2024-03-11T19:16:02.490 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-11T19:16:02.490 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272 --name teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} -- /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272/orig.config.yaml 2024-03-11T19:16:02.547 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-11T19:16:02.636 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7591272' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 94a539707733325322964c6a187c6a1b41be1daf subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 94a539707733325322964c6a187c6a1b41be1daf kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad owner: scheduled_teuthology@teuthology priority: 250 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8179 sha1: 94a539707733325322964c6a187c6a1b41be1daf sleep_before_teardown: 0 subset: 297/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLOuY+M2QlviJnbLDcvky6Z2VXTmnRXIgRlRysT1A53w960iPFT5YgOjY06lf6CKf7Nf12BLKe5QWDhcd/OWMM= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRArQaZruB8rQ7ogXnUlBsEfp+U4nllr8h7QeGdH4Sh7q/7ttds73kas0cP1hVFGLNgfveTI+32RPmk/BiTsvw= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEk29REzAi1N7zcW/tXY8+rqq4cQhg/zAIj4G22CDXWyOqOT0bCWdyI0R7KWbkVYYoKifB0PwQIoyxgddFpTuMw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-11_14:02:35 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-11T19:16:02.636 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa; will attempt to use it 2024-03-11T19:16:02.637 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks 2024-03-11T19:16:02.637 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-11T19:16:02.639 INFO:teuthology.task.internal:Checking packages... 2024-03-11T19:16:02.662 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '94a539707733325322964c6a187c6a1b41be1daf' 2024-03-11T19:16:02.662 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-11T19:16:02.662 INFO:teuthology.packaging:ref: None 2024-03-11T19:16:02.662 INFO:teuthology.packaging:tag: None 2024-03-11T19:16:02.662 INFO:teuthology.packaging:branch: squid 2024-03-11T19:16:02.662 INFO:teuthology.packaging:sha1: 94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:16:02.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-03-11T19:16:02.801 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1607.g94a53970 2024-03-11T19:16:02.802 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-11T19:16:02.810 INFO:teuthology.task.internal:no buildpackages task found 2024-03-11T19:16:02.810 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-11T19:16:02.827 INFO:teuthology.task.internal:Saving configuration 2024-03-11T19:16:02.847 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-11T19:16:02.858 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T19:16:02.884 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:10:49.730995', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLOuY+M2QlviJnbLDcvky6Z2VXTmnRXIgRlRysT1A53w960iPFT5YgOjY06lf6CKf7Nf12BLKe5QWDhcd/OWMM='} 2024-03-11T19:16:02.910 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:10:49.732958', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRArQaZruB8rQ7ogXnUlBsEfp+U4nllr8h7QeGdH4Sh7q/7ttds73kas0cP1hVFGLNgfveTI+32RPmk/BiTsvw='} 2024-03-11T19:16:02.931 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:10:49.734627', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEk29REzAi1N7zcW/tXY8+rqq4cQhg/zAIj4G22CDXWyOqOT0bCWdyI0R7KWbkVYYoKifB0PwQIoyxgddFpTuMw='} 2024-03-11T19:16:02.931 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-11T19:16:02.942 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-11T19:16:02.943 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-11T19:16:02.943 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-11T19:16:02.943 INFO:teuthology.run_tasks:Running task console_log... 2024-03-11T19:16:03.053 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f45f30f5160>, signals=[15]) 2024-03-11T19:16:03.054 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-11T19:16:03.067 INFO:teuthology.task.internal:Opening connections... 2024-03-11T19:16:03.067 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-03-11T19:16:03.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:16:03.161 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-03-11T19:16:03.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:16:03.250 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-03-11T19:16:03.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:16:03.336 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-11T19:16:03.343 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-03-11T19:16:03.360 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-03-11T19:16:03.361 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-03-11T19:16:03.417 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-03-11T19:16:03.417 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-11T19:16:03.418 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-11T19:16:03.419 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-03-11T19:16:03.447 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-03-11T19:16:03.464 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-03-11T19:16:03.465 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-03-11T19:16:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-11T19:16:03.522 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-11T19:16:03.522 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-11T19:16:03.522 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-03-11T19:16:03.547 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-03-11T19:16:03.564 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-03-11T19:16:03.564 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-03-11T19:16:03.621 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-03-11T19:16:03.621 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-03-11T19:16:03.621 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-03-11T19:16:03.621 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-03-11T19:16:03.621 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-03-11T19:16:03.621 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-03-11T19:16:03.621 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-11T19:16:03.621 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-03-11T19:16:03.621 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-03-11T19:16:03.622 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-11T19:16:03.622 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-03-11T19:16:03.622 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-11T19:16:03.622 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-11T19:16:03.622 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-11T19:16:03.622 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-03-11T19:16:03.647 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-11T19:16:03.658 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-11T19:16:03.665 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-11T19:16:03.666 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-03-11T19:16:03.668 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-03-11T19:16:03.671 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-03-11T19:16:03.686 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-11T19:16:03.693 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-11T19:16:03.694 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-03-11T19:16:03.725 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-03-11T19:16:03.728 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-03-11T19:16:03.840 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-11T19:16:03.919 INFO:teuthology.run_tasks:Running task kernel... 2024-03-11T19:16:03.936 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-11T19:16:03.936 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-11T19:16:03.936 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-11T19:16:03.936 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-11T19:16:03.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:16:03.953 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-03-11T19:16:04.009 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-361.el9.x86_64 2024-03-11T19:16:04.009 INFO:teuthology.task.kernel:Running kernel on smithi083: 5.14.0-361.el9.x86_64 2024-03-11T19:16:04.009 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum install -y kernel 2024-03-11T19:16:06.231 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-11T19:16:08.182 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - BaseOS 2.4 MB/s | 2.0 MB 00:00 2024-03-11T19:16:11.863 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.0 MB 00:03 2024-03-11T19:16:13.817 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-11T19:16:18.174 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 6.1 MB 00:00 2024-03-11T19:16:19.674 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 15 kB 00:00 2024-03-11T19:16:21.037 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-11T19:16:25.967 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-03-11T19:16:28.133 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-11T19:16:28.133 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-11T19:16:28.214 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-03-11T19:16:28.224 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-03-11T19:16:28.224 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-03-11T19:16:28.224 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-03-11T19:16:28.225 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-03-11T19:16:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-11T19:16:28.225 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-03-11T19:16:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-11T19:16:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-11T19:16:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-11T19:16:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:16:28.225 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-03-11T19:16:28.225 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-03-11T19:16:28.225 INFO:teuthology.orchestra.run.smithi083.stdout:Install 4 Packages 2024-03-11T19:16:28.225 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:16:28.226 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 100 M 2024-03-11T19:16:28.226 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 123 M 2024-03-11T19:16:28.226 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-03-11T19:16:40.543 INFO:teuthology.orchestra.run.smithi083.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 500 kB/s | 6.0 MB 00:12 2024-03-11T19:17:13.082 INFO:teuthology.orchestra.run.smithi083.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 476 kB/s | 21 MB 00:44 2024-03-11T19:18:02.693 INFO:teuthology.orchestra.run.smithi083.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 426 kB/s | 39 MB 01:34 2024-03-11T19:18:07.743 INFO:teuthology.orchestra.run.smithi083.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 392 kB/s | 33 MB 01:27 2024-03-11T19:18:07.743 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:18:07.743 INFO:teuthology.orchestra.run.smithi083.stdout:Total 1.0 MB/s | 100 MB 01:39 2024-03-11T19:18:08.628 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-03-11T19:18:08.999 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-03-11T19:18:09.000 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-03-11T19:18:10.364 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-03-11T19:18:10.364 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-03-11T19:18:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-03-11T19:18:13.970 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-11T19:18:14.237 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-11T19:18:16.903 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-11T19:18:17.058 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-11T19:18:21.824 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-11T19:18:21.952 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-11T19:18:26.737 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-11T19:18:56.066 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-11T19:18:56.067 INFO:teuthology.orchestra.run.smithi083.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-11T19:18:56.067 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:18:56.151 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-11T19:18:56.646 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-11T19:18:56.646 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-11T19:18:56.646 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-11T19:18:56.646 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-11T19:18:57.092 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-11T19:18:57.092 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:18:57.093 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-03-11T19:18:57.093 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-11T19:18:57.093 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-11T19:18:57.093 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-11T19:18:57.093 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-11T19:18:57.093 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:18:57.093 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-03-11T19:18:57.604 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2024-03-11T19:18:57.736 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-11T19:18:57.736 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-11T19:18:57.736 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-11T19:18:57.736 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-11T19:18:57.736 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-11T19:18:57.736 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-11T19:18:57.737 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=None, version=distro) 2024-03-11T19:18:57.737 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2024-03-11T19:18:57.856 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-11T19:18:57.856 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-11T19:18:57.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -qi grub2-tools 2024-03-11T19:18:57.896 INFO:teuthology.orchestra.run.smithi083.stdout:Name : grub2-tools 2024-03-11T19:18:57.896 INFO:teuthology.orchestra.run.smithi083.stdout:Epoch : 1 2024-03-11T19:18:57.896 INFO:teuthology.orchestra.run.smithi083.stdout:Version : 2.06 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Release : 68.el9 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: x86_64 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Group : Unspecified 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Size : 8264254 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:License : GPLv3+ 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Packager : builder@centos.org 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Vendor : CentOS 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-11T19:18:57.897 INFO:teuthology.orchestra.run.smithi083.stdout:Summary : Support tools for GRUB. 2024-03-11T19:18:57.898 INFO:teuthology.orchestra.run.smithi083.stdout:Description : 2024-03-11T19:18:57.898 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:18:57.898 INFO:teuthology.orchestra.run.smithi083.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-11T19:18:57.898 INFO:teuthology.orchestra.run.smithi083.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-11T19:18:57.898 INFO:teuthology.orchestra.run.smithi083.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-11T19:18:57.898 INFO:teuthology.orchestra.run.smithi083.stdout:hardware devices. 2024-03-11T19:18:57.898 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:18:57.898 INFO:teuthology.orchestra.run.smithi083.stdout:This subpackage provides tools for support of all platforms. 2024-03-11T19:18:57.899 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-11T19:18:57.899 INFO:teuthology.task.kernel:Updating grub on smithi083 to boot 5.14.0-427.el9.x86_64 2024-03-11T19:18:57.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-11T19:18:58.490 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-03-11T19:18:59.820 INFO:teuthology.orchestra.run.smithi083.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-11T19:18:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-03-11T19:18:59.858 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-03-11T19:18:59.877 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.Zs9RG6jUhW 2024-03-11T19:18:59.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Zs9RG6jUhW 2024-03-11T19:18:59.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.Zs9RG6jUhW 2024-03-11T19:19:00.097 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.Zs9RG6jUhW is 6KB 2024-03-11T19:19:00.146 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.Zs9RG6jUhW 2024-03-11T19:19:00.160 DEBUG:teuthology.orchestra.run.smithi083:> sudo /bin/ls /boot/loader/entries || true 2024-03-11T19:19:00.224 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-11T19:19:00.225 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-11T19:19:00.225 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-11T19:19:00.225 INFO:teuthology.orchestra.run.smithi083.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-11T19:19:00.226 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-11T19:19:00.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-03-11T19:19:30.402 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-11T19:19:30.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-03-11T19:19:30.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:19:48.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-03-11T19:19:57.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-03-11T19:19:57.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:20:57.773 DEBUG:teuthology.orchestra.remote:timed out 2024-03-11T19:21:09.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-03-11T19:21:09.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:21:10.229 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-03-11T19:21:10.605 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-03-11T19:21:10.605 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-11T19:21:10.606 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-03-11T19:21:10.621 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-427.el9.x86_64 2024-03-11T19:21:10.621 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-11T19:21:10.621 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-11T19:21:10.621 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-11T19:21:11.622 DEBUG:teuthology.parallel:result is None 2024-03-11T19:21:11.623 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-11T19:21:11.630 INFO:teuthology.task.internal:Creating test directory... 2024-03-11T19:21:11.630 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-11T19:21:11.633 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-11T19:21:11.635 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-11T19:21:11.653 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-11T19:21:11.677 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-11T19:21:11.684 INFO:teuthology.task.internal:Creating archive directory... 2024-03-11T19:21:11.684 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-11T19:21:11.690 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-11T19:21:11.693 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-11T19:21:12.013 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-11T19:21:12.019 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-11T19:21:12.019 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-11T19:21:12.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:21:12.036 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-11T19:21:12.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:21:12.053 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-11T19:21:12.055 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-11T19:21:12.079 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-11T19:21:12.148 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:21:12.153 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:21:12.163 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:21:12.170 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:21:12.183 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:21:12.184 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T19:21:12.186 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-11T19:21:12.194 INFO:teuthology.task.internal:Configuring sudo... 2024-03-11T19:21:12.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-11T19:21:12.196 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-11T19:21:12.199 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-11T19:21:12.228 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-11T19:21:12.236 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-11T19:21:12.236 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-11T19:21:12.266 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-11T19:21:12.269 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-11T19:21:12.284 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:21:12.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:21:12.424 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:21:12.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:21:12.511 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:21:12.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-11T19:21:12.579 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:21:12.623 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:21:12.679 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:21:12.763 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:21:12.808 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-11T19:21:12.808 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-11T19:21:12.877 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:21:12.919 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T19:21:12.979 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:21:13.058 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T19:21:13.107 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-11T19:21:13.107 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-11T19:21:13.176 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-03-11T19:21:13.179 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-03-11T19:21:13.181 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-03-11T19:21:13.223 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:21:13.226 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:21:13.265 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:21:13.682 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-11T19:21:13.690 INFO:teuthology.task.internal:Starting timer... 2024-03-11T19:21:13.690 INFO:teuthology.run_tasks:Running task pcp... 2024-03-11T19:21:13.700 INFO:teuthology.run_tasks:Running task selinux... 2024-03-11T19:21:13.708 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-11T19:21:13.709 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-03-11T19:21:13.794 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-03-11T19:21:13.796 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-03-11T19:21:13.860 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-03-11T19:21:13.862 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-03-11T19:21:13.925 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-03-11T19:21:13.927 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-11T19:21:13.927 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-03-11T19:21:13.951 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-03-11T19:21:13.951 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-03-11T19:21:13.973 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-03-11T19:21:13.974 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-03-11T19:21:13.997 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-03-11T19:21:13.997 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-03-11T19:21:13.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:21:14.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:21:14.027 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:21:14.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:21:14.056 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:21:14.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:21:14.083 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-11T19:21:14.083 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-03-11T19:21:14.120 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-03-11T19:21:14.150 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-03-11T19:21:14.190 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-11T19:21:14.200 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-11T19:21:14.205 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-11T19:21:14.347 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-11T19:21:14.363 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-11T19:21:14.365 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi137.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-11T19:25:59.424 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-03-11T19:25:59.425 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-03-11T19:25:59.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:25:59.516 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-03-11T19:25:59.597 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-03-11T19:25:59.597 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-11T19:25:59.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:25:59.687 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-03-11T19:25:59.759 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-11T19:25:59.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-11T19:25:59.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T19:25:59.846 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-03-11T19:25:59.914 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-11T19:25:59.914 INFO:teuthology.run_tasks:Running task clock... 2024-03-11T19:25:59.925 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-11T19:25:59.925 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-11T19:25:59.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:25:59.928 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-11T19:25:59.928 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:25:59.931 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-11T19:25:59.931 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:25:59.957 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-11T19:25:59.964 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-11T19:25:59.974 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-11T19:25:59.982 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-11T19:25:59.986 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-11T19:26:00.002 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-03-11T19:26:00.003 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-11T19:26:00.013 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-03-11T19:26:00.015 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-03-11T19:26:00.027 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-03-11T19:26:00.031 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-11T19:26:00.033 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-03-11T19:26:00.044 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-11T19:26:00.046 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-03-11T19:26:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-11T19:26:00.061 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-11T19:26:00.063 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-11T19:26:00.079 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-11T19:26:00.099 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:26:00.103 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:26:00.103 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-03-11T19:26:00.103 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.103 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.103 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.103 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.105 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:26:00.109 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:26:00.109 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-03-11T19:26:00.109 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.109 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.109 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.109 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.123 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:26:00.126 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:26:00.127 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-03-11T19:26:00.127 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.127 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.127 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.127 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:26:00.128 INFO:teuthology.run_tasks:Running task install... 2024-03-11T19:26:00.136 DEBUG:teuthology.task.install:project ceph 2024-03-11T19:26:00.136 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'}} 2024-03-11T19:26:00.137 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'} 2024-03-11T19:26:00.137 INFO:teuthology.task.install:Using flavor: default 2024-03-11T19:26:00.145 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-11T19:26:00.145 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-11T19:26:00.145 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-11T19:26:00.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:26:00.148 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-11T19:26:00.148 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:26:00.149 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'tag': None, 'wait_for_package': False} 2024-03-11T19:26:00.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:26:00.297 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-11T19:26:00.297 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-11T19:26:00.480 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-11T19:26:00.480 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-11T19:26:00.498 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/ 2024-03-11T19:26:00.499 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1607.g94a53970 2024-03-11T19:26:00.545 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-11T19:26:00.546 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-11T19:26:00.546 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-11T19:26:00.574 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-11T19:26:00.574 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-11T19:26:00.589 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-11T19:26:00.589 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:26:00.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-11T19:26:00.607 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-11T19:26:00.607 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-11T19:26:00.607 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-11T19:26:00.616 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-11T19:26:00.617 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-11T19:26:00.635 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-11T19:26:00.635 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/94a539707733325322964c6a187c6a1b41be1daf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-11T19:26:00.642 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-11T19:26:00.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-11T19:26:00.703 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-11T19:26:00.722 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-11T19:26:00.765 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-11T19:26:00.783 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-11T19:26:00.789 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-03-11T19:26:00.791 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-03-11T19:26:00.832 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-03-11T19:26:00.834 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-03-11T19:26:00.849 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-03-11T19:26:00.851 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-03-11T19:26:01.026 INFO:teuthology.orchestra.run.smithi137.stdout:74 files removed 2024-03-11T19:26:01.054 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-11T19:26:01.085 INFO:teuthology.orchestra.run.smithi204.stdout:74 files removed 2024-03-11T19:26:01.091 INFO:teuthology.orchestra.run.smithi083.stdout:74 files removed 2024-03-11T19:26:01.113 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-11T19:26:01.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-11T19:26:01.706 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 228 kB/s | 82 kB 00:00 2024-03-11T19:26:01.828 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 193 kB/s | 82 kB 00:00 2024-03-11T19:26:01.907 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 173 kB/s | 82 kB 00:00 2024-03-11T19:26:02.056 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-11T19:26:02.220 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 28 kB/s | 10 kB 00:00 2024-03-11T19:26:02.333 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-11T19:26:02.457 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-03-11T19:26:02.562 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-03-11T19:26:02.775 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-03-11T19:26:02.943 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-11T19:26:03.628 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - AppStream 8.2 MB/s | 6.7 MB 00:00 2024-03-11T19:26:04.148 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - AppStream 4.3 MB/s | 6.7 MB 00:01 2024-03-11T19:26:04.443 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2024-03-11T19:26:05.311 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-03-11T19:26:05.790 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-11T19:26:06.167 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.0 MB 00:01 2024-03-11T19:26:06.639 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-03-11T19:26:07.993 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.0 MB 00:02 2024-03-11T19:26:12.026 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 19 MB 00:04 2024-03-11T19:26:12.302 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 19 MB 00:03 2024-03-11T19:26:17.537 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.1 MB 00:01 2024-03-11T19:26:19.747 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 15 kB 00:00 2024-03-11T19:26:21.242 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-11T19:26:25.476 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 626 kB/s | 6.1 MB 00:10 2024-03-11T19:26:26.214 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-11T19:26:27.042 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 15 kB 00:00 2024-03-11T19:26:28.744 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:26:28.745 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:26:28.759 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-11T19:26:28.761 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-11T19:26:28.762 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-11T19:26:28.763 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-11T19:26:28.765 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-11T19:26:28.768 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-11T19:26:28.769 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-11T19:26:28.849 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-03-11T19:26:28.871 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================================ 2024-03-11T19:26:28.871 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-03-11T19:26:28.871 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================================ 2024-03-11T19:26:28.871 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-11T19:26:28.872 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-11T19:26:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-03-11T19:26:28.874 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-11T19:26:28.875 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-11T19:26:28.876 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-11T19:26:28.877 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-11T19:26:28.877 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-11T19:26:28.877 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-11T19:26:28.877 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-11T19:26:28.877 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-11T19:26:28.877 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-11T19:26:28.877 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-11T19:26:28.877 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-11T19:26:28.877 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-11T19:26:28.877 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-11T19:26:28.877 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-11T19:26:28.878 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-11T19:26:28.878 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-11T19:26:28.878 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-11T19:26:28.878 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-11T19:26:28.878 INFO:teuthology.orchestra.run.smithi083.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-11T19:26:28.878 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-11T19:26:28.878 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-11T19:26:28.878 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-11T19:26:28.878 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-11T19:26:28.878 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-11T19:26:28.878 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-11T19:26:28.879 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-11T19:26:28.880 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-11T19:26:28.881 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-11T19:26:28.882 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-11T19:26:28.883 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-11T19:26:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-11T19:26:28.885 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:26:28.886 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-03-11T19:26:28.886 INFO:teuthology.orchestra.run.smithi083.stdout:============================================================================================ 2024-03-11T19:26:28.886 INFO:teuthology.orchestra.run.smithi083.stdout:Install 150 Packages 2024-03-11T19:26:28.886 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 24 Packages 2024-03-11T19:26:28.886 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:26:28.886 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 223 M 2024-03-11T19:26:28.886 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-03-11T19:26:30.216 INFO:teuthology.orchestra.run.smithi083.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-03-11T19:26:30.494 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 21 MB 00:03 2024-03-11T19:26:30.682 INFO:teuthology.orchestra.run.smithi083.stdout:(2/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 6.3 MB/s | 5.3 MB 00:00 2024-03-11T19:26:30.732 INFO:teuthology.orchestra.run.smithi083.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 2.9 MB/s | 145 kB 00:00 2024-03-11T19:26:30.891 INFO:teuthology.orchestra.run.smithi083.stdout:(4/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 13 MB/s | 2.1 MB 00:00 2024-03-11T19:26:30.916 INFO:teuthology.orchestra.run.smithi083.stdout:(5/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 1.2 MB/s | 886 kB 00:00 2024-03-11T19:26:31.008 INFO:teuthology.orchestra.run.smithi083.stdout:(6/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 13 MB/s | 1.5 MB 00:00 2024-03-11T19:26:32.058 INFO:teuthology.orchestra.run.smithi083.stdout:(7/174): ceph-osd-19.0.0-1607.g94a53970.el9.x86 16 MB/s | 17 MB 00:01 2024-03-11T19:26:32.767 INFO:teuthology.orchestra.run.smithi083.stdout:(8/174): ceph-radosgw-19.0.0-1607.g94a53970.el9 14 MB/s | 9.8 MB 00:00 2024-03-11T19:26:32.809 INFO:teuthology.orchestra.run.smithi083.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 609 kB/s | 25 kB 00:00 2024-03-11T19:26:32.951 INFO:teuthology.orchestra.run.smithi083.stdout:(10/174): ceph-mon-19.0.0-1607.g94a53970.el9.x8 2.3 MB/s | 4.7 MB 00:02 2024-03-11T19:26:33.202 INFO:teuthology.orchestra.run.smithi083.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 869 kB/s | 217 kB 00:00 2024-03-11T19:26:33.302 INFO:teuthology.orchestra.run.smithi083.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 271 kB/s | 27 kB 00:00 2024-03-11T19:26:33.402 INFO:teuthology.orchestra.run.smithi083.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 312 kB/s | 31 kB 00:00 2024-03-11T19:26:33.911 INFO:teuthology.orchestra.run.smithi083.stdout:(14/174): libcephfs2-19.0.0-1607.g94a53970.el9. 1.4 MB/s | 742 kB 00:00 2024-03-11T19:26:34.036 INFO:teuthology.orchestra.run.smithi083.stdout:(15/174): libcephsqlite-19.0.0-1607.g94a53970.e 1.3 MB/s | 163 kB 00:00 2024-03-11T19:26:34.178 INFO:teuthology.orchestra.run.smithi083.stdout:(16/174): librados-devel-19.0.0-1607.g94a53970. 894 kB/s | 127 kB 00:00 2024-03-11T19:26:34.463 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 721 kB/s | 19 MB 00:26 2024-03-11T19:26:34.528 INFO:teuthology.orchestra.run.smithi083.stdout:(17/174): libradosstriper1-19.0.0-1607.g94a5397 1.4 MB/s | 498 kB 00:00 2024-03-11T19:26:35.327 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-03-11T19:26:36.153 INFO:teuthology.orchestra.run.smithi083.stdout:(18/174): ceph-test-19.0.0-1607.g94a53970.el9.x 14 MB/s | 47 MB 00:03 2024-03-11T19:26:36.212 INFO:teuthology.orchestra.run.smithi083.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 779 kB/s | 45 kB 00:00 2024-03-11T19:26:36.270 INFO:teuthology.orchestra.run.smithi083.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 2.2 MB/s | 131 kB 00:00 2024-03-11T19:26:36.329 INFO:teuthology.orchestra.run.smithi083.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 2.7 MB/s | 161 kB 00:00 2024-03-11T19:26:36.429 INFO:teuthology.orchestra.run.smithi083.stdout:(22/174): python3-rados-19.0.0-1607.g94a53970.e 3.1 MB/s | 320 kB 00:00 2024-03-11T19:26:36.496 INFO:teuthology.orchestra.run.smithi083.stdout:(23/174): python3-rbd-19.0.0-1607.g94a53970.el9 4.4 MB/s | 298 kB 00:00 2024-03-11T19:26:36.555 INFO:teuthology.orchestra.run.smithi083.stdout:(24/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.7 MB/s | 99 kB 00:00 2024-03-11T19:26:36.613 INFO:teuthology.orchestra.run.smithi083.stdout:(25/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-11T19:26:36.872 INFO:teuthology.orchestra.run.smithi083.stdout:(26/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 12 MB/s | 3.1 MB 00:00 2024-03-11T19:26:36.922 INFO:teuthology.orchestra.run.smithi083.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 3.4 MB/s | 171 kB 00:00 2024-03-11T19:26:36.964 INFO:teuthology.orchestra.run.smithi083.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1607.g 757 kB/s | 31 kB 00:00 2024-03-11T19:26:37.014 INFO:teuthology.orchestra.run.smithi083.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 2.7 MB/s | 138 kB 00:00 2024-03-11T19:26:37.156 INFO:teuthology.orchestra.run.smithi083.stdout:(30/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 13 MB/s | 1.8 MB 00:00 2024-03-11T19:26:37.665 INFO:teuthology.orchestra.run.smithi083.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-03-11T19:26:37.740 INFO:teuthology.orchestra.run.smithi083.stdout:(32/174): librgw2-19.0.0-1607.g94a53970.el9.x86 1.6 MB/s | 5.1 MB 00:03 2024-03-11T19:26:37.766 INFO:teuthology.orchestra.run.smithi083.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1607.g94 2.4 MB/s | 247 kB 00:00 2024-03-11T19:26:37.790 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:26:37.792 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:26:37.805 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-11T19:26:37.806 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-11T19:26:37.807 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-11T19:26:37.807 INFO:teuthology.orchestra.run.smithi083.stdout:(34/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 739 kB/s | 49 kB 00:00 2024-03-11T19:26:37.808 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-11T19:26:37.810 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-11T19:26:37.813 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-11T19:26:37.814 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-11T19:26:37.833 INFO:teuthology.orchestra.run.smithi083.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1607.g9 229 kB/s | 15 kB 00:00 2024-03-11T19:26:37.893 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-03-11T19:26:37.905 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-03-11T19:26:37.905 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-03-11T19:26:37.905 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-03-11T19:26:37.905 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-03-11T19:26:37.905 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-11T19:26:37.905 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-11T19:26:37.905 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-11T19:26:37.906 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-11T19:26:37.907 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-11T19:26:37.908 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-11T19:26:37.909 INFO:teuthology.orchestra.run.smithi137.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-11T19:26:37.910 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-11T19:26:37.911 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-11T19:26:37.912 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-11T19:26:37.913 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-11T19:26:37.914 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-11T19:26:37.915 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-11T19:26:37.916 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:26:37.917 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-03-11T19:26:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-03-11T19:26:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:Install 150 Packages 2024-03-11T19:26:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 24 Packages 2024-03-11T19:26:37.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:26:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 223 M 2024-03-11T19:26:37.918 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-03-11T19:26:37.925 INFO:teuthology.orchestra.run.smithi083.stdout:(36/174): cephadm-19.0.0-1607.g94a53970.el9.noa 8.0 MB/s | 748 kB 00:00 2024-03-11T19:26:37.983 INFO:teuthology.orchestra.run.smithi083.stdout:(37/174): cephfs-top-19.0.0-1607.g94a53970.el9. 313 kB/s | 18 kB 00:00 2024-03-11T19:26:38.008 INFO:teuthology.orchestra.run.smithi083.stdout:(38/174): ceph-volume-19.0.0-1607.g94a53970.el9 1.3 MB/s | 264 kB 00:00 2024-03-11T19:26:38.184 INFO:teuthology.orchestra.run.smithi083.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2024-03-11T19:26:38.218 INFO:teuthology.orchestra.run.smithi083.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.3 MB/s | 920 kB 00:00 2024-03-11T19:26:38.243 INFO:teuthology.orchestra.run.smithi083.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-11T19:26:38.269 INFO:teuthology.orchestra.run.smithi083.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-11T19:26:38.285 INFO:teuthology.orchestra.run.smithi083.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 765 kB/s | 32 kB 00:00 2024-03-11T19:26:38.369 INFO:teuthology.orchestra.run.smithi083.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-11T19:26:38.386 INFO:teuthology.orchestra.run.smithi083.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 172 kB/s | 17 kB 00:00 2024-03-11T19:26:38.403 INFO:teuthology.orchestra.run.smithi083.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 299 kB/s | 10 kB 00:00 2024-03-11T19:26:38.428 INFO:teuthology.orchestra.run.smithi083.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-11T19:26:38.453 INFO:teuthology.orchestra.run.smithi083.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-03-11T19:26:38.487 INFO:teuthology.orchestra.run.smithi083.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-11T19:26:38.513 INFO:teuthology.orchestra.run.smithi083.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-03-11T19:26:38.538 INFO:teuthology.orchestra.run.smithi083.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 937 kB/s | 46 kB 00:00 2024-03-11T19:26:38.563 INFO:teuthology.orchestra.run.smithi083.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-11T19:26:38.588 INFO:teuthology.orchestra.run.smithi083.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-11T19:26:38.614 INFO:teuthology.orchestra.run.smithi083.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-11T19:26:38.639 INFO:teuthology.orchestra.run.smithi083.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 684 kB/s | 34 kB 00:00 2024-03-11T19:26:38.720 INFO:teuthology.orchestra.run.smithi137.stdout:(1/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-03-11T19:26:38.756 INFO:teuthology.orchestra.run.smithi083.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-03-11T19:26:38.798 INFO:teuthology.orchestra.run.smithi083.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 9.8 MB/s | 1.5 MB 00:00 2024-03-11T19:26:38.831 INFO:teuthology.orchestra.run.smithi083.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-11T19:26:38.857 INFO:teuthology.orchestra.run.smithi083.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-11T19:26:38.995 INFO:teuthology.orchestra.run.smithi137.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 3.2 MB/s | 886 kB 00:00 2024-03-11T19:26:38.999 INFO:teuthology.orchestra.run.smithi083.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-03-11T19:26:39.032 INFO:teuthology.orchestra.run.smithi083.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.9 MB/s | 1.0 MB 00:00 2024-03-11T19:26:39.057 INFO:teuthology.orchestra.run.smithi083.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-11T19:26:39.087 INFO:teuthology.orchestra.run.smithi137.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 1.6 MB/s | 145 kB 00:00 2024-03-11T19:26:39.208 INFO:teuthology.orchestra.run.smithi083.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-03-11T19:26:39.241 INFO:teuthology.orchestra.run.smithi083.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 134 kB/s | 24 kB 00:00 2024-03-11T19:26:39.266 INFO:teuthology.orchestra.run.smithi083.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 309 kB/s | 18 kB 00:00 2024-03-11T19:26:39.292 INFO:teuthology.orchestra.run.smithi083.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-11T19:26:39.317 INFO:teuthology.orchestra.run.smithi083.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 627 kB/s | 32 kB 00:00 2024-03-11T19:26:39.451 INFO:teuthology.orchestra.run.smithi083.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2024-03-11T19:26:39.477 INFO:teuthology.orchestra.run.smithi083.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-11T19:26:39.502 INFO:teuthology.orchestra.run.smithi083.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-11T19:26:39.503 INFO:teuthology.orchestra.run.smithi137.stdout:(4/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 5.1 MB/s | 2.1 MB 00:00 2024-03-11T19:26:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-11T19:26:39.552 INFO:teuthology.orchestra.run.smithi083.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 819 kB/s | 41 kB 00:00 2024-03-11T19:26:39.578 INFO:teuthology.orchestra.run.smithi083.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-11T19:26:39.603 INFO:teuthology.orchestra.run.smithi083.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-11T19:26:39.628 INFO:teuthology.orchestra.run.smithi083.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-11T19:26:39.654 INFO:teuthology.orchestra.run.smithi083.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2024-03-11T19:26:39.670 INFO:teuthology.orchestra.run.smithi083.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-03-11T19:26:39.771 INFO:teuthology.orchestra.run.smithi137.stdout:(5/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-03-11T19:26:39.837 INFO:teuthology.orchestra.run.smithi083.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 232 kB/s | 39 kB 00:00 2024-03-11T19:26:40.104 INFO:teuthology.orchestra.run.smithi083.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-11T19:26:40.122 INFO:teuthology.orchestra.run.smithi083.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 145 kB/s | 41 kB 00:00 2024-03-11T19:26:40.147 INFO:teuthology.orchestra.run.smithi083.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-11T19:26:40.163 INFO:teuthology.orchestra.run.smithi083.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-03-11T19:26:40.180 INFO:teuthology.orchestra.run.smithi083.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-11T19:26:40.197 INFO:teuthology.orchestra.run.smithi083.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-11T19:26:40.223 INFO:teuthology.orchestra.run.smithi083.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-03-11T19:26:40.248 INFO:teuthology.orchestra.run.smithi083.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-11T19:26:40.290 INFO:teuthology.orchestra.run.smithi083.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-11T19:26:40.315 INFO:teuthology.orchestra.run.smithi083.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-11T19:26:40.340 INFO:teuthology.orchestra.run.smithi083.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-11T19:26:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 187 kB 00:00 2024-03-11T19:26:40.396 INFO:teuthology.orchestra.run.smithi137.stdout:(6/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 7.5 MB/s | 4.7 MB 00:00 2024-03-11T19:26:40.641 INFO:teuthology.orchestra.run.smithi083.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 624 kB/s | 187 kB 00:00 2024-03-11T19:26:40.758 INFO:teuthology.orchestra.run.smithi083.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-03-11T19:26:40.810 INFO:teuthology.orchestra.run.smithi083.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-11T19:26:40.830 INFO:teuthology.orchestra.run.smithi137.stdout:(7/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 2.3 MB/s | 5.3 MB 00:02 2024-03-11T19:26:40.842 INFO:teuthology.orchestra.run.smithi083.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-03-11T19:26:40.951 INFO:teuthology.orchestra.run.smithi083.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 151 kB/s | 21 kB 00:00 2024-03-11T19:26:40.976 INFO:teuthology.orchestra.run.smithi083.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 190 kB/s | 25 kB 00:00 2024-03-11T19:26:41.226 INFO:teuthology.orchestra.run.smithi083.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 982 kB/s | 245 kB 00:00 2024-03-11T19:26:41.368 INFO:teuthology.orchestra.run.smithi083.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-03-11T19:26:41.494 INFO:teuthology.orchestra.run.smithi083.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.7 MB/s | 216 kB 00:00 2024-03-11T19:26:41.552 INFO:teuthology.orchestra.run.smithi083.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 600 kB/s | 35 kB 00:00 2024-03-11T19:26:41.644 INFO:teuthology.orchestra.run.smithi083.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 485 kB/s | 44 kB 00:00 2024-03-11T19:26:41.768 INFO:teuthology.orchestra.run.smithi137.stdout:(8/174): ceph-osd-19.0.0-1607.g94a53970.el9.x86 12 MB/s | 17 MB 00:01 2024-03-11T19:26:41.769 INFO:teuthology.orchestra.run.smithi083.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 750 kB/s | 94 kB 00:00 2024-03-11T19:26:41.813 INFO:teuthology.orchestra.run.smithi137.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 555 kB/s | 25 kB 00:00 2024-03-11T19:26:41.895 INFO:teuthology.orchestra.run.smithi083.stdout:(103/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.4 MB/s | 4.1 MB 00:00 2024-03-11T19:26:41.970 INFO:teuthology.orchestra.run.smithi083.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 111 kB/s | 22 kB 00:00 2024-03-11T19:26:42.171 INFO:teuthology.orchestra.run.smithi083.stdout:(105/174): ceph-common-19.0.0-1607.g94a53970.el 1.6 MB/s | 20 MB 00:12 2024-03-11T19:26:42.254 INFO:teuthology.orchestra.run.smithi083.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-03-11T19:26:42.279 INFO:teuthology.orchestra.run.smithi083.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-03-11T19:26:42.304 INFO:teuthology.orchestra.run.smithi083.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-03-11T19:26:42.338 INFO:teuthology.orchestra.run.smithi083.stdout:(109/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-03-11T19:26:42.363 INFO:teuthology.orchestra.run.smithi083.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-03-11T19:26:42.397 INFO:teuthology.orchestra.run.smithi083.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-03-11T19:26:42.422 INFO:teuthology.orchestra.run.smithi083.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 579 kB/s | 49 kB 00:00 2024-03-11T19:26:42.448 INFO:teuthology.orchestra.run.smithi083.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-03-11T19:26:42.514 INFO:teuthology.orchestra.run.smithi083.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-03-11T19:26:42.540 INFO:teuthology.orchestra.run.smithi083.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-03-11T19:26:42.557 INFO:teuthology.orchestra.run.smithi083.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-03-11T19:26:42.573 INFO:teuthology.orchestra.run.smithi083.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 959 kB/s | 32 kB 00:00 2024-03-11T19:26:42.597 INFO:teuthology.orchestra.run.smithi137.stdout:(10/174): ceph-radosgw-19.0.0-1607.g94a53970.el 5.5 MB/s | 9.8 MB 00:01 2024-03-11T19:26:42.599 INFO:teuthology.orchestra.run.smithi083.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 359 kB/s | 15 kB 00:00 2024-03-11T19:26:42.624 INFO:teuthology.orchestra.run.smithi083.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-03-11T19:26:42.649 INFO:teuthology.orchestra.run.smithi083.stdout:(120/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-03-11T19:26:42.683 INFO:teuthology.orchestra.run.smithi083.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-11T19:26:42.716 INFO:teuthology.orchestra.run.smithi083.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-03-11T19:26:42.722 INFO:teuthology.orchestra.run.smithi137.stdout:(11/174): cephfs-mirror-19.0.0-1607.g94a53970.e 1.7 MB/s | 217 kB 00:00 2024-03-11T19:26:42.733 INFO:teuthology.orchestra.run.smithi083.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-03-11T19:26:42.759 INFO:teuthology.orchestra.run.smithi083.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-11T19:26:42.793 INFO:teuthology.orchestra.run.smithi083.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-11T19:26:42.818 INFO:teuthology.orchestra.run.smithi083.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 181 kB/s | 11 kB 00:00 2024-03-11T19:26:42.831 INFO:teuthology.orchestra.run.smithi137.stdout:(12/174): cephfs-shell-19.0.0-1607.g94a53970.el 249 kB/s | 27 kB 00:00 2024-03-11T19:26:42.843 INFO:teuthology.orchestra.run.smithi083.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-03-11T19:26:42.868 INFO:teuthology.orchestra.run.smithi083.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-03-11T19:26:42.894 INFO:teuthology.orchestra.run.smithi083.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-03-11T19:26:42.919 INFO:teuthology.orchestra.run.smithi083.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-03-11T19:26:42.948 INFO:teuthology.orchestra.run.smithi137.stdout:(13/174): libcephfs-devel-19.0.0-1607.g94a53970 268 kB/s | 31 kB 00:00 2024-03-11T19:26:42.969 INFO:teuthology.orchestra.run.smithi083.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-03-11T19:26:43.003 INFO:teuthology.orchestra.run.smithi083.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-03-11T19:26:43.028 INFO:teuthology.orchestra.run.smithi083.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-03-11T19:26:43.053 INFO:teuthology.orchestra.run.smithi083.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 941 kB/s | 79 kB 00:00 2024-03-11T19:26:43.078 INFO:teuthology.orchestra.run.smithi083.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-11T19:26:43.104 INFO:teuthology.orchestra.run.smithi083.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-03-11T19:26:43.129 INFO:teuthology.orchestra.run.smithi083.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-03-11T19:26:43.155 INFO:teuthology.orchestra.run.smithi083.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-03-11T19:26:43.180 INFO:teuthology.orchestra.run.smithi083.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-03-11T19:26:43.205 INFO:teuthology.orchestra.run.smithi083.stdout:(140/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-03-11T19:26:43.230 INFO:teuthology.orchestra.run.smithi083.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-11T19:26:43.256 INFO:teuthology.orchestra.run.smithi083.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-03-11T19:26:43.281 INFO:teuthology.orchestra.run.smithi083.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-03-11T19:26:43.306 INFO:teuthology.orchestra.run.smithi083.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-03-11T19:26:43.332 INFO:teuthology.orchestra.run.smithi083.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-03-11T19:26:43.365 INFO:teuthology.orchestra.run.smithi083.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-03-11T19:26:43.393 INFO:teuthology.orchestra.run.smithi083.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-03-11T19:26:43.416 INFO:teuthology.orchestra.run.smithi083.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-03-11T19:26:43.449 INFO:teuthology.orchestra.run.smithi083.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-03-11T19:26:43.473 INFO:teuthology.orchestra.run.smithi137.stdout:(14/174): libcephfs2-19.0.0-1607.g94a53970.el9. 1.4 MB/s | 742 kB 00:00 2024-03-11T19:26:43.491 INFO:teuthology.orchestra.run.smithi083.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-03-11T19:26:43.583 INFO:teuthology.orchestra.run.smithi083.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.4 MB/s | 321 kB 00:00 2024-03-11T19:26:43.606 INFO:teuthology.orchestra.run.smithi137.stdout:(15/174): libcephsqlite-19.0.0-1607.g94a53970.e 1.2 MB/s | 163 kB 00:00 2024-03-11T19:26:43.732 INFO:teuthology.orchestra.run.smithi137.stdout:(16/174): librados-devel-19.0.0-1607.g94a53970. 1.0 MB/s | 127 kB 00:00 2024-03-11T19:26:43.857 INFO:teuthology.orchestra.run.smithi137.stdout:(17/174): libradosstriper1-19.0.0-1607.g94a5397 3.9 MB/s | 498 kB 00:00 2024-03-11T19:26:44.084 INFO:teuthology.orchestra.run.smithi083.stdout:(152/174): librbd1-19.0.0-1607.g94a53970.el9.x8 4.9 MB/s | 3.1 MB 00:00 2024-03-11T19:26:44.158 INFO:teuthology.orchestra.run.smithi083.stdout:(153/174): NetworkManager-1.46.0-1.el9.x86_64.r 4.0 MB/s | 2.3 MB 00:00 2024-03-11T19:26:44.209 INFO:teuthology.orchestra.run.smithi083.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 848 kB/s | 42 kB 00:00 2024-03-11T19:26:44.268 INFO:teuthology.orchestra.run.smithi083.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-11T19:26:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-11T19:26:44.377 INFO:teuthology.orchestra.run.smithi083.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 729 kB/s | 36 kB 00:00 2024-03-11T19:26:44.435 INFO:teuthology.orchestra.run.smithi083.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-11T19:26:44.502 INFO:teuthology.orchestra.run.smithi083.stdout:(159/174): NetworkManager-libnm-1.46.0-1.el9.x8 4.3 MB/s | 1.8 MB 00:00 2024-03-11T19:26:44.536 INFO:teuthology.orchestra.run.smithi083.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.5 MB/s | 256 kB 00:00 2024-03-11T19:26:44.561 INFO:teuthology.orchestra.run.smithi083.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 395 kB/s | 23 kB 00:00 2024-03-11T19:26:44.586 INFO:teuthology.orchestra.run.smithi083.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-11T19:26:44.599 INFO:teuthology.orchestra.run.smithi137.stdout:(18/174): ceph-test-19.0.0-1607.g94a53970.el9.x 17 MB/s | 47 MB 00:02 2024-03-11T19:26:44.620 INFO:teuthology.orchestra.run.smithi083.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-03-11T19:26:44.645 INFO:teuthology.orchestra.run.smithi083.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 136 kB 00:00 2024-03-11T19:26:44.657 INFO:teuthology.orchestra.run.smithi137.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 779 kB/s | 45 kB 00:00 2024-03-11T19:26:44.679 INFO:teuthology.orchestra.run.smithi083.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-11T19:26:44.704 INFO:teuthology.orchestra.run.smithi083.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 490 kB/s | 29 kB 00:00 2024-03-11T19:26:44.724 INFO:teuthology.orchestra.run.smithi137.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.9 MB/s | 131 kB 00:00 2024-03-11T19:26:44.863 INFO:teuthology.orchestra.run.smithi083.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 6.6 MB/s | 1.2 MB 00:00 2024-03-11T19:26:44.908 INFO:teuthology.orchestra.run.smithi137.stdout:(21/174): ceph-common-19.0.0-1607.g94a53970.el9 3.1 MB/s | 20 MB 00:06 2024-03-11T19:26:44.933 INFO:teuthology.orchestra.run.smithi137.stdout:(22/174): python3-cephfs-19.0.0-1607.g94a53970. 774 kB/s | 161 kB 00:00 2024-03-11T19:26:44.963 INFO:teuthology.orchestra.run.smithi083.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 8.3 MB/s | 2.2 MB 00:00 2024-03-11T19:26:45.000 INFO:teuthology.orchestra.run.smithi137.stdout:(23/174): librgw2-19.0.0-1607.g94a53970.el9.x86 4.5 MB/s | 5.1 MB 00:01 2024-03-11T19:26:45.025 INFO:teuthology.orchestra.run.smithi137.stdout:(24/174): python3-rbd-19.0.0-1607.g94a53970.el9 3.2 MB/s | 298 kB 00:00 2024-03-11T19:26:45.063 INFO:teuthology.orchestra.run.smithi083.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 4.6 MB/s | 465 kB 00:00 2024-03-11T19:26:45.076 INFO:teuthology.orchestra.run.smithi137.stdout:(25/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.3 MB/s | 99 kB 00:00 2024-03-11T19:26:45.101 INFO:teuthology.orchestra.run.smithi137.stdout:(26/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-11T19:26:45.113 INFO:teuthology.orchestra.run.smithi083.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 623 kB/s | 31 kB 00:00 2024-03-11T19:26:45.126 INFO:teuthology.orchestra.run.smithi137.stdout:(27/174): python3-rados-19.0.0-1607.g94a53970.e 1.4 MB/s | 320 kB 00:00 2024-03-11T19:26:45.160 INFO:teuthology.orchestra.run.smithi137.stdout:(28/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 2.9 MB/s | 171 kB 00:00 2024-03-11T19:26:45.164 INFO:teuthology.orchestra.run.smithi083.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.6 MB/s | 2.3 MB 00:00 2024-03-11T19:26:45.202 INFO:teuthology.orchestra.run.smithi137.stdout:(29/174): ceph-grafana-dashboards-19.0.0-1607.g 418 kB/s | 31 kB 00:00 2024-03-11T19:26:45.227 INFO:teuthology.orchestra.run.smithi137.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 2.0 MB/s | 138 kB 00:00 2024-03-11T19:26:45.230 INFO:teuthology.orchestra.run.smithi083.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 74 kB 00:00 2024-03-11T19:26:45.273 INFO:teuthology.orchestra.run.smithi083.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2024-03-11T19:26:45.348 INFO:teuthology.orchestra.run.smithi083.stdout:(174/174): librados2-19.0.0-1607.g94a53970.el9. 1.7 MB/s | 3.4 MB 00:01 2024-03-11T19:26:45.353 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:26:45.353 INFO:teuthology.orchestra.run.smithi083.stdout:Total 14 MB/s | 223 MB 00:16 2024-03-11T19:26:45.577 INFO:teuthology.orchestra.run.smithi137.stdout:(31/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 6.2 MB/s | 3.1 MB 00:00 2024-03-11T19:26:45.694 INFO:teuthology.orchestra.run.smithi137.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-03-11T19:26:45.719 INFO:teuthology.orchestra.run.smithi137.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1607.g94 1.7 MB/s | 247 kB 00:00 2024-03-11T19:26:45.744 INFO:teuthology.orchestra.run.smithi137.stdout:(34/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 990 kB/s | 49 kB 00:00 2024-03-11T19:26:45.778 INFO:teuthology.orchestra.run.smithi137.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1607.g9 262 kB/s | 15 kB 00:00 2024-03-11T19:26:45.803 INFO:teuthology.orchestra.run.smithi137.stdout:(36/174): ceph-volume-19.0.0-1607.g94a53970.el9 4.4 MB/s | 264 kB 00:00 2024-03-11T19:26:45.837 INFO:teuthology.orchestra.run.smithi137.stdout:(37/174): cephfs-top-19.0.0-1607.g94a53970.el9. 542 kB/s | 18 kB 00:00 2024-03-11T19:26:45.838 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-03-11T19:26:45.938 INFO:teuthology.orchestra.run.smithi137.stdout:(38/174): cephadm-19.0.0-1607.g94a53970.el9.noa 4.6 MB/s | 748 kB 00:00 2024-03-11T19:26:46.071 INFO:teuthology.orchestra.run.smithi137.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 662 kB 00:00 2024-03-11T19:26:46.105 INFO:teuthology.orchestra.run.smithi137.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 5.4 MB/s | 920 kB 00:00 2024-03-11T19:26:46.130 INFO:teuthology.orchestra.run.smithi137.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-11T19:26:46.151 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-03-11T19:26:46.151 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-03-11T19:26:46.172 INFO:teuthology.orchestra.run.smithi137.stdout:(42/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 1.8 MB/s | 1.8 MB 00:00 2024-03-11T19:26:46.197 INFO:teuthology.orchestra.run.smithi137.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-11T19:26:46.222 INFO:teuthology.orchestra.run.smithi137.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 344 kB/s | 32 kB 00:00 2024-03-11T19:26:46.331 INFO:teuthology.orchestra.run.smithi137.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-03-11T19:26:46.356 INFO:teuthology.orchestra.run.smithi137.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 76 kB/s | 10 kB 00:00 2024-03-11T19:26:46.382 INFO:teuthology.orchestra.run.smithi137.stdout:(47/174): flexiblas-openblas-openmp-3.0.4-8.el9 109 kB/s | 17 kB 00:00 2024-03-11T19:26:46.407 INFO:teuthology.orchestra.run.smithi137.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-11T19:26:46.457 INFO:teuthology.orchestra.run.smithi137.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 441 kB/s | 44 kB 00:00 2024-03-11T19:26:46.482 INFO:teuthology.orchestra.run.smithi137.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 578 kB 00:00 2024-03-11T19:26:46.507 INFO:teuthology.orchestra.run.smithi137.stdout:(51/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 661 kB 00:00 2024-03-11T19:26:46.533 INFO:teuthology.orchestra.run.smithi137.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 624 kB/s | 46 kB 00:00 2024-03-11T19:26:46.558 INFO:teuthology.orchestra.run.smithi137.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-11T19:26:46.583 INFO:teuthology.orchestra.run.smithi137.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-11T19:26:46.616 INFO:teuthology.orchestra.run.smithi137.stdout:(55/174): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-03-11T19:26:46.642 INFO:teuthology.orchestra.run.smithi137.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 409 kB/s | 34 kB 00:00 2024-03-11T19:26:46.751 INFO:teuthology.orchestra.run.smithi137.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-03-11T19:26:46.784 INFO:teuthology.orchestra.run.smithi137.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 5.7 MB/s | 811 kB 00:00 2024-03-11T19:26:46.901 INFO:teuthology.orchestra.run.smithi137.stdout:(59/174): postgresql-13.11-1.el9.x86_64.rpm 5.9 MB/s | 1.5 MB 00:00 2024-03-11T19:26:46.985 INFO:teuthology.orchestra.run.smithi137.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 573 kB/s | 134 kB 00:00 2024-03-11T19:26:47.177 INFO:teuthology.orchestra.run.smithi137.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2024-03-11T19:26:47.202 INFO:teuthology.orchestra.run.smithi137.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 363 kB/s | 78 kB 00:00 2024-03-11T19:26:47.244 INFO:teuthology.orchestra.run.smithi137.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-03-11T19:26:47.413 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 664 kB/s | 6.1 MB 00:09 2024-03-11T19:26:47.419 INFO:teuthology.orchestra.run.smithi137.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 5.8 MB 00:00 2024-03-11T19:26:47.444 INFO:teuthology.orchestra.run.smithi137.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 101 kB/s | 24 kB 00:00 2024-03-11T19:26:47.469 INFO:teuthology.orchestra.run.smithi137.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 81 kB/s | 18 kB 00:00 2024-03-11T19:26:47.503 INFO:teuthology.orchestra.run.smithi137.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-11T19:26:47.536 INFO:teuthology.orchestra.run.smithi137.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 476 kB/s | 32 kB 00:00 2024-03-11T19:26:47.803 INFO:teuthology.orchestra.run.smithi137.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 4.8 MB 00:00 2024-03-11T19:26:47.829 INFO:teuthology.orchestra.run.smithi137.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 668 kB/s | 217 kB 00:00 2024-03-11T19:26:47.854 INFO:teuthology.orchestra.run.smithi137.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 552 kB/s | 174 kB 00:00 2024-03-11T19:26:47.888 INFO:teuthology.orchestra.run.smithi137.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 828 kB/s | 69 kB 00:00 2024-03-11T19:26:47.988 INFO:teuthology.orchestra.run.smithi137.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 309 kB/s | 41 kB 00:00 2024-03-11T19:26:48.013 INFO:teuthology.orchestra.run.smithi137.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 238 kB 00:00 2024-03-11T19:26:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 879 kB/s | 132 kB 00:00 2024-03-11T19:26:48.063 INFO:teuthology.orchestra.run.smithi137.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-11T19:26:48.097 INFO:teuthology.orchestra.run.smithi137.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 738 kB/s | 43 kB 00:00 2024-03-11T19:26:48.114 INFO:teuthology.orchestra.run.smithi137.stdout:(78/174): python3-pytz-2021.1-5.el9.noarch.rpm 630 kB/s | 47 kB 00:00 2024-03-11T19:26:48.182 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-03-11T19:26:48.183 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-03-11T19:26:48.564 INFO:teuthology.orchestra.run.smithi137.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 83 kB/s | 39 kB 00:00 2024-03-11T19:26:48.606 INFO:teuthology.orchestra.run.smithi137.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 83 kB/s | 41 kB 00:00 2024-03-11T19:26:48.790 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 15 kB 00:00 2024-03-11T19:26:48.865 INFO:teuthology.orchestra.run.smithi137.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-03-11T19:26:48.898 INFO:teuthology.orchestra.run.smithi137.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 902 kB/s | 300 kB 00:00 2024-03-11T19:26:48.927 INFO:teuthology.orchestra.run.smithi137.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 176 kB/s | 56 kB 00:00 2024-03-11T19:26:48.952 INFO:teuthology.orchestra.run.smithi137.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 727 kB/s | 63 kB 00:00 2024-03-11T19:26:48.977 INFO:teuthology.orchestra.run.smithi137.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 884 kB/s | 44 kB 00:00 2024-03-11T19:26:49.002 INFO:teuthology.orchestra.run.smithi137.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-11T19:26:49.028 INFO:teuthology.orchestra.run.smithi137.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-11T19:26:49.045 INFO:teuthology.orchestra.run.smithi137.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 2.4 MB/s | 103 kB 00:00 2024-03-11T19:26:49.087 INFO:teuthology.orchestra.run.smithi137.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-11T19:26:49.112 INFO:teuthology.orchestra.run.smithi137.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-03-11T19:26:49.137 INFO:teuthology.orchestra.run.smithi137.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-11T19:26:49.913 INFO:teuthology.orchestra.run.smithi137.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 252 kB/s | 195 kB 00:00 2024-03-11T19:26:50.071 INFO:teuthology.orchestra.run.smithi137.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 190 kB/s | 187 kB 00:00 2024-03-11T19:26:50.138 INFO:teuthology.orchestra.run.smithi137.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 554 kB/s | 125 kB 00:00 2024-03-11T19:26:50.255 INFO:teuthology.orchestra.run.smithi137.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 117 kB/s | 21 kB 00:00 2024-03-11T19:26:50.280 INFO:teuthology.orchestra.run.smithi137.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 179 kB/s | 25 kB 00:00 2024-03-11T19:26:50.372 INFO:teuthology.orchestra.run.smithi137.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 651 kB/s | 803 kB 00:01 2024-03-11T19:26:50.589 INFO:teuthology.orchestra.run.smithi137.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 795 kB/s | 245 kB 00:00 2024-03-11T19:26:50.773 INFO:teuthology.orchestra.run.smithi137.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.2 MB/s | 216 kB 00:00 2024-03-11T19:26:50.798 INFO:teuthology.orchestra.run.smithi137.stdout:(100/174): python3-jinja2-2.11.3-5.el9.noarch.r 583 kB/s | 248 kB 00:00 2024-03-11T19:26:50.832 INFO:teuthology.orchestra.run.smithi137.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 595 kB/s | 35 kB 00:00 2024-03-11T19:26:50.849 INFO:teuthology.orchestra.run.smithi137.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 883 kB/s | 44 kB 00:00 2024-03-11T19:26:50.924 INFO:teuthology.orchestra.run.smithi137.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.0 MB/s | 94 kB 00:00 2024-03-11T19:26:50.941 INFO:teuthology.orchestra.run.smithi137.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-03-11T19:26:51.100 INFO:teuthology.orchestra.run.smithi137.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-03-11T19:26:51.268 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 21 MB 00:02 2024-03-11T19:26:51.325 INFO:teuthology.orchestra.run.smithi137.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.9 MB/s | 4.1 MB 00:01 2024-03-11T19:26:51.350 INFO:teuthology.orchestra.run.smithi137.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-03-11T19:26:51.450 INFO:teuthology.orchestra.run.smithi137.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 881 kB/s | 309 kB 00:00 2024-03-11T19:26:51.509 INFO:teuthology.orchestra.run.smithi137.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 456 kB/s | 26 kB 00:00 2024-03-11T19:26:51.542 INFO:teuthology.orchestra.run.smithi137.stdout:(110/174): grpc-data-1.46.7-10.el9.noarch.rpm 90 kB/s | 19 kB 00:00 2024-03-11T19:26:51.568 INFO:teuthology.orchestra.run.smithi137.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 831 kB/s | 49 kB 00:00 2024-03-11T19:26:51.635 INFO:teuthology.orchestra.run.smithi137.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-11T19:26:51.660 INFO:teuthology.orchestra.run.smithi137.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 575 kB/s | 67 kB 00:00 2024-03-11T19:26:51.811 INFO:teuthology.orchestra.run.smithi137.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-03-11T19:26:51.844 INFO:teuthology.orchestra.run.smithi137.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-03-11T19:26:51.877 INFO:teuthology.orchestra.run.smithi137.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-03-11T19:26:51.902 INFO:teuthology.orchestra.run.smithi137.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-03-11T19:26:51.928 INFO:teuthology.orchestra.run.smithi137.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 383 kB/s | 32 kB 00:00 2024-03-11T19:26:51.953 INFO:teuthology.orchestra.run.smithi137.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 198 kB/s | 15 kB 00:00 2024-03-11T19:26:51.978 INFO:teuthology.orchestra.run.smithi137.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-03-11T19:26:52.004 INFO:teuthology.orchestra.run.smithi137.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-03-11T19:26:52.037 INFO:teuthology.orchestra.run.smithi137.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-03-11T19:26:52.062 INFO:teuthology.orchestra.run.smithi137.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-03-11T19:26:52.088 INFO:teuthology.orchestra.run.smithi137.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-11T19:26:52.121 INFO:teuthology.orchestra.run.smithi137.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-11T19:26:52.146 INFO:teuthology.orchestra.run.smithi137.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-03-11T19:26:52.180 INFO:teuthology.orchestra.run.smithi137.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-03-11T19:26:52.205 INFO:teuthology.orchestra.run.smithi137.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-03-11T19:26:52.247 INFO:teuthology.orchestra.run.smithi137.stdout:(129/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.7 MB/s | 2.0 MB 00:00 2024-03-11T19:26:52.273 INFO:teuthology.orchestra.run.smithi137.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 210 kB/s | 19 kB 00:00 2024-03-11T19:26:52.298 INFO:teuthology.orchestra.run.smithi137.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-03-11T19:26:52.381 INFO:teuthology.orchestra.run.smithi137.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 556 kB/s | 46 kB 00:00 2024-03-11T19:26:52.407 INFO:teuthology.orchestra.run.smithi137.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 727 kB/s | 79 kB 00:00 2024-03-11T19:26:52.449 INFO:teuthology.orchestra.run.smithi137.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-03-11T19:26:52.474 INFO:teuthology.orchestra.run.smithi137.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 625 kB/s | 58 kB 00:00 2024-03-11T19:26:52.516 INFO:teuthology.orchestra.run.smithi137.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-03-11T19:26:52.541 INFO:teuthology.orchestra.run.smithi137.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-03-11T19:26:52.567 INFO:teuthology.orchestra.run.smithi137.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-03-11T19:26:52.593 INFO:teuthology.orchestra.run.smithi137.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 370 kB/s | 28 kB 00:00 2024-03-11T19:26:52.618 INFO:teuthology.orchestra.run.smithi137.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 404 kB/s | 31 kB 00:00 2024-03-11T19:26:52.643 INFO:teuthology.orchestra.run.smithi137.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-11T19:26:52.660 INFO:teuthology.orchestra.run.smithi137.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 883 kB/s | 59 kB 00:00 2024-03-11T19:26:52.677 INFO:teuthology.orchestra.run.smithi137.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 608 kB/s | 36 kB 00:00 2024-03-11T19:26:52.702 INFO:teuthology.orchestra.run.smithi137.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 868 kB/s | 51 kB 00:00 2024-03-11T19:26:52.727 INFO:teuthology.orchestra.run.smithi137.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-03-11T19:26:52.753 INFO:teuthology.orchestra.run.smithi137.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-11T19:26:52.778 INFO:teuthology.orchestra.run.smithi137.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-03-11T19:26:52.795 INFO:teuthology.orchestra.run.smithi137.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 297 kB/s | 20 kB 00:00 2024-03-11T19:26:52.820 INFO:teuthology.orchestra.run.smithi137.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-03-11T19:26:53.029 INFO:teuthology.orchestra.run.smithi137.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-03-11T19:26:53.171 INFO:teuthology.orchestra.run.smithi137.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.2 MB/s | 321 kB 00:00 2024-03-11T19:26:53.529 INFO:teuthology.orchestra.run.smithi137.stdout:(152/174): librbd1-19.0.0-1607.g94a53970.el9.x8 4.4 MB/s | 3.1 MB 00:00 2024-03-11T19:26:54.071 INFO:teuthology.orchestra.run.smithi137.stdout:(153/174): librados2-19.0.0-1607.g94a53970.el9. 2.6 MB/s | 3.4 MB 00:01 2024-03-11T19:26:54.596 INFO:teuthology.orchestra.run.smithi137.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 81 kB/s | 42 kB 00:00 2024-03-11T19:26:55.061 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-03-11T19:26:55.153 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-11T19:26:55.178 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-11T19:26:55.283 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:26:55.329 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:26:55.371 INFO:teuthology.orchestra.run.smithi137.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 320 kB/s | 248 kB 00:00 2024-03-11T19:26:55.501 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-11T19:26:55.573 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-11T19:26:55.612 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-11T19:26:55.637 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-11T19:26:55.687 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-11T19:26:55.696 INFO:teuthology.orchestra.run.smithi137.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 220 kB/s | 71 kB 00:00 2024-03-11T19:26:55.788 INFO:teuthology.orchestra.run.smithi137.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 398 kB/s | 36 kB 00:00 2024-03-11T19:26:56.018 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-11T19:26:56.064 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:26:56.185 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:26:56.223 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:26:56.240 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-11T19:26:56.263 INFO:teuthology.orchestra.run.smithi137.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 405 kB/s | 192 kB 00:00 2024-03-11T19:26:56.317 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:26:56.350 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:26:56.406 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-11T19:26:56.595 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-11T19:26:56.649 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-11T19:26:56.677 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:26:56.763 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:26:56.786 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:26:56.835 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:26:56.865 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-11T19:26:56.888 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:26:56.903 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:26:57.103 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:26:57.136 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:26:57.176 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-11T19:26:57.361 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-11T19:26:57.414 INFO:teuthology.orchestra.run.smithi137.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 222 kB/s | 256 kB 00:01 2024-03-11T19:26:57.510 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:26:57.554 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:26:57.589 INFO:teuthology.orchestra.run.smithi137.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 131 kB/s | 23 kB 00:00 2024-03-11T19:26:57.680 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:26:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:26:57.806 INFO:teuthology.orchestra.run.smithi137.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 500 kB/s | 108 kB 00:00 2024-03-11T19:26:57.957 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:26:58.006 INFO:teuthology.orchestra.run.smithi137.stdout:(162/174): NetworkManager-1.46.0-1.el9.x86_64.r 485 kB/s | 2.3 MB 00:04 2024-03-11T19:26:58.062 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-11T19:26:58.157 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-11T19:26:58.188 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-11T19:26:58.198 INFO:teuthology.orchestra.run.smithi137.stdout:(163/174): NetworkManager-libnm-1.46.0-1.el9.x8 399 kB/s | 1.8 MB 00:04 2024-03-11T19:26:58.214 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-11T19:26:58.249 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-11T19:26:58.257 INFO:teuthology.orchestra.run.smithi137.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 344 kB/s | 155 kB 00:00 2024-03-11T19:26:58.323 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-11T19:26:58.349 INFO:teuthology.orchestra.run.smithi137.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 313 kB/s | 29 kB 00:00 2024-03-11T19:26:58.374 INFO:teuthology.orchestra.run.smithi137.stdout:(166/174): libmount-2.37.4-18.el9.x86_64.rpm 369 kB/s | 136 kB 00:00 2024-03-11T19:26:58.377 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-11T19:26:58.416 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-11T19:26:58.436 INFO:teuthology.orchestra.run.smithi137.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 268 kB/s | 64 kB 00:00 2024-03-11T19:26:58.499 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-11T19:26:58.534 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-11T19:26:58.578 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-11T19:26:58.611 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-11T19:26:58.638 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-11T19:26:58.706 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-11T19:26:58.720 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:26:58.721 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-11T19:26:58.735 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-11T19:26:58.736 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-11T19:26:58.737 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-11T19:26:58.738 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-11T19:26:58.740 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-11T19:26:58.743 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-11T19:26:58.744 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-11T19:26:58.751 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-11T19:26:58.794 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-11T19:26:58.823 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-03-11T19:26:58.832 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-11T19:26:58.851 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-03-11T19:26:58.851 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-03-11T19:26:58.851 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-03-11T19:26:58.852 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-03-11T19:26:58.852 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-11T19:26:58.852 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.0.0-1607.g94a53970.el9 ceph 6.6 k 2024-03-11T19:26:58.852 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.3 M 2024-03-11T19:26:58.852 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 886 k 2024-03-11T19:26:58.852 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1607.g94a53970.el9 ceph 145 k 2024-03-11T19:26:58.853 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.0.0-1607.g94a53970.el9 ceph 1.5 M 2024-03-11T19:26:58.853 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 138 k 2024-03-11T19:26:58.853 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 1.8 M 2024-03-11T19:26:58.853 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 7.4 M 2024-03-11T19:26:58.853 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 49 k 2024-03-11T19:26:58.853 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 9.8 M 2024-03-11T19:26:58.853 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.0.0-1607.g94a53970.el9 ceph 47 M 2024-03-11T19:26:58.853 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 264 k 2024-03-11T19:26:58.854 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 748 k 2024-03-11T19:26:58.854 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 217 k 2024-03-11T19:26:58.854 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.0.0-1607.g94a53970.el9 ceph 27 k 2024-03-11T19:26:58.854 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 18 k 2024-03-11T19:26:58.854 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-11T19:26:58.854 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-11T19:26:58.854 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-11T19:26:58.854 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 31 k 2024-03-11T19:26:58.855 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 742 k 2024-03-11T19:26:58.855 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.0.0-1607.g94a53970.el9 ceph 127 k 2024-03-11T19:26:58.855 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-11T19:26:58.855 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-11T19:26:58.855 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-11T19:26:58.855 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-11T19:26:58.855 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-11T19:26:58.855 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-11T19:26:58.856 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.0.0-1607.g94a53970.el9 ceph 161 k 2024-03-11T19:26:58.856 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.0.0-1607.g94a53970.el9 ceph 320 k 2024-03-11T19:26:58.856 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 298 k 2024-03-11T19:26:58.856 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.0.0-1607.g94a53970.el9 ceph 99 k 2024-03-11T19:26:58.856 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 85 k 2024-03-11T19:26:58.856 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:26:58.856 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 171 k 2024-03-11T19:26:58.856 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-11T19:26:58.857 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-03-11T19:26:58.857 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-11T19:26:58.857 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-11T19:26:58.857 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-11T19:26:58.857 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-11T19:26:58.857 INFO:teuthology.orchestra.run.smithi204.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-11T19:26:58.857 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-11T19:26:58.857 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-11T19:26:58.858 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-11T19:26:58.858 INFO:teuthology.orchestra.run.smithi204.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-11T19:26:58.858 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-11T19:26:58.858 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-11T19:26:58.858 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-11T19:26:58.858 INFO:teuthology.orchestra.run.smithi204.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-11T19:26:58.858 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.4 M 2024-03-11T19:26:58.858 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 3.1 M 2024-03-11T19:26:58.859 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-11T19:26:58.859 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-11T19:26:58.859 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-11T19:26:58.859 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-11T19:26:58.859 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-11T19:26:58.859 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-11T19:26:58.859 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-11T19:26:58.859 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-11T19:26:58.860 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-11T19:26:58.860 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-03-11T19:26:58.860 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-11T19:26:58.860 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-11T19:26:58.860 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-11T19:26:58.860 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 20 M 2024-03-11T19:26:58.860 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 31 k 2024-03-11T19:26:58.860 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.0.0-1607.g94a53970.el9 ceph 2.1 M 2024-03-11T19:26:58.860 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 247 k 2024-03-11T19:26:58.861 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.0.0-1607.g94a53970.el9 ceph 4.7 M 2024-03-11T19:26:58.861 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.0.0-1607.g94a53970.el9 ceph 17 M 2024-03-11T19:26:58.861 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1607.g94a53970.el9 ceph-noarch 15 k 2024-03-11T19:26:58.861 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.0.0-1607.g94a53970.el9 ceph 25 k 2024-03-11T19:26:58.861 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-11T19:26:58.861 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-11T19:26:58.862 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-11T19:26:58.862 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-11T19:26:58.862 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-11T19:26:58.862 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-11T19:26:58.862 INFO:teuthology.orchestra.run.smithi204.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-11T19:26:58.862 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-11T19:26:58.862 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-11T19:26:58.862 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.0.0-1607.g94a53970.el9 ceph 163 k 2024-03-11T19:26:58.863 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-11T19:26:58.863 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-11T19:26:58.863 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-11T19:26:58.863 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-11T19:26:58.863 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-11T19:26:58.863 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 498 k 2024-03-11T19:26:58.863 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-11T19:26:58.863 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.0.0-1607.g94a53970.el9 ceph 5.1 M 2024-03-11T19:26:58.864 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-11T19:26:58.864 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-11T19:26:58.864 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-11T19:26:58.864 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-11T19:26:58.864 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-11T19:26:58.864 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-11T19:26:58.864 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-11T19:26:58.865 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-11T19:26:58.865 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-11T19:26:58.865 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-11T19:26:58.865 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-11T19:26:58.865 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-11T19:26:58.865 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-11T19:26:58.865 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-11T19:26:58.865 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-11T19:26:58.866 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-11T19:26:58.866 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-11T19:26:58.866 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-11T19:26:58.866 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.0.0-1607.g94a53970.el9 ceph 45 k 2024-03-11T19:26:58.866 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.0.0-1607.g94a53970.el9 ceph 131 k 2024-03-11T19:26:58.866 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-11T19:26:58.866 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-11T19:26:58.867 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-11T19:26:58.867 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-11T19:26:58.867 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-11T19:26:58.867 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-11T19:26:58.867 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-11T19:26:58.867 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-11T19:26:58.867 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-11T19:26:58.867 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-11T19:26:58.868 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-11T19:26:58.868 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-11T19:26:58.868 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-11T19:26:58.868 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-11T19:26:58.868 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-11T19:26:58.868 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-11T19:26:58.868 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-11T19:26:58.868 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-11T19:26:58.869 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-11T19:26:58.869 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-11T19:26:58.869 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-11T19:26:58.869 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-11T19:26:58.869 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-11T19:26:58.869 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-11T19:26:58.869 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-11T19:26:58.869 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-11T19:26:58.870 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-11T19:26:58.870 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-11T19:26:58.870 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-11T19:26:58.870 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-11T19:26:58.870 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-11T19:26:58.870 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-11T19:26:58.870 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-11T19:26:58.871 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-11T19:26:58.871 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-11T19:26:58.871 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-11T19:26:58.871 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-11T19:26:58.871 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-11T19:26:58.871 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-11T19:26:58.871 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-11T19:26:58.871 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-11T19:26:58.872 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-11T19:26:58.872 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-11T19:26:58.872 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-11T19:26:58.872 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-11T19:26:58.872 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-11T19:26:58.872 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-11T19:26:58.872 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-11T19:26:58.872 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-11T19:26:58.873 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-11T19:26:58.873 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-11T19:26:58.873 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-11T19:26:58.873 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-11T19:26:58.873 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-11T19:26:58.873 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-11T19:26:58.873 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-11T19:26:58.874 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-11T19:26:58.874 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-11T19:26:58.874 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-11T19:26:58.874 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-11T19:26:58.874 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-11T19:26:58.874 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-11T19:26:58.874 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-11T19:26:58.875 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-03-11T19:26:58.875 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-11T19:26:58.875 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:26:58.875 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-03-11T19:26:58.875 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-03-11T19:26:58.875 INFO:teuthology.orchestra.run.smithi204.stdout:Install 150 Packages 2024-03-11T19:26:58.875 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 24 Packages 2024-03-11T19:26:58.875 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:26:58.876 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 223 M 2024-03-11T19:26:58.876 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-03-11T19:26:58.901 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-11T19:26:58.946 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-11T19:26:59.019 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-11T19:26:59.154 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-11T19:26:59.207 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-11T19:26:59.358 INFO:teuthology.orchestra.run.smithi137.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 1.2 MB/s | 1.2 MB 00:01 2024-03-11T19:26:59.725 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-11T19:26:59.759 INFO:teuthology.orchestra.run.smithi204.stdout:(1/174): ceph-base-19.0.0-1607.g94a53970.el9.x8 9.9 MB/s | 5.3 MB 00:00 2024-03-11T19:26:59.803 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-11T19:26:59.859 INFO:teuthology.orchestra.run.smithi204.stdout:(2/174): ceph-fuse-19.0.0-1607.g94a53970.el9.x8 8.7 MB/s | 886 kB 00:00 2024-03-11T19:26:59.883 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-11T19:26:59.918 INFO:teuthology.orchestra.run.smithi204.stdout:(3/174): ceph-immutable-object-cache-19.0.0-160 2.4 MB/s | 145 kB 00:00 2024-03-11T19:26:59.940 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:26:59.949 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:26:59.981 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-11T19:27:00.029 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-11T19:27:00.066 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-11T19:27:00.093 INFO:teuthology.orchestra.run.smithi204.stdout:(4/174): ceph-mds-19.0.0-1607.g94a53970.el9.x86 12 MB/s | 2.1 MB 00:00 2024-03-11T19:27:00.097 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-11T19:27:00.235 INFO:teuthology.orchestra.run.smithi204.stdout:(5/174): ceph-mgr-19.0.0-1607.g94a53970.el9.x86 11 MB/s | 1.5 MB 00:00 2024-03-11T19:27:00.351 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-11T19:27:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-11T19:27:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-11T19:27:00.643 INFO:teuthology.orchestra.run.smithi204.stdout:(6/174): ceph-mon-19.0.0-1607.g94a53970.el9.x86 11 MB/s | 4.7 MB 00:00 2024-03-11T19:27:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-11T19:27:00.867 INFO:teuthology.orchestra.run.smithi137.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 308 kB/s | 465 kB 00:01 2024-03-11T19:27:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-11T19:27:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-11T19:27:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-11T19:27:01.050 INFO:teuthology.orchestra.run.smithi137.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 170 kB/s | 31 kB 00:00 2024-03-11T19:27:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-11T19:27:01.094 INFO:teuthology.orchestra.run.smithi204.stdout:(7/174): ceph-common-19.0.0-1607.g94a53970.el9. 11 MB/s | 20 MB 00:01 2024-03-11T19:27:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-11T19:27:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-11T19:27:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-11T19:27:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-11T19:27:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-11T19:27:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-11T19:27:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-11T19:27:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-11T19:27:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-11T19:27:01.569 INFO:teuthology.orchestra.run.smithi204.stdout:(8/174): ceph-19.0.0-1607.g94a53970.el9.x86_64. 2.8 kB/s | 6.6 kB 00:02 2024-03-11T19:27:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-11T19:27:01.625 INFO:teuthology.orchestra.run.smithi137.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 679 kB/s | 2.2 MB 00:03 2024-03-11T19:27:01.689 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-11T19:27:01.719 INFO:teuthology.orchestra.run.smithi204.stdout:(9/174): ceph-selinux-19.0.0-1607.g94a53970.el9 168 kB/s | 25 kB 00:00 2024-03-11T19:27:01.729 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-11T19:27:01.770 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-11T19:27:01.784 INFO:teuthology.orchestra.run.smithi137.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 469 kB/s | 74 kB 00:00 2024-03-11T19:27:01.795 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-11T19:27:01.820 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-11T19:27:01.912 INFO:teuthology.orchestra.run.smithi204.stdout:(10/174): ceph-osd-19.0.0-1607.g94a53970.el9.x8 13 MB/s | 17 MB 00:01 2024-03-11T19:27:01.923 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-11T19:27:02.028 INFO:teuthology.orchestra.run.smithi204.stdout:(11/174): ceph-radosgw-19.0.0-1607.g94a53970.el 10 MB/s | 9.8 MB 00:00 2024-03-11T19:27:02.046 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-11T19:27:02.053 INFO:teuthology.orchestra.run.smithi204.stdout:(12/174): cephfs-mirror-19.0.0-1607.g94a53970.e 1.5 MB/s | 217 kB 00:00 2024-03-11T19:27:02.070 INFO:teuthology.orchestra.run.smithi204.stdout:(13/174): cephfs-shell-19.0.0-1607.g94a53970.el 647 kB/s | 27 kB 00:00 2024-03-11T19:27:02.087 INFO:teuthology.orchestra.run.smithi204.stdout:(14/174): libcephfs-devel-19.0.0-1607.g94a53970 930 kB/s | 31 kB 00:00 2024-03-11T19:27:02.146 INFO:teuthology.orchestra.run.smithi204.stdout:(15/174): libcephfs2-19.0.0-1607.g94a53970.el9. 9.6 MB/s | 742 kB 00:00 2024-03-11T19:27:02.161 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-11T19:27:02.171 INFO:teuthology.orchestra.run.smithi204.stdout:(16/174): libcephsqlite-19.0.0-1607.g94a53970.e 1.9 MB/s | 163 kB 00:00 2024-03-11T19:27:02.203 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-11T19:27:02.213 INFO:teuthology.orchestra.run.smithi204.stdout:(17/174): librados-devel-19.0.0-1607.g94a53970. 1.8 MB/s | 127 kB 00:00 2024-03-11T19:27:02.252 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-11T19:27:02.264 INFO:teuthology.orchestra.run.smithi204.stdout:(18/174): libradosstriper1-19.0.0-1607.g94a5397 5.3 MB/s | 498 kB 00:00 2024-03-11T19:27:02.307 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-11T19:27:02.356 INFO:teuthology.orchestra.run.smithi204.stdout:(19/174): python3-ceph-argparse-19.0.0-1607.g94 492 kB/s | 45 kB 00:00 2024-03-11T19:27:02.456 INFO:teuthology.orchestra.run.smithi204.stdout:(20/174): python3-ceph-common-19.0.0-1607.g94a5 1.3 MB/s | 131 kB 00:00 2024-03-11T19:27:02.491 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-11T19:27:02.541 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-11T19:27:02.565 INFO:teuthology.orchestra.run.smithi204.stdout:(21/174): python3-cephfs-19.0.0-1607.g94a53970. 1.5 MB/s | 161 kB 00:00 2024-03-11T19:27:02.670 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-11T19:27:02.711 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-11T19:27:02.746 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-11T19:27:02.777 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-11T19:27:02.790 INFO:teuthology.orchestra.run.smithi204.stdout:(22/174): librgw2-19.0.0-1607.g94a53970.el9.x86 8.9 MB/s | 5.1 MB 00:00 2024-03-11T19:27:02.804 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-11T19:27:02.856 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-11T19:27:02.857 INFO:teuthology.orchestra.run.smithi204.stdout:(23/174): python3-rbd-19.0.0-1607.g94a53970.el9 4.4 MB/s | 298 kB 00:00 2024-03-11T19:27:02.903 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-11T19:27:02.924 INFO:teuthology.orchestra.run.smithi204.stdout:(24/174): python3-rgw-19.0.0-1607.g94a53970.el9 1.5 MB/s | 99 kB 00:00 2024-03-11T19:27:02.935 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-11T19:27:02.965 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-11T19:27:02.991 INFO:teuthology.orchestra.run.smithi204.stdout:(25/174): rbd-fuse-19.0.0-1607.g94a53970.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-11T19:27:03.024 INFO:teuthology.orchestra.run.smithi204.stdout:(26/174): python3-rados-19.0.0-1607.g94a53970.e 697 kB/s | 320 kB 00:00 2024-03-11T19:27:03.041 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-11T19:27:03.076 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-11T19:27:03.109 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-11T19:27:03.150 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-11T19:27:03.185 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-11T19:27:03.266 INFO:teuthology.orchestra.run.smithi204.stdout:(27/174): rbd-nbd-19.0.0-1607.g94a53970.el9.x86 706 kB/s | 171 kB 00:00 2024-03-11T19:27:03.275 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-11T19:27:03.358 INFO:teuthology.orchestra.run.smithi204.stdout:(28/174): rbd-mirror-19.0.0-1607.g94a53970.el9. 8.4 MB/s | 3.1 MB 00:00 2024-03-11T19:27:03.363 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-11T19:27:03.383 INFO:teuthology.orchestra.run.smithi204.stdout:(29/174): ceph-grafana-dashboards-19.0.0-1607.g 269 kB/s | 31 kB 00:00 2024-03-11T19:27:03.407 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-11T19:27:03.409 INFO:teuthology.orchestra.run.smithi204.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1607.g94a5397 2.7 MB/s | 138 kB 00:00 2024-03-11T19:27:03.439 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-11T19:27:03.487 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-11T19:27:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-11T19:27:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-11T19:27:03.767 INFO:teuthology.orchestra.run.smithi137.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 404 kB/s | 1.1 MB 00:02 2024-03-11T19:27:04.109 INFO:teuthology.orchestra.run.smithi137.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 412 kB/s | 2.3 MB 00:05 2024-03-11T19:27:04.116 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:27:04.116 INFO:teuthology.orchestra.run.smithi137.stdout:Total 8.5 MB/s | 223 MB 00:26 2024-03-11T19:27:04.193 INFO:teuthology.orchestra.run.smithi204.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 9.4 MB/s | 7.4 MB 00:00 2024-03-11T19:27:04.259 INFO:teuthology.orchestra.run.smithi204.stdout:(32/174): ceph-mgr-dashboard-19.0.0-1607.g94a53 2.0 MB/s | 1.8 MB 00:00 2024-03-11T19:27:04.285 INFO:teuthology.orchestra.run.smithi204.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1607.g94 2.6 MB/s | 247 kB 00:00 2024-03-11T19:27:04.318 INFO:teuthology.orchestra.run.smithi204.stdout:(34/174): ceph-mgr-rook-19.0.0-1607.g94a53970.e 841 kB/s | 49 kB 00:00 2024-03-11T19:27:04.343 INFO:teuthology.orchestra.run.smithi204.stdout:(35/174): ceph-prometheus-alerts-19.0.0-1607.g9 263 kB/s | 15 kB 00:00 2024-03-11T19:27:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-11T19:27:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-11T19:27:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-11T19:27:04.452 INFO:teuthology.orchestra.run.smithi204.stdout:(36/174): ceph-volume-19.0.0-1607.g94a53970.el9 1.9 MB/s | 264 kB 00:00 2024-03-11T19:27:04.477 INFO:teuthology.orchestra.run.smithi204.stdout:(37/174): cephadm-19.0.0-1607.g94a53970.el9.noa 5.5 MB/s | 748 kB 00:00 2024-03-11T19:27:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-11T19:27:04.511 INFO:teuthology.orchestra.run.smithi204.stdout:(38/174): cephfs-top-19.0.0-1607.g94a53970.el9. 312 kB/s | 18 kB 00:00 2024-03-11T19:27:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-11T19:27:04.582 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-03-11T19:27:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-11T19:27:04.791 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-11T19:27:04.820 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-11T19:27:04.850 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-03-11T19:27:04.851 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-03-11T19:27:04.898 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-11T19:27:04.970 INFO:teuthology.orchestra.run.smithi204.stdout:(39/174): ceph-test-19.0.0-1607.g94a53970.el9.x 15 MB/s | 47 MB 00:03 2024-03-11T19:27:05.120 INFO:teuthology.orchestra.run.smithi204.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 662 kB 00:00 2024-03-11T19:27:05.145 INFO:teuthology.orchestra.run.smithi204.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 721 kB/s | 106 kB 00:00 2024-03-11T19:27:05.179 INFO:teuthology.orchestra.run.smithi204.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-03-11T19:27:05.204 INFO:teuthology.orchestra.run.smithi204.stdout:(43/174): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 920 kB 00:00 2024-03-11T19:27:05.221 INFO:teuthology.orchestra.run.smithi204.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 422 kB/s | 32 kB 00:00 2024-03-11T19:27:05.305 INFO:teuthology.orchestra.run.smithi204.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 123 kB/s | 10 kB 00:00 2024-03-11T19:27:05.330 INFO:teuthology.orchestra.run.smithi204.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 159 kB/s | 17 kB 00:00 2024-03-11T19:27:05.381 INFO:teuthology.orchestra.run.smithi204.stdout:(47/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-03-11T19:27:05.406 INFO:teuthology.orchestra.run.smithi204.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 159 kB 00:00 2024-03-11T19:27:05.423 INFO:teuthology.orchestra.run.smithi204.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 484 kB/s | 44 kB 00:00 2024-03-11T19:27:05.448 INFO:teuthology.orchestra.run.smithi204.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-11T19:27:05.465 INFO:teuthology.orchestra.run.smithi204.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-11T19:27:05.499 INFO:teuthology.orchestra.run.smithi204.stdout:(52/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-11T19:27:05.524 INFO:teuthology.orchestra.run.smithi204.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-11T19:27:05.549 INFO:teuthology.orchestra.run.smithi204.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-03-11T19:27:05.549 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-11T19:27:05.574 INFO:teuthology.orchestra.run.smithi204.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-11T19:27:05.591 INFO:teuthology.orchestra.run.smithi204.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 828 kB/s | 34 kB 00:00 2024-03-11T19:27:05.653 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-11T19:27:05.742 INFO:teuthology.orchestra.run.smithi204.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-03-11T19:27:05.775 INFO:teuthology.orchestra.run.smithi204.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 7.6 MB/s | 1.5 MB 00:00 2024-03-11T19:27:05.809 INFO:teuthology.orchestra.run.smithi204.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 3.7 MB/s | 811 kB 00:00 2024-03-11T19:27:05.834 INFO:teuthology.orchestra.run.smithi204.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2024-03-11T19:27:05.918 INFO:teuthology.orchestra.run.smithi204.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-03-11T19:27:06.043 INFO:teuthology.orchestra.run.smithi204.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 24 MB/s | 5.7 MB 00:00 2024-03-11T19:27:06.068 INFO:teuthology.orchestra.run.smithi204.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 332 kB/s | 78 kB 00:00 2024-03-11T19:27:06.194 INFO:teuthology.orchestra.run.smithi204.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-03-11T19:27:06.218 INFO:teuthology.orchestra.run.smithi204.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 141 kB/s | 24 kB 00:00 2024-03-11T19:27:06.244 INFO:teuthology.orchestra.run.smithi204.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 105 kB/s | 18 kB 00:00 2024-03-11T19:27:06.261 INFO:teuthology.orchestra.run.smithi204.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-03-11T19:27:06.303 INFO:teuthology.orchestra.run.smithi204.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 542 kB/s | 32 kB 00:00 2024-03-11T19:27:06.497 INFO:teuthology.orchestra.run.smithi204.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2024-03-11T19:27:06.522 INFO:teuthology.orchestra.run.smithi204.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 834 kB/s | 217 kB 00:00 2024-03-11T19:27:06.536 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-03-11T19:27:06.537 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-03-11T19:27:06.547 INFO:teuthology.orchestra.run.smithi204.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 720 kB/s | 174 kB 00:00 2024-03-11T19:27:06.572 INFO:teuthology.orchestra.run.smithi204.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 919 kB/s | 69 kB 00:00 2024-03-11T19:27:06.623 INFO:teuthology.orchestra.run.smithi204.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-11T19:27:06.640 INFO:teuthology.orchestra.run.smithi204.stdout:(74/174): python3-prettytable-0.7.2-27.el9.noar 446 kB/s | 41 kB 00:00 2024-03-11T19:27:06.665 INFO:teuthology.orchestra.run.smithi204.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 132 kB 00:00 2024-03-11T19:27:06.690 INFO:teuthology.orchestra.run.smithi204.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-11T19:27:06.716 INFO:teuthology.orchestra.run.smithi204.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 861 kB/s | 43 kB 00:00 2024-03-11T19:27:06.732 INFO:teuthology.orchestra.run.smithi204.stdout:(78/174): python3-pytz-2021.1-5.el9.noarch.rpm 710 kB/s | 47 kB 00:00 2024-03-11T19:27:06.877 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-11T19:27:06.966 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-11T19:27:06.997 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-11T19:27:07.025 INFO:teuthology.orchestra.run.smithi204.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 125 kB/s | 39 kB 00:00 2024-03-11T19:27:07.028 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-11T19:27:07.041 INFO:teuthology.orchestra.run.smithi204.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 133 kB/s | 41 kB 00:00 2024-03-11T19:27:07.055 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-11T19:27:07.225 INFO:teuthology.orchestra.run.smithi204.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-03-11T19:27:07.259 INFO:teuthology.orchestra.run.smithi204.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 300 kB 00:00 2024-03-11T19:27:07.276 INFO:teuthology.orchestra.run.smithi204.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 243 kB/s | 56 kB 00:00 2024-03-11T19:27:07.293 INFO:teuthology.orchestra.run.smithi204.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 943 kB/s | 63 kB 00:00 2024-03-11T19:27:07.310 INFO:teuthology.orchestra.run.smithi204.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-11T19:27:07.335 INFO:teuthology.orchestra.run.smithi204.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-11T19:27:07.360 INFO:teuthology.orchestra.run.smithi204.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-11T19:27:07.365 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-11T19:27:07.394 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:27:07.394 INFO:teuthology.orchestra.run.smithi204.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-11T19:27:07.476 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:27:07.502 INFO:teuthology.orchestra.run.smithi204.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 620 kB/s | 103 kB 00:00 2024-03-11T19:27:07.507 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-11T19:27:07.544 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-11T19:27:07.578 INFO:teuthology.orchestra.run.smithi204.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 575 kB/s | 124 kB 00:00 2024-03-11T19:27:07.603 INFO:teuthology.orchestra.run.smithi204.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 899 kB/s | 187 kB 00:00 2024-03-11T19:27:08.003 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-11T19:27:08.128 INFO:teuthology.orchestra.run.smithi204.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 340 kB/s | 187 kB 00:00 2024-03-11T19:27:08.153 INFO:teuthology.orchestra.run.smithi204.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 355 kB/s | 195 kB 00:00 2024-03-11T19:27:08.246 INFO:teuthology.orchestra.run.smithi204.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-03-11T19:27:08.255 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:27:08.396 INFO:teuthology.orchestra.run.smithi204.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 983 kB/s | 803 kB 00:00 2024-03-11T19:27:08.454 INFO:teuthology.orchestra.run.smithi204.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 71 kB/s | 21 kB 00:00 2024-03-11T19:27:08.494 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:27:08.513 INFO:teuthology.orchestra.run.smithi204.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 95 kB/s | 25 kB 00:00 2024-03-11T19:27:08.524 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-11T19:27:08.646 INFO:teuthology.orchestra.run.smithi204.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-03-11T19:27:08.772 INFO:teuthology.orchestra.run.smithi204.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.7 MB/s | 216 kB 00:00 2024-03-11T19:27:08.905 INFO:teuthology.orchestra.run.smithi204.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 262 kB/s | 35 kB 00:00 2024-03-11T19:27:08.931 INFO:teuthology.orchestra.run.smithi204.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 594 kB/s | 248 kB 00:00 2024-03-11T19:27:09.031 INFO:teuthology.orchestra.run.smithi204.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 355 kB/s | 44 kB 00:00 2024-03-11T19:27:09.114 INFO:teuthology.orchestra.run.smithi204.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 510 kB/s | 94 kB 00:00 2024-03-11T19:27:09.906 INFO:teuthology.orchestra.run.smithi204.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 25 kB/s | 22 kB 00:00 2024-03-11T19:27:10.365 INFO:teuthology.orchestra.run.smithi204.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-03-11T19:27:10.465 INFO:teuthology.orchestra.run.smithi204.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-03-11T19:27:10.541 INFO:teuthology.orchestra.run.smithi204.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-03-11T19:27:10.618 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:27:10.670 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:27:10.816 INFO:teuthology.orchestra.run.smithi204.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-03-11T19:27:10.883 INFO:teuthology.orchestra.run.smithi204.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-03-11T19:27:10.966 INFO:teuthology.orchestra.run.smithi204.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 584 kB/s | 49 kB 00:00 2024-03-11T19:27:11.042 INFO:teuthology.orchestra.run.smithi204.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 893 kB/s | 67 kB 00:00 2024-03-11T19:27:11.117 INFO:teuthology.orchestra.run.smithi204.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-11T19:27:11.209 INFO:teuthology.orchestra.run.smithi204.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-03-11T19:27:11.227 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:27:11.274 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:27:11.309 INFO:teuthology.orchestra.run.smithi204.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-03-11T19:27:11.376 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:27:11.376 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-11T19:27:11.376 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-11T19:27:11.376 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:11.385 INFO:teuthology.orchestra.run.smithi204.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 578 kB/s | 43 kB 00:00 2024-03-11T19:27:11.416 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:27:11.452 INFO:teuthology.orchestra.run.smithi204.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 483 kB/s | 32 kB 00:00 2024-03-11T19:27:11.519 INFO:teuthology.orchestra.run.smithi204.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 225 kB/s | 15 kB 00:00 2024-03-11T19:27:11.602 INFO:teuthology.orchestra.run.smithi204.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-03-11T19:27:11.678 INFO:teuthology.orchestra.run.smithi204.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-03-11T19:27:11.753 INFO:teuthology.orchestra.run.smithi204.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-03-11T19:27:11.820 INFO:teuthology.orchestra.run.smithi204.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 529 kB/s | 35 kB 00:00 2024-03-11T19:27:11.895 INFO:teuthology.orchestra.run.smithi204.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-11T19:27:12.007 INFO:teuthology.orchestra.run.smithi204.stdout:(123/174): python3-grpcio-1.46.7-10.el9.x86_64. 18 MB/s | 2.0 MB 00:00 2024-03-11T19:27:12.032 INFO:teuthology.orchestra.run.smithi204.stdout:(124/174): protobuf-compiler-3.14.0-13.el9.x86_ 296 kB/s | 863 kB 00:02 2024-03-11T19:27:12.091 INFO:teuthology.orchestra.run.smithi204.stdout:(125/174): openssl-devel-3.0.7-27.el9.x86_64.rp 1.1 MB/s | 4.1 MB 00:03 2024-03-11T19:27:12.142 INFO:teuthology.orchestra.run.smithi204.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-03-11T19:27:12.216 INFO:teuthology.orchestra.run.smithi204.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 310 kB/s | 23 kB 00:00 2024-03-11T19:27:12.275 INFO:teuthology.orchestra.run.smithi204.stdout:(128/174): python3-jaraco-8.2.1-3.el9.noarch.rp 44 kB/s | 11 kB 00:00 2024-03-11T19:27:12.300 INFO:teuthology.orchestra.run.smithi204.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-03-11T19:27:12.367 INFO:teuthology.orchestra.run.smithi204.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-03-11T19:27:12.393 INFO:teuthology.orchestra.run.smithi204.stdout:(131/174): python3-jaraco-classes-3.2.1-5.el9.n 64 kB/s | 18 kB 00:00 2024-03-11T19:27:12.445 INFO:teuthology.orchestra.run.smithi204.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 7.1 MB/s | 1.0 MB 00:00 2024-03-11T19:27:12.487 INFO:teuthology.orchestra.run.smithi204.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 388 kB/s | 46 kB 00:00 2024-03-11T19:27:12.512 INFO:teuthology.orchestra.run.smithi204.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-03-11T19:27:12.563 INFO:teuthology.orchestra.run.smithi204.stdout:(135/174): python3-more-itertools-8.12.0-2.el9. 465 kB/s | 79 kB 00:00 2024-03-11T19:27:12.588 INFO:teuthology.orchestra.run.smithi204.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-03-11T19:27:12.655 INFO:teuthology.orchestra.run.smithi204.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-03-11T19:27:12.680 INFO:teuthology.orchestra.run.smithi204.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 769 kB/s | 90 kB 00:00 2024-03-11T19:27:12.705 INFO:teuthology.orchestra.run.smithi204.stdout:(139/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-03-11T19:27:12.731 INFO:teuthology.orchestra.run.smithi204.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-03-11T19:27:12.781 INFO:teuthology.orchestra.run.smithi204.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-03-11T19:27:12.806 INFO:teuthology.orchestra.run.smithi204.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-03-11T19:27:12.831 INFO:teuthology.orchestra.run.smithi204.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-03-11T19:27:12.857 INFO:teuthology.orchestra.run.smithi204.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-03-11T19:27:12.890 INFO:teuthology.orchestra.run.smithi204.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-11T19:27:12.915 INFO:teuthology.orchestra.run.smithi204.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-11T19:27:12.966 INFO:teuthology.orchestra.run.smithi204.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-03-11T19:27:12.999 INFO:teuthology.orchestra.run.smithi204.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-03-11T19:27:13.025 INFO:teuthology.orchestra.run.smithi204.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-03-11T19:27:13.067 INFO:teuthology.orchestra.run.smithi204.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-03-11T19:27:13.404 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-03-11T19:27:13.408 INFO:teuthology.orchestra.run.smithi204.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.1 MB/s | 321 kB 00:00 2024-03-11T19:27:13.513 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-11T19:27:13.547 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-11T19:27:13.586 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:27:13.596 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:27:13.635 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-11T19:27:13.688 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-11T19:27:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-11T19:27:13.756 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-11T19:27:13.799 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-11T19:27:14.145 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-11T19:27:14.216 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:27:14.225 INFO:teuthology.orchestra.run.smithi204.stdout:(152/174): librados2-19.0.0-1607.g94a53970.el9. 2.8 MB/s | 3.4 MB 00:01 2024-03-11T19:27:14.337 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:27:14.375 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:27:14.401 INFO:teuthology.orchestra.run.smithi204.stdout:(153/174): librbd1-19.0.0-1607.g94a53970.el9.x8 2.3 MB/s | 3.1 MB 00:01 2024-03-11T19:27:14.476 INFO:teuthology.orchestra.run.smithi204.stdout:(154/174): NetworkManager-1.46.0-1.el9.x86_64.r 2.1 MB/s | 2.3 MB 00:01 2024-03-11T19:27:14.483 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:27:14.521 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:27:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-11T19:27:14.601 INFO:teuthology.orchestra.run.smithi204.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 2.0 MB/s | 248 kB 00:00 2024-03-11T19:27:14.718 INFO:teuthology.orchestra.run.smithi204.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 612 kB/s | 71 kB 00:00 2024-03-11T19:27:14.726 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-11T19:27:14.777 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-11T19:27:14.802 INFO:teuthology.orchestra.run.smithi204.stdout:(157/174): NetworkManager-team-1.46.0-1.el9.x86 106 kB/s | 42 kB 00:00 2024-03-11T19:27:14.805 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:27:14.835 INFO:teuthology.orchestra.run.smithi204.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 311 kB/s | 36 kB 00:00 2024-03-11T19:27:14.909 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:27:14.931 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:27:14.952 INFO:teuthology.orchestra.run.smithi204.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-03-11T19:27:14.995 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:27:15.035 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-11T19:27:15.058 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:27:15.073 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:27:15.119 INFO:teuthology.orchestra.run.smithi204.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 606 kB/s | 192 kB 00:00 2024-03-11T19:27:15.220 INFO:teuthology.orchestra.run.smithi204.stdout:(161/174): NetworkManager-libnm-1.46.0-1.el9.x8 1.8 MB/s | 1.8 MB 00:00 2024-03-11T19:27:15.224 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:27:15.245 INFO:teuthology.orchestra.run.smithi204.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 863 kB/s | 108 kB 00:00 2024-03-11T19:27:15.255 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:27:15.270 INFO:teuthology.orchestra.run.smithi204.stdout:(163/174): libacl-2.3.1-4.el9.x86_64.rpm 72 kB/s | 23 kB 00:00 2024-03-11T19:27:15.287 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-11T19:27:15.345 INFO:teuthology.orchestra.run.smithi204.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 155 kB 00:00 2024-03-11T19:27:15.371 INFO:teuthology.orchestra.run.smithi204.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-03-11T19:27:15.383 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-11T19:27:15.471 INFO:teuthology.orchestra.run.smithi204.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 229 kB/s | 29 kB 00:00 2024-03-11T19:27:15.473 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:27:15.505 INFO:teuthology.orchestra.run.smithi204.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 272 kB/s | 64 kB 00:00 2024-03-11T19:27:15.533 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:27:15.602 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:27:15.667 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:27:15.747 INFO:teuthology.orchestra.run.smithi204.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.8 MB/s | 2.2 MB 00:00 2024-03-11T19:27:15.830 INFO:teuthology.orchestra.run.smithi204.stdout:(169/174): openssl-3.0.7-27.el9.x86_64.rpm 2.6 MB/s | 1.2 MB 00:00 2024-03-11T19:27:15.882 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:27:15.889 INFO:teuthology.orchestra.run.smithi204.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.2 MB/s | 465 kB 00:00 2024-03-11T19:27:15.973 INFO:teuthology.orchestra.run.smithi204.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 219 kB/s | 31 kB 00:00 2024-03-11T19:27:15.988 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-11T19:27:16.084 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-11T19:27:16.114 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-11T19:27:16.140 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-11T19:27:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-11T19:27:16.248 INFO:teuthology.orchestra.run.smithi204.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 270 kB/s | 74 kB 00:00 2024-03-11T19:27:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-11T19:27:16.303 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-11T19:27:16.344 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-11T19:27:16.381 INFO:teuthology.orchestra.run.smithi204.stdout:(173/174): util-linux-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 2.3 MB 00:00 2024-03-11T19:27:16.408 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-11T19:27:16.443 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-11T19:27:16.487 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-11T19:27:16.537 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-11T19:27:16.564 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-11T19:27:16.633 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-11T19:27:16.669 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-11T19:27:16.712 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-11T19:27:16.741 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-11T19:27:16.809 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-11T19:27:16.855 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-11T19:27:16.929 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-11T19:27:17.061 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-11T19:27:17.116 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-11T19:27:17.632 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-11T19:27:17.711 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-11T19:27:17.796 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-11T19:27:17.831 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:27:17.838 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:27:17.871 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-11T19:27:17.920 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-11T19:27:17.958 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-11T19:27:17.988 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-11T19:27:18.173 INFO:teuthology.orchestra.run.smithi204.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 481 kB/s | 1.1 MB 00:02 2024-03-11T19:27:18.177 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:27:18.177 INFO:teuthology.orchestra.run.smithi204.stdout:Total 12 MB/s | 223 MB 00:19 2024-03-11T19:27:18.220 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-11T19:27:18.378 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-11T19:27:18.477 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-11T19:27:18.526 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-11T19:27:18.630 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-03-11T19:27:18.664 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-11T19:27:18.704 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-11T19:27:18.758 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-11T19:27:18.831 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-11T19:27:18.871 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-11T19:27:18.905 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-03-11T19:27:18.905 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-03-11T19:27:18.907 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-11T19:27:18.954 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-11T19:27:18.990 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-11T19:27:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-11T19:27:19.075 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-11T19:27:19.207 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-11T19:27:19.248 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-11T19:27:19.278 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-11T19:27:19.319 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-11T19:27:19.414 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-11T19:27:19.453 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-11T19:27:19.492 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-11T19:27:19.518 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-11T19:27:19.552 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-11T19:27:19.623 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-11T19:27:19.744 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-11T19:27:19.860 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-11T19:27:19.901 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-11T19:27:19.950 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-11T19:27:20.005 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-11T19:27:20.190 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-11T19:27:20.256 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-11T19:27:20.383 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-11T19:27:20.420 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-11T19:27:20.452 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-11T19:27:20.483 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-11T19:27:20.511 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-11T19:27:20.545 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-11T19:27:20.584 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-11T19:27:20.612 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-03-11T19:27:20.613 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-03-11T19:27:20.616 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-11T19:27:20.646 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-11T19:27:20.722 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-11T19:27:20.756 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-11T19:27:20.789 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-11T19:27:20.831 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-11T19:27:20.866 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-11T19:27:20.955 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-11T19:27:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-11T19:27:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-11T19:27:21.128 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-11T19:27:21.176 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-11T19:27:21.273 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-11T19:27:21.321 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-11T19:27:22.048 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-11T19:27:22.093 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-11T19:27:22.127 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-11T19:27:22.175 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-11T19:27:22.263 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-11T19:27:22.294 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-11T19:27:22.471 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-11T19:27:22.517 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-11T19:27:22.595 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-11T19:27:23.156 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:27:23.156 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-03-11T19:27:23.156 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-03-11T19:27:23.156 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-03-11T19:27:23.157 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-03-11T19:27:23.157 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-03-11T19:27:23.157 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-03-11T19:27:23.157 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-03-11T19:27:23.157 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:23.233 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-11T19:27:23.321 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:27:23.337 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:27:23.337 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:23.337 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-11T19:27:23.337 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:23.340 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-11T19:27:24.539 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-11T19:27:24.634 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-11T19:27:24.665 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-11T19:27:24.704 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-11T19:27:24.732 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-11T19:27:24.807 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:27:24.833 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:27:24.833 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:24.833 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-11T19:27:24.833 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:27:24.833 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:27:24.833 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:25.041 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-11T19:27:25.042 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:27:25.065 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:27:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-11T19:27:25.065 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:27:25.066 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:27:25.066 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:25.070 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:27:25.161 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:27:25.192 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-11T19:27:25.273 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-11T19:27:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:27:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:27:25.533 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:25.533 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-11T19:27:25.534 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:27:25.534 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:27:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:25.629 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-11T19:27:25.871 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-11T19:27:25.898 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:27:25.952 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:27:25.975 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-11T19:27:26.038 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:27:26.069 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:27:26.101 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-11T19:27:26.161 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-11T19:27:26.184 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:27:26.416 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:27:26.452 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:27:27.412 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-03-11T19:27:27.417 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:27:27.444 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:27:27.527 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-11T19:27:27.545 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:27:27.561 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-11T19:27:27.600 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:27:27.609 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-11T19:27:27.657 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-11T19:27:27.706 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-11T19:27:27.711 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-11T19:27:27.738 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:27:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-11T19:27:27.762 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:27:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-11T19:27:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:27:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:27:27.762 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:27.771 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-11T19:27:27.805 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:27:27.813 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-11T19:27:27.817 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:27:27.870 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-11T19:27:28.086 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:27:28.113 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:27:28.133 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-11T19:27:28.189 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:27:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:27:28.323 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:27:28.398 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:27:28.427 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:27:28.468 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-11T19:27:28.645 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-11T19:27:28.646 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:27:28.682 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:27:28.692 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-11T19:27:28.720 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:27:28.738 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-11T19:27:28.763 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:27:28.763 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-11T19:27:28.763 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-11T19:27:28.763 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:27:28.765 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:27:28.788 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:27:28.789 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:28.789 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-11T19:27:28.789 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:27:28.789 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:27:28.789 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:28.792 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:27:28.807 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:27:28.830 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:27:28.833 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:27:28.855 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:27:28.855 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:28.855 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-11T19:27:28.855 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:28.876 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 17/198 2024-03-11T19:27:28.900 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:27:28.917 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-11T19:27:28.923 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:27:28.923 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:28.923 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-11T19:27:28.923 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:27:28.923 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:27:28.923 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:28.940 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:27:28.955 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-11T19:27:29.106 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:27:29.137 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-11T19:27:29.169 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-11T19:27:29.243 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:27:29.266 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:27:29.266 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:29.266 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-11T19:27:29.266 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:27:29.266 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:27:29.266 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:29.274 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-11T19:27:29.396 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:27:29.457 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-11T19:27:29.529 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:27:29.575 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:27:29.806 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-11T19:27:29.921 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-11T19:27:30.017 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-11T19:27:30.046 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-11T19:27:30.073 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-11T19:27:30.108 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-11T19:27:30.183 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-11T19:27:30.237 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-11T19:27:30.275 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-11T19:27:30.340 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-11T19:27:30.374 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-11T19:27:30.418 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-11T19:27:30.450 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-11T19:27:30.478 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-11T19:27:30.547 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-11T19:27:30.583 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 39/198 2024-03-11T19:27:30.618 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 40/198 2024-03-11T19:27:30.638 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 41/198 2024-03-11T19:27:30.710 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-11T19:27:30.753 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-11T19:27:30.827 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-11T19:27:30.962 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-11T19:27:31.022 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-11T19:27:31.543 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-11T19:27:31.619 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-11T19:27:31.700 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-11T19:27:31.738 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:27:31.745 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-11T19:27:31.778 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-11T19:27:31.818 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-11T19:27:31.857 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 53/198 2024-03-11T19:27:31.886 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-11T19:27:32.118 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-11T19:27:32.283 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-11T19:27:32.364 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-11T19:27:32.408 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-11T19:27:32.536 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-11T19:27:32.623 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-11T19:27:32.684 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-11T19:27:32.757 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-11T19:27:32.787 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-11T19:27:32.823 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-11T19:27:32.870 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-11T19:27:32.906 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-11T19:27:32.949 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-11T19:27:32.983 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-11T19:27:33.118 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-11T19:27:33.172 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-11T19:27:33.202 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-11T19:27:33.247 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-11T19:27:33.341 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-11T19:27:33.372 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-11T19:27:33.412 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-11T19:27:33.438 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-11T19:27:33.463 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-11T19:27:33.526 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-11T19:27:33.673 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-11T19:27:33.698 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:27:33.738 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-11T19:27:33.769 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-11T19:27:33.781 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-11T19:27:33.796 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-11T19:27:33.821 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-11T19:27:33.870 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-11T19:27:33.926 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-11T19:27:33.982 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-11T19:27:34.095 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-11T19:27:34.119 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-11T19:27:34.135 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-11T19:27:34.167 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-11T19:27:34.187 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-11T19:27:34.253 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-11T19:27:34.296 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-11T19:27:34.297 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-11T19:27:34.328 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-11T19:27:34.332 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-11T19:27:34.352 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-11T19:27:34.364 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-11T19:27:34.382 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-11T19:27:34.387 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-11T19:27:34.414 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-11T19:27:34.449 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-11T19:27:34.450 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-11T19:27:34.488 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-11T19:27:34.520 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-11T19:27:34.524 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-11T19:27:34.550 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-11T19:27:34.564 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-11T19:27:34.634 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-11T19:27:34.669 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-11T19:27:34.702 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-11T19:27:34.744 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-11T19:27:34.778 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-11T19:27:34.861 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-11T19:27:34.966 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-11T19:27:35.009 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-11T19:27:35.040 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-11T19:27:35.088 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-11T19:27:35.153 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:27:35.185 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-11T19:27:35.187 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:27:35.208 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:27:35.234 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-11T19:27:35.966 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-11T19:27:36.015 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-11T19:27:36.020 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-11T19:27:36.048 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-11T19:27:36.097 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-11T19:27:36.102 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-11T19:27:36.135 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-11T19:27:36.185 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-11T19:27:36.216 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-11T19:27:36.236 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-11T19:27:36.272 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-11T19:27:36.297 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-11T19:27:36.322 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-11T19:27:36.349 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-11T19:27:36.349 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:27:36.393 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-11T19:27:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-11T19:27:36.428 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:27:36.458 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-11T19:27:36.458 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:27:36.499 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-11T19:27:36.543 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:27:36.569 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-11T19:27:36.602 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-11T19:27:36.628 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-11T19:27:36.653 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-11T19:27:36.695 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-11T19:27:36.733 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-11T19:27:36.774 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-11T19:27:36.799 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-11T19:27:36.825 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-11T19:27:36.850 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:27:36.876 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-11T19:27:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:27:36.882 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:27:37.061 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:27:37.061 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:27:37.141 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-11T19:27:37.153 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:27:37.200 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:27:37.228 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-11T19:27:37.253 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-11T19:27:37.260 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-11T19:27:37.287 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-11T19:27:37.353 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:27:38.484 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-11T19:27:38.565 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-11T19:27:38.596 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-11T19:27:38.627 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-11T19:27:38.655 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-11T19:27:38.962 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-11T19:27:39.001 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:27:39.092 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 124/198 2024-03-11T19:27:39.123 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 125/198 2024-03-11T19:27:39.159 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-11T19:27:39.613 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-11T19:27:39.638 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:27:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 128/198 2024-03-11T19:27:39.731 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 129/198 2024-03-11T19:27:40.431 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:27:40.431 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-03-11T19:27:40.431 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-03-11T19:27:40.431 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-03-11T19:27:40.431 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-03-11T19:27:40.432 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-03-11T19:27:40.432 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-03-11T19:27:40.432 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-03-11T19:27:40.432 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:27:40.588 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:27:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:27:40.606 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:40.606 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-11T19:27:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:27:41.880 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:27:41.919 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:27:42.058 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:27:42.083 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:27:42.083 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:42.083 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-11T19:27:42.083 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:27:42.083 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:27:42.083 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:27:42.292 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:27:42.314 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:27:42.314 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:42.314 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-11T19:27:42.314 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:27:42.315 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:27:42.315 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:27:42.332 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:27:42.333 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-11T19:27:42.334 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-11T19:27:42.335 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-11T19:27:42.335 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-11T19:27:42.335 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-11T19:27:42.335 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-11T19:27:42.335 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-11T19:27:42.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-11T19:27:42.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-11T19:27:42.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-11T19:27:42.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-11T19:27:42.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-11T19:27:42.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-11T19:27:42.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-11T19:27:42.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-11T19:27:42.350 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-11T19:27:42.351 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-11T19:27:42.351 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-11T19:27:42.351 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-11T19:27:42.351 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-11T19:27:42.351 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-11T19:27:42.351 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-11T19:27:42.351 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-11T19:27:42.351 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-11T19:27:42.351 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-11T19:27:42.351 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-11T19:27:42.352 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-11T19:27:42.353 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-11T19:27:42.354 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-11T19:27:42.355 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-11T19:27:42.356 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-11T19:27:42.357 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-11T19:27:42.358 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:27:42.359 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-11T19:27:42.360 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-11T19:27:42.361 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-11T19:27:42.362 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:27:42.363 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-11T19:27:42.363 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-11T19:27:42.363 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-11T19:27:42.454 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 130/198 2024-03-11T19:27:42.489 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:27:42.579 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 131/198 2024-03-11T19:27:42.580 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-11T19:27:42.580 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-11T19:27:42.580 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:27:42.616 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:27:42.751 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:27:42.773 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:27:42.773 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:42.773 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-11T19:27:42.773 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:27:42.773 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:27:42.773 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:27:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-11T19:27:43.270 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:27:43.300 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:27:43.332 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-11T19:27:43.392 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-11T19:27:43.415 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:27:43.650 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:27:43.683 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:27:44.649 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:27:44.683 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:27:44.782 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:27:44.945 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-11T19:27:44.977 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:27:44.999 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:27:45.000 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:45.000 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-11T19:27:45.000 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:27:45.000 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:27:45.000 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:27:45.044 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:27:45.055 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:27:45.099 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-11T19:27:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-11T19:27:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: acl-2.3.1-4.el9.x86_64 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-11T19:27:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-11T19:27:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: jq-1.6-16.el9.x86_64 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-11T19:27:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.11-1.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-11T19:27:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-11T19:27:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-11T19:27:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-11T19:27:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-11T19:27:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-11T19:27:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-11T19:27:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-11T19:27:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-11T19:27:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-11T19:27:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-11T19:27:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-11T19:27:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-11T19:27:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-11T19:27:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-11T19:27:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-11T19:27:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:27:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-3.el9.x86_64 2024-03-11T19:27:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-11T19:27:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-11T19:27:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-11T19:27:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-11T19:27:45.547 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-11T19:27:45.547 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-11T19:27:45.547 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:27:45.547 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-03-11T19:27:45.954 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-11T19:27:45.988 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:27:46.010 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:27:46.010 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:46.010 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-11T19:27:46.010 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:27:46.010 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:27:46.010 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:27:46.021 DEBUG:teuthology.parallel:result is None 2024-03-11T19:27:46.047 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:27:46.069 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:27:46.069 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:46.069 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-11T19:27:46.069 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:27:46.122 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:27:46.144 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:27:46.144 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:46.145 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-11T19:27:46.145 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:27:46.145 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:27:46.145 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:27:46.456 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:27:46.478 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:27:46.479 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:46.479 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-11T19:27:46.479 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:27:46.479 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:27:46.479 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:27:50.902 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:27:50.943 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-11T19:27:50.977 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-11T19:27:51.013 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-11T19:27:51.187 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-11T19:27:51.308 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-11T19:27:51.365 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-11T19:27:51.405 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-11T19:27:51.457 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-11T19:27:51.492 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-11T19:27:51.516 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-11T19:27:51.540 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-11T19:27:51.570 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-11T19:27:51.624 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-11T19:27:51.684 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-11T19:27:51.718 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-11T19:27:52.274 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:27:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:27:52.336 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:27:53.148 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-11T19:27:53.230 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-11T19:27:53.271 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-11T19:27:53.352 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-11T19:27:53.392 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-11T19:27:53.417 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-11T19:27:53.442 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-11T19:27:53.469 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-11T19:27:53.469 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:27:53.556 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:27:53.586 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-11T19:27:53.586 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:27:53.670 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:27:53.696 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-11T19:27:53.730 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-11T19:27:53.756 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-11T19:27:53.780 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-11T19:27:53.813 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-11T19:27:53.861 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-11T19:27:53.902 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-11T19:27:53.927 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-11T19:27:53.952 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-11T19:27:53.977 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:27:54.003 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-11T19:27:54.009 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:27:54.009 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:27:54.189 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:27:54.189 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:27:54.272 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:27:54.275 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 132/198 2024-03-11T19:27:54.275 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-03-11T19:27:54.275 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-03-11T19:27:54.276 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-03-11T19:27:54.276 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-03-11T19:27:54.276 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-03-11T19:27:54.276 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-03-11T19:27:54.276 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-03-11T19:27:54.276 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:27:54.297 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:27:54.322 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-11T19:27:54.356 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-11T19:27:54.381 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-11T19:27:54.439 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:27:54.456 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 133/198 2024-03-11T19:27:54.456 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:54.456 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-11T19:27:54.456 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:27:54.471 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:27:55.934 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:27:55.959 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 134/198 2024-03-11T19:27:55.959 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:55.959 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-11T19:27:55.959 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:27:55.959 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-11T19:27:55.959 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:27:56.176 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:27:56.198 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 135/198 2024-03-11T19:27:56.198 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:56.198 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-11T19:27:56.198 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:27:56.198 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-11T19:27:56.198 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:27:56.643 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:27:56.665 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 136/198 2024-03-11T19:27:56.665 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:56.666 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-11T19:27:56.666 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:27:56.666 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-11T19:27:56.666 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:27:56.754 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-11T19:27:57.123 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:27:57.146 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 138/198 2024-03-11T19:27:57.177 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 139/198 2024-03-11T19:27:57.238 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 140/198 2024-03-11T19:27:57.269 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:27:57.502 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 141/198 2024-03-11T19:27:57.536 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:27:58.500 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 142/198 2024-03-11T19:27:58.528 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:27:58.630 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 143/198 2024-03-11T19:27:58.782 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 144/198 2024-03-11T19:27:58.805 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:27:58.828 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 145/198 2024-03-11T19:27:58.828 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:58.828 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-11T19:27:58.828 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:27:58.828 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-11T19:27:58.828 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:27:58.865 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:27:58.875 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 146/198 2024-03-11T19:27:58.912 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 147/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:27:59.257 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-11T19:27:59.258 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-11T19:27:59.259 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-11T19:27:59.260 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-11T19:27:59.261 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-11T19:27:59.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-11T19:27:59.263 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-11T19:27:59.264 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-11T19:27:59.265 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-11T19:27:59.266 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-11T19:27:59.267 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-11T19:27:59.268 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-11T19:27:59.269 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-11T19:27:59.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-11T19:27:59.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:27:59.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-11T19:27:59.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-11T19:27:59.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-11T19:27:59.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:27:59.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-11T19:27:59.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-11T19:27:59.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-11T19:27:59.766 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-11T19:27:59.800 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:27:59.822 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 149/198 2024-03-11T19:27:59.823 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:59.823 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-11T19:27:59.823 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:27:59.823 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-11T19:27:59.823 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:27:59.867 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:27:59.889 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1607.g94a53 150/198 2024-03-11T19:27:59.889 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:59.889 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-11T19:27:59.889 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:27:59.935 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:27:59.957 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:27:59.957 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:27:59.957 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-11T19:27:59.957 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:27:59.957 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-11T19:27:59.957 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:28:00.261 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:28:00.282 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 152/198 2024-03-11T19:28:00.283 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-11T19:28:00.283 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-11T19:28:00.283 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:28:00.283 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T19:28:00.283 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:28:02.250 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-11T19:28:02.250 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:28:02.250 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-03-11T19:28:02.250 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-11T19:28:02.250 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: acl-2.3.1-4.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.251 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.2-8.el9.noarch 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.7.4-5.el9.x86_64 2024-03-11T19:28:02.252 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:02.253 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.4-9.el9.x86_64 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-11T19:28:02.254 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: jq-1.6-16.el9.x86_64 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-11T19:28:02.255 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-03-11T19:28:02.256 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-13.11-1.el9.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-11T19:28:02.257 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-11T19:28:02.258 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-11T19:28:02.259 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-11T19:28:02.260 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-11T19:28:02.261 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-11T19:28:02.262 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.263 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.263 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:02.263 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-3.el9.x86_64 2024-03-11T19:28:02.263 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-11T19:28:02.263 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-11T19:28:02.263 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-11T19:28:02.263 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-11T19:28:02.263 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-11T19:28:02.263 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-11T19:28:02.264 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-11T19:28:02.264 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-03-11T19:28:02.770 DEBUG:teuthology.parallel:result is None 2024-03-11T19:28:04.690 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:28:04.740 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 154/198 2024-03-11T19:28:04.780 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 155/198 2024-03-11T19:28:04.814 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-11T19:28:04.996 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 157/198 2024-03-11T19:28:05.114 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-11T19:28:05.154 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 159/198 2024-03-11T19:28:05.193 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-11T19:28:05.254 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-11T19:28:05.289 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-11T19:28:05.322 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 163/198 2024-03-11T19:28:05.362 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-11T19:28:05.392 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-11T19:28:05.446 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-11T19:28:05.515 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-11T19:28:05.540 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-11T19:28:06.078 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:28:06.122 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:28:06.141 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-11T19:28:06.977 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 170/198 2024-03-11T19:28:07.051 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-11T19:28:07.092 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-11T19:28:07.183 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-11T19:28:07.230 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-11T19:28:07.254 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-11T19:28:07.280 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-11T19:28:07.306 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-11T19:28:07.306 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:28:07.394 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-11T19:28:07.424 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-11T19:28:07.424 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:28:07.517 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-11T19:28:07.543 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-11T19:28:07.576 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-11T19:28:07.602 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-11T19:28:07.627 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-11T19:28:07.660 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-11T19:28:07.708 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-11T19:28:07.748 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-11T19:28:07.792 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-11T19:28:07.824 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-11T19:28:07.850 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:28:07.874 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-11T19:28:07.880 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:28:07.880 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:28:08.061 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-11T19:28:08.061 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:28:08.135 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-11T19:28:08.160 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:28:08.186 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-11T19:28:08.218 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-11T19:28:08.245 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-11T19:28:08.309 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:28:13.148 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-11T19:28:13.148 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.0.0-1607.g94a53970.el9.x86_64 1/198 2024-03-11T19:28:13.148 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2/198 2024-03-11T19:28:13.148 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 3/198 2024-03-11T19:28:13.148 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 4/198 2024-03-11T19:28:13.148 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1607.g94a53 5/198 2024-03-11T19:28:13.148 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 6/198 2024-03-11T19:28:13.148 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 7/198 2024-03-11T19:28:13.148 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 8/198 2024-03-11T19:28:13.148 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 9/198 2024-03-11T19:28:13.148 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 10/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 11/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 12/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 13/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 14/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_ 15/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 16/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 17/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.0.0-1607.g94a53970.el9.x86_6 18/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86 19/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 20/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1607.g94a53970.el 21/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.0.0-1607.g94a53970.el9. 22/198 2024-03-11T19:28:13.149 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_6 23/198 2024-03-11T19:28:13.150 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 24/198 2024-03-11T19:28:13.150 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 25/198 2024-03-11T19:28:13.150 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 26/198 2024-03-11T19:28:13.150 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 27/198 2024-03-11T19:28:13.150 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 28/198 2024-03-11T19:28:13.150 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 29/198 2024-03-11T19:28:13.166 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1607.g94a53970. 30/198 2024-03-11T19:28:13.166 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noa 31/198 2024-03-11T19:28:13.166 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.n 32/198 2024-03-11T19:28:13.166 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a 33/198 2024-03-11T19:28:13.166 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el 34/198 2024-03-11T19:28:13.166 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 35/198 2024-03-11T19:28:13.166 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.e 36/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 37/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.0.0-1607.g94a53970.el9.noarch 38/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 39/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-11T19:28:13.167 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-11T19:28:13.168 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-11T19:28:13.169 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-11T19:28:13.170 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-11T19:28:13.171 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-11T19:28:13.172 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-11T19:28:13.173 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-11T19:28:13.174 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.0.0-1607.g94a53970.el9.x86_64 151/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 153/198 2024-03-11T19:28:13.175 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-11T19:28:13.176 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-11T19:28:13.177 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-11T19:28:13.178 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-11T19:28:16.070 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-11T19:28:16.070 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:28:16.070 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-03-11T19:28:16.070 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-11T19:28:16.070 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: acl-2.3.1-4.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.071 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-03-11T19:28:16.072 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.0.0-1607.g94a53970.el9.noarch 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-03-11T19:28:16.074 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: jq-1.6-16.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-11T19:28:16.075 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-11T19:28:16.076 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.11-1.el9.x86_64 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-11T19:28:16.077 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-11T19:28:16.078 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-11T19:28:16.079 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-11T19:28:16.080 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-11T19:28:16.081 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-11T19:28:16.082 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.083 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.083 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.0.0-1607.g94a53970.el9.x86_64 2024-03-11T19:28:16.083 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-03-11T19:28:16.083 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-11T19:28:16.083 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-11T19:28:16.083 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-11T19:28:16.083 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-11T19:28:16.083 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-11T19:28:16.083 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-11T19:28:16.084 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:28:16.084 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-03-11T19:28:16.518 DEBUG:teuthology.parallel:result is None 2024-03-11T19:28:16.518 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:28:16.886 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-11T19:28:16.912 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-1607.g94a53970.el9 2024-03-11T19:28:16.913 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-11T19:28:16.913 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-11T19:28:16.915 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:28:17.066 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-11T19:28:17.092 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-1607.g94a53970.el9 2024-03-11T19:28:17.092 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-11T19:28:17.092 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-11T19:28:17.094 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:28:17.239 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-11T19:28:17.265 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-1607.g94a53970.el9 2024-03-11T19:28:17.265 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607.g94a53970.el9 2024-03-11T19:28:17.265 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607.g94a53970 is installed. 2024-03-11T19:28:17.267 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-11T19:28:17.267 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:28:17.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-11T19:28:17.302 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-11T19:28:17.303 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-11T19:28:17.337 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-11T19:28:17.338 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-11T19:28:17.372 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-11T19:28:17.372 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:28:17.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-03-11T19:28:17.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-11T19:28:17.474 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-11T19:28:17.475 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-03-11T19:28:17.506 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-11T19:28:17.575 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-11T19:28:17.575 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-03-11T19:28:17.607 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-11T19:28:17.679 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-11T19:28:17.679 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:28:17.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-11T19:28:17.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-11T19:28:17.773 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-11T19:28:17.773 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-11T19:28:17.804 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-11T19:28:17.876 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-11T19:28:17.876 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-11T19:28:17.907 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-11T19:28:17.978 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-11T19:28:17.978 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:28:17.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-03-11T19:28:18.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-11T19:28:18.084 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-11T19:28:18.084 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-03-11T19:28:18.115 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-11T19:28:18.185 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-11T19:28:18.185 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-03-11T19:28:18.216 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-11T19:28:18.287 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-11T19:28:18.409 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'subvols': {'create': 2}} 2024-03-11T19:28:18.409 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:28:18.410 INFO:tasks.cephadm:Cluster fsid is 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:28:18.410 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-11T19:28:18.410 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.83', 'mon.b': '172.21.15.137', 'mon.c': '172.21.15.204'} 2024-03-11T19:28:18.411 INFO:tasks.cephadm:First mon is mon.a on smithi083 2024-03-11T19:28:18.411 INFO:tasks.cephadm:First mgr is x 2024-03-11T19:28:18.411 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-11T19:28:18.411 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-03-11T19:28:18.439 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-03-11T19:28:18.470 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-03-11T19:28:18.502 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-11T19:28:18.502 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:28:18.653 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77431/', 'root_build_cause': 'SCMTRIGGER,MANUALTRIGGER', 'version': '19.0.0-1607-g94a53970', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1607.g94a53970'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 04:08:19.449750', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77429/', 'root_build_cause': 'SCMTRIGGER,MANUALTRIGGER', 'version': '19.0.0-1607-g94a53970', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1607.g94a53970'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 04:06:59.696700', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-11T19:28:18.800 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 94a539707733325322964c6a187c6a1b41be1daf from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:28:18.802 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm 2024-03-11T19:28:18.802 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-11T19:28:19.439 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 11 19:28 /home/ubuntu/cephtest/cephadm 2024-03-11T19:28:19.439 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-11T19:28:20.099 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 11 19:28 /home/ubuntu/cephtest/cephadm 2024-03-11T19:28:20.099 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-11T19:28:20.962 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 766894 Mar 11 19:28 /home/ubuntu/cephtest/cephadm 2024-03-11T19:28:20.962 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-11T19:28:20.981 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-11T19:28:20.999 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-11T19:28:21.033 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf on all hosts... 2024-03-11T19:28:21.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-11T19:28:21.036 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-11T19:28:21.042 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf pull 2024-03-11T19:28:21.901 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:28:21.970 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:28:22.017 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:28:48.263 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:28:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T19:28:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-11T19:28:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-03-11T19:28:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-11T19:28:48.263 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-03-11T19:28:48.263 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:28:52.235 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-03-11T19:28:52.235 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T19:28:52.236 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-11T19:28:52.236 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-03-11T19:28:52.236 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-11T19:28:52.236 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-03-11T19:28:52.236 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-03-11T19:28:55.291 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-11T19:28:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T19:28:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e", 2024-03-11T19:28:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-03-11T19:28:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b" 2024-03-11T19:28:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-11T19:28:55.291 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-11T19:28:55.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-03-11T19:28:55.351 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-03-11T19:28:55.380 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-03-11T19:28:55.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-03-11T19:28:55.447 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-03-11T19:28:55.474 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-03-11T19:28:55.505 INFO:tasks.cephadm:Writing seed config... 2024-03-11T19:28:55.507 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-11T19:28:55.507 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-11T19:28:55.507 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-11T19:28:55.507 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-11T19:28:55.507 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-11T19:28:55.507 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-11T19:28:55.507 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-11T19:28:55.507 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-11T19:28:55.507 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-11T19:28:55.507 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-11T19:28:55.508 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-11T19:28:55.509 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-11T19:28:55.510 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-11T19:28:55.510 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-11T19:28:55.510 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-11T19:28:55.510 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-11T19:28:55.510 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-11T19:28:55.510 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-11T19:28:55.510 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-11T19:28:55.510 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-11T19:28:55.511 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:28:55.511 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-11T19:28:55.529 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 839f93c8-dfdd-11ee-95c8-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-11T19:28:55.530 DEBUG:teuthology.orchestra.run.smithi083:mon.a> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.a.service 2024-03-11T19:28:55.572 DEBUG:teuthology.orchestra.run.smithi083:mgr.x> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.x.service 2024-03-11T19:28:55.614 INFO:tasks.cephadm:Bootstrapping... 2024-03-11T19:28:55.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf -v bootstrap --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.83 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:28:55.808 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-03-11T19:28:55.809 INFO:teuthology.orchestra.run.smithi083.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf', '-v', 'bootstrap', '--fsid', '839f93c8-dfdd-11ee-95c8-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.83', '--skip-admin-label'] 2024-03-11T19:28:55.833 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-11T19:28:55.833 INFO:teuthology.orchestra.run.smithi083.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-11T19:28:55.834 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying podman|docker is present... 2024-03-11T19:28:55.857 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-11T19:28:55.857 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying lvm2 is present... 2024-03-11T19:28:55.857 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying time synchronization is in place... 2024-03-11T19:28:55.864 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-11T19:28:55.864 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-11T19:28:55.870 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-11T19:28:55.870 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-03-11T19:28:55.876 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-03-11T19:28:55.882 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-03-11T19:28:55.882 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-03-11T19:28:55.882 INFO:teuthology.orchestra.run.smithi083.stdout:Repeating the final host check... 2024-03-11T19:28:55.906 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-11T19:28:55.906 INFO:teuthology.orchestra.run.smithi083.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-11T19:28:55.906 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl is present 2024-03-11T19:28:55.906 INFO:teuthology.orchestra.run.smithi083.stdout:lvcreate is present 2024-03-11T19:28:55.912 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-11T19:28:55.912 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-11T19:28:55.918 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-11T19:28:55.918 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-03-11T19:28:55.924 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-03-11T19:28:55.930 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-03-11T19:28:55.930 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-03-11T19:28:55.930 INFO:teuthology.orchestra.run.smithi083.stdout:Host looks OK 2024-03-11T19:28:55.930 INFO:teuthology.orchestra.run.smithi083.stdout:Cluster fsid: 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:28:55.931 INFO:teuthology.orchestra.run.smithi083.stdout:Acquiring lock 140693026254800 on /run/cephadm/839f93c8-dfdd-11ee-95c8-87774f69a715.lock 2024-03-11T19:28:55.931 INFO:teuthology.orchestra.run.smithi083.stdout:Lock 140693026254800 acquired on /run/cephadm/839f93c8-dfdd-11ee-95c8-87774f69a715.lock 2024-03-11T19:28:55.931 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 3300 ... 2024-03-11T19:28:55.931 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 6789 ... 2024-03-11T19:28:55.931 INFO:teuthology.orchestra.run.smithi083.stdout:Base mon IP(s) is [172.21.15.83:3300, 172.21.15.83:6789], mon addrv is [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-03-11T19:28:55.934 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.83 metric 100 2024-03-11T19:28:55.935 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.83 metric 100 2024-03-11T19:28:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-11T19:28:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-11T19:28:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-03-11T19:28:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-11T19:28:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-11T19:28:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-11T19:28:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-11T19:28:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-03-11T19:28:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-11T19:28:55.941 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-03-11T19:28:55.941 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-03-11T19:28:55.941 INFO:teuthology.orchestra.run.smithi083.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-11T19:28:55.941 INFO:teuthology.orchestra.run.smithi083.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-11T19:28:55.942 INFO:teuthology.orchestra.run.smithi083.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:28:56.389 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e 2024-03-11T19:28:56.389 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf... 2024-03-11T19:28:56.389 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Getting image source signatures 2024-03-11T19:28:56.389 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:0b3c7bee4e66dab4b5376aaf0baf8ba5fc1c3e66c27dc601157782d617bd9dc6 2024-03-11T19:28:56.389 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-11T19:28:56.389 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying config sha256:a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e 2024-03-11T19:28:56.389 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-11T19:28:57.310 INFO:teuthology.orchestra.run.smithi083.stdout:ceph: stdout ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 2024-03-11T19:28:57.311 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph version: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev) 2024-03-11T19:28:57.311 INFO:teuthology.orchestra.run.smithi083.stdout:Extracting ceph user uid/gid from container image... 2024-03-11T19:28:58.208 INFO:teuthology.orchestra.run.smithi083.stdout:stat: stdout 167 167 2024-03-11T19:28:58.208 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial keys... 2024-03-11T19:28:59.215 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQB6W+9lBlcbIRAAagpl4Tt2PmdiU4CURC4B2w== 2024-03-11T19:29:00.004 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQB7W+9lgFqYHRAAj4cXBdahfGtmwiYXa2XceQ== 2024-03-11T19:29:00.818 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQB8W+9ls0KHEhAARershTEyf+YBVxN45NYGDw== 2024-03-11T19:29:00.819 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial monmap... 2024-03-11T19:29:01.742 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-11T19:29:01.742 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-11T19:29:01.742 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:29:01.742 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-11T19:29:01.742 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool for a [v2:172.21.15.83:3300,v1:172.21.15.83:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-11T19:29:01.742 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-03-11T19:29:01.742 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: set fsid to 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:29:01.742 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-11T19:29:01.743 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:29:01.743 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mon... 2024-03-11T19:29:02.675 INFO:teuthology.orchestra.run.smithi083.stdout:create mon.a on 2024-03-11T19:29:02.870 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-11T19:29:03.005 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-11T19:29:03.169 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-839f93c8-dfdd-11ee-95c8-87774f69a715.target → /etc/systemd/system/ceph-839f93c8-dfdd-11ee-95c8-87774f69a715.target. 2024-03-11T19:29:03.169 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-839f93c8-dfdd-11ee-95c8-87774f69a715.target → /etc/systemd/system/ceph-839f93c8-dfdd-11ee-95c8-87774f69a715.target. 2024-03-11T19:29:03.394 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.a 2024-03-11T19:29:03.394 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.a.service: Unit ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.a.service not loaded. 2024-03-11T19:29:03.550 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-839f93c8-dfdd-11ee-95c8-87774f69a715.target.wants/ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.a.service → /etc/systemd/system/ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@.service. 2024-03-11T19:29:04.102 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:29:04.102 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-03-11T19:29:04.108 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:29:04.108 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-03-11T19:29:04.108 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-03-11T19:29:04.108 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-03-11T19:29:04.109 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon to start... 2024-03-11T19:29:04.109 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon... 2024-03-11T19:29:05.175 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout cluster: 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout id: 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout services: 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.554957s) 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout data: 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-11T19:29:05.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pgs: 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.547+0000 7f442e47c700 1 Processor -- start 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.547+0000 7f442e47c700 1 -- start start 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.547+0000 7f442e47c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f44281068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.547+0000 7f442e47c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4428106ec0 con 0x7f44281064d0 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.548+0000 7f4427fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f44281068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.548+0000 7f4427fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f44281068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38268/0 (socket says 172.21.15.83:38268) 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.548+0000 7f4427fff700 1 -- 172.21.15.83:0/184963114 learned_addr learned my addr 172.21.15.83:0/184963114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.548+0000 7f4427fff700 1 -- 172.21.15.83:0/184963114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44281076d0 con 0x7f44281064d0 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.548+0000 7f4427fff700 1 --2- 172.21.15.83:0/184963114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f44281068f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44180098a0 tx=0x7f4418009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc9050ac5b3360e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.548+0000 7f4426ffd700 1 -- 172.21.15.83:0/184963114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4418004160 con 0x7f44281064d0 2024-03-11T19:29:05.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.548+0000 7f4426ffd700 1 -- 172.21.15.83:0/184963114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44180042c0 con 0x7f44281064d0 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.548+0000 7f4426ffd700 1 -- 172.21.15.83:0/184963114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44180045a0 con 0x7f44281064d0 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.549+0000 7f442e47c700 1 -- 172.21.15.83:0/184963114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 msgr2=0x7f44281068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.549+0000 7f442e47c700 1 --2- 172.21.15.83:0/184963114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f44281068f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44180098a0 tx=0x7f4418009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.549+0000 7f442e47c700 1 -- 172.21.15.83:0/184963114 shutdown_connections 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.549+0000 7f442e47c700 1 --2- 172.21.15.83:0/184963114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f44281068f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.549+0000 7f442e47c700 1 -- 172.21.15.83:0/184963114 >> 172.21.15.83:0/184963114 conn(0x7f4428101a70 msgr2=0x7f4428103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.549+0000 7f442e47c700 1 -- 172.21.15.83:0/184963114 shutdown_connections 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.549+0000 7f442e47c700 1 -- 172.21.15.83:0/184963114 wait complete. 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.549+0000 7f442e47c700 1 Processor -- start 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.549+0000 7f442e47c700 1 -- start start 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f442e47c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f4428198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f442e47c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4428198e50 con 0x7f44281064d0 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f4427fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f4428198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:05.178 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f4427fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f4428198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38280/0 (socket says 172.21.15.83:38280) 2024-03-11T19:29:05.179 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f4427fff700 1 -- 172.21.15.83:0/3165448851 learned_addr learned my addr 172.21.15.83:0/3165448851 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:05.179 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f4427fff700 1 -- 172.21.15.83:0/3165448851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4418009550 con 0x7f44281064d0 2024-03-11T19:29:05.179 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f4427fff700 1 --2- 172.21.15.83:0/3165448851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f4428198910 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4418004000 tx=0x7f4418004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:05.179 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f44257fa700 1 -- 172.21.15.83:0/3165448851 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4418011630 con 0x7f44281064d0 2024-03-11T19:29:05.179 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f44257fa700 1 -- 172.21.15.83:0/3165448851 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4418011790 con 0x7f44281064d0 2024-03-11T19:29:05.179 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f442e47c700 1 -- 172.21.15.83:0/3165448851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4428199050 con 0x7f44281064d0 2024-03-11T19:29:05.180 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f44257fa700 1 -- 172.21.15.83:0/3165448851 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4418011a30 con 0x7f44281064d0 2024-03-11T19:29:05.180 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.550+0000 7f442e47c700 1 -- 172.21.15.83:0/3165448851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44281994f0 con 0x7f44281064d0 2024-03-11T19:29:05.180 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.551+0000 7f44257fa700 1 -- 172.21.15.83:0/3165448851 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4418022070 con 0x7f44281064d0 2024-03-11T19:29:05.180 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.551+0000 7f44257fa700 1 -- 172.21.15.83:0/3165448851 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f441801c450 con 0x7f44281064d0 2024-03-11T19:29:05.180 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.551+0000 7f442e47c700 1 -- 172.21.15.83:0/3165448851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44281925a0 con 0x7f44281064d0 2024-03-11T19:29:05.180 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.554+0000 7f44257fa700 1 -- 172.21.15.83:0/3165448851 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4418011b90 con 0x7f44281064d0 2024-03-11T19:29:05.180 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.622+0000 7f442e47c700 1 -- 172.21.15.83:0/3165448851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f4428107cb0 con 0x7f44281064d0 2024-03-11T19:29:05.180 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.622+0000 7f44257fa700 1 -- 172.21.15.83:0/3165448851 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f4418011d30 con 0x7f44281064d0 2024-03-11T19:29:05.180 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.623+0000 7f442e47c700 1 -- 172.21.15.83:0/3165448851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 msgr2=0x7f4428198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:05.180 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.623+0000 7f442e47c700 1 --2- 172.21.15.83:0/3165448851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f4428198910 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4418004000 tx=0x7f4418004750 comp rx=0 tx=0).stop 2024-03-11T19:29:05.180 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.623+0000 7f442e47c700 1 -- 172.21.15.83:0/3165448851 shutdown_connections 2024-03-11T19:29:05.181 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.623+0000 7f442e47c700 1 --2- 172.21.15.83:0/3165448851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44281064d0 0x7f4428198910 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:05.181 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.623+0000 7f442e47c700 1 -- 172.21.15.83:0/3165448851 >> 172.21.15.83:0/3165448851 conn(0x7f4428101a70 msgr2=0x7f442818f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:05.181 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.623+0000 7f442e47c700 1 -- 172.21.15.83:0/3165448851 shutdown_connections 2024-03-11T19:29:05.181 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:04.623+0000 7f442e47c700 1 -- 172.21.15.83:0/3165448851 wait complete. 2024-03-11T19:29:05.181 INFO:teuthology.orchestra.run.smithi083.stdout:mon is available 2024-03-11T19:29:05.181 INFO:teuthology.orchestra.run.smithi083.stdout:Assimilating anything we can from ceph.conf... 2024-03-11T19:29:06.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:06 smithi083 ceph-mon[31480]: from='client.? 172.21.15.83:0/976877869' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-11T19:29:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-03-11T19:29:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:29:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-03-11T19:29:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-11T19:29:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-11T19:29:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.597+0000 7f3b10bed700 1 Processor -- start 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.597+0000 7f3b10bed700 1 -- start start 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.597+0000 7f3b10bed700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c106310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.597+0000 7f3b10bed700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c07ae20 con 0x7f3b0c105ef0 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.597+0000 7f3b0a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c106310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.597+0000 7f3b0a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c106310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38286/0 (socket says 172.21.15.83:38286) 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.597+0000 7f3b0a59c700 1 -- 172.21.15.83:0/598726597 learned_addr learned my addr 172.21.15.83:0/598726597 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.598+0000 7f3b0a59c700 1 -- 172.21.15.83:0/598726597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c07af60 con 0x7f3b0c105ef0 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.598+0000 7f3b0a59c700 1 --2- 172.21.15.83:0/598726597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c106310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0098a0 tx=0x7f3afc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52158c16c11736b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.598+0000 7f3b09d9b700 1 -- 172.21.15.83:0/598726597 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3afc004160 con 0x7f3b0c105ef0 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.598+0000 7f3b09d9b700 1 -- 172.21.15.83:0/598726597 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc0042c0 con 0x7f3b0c105ef0 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.598+0000 7f3b09d9b700 1 -- 172.21.15.83:0/598726597 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3afc0045a0 con 0x7f3b0c105ef0 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.598+0000 7f3b10bed700 1 -- 172.21.15.83:0/598726597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 msgr2=0x7f3b0c106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.598+0000 7f3b10bed700 1 --2- 172.21.15.83:0/598726597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c106310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0098a0 tx=0x7f3afc009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.598+0000 7f3b10bed700 1 -- 172.21.15.83:0/598726597 shutdown_connections 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.598+0000 7f3b10bed700 1 --2- 172.21.15.83:0/598726597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c106310 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.598+0000 7f3b10bed700 1 -- 172.21.15.83:0/598726597 >> 172.21.15.83:0/598726597 conn(0x7f3b0c101b30 msgr2=0x7f3b0c103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.599+0000 7f3b10bed700 1 -- 172.21.15.83:0/598726597 shutdown_connections 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.599+0000 7f3b10bed700 1 -- 172.21.15.83:0/598726597 wait complete. 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.599+0000 7f3b10bed700 1 Processor -- start 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.599+0000 7f3b10bed700 1 -- start start 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b10bed700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b10bed700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c19d1e0 con 0x7f3b0c105ef0 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b0a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b0a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38302/0 (socket says 172.21.15.83:38302) 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b0a59c700 1 -- 172.21.15.83:0/976877869 learned_addr learned my addr 172.21.15.83:0/976877869 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b0a59c700 1 -- 172.21.15.83:0/976877869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc009550 con 0x7f3b0c105ef0 2024-03-11T19:29:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b0a59c700 1 --2- 172.21.15.83:0/976877869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c19cca0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c07a260 tx=0x7f3afc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b03fff700 1 -- 172.21.15.83:0/976877869 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3afc011630 con 0x7f3b0c105ef0 2024-03-11T19:29:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b03fff700 1 -- 172.21.15.83:0/976877869 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc011790 con 0x7f3b0c105ef0 2024-03-11T19:29:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b03fff700 1 -- 172.21.15.83:0/976877869 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3afc011a30 con 0x7f3b0c105ef0 2024-03-11T19:29:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b10bed700 1 -- 172.21.15.83:0/976877869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c19d3e0 con 0x7f3b0c105ef0 2024-03-11T19:29:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.600+0000 7f3b10bed700 1 -- 172.21.15.83:0/976877869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c19d880 con 0x7f3b0c105ef0 2024-03-11T19:29:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.601+0000 7f3b03fff700 1 -- 172.21.15.83:0/976877869 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3afc022070 con 0x7f3b0c105ef0 2024-03-11T19:29:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.601+0000 7f3b03fff700 1 -- 172.21.15.83:0/976877869 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3afc01c450 con 0x7f3b0c105ef0 2024-03-11T19:29:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.601+0000 7f3b10bed700 1 -- 172.21.15.83:0/976877869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c106a30 con 0x7f3b0c105ef0 2024-03-11T19:29:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.604+0000 7f3b03fff700 1 -- 172.21.15.83:0/976877869 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3afc011b90 con 0x7f3b0c105ef0 2024-03-11T19:29:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.669+0000 7f3b10bed700 1 -- 172.21.15.83:0/976877869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f3b0c1a0160 con 0x7f3b0c105ef0 2024-03-11T19:29:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.672+0000 7f3b03fff700 1 -- 172.21.15.83:0/976877869 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f3afc011de0 con 0x7f3b0c105ef0 2024-03-11T19:29:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.673+0000 7f3b03fff700 1 -- 172.21.15.83:0/976877869 <== mon.0 v2:172.21.15.83:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3afc01c6b0 con 0x7f3b0c105ef0 2024-03-11T19:29:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.674+0000 7f3b10bed700 1 -- 172.21.15.83:0/976877869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 msgr2=0x7f3b0c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.674+0000 7f3b10bed700 1 --2- 172.21.15.83:0/976877869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c19cca0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c07a260 tx=0x7f3afc004750 comp rx=0 tx=0).stop 2024-03-11T19:29:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.674+0000 7f3b10bed700 1 -- 172.21.15.83:0/976877869 shutdown_connections 2024-03-11T19:29:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.674+0000 7f3b10bed700 1 --2- 172.21.15.83:0/976877869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b0c105ef0 0x7f3b0c19cca0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.674+0000 7f3b10bed700 1 -- 172.21.15.83:0/976877869 >> 172.21.15.83:0/976877869 conn(0x7f3b0c101b30 msgr2=0x7f3b0c193430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.674+0000 7f3b10bed700 1 -- 172.21.15.83:0/976877869 shutdown_connections 2024-03-11T19:29:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:05.674+0000 7f3b10bed700 1 -- 172.21.15.83:0/976877869 wait complete. 2024-03-11T19:29:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:Generating new minimal ceph.conf... 2024-03-11T19:29:14.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:14 smithi083 ceph-mon[31480]: from='client.? 172.21.15.83:0/2444509988' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:29:18.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.091+0000 7f0e6322b700 1 Processor -- start 2024-03-11T19:29:18.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.091+0000 7f0e6322b700 1 -- start start 2024-03-11T19:29:18.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.091+0000 7f0e6322b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:18.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.091+0000 7f0e6322b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c108050 con 0x7f0e5c107660 2024-03-11T19:29:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.091+0000 7f0e60fc7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.091+0000 7f0e60fc7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55798/0 (socket says 172.21.15.83:55798) 2024-03-11T19:29:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.091+0000 7f0e60fc7700 1 -- 172.21.15.83:0/3693481474 learned_addr learned my addr 172.21.15.83:0/3693481474 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.091+0000 7f0e60fc7700 1 -- 172.21.15.83:0/3693481474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c108810 con 0x7f0e5c107660 2024-03-11T19:29:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.091+0000 7f0e60fc7700 1 --2- 172.21.15.83:0/3693481474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c107a80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e500098a0 tx=0x7f0e50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b22fae4be821fdf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.091+0000 7f0e5b7fe700 1 -- 172.21.15.83:0/3693481474 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e50004160 con 0x7f0e5c107660 2024-03-11T19:29:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.092+0000 7f0e5b7fe700 1 -- 172.21.15.83:0/3693481474 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e500042c0 con 0x7f0e5c107660 2024-03-11T19:29:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.092+0000 7f0e5b7fe700 1 -- 172.21.15.83:0/3693481474 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e50011ac0 con 0x7f0e5c107660 2024-03-11T19:29:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.092+0000 7f0e6322b700 1 -- 172.21.15.83:0/3693481474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 msgr2=0x7f0e5c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.092+0000 7f0e6322b700 1 --2- 172.21.15.83:0/3693481474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c107a80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e500098a0 tx=0x7f0e50009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.092+0000 7f0e6322b700 1 -- 172.21.15.83:0/3693481474 shutdown_connections 2024-03-11T19:29:18.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.092+0000 7f0e6322b700 1 --2- 172.21.15.83:0/3693481474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c107a80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:18.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.092+0000 7f0e6322b700 1 -- 172.21.15.83:0/3693481474 >> 172.21.15.83:0/3693481474 conn(0x7f0e5c102cc0 msgr2=0x7f0e5c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:18.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.092+0000 7f0e6322b700 1 -- 172.21.15.83:0/3693481474 shutdown_connections 2024-03-11T19:29:18.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.092+0000 7f0e6322b700 1 -- 172.21.15.83:0/3693481474 wait complete. 2024-03-11T19:29:18.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.093+0000 7f0e6322b700 1 Processor -- start 2024-03-11T19:29:18.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.093+0000 7f0e6322b700 1 -- start start 2024-03-11T19:29:18.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.093+0000 7f0e6322b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:18.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.093+0000 7f0e6322b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5c19b8f0 con 0x7f0e5c107660 2024-03-11T19:29:18.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.093+0000 7f0e60fc7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:18.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.093+0000 7f0e60fc7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55812/0 (socket says 172.21.15.83:55812) 2024-03-11T19:29:18.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.093+0000 7f0e60fc7700 1 -- 172.21.15.83:0/2444509988 learned_addr learned my addr 172.21.15.83:0/2444509988 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:18.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.093+0000 7f0e60fc7700 1 -- 172.21.15.83:0/2444509988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e50009550 con 0x7f0e5c107660 2024-03-11T19:29:18.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.093+0000 7f0e60fc7700 1 --2- 172.21.15.83:0/2444509988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c19b3b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009fb0 tx=0x7f0e50011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:18.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.093+0000 7f0e59ffb700 1 -- 172.21.15.83:0/2444509988 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e500038b0 con 0x7f0e5c107660 2024-03-11T19:29:18.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.094+0000 7f0e59ffb700 1 -- 172.21.15.83:0/2444509988 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e50003a10 con 0x7f0e5c107660 2024-03-11T19:29:18.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.094+0000 7f0e6322b700 1 -- 172.21.15.83:0/2444509988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c19baf0 con 0x7f0e5c107660 2024-03-11T19:29:18.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.094+0000 7f0e59ffb700 1 -- 172.21.15.83:0/2444509988 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e500195e0 con 0x7f0e5c107660 2024-03-11T19:29:18.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.094+0000 7f0e6322b700 1 -- 172.21.15.83:0/2444509988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5c19bf90 con 0x7f0e5c107660 2024-03-11T19:29:18.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.094+0000 7f0e59ffb700 1 -- 172.21.15.83:0/2444509988 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0e50019be0 con 0x7f0e5c107660 2024-03-11T19:29:18.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.094+0000 7f0e59ffb700 1 -- 172.21.15.83:0/2444509988 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0e50024990 con 0x7f0e5c107660 2024-03-11T19:29:18.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.095+0000 7f0e6322b700 1 -- 172.21.15.83:0/2444509988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5c0632a0 con 0x7f0e5c107660 2024-03-11T19:29:18.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.097+0000 7f0e59ffb700 1 -- 172.21.15.83:0/2444509988 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0e50003b80 con 0x7f0e5c107660 2024-03-11T19:29:18.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.165+0000 7f0e6322b700 1 -- 172.21.15.83:0/2444509988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f0e5c19eb00 con 0x7f0e5c107660 2024-03-11T19:29:18.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.165+0000 7f0e59ffb700 1 -- 172.21.15.83:0/2444509988 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f0e5001aa10 con 0x7f0e5c107660 2024-03-11T19:29:18.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.166+0000 7f0e6322b700 1 -- 172.21.15.83:0/2444509988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 msgr2=0x7f0e5c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:18.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.166+0000 7f0e6322b700 1 --2- 172.21.15.83:0/2444509988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c19b3b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009fb0 tx=0x7f0e50011b20 comp rx=0 tx=0).stop 2024-03-11T19:29:18.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.166+0000 7f0e6322b700 1 -- 172.21.15.83:0/2444509988 shutdown_connections 2024-03-11T19:29:18.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.166+0000 7f0e6322b700 1 --2- 172.21.15.83:0/2444509988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e5c107660 0x7f0e5c19b3b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:18.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.166+0000 7f0e6322b700 1 -- 172.21.15.83:0/2444509988 >> 172.21.15.83:0/2444509988 conn(0x7f0e5c102cc0 msgr2=0x7f0e5c103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:18.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.166+0000 7f0e6322b700 1 -- 172.21.15.83:0/2444509988 shutdown_connections 2024-03-11T19:29:18.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:14.166+0000 7f0e6322b700 1 -- 172.21.15.83:0/2444509988 wait complete. 2024-03-11T19:29:18.588 INFO:teuthology.orchestra.run.smithi083.stdout:Restarting the monitor... 2024-03-11T19:29:18.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:18 smithi083 systemd[1]: Stopping Ceph mon.a for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:29:19.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:18 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a[31476]: 2024-03-11T19:29:18.859+0000 7f4542af0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:29:19.280 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:18 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a[31476]: 2024-03-11T19:29:18.859+0000 7f4542af0700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-11T19:29:19.280 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:19 smithi083 podman[31735]: 2024-03-11 19:29:19.00245633 +0000 UTC m=+0.249718335 container died 07c7b5e4a89931fe36fb9caa3e06cac323c012fa3957cdccedb8b76f4a97a821 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:29:19.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:19 smithi083 podman[31735]: 2024-03-11 19:29:19.36793677 +0000 UTC m=+0.615198770 container cleanup 07c7b5e4a89931fe36fb9caa3e06cac323c012fa3957cdccedb8b76f4a97a821 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-11T19:29:19.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:19 smithi083 bash[31735]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a 2024-03-11T19:29:19.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:19 smithi083 podman[31749]: 2024-03-11 19:29:19.618996235 +0000 UTC m=+0.615445531 container remove 07c7b5e4a89931fe36fb9caa3e06cac323c012fa3957cdccedb8b76f4a97a821 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-11T19:29:19.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:19 smithi083 systemd[1]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-11T19:29:19.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:19 smithi083 systemd[1]: Stopped Ceph mon.a for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:29:19.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:19 smithi083 systemd[1]: Starting Ceph mon.a for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:29:20.191 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 podman[31846]: 2024-03-11 19:29:19.907398714 +0000 UTC m=+0.020235376 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T19:29:20.192 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 podman[31846]: 2024-03-11 19:29:20.011533576 +0000 UTC m=+0.124370218 container create 5c07b05df3a9d4523dad626fbf69d189090e5ed76926a7f06b278a3894a7ddf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-11T19:29:20.192 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 podman[31846]: 2024-03-11 19:29:20.188797865 +0000 UTC m=+0.301634520 container init 5c07b05df3a9d4523dad626fbf69d189090e5ed76926a7f06b278a3894a7ddf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS) 2024-03-11T19:29:20.287 INFO:teuthology.orchestra.run.smithi083.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-11T19:29:20.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 podman[31846]: 2024-03-11 19:29:20.191604156 +0000 UTC m=+0.304440791 container start 5c07b05df3a9d4523dad626fbf69d189090e5ed76926a7f06b278a3894a7ddf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-03-11T19:29:20.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: set uid:gid to 167:167 (ceph:ceph) 2024-03-11T19:29:20.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev), process ceph-mon, pid 2 2024-03-11T19:29:20.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: pidfile_write: ignore empty --pid-file 2024-03-11T19:29:20.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: load: jerasure load: lrc 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: RocksDB version: 7.9.2 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Git sha 0 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Compile date 2024-03-08 02:48:09 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: DB SUMMARY 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: DB Session ID: LN7ETJYCXAXGZT96XWIS 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: CURRENT file: CURRENT 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: IDENTITY file: IDENTITY 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100090 ; 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.error_if_exists: 0 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.create_if_missing: 0 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.paranoid_checks: 1 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-11T19:29:20.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.env: 0x5623d338eae0 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.fs: PosixFileSystem 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.info_log: 0x5623d4dd18c0 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_file_opening_threads: 16 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.statistics: (nil) 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.use_fsync: 0 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_log_file_size: 0 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.keep_log_file_num: 1000 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.recycle_log_file_num: 0 2024-03-11T19:29:20.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.allow_fallocate: 1 2024-03-11T19:29:20.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.allow_mmap_reads: 0 2024-03-11T19:29:20.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.allow_mmap_writes: 0 2024-03-11T19:29:20.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.use_direct_reads: 0 2024-03-11T19:29:20.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-11T19:29:20.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.create_missing_column_families: 0 2024-03-11T19:29:20.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.db_log_dir: 2024-03-11T19:29:20.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.wal_dir: 2024-03-11T19:29:20.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-11T19:29:20.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-11T19:29:20.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-11T19:29:20.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.advise_random_on_open: 1 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.db_write_buffer_size: 0 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.write_buffer_manager: 0x5623d4da5400 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.rate_limiter: (nil) 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.wal_recovery_mode: 2 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.enable_thread_tracking: 0 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.enable_pipelined_write: 0 2024-03-11T19:29:20.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.unordered_write: 0 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.row_cache: None 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.wal_filter: None 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.allow_ingest_behind: 0 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.two_write_queues: 0 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.manual_wal_flush: 0 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.wal_compression: 0 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.atomic_flush: 0 2024-03-11T19:29:20.530 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.log_readahead_size: 0 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.best_efforts_recovery: 0 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.allow_data_in_errors: 0 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.db_host_id: __hostname__ 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_background_jobs: 2 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_background_compactions: -1 2024-03-11T19:29:20.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_subcompactions: 1 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_total_wal_size: 0 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_open_files: -1 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bytes_per_sync: 0 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-11T19:29:20.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_readahead_size: 0 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_background_flushes: -1 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Compression algorithms supported: 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: kZSTD supported: 0 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: kXpressCompression supported: 0 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: kLZ4HCCompression supported: 1 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: kZlibCompression supported: 1 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: kSnappyCompression supported: 1 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: kLZ4Compression supported: 1 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: kBZip2Compression supported: 0 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-11T19:29:20.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.merge_operator: 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_filter: None 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_filter_factory: None 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.sst_partitioner_factory: None 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5623d4dd19c0) 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks: 1 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_top_level_index_and_filter: 1 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_type: 0 2024-03-11T19:29:20.534 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_index_type: 0 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_shortening: 1 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: checksum: 4 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: no_block_cache: 0 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache: 0x5623d4dd5090 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_name: BinnedLRUCache 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_options: 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: capacity : 536870912 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_shard_bits : 4 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: strict_capacity_limit : 0 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: high_pri_pool_ratio: 0.000 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_compressed: (nil) 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: persistent_cache: (nil) 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size: 4096 2024-03-11T19:29:20.535 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size_deviation: 10 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_restart_interval: 16 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_block_restart_interval: 1 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: metadata_block_size: 4096 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: partition_filters: 0 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: use_delta_encoding: 1 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: filter_policy: bloomfilter 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: whole_key_filtering: 1 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: verify_compression: 0 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: read_amp_bytes_per_bit: 0 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: format_version: 5 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_index_compression: 1 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_align: 0 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: max_auto_readahead_size: 262144 2024-03-11T19:29:20.536 INFO:journalctl@ceph.mon.a.smithi083.stdout: prepopulate_block_cache: 0 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout: initial_auto_readahead_size: 8192 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_file_reads_for_auto_readahead: 2 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.write_buffer_size: 33554432 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_write_buffer_number: 2 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compression: NoCompression 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bottommost_compression: Disabled 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.prefix_extractor: nullptr 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.num_levels: 7 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-11T19:29:20.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compression_opts.level: 32767 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compression_opts.strategy: 0 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-11T19:29:20.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compression_opts.enabled: false 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.target_file_size_base: 67108864 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-11T19:29:20.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.arena_block_size: 1048576 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.disable_auto_compactions: 0 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-11T19:29:20.540 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.table_properties_collectors: 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.inplace_update_support: 0 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.bloom_locality: 0 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.max_successive_merges: 0 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.paranoid_file_checks: 0 2024-03-11T19:29:20.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.force_consistency_checks: 1 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.report_bg_io_stats: 0 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.ttl: 2592000 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.enable_blob_files: false 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.min_blob_size: 0 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.blob_file_size: 268435456 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-11T19:29:20.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.blob_file_starting_level: 0 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7a6fce6a-6ead-44fa-82c9-936e554f7db4 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710185360220108, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710185360221098, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95265, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93263, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13878, "raw_average_key_size": 50, "raw_value_size": 85683, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710185360, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7a6fce6a-6ead-44fa-82c9-936e554f7db4", "db_session_id": "LN7ETJYCXAXGZT96XWIS", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710185360221157, "job": 1, "event": "recovery_finished"} 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5623d4ec6000 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: DB pointer 0x5623d4eb0000 2024-03-11T19:29:20.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** DB Stats ** 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-11T19:29:20.544 INFO:journalctl@ceph.mon.a.smithi083.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: L0 2/0 94.87 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: Sum 2/0 94.87 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 140.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-11T19:29:20.545 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative compaction: 0.00 GB write, 31.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval compaction: 0.00 GB write, 31.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache BinnedLRUCache@0x5623d4dd5090#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: starting mon.a rank 0 at public addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] at bind addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: mon.a@-1(???) e1 preinit fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:29:20.546 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: mon.a@-1(???).mds e1 new map 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: mon.a@-1(???).mds e1 print_map 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: e1 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: legacy client fscid: -1 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout: No filesystems configured 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 bash[31846]: 5c07b05df3a9d4523dad626fbf69d189090e5ed76926a7f06b278a3894a7ddf6 2024-03-11T19:29:20.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 systemd[1]: Started Ceph mon.a for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:29:20.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-11T19:29:20.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: monmap e1: 1 mons at {a=[v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:29:20.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: fsmap 2024-03-11T19:29:20.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: osdmap e1: 0 total, 0 up, 0 in 2024-03-11T19:29:20.548 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:20 smithi083 ceph-mon[31864]: mgrmap e1: no daemons active 2024-03-11T19:29:21.335 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.734+0000 7fc04b952700 1 Processor -- start 2024-03-11T19:29:21.339 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.734+0000 7fc04b952700 1 -- start start 2024-03-11T19:29:21.339 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.734+0000 7fc04b952700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc044107ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.734+0000 7fc04b952700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc044108490 con 0x7fc044107aa0 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.735+0000 7fc0496ee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc044107ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.735+0000 7fc0496ee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc044107ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35542/0 (socket says 172.21.15.83:35542) 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.735+0000 7fc0496ee700 1 -- 172.21.15.83:0/50868420 learned_addr learned my addr 172.21.15.83:0/50868420 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.735+0000 7fc0496ee700 1 -- 172.21.15.83:0/50868420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0441066b0 con 0x7fc044107aa0 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.735+0000 7fc0496ee700 1 --2- 172.21.15.83:0/50868420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc044107ec0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc0400098a0 tx=0x7fc040009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2587a01bc023043e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.735+0000 7fc03bfff700 1 -- 172.21.15.83:0/50868420 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc040004160 con 0x7fc044107aa0 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.735+0000 7fc03bfff700 1 -- 172.21.15.83:0/50868420 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0400042c0 con 0x7fc044107aa0 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.735+0000 7fc03bfff700 1 -- 172.21.15.83:0/50868420 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc040011ac0 con 0x7fc044107aa0 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.736+0000 7fc04b952700 1 -- 172.21.15.83:0/50868420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 msgr2=0x7fc044107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.736+0000 7fc04b952700 1 --2- 172.21.15.83:0/50868420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc044107ec0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc0400098a0 tx=0x7fc040009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.736+0000 7fc04b952700 1 -- 172.21.15.83:0/50868420 shutdown_connections 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.736+0000 7fc04b952700 1 --2- 172.21.15.83:0/50868420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc044107ec0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:21.340 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.736+0000 7fc04b952700 1 -- 172.21.15.83:0/50868420 >> 172.21.15.83:0/50868420 conn(0x7fc044101200 msgr2=0x7fc044103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.736+0000 7fc04b952700 1 -- 172.21.15.83:0/50868420 shutdown_connections 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.736+0000 7fc04b952700 1 -- 172.21.15.83:0/50868420 wait complete. 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.736+0000 7fc04b952700 1 Processor -- start 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.736+0000 7fc04b952700 1 -- start start 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc04b952700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc04419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc04b952700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04419b910 con 0x7fc044107aa0 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc0496ee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc04419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc0496ee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc04419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35550/0 (socket says 172.21.15.83:35550) 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc0496ee700 1 -- 172.21.15.83:0/130530679 learned_addr learned my addr 172.21.15.83:0/130530679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc0496ee700 1 -- 172.21.15.83:0/130530679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc040009550 con 0x7fc044107aa0 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc0496ee700 1 --2- 172.21.15.83:0/130530679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc04419b3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc040000c00 tx=0x7fc040011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc03a7fc700 1 -- 172.21.15.83:0/130530679 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc040003aa0 con 0x7fc044107aa0 2024-03-11T19:29:21.341 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc03a7fc700 1 -- 172.21.15.83:0/130530679 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc040003c00 con 0x7fc044107aa0 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc03a7fc700 1 -- 172.21.15.83:0/130530679 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0400248a0 con 0x7fc044107aa0 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc04b952700 1 -- 172.21.15.83:0/130530679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04419bb10 con 0x7fc044107aa0 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.737+0000 7fc04b952700 1 -- 172.21.15.83:0/130530679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04419bfb0 con 0x7fc044107aa0 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.738+0000 7fc03a7fc700 1 -- 172.21.15.83:0/130530679 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc04002d3f0 con 0x7fc044107aa0 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.738+0000 7fc03a7fc700 1 -- 172.21.15.83:0/130530679 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc040019770 con 0x7fc044107aa0 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.738+0000 7fc04b952700 1 -- 172.21.15.83:0/130530679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc044195560 con 0x7fc044107aa0 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.740+0000 7fc03a7fc700 1 -- 172.21.15.83:0/130530679 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc04001ac60 con 0x7fc044107aa0 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.807+0000 7fc04b952700 1 -- 172.21.15.83:0/130530679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fc04410ab60 con 0x7fc044107aa0 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.808+0000 7fc03a7fc700 1 -- 172.21.15.83:0/130530679 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc0400350c0 con 0x7fc044107aa0 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.809+0000 7fc04b952700 1 -- 172.21.15.83:0/130530679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 msgr2=0x7fc04419b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.809+0000 7fc04b952700 1 --2- 172.21.15.83:0/130530679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc04419b3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc040000c00 tx=0x7fc040011b40 comp rx=0 tx=0).stop 2024-03-11T19:29:21.342 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.809+0000 7fc04b952700 1 -- 172.21.15.83:0/130530679 shutdown_connections 2024-03-11T19:29:21.343 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.809+0000 7fc04b952700 1 --2- 172.21.15.83:0/130530679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc044107aa0 0x7fc04419b3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:21.343 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.809+0000 7fc04b952700 1 -- 172.21.15.83:0/130530679 >> 172.21.15.83:0/130530679 conn(0x7fc044101200 msgr2=0x7fc0441020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:21.343 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.809+0000 7fc04b952700 1 -- 172.21.15.83:0/130530679 shutdown_connections 2024-03-11T19:29:21.343 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:20.809+0000 7fc04b952700 1 -- 172.21.15.83:0/130530679 wait complete. 2024-03-11T19:29:21.343 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-11T19:29:21.365 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:29:21.365 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mgr... 2024-03-11T19:29:21.365 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-11T19:29:21.366 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-11T19:29:21.614 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.x 2024-03-11T19:29:21.614 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.x.service: Unit ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.x.service not loaded. 2024-03-11T19:29:21.756 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-839f93c8-dfdd-11ee-95c8-87774f69a715.target.wants/ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@.service. 2024-03-11T19:29:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:21 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/130530679' entity='client.admin' 2024-03-11T19:29:22.068 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:21 smithi083 podman[32083]: 2024-03-11 19:29:21.989116263 +0000 UTC m=+0.120388041 container create 23a489d69253aef9bddcef970c0e4b27e94918711d45826b7c4dc2aecedcbba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-11T19:29:22.258 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:29:22.258 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-03-11T19:29:22.265 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:29:22.266 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-03-11T19:29:22.266 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-03-11T19:29:22.266 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-03-11T19:29:22.274 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:29:22.274 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-03-11T19:29:22.281 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:29:22.281 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-03-11T19:29:22.281 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-03-11T19:29:22.281 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-11T19:29:22.281 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr to start... 2024-03-11T19:29:22.281 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr... 2024-03-11T19:29:22.388 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:22 smithi083 podman[32083]: 2024-03-11 19:29:22.166518753 +0000 UTC m=+0.297790531 container init 23a489d69253aef9bddcef970c0e4b27e94918711d45826b7c4dc2aecedcbba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-11T19:29:22.388 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:22 smithi083 podman[32083]: 2024-03-11 19:29:22.168888941 +0000 UTC m=+0.300160720 container start 23a489d69253aef9bddcef970c0e4b27e94918711d45826b7c4dc2aecedcbba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-11T19:29:22.388 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:22 smithi083 bash[32083]: 23a489d69253aef9bddcef970c0e4b27e94918711d45826b7c4dc2aecedcbba9 2024-03-11T19:29:22.388 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:22 smithi083 systemd[1]: Started Ceph mgr.x for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:29:23.050 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:22 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1248613543' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:29:23.051 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:22 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:22.729+0000 7f11e4adf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:29:23.051 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:22 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:22.829+0000 7f11e4adf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:29:23.364 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:23.364 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:29:23.364 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "839f93c8-dfdd-11ee-95c8-87774f69a715", 2024-03-11T19:29:23.364 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:29:23.364 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:23.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:29:23.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:29:23.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:23.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:29:23.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:23.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:29:23.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:29:23.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:29:23.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:29:23.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:23.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:29:23.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:23.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:29:23.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:29:23.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:23.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:29:23.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-11T19:29:23.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:29:04.070734+0000", 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:29:23.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.668+0000 7f51386fc700 1 Processor -- start 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.668+0000 7f51386fc700 1 -- start start 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.668+0000 7f51386fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.668+0000 7f51386fc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130106ef0 con 0x7f5130106500 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.669+0000 7f5136498700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.669+0000 7f5136498700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35576/0 (socket says 172.21.15.83:35576) 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.669+0000 7f5136498700 1 -- 172.21.15.83:0/223358498 learned_addr learned my addr 172.21.15.83:0/223358498 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.669+0000 7f5136498700 1 -- 172.21.15.83:0/223358498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130107700 con 0x7f5130106500 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.669+0000 7f5136498700 1 --2- 172.21.15.83:0/223358498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130106920 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f512c0098a0 tx=0x7f512c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ce1e9f1541f2ea server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.669+0000 7f5135496700 1 -- 172.21.15.83:0/223358498 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f512c004160 con 0x7f5130106500 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.669+0000 7f5135496700 1 -- 172.21.15.83:0/223358498 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f512c0042c0 con 0x7f5130106500 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.669+0000 7f5135496700 1 -- 172.21.15.83:0/223358498 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f512c011ac0 con 0x7f5130106500 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.670+0000 7f51386fc700 1 -- 172.21.15.83:0/223358498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 msgr2=0x7f5130106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:23.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.670+0000 7f51386fc700 1 --2- 172.21.15.83:0/223358498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130106920 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f512c0098a0 tx=0x7f512c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.670+0000 7f51386fc700 1 -- 172.21.15.83:0/223358498 shutdown_connections 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.670+0000 7f51386fc700 1 --2- 172.21.15.83:0/223358498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130106920 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.670+0000 7f51386fc700 1 -- 172.21.15.83:0/223358498 >> 172.21.15.83:0/223358498 conn(0x7f5130101ac0 msgr2=0x7f5130103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.670+0000 7f51386fc700 1 -- 172.21.15.83:0/223358498 shutdown_connections 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.670+0000 7f51386fc700 1 -- 172.21.15.83:0/223358498 wait complete. 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.671+0000 7f51386fc700 1 Processor -- start 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.671+0000 7f51386fc700 1 -- start start 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.671+0000 7f51386fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.671+0000 7f51386fc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130199790 con 0x7f5130106500 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.671+0000 7f5136498700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.671+0000 7f5136498700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35588/0 (socket says 172.21.15.83:35588) 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.671+0000 7f5136498700 1 -- 172.21.15.83:0/1248613543 learned_addr learned my addr 172.21.15.83:0/1248613543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:23.378 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.671+0000 7f5136498700 1 -- 172.21.15.83:0/1248613543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c009550 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.672+0000 7f5136498700 1 --2- 172.21.15.83:0/1248613543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130199250 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f512c009fb0 tx=0x7f512c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.672+0000 7f51277fe700 1 -- 172.21.15.83:0/1248613543 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f512c0038b0 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.672+0000 7f51277fe700 1 -- 172.21.15.83:0/1248613543 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f512c003a10 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.672+0000 7f51386fc700 1 -- 172.21.15.83:0/1248613543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5130199990 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.672+0000 7f51277fe700 1 -- 172.21.15.83:0/1248613543 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f512c0195e0 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.672+0000 7f51386fc700 1 -- 172.21.15.83:0/1248613543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5130199e30 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.672+0000 7f51277fe700 1 -- 172.21.15.83:0/1248613543 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f512c019c00 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.672+0000 7f51277fe700 1 -- 172.21.15.83:0/1248613543 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f512c024990 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.673+0000 7f51386fc700 1 -- 172.21.15.83:0/1248613543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51300632a0 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.675+0000 7f51277fe700 1 -- 172.21.15.83:0/1248613543 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f512c003b80 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.748+0000 7f51386fc700 1 -- 172.21.15.83:0/1248613543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5130192920 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.749+0000 7f51277fe700 1 -- 172.21.15.83:0/1248613543 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f512c01aac0 con 0x7f5130106500 2024-03-11T19:29:23.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.749+0000 7f51386fc700 1 -- 172.21.15.83:0/1248613543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 msgr2=0x7f5130199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:23.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.749+0000 7f51386fc700 1 --2- 172.21.15.83:0/1248613543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130199250 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f512c009fb0 tx=0x7f512c011b20 comp rx=0 tx=0).stop 2024-03-11T19:29:23.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.749+0000 7f51386fc700 1 -- 172.21.15.83:0/1248613543 shutdown_connections 2024-03-11T19:29:23.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.749+0000 7f51386fc700 1 --2- 172.21.15.83:0/1248613543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5130106500 0x7f5130199250 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:23.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.749+0000 7f51386fc700 1 -- 172.21.15.83:0/1248613543 >> 172.21.15.83:0/1248613543 conn(0x7f5130101ac0 msgr2=0x7f513018f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:23.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.750+0000 7f51386fc700 1 -- 172.21.15.83:0/1248613543 shutdown_connections 2024-03-11T19:29:23.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:22.750+0000 7f51386fc700 1 -- 172.21.15.83:0/1248613543 wait complete. 2024-03-11T19:29:23.380 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (1/15)... 2024-03-11T19:29:23.711 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:23 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:23.476+0000 7f11e4adf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:29:23.712 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:23 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:23.567+0000 7f11e4adf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:29:23.712 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:23 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:23.708+0000 7f11e4adf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:29:24.067 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:23 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:23.804+0000 7f11e4adf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:29:24.068 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:23 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:23.887+0000 7f11e4adf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:29:24.445 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:24 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:24.151+0000 7f11e4adf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:29:24.445 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:24 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:24.259+0000 7f11e4adf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:29:24.445 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:24 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:24.346+0000 7f11e4adf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:29:24.445 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:24 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:24.441+0000 7f11e4adf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:29:24.817 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:24 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:24.613+0000 7f11e4adf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:29:25.768 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:25 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:25.584+0000 7f11e4adf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:29:25.768 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:25 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:25.717+0000 7f11e4adf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:29:26.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:25 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2345568611' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:29:26.317 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:26 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:26.052+0000 7f11e4adf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:29:26.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:26 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:26.145+0000 7f11e4adf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:29:26.434 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "839f93c8-dfdd-11ee-95c8-87774f69a715", 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:26.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:29:26.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:29:26.445 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:29:26.446 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:29:26.446 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:29:26.446 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:26.446 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:29:26.446 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:26.446 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:29:26.446 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:29:26.446 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:26.446 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:29:26.447 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-11T19:29:26.447 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:29:26.447 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:29:26.447 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:29:26.447 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:29:26.447 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:29:26.447 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:26.447 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:29:26.447 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:26.447 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:29:26.447 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:26.448 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:29:04.070734+0000", 2024-03-11T19:29:26.448 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:29:26.448 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:26.448 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:29:26.448 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:29:26.448 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.779+0000 7f2ac1bc3700 1 Processor -- start 2024-03-11T19:29:26.448 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.779+0000 7f2ac1bc3700 1 -- start start 2024-03-11T19:29:26.448 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.779+0000 7f2ac1bc3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:26.448 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.779+0000 7f2ac1bc3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc106ef0 con 0x7f2abc106500 2024-03-11T19:29:26.448 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.780+0000 7f2abb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:26.448 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.780+0000 7f2abb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35602/0 (socket says 172.21.15.83:35602) 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.780+0000 7f2abb7fe700 1 -- 172.21.15.83:0/3588889272 learned_addr learned my addr 172.21.15.83:0/3588889272 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.780+0000 7f2abb7fe700 1 -- 172.21.15.83:0/3588889272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc107700 con 0x7f2abc106500 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.780+0000 7f2abb7fe700 1 --2- 172.21.15.83:0/3588889272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc106920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00098a0 tx=0x7f2ab0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303463853e128830 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.780+0000 7f2aba7fc700 1 -- 172.21.15.83:0/3588889272 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ab0004160 con 0x7f2abc106500 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.780+0000 7f2aba7fc700 1 -- 172.21.15.83:0/3588889272 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ab00042c0 con 0x7f2abc106500 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.780+0000 7f2aba7fc700 1 -- 172.21.15.83:0/3588889272 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ab0011ac0 con 0x7f2abc106500 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.781+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/3588889272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 msgr2=0x7f2abc106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.781+0000 7f2ac1bc3700 1 --2- 172.21.15.83:0/3588889272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc106920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00098a0 tx=0x7f2ab0009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.781+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/3588889272 shutdown_connections 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.781+0000 7f2ac1bc3700 1 --2- 172.21.15.83:0/3588889272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc106920 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.781+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/3588889272 >> 172.21.15.83:0/3588889272 conn(0x7f2abc101b00 msgr2=0x7f2abc103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.781+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/3588889272 shutdown_connections 2024-03-11T19:29:26.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.781+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/3588889272 wait complete. 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.781+0000 7f2ac1bc3700 1 Processor -- start 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.782+0000 7f2ac1bc3700 1 -- start start 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.782+0000 7f2ac1bc3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc07aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.782+0000 7f2ac1bc3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc19e170 con 0x7f2abc106500 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.782+0000 7f2abb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc07aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.782+0000 7f2abb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc07aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35608/0 (socket says 172.21.15.83:35608) 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.782+0000 7f2abb7fe700 1 -- 172.21.15.83:0/2345568611 learned_addr learned my addr 172.21.15.83:0/2345568611 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.782+0000 7f2abb7fe700 1 -- 172.21.15.83:0/2345568611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab0009550 con 0x7f2abc106500 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.782+0000 7f2abb7fe700 1 --2- 172.21.15.83:0/2345568611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc07aa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009fb0 tx=0x7f2ab0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.783+0000 7f2ab8ff9700 1 -- 172.21.15.83:0/2345568611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ab00038b0 con 0x7f2abc106500 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.783+0000 7f2ab8ff9700 1 -- 172.21.15.83:0/2345568611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ab0003a10 con 0x7f2abc106500 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.783+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/2345568611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc19e2b0 con 0x7f2abc106500 2024-03-11T19:29:26.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.783+0000 7f2ab8ff9700 1 -- 172.21.15.83:0/2345568611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ab00195e0 con 0x7f2abc106500 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.783+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/2345568611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc19e690 con 0x7f2abc106500 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.783+0000 7f2ab8ff9700 1 -- 172.21.15.83:0/2345568611 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2ab0019c50 con 0x7f2abc106500 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.783+0000 7f2ab8ff9700 1 -- 172.21.15.83:0/2345568611 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2ab0024ac0 con 0x7f2abc106500 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.784+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/2345568611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc0632a0 con 0x7f2abc106500 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.786+0000 7f2ab8ff9700 1 -- 172.21.15.83:0/2345568611 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2ab0003b80 con 0x7f2abc106500 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.857+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/2345568611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2abc10a900 con 0x7f2abc106500 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.857+0000 7f2ab8ff9700 1 -- 172.21.15.83:0/2345568611 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f2ab001aa10 con 0x7f2abc106500 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.858+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/2345568611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 msgr2=0x7f2abc07aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.858+0000 7f2ac1bc3700 1 --2- 172.21.15.83:0/2345568611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc07aa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009fb0 tx=0x7f2ab0011b20 comp rx=0 tx=0).stop 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.858+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/2345568611 shutdown_connections 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.858+0000 7f2ac1bc3700 1 --2- 172.21.15.83:0/2345568611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2abc106500 0x7f2abc07aa80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.858+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/2345568611 >> 172.21.15.83:0/2345568611 conn(0x7f2abc101b00 msgr2=0x7f2abc102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.858+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/2345568611 shutdown_connections 2024-03-11T19:29:26.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:25.858+0000 7f2ac1bc3700 1 -- 172.21.15.83:0/2345568611 wait complete. 2024-03-11T19:29:26.452 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (2/15)... 2024-03-11T19:29:26.810 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:26 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:26.338+0000 7f11e4adf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:29:26.810 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:26 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:26.527+0000 7f11e4adf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:29:27.062 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:26 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:26.806+0000 7f11e4adf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:29:27.062 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:27 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:27.058+0000 7f11e4adf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:29:27.317 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:27 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:27.287+0000 7f11e4adf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:29:28.317 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:28 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:28.063+0000 7f11e4adf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:29:28.816 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:28 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:28.624+0000 7f11e4adf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:29:28.816 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:28 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:28.729+0000 7f11e4adf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:29:29.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:28 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/810677521' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:29:29.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:28 smithi083 ceph-mon[31864]: Activating manager daemon x 2024-03-11T19:29:29.068 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:28 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:28.958+0000 7f11e4adf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:29:29.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:29.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:29:29.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "839f93c8-dfdd-11ee-95c8-87774f69a715", 2024-03-11T19:29:29.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:29:29.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:29:29.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:29:29.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:29:29.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:29.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:29:29.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:29:29.492 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:29:29.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:29:29.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:29:29.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:29:29.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:29:29.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:29.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:29:29.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:29:29.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:29:29.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:29:29.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:29:29.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:29:29.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:29:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:29:04.070734+0000", 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.842+0000 7f70d15f7700 1 Processor -- start 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.842+0000 7f70d15f7700 1 -- start start 2024-03-11T19:29:29.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.842+0000 7f70d15f7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc105cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:29.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.842+0000 7f70d15f7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc106280 con 0x7f70cc105890 2024-03-11T19:29:29.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.843+0000 7f70caffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc105cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:29.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.843+0000 7f70caffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc105cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35612/0 (socket says 172.21.15.83:35612) 2024-03-11T19:29:29.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.843+0000 7f70caffd700 1 -- 172.21.15.83:0/3045648011 learned_addr learned my addr 172.21.15.83:0/3045648011 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:29.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.843+0000 7f70caffd700 1 -- 172.21.15.83:0/3045648011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc1063c0 con 0x7f70cc105890 2024-03-11T19:29:29.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.843+0000 7f70caffd700 1 --2- 172.21.15.83:0/3045648011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc105cb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0098a0 tx=0x7f70bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c9f927eee33252 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:29.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.843+0000 7f70c9ffb700 1 -- 172.21.15.83:0/3045648011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70bc004160 con 0x7f70cc105890 2024-03-11T19:29:29.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.843+0000 7f70c9ffb700 1 -- 172.21.15.83:0/3045648011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70bc0042c0 con 0x7f70cc105890 2024-03-11T19:29:29.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.843+0000 7f70c9ffb700 1 -- 172.21.15.83:0/3045648011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70bc011ac0 con 0x7f70cc105890 2024-03-11T19:29:29.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.844+0000 7f70d15f7700 1 -- 172.21.15.83:0/3045648011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 msgr2=0x7f70cc105cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:29.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.844+0000 7f70d15f7700 1 --2- 172.21.15.83:0/3045648011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc105cb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0098a0 tx=0x7f70bc009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.844+0000 7f70d15f7700 1 -- 172.21.15.83:0/3045648011 shutdown_connections 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.844+0000 7f70d15f7700 1 --2- 172.21.15.83:0/3045648011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc105cb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.844+0000 7f70d15f7700 1 -- 172.21.15.83:0/3045648011 >> 172.21.15.83:0/3045648011 conn(0x7f70cc101230 msgr2=0x7f70cc103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.844+0000 7f70d15f7700 1 -- 172.21.15.83:0/3045648011 shutdown_connections 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.844+0000 7f70d15f7700 1 -- 172.21.15.83:0/3045648011 wait complete. 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.845+0000 7f70d15f7700 1 Processor -- start 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.845+0000 7f70d15f7700 1 -- start start 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.845+0000 7f70d15f7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.845+0000 7f70d15f7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc19db50 con 0x7f70cc105890 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.845+0000 7f70caffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.845+0000 7f70caffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc19d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35628/0 (socket says 172.21.15.83:35628) 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.845+0000 7f70caffd700 1 -- 172.21.15.83:0/810677521 learned_addr learned my addr 172.21.15.83:0/810677521 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:29.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.845+0000 7f70caffd700 1 -- 172.21.15.83:0/810677521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc009550 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.846+0000 7f70caffd700 1 --2- 172.21.15.83:0/810677521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc19d610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f70bc003f70 tx=0x7f70bc011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.846+0000 7f70b3fff700 1 -- 172.21.15.83:0/810677521 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70bc0039f0 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.846+0000 7f70d15f7700 1 -- 172.21.15.83:0/810677521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70cc19dd50 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.846+0000 7f70b3fff700 1 -- 172.21.15.83:0/810677521 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70bc003b50 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.846+0000 7f70b3fff700 1 -- 172.21.15.83:0/810677521 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70bc01a690 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.846+0000 7f70d15f7700 1 -- 172.21.15.83:0/810677521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70cc19e1f0 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.846+0000 7f70b3fff700 1 -- 172.21.15.83:0/810677521 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f70bc01acb0 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.846+0000 7f70b3fff700 1 -- 172.21.15.83:0/810677521 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f70bc019970 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.847+0000 7f70d15f7700 1 -- 172.21.15.83:0/810677521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70cc0632a0 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.851+0000 7f70b3fff700 1 -- 172.21.15.83:0/810677521 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f70bc0249f0 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.922+0000 7f70d15f7700 1 -- 172.21.15.83:0/810677521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f70cc1a0dc0 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.923+0000 7f70b3fff700 1 -- 172.21.15.83:0/810677521 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f70bc019b30 con 0x7f70cc105890 2024-03-11T19:29:29.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.924+0000 7f70d15f7700 1 -- 172.21.15.83:0/810677521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 msgr2=0x7f70cc19d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:29.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.924+0000 7f70d15f7700 1 --2- 172.21.15.83:0/810677521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc19d610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f70bc003f70 tx=0x7f70bc011b40 comp rx=0 tx=0).stop 2024-03-11T19:29:29.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.924+0000 7f70d15f7700 1 -- 172.21.15.83:0/810677521 shutdown_connections 2024-03-11T19:29:29.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.924+0000 7f70d15f7700 1 --2- 172.21.15.83:0/810677521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f70cc105890 0x7f70cc19d610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:29.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.924+0000 7f70d15f7700 1 -- 172.21.15.83:0/810677521 >> 172.21.15.83:0/810677521 conn(0x7f70cc101230 msgr2=0x7f70cc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:29.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.924+0000 7f70d15f7700 1 -- 172.21.15.83:0/810677521 shutdown_connections 2024-03-11T19:29:29.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:28.924+0000 7f70d15f7700 1 -- 172.21.15.83:0/810677521 wait complete. 2024-03-11T19:29:29.509 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (3/15)... 2024-03-11T19:29:30.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: mgrmap e2: x(active, starting, since 0.00300912s) 2024-03-11T19:29:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: from='mgr.14100 172.21.15.83:0/2965581001' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:29:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: from='mgr.14100 172.21.15.83:0/2965581001' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:29:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: from='mgr.14100 172.21.15.83:0/2965581001' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:29:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: from='mgr.14100 172.21.15.83:0/2965581001' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:29:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: from='mgr.14100 172.21.15.83:0/2965581001' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:29:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: Manager daemon x is now available 2024-03-11T19:29:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: from='mgr.14100 172.21.15.83:0/2965581001' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:29:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: from='mgr.14100 172.21.15.83:0/2965581001' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:29:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: from='mgr.14100 172.21.15.83:0/2965581001' entity='mgr.x' 2024-03-11T19:29:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: from='mgr.14100 172.21.15.83:0/2965581001' entity='mgr.x' 2024-03-11T19:29:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:29 smithi083 ceph-mon[31864]: from='mgr.14100 172.21.15.83:0/2965581001' entity='mgr.x' 2024-03-11T19:29:31.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:30 smithi083 ceph-mon[31864]: mgrmap e3: x(active, since 1.0107s) 2024-03-11T19:29:32.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:32.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:29:32.773 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "839f93c8-dfdd-11ee-95c8-87774f69a715", 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:32.774 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-11T19:29:32.775 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-11T19:29:32.776 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-11T19:29:32.777 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-03-11T19:29:04.070734+0000", 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.941+0000 7f39cf7a3700 1 Processor -- start 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.941+0000 7f39cf7a3700 1 -- start start 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.941+0000 7f39cf7a3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c807b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.941+0000 7f39cf7a3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c807b8e0 con 0x7f39c807ce60 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.942+0000 7f39cd53f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.942+0000 7f39cd53f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c807b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47592/0 (socket says 172.21.15.83:47592) 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.942+0000 7f39cd53f700 1 -- 172.21.15.83:0/4102293268 learned_addr learned my addr 172.21.15.83:0/4102293268 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.942+0000 7f39cd53f700 1 -- 172.21.15.83:0/4102293268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c807ba20 con 0x7f39c807ce60 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.942+0000 7f39cd53f700 1 --2- 172.21.15.83:0/4102293268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c807b310 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009e80 tx=0x7f39c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4e81556ee18a8c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:32.778 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.942+0000 7f39bffff700 1 -- 172.21.15.83:0/4102293268 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39c400e070 con 0x7f39c807ce60 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.942+0000 7f39bffff700 1 -- 172.21.15.83:0/4102293268 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39c400b8c0 con 0x7f39c807ce60 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.942+0000 7f39bffff700 1 -- 172.21.15.83:0/4102293268 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39c4004550 con 0x7f39c807ce60 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.943+0000 7f39cf7a3700 1 -- 172.21.15.83:0/4102293268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 msgr2=0x7f39c807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.943+0000 7f39cf7a3700 1 --2- 172.21.15.83:0/4102293268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c807b310 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009e80 tx=0x7f39c400b330 comp rx=0 tx=0).stop 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.943+0000 7f39cf7a3700 1 -- 172.21.15.83:0/4102293268 shutdown_connections 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.943+0000 7f39cf7a3700 1 --2- 172.21.15.83:0/4102293268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c807b310 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.943+0000 7f39cf7a3700 1 -- 172.21.15.83:0/4102293268 >> 172.21.15.83:0/4102293268 conn(0x7f39c8102bd0 msgr2=0x7f39c8104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.943+0000 7f39cf7a3700 1 -- 172.21.15.83:0/4102293268 shutdown_connections 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.943+0000 7f39cf7a3700 1 -- 172.21.15.83:0/4102293268 wait complete. 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39cf7a3700 1 Processor -- start 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39cf7a3700 1 -- start start 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39cf7a3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39cf7a3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c819da00 con 0x7f39c807ce60 2024-03-11T19:29:32.779 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39cd53f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39cd53f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c819d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47594/0 (socket says 172.21.15.83:47594) 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39cd53f700 1 -- 172.21.15.83:0/893290153 learned_addr learned my addr 172.21.15.83:0/893290153 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39cd53f700 1 -- 172.21.15.83:0/893290153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c4009b30 con 0x7f39c807ce60 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39cd53f700 1 --2- 172.21.15.83:0/893290153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c819d4c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f39c4004070 tx=0x7f39c4003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39be7fc700 1 -- 172.21.15.83:0/893290153 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39c400e040 con 0x7f39c807ce60 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39be7fc700 1 -- 172.21.15.83:0/893290153 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39c400bdf0 con 0x7f39c807ce60 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.944+0000 7f39cf7a3700 1 -- 172.21.15.83:0/893290153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c819dc00 con 0x7f39c807ce60 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.945+0000 7f39be7fc700 1 -- 172.21.15.83:0/893290153 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39c401c840 con 0x7f39c807ce60 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.945+0000 7f39cf7a3700 1 -- 172.21.15.83:0/893290153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c819e080 con 0x7f39c807ce60 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.945+0000 7f39be7fc700 1 -- 172.21.15.83:0/893290153 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 3) v1 ==== 45652+0+0 (secure 0 0 0) 0x7f39c401cb00 con 0x7f39c807ce60 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.945+0000 7f39be7fc700 1 --2- 172.21.15.83:0/893290153 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f39b4038ae0 0x7f39b403afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.945+0000 7f39cf7a3700 1 -- 172.21.15.83:0/893290153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c810d9a0 con 0x7f39c807ce60 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.946+0000 7f39be7fc700 1 -- 172.21.15.83:0/893290153 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f39c404cbb0 con 0x7f39c807ce60 2024-03-11T19:29:32.780 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.946+0000 7f39ccd3e700 1 --2- 172.21.15.83:0/893290153 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f39b4038ae0 0x7f39b403afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.946+0000 7f39ccd3e700 1 --2- 172.21.15.83:0/893290153 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f39b4038ae0 0x7f39b403afa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006fd0 tx=0x7f39b8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.951+0000 7f39be7fc700 1 -- 172.21.15.83:0/893290153 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f39c4017080 con 0x7f39c807ce60 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:31.985+0000 7f39be7fc700 1 -- 172.21.15.83:0/893290153 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f39c401b3a0 con 0x7f39c807ce60 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.223+0000 7f39cf7a3700 1 -- 172.21.15.83:0/893290153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f39c81a0d20 con 0x7f39c807ce60 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.223+0000 7f39be7fc700 1 -- 172.21.15.83:0/893290153 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f39c4011960 con 0x7f39c807ce60 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.225+0000 7f39cf7a3700 1 -- 172.21.15.83:0/893290153 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f39b4038ae0 msgr2=0x7f39b403afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.225+0000 7f39cf7a3700 1 --2- 172.21.15.83:0/893290153 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f39b4038ae0 0x7f39b403afa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f39b8006fd0 tx=0x7f39b8006e40 comp rx=0 tx=0).stop 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.225+0000 7f39cf7a3700 1 -- 172.21.15.83:0/893290153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 msgr2=0x7f39c819d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.225+0000 7f39cf7a3700 1 --2- 172.21.15.83:0/893290153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c819d4c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f39c4004070 tx=0x7f39c4003d00 comp rx=0 tx=0).stop 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.225+0000 7f39cf7a3700 1 -- 172.21.15.83:0/893290153 shutdown_connections 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.225+0000 7f39cf7a3700 1 --2- 172.21.15.83:0/893290153 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f39b4038ae0 0x7f39b403afa0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.225+0000 7f39cf7a3700 1 --2- 172.21.15.83:0/893290153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c807ce60 0x7f39c819d4c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.226+0000 7f39cf7a3700 1 -- 172.21.15.83:0/893290153 >> 172.21.15.83:0/893290153 conn(0x7f39c8102bd0 msgr2=0x7f39c81036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:32.781 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.226+0000 7f39cf7a3700 1 -- 172.21.15.83:0/893290153 shutdown_connections 2024-03-11T19:29:32.782 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:32.226+0000 7f39cf7a3700 1 -- 172.21.15.83:0/893290153 wait complete. 2024-03-11T19:29:32.782 INFO:teuthology.orchestra.run.smithi083.stdout:mgr is available 2024-03-11T19:29:33.163 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:32 smithi083 ceph-mon[31864]: mgrmap e4: x(active, since 3s) 2024-03-11T19:29:33.163 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:32 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/893290153' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-03-11T19:29:33.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.173+0000 7f53ce366700 1 Processor -- start 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.173+0000 7f53ce366700 1 -- start start 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.174+0000 7f53ce366700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c8105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.174+0000 7f53ce366700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c81062f0 con 0x7f53c8105900 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.174+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c8105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.174+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c8105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47598/0 (socket says 172.21.15.83:47598) 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.174+0000 7f53c7fff700 1 -- 172.21.15.83:0/2648723285 learned_addr learned my addr 172.21.15.83:0/2648723285 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.174+0000 7f53c7fff700 1 -- 172.21.15.83:0/2648723285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c8106430 con 0x7f53c8105900 2024-03-11T19:29:33.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.174+0000 7f53c7fff700 1 --2- 172.21.15.83:0/2648723285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c8105d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f53b80098a0 tx=0x7f53b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66165a42f2da962 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.174+0000 7f53c77fe700 1 -- 172.21.15.83:0/2648723285 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53b8004160 con 0x7f53c8105900 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.174+0000 7f53c77fe700 1 -- 172.21.15.83:0/2648723285 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53b80042c0 con 0x7f53c8105900 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.174+0000 7f53c77fe700 1 -- 172.21.15.83:0/2648723285 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53b8011ac0 con 0x7f53c8105900 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.175+0000 7f53ce366700 1 -- 172.21.15.83:0/2648723285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 msgr2=0x7f53c8105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.175+0000 7f53ce366700 1 --2- 172.21.15.83:0/2648723285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c8105d20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f53b80098a0 tx=0x7f53b8009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.175+0000 7f53ce366700 1 -- 172.21.15.83:0/2648723285 shutdown_connections 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.175+0000 7f53ce366700 1 --2- 172.21.15.83:0/2648723285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c8105d20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.175+0000 7f53ce366700 1 -- 172.21.15.83:0/2648723285 >> 172.21.15.83:0/2648723285 conn(0x7f53c81012c0 msgr2=0x7f53c81036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.175+0000 7f53ce366700 1 -- 172.21.15.83:0/2648723285 shutdown_connections 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.175+0000 7f53ce366700 1 -- 172.21.15.83:0/2648723285 wait complete. 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.175+0000 7f53ce366700 1 Processor -- start 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.175+0000 7f53ce366700 1 -- start start 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53ce366700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:33.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53ce366700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c819b900 con 0x7f53c8105900 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47604/0 (socket says 172.21.15.83:47604) 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53c7fff700 1 -- 172.21.15.83:0/1886939269 learned_addr learned my addr 172.21.15.83:0/1886939269 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53c7fff700 1 -- 172.21.15.83:0/1886939269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b8009550 con 0x7f53c8105900 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53c7fff700 1 --2- 172.21.15.83:0/1886939269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c819b3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009fb0 tx=0x7f53b8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53c5ffb700 1 -- 172.21.15.83:0/1886939269 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53b80038b0 con 0x7f53c8105900 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53c5ffb700 1 -- 172.21.15.83:0/1886939269 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53b8003a10 con 0x7f53c8105900 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53ce366700 1 -- 172.21.15.83:0/1886939269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c819bb60 con 0x7f53c8105900 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53c5ffb700 1 -- 172.21.15.83:0/1886939269 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53b80195e0 con 0x7f53c8105900 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.176+0000 7f53ce366700 1 -- 172.21.15.83:0/1886939269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c819e880 con 0x7f53c8105900 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.177+0000 7f53c5ffb700 1 -- 172.21.15.83:0/1886939269 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f53b8019850 con 0x7f53c8105900 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.177+0000 7f53c5ffb700 1 --2- 172.21.15.83:0/1886939269 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f53b4038c60 0x7f53b403b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.177+0000 7f53ce366700 1 -- 172.21.15.83:0/1886939269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c80632a0 con 0x7f53c8105900 2024-03-11T19:29:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.177+0000 7f53c5ffb700 1 -- 172.21.15.83:0/1886939269 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f53b804cd20 con 0x7f53c8105900 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.177+0000 7f53bffff700 1 --2- 172.21.15.83:0/1886939269 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f53b4038c60 0x7f53b403b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.178+0000 7f53bffff700 1 --2- 172.21.15.83:0/1886939269 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f53b4038c60 0x7f53b403b120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f53b0006fd0 tx=0x7f53b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.182+0000 7f53c5ffb700 1 -- 172.21.15.83:0/1886939269 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53b8019b00 con 0x7f53c8105900 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.374+0000 7f53ce366700 1 -- 172.21.15.83:0/1886939269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f53c819eac0 con 0x7f53c8105900 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.376+0000 7f53c5ffb700 1 -- 172.21.15.83:0/1886939269 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f53b8018be0 con 0x7f53c8105900 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.378+0000 7f53ce366700 1 -- 172.21.15.83:0/1886939269 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f53b4038c60 msgr2=0x7f53b403b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.378+0000 7f53ce366700 1 --2- 172.21.15.83:0/1886939269 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f53b4038c60 0x7f53b403b120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f53b0006fd0 tx=0x7f53b0006e40 comp rx=0 tx=0).stop 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.378+0000 7f53ce366700 1 -- 172.21.15.83:0/1886939269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 msgr2=0x7f53c819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.378+0000 7f53ce366700 1 --2- 172.21.15.83:0/1886939269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c819b3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009fb0 tx=0x7f53b8011b20 comp rx=0 tx=0).stop 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.378+0000 7f53ce366700 1 -- 172.21.15.83:0/1886939269 shutdown_connections 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.378+0000 7f53ce366700 1 --2- 172.21.15.83:0/1886939269 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f53b4038c60 0x7f53b403b120 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.378+0000 7f53ce366700 1 --2- 172.21.15.83:0/1886939269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53c8105900 0x7f53c819b3c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.378+0000 7f53ce366700 1 -- 172.21.15.83:0/1886939269 >> 172.21.15.83:0/1886939269 conn(0x7f53c81012c0 msgr2=0x7f53c8101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.378+0000 7f53ce366700 1 -- 172.21.15.83:0/1886939269 shutdown_connections 2024-03-11T19:29:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:33.378+0000 7f53ce366700 1 -- 172.21.15.83:0/1886939269 wait complete. 2024-03-11T19:29:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling cephadm module... 2024-03-11T19:29:34.261 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:33 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1886939269' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-11T19:29:35.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:34 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2562410826' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-11T19:29:35.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: ignoring --setuser ceph since I am not root 2024-03-11T19:29:35.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: ignoring --setgroup ceph since I am not root 2024-03-11T19:29:35.573 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:35.547+0000 7ff9eff0f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:29:35.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.359+0000 7f854419f700 1 Processor -- start 2024-03-11T19:29:35.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.360+0000 7f854419f700 1 -- start start 2024-03-11T19:29:35.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.360+0000 7f854419f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:35.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.360+0000 7f854419f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c07b8e0 con 0x7f853c07ce60 2024-03-11T19:29:35.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.360+0000 7f8541f3b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:35.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.360+0000 7f8541f3b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c07b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47616/0 (socket says 172.21.15.83:47616) 2024-03-11T19:29:35.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.360+0000 7f8541f3b700 1 -- 172.21.15.83:0/3752472115 learned_addr learned my addr 172.21.15.83:0/3752472115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:35.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.360+0000 7f8541f3b700 1 -- 172.21.15.83:0/3752472115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c07ba20 con 0x7f853c07ce60 2024-03-11T19:29:35.580 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.360+0000 7f8541f3b700 1 --2- 172.21.15.83:0/3752472115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c07b310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f852c0098a0 tx=0x7f852c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6a0ec090366e3c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.360+0000 7f8540f39700 1 -- 172.21.15.83:0/3752472115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f852c004160 con 0x7f853c07ce60 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.361+0000 7f8540f39700 1 -- 172.21.15.83:0/3752472115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f852c0042c0 con 0x7f853c07ce60 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.361+0000 7f8540f39700 1 -- 172.21.15.83:0/3752472115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f852c011ac0 con 0x7f853c07ce60 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.361+0000 7f854419f700 1 -- 172.21.15.83:0/3752472115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 msgr2=0x7f853c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.361+0000 7f854419f700 1 --2- 172.21.15.83:0/3752472115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c07b310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f852c0098a0 tx=0x7f852c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.361+0000 7f854419f700 1 -- 172.21.15.83:0/3752472115 shutdown_connections 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.361+0000 7f854419f700 1 --2- 172.21.15.83:0/3752472115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c07b310 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.361+0000 7f854419f700 1 -- 172.21.15.83:0/3752472115 >> 172.21.15.83:0/3752472115 conn(0x7f853c102cc0 msgr2=0x7f853c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.361+0000 7f854419f700 1 -- 172.21.15.83:0/3752472115 shutdown_connections 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.361+0000 7f854419f700 1 -- 172.21.15.83:0/3752472115 wait complete. 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.362+0000 7f854419f700 1 Processor -- start 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.362+0000 7f854419f700 1 -- start start 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.362+0000 7f854419f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:35.581 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.362+0000 7f854419f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c19dae0 con 0x7f853c07ce60 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.362+0000 7f8541f3b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.362+0000 7f8541f3b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47624/0 (socket says 172.21.15.83:47624) 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.362+0000 7f8541f3b700 1 -- 172.21.15.83:0/2562410826 learned_addr learned my addr 172.21.15.83:0/2562410826 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.362+0000 7f8541f3b700 1 -- 172.21.15.83:0/2562410826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c009550 con 0x7f853c07ce60 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.362+0000 7f8541f3b700 1 --2- 172.21.15.83:0/2562410826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c19d5a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f852c011a70 tx=0x7f852c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.363+0000 7f8532ffd700 1 -- 172.21.15.83:0/2562410826 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f852c0039c0 con 0x7f853c07ce60 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.363+0000 7f8532ffd700 1 -- 172.21.15.83:0/2562410826 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f852c003b20 con 0x7f853c07ce60 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.363+0000 7f8532ffd700 1 -- 172.21.15.83:0/2562410826 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f852c024610 con 0x7f853c07ce60 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.363+0000 7f854419f700 1 -- 172.21.15.83:0/2562410826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853c19dce0 con 0x7f853c07ce60 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.363+0000 7f854419f700 1 -- 172.21.15.83:0/2562410826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c19e180 con 0x7f853c07ce60 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.363+0000 7f8532ffd700 1 -- 172.21.15.83:0/2562410826 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) v1 ==== 45758+0+0 (secure 0 0 0) 0x7f852c01a430 con 0x7f853c07ce60 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.364+0000 7f854419f700 1 -- 172.21.15.83:0/2562410826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853c10d9d0 con 0x7f853c07ce60 2024-03-11T19:29:35.582 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.364+0000 7f8532ffd700 1 --2- 172.21.15.83:0/2562410826 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f8528038cb0 0x7f852803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.364+0000 7f8532ffd700 1 -- 172.21.15.83:0/2562410826 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f852c04cca0 con 0x7f853c07ce60 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.364+0000 7f854173a700 1 --2- 172.21.15.83:0/2562410826 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f8528038cb0 0x7f852803b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.364+0000 7f854173a700 1 --2- 172.21.15.83:0/2562410826 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f8528038cb0 0x7f852803b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8538006fd0 tx=0x7f8538006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.369+0000 7f8532ffd700 1 -- 172.21.15.83:0/2562410826 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f852c010380 con 0x7f853c07ce60 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.616+0000 7f854419f700 1 -- 172.21.15.83:0/2562410826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f853c1a0f50 con 0x7f853c07ce60 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.988+0000 7f8532ffd700 1 -- 172.21.15.83:0/2562410826 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f852c018be0 con 0x7f853c07ce60 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.989+0000 7f8532ffd700 1 -- 172.21.15.83:0/2562410826 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f852c01aac0 con 0x7f853c07ce60 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.991+0000 7f854419f700 1 -- 172.21.15.83:0/2562410826 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f8528038cb0 msgr2=0x7f852803b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.991+0000 7f854419f700 1 --2- 172.21.15.83:0/2562410826 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f8528038cb0 0x7f852803b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8538006fd0 tx=0x7f8538006e40 comp rx=0 tx=0).stop 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.991+0000 7f854419f700 1 -- 172.21.15.83:0/2562410826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 msgr2=0x7f853c19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.991+0000 7f854419f700 1 --2- 172.21.15.83:0/2562410826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c19d5a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f852c011a70 tx=0x7f852c011b50 comp rx=0 tx=0).stop 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.991+0000 7f854419f700 1 -- 172.21.15.83:0/2562410826 shutdown_connections 2024-03-11T19:29:35.583 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.991+0000 7f854419f700 1 --2- 172.21.15.83:0/2562410826 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f8528038cb0 0x7f852803b170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:35.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.991+0000 7f854419f700 1 --2- 172.21.15.83:0/2562410826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f853c07ce60 0x7f853c19d5a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:35.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.991+0000 7f854419f700 1 -- 172.21.15.83:0/2562410826 >> 172.21.15.83:0/2562410826 conn(0x7f853c102cc0 msgr2=0x7f853c103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:35.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.991+0000 7f854419f700 1 -- 172.21.15.83:0/2562410826 shutdown_connections 2024-03-11T19:29:35.584 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:34.991+0000 7f854419f700 1 -- 172.21.15.83:0/2562410826 wait complete. 2024-03-11T19:29:35.867 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:35.650+0000 7ff9eff0f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:29:36.171 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:35 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2562410826' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-11T19:29:36.171 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:35 smithi083 ceph-mon[31864]: mgrmap e5: x(active, since 6s) 2024-03-11T19:29:36.530 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:36 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:36.311+0000 7ff9eff0f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:29:36.530 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:36 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:36.406+0000 7ff9eff0f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:29:36.817 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:36 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:36.550+0000 7ff9eff0f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:29:36.818 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:36 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:36.647+0000 7ff9eff0f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:29:36.818 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:36 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:36.729+0000 7ff9eff0f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:29:36.854 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:29:36.854 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-11T19:29:36.854 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-11T19:29:36.854 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-11T19:29:36.854 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.962+0000 7f4754f67700 1 Processor -- start 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.962+0000 7f4754f67700 1 -- start start 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.962+0000 7f4754f67700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f47501062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.962+0000 7f4754f67700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475007ae40 con 0x7f4750105ea0 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.962+0000 7f474e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f47501062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.962+0000 7f474e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f47501062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47648/0 (socket says 172.21.15.83:47648) 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.962+0000 7f474e59c700 1 -- 172.21.15.83:0/282481892 learned_addr learned my addr 172.21.15.83:0/282481892 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.962+0000 7f474e59c700 1 -- 172.21.15.83:0/282481892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4750106800 con 0x7f4750105ea0 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.963+0000 7f474e59c700 1 --2- 172.21.15.83:0/282481892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f47501062c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47400098a0 tx=0x7f4740009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85c5350e955424d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.963+0000 7f474d59a700 1 -- 172.21.15.83:0/282481892 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4740004160 con 0x7f4750105ea0 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.963+0000 7f474d59a700 1 -- 172.21.15.83:0/282481892 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47400042c0 con 0x7f4750105ea0 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.963+0000 7f474d59a700 1 -- 172.21.15.83:0/282481892 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4740011ac0 con 0x7f4750105ea0 2024-03-11T19:29:36.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.963+0000 7f4754f67700 1 -- 172.21.15.83:0/282481892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 msgr2=0x7f47501062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.963+0000 7f4754f67700 1 --2- 172.21.15.83:0/282481892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f47501062c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47400098a0 tx=0x7f4740009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.964+0000 7f4754f67700 1 -- 172.21.15.83:0/282481892 shutdown_connections 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.964+0000 7f4754f67700 1 --2- 172.21.15.83:0/282481892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f47501062c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.964+0000 7f4754f67700 1 -- 172.21.15.83:0/282481892 >> 172.21.15.83:0/282481892 conn(0x7f4750101aa0 msgr2=0x7f4750103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.964+0000 7f4754f67700 1 -- 172.21.15.83:0/282481892 shutdown_connections 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.964+0000 7f4754f67700 1 -- 172.21.15.83:0/282481892 wait complete. 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.964+0000 7f4754f67700 1 Processor -- start 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.964+0000 7f4754f67700 1 -- start start 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.964+0000 7f4754f67700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f475019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.964+0000 7f4754f67700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475019dac0 con 0x7f4750105ea0 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.965+0000 7f474e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f475019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.965+0000 7f474e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f475019d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47664/0 (socket says 172.21.15.83:47664) 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.965+0000 7f474e59c700 1 -- 172.21.15.83:0/2712953165 learned_addr learned my addr 172.21.15.83:0/2712953165 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.965+0000 7f474e59c700 1 -- 172.21.15.83:0/2712953165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4740009550 con 0x7f4750105ea0 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.965+0000 7f474e59c700 1 --2- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f475019d580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4740003f70 tx=0x7f4740011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.965+0000 7f473f7fe700 1 -- 172.21.15.83:0/2712953165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47400039f0 con 0x7f4750105ea0 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.965+0000 7f473f7fe700 1 -- 172.21.15.83:0/2712953165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4740003b50 con 0x7f4750105ea0 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.965+0000 7f4754f67700 1 -- 172.21.15.83:0/2712953165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475019dcc0 con 0x7f4750105ea0 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.965+0000 7f473f7fe700 1 -- 172.21.15.83:0/2712953165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f474001a760 con 0x7f4750105ea0 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.965+0000 7f4754f67700 1 -- 172.21.15.83:0/2712953165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475019e160 con 0x7f4750105ea0 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.966+0000 7f473f7fe700 1 -- 172.21.15.83:0/2712953165 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f474001a9a0 con 0x7f4750105ea0 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.966+0000 7f473f7fe700 1 --2- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4738038cb0 0x7f473803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.966+0000 7f4754f67700 1 -- 172.21.15.83:0/2712953165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4750197220 con 0x7f4750105ea0 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.966+0000 7f473f7fe700 1 -- 172.21.15.83:0/2712953165 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f474004ce60 con 0x7f4750105ea0 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.966+0000 7f474dd9b700 1 -- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4738038cb0 msgr2=0x7f473803b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2370098752 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.967+0000 7f474dd9b700 1 --2- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4738038cb0 0x7f473803b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:35.972+0000 7f473f7fe700 1 -- 172.21.15.83:0/2712953165 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f474001ae30 con 0x7f4750105ea0 2024-03-11T19:29:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.168+0000 7f474dd9b700 1 -- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4738038cb0 msgr2=0x7f473803b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2370098752 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.168+0000 7f474dd9b700 1 --2- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4738038cb0 0x7f473803b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.225+0000 7f4754f67700 1 -- 172.21.15.83:0/2712953165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f475002d9b0 con 0x7f4750105ea0 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.226+0000 7f473f7fe700 1 -- 172.21.15.83:0/2712953165 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f474004a030 con 0x7f4750105ea0 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.228+0000 7f4754f67700 1 -- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4738038cb0 msgr2=0x7f473803b170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.228+0000 7f4754f67700 1 --2- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4738038cb0 0x7f473803b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.228+0000 7f4754f67700 1 -- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 msgr2=0x7f475019d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.228+0000 7f4754f67700 1 --2- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f475019d580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4740003f70 tx=0x7f4740011b40 comp rx=0 tx=0).stop 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.228+0000 7f4754f67700 1 -- 172.21.15.83:0/2712953165 shutdown_connections 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.228+0000 7f4754f67700 1 --2- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4738038cb0 0x7f473803b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.228+0000 7f4754f67700 1 --2- 172.21.15.83:0/2712953165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4750105ea0 0x7f475019d580 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.228+0000 7f4754f67700 1 -- 172.21.15.83:0/2712953165 >> 172.21.15.83:0/2712953165 conn(0x7f4750101aa0 msgr2=0x7f4750194360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.228+0000 7f4754f67700 1 -- 172.21.15.83:0/2712953165 shutdown_connections 2024-03-11T19:29:36.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:36.228+0000 7f4754f67700 1 -- 172.21.15.83:0/2712953165 wait complete. 2024-03-11T19:29:36.859 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-03-11T19:29:36.859 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 5... 2024-03-11T19:29:37.121 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:36 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2712953165' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-11T19:29:37.121 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:37.006+0000 7ff9eff0f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:29:37.121 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:37.117+0000 7ff9eff0f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:29:37.487 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:37.207+0000 7ff9eff0f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:29:37.487 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:37.305+0000 7ff9eff0f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:29:37.817 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:37.483+0000 7ff9eff0f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:29:38.756 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:38 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:38.427+0000 7ff9eff0f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:29:38.757 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:38 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:38.555+0000 7ff9eff0f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:29:39.068 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:38 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:38.884+0000 7ff9eff0f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:29:39.068 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:38 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:38.974+0000 7ff9eff0f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:29:39.567 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:39 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:39.166+0000 7ff9eff0f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:29:39.567 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:39 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:39.347+0000 7ff9eff0f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:29:39.889 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:39 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:39.629+0000 7ff9eff0f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:29:40.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:39 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:39.885+0000 7ff9eff0f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:29:40.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:40 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:40.118+0000 7ff9eff0f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:29:41.317 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:40 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:40.909+0000 7ff9eff0f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:29:41.789 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:41.472+0000 7ff9eff0f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:29:41.789 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:41.570+0000 7ff9eff0f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:29:41.789 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:29:41.784+0000 7ff9eff0f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:29:42.067 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: Active manager daemon x restarted 2024-03-11T19:29:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: Activating manager daemon x 2024-03-11T19:29:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: osdmap e2: 0 total, 0 up, 0 in 2024-03-11T19:29:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: mgrmap e6: x(active, starting, since 0.00328854s) 2024-03-11T19:29:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:29:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:29:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:29:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:29:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:29:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: Manager daemon x is now available 2024-03-11T19:29:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:42.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:29:42.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:41 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:29:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:42 smithi083 ceph-mon[31864]: Found migration_current of "None". Setting to last migration. 2024-03-11T19:29:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:42 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:29:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:42 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:29:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:42 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:29:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:42 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:42 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:42 smithi083 ceph-mon[31864]: mgrmap e7: x(active, since 1.00547s) 2024-03-11T19:29:43.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:29:43.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-11T19:29:43.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-11T19:29:43.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:29:43.379 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.345+0000 7f429bd27700 1 Processor -- start 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.345+0000 7f429bd27700 1 -- start start 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.345+0000 7f429bd27700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f42941062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.345+0000 7f429bd27700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429407ae60 con 0x7f4294105e80 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.345+0000 7f4299ac3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f42941062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.346+0000 7f4299ac3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f42941062a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47678/0 (socket says 172.21.15.83:47678) 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.346+0000 7f4299ac3700 1 -- 172.21.15.83:0/1945193096 learned_addr learned my addr 172.21.15.83:0/1945193096 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.346+0000 7f4299ac3700 1 -- 172.21.15.83:0/1945193096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42941067e0 con 0x7f4294105e80 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.346+0000 7f4299ac3700 1 --2- 172.21.15.83:0/1945193096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f42941062a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f42840098a0 tx=0x7f4284009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675aac83c67edac5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.346+0000 7f4298ac1700 1 -- 172.21.15.83:0/1945193096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4284004160 con 0x7f4294105e80 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.346+0000 7f4298ac1700 1 -- 172.21.15.83:0/1945193096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42840042c0 con 0x7f4294105e80 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.346+0000 7f4298ac1700 1 -- 172.21.15.83:0/1945193096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4284011ac0 con 0x7f4294105e80 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.347+0000 7f429bd27700 1 -- 172.21.15.83:0/1945193096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 msgr2=0x7f42941062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:43.380 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.347+0000 7f429bd27700 1 --2- 172.21.15.83:0/1945193096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f42941062a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f42840098a0 tx=0x7f4284009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.347+0000 7f429bd27700 1 -- 172.21.15.83:0/1945193096 shutdown_connections 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.347+0000 7f429bd27700 1 --2- 172.21.15.83:0/1945193096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f42941062a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.347+0000 7f429bd27700 1 -- 172.21.15.83:0/1945193096 >> 172.21.15.83:0/1945193096 conn(0x7f4294101aa0 msgr2=0x7f4294103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.347+0000 7f429bd27700 1 -- 172.21.15.83:0/1945193096 shutdown_connections 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.347+0000 7f429bd27700 1 -- 172.21.15.83:0/1945193096 wait complete. 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.347+0000 7f429bd27700 1 Processor -- start 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f429bd27700 1 -- start start 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f429bd27700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f429419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f429bd27700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429419da80 con 0x7f4294105e80 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f4299ac3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f429419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f4299ac3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f429419d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47686/0 (socket says 172.21.15.83:47686) 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f4299ac3700 1 -- 172.21.15.83:0/3223377831 learned_addr learned my addr 172.21.15.83:0/3223377831 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:43.381 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f4299ac3700 1 -- 172.21.15.83:0/3223377831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4284009550 con 0x7f4294105e80 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f4299ac3700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f429419d540 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4284003f70 tx=0x7f4284011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42840039f0 con 0x7f4294105e80 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4284003b50 con 0x7f4294105e80 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f429bd27700 1 -- 172.21.15.83:0/3223377831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429419dc80 con 0x7f4294105e80 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f428401a760 con 0x7f4294105e80 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.348+0000 7f429bd27700 1 -- 172.21.15.83:0/3223377831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429419e120 con 0x7f4294105e80 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.349+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f4284024430 con 0x7f4294105e80 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.349+0000 7f428affd700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 0x7f428003b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.349+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 --> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f428003b880 con 0x7f4280038cb0 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.349+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f428404ce90 con 0x7f4294105e80 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.349+0000 7f42992c2700 1 -- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 msgr2=0x7f428003b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2370098752 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.349+0000 7f42992c2700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 0x7f428003b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:29:43.382 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.551+0000 7f42992c2700 1 -- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 msgr2=0x7f428003b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2370098752 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.551+0000 7f42992c2700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 0x7f428003b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.952+0000 7f42992c2700 1 -- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 msgr2=0x7f428003b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2370098752 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:37.952+0000 7f42992c2700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 0x7f428003b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:38.753+0000 7f42992c2700 1 -- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 msgr2=0x7f428003b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2370098752 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:38.753+0000 7f42992c2700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 0x7f428003b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:40.353+0000 7f42992c2700 1 -- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 msgr2=0x7f428003b170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2370098752 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:40.353+0000 7f42992c2700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 0x7f428003b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:41.799+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mgrmap(e 6) v1 ==== 45536+0+0 (secure 0 0 0) 0x7f428402d910 con 0x7f4294105e80 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:41.799+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 msgr2=0x7f428003b170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:41.799+0000 7f428affd700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 0x7f428003b170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.800+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 7) v1 ==== 45663+0+0 (secure 0 0 0) 0x7f428404ec40 con 0x7f4294105e80 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.800+0000 7f428affd700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f428003c5a0 0x7f428003e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:43.383 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.800+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 --> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f428003b880 con 0x7f428003c5a0 2024-03-11T19:29:43.384 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.800+0000 7f42992c2700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f428003c5a0 0x7f428003e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:43.384 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.801+0000 7f42992c2700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f428003c5a0 0x7f428003e990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4290003a10 tx=0x7f42900092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:43.384 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.801+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 <== mgr.14120 v2:172.21.15.83:6800/4058944299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f428003b880 con 0x7f428003c5a0 2024-03-11T19:29:43.384 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 -- 172.21.15.83:0/3223377831 --> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f42941076b0 con 0x7f428003c5a0 2024-03-11T19:29:43.384 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f428affd700 1 -- 172.21.15.83:0/3223377831 <== mgr.14120 v2:172.21.15.83:6800/4058944299 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f42941076b0 con 0x7f428003c5a0 2024-03-11T19:29:43.384 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 -- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f428003c5a0 msgr2=0x7f428003e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:43.384 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f428003c5a0 0x7f428003e990 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4290003a10 tx=0x7f42900092b0 comp rx=0 tx=0).stop 2024-03-11T19:29:43.384 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 -- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 msgr2=0x7f429419d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:43.384 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f429419d540 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4284003f70 tx=0x7f4284011b40 comp rx=0 tx=0).stop 2024-03-11T19:29:43.384 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 -- 172.21.15.83:0/3223377831 shutdown_connections 2024-03-11T19:29:43.385 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f428003c5a0 0x7f428003e990 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:43.385 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294105e80 0x7f429419d540 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:43.385 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 --2- 172.21.15.83:0/3223377831 >> [v2:172.21.15.83:6800/2370098752,v1:172.21.15.83:6801/2370098752] conn(0x7f4280038cb0 0x7f428003b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:43.385 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 -- 172.21.15.83:0/3223377831 >> 172.21.15.83:0/3223377831 conn(0x7f4294101aa0 msgr2=0x7f4294194340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:43.385 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 -- 172.21.15.83:0/3223377831 shutdown_connections 2024-03-11T19:29:43.385 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:42.808+0000 7f429bd27700 1 -- 172.21.15.83:0/3223377831 wait complete. 2024-03-11T19:29:43.385 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 5 is available 2024-03-11T19:29:43.385 INFO:teuthology.orchestra.run.smithi083.stdout:Setting orchestrator backend to cephadm... 2024-03-11T19:29:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:43 smithi083 ceph-mon[31864]: [11/Mar/2024:19:29:42] ENGINE Bus STARTING 2024-03-11T19:29:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:43 smithi083 ceph-mon[31864]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-11T19:29:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:43 smithi083 ceph-mon[31864]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-11T19:29:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:43 smithi083 ceph-mon[31864]: [11/Mar/2024:19:29:42] ENGINE Serving on https://172.21.15.83:7150 2024-03-11T19:29:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:43 smithi083 ceph-mon[31864]: [11/Mar/2024:19:29:42] ENGINE Serving on http://172.21.15.83:8765 2024-03-11T19:29:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:43 smithi083 ceph-mon[31864]: [11/Mar/2024:19:29:42] ENGINE Bus STARTED 2024-03-11T19:29:44.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:43 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:29:44.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.778+0000 7fe9f74ea700 1 Processor -- start 2024-03-11T19:29:44.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.778+0000 7fe9f74ea700 1 -- start start 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.779+0000 7fe9f74ea700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.779+0000 7fe9f74ea700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f0106ef0 con 0x7fe9f0106500 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.779+0000 7fe9f5286700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.779+0000 7fe9f5286700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53174/0 (socket says 172.21.15.83:53174) 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.779+0000 7fe9f5286700 1 -- 172.21.15.83:0/2561446141 learned_addr learned my addr 172.21.15.83:0/2561446141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.779+0000 7fe9f5286700 1 -- 172.21.15.83:0/2561446141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0107700 con 0x7fe9f0106500 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.779+0000 7fe9f5286700 1 --2- 172.21.15.83:0/2561446141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0106920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009e80 tx=0x7fe9e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55ebe2c68b78c30 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.779+0000 7fe9effff700 1 -- 172.21.15.83:0/2561446141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9e000e070 con 0x7fe9f0106500 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.779+0000 7fe9effff700 1 -- 172.21.15.83:0/2561446141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9e000b8c0 con 0x7fe9f0106500 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.779+0000 7fe9effff700 1 -- 172.21.15.83:0/2561446141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9e0004550 con 0x7fe9f0106500 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.780+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2561446141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 msgr2=0x7fe9f0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.780+0000 7fe9f74ea700 1 --2- 172.21.15.83:0/2561446141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0106920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009e80 tx=0x7fe9e000b330 comp rx=0 tx=0).stop 2024-03-11T19:29:44.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.780+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2561446141 shutdown_connections 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.780+0000 7fe9f74ea700 1 --2- 172.21.15.83:0/2561446141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0106920 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.780+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2561446141 >> 172.21.15.83:0/2561446141 conn(0x7fe9f0101ac0 msgr2=0x7fe9f0103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.780+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2561446141 shutdown_connections 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.780+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2561446141 wait complete. 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9f74ea700 1 Processor -- start 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9f74ea700 1 -- start start 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9f74ea700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9f74ea700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f0199670 con 0x7fe9f0106500 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9f5286700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9f5286700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53186/0 (socket says 172.21.15.83:53186) 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9f5286700 1 -- 172.21.15.83:0/2361107363 learned_addr learned my addr 172.21.15.83:0/2361107363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9f5286700 1 -- 172.21.15.83:0/2361107363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e0009b30 con 0x7fe9f0106500 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9f5286700 1 --2- 172.21.15.83:0/2361107363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0199130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000b7b0 tx=0x7fe9e0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:44.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9ee7fc700 1 -- 172.21.15.83:0/2361107363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9e000e040 con 0x7fe9f0106500 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2361107363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f0199870 con 0x7fe9f0106500 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9ee7fc700 1 -- 172.21.15.83:0/2361107363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9e000bd70 con 0x7fe9f0106500 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.781+0000 7fe9ee7fc700 1 -- 172.21.15.83:0/2361107363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9e001c770 con 0x7fe9f0106500 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.782+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2361107363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f0199cf0 con 0x7fe9f0106500 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.782+0000 7fe9ee7fc700 1 -- 172.21.15.83:0/2361107363 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) v1 ==== 45663+0+0 (secure 0 0 0) 0x7fe9e001c970 con 0x7fe9f0106500 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.782+0000 7fe9ee7fc700 1 --2- 172.21.15.83:0/2361107363 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe9d803ad50 0x7fe9d803d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.783+0000 7fe9ee7fc700 1 -- 172.21.15.83:0/2361107363 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe9e004caa0 con 0x7fe9f0106500 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.782+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2361107363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f00632a0 con 0x7fe9f0106500 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.783+0000 7fe9f4a85700 1 --2- 172.21.15.83:0/2361107363 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe9d803ad50 0x7fe9d803d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.783+0000 7fe9f4a85700 1 --2- 172.21.15.83:0/2361107363 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe9d803ad50 0x7fe9d803d210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4006fd0 tx=0x7fe9e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.788+0000 7fe9ee7fc700 1 -- 172.21.15.83:0/2361107363 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe9e0017070 con 0x7fe9f0106500 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.851+0000 7fe9ee7fc700 1 -- 172.21.15.83:0/2361107363 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fe9e001b580 con 0x7fe9f0106500 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:43.998+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2361107363 --> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9f00620b0 con 0x7fe9d803ad50 2024-03-11T19:29:44.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:44.003+0000 7fe9ee7fc700 1 -- 172.21.15.83:0/2361107363 <== mgr.14120 v2:172.21.15.83:6800/4058944299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe9f00620b0 con 0x7fe9d803ad50 2024-03-11T19:29:44.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:44.006+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2361107363 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe9d803ad50 msgr2=0x7fe9d803d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:44.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:44.006+0000 7fe9f74ea700 1 --2- 172.21.15.83:0/2361107363 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe9d803ad50 0x7fe9d803d210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4006fd0 tx=0x7fe9e4006e40 comp rx=0 tx=0).stop 2024-03-11T19:29:44.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:44.006+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2361107363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 msgr2=0x7fe9f0199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:44.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:44.006+0000 7fe9f74ea700 1 --2- 172.21.15.83:0/2361107363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0199130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000b7b0 tx=0x7fe9e0003d00 comp rx=0 tx=0).stop 2024-03-11T19:29:44.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:44.006+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2361107363 shutdown_connections 2024-03-11T19:29:44.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:44.006+0000 7fe9f74ea700 1 --2- 172.21.15.83:0/2361107363 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe9d803ad50 0x7fe9d803d210 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:44.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:44.006+0000 7fe9f74ea700 1 --2- 172.21.15.83:0/2361107363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0106500 0x7fe9f0199130 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:44.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:44.006+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2361107363 >> 172.21.15.83:0/2361107363 conn(0x7fe9f0101ac0 msgr2=0x7fe9f018f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:44.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:44.006+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2361107363 shutdown_connections 2024-03-11T19:29:44.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:44.006+0000 7fe9f74ea700 1 -- 172.21.15.83:0/2361107363 wait complete. 2024-03-11T19:29:45.112 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:44 smithi083 ceph-mon[31864]: mgrmap e8: x(active, since 2s) 2024-03-11T19:29:45.112 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:44 smithi083 ceph-mon[31864]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:29:45.112 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:44 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:45.112 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:44 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:29:46.016 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-11T19:29:46.016 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.213+0000 7f8fa846a700 1 Processor -- start 2024-03-11T19:29:46.016 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.213+0000 7f8fa846a700 1 -- start start 2024-03-11T19:29:46.016 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.213+0000 7f8fa846a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa0105d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:46.016 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.213+0000 7f8fa846a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa01062d0 con 0x7f8fa01058e0 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.213+0000 7f8fa6206700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa0105d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.213+0000 7f8fa6206700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa0105d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53192/0 (socket says 172.21.15.83:53192) 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.213+0000 7f8fa6206700 1 -- 172.21.15.83:0/471253753 learned_addr learned my addr 172.21.15.83:0/471253753 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.214+0000 7f8fa6206700 1 -- 172.21.15.83:0/471253753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa0106410 con 0x7f8fa01058e0 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.214+0000 7f8fa6206700 1 --2- 172.21.15.83:0/471253753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa0105d00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0098a0 tx=0x7f8f9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a10a7db6eb937c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.214+0000 7f8fa5204700 1 -- 172.21.15.83:0/471253753 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f9c004160 con 0x7f8fa01058e0 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.214+0000 7f8fa5204700 1 -- 172.21.15.83:0/471253753 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f9c0042c0 con 0x7f8fa01058e0 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.214+0000 7f8fa5204700 1 -- 172.21.15.83:0/471253753 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f9c011ac0 con 0x7f8fa01058e0 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.214+0000 7f8fa846a700 1 -- 172.21.15.83:0/471253753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 msgr2=0x7f8fa0105d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.214+0000 7f8fa846a700 1 --2- 172.21.15.83:0/471253753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa0105d00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0098a0 tx=0x7f8f9c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.215+0000 7f8fa846a700 1 -- 172.21.15.83:0/471253753 shutdown_connections 2024-03-11T19:29:46.017 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.215+0000 7f8fa846a700 1 --2- 172.21.15.83:0/471253753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa0105d00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.215+0000 7f8fa846a700 1 -- 172.21.15.83:0/471253753 >> 172.21.15.83:0/471253753 conn(0x7f8fa01012c0 msgr2=0x7f8fa01036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.215+0000 7f8fa846a700 1 -- 172.21.15.83:0/471253753 shutdown_connections 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.215+0000 7f8fa846a700 1 -- 172.21.15.83:0/471253753 wait complete. 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.215+0000 7f8fa846a700 1 Processor -- start 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.215+0000 7f8fa846a700 1 -- start start 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.215+0000 7f8fa846a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.215+0000 7f8fa846a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa019b970 con 0x7f8fa01058e0 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.216+0000 7f8fa6206700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.216+0000 7f8fa6206700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53202/0 (socket says 172.21.15.83:53202) 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.216+0000 7f8fa6206700 1 -- 172.21.15.83:0/248202481 learned_addr learned my addr 172.21.15.83:0/248202481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.216+0000 7f8fa6206700 1 -- 172.21.15.83:0/248202481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c009550 con 0x7f8fa01058e0 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.216+0000 7f8fa6206700 1 --2- 172.21.15.83:0/248202481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa019b430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c011780 tx=0x7f8f9c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:46.018 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.216+0000 7f8f977fe700 1 -- 172.21.15.83:0/248202481 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f9c003af0 con 0x7f8fa01058e0 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.216+0000 7f8f977fe700 1 -- 172.21.15.83:0/248202481 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f9c003c50 con 0x7f8fa01058e0 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.216+0000 7f8fa846a700 1 -- 172.21.15.83:0/248202481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa019bb70 con 0x7f8fa01058e0 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.216+0000 7f8f977fe700 1 -- 172.21.15.83:0/248202481 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f9c0197e0 con 0x7f8fa01058e0 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.216+0000 7f8fa846a700 1 -- 172.21.15.83:0/248202481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa019c010 con 0x7f8fa01058e0 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.217+0000 7f8f977fe700 1 -- 172.21.15.83:0/248202481 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f8f9c019a50 con 0x7f8fa01058e0 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.217+0000 7f8f977fe700 1 --2- 172.21.15.83:0/248202481 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f8f8c038d00 0x7f8f8c03b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.217+0000 7f8f977fe700 1 -- 172.21.15.83:0/248202481 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8f9c04d120 con 0x7f8fa01058e0 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.217+0000 7f8fa846a700 1 -- 172.21.15.83:0/248202481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa00632a0 con 0x7f8fa01058e0 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.217+0000 7f8fa5a05700 1 --2- 172.21.15.83:0/248202481 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f8f8c038d00 0x7f8f8c03b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.218+0000 7f8fa5a05700 1 --2- 172.21.15.83:0/248202481 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f8f8c038d00 0x7f8f8c03b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f90006fd0 tx=0x7f8f90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.222+0000 7f8f977fe700 1 -- 172.21.15.83:0/248202481 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f9c0188f0 con 0x7f8fa01058e0 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.417+0000 7f8fa846a700 1 -- 172.21.15.83:0/248202481 --> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fa0102540 con 0x7f8f8c038d00 2024-03-11T19:29:46.019 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.418+0000 7f8f977fe700 1 -- 172.21.15.83:0/248202481 <== mgr.14120 v2:172.21.15.83:6800/4058944299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f8fa0102540 con 0x7f8f8c038d00 2024-03-11T19:29:46.020 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.420+0000 7f8fa846a700 1 -- 172.21.15.83:0/248202481 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f8f8c038d00 msgr2=0x7f8f8c03b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:46.020 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.420+0000 7f8fa846a700 1 --2- 172.21.15.83:0/248202481 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f8f8c038d00 0x7f8f8c03b1c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8f90006fd0 tx=0x7f8f90006e40 comp rx=0 tx=0).stop 2024-03-11T19:29:46.020 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.420+0000 7f8fa846a700 1 -- 172.21.15.83:0/248202481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 msgr2=0x7f8fa019b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:46.020 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.420+0000 7f8fa846a700 1 --2- 172.21.15.83:0/248202481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa019b430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c011780 tx=0x7f8f9c011b40 comp rx=0 tx=0).stop 2024-03-11T19:29:46.020 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.420+0000 7f8fa846a700 1 -- 172.21.15.83:0/248202481 shutdown_connections 2024-03-11T19:29:46.020 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.420+0000 7f8fa846a700 1 --2- 172.21.15.83:0/248202481 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f8f8c038d00 0x7f8f8c03b1c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:46.020 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.420+0000 7f8fa846a700 1 --2- 172.21.15.83:0/248202481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa01058e0 0x7f8fa019b430 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:46.020 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.420+0000 7f8fa846a700 1 -- 172.21.15.83:0/248202481 >> 172.21.15.83:0/248202481 conn(0x7f8fa01012c0 msgr2=0x7f8fa0101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:46.020 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.420+0000 7f8fa846a700 1 -- 172.21.15.83:0/248202481 shutdown_connections 2024-03-11T19:29:46.020 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:45.420+0000 7f8fa846a700 1 -- 172.21.15.83:0/248202481 wait complete. 2024-03-11T19:29:46.020 INFO:teuthology.orchestra.run.smithi083.stdout:Generating ssh key... 2024-03-11T19:29:46.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-mon[31864]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:29:47.198 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: Generating public/private rsa key pair. 2024-03-11T19:29:47.198 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: Your identification has been saved in /tmp/tmpw6z0_brb/key. 2024-03-11T19:29:47.198 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: Your public key has been saved in /tmp/tmpw6z0_brb/key.pub. 2024-03-11T19:29:47.198 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: The key fingerprint is: 2024-03-11T19:29:47.198 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: SHA256:Ev0XdY29sS5424qrKiZmWdY1B1QiJbaIS/Wwsk4tkv4 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:29:47.198 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: The key's randomart image is: 2024-03-11T19:29:47.198 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: +---[RSA 3072]----+ 2024-03-11T19:29:47.198 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: | o +o+.. .oo| 2024-03-11T19:29:47.198 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: | o *.+.. ..oo| 2024-03-11T19:29:47.199 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: | + o.o. . . +| 2024-03-11T19:29:47.199 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: | o = . + . . o | 2024-03-11T19:29:47.199 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: | o = .o S + o . | 2024-03-11T19:29:47.199 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: |. + .o o o o . | 2024-03-11T19:29:47.199 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: | . .+ . + | 2024-03-11T19:29:47.199 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: | .= o .. . | 2024-03-11T19:29:47.199 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: | oEo .....o... | 2024-03-11T19:29:47.199 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:29:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: +----[SHA256]-----+ 2024-03-11T19:29:47.481 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.422+0000 7f9aea019700 1 Processor -- start 2024-03-11T19:29:47.481 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.422+0000 7f9aea019700 1 -- start start 2024-03-11T19:29:47.481 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.422+0000 7f9aea019700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae41062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.422+0000 7f9aea019700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae407ae80 con 0x7f9ae4105e90 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.422+0000 7f9ae37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae41062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.422+0000 7f9ae37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae41062b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53210/0 (socket says 172.21.15.83:53210) 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.422+0000 7f9ae37fe700 1 -- 172.21.15.83:0/2566481137 learned_addr learned my addr 172.21.15.83:0/2566481137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.423+0000 7f9ae37fe700 1 -- 172.21.15.83:0/2566481137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae41067f0 con 0x7f9ae4105e90 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.423+0000 7f9ae37fe700 1 --2- 172.21.15.83:0/2566481137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae41062b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009e80 tx=0x7f9ad400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549b2fabde42291a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.423+0000 7f9ae27fc700 1 -- 172.21.15.83:0/2566481137 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ad400e070 con 0x7f9ae4105e90 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.423+0000 7f9ae27fc700 1 -- 172.21.15.83:0/2566481137 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ad400b8c0 con 0x7f9ae4105e90 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.423+0000 7f9ae27fc700 1 -- 172.21.15.83:0/2566481137 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ad4004550 con 0x7f9ae4105e90 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.424+0000 7f9aea019700 1 -- 172.21.15.83:0/2566481137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 msgr2=0x7f9ae41062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.424+0000 7f9aea019700 1 --2- 172.21.15.83:0/2566481137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae41062b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009e80 tx=0x7f9ad400b330 comp rx=0 tx=0).stop 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.424+0000 7f9aea019700 1 -- 172.21.15.83:0/2566481137 shutdown_connections 2024-03-11T19:29:47.482 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.424+0000 7f9aea019700 1 --2- 172.21.15.83:0/2566481137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae41062b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.424+0000 7f9aea019700 1 -- 172.21.15.83:0/2566481137 >> 172.21.15.83:0/2566481137 conn(0x7f9ae4101b30 msgr2=0x7f9ae4103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.424+0000 7f9aea019700 1 -- 172.21.15.83:0/2566481137 shutdown_connections 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.424+0000 7f9aea019700 1 -- 172.21.15.83:0/2566481137 wait complete. 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.424+0000 7f9aea019700 1 Processor -- start 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.424+0000 7f9aea019700 1 -- start start 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9aea019700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9aea019700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae419d9a0 con 0x7f9ae4105e90 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9ae37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9ae37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53224/0 (socket says 172.21.15.83:53224) 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9ae37fe700 1 -- 172.21.15.83:0/1925709318 learned_addr learned my addr 172.21.15.83:0/1925709318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9ae37fe700 1 -- 172.21.15.83:0/1925709318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009b30 con 0x7f9ae4105e90 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9ae37fe700 1 --2- 172.21.15.83:0/1925709318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae419d460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40042e0 tx=0x7f9ad4003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:47.483 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9ae0ff9700 1 -- 172.21.15.83:0/1925709318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ad400e040 con 0x7f9ae4105e90 2024-03-11T19:29:47.484 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9ae0ff9700 1 -- 172.21.15.83:0/1925709318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ad400bd70 con 0x7f9ae4105e90 2024-03-11T19:29:47.484 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9aea019700 1 -- 172.21.15.83:0/1925709318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae419dba0 con 0x7f9ae4105e90 2024-03-11T19:29:47.484 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9ae0ff9700 1 -- 172.21.15.83:0/1925709318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ad401c7f0 con 0x7f9ae4105e90 2024-03-11T19:29:47.485 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.425+0000 7f9aea019700 1 -- 172.21.15.83:0/1925709318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae419e020 con 0x7f9ae4105e90 2024-03-11T19:29:47.485 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.426+0000 7f9ae0ff9700 1 -- 172.21.15.83:0/1925709318 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f9ad401ca60 con 0x7f9ae4105e90 2024-03-11T19:29:47.486 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.426+0000 7f9aea019700 1 -- 172.21.15.83:0/1925709318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae4197170 con 0x7f9ae4105e90 2024-03-11T19:29:47.486 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.426+0000 7f9ae0ff9700 1 --2- 172.21.15.83:0/1925709318 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f9acc038cb0 0x7f9acc03b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:47.486 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.427+0000 7f9ae0ff9700 1 -- 172.21.15.83:0/1925709318 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9ad404cd90 con 0x7f9ae4105e90 2024-03-11T19:29:47.486 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.427+0000 7f9ae2ffd700 1 --2- 172.21.15.83:0/1925709318 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f9acc038cb0 0x7f9acc03b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:47.486 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.427+0000 7f9ae2ffd700 1 --2- 172.21.15.83:0/1925709318 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f9acc038cb0 0x7f9acc03b170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006fd0 tx=0x7f9ad8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:47.486 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.431+0000 7f9ae0ff9700 1 -- 172.21.15.83:0/1925709318 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ad4024ca0 con 0x7f9ae4105e90 2024-03-11T19:29:47.486 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.622+0000 7f9aea019700 1 -- 172.21.15.83:0/1925709318 --> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ae402d960 con 0x7f9acc038cb0 2024-03-11T19:29:47.486 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.899+0000 7f9ae0ff9700 1 -- 172.21.15.83:0/1925709318 <== mgr.14120 v2:172.21.15.83:6800/4058944299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9ae402d960 con 0x7f9acc038cb0 2024-03-11T19:29:47.486 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.901+0000 7f9aea019700 1 -- 172.21.15.83:0/1925709318 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f9acc038cb0 msgr2=0x7f9acc03b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:47.486 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.901+0000 7f9aea019700 1 --2- 172.21.15.83:0/1925709318 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f9acc038cb0 0x7f9acc03b170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006fd0 tx=0x7f9ad8006e40 comp rx=0 tx=0).stop 2024-03-11T19:29:47.486 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.901+0000 7f9aea019700 1 -- 172.21.15.83:0/1925709318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 msgr2=0x7f9ae419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:47.487 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.901+0000 7f9aea019700 1 --2- 172.21.15.83:0/1925709318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae419d460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40042e0 tx=0x7f9ad4003d00 comp rx=0 tx=0).stop 2024-03-11T19:29:47.487 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.901+0000 7f9aea019700 1 -- 172.21.15.83:0/1925709318 shutdown_connections 2024-03-11T19:29:47.487 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.901+0000 7f9aea019700 1 --2- 172.21.15.83:0/1925709318 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f9acc038cb0 0x7f9acc03b170 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:47.487 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.901+0000 7f9aea019700 1 --2- 172.21.15.83:0/1925709318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ae4105e90 0x7f9ae419d460 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:47.487 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.901+0000 7f9aea019700 1 -- 172.21.15.83:0/1925709318 >> 172.21.15.83:0/1925709318 conn(0x7f9ae4101b30 msgr2=0x7f9ae41942b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:47.487 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.902+0000 7f9aea019700 1 -- 172.21.15.83:0/1925709318 shutdown_connections 2024-03-11T19:29:47.487 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:46.902+0000 7f9aea019700 1 -- 172.21.15.83:0/1925709318 wait complete. 2024-03-11T19:29:48.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:47 smithi083 ceph-mon[31864]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:29:48.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:47 smithi083 ceph-mon[31864]: Generating ssh key... 2024-03-11T19:29:48.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:47 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:48.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:47 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:48.655 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDalrRLnJ6uujR52Yhg//lOQLVxM5vMVfa4TgVGNTbPLWijvX11jW9SNDfiZZSzUwVByQ7csArIOOxA5298Wl4P1Mq+IND5LkhG7hH/UqBKzkZ+12n61+SblAoKOEC/L+GSv+gOYKDSkwCYJoR8cq3mu67dx6e7nEX/0rBY6uz0fPAqi8hNuRjylVe5XP46jzyLeAiiTh2Os5D6pGt6flDK7TceawNOR+fR/h61gb8B6DZ0vdMczaqv3UF43vWiERjmBAjVXUzSFzEgEC14/hQzn0w9z4BwtaPNZ8WKR6Ony/vOfEhwbz1JF1MOIZDGE0vbc2idpQ3PWWABGK8HwouHHt5RGrfbIa8NC1gqRYzFfEDNf7XbICzXM7BAv5wx8/gxKn+vEUC0fUcdT2yMRWqpkLtjPbYk6g+0rLIQSRtxRDIom3qhswj/QvDwXJ1Pe+yYNJx1R0HHBXvacIkVHuI7vJG36tNMPjEEyKTUoItZ7GIcLMOrDA5V8+T4J0ps+B8= ceph-839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:29:48.656 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.853+0000 7fa7b75ef700 1 Processor -- start 2024-03-11T19:29:48.656 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.853+0000 7fa7b75ef700 1 -- start start 2024-03-11T19:29:48.656 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.854+0000 7fa7b75ef700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b010a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:48.656 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.854+0000 7fa7b75ef700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b00754f0 con 0x7fa7b01080b0 2024-03-11T19:29:48.656 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.854+0000 7fa7b538b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b010a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:48.656 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.854+0000 7fa7b538b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b010a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53228/0 (socket says 172.21.15.83:53228) 2024-03-11T19:29:48.656 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.854+0000 7fa7b538b700 1 -- 172.21.15.83:0/1617971238 learned_addr learned my addr 172.21.15.83:0/1617971238 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:48.656 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.854+0000 7fa7b538b700 1 -- 172.21.15.83:0/1617971238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b010a9e0 con 0x7fa7b01080b0 2024-03-11T19:29:48.656 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.854+0000 7fa7b538b700 1 --2- 172.21.15.83:0/1617971238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b010a4a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00098a0 tx=0x7fa7a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c6eaf495be73f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.854+0000 7fa7a7fff700 1 -- 172.21.15.83:0/1617971238 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7a0004160 con 0x7fa7b01080b0 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.854+0000 7fa7a7fff700 1 -- 172.21.15.83:0/1617971238 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7a00042c0 con 0x7fa7b01080b0 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.854+0000 7fa7a7fff700 1 -- 172.21.15.83:0/1617971238 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7a0011ac0 con 0x7fa7b01080b0 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.855+0000 7fa7b75ef700 1 -- 172.21.15.83:0/1617971238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 msgr2=0x7fa7b010a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.855+0000 7fa7b75ef700 1 --2- 172.21.15.83:0/1617971238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b010a4a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00098a0 tx=0x7fa7a0009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.855+0000 7fa7b75ef700 1 -- 172.21.15.83:0/1617971238 shutdown_connections 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.855+0000 7fa7b75ef700 1 --2- 172.21.15.83:0/1617971238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b010a4a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.855+0000 7fa7b75ef700 1 -- 172.21.15.83:0/1617971238 >> 172.21.15.83:0/1617971238 conn(0x7fa7b0101ac0 msgr2=0x7fa7b0103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.855+0000 7fa7b75ef700 1 -- 172.21.15.83:0/1617971238 shutdown_connections 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.855+0000 7fa7b75ef700 1 -- 172.21.15.83:0/1617971238 wait complete. 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7b75ef700 1 Processor -- start 2024-03-11T19:29:48.657 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7b75ef700 1 -- start start 2024-03-11T19:29:48.658 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7b75ef700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:48.658 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7b75ef700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b019b990 con 0x7fa7b01080b0 2024-03-11T19:29:48.658 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7b538b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:48.658 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7b538b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53238/0 (socket says 172.21.15.83:53238) 2024-03-11T19:29:48.658 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7b538b700 1 -- 172.21.15.83:0/696795785 learned_addr learned my addr 172.21.15.83:0/696795785 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:48.658 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7b538b700 1 -- 172.21.15.83:0/696795785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a0009550 con 0x7fa7b01080b0 2024-03-11T19:29:48.658 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7b538b700 1 --2- 172.21.15.83:0/696795785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b019b450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0011a70 tx=0x7fa7a0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:48.658 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7a67fc700 1 -- 172.21.15.83:0/696795785 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7a00039c0 con 0x7fa7b01080b0 2024-03-11T19:29:48.658 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7a67fc700 1 -- 172.21.15.83:0/696795785 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7a0003b20 con 0x7fa7b01080b0 2024-03-11T19:29:48.658 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.856+0000 7fa7b75ef700 1 -- 172.21.15.83:0/696795785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b019bb90 con 0x7fa7b01080b0 2024-03-11T19:29:48.658 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.857+0000 7fa7a67fc700 1 -- 172.21.15.83:0/696795785 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7a0024610 con 0x7fa7b01080b0 2024-03-11T19:29:48.659 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.857+0000 7fa7b75ef700 1 -- 172.21.15.83:0/696795785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b019c030 con 0x7fa7b01080b0 2024-03-11T19:29:48.659 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.857+0000 7fa7a67fc700 1 -- 172.21.15.83:0/696795785 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fa7a001aa10 con 0x7fa7b01080b0 2024-03-11T19:29:48.659 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.857+0000 7fa7b75ef700 1 -- 172.21.15.83:0/696795785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b010b7e0 con 0x7fa7b01080b0 2024-03-11T19:29:48.659 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.857+0000 7fa7a67fc700 1 --2- 172.21.15.83:0/696795785 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fa79c038cb0 0x7fa79c03b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:48.659 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.858+0000 7fa7a67fc700 1 -- 172.21.15.83:0/696795785 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa7a004ccc0 con 0x7fa7b01080b0 2024-03-11T19:29:48.659 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.858+0000 7fa7b4b8a700 1 --2- 172.21.15.83:0/696795785 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fa79c038cb0 0x7fa79c03b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:48.659 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.858+0000 7fa7b4b8a700 1 --2- 172.21.15.83:0/696795785 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fa79c038cb0 0x7fa79c03b170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac006fd0 tx=0x7fa7ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:48.659 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:47.862+0000 7fa7a67fc700 1 -- 172.21.15.83:0/696795785 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7a0018dc0 con 0x7fa7b01080b0 2024-03-11T19:29:48.659 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.055+0000 7fa7b75ef700 1 -- 172.21.15.83:0/696795785 --> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7b0102d00 con 0x7fa79c038cb0 2024-03-11T19:29:48.659 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.056+0000 7fa7a67fc700 1 -- 172.21.15.83:0/696795785 <== mgr.14120 v2:172.21.15.83:6800/4058944299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fa7b0102d00 con 0x7fa79c038cb0 2024-03-11T19:29:48.659 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.058+0000 7fa7b75ef700 1 -- 172.21.15.83:0/696795785 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fa79c038cb0 msgr2=0x7fa79c03b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:48.660 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.058+0000 7fa7b75ef700 1 --2- 172.21.15.83:0/696795785 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fa79c038cb0 0x7fa79c03b170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac006fd0 tx=0x7fa7ac006e40 comp rx=0 tx=0).stop 2024-03-11T19:29:48.660 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.058+0000 7fa7b75ef700 1 -- 172.21.15.83:0/696795785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 msgr2=0x7fa7b019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:48.660 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.058+0000 7fa7b75ef700 1 --2- 172.21.15.83:0/696795785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b019b450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0011a70 tx=0x7fa7a0011b50 comp rx=0 tx=0).stop 2024-03-11T19:29:48.660 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.058+0000 7fa7b75ef700 1 -- 172.21.15.83:0/696795785 shutdown_connections 2024-03-11T19:29:48.660 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.058+0000 7fa7b75ef700 1 --2- 172.21.15.83:0/696795785 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fa79c038cb0 0x7fa79c03b170 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:48.660 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.058+0000 7fa7b75ef700 1 --2- 172.21.15.83:0/696795785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7b01080b0 0x7fa7b019b450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:48.660 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.058+0000 7fa7b75ef700 1 -- 172.21.15.83:0/696795785 >> 172.21.15.83:0/696795785 conn(0x7fa7b0101ac0 msgr2=0x7fa7b01025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:48.660 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.058+0000 7fa7b75ef700 1 -- 172.21.15.83:0/696795785 shutdown_connections 2024-03-11T19:29:48.660 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:48.058+0000 7fa7b75ef700 1 -- 172.21.15.83:0/696795785 wait complete. 2024-03-11T19:29:48.660 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-11T19:29:48.660 INFO:teuthology.orchestra.run.smithi083.stdout:Adding key to root@localhost authorized_keys... 2024-03-11T19:29:48.661 INFO:teuthology.orchestra.run.smithi083.stdout:Adding host smithi083... 2024-03-11T19:29:49.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:49 smithi083 ceph-mon[31864]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:29:50.264 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:50 smithi083 ceph-mon[31864]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:29:50.264 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:50 smithi083 ceph-mon[31864]: Deploying cephadm binary to smithi083 2024-03-11T19:29:54.121 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Added host 'smithi083' with addr '172.21.15.83' 2024-03-11T19:29:54.121 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.039+0000 7f4954c89700 1 Processor -- start 2024-03-11T19:29:54.121 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.039+0000 7f4954c89700 1 -- start start 2024-03-11T19:29:54.121 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.040+0000 7f4954c89700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f4950106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.040+0000 7f4954c89700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495007ae80 con 0x7f4950105e60 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.040+0000 7f494e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f4950106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.040+0000 7f494e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f4950106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53254/0 (socket says 172.21.15.83:53254) 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.040+0000 7f494e59c700 1 -- 172.21.15.83:0/2573979668 learned_addr learned my addr 172.21.15.83:0/2573979668 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.040+0000 7f494e59c700 1 -- 172.21.15.83:0/2573979668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49501067c0 con 0x7f4950105e60 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.040+0000 7f494e59c700 1 --2- 172.21.15.83:0/2573979668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f4950106280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f49440098a0 tx=0x7f4944009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca1447f6caed037 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.041+0000 7f494d59a700 1 -- 172.21.15.83:0/2573979668 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4944004160 con 0x7f4950105e60 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.041+0000 7f494d59a700 1 -- 172.21.15.83:0/2573979668 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49440042c0 con 0x7f4950105e60 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.041+0000 7f494d59a700 1 -- 172.21.15.83:0/2573979668 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4944011ac0 con 0x7f4950105e60 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.041+0000 7f4954c89700 1 -- 172.21.15.83:0/2573979668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 msgr2=0x7f4950106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.041+0000 7f4954c89700 1 --2- 172.21.15.83:0/2573979668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f4950106280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f49440098a0 tx=0x7f4944009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:54.122 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.041+0000 7f4954c89700 1 -- 172.21.15.83:0/2573979668 shutdown_connections 2024-03-11T19:29:54.123 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.041+0000 7f4954c89700 1 --2- 172.21.15.83:0/2573979668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f4950106280 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:54.123 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.041+0000 7f4954c89700 1 -- 172.21.15.83:0/2573979668 >> 172.21.15.83:0/2573979668 conn(0x7f4950101aa0 msgr2=0x7f4950103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:54.123 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.041+0000 7f4954c89700 1 -- 172.21.15.83:0/2573979668 shutdown_connections 2024-03-11T19:29:54.123 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.041+0000 7f4954c89700 1 -- 172.21.15.83:0/2573979668 wait complete. 2024-03-11T19:29:54.123 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.042+0000 7f4954c89700 1 Processor -- start 2024-03-11T19:29:54.123 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.042+0000 7f4954c89700 1 -- start start 2024-03-11T19:29:54.123 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.042+0000 7f4954c89700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f495019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:54.125 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.042+0000 7f4954c89700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495019dab0 con 0x7f4950105e60 2024-03-11T19:29:54.125 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.042+0000 7f494e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f495019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:54.125 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.042+0000 7f494e59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f495019d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53264/0 (socket says 172.21.15.83:53264) 2024-03-11T19:29:54.125 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.042+0000 7f494e59c700 1 -- 172.21.15.83:0/3975838379 learned_addr learned my addr 172.21.15.83:0/3975838379 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:54.125 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.043+0000 7f494e59c700 1 -- 172.21.15.83:0/3975838379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4944009550 con 0x7f4950105e60 2024-03-11T19:29:54.125 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.043+0000 7f494e59c700 1 --2- 172.21.15.83:0/3975838379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f495019d570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4944009130 tx=0x7f4944011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.043+0000 7f493f7fe700 1 -- 172.21.15.83:0/3975838379 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4944003aa0 con 0x7f4950105e60 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.043+0000 7f493f7fe700 1 -- 172.21.15.83:0/3975838379 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4944003c00 con 0x7f4950105e60 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.043+0000 7f4954c89700 1 -- 172.21.15.83:0/3975838379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495019dcb0 con 0x7f4950105e60 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.044+0000 7f493f7fe700 1 -- 172.21.15.83:0/3975838379 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4944024800 con 0x7f4950105e60 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.044+0000 7f4954c89700 1 -- 172.21.15.83:0/3975838379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495019e150 con 0x7f4950105e60 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.044+0000 7f493f7fe700 1 -- 172.21.15.83:0/3975838379 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f4944024af0 con 0x7f4950105e60 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.044+0000 7f493f7fe700 1 --2- 172.21.15.83:0/3975838379 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f4938038d50 0x7f493803b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.044+0000 7f493f7fe700 1 -- 172.21.15.83:0/3975838379 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f494401a840 con 0x7f4950105e60 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.044+0000 7f494dd9b700 1 --2- 172.21.15.83:0/3975838379 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f4938038d50 0x7f493803b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.045+0000 7f494dd9b700 1 --2- 172.21.15.83:0/3975838379 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f4938038d50 0x7f493803b210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4940006fd0 tx=0x7f4940006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.045+0000 7f4954c89700 1 -- 172.21.15.83:0/3975838379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4950197180 con 0x7f4950105e60 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.050+0000 7f493f7fe700 1 -- 172.21.15.83:0/3975838379 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4944023e90 con 0x7f4950105e60 2024-03-11T19:29:54.126 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:49.256+0000 7f4954c89700 1 -- 172.21.15.83:0/3975838379 --> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}) v1 -- 0x7f495002d910 con 0x7f4938038d50 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:53.454+0000 7f493f7fe700 1 -- 172.21.15.83:0/3975838379 <== mgr.14120 v2:172.21.15.83:6800/4058944299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f495002d910 con 0x7f4938038d50 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:53.457+0000 7f4954c89700 1 -- 172.21.15.83:0/3975838379 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f4938038d50 msgr2=0x7f493803b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:53.457+0000 7f4954c89700 1 --2- 172.21.15.83:0/3975838379 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f4938038d50 0x7f493803b210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4940006fd0 tx=0x7f4940006e40 comp rx=0 tx=0).stop 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:53.457+0000 7f4954c89700 1 -- 172.21.15.83:0/3975838379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 msgr2=0x7f495019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:53.457+0000 7f4954c89700 1 --2- 172.21.15.83:0/3975838379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f495019d570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4944009130 tx=0x7f4944011b40 comp rx=0 tx=0).stop 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:53.457+0000 7f4954c89700 1 -- 172.21.15.83:0/3975838379 shutdown_connections 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:53.457+0000 7f4954c89700 1 --2- 172.21.15.83:0/3975838379 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f4938038d50 0x7f493803b210 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:53.457+0000 7f4954c89700 1 --2- 172.21.15.83:0/3975838379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4950105e60 0x7f495019d570 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:53.457+0000 7f4954c89700 1 -- 172.21.15.83:0/3975838379 >> 172.21.15.83:0/3975838379 conn(0x7f4950101aa0 msgr2=0x7f49501942c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:53.457+0000 7f4954c89700 1 -- 172.21.15.83:0/3975838379 shutdown_connections 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:53.457+0000 7f4954c89700 1 -- 172.21.15.83:0/3975838379 wait complete. 2024-03-11T19:29:54.127 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mon service... 2024-03-11T19:29:54.776 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:54 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:54.776 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:54 smithi083 ceph-mon[31864]: Added host smithi083 2024-03-11T19:29:54.776 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:54 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:29:55.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-11T19:29:55.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.524+0000 7fd7c8bee700 1 Processor -- start 2024-03-11T19:29:55.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.524+0000 7fd7c8bee700 1 -- start start 2024-03-11T19:29:55.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.524+0000 7fd7c8bee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c4106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:55.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.524+0000 7fd7c8bee700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c4106ef0 con 0x7fd7c4106500 2024-03-11T19:29:55.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.524+0000 7fd7c259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c4106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:55.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.525+0000 7fd7c259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c4106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50564/0 (socket says 172.21.15.83:50564) 2024-03-11T19:29:55.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.525+0000 7fd7c259c700 1 -- 172.21.15.83:0/189672654 learned_addr learned my addr 172.21.15.83:0/189672654 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:55.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.525+0000 7fd7c259c700 1 -- 172.21.15.83:0/189672654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c4107700 con 0x7fd7c4106500 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.525+0000 7fd7c259c700 1 --2- 172.21.15.83:0/189672654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c4106920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40098a0 tx=0x7fd7b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89aa6cd080f47e86 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.525+0000 7fd7c1d9b700 1 -- 172.21.15.83:0/189672654 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7b4004160 con 0x7fd7c4106500 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.525+0000 7fd7c1d9b700 1 -- 172.21.15.83:0/189672654 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7b40042c0 con 0x7fd7c4106500 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.525+0000 7fd7c1d9b700 1 -- 172.21.15.83:0/189672654 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7b4011ac0 con 0x7fd7c4106500 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.526+0000 7fd7c8bee700 1 -- 172.21.15.83:0/189672654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 msgr2=0x7fd7c4106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.526+0000 7fd7c8bee700 1 --2- 172.21.15.83:0/189672654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c4106920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40098a0 tx=0x7fd7b4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.526+0000 7fd7c8bee700 1 -- 172.21.15.83:0/189672654 shutdown_connections 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.526+0000 7fd7c8bee700 1 --2- 172.21.15.83:0/189672654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c4106920 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.526+0000 7fd7c8bee700 1 -- 172.21.15.83:0/189672654 >> 172.21.15.83:0/189672654 conn(0x7fd7c4101ac0 msgr2=0x7fd7c4103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.526+0000 7fd7c8bee700 1 -- 172.21.15.83:0/189672654 shutdown_connections 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.526+0000 7fd7c8bee700 1 -- 172.21.15.83:0/189672654 wait complete. 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.526+0000 7fd7c8bee700 1 Processor -- start 2024-03-11T19:29:55.507 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.526+0000 7fd7c8bee700 1 -- start start 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7c8bee700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c40750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7c8bee700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c4075610 con 0x7fd7c4106500 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7c259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c40750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7c259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c40750d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50574/0 (socket says 172.21.15.83:50574) 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7c259c700 1 -- 172.21.15.83:0/30897719 learned_addr learned my addr 172.21.15.83:0/30897719 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7c259c700 1 -- 172.21.15.83:0/30897719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b4009550 con 0x7fd7c4106500 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7c259c700 1 --2- 172.21.15.83:0/30897719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c40750d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009fb0 tx=0x7fd7b4011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7bb7fe700 1 -- 172.21.15.83:0/30897719 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7b40038b0 con 0x7fd7c4106500 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7c8bee700 1 -- 172.21.15.83:0/30897719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c419e170 con 0x7fd7c4106500 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7bb7fe700 1 -- 172.21.15.83:0/30897719 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7b4003a10 con 0x7fd7c4106500 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7bb7fe700 1 -- 172.21.15.83:0/30897719 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7b40195e0 con 0x7fd7c4106500 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.527+0000 7fd7c8bee700 1 -- 172.21.15.83:0/30897719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c419e610 con 0x7fd7c4106500 2024-03-11T19:29:55.508 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.528+0000 7fd7bb7fe700 1 -- 172.21.15.83:0/30897719 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7fd7b40198a0 con 0x7fd7c4106500 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.528+0000 7fd7c8bee700 1 -- 172.21.15.83:0/30897719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c410a930 con 0x7fd7c4106500 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.528+0000 7fd7bb7fe700 1 --2- 172.21.15.83:0/30897719 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd7b0038c60 0x7fd7b003b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.528+0000 7fd7bb7fe700 1 -- 172.21.15.83:0/30897719 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd7b404cfc0 con 0x7fd7c4106500 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.528+0000 7fd7bbfff700 1 --2- 172.21.15.83:0/30897719 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd7b0038c60 0x7fd7b003b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.529+0000 7fd7bbfff700 1 --2- 172.21.15.83:0/30897719 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd7b0038c60 0x7fd7b003b120 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac006fd0 tx=0x7fd7ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.533+0000 7fd7bb7fe700 1 -- 172.21.15.83:0/30897719 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7b4080d20 con 0x7fd7c4106500 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.772+0000 7fd7c8bee700 1 -- 172.21.15.83:0/30897719 --> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd7c4102cc0 con 0x7fd7b0038c60 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.776+0000 7fd7bb7fe700 1 -- 172.21.15.83:0/30897719 <== mgr.14120 v2:172.21.15.83:6800/4058944299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd7c4102cc0 con 0x7fd7b0038c60 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.778+0000 7fd7c8bee700 1 -- 172.21.15.83:0/30897719 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd7b0038c60 msgr2=0x7fd7b003b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.778+0000 7fd7c8bee700 1 --2- 172.21.15.83:0/30897719 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd7b0038c60 0x7fd7b003b120 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac006fd0 tx=0x7fd7ac006e40 comp rx=0 tx=0).stop 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.778+0000 7fd7c8bee700 1 -- 172.21.15.83:0/30897719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 msgr2=0x7fd7c40750d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:55.509 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.778+0000 7fd7c8bee700 1 --2- 172.21.15.83:0/30897719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c40750d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009fb0 tx=0x7fd7b4011b20 comp rx=0 tx=0).stop 2024-03-11T19:29:55.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.778+0000 7fd7c8bee700 1 -- 172.21.15.83:0/30897719 shutdown_connections 2024-03-11T19:29:55.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.778+0000 7fd7c8bee700 1 --2- 172.21.15.83:0/30897719 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd7b0038c60 0x7fd7b003b120 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:55.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.778+0000 7fd7c8bee700 1 --2- 172.21.15.83:0/30897719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7c4106500 0x7fd7c40750d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:55.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.778+0000 7fd7c8bee700 1 -- 172.21.15.83:0/30897719 >> 172.21.15.83:0/30897719 conn(0x7fd7c4101ac0 msgr2=0x7fd7c41025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:55.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.779+0000 7fd7c8bee700 1 -- 172.21.15.83:0/30897719 shutdown_connections 2024-03-11T19:29:55.510 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:54.779+0000 7fd7c8bee700 1 -- 172.21.15.83:0/30897719 wait complete. 2024-03-11T19:29:55.510 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mgr service... 2024-03-11T19:29:56.067 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:55 smithi083 ceph-mon[31864]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:29:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:55 smithi083 ceph-mon[31864]: Saving service mon spec with placement count:5 2024-03-11T19:29:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:55 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:57.444 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:57 smithi083 ceph-mon[31864]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:29:57.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:57 smithi083 ceph-mon[31864]: Saving service mgr spec with placement count:2 2024-03-11T19:29:57.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:57 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:57.586 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.900+0000 7f79efee6700 1 Processor -- start 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.900+0000 7f79efee6700 1 -- start start 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.900+0000 7f79efee6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.900+0000 7f79efee6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e81073a0 con 0x7f79e81069b0 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.901+0000 7f79edc82700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.901+0000 7f79edc82700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8106dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50580/0 (socket says 172.21.15.83:50580) 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.901+0000 7f79edc82700 1 -- 172.21.15.83:0/4203415935 learned_addr learned my addr 172.21.15.83:0/4203415935 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.901+0000 7f79edc82700 1 -- 172.21.15.83:0/4203415935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e81076d0 con 0x7f79e81069b0 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.901+0000 7f79edc82700 1 --2- 172.21.15.83:0/4203415935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8106dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f79e40098a0 tx=0x7f79e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd76a2eb79ad70b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.901+0000 7f79ecc80700 1 -- 172.21.15.83:0/4203415935 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79e4004160 con 0x7f79e81069b0 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.901+0000 7f79ecc80700 1 -- 172.21.15.83:0/4203415935 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79e40042c0 con 0x7f79e81069b0 2024-03-11T19:29:57.587 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.901+0000 7f79ecc80700 1 -- 172.21.15.83:0/4203415935 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79e4011ac0 con 0x7f79e81069b0 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.902+0000 7f79efee6700 1 -- 172.21.15.83:0/4203415935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 msgr2=0x7f79e8106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.902+0000 7f79efee6700 1 --2- 172.21.15.83:0/4203415935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8106dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f79e40098a0 tx=0x7f79e4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.902+0000 7f79efee6700 1 -- 172.21.15.83:0/4203415935 shutdown_connections 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.902+0000 7f79efee6700 1 --2- 172.21.15.83:0/4203415935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8106dd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.902+0000 7f79efee6700 1 -- 172.21.15.83:0/4203415935 >> 172.21.15.83:0/4203415935 conn(0x7f79e8101210 msgr2=0x7f79e8103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.902+0000 7f79efee6700 1 -- 172.21.15.83:0/4203415935 shutdown_connections 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.902+0000 7f79efee6700 1 -- 172.21.15.83:0/4203415935 wait complete. 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.902+0000 7f79efee6700 1 Processor -- start 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.902+0000 7f79efee6700 1 -- start start 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79efee6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8105fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79efee6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e819e070 con 0x7f79e81069b0 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79edc82700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8105fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79edc82700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8105fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50588/0 (socket says 172.21.15.83:50588) 2024-03-11T19:29:57.588 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79edc82700 1 -- 172.21.15.83:0/3904443067 learned_addr learned my addr 172.21.15.83:0/3904443067 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79edc82700 1 -- 172.21.15.83:0/3904443067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e4009550 con 0x7f79e81069b0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79edc82700 1 --2- 172.21.15.83:0/3904443067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8105fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009130 tx=0x7f79e4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79deffd700 1 -- 172.21.15.83:0/3904443067 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79e4003aa0 con 0x7f79e81069b0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79deffd700 1 -- 172.21.15.83:0/3904443067 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79e4003c00 con 0x7f79e81069b0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79efee6700 1 -- 172.21.15.83:0/3904443067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e819e1b0 con 0x7f79e81069b0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79deffd700 1 -- 172.21.15.83:0/3904443067 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79e4024800 con 0x7f79e81069b0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.903+0000 7f79efee6700 1 -- 172.21.15.83:0/3904443067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e819e590 con 0x7f79e81069b0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.904+0000 7f79deffd700 1 -- 172.21.15.83:0/3904443067 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f79e401abd0 con 0x7f79e81069b0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.904+0000 7f79efee6700 1 -- 172.21.15.83:0/3904443067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e810a900 con 0x7f79e81069b0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.904+0000 7f79deffd700 1 --2- 172.21.15.83:0/3904443067 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f79d4038d00 0x7f79d403b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.904+0000 7f79deffd700 1 -- 172.21.15.83:0/3904443067 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f79e404ceb0 con 0x7f79e81069b0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.904+0000 7f79ed481700 1 --2- 172.21.15.83:0/3904443067 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f79d4038d00 0x7f79d403b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.905+0000 7f79ed481700 1 --2- 172.21.15.83:0/3904443067 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f79d4038d00 0x7f79d403b1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f79d8006fd0 tx=0x7f79d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:57.589 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:55.909+0000 7f79deffd700 1 -- 172.21.15.83:0/3904443067 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f79e4018760 con 0x7f79e81069b0 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.129+0000 7f79efee6700 1 -- 172.21.15.83:0/3904443067 --> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f79e8102490 con 0x7f79d4038d00 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.134+0000 7f79deffd700 1 -- 172.21.15.83:0/3904443067 <== mgr.14120 v2:172.21.15.83:6800/4058944299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f79e8102490 con 0x7f79d4038d00 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.136+0000 7f79efee6700 1 -- 172.21.15.83:0/3904443067 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f79d4038d00 msgr2=0x7f79d403b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.136+0000 7f79efee6700 1 --2- 172.21.15.83:0/3904443067 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f79d4038d00 0x7f79d403b1c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f79d8006fd0 tx=0x7f79d8006e40 comp rx=0 tx=0).stop 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.136+0000 7f79efee6700 1 -- 172.21.15.83:0/3904443067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 msgr2=0x7f79e8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.136+0000 7f79efee6700 1 --2- 172.21.15.83:0/3904443067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8105fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009130 tx=0x7f79e4011b40 comp rx=0 tx=0).stop 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.136+0000 7f79efee6700 1 -- 172.21.15.83:0/3904443067 shutdown_connections 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.136+0000 7f79efee6700 1 --2- 172.21.15.83:0/3904443067 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f79d4038d00 0x7f79d403b1c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.136+0000 7f79efee6700 1 --2- 172.21.15.83:0/3904443067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e81069b0 0x7f79e8105fb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.136+0000 7f79efee6700 1 -- 172.21.15.83:0/3904443067 >> 172.21.15.83:0/3904443067 conn(0x7f79e8101210 msgr2=0x7f79e8101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.136+0000 7f79efee6700 1 -- 172.21.15.83:0/3904443067 shutdown_connections 2024-03-11T19:29:57.590 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:56.136+0000 7f79efee6700 1 -- 172.21.15.83:0/3904443067 wait complete. 2024-03-11T19:29:58.754 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:58 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:58.755 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:58 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:29:58.755 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:29:58 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2406067349' entity='client.admin' 2024-03-11T19:29:58.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.027+0000 7f686d207700 1 Processor -- start 2024-03-11T19:29:58.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.027+0000 7f686d207700 1 -- start start 2024-03-11T19:29:58.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.027+0000 7f686d207700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686810a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:58.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.027+0000 7f686d207700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686807ae40 con 0x7f68681080f0 2024-03-11T19:29:58.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.027+0000 7f6866d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686810a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:58.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.027+0000 7f6866d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686810a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50602/0 (socket says 172.21.15.83:50602) 2024-03-11T19:29:58.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.027+0000 7f6866d9d700 1 -- 172.21.15.83:0/31932831 learned_addr learned my addr 172.21.15.83:0/31932831 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:58.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.028+0000 7f6866d9d700 1 -- 172.21.15.83:0/31932831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686810aa20 con 0x7f68681080f0 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.028+0000 7f6866d9d700 1 --2- 172.21.15.83:0/31932831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686810a4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f68580098a0 tx=0x7f6858009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52161efb8c4101ff server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.028+0000 7f6865d9b700 1 -- 172.21.15.83:0/31932831 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6858004160 con 0x7f68681080f0 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.028+0000 7f6865d9b700 1 -- 172.21.15.83:0/31932831 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68580042c0 con 0x7f68681080f0 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.028+0000 7f6865d9b700 1 -- 172.21.15.83:0/31932831 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6858011a80 con 0x7f68681080f0 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.028+0000 7f686d207700 1 -- 172.21.15.83:0/31932831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 msgr2=0x7f686810a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.028+0000 7f686d207700 1 --2- 172.21.15.83:0/31932831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686810a4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f68580098a0 tx=0x7f6858009bb0 comp rx=0 tx=0).stop 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.029+0000 7f686d207700 1 -- 172.21.15.83:0/31932831 shutdown_connections 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.029+0000 7f686d207700 1 --2- 172.21.15.83:0/31932831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686810a4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.029+0000 7f686d207700 1 -- 172.21.15.83:0/31932831 >> 172.21.15.83:0/31932831 conn(0x7f6868101b00 msgr2=0x7f6868103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.029+0000 7f686d207700 1 -- 172.21.15.83:0/31932831 shutdown_connections 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.029+0000 7f686d207700 1 -- 172.21.15.83:0/31932831 wait complete. 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.029+0000 7f686d207700 1 Processor -- start 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.030+0000 7f686d207700 1 -- start start 2024-03-11T19:29:58.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.030+0000 7f686d207700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.030+0000 7f686d207700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686819b8c0 con 0x7f68681080f0 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.030+0000 7f6866d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.030+0000 7f6866d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50608/0 (socket says 172.21.15.83:50608) 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.030+0000 7f6866d9d700 1 -- 172.21.15.83:0/2406067349 learned_addr learned my addr 172.21.15.83:0/2406067349 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.030+0000 7f6866d9d700 1 -- 172.21.15.83:0/2406067349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6858009550 con 0x7f68681080f0 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.030+0000 7f6866d9d700 1 --2- 172.21.15.83:0/2406067349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686819b380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6858009130 tx=0x7f6858011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.030+0000 7f684ffff700 1 -- 172.21.15.83:0/2406067349 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6858003aa0 con 0x7f68681080f0 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.030+0000 7f684ffff700 1 -- 172.21.15.83:0/2406067349 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6858003c00 con 0x7f68681080f0 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.030+0000 7f686d207700 1 -- 172.21.15.83:0/2406067349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686819bac0 con 0x7f68681080f0 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.031+0000 7f684ffff700 1 -- 172.21.15.83:0/2406067349 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68580248a0 con 0x7f68681080f0 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.031+0000 7f686d207700 1 -- 172.21.15.83:0/2406067349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686819bf60 con 0x7f68681080f0 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.031+0000 7f684ffff700 1 -- 172.21.15.83:0/2406067349 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f6858024ae0 con 0x7f68681080f0 2024-03-11T19:29:58.804 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.031+0000 7f686d207700 1 -- 172.21.15.83:0/2406067349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686810b810 con 0x7f68681080f0 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.032+0000 7f684ffff700 1 --2- 172.21.15.83:0/2406067349 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f6850038d00 0x7f685003b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.032+0000 7f684ffff700 1 -- 172.21.15.83:0/2406067349 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f685801ac10 con 0x7f68681080f0 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.032+0000 7f686659c700 1 --2- 172.21.15.83:0/2406067349 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f6850038d00 0x7f685003b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.032+0000 7f686659c700 1 --2- 172.21.15.83:0/2406067349 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f6850038d00 0x7f685003b1c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f685c006fd0 tx=0x7f685c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.037+0000 7f684ffff700 1 -- 172.21.15.83:0/2406067349 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6858024d90 con 0x7f68681080f0 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.236+0000 7f686d207700 1 -- 172.21.15.83:0/2406067349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f686819e8a0 con 0x7f68681080f0 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.238+0000 7f684ffff700 1 -- 172.21.15.83:0/2406067349 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f685802c350 con 0x7f68681080f0 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.241+0000 7f686d207700 1 -- 172.21.15.83:0/2406067349 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f6850038d00 msgr2=0x7f685003b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.241+0000 7f686d207700 1 --2- 172.21.15.83:0/2406067349 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f6850038d00 0x7f685003b1c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f685c006fd0 tx=0x7f685c006e40 comp rx=0 tx=0).stop 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.241+0000 7f686d207700 1 -- 172.21.15.83:0/2406067349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 msgr2=0x7f686819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.241+0000 7f686d207700 1 --2- 172.21.15.83:0/2406067349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686819b380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6858009130 tx=0x7f6858011b00 comp rx=0 tx=0).stop 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.241+0000 7f686d207700 1 -- 172.21.15.83:0/2406067349 shutdown_connections 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.241+0000 7f686d207700 1 --2- 172.21.15.83:0/2406067349 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f6850038d00 0x7f685003b1c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:58.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.241+0000 7f686d207700 1 --2- 172.21.15.83:0/2406067349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68681080f0 0x7f686819b380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:29:58.806 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.241+0000 7f686d207700 1 -- 172.21.15.83:0/2406067349 >> 172.21.15.83:0/2406067349 conn(0x7f6868101b00 msgr2=0x7f68681025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:29:58.807 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.241+0000 7f686d207700 1 -- 172.21.15.83:0/2406067349 shutdown_connections 2024-03-11T19:29:58.807 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:58.241+0000 7f686d207700 1 -- 172.21.15.83:0/2406067349 wait complete. 2024-03-11T19:30:00.160 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.218+0000 7f65c0ec2700 1 Processor -- start 2024-03-11T19:30:00.160 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.218+0000 7f65c0ec2700 1 -- start start 2024-03-11T19:30:00.160 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.218+0000 7f65c0ec2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:00.160 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.218+0000 7f65c0ec2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc106ef0 con 0x7f65bc106500 2024-03-11T19:30:00.160 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.218+0000 7f65ba59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:00.160 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.218+0000 7f65ba59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50616/0 (socket says 172.21.15.83:50616) 2024-03-11T19:30:00.160 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.218+0000 7f65ba59c700 1 -- 172.21.15.83:0/515089644 learned_addr learned my addr 172.21.15.83:0/515089644 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:00.160 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.219+0000 7f65ba59c700 1 -- 172.21.15.83:0/515089644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc107700 con 0x7f65bc106500 2024-03-11T19:30:00.160 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.219+0000 7f65ba59c700 1 --2- 172.21.15.83:0/515089644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc106920 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0098a0 tx=0x7f65ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c6598cbf2b1eed server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:00.160 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.219+0000 7f65b959a700 1 -- 172.21.15.83:0/515089644 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac004160 con 0x7f65bc106500 2024-03-11T19:30:00.160 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.219+0000 7f65b959a700 1 -- 172.21.15.83:0/515089644 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65ac0042c0 con 0x7f65bc106500 2024-03-11T19:30:00.161 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.219+0000 7f65b959a700 1 -- 172.21.15.83:0/515089644 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac011a80 con 0x7f65bc106500 2024-03-11T19:30:00.161 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.219+0000 7f65c0ec2700 1 -- 172.21.15.83:0/515089644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 msgr2=0x7f65bc106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:00.161 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.219+0000 7f65c0ec2700 1 --2- 172.21.15.83:0/515089644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc106920 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0098a0 tx=0x7f65ac009bb0 comp rx=0 tx=0).stop 2024-03-11T19:30:00.161 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.220+0000 7f65c0ec2700 1 -- 172.21.15.83:0/515089644 shutdown_connections 2024-03-11T19:30:00.161 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.220+0000 7f65c0ec2700 1 --2- 172.21.15.83:0/515089644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc106920 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:00.161 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.220+0000 7f65c0ec2700 1 -- 172.21.15.83:0/515089644 >> 172.21.15.83:0/515089644 conn(0x7f65bc101b40 msgr2=0x7f65bc103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:00.161 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.220+0000 7f65c0ec2700 1 -- 172.21.15.83:0/515089644 shutdown_connections 2024-03-11T19:30:00.161 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.220+0000 7f65c0ec2700 1 -- 172.21.15.83:0/515089644 wait complete. 2024-03-11T19:30:00.161 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.220+0000 7f65c0ec2700 1 Processor -- start 2024-03-11T19:30:00.161 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.220+0000 7f65c0ec2700 1 -- start start 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.220+0000 7f65c0ec2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc07aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.220+0000 7f65c0ec2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc07afa0 con 0x7f65bc106500 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.221+0000 7f65ba59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc07aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.221+0000 7f65ba59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc07aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50620/0 (socket says 172.21.15.83:50620) 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.221+0000 7f65ba59c700 1 -- 172.21.15.83:0/1269325567 learned_addr learned my addr 172.21.15.83:0/1269325567 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.221+0000 7f65ba59c700 1 -- 172.21.15.83:0/1269325567 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac009550 con 0x7f65bc106500 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.221+0000 7f65ba59c700 1 --2- 172.21.15.83:0/1269325567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc07aa60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f65ac011740 tx=0x7f65ac011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.221+0000 7f65ab7fe700 1 -- 172.21.15.83:0/1269325567 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac0039e0 con 0x7f65bc106500 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.221+0000 7f65c0ec2700 1 -- 172.21.15.83:0/1269325567 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc19bec0 con 0x7f65bc106500 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.221+0000 7f65ab7fe700 1 -- 172.21.15.83:0/1269325567 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65ac003b40 con 0x7f65bc106500 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.221+0000 7f65ab7fe700 1 -- 172.21.15.83:0/1269325567 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac024740 con 0x7f65bc106500 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.221+0000 7f65c0ec2700 1 -- 172.21.15.83:0/1269325567 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc1a0780 con 0x7f65bc106500 2024-03-11T19:30:00.162 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.222+0000 7f65ab7fe700 1 -- 172.21.15.83:0/1269325567 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f65ac02d3f0 con 0x7f65bc106500 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.222+0000 7f65c0ec2700 1 -- 172.21.15.83:0/1269325567 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc10a930 con 0x7f65bc106500 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.222+0000 7f65ab7fe700 1 --2- 172.21.15.83:0/1269325567 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f65a4038cb0 0x7f65a403b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.222+0000 7f65ab7fe700 1 -- 172.21.15.83:0/1269325567 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f65ac04d1c0 con 0x7f65bc106500 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.222+0000 7f65b9d9b700 1 --2- 172.21.15.83:0/1269325567 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f65a4038cb0 0x7f65a403b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.223+0000 7f65b9d9b700 1 --2- 172.21.15.83:0/1269325567 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f65a4038cb0 0x7f65a403b170 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f65b0006fd0 tx=0x7f65b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.227+0000 7f65ab7fe700 1 -- 172.21.15.83:0/1269325567 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65ac010970 con 0x7f65bc106500 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.432+0000 7f65c0ec2700 1 -- 172.21.15.83:0/1269325567 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f65bc1a09c0 con 0x7f65bc106500 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.434+0000 7f65ab7fe700 1 -- 172.21.15.83:0/1269325567 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f65ac04cb30 con 0x7f65bc106500 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.437+0000 7f65c0ec2700 1 -- 172.21.15.83:0/1269325567 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f65a4038cb0 msgr2=0x7f65a403b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.437+0000 7f65c0ec2700 1 --2- 172.21.15.83:0/1269325567 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f65a4038cb0 0x7f65a403b170 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f65b0006fd0 tx=0x7f65b0006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.437+0000 7f65c0ec2700 1 -- 172.21.15.83:0/1269325567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 msgr2=0x7f65bc07aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.437+0000 7f65c0ec2700 1 --2- 172.21.15.83:0/1269325567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc07aa60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f65ac011740 tx=0x7f65ac011b00 comp rx=0 tx=0).stop 2024-03-11T19:30:00.163 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.437+0000 7f65c0ec2700 1 -- 172.21.15.83:0/1269325567 shutdown_connections 2024-03-11T19:30:00.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.437+0000 7f65c0ec2700 1 --2- 172.21.15.83:0/1269325567 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f65a4038cb0 0x7f65a403b170 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:00.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.437+0000 7f65c0ec2700 1 --2- 172.21.15.83:0/1269325567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65bc106500 0x7f65bc07aa60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:00.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.437+0000 7f65c0ec2700 1 -- 172.21.15.83:0/1269325567 >> 172.21.15.83:0/1269325567 conn(0x7f65bc101b40 msgr2=0x7f65bc1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:00.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.437+0000 7f65c0ec2700 1 -- 172.21.15.83:0/1269325567 shutdown_connections 2024-03-11T19:30:00.164 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:29:59.437+0000 7f65c0ec2700 1 -- 172.21.15.83:0/1269325567 wait complete. 2024-03-11T19:30:00.164 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling the dashboard module... 2024-03-11T19:30:00.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:00 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1269325567' entity='client.admin' 2024-03-11T19:30:01.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:01 smithi083 ceph-mon[31864]: from='mgr.14120 172.21.15.83:0/4068167448' entity='mgr.x' 2024-03-11T19:30:01.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:01 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2273773836' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-11T19:30:01.794 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:01 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: ignoring --setuser ceph since I am not root 2024-03-11T19:30:01.794 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:01 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: ignoring --setgroup ceph since I am not root 2024-03-11T19:30:02.086 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.857+0000 7f0cfeb8d700 1 Processor -- start 2024-03-11T19:30:02.087 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.857+0000 7f0cfeb8d700 1 -- start start 2024-03-11T19:30:02.087 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.857+0000 7f0cfeb8d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf810a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:02.087 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.857+0000 7f0cfeb8d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80754f0 con 0x7f0cf81080b0 2024-03-11T19:30:02.087 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.857+0000 7f0cfc929700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf810a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:02.087 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.857+0000 7f0cfc929700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf810a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49376/0 (socket says 172.21.15.83:49376) 2024-03-11T19:30:02.087 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.857+0000 7f0cfc929700 1 -- 172.21.15.83:0/784942009 learned_addr learned my addr 172.21.15.83:0/784942009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:02.087 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.857+0000 7f0cfc929700 1 -- 172.21.15.83:0/784942009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf810a9e0 con 0x7f0cf81080b0 2024-03-11T19:30:02.087 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.858+0000 7f0cfc929700 1 --2- 172.21.15.83:0/784942009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf810a4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0098a0 tx=0x7f0cec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3451f5e310bf6d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:02.087 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.858+0000 7f0cf77fe700 1 -- 172.21.15.83:0/784942009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cec004160 con 0x7f0cf81080b0 2024-03-11T19:30:02.087 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.858+0000 7f0cf77fe700 1 -- 172.21.15.83:0/784942009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cec0042c0 con 0x7f0cf81080b0 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.858+0000 7f0cf77fe700 1 -- 172.21.15.83:0/784942009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cec011a80 con 0x7f0cf81080b0 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.858+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/784942009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 msgr2=0x7f0cf810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.858+0000 7f0cfeb8d700 1 --2- 172.21.15.83:0/784942009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf810a4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0098a0 tx=0x7f0cec009bb0 comp rx=0 tx=0).stop 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.858+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/784942009 shutdown_connections 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.858+0000 7f0cfeb8d700 1 --2- 172.21.15.83:0/784942009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf810a4a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.858+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/784942009 >> 172.21.15.83:0/784942009 conn(0x7f0cf8101ac0 msgr2=0x7f0cf8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.859+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/784942009 shutdown_connections 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.859+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/784942009 wait complete. 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.859+0000 7f0cfeb8d700 1 Processor -- start 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.859+0000 7f0cfeb8d700 1 -- start start 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.859+0000 7f0cfeb8d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.859+0000 7f0cfeb8d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf819b8f0 con 0x7f0cf81080b0 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.860+0000 7f0cfc929700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.860+0000 7f0cfc929700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49378/0 (socket says 172.21.15.83:49378) 2024-03-11T19:30:02.088 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.860+0000 7f0cfc929700 1 -- 172.21.15.83:0/2273773836 learned_addr learned my addr 172.21.15.83:0/2273773836 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.860+0000 7f0cfc929700 1 -- 172.21.15.83:0/2273773836 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec009550 con 0x7f0cf81080b0 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.860+0000 7f0cfc929700 1 --2- 172.21.15.83:0/2273773836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf819b3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0cec003f20 tx=0x7f0cec011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.860+0000 7f0cf5ffb700 1 -- 172.21.15.83:0/2273773836 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cec0039f0 con 0x7f0cf81080b0 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.860+0000 7f0cf5ffb700 1 -- 172.21.15.83:0/2273773836 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cec003b50 con 0x7f0cf81080b0 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.860+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/2273773836 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf819baf0 con 0x7f0cf81080b0 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.860+0000 7f0cf5ffb700 1 -- 172.21.15.83:0/2273773836 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cec01a6d0 con 0x7f0cf81080b0 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.860+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/2273773836 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf819bf90 con 0x7f0cf81080b0 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.861+0000 7f0cf5ffb700 1 -- 172.21.15.83:0/2273773836 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) v1 ==== 45769+0+0 (secure 0 0 0) 0x7f0cec01a940 con 0x7f0cf81080b0 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.861+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/2273773836 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf810b820 con 0x7f0cf81080b0 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.861+0000 7f0cf5ffb700 1 --2- 172.21.15.83:0/2273773836 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f0ce8038cb0 0x7f0ce803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.862+0000 7f0cf5ffb700 1 -- 172.21.15.83:0/2273773836 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0cec04cf80 con 0x7f0cf81080b0 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.862+0000 7f0cf7fff700 1 --2- 172.21.15.83:0/2273773836 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f0ce8038cb0 0x7f0ce803b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:02.089 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.862+0000 7f0cf7fff700 1 --2- 172.21.15.83:0/2273773836 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f0ce8038cb0 0x7f0ce803b170 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4006fd0 tx=0x7f0ce4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:00.866+0000 7f0cf5ffb700 1 -- 172.21.15.83:0/2273773836 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0cec018dc0 con 0x7f0cf81080b0 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.156+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/2273773836 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f0cf819e870 con 0x7f0cf81080b0 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.477+0000 7f0cf5ffb700 1 -- 172.21.15.83:0/2273773836 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f0cec018be0 con 0x7f0cf81080b0 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.479+0000 7f0cf5ffb700 1 -- 172.21.15.83:0/2273773836 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f0cec02d9b0 con 0x7f0cf81080b0 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.480+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/2273773836 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f0ce8038cb0 msgr2=0x7f0ce803b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.480+0000 7f0cfeb8d700 1 --2- 172.21.15.83:0/2273773836 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f0ce8038cb0 0x7f0ce803b170 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4006fd0 tx=0x7f0ce4006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.480+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/2273773836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 msgr2=0x7f0cf819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.480+0000 7f0cfeb8d700 1 --2- 172.21.15.83:0/2273773836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf819b3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0cec003f20 tx=0x7f0cec011b00 comp rx=0 tx=0).stop 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.481+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/2273773836 shutdown_connections 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.481+0000 7f0cfeb8d700 1 --2- 172.21.15.83:0/2273773836 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7f0ce8038cb0 0x7f0ce803b170 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.481+0000 7f0cfeb8d700 1 --2- 172.21.15.83:0/2273773836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf81080b0 0x7f0cf819b3b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.481+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/2273773836 >> 172.21.15.83:0/2273773836 conn(0x7f0cf8101ac0 msgr2=0x7f0cf81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.481+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/2273773836 shutdown_connections 2024-03-11T19:30:02.090 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:01.481+0000 7f0cfeb8d700 1 -- 172.21.15.83:0/2273773836 wait complete. 2024-03-11T19:30:02.317 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:02 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:02.051+0000 7f856c77a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:30:02.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:02 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:02.155+0000 7f856c77a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:30:02.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:02 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2273773836' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-11T19:30:02.728 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:02 smithi083 ceph-mon[31864]: mgrmap e9: x(active, since 19s) 2024-03-11T19:30:03.058 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:02 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:02.818+0000 7f856c77a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:30:03.058 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:02 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:02.912+0000 7f856c77a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:30:03.058 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:03.054+0000 7f856c77a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:30:03.317 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:03.152+0000 7f856c77a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:30:03.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:03.235+0000 7f856c77a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:30:03.434 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:30:03.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-11T19:30:03.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-11T19:30:03.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-11T19:30:03.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-11T19:30:03.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:30:03.435 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.518+0000 7fd5ae937700 1 Processor -- start 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.518+0000 7fd5ae937700 1 -- start start 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.518+0000 7fd5ae937700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a810a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.518+0000 7fd5ae937700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a807ae80 con 0x7fd5a81080b0 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.518+0000 7fd5a7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a810a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.518+0000 7fd5a7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a810a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49412/0 (socket says 172.21.15.83:49412) 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.518+0000 7fd5a7fff700 1 -- 172.21.15.83:0/2866239586 learned_addr learned my addr 172.21.15.83:0/2866239586 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.519+0000 7fd5a7fff700 1 -- 172.21.15.83:0/2866239586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a810a9e0 con 0x7fd5a81080b0 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.519+0000 7fd5a7fff700 1 --2- 172.21.15.83:0/2866239586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a810a4a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0098a0 tx=0x7fd59c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ed394132e18313 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.519+0000 7fd5a6ffd700 1 -- 172.21.15.83:0/2866239586 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd59c004160 con 0x7fd5a81080b0 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.519+0000 7fd5a6ffd700 1 -- 172.21.15.83:0/2866239586 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd59c0042c0 con 0x7fd5a81080b0 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.519+0000 7fd5a6ffd700 1 -- 172.21.15.83:0/2866239586 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd59c011a80 con 0x7fd5a81080b0 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.520+0000 7fd5ae937700 1 -- 172.21.15.83:0/2866239586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 msgr2=0x7fd5a810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:03.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.520+0000 7fd5ae937700 1 --2- 172.21.15.83:0/2866239586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a810a4a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0098a0 tx=0x7fd59c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.520+0000 7fd5ae937700 1 -- 172.21.15.83:0/2866239586 shutdown_connections 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.520+0000 7fd5ae937700 1 --2- 172.21.15.83:0/2866239586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a810a4a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.520+0000 7fd5ae937700 1 -- 172.21.15.83:0/2866239586 >> 172.21.15.83:0/2866239586 conn(0x7fd5a8101b00 msgr2=0x7fd5a8103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.520+0000 7fd5ae937700 1 -- 172.21.15.83:0/2866239586 shutdown_connections 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.520+0000 7fd5ae937700 1 -- 172.21.15.83:0/2866239586 wait complete. 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.520+0000 7fd5ae937700 1 Processor -- start 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.520+0000 7fd5ae937700 1 -- start start 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.521+0000 7fd5ae937700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a8107930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.521+0000 7fd5ae937700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a8107e70 con 0x7fd5a81080b0 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.521+0000 7fd5a7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a8107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.521+0000 7fd5a7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a8107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49426/0 (socket says 172.21.15.83:49426) 2024-03-11T19:30:03.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.521+0000 7fd5a7fff700 1 -- 172.21.15.83:0/3389698557 learned_addr learned my addr 172.21.15.83:0/3389698557 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.521+0000 7fd5a7fff700 1 -- 172.21.15.83:0/3389698557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c009550 con 0x7fd5a81080b0 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.521+0000 7fd5a7fff700 1 --2- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a8107930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd59c003f20 tx=0x7fd59c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.521+0000 7fd5a57fa700 1 -- 172.21.15.83:0/3389698557 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd59c0039f0 con 0x7fd5a81080b0 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.521+0000 7fd5a57fa700 1 -- 172.21.15.83:0/3389698557 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd59c003b50 con 0x7fd5a81080b0 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.521+0000 7fd5ae937700 1 -- 172.21.15.83:0/3389698557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a8106060 con 0x7fd5a81080b0 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.522+0000 7fd5a57fa700 1 -- 172.21.15.83:0/3389698557 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd59c01a6d0 con 0x7fd5a81080b0 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.522+0000 7fd5ae937700 1 -- 172.21.15.83:0/3389698557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a8106500 con 0x7fd5a81080b0 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.522+0000 7fd5ae937700 1 -- 172.21.15.83:0/3389698557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a810b820 con 0x7fd5a81080b0 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.523+0000 7fd5a57fa700 1 -- 172.21.15.83:0/3389698557 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fd59c01a940 con 0x7fd5a81080b0 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.523+0000 7fd5a57fa700 1 --2- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd590038d10 0x7fd59003b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.523+0000 7fd5a57fa700 1 -- 172.21.15.83:0/3389698557 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd59c04cfb0 con 0x7fd5a81080b0 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.523+0000 7fd5a77fe700 1 -- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd590038d10 msgr2=0x7fd59003b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/4058944299 2024-03-11T19:30:03.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.523+0000 7fd5a77fe700 1 --2- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd590038d10 0x7fd59003b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.528+0000 7fd5a57fa700 1 -- 172.21.15.83:0/3389698557 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd59c018dc0 con 0x7fd5a81080b0 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.724+0000 7fd5a77fe700 1 -- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd590038d10 msgr2=0x7fd59003b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/4058944299 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.724+0000 7fd5a77fe700 1 --2- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd590038d10 0x7fd59003b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.783+0000 7fd5ae937700 1 -- 172.21.15.83:0/3389698557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fd5a8106b00 con 0x7fd5a81080b0 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.784+0000 7fd5a57fa700 1 -- 172.21.15.83:0/3389698557 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fd59c018be0 con 0x7fd5a81080b0 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.786+0000 7fd5ae937700 1 -- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd590038d10 msgr2=0x7fd59003b1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.786+0000 7fd5ae937700 1 --2- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd590038d10 0x7fd59003b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.786+0000 7fd5ae937700 1 -- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 msgr2=0x7fd5a8107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.786+0000 7fd5ae937700 1 --2- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a8107930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd59c003f20 tx=0x7fd59c011b00 comp rx=0 tx=0).stop 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.786+0000 7fd5ae937700 1 -- 172.21.15.83:0/3389698557 shutdown_connections 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.786+0000 7fd5ae937700 1 --2- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fd590038d10 0x7fd59003b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.786+0000 7fd5ae937700 1 --2- 172.21.15.83:0/3389698557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a81080b0 0x7fd5a8107930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:03.439 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.786+0000 7fd5ae937700 1 -- 172.21.15.83:0/3389698557 >> 172.21.15.83:0/3389698557 conn(0x7fd5a8101b00 msgr2=0x7fd5a81025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:03.440 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.786+0000 7fd5ae937700 1 -- 172.21.15.83:0/3389698557 shutdown_connections 2024-03-11T19:30:03.440 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:02.786+0000 7fd5ae937700 1 -- 172.21.15.83:0/3389698557 wait complete. 2024-03-11T19:30:03.440 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-03-11T19:30:03.440 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 9... 2024-03-11T19:30:03.682 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:03 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3389698557' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-11T19:30:03.682 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:03.509+0000 7f856c77a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:30:03.682 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:03.618+0000 7f856c77a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:30:03.989 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:03.708+0000 7f856c77a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:30:03.989 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:03.805+0000 7f856c77a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:30:04.317 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:03.985+0000 7f856c77a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:30:05.304 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:04 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:04.942+0000 7f856c77a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:30:05.305 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:05.072+0000 7f856c77a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:30:05.567 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:05.416+0000 7f856c77a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:30:05.568 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:05.508+0000 7f856c77a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:30:06.067 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:05.708+0000 7f856c77a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:30:06.068 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:05.895+0000 7f856c77a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:30:06.457 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:06 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:06.183+0000 7f856c77a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:30:06.457 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:06 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:06.453+0000 7f856c77a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:30:06.817 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:06 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:06.690+0000 7f856c77a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:30:07.817 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:07 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:07.496+0000 7f856c77a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:30:08.417 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:08.096+0000 7f856c77a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:30:08.417 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:08.195+0000 7f856c77a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:30:08.417 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:30:08.413+0000 7f856c77a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:30:08.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: Active manager daemon x restarted 2024-03-11T19:30:08.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: Activating manager daemon x 2024-03-11T19:30:08.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: osdmap e3: 0 total, 0 up, 0 in 2024-03-11T19:30:08.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: mgrmap e10: x(active, starting, since 0.00338968s) 2024-03-11T19:30:08.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:30:08.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-11T19:30:08.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-11T19:30:08.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-11T19:30:08.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-11T19:30:08.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: Manager daemon x is now available 2024-03-11T19:30:08.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:30:08.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:30:09.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-11T19:30:09.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-11T19:30:09.688 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:09 smithi083 ceph-mon[31864]: mgrmap e11: x(active, since 1.00547s) 2024-03-11T19:30:10.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-03-11T19:30:10.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-11T19:30:10.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-11T19:30:10.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-03-11T19:30:10.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.893+0000 7fe182b33700 1 Processor -- start 2024-03-11T19:30:10.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.893+0000 7fe182b33700 1 -- start start 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.894+0000 7fe182b33700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c10a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.894+0000 7fe182b33700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c07ae80 con 0x7fe17c1080b0 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.894+0000 7fe1808cf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c10a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.894+0000 7fe1808cf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c10a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49442/0 (socket says 172.21.15.83:49442) 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.894+0000 7fe1808cf700 1 -- 172.21.15.83:0/483566943 learned_addr learned my addr 172.21.15.83:0/483566943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.894+0000 7fe1808cf700 1 -- 172.21.15.83:0/483566943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c10a9e0 con 0x7fe17c1080b0 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.894+0000 7fe1808cf700 1 --2- 172.21.15.83:0/483566943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c10a4a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098a0 tx=0x7fe170009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1167bdf11e59de3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.894+0000 7fe17b7fe700 1 -- 172.21.15.83:0/483566943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe170004160 con 0x7fe17c1080b0 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.894+0000 7fe17b7fe700 1 -- 172.21.15.83:0/483566943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1700042c0 con 0x7fe17c1080b0 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.894+0000 7fe17b7fe700 1 -- 172.21.15.83:0/483566943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe170011a80 con 0x7fe17c1080b0 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.895+0000 7fe182b33700 1 -- 172.21.15.83:0/483566943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 msgr2=0x7fe17c10a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.895+0000 7fe182b33700 1 --2- 172.21.15.83:0/483566943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c10a4a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1700098a0 tx=0x7fe170009bb0 comp rx=0 tx=0).stop 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.895+0000 7fe182b33700 1 -- 172.21.15.83:0/483566943 shutdown_connections 2024-03-11T19:30:10.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.895+0000 7fe182b33700 1 --2- 172.21.15.83:0/483566943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c10a4a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.895+0000 7fe182b33700 1 -- 172.21.15.83:0/483566943 >> 172.21.15.83:0/483566943 conn(0x7fe17c101b00 msgr2=0x7fe17c103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.895+0000 7fe182b33700 1 -- 172.21.15.83:0/483566943 shutdown_connections 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.895+0000 7fe182b33700 1 -- 172.21.15.83:0/483566943 wait complete. 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.896+0000 7fe182b33700 1 Processor -- start 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.896+0000 7fe182b33700 1 -- start start 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.896+0000 7fe182b33700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.896+0000 7fe182b33700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c19b8c0 con 0x7fe17c1080b0 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.896+0000 7fe1808cf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.896+0000 7fe1808cf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49452/0 (socket says 172.21.15.83:49452) 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.896+0000 7fe1808cf700 1 -- 172.21.15.83:0/455973166 learned_addr learned my addr 172.21.15.83:0/455973166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.896+0000 7fe1808cf700 1 -- 172.21.15.83:0/455973166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe170009550 con 0x7fe17c1080b0 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.897+0000 7fe1808cf700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c19b380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe170003f20 tx=0x7fe170011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.897+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1700039f0 con 0x7fe17c1080b0 2024-03-11T19:30:10.197 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.897+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe170003b50 con 0x7fe17c1080b0 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.897+0000 7fe182b33700 1 -- 172.21.15.83:0/455973166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c19bac0 con 0x7fe17c1080b0 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.897+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe17001a6d0 con 0x7fe17c1080b0 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.897+0000 7fe182b33700 1 -- 172.21.15.83:0/455973166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c19bf60 con 0x7fe17c1080b0 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.898+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fe170024430 con 0x7fe17c1080b0 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.898+0000 7fe179ffb700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 0x7fe16c03b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.898+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 --> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe16c03b8e0 con 0x7fe16c038d10 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.898+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe17004cff0 con 0x7fe17c1080b0 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.898+0000 7fe17bfff700 1 -- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 msgr2=0x7fe16c03b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/4058944299 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:03.898+0000 7fe17bfff700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 0x7fe16c03b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:04.099+0000 7fe17bfff700 1 -- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 msgr2=0x7fe16c03b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/4058944299 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:04.099+0000 7fe17bfff700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 0x7fe16c03b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:04.500+0000 7fe17bfff700 1 -- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 msgr2=0x7fe16c03b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/4058944299 2024-03-11T19:30:10.198 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:04.500+0000 7fe17bfff700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 0x7fe16c03b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:05.301+0000 7fe17bfff700 1 -- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 msgr2=0x7fe16c03b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/4058944299 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:05.301+0000 7fe17bfff700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 0x7fe16c03b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:06.902+0000 7fe17bfff700 1 -- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 msgr2=0x7fe16c03b1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/4058944299 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:06.902+0000 7fe17bfff700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 0x7fe16c03b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:08.429+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mgrmap(e 10) v1 ==== 45549+0+0 (secure 0 0 0) 0x7fe17002d910 con 0x7fe17c1080b0 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:08.429+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 msgr2=0x7fe16c03b1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:08.429+0000 7fe179ffb700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 0x7fe16c03b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.430+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 11) v1 ==== 45676+0+0 (secure 0 0 0) 0x7fe17004e600 con 0x7fe17c1080b0 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.430+0000 7fe179ffb700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe16c03c7c0 0x7fe16c03ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.430+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe16c03b8e0 con 0x7fe16c03c7c0 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.431+0000 7fe17bfff700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe16c03c7c0 0x7fe16c03ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.431+0000 7fe17bfff700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe16c03c7c0 0x7fe16c03ebb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe168003a10 tx=0x7fe168009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:10.199 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.432+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fe16c03b8e0 con 0x7fe16c03c7c0 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.443+0000 7fe182b33700 1 -- 172.21.15.83:0/455973166 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe17c10b820 con 0x7fe16c03c7c0 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe179ffb700 1 -- 172.21.15.83:0/455973166 <== mgr.14152 v2:172.21.15.83:6800/3477661212 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fe17c10b820 con 0x7fe16c03c7c0 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe182b33700 1 -- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe16c03c7c0 msgr2=0x7fe16c03ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe182b33700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe16c03c7c0 0x7fe16c03ebb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe168003a10 tx=0x7fe168009270 comp rx=0 tx=0).stop 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe182b33700 1 -- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 msgr2=0x7fe17c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe182b33700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c19b380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe170003f20 tx=0x7fe170011b00 comp rx=0 tx=0).stop 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe182b33700 1 -- 172.21.15.83:0/455973166 shutdown_connections 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe182b33700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe16c03c7c0 0x7fe16c03ebb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe182b33700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe17c1080b0 0x7fe17c19b380 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe182b33700 1 --2- 172.21.15.83:0/455973166 >> [v2:172.21.15.83:6800/4058944299,v1:172.21.15.83:6801/4058944299] conn(0x7fe16c038d10 0x7fe16c03b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe182b33700 1 -- 172.21.15.83:0/455973166 >> 172.21.15.83:0/455973166 conn(0x7fe17c101b00 msgr2=0x7fe17c1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe182b33700 1 -- 172.21.15.83:0/455973166 shutdown_connections 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:09.444+0000 7fe182b33700 1 -- 172.21.15.83:0/455973166 wait complete. 2024-03-11T19:30:10.200 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 9 is available 2024-03-11T19:30:10.201 INFO:teuthology.orchestra.run.smithi083.stdout:Generating a dashboard self-signed certificate... 2024-03-11T19:30:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:10 smithi083 ceph-mon[31864]: [11/Mar/2024:19:30:08] ENGINE Bus STARTING 2024-03-11T19:30:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:10 smithi083 ceph-mon[31864]: [11/Mar/2024:19:30:08] ENGINE Serving on https://172.21.15.83:7150 2024-03-11T19:30:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:10 smithi083 ceph-mon[31864]: [11/Mar/2024:19:30:08] ENGINE Serving on http://172.21.15.83:8765 2024-03-11T19:30:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:10 smithi083 ceph-mon[31864]: [11/Mar/2024:19:30:08] ENGINE Bus STARTED 2024-03-11T19:30:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:10 smithi083 ceph-mon[31864]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-11T19:30:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:10 smithi083 ceph-mon[31864]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-11T19:30:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:11.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-11T19:30:11.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.643+0000 7fc41d6bf700 1 Processor -- start 2024-03-11T19:30:11.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.643+0000 7fc41d6bf700 1 -- start start 2024-03-11T19:30:11.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.643+0000 7fc41d6bf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41810a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:11.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.643+0000 7fc41d6bf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180754f0 con 0x7fc4181080b0 2024-03-11T19:30:11.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.644+0000 7fc416ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41810a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:11.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.644+0000 7fc416ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41810a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32860/0 (socket says 172.21.15.83:32860) 2024-03-11T19:30:11.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.644+0000 7fc416ffd700 1 -- 172.21.15.83:0/926611228 learned_addr learned my addr 172.21.15.83:0/926611228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:11.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.644+0000 7fc416ffd700 1 -- 172.21.15.83:0/926611228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41810a9e0 con 0x7fc4181080b0 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.644+0000 7fc416ffd700 1 --2- 172.21.15.83:0/926611228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41810a4a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0098a0 tx=0x7fc40c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f0765766130c00 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.644+0000 7fc415ffb700 1 -- 172.21.15.83:0/926611228 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc40c004160 con 0x7fc4181080b0 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.644+0000 7fc415ffb700 1 -- 172.21.15.83:0/926611228 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc40c0042c0 con 0x7fc4181080b0 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.644+0000 7fc415ffb700 1 -- 172.21.15.83:0/926611228 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc40c011a80 con 0x7fc4181080b0 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.645+0000 7fc41d6bf700 1 -- 172.21.15.83:0/926611228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 msgr2=0x7fc41810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.645+0000 7fc41d6bf700 1 --2- 172.21.15.83:0/926611228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41810a4a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0098a0 tx=0x7fc40c009bb0 comp rx=0 tx=0).stop 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.645+0000 7fc41d6bf700 1 -- 172.21.15.83:0/926611228 shutdown_connections 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.645+0000 7fc41d6bf700 1 --2- 172.21.15.83:0/926611228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41810a4a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.645+0000 7fc41d6bf700 1 -- 172.21.15.83:0/926611228 >> 172.21.15.83:0/926611228 conn(0x7fc418101ac0 msgr2=0x7fc418103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.645+0000 7fc41d6bf700 1 -- 172.21.15.83:0/926611228 shutdown_connections 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.645+0000 7fc41d6bf700 1 -- 172.21.15.83:0/926611228 wait complete. 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.646+0000 7fc41d6bf700 1 Processor -- start 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.646+0000 7fc41d6bf700 1 -- start start 2024-03-11T19:30:11.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.646+0000 7fc41d6bf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.646+0000 7fc41d6bf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41819b8f0 con 0x7fc4181080b0 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.646+0000 7fc416ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.646+0000 7fc416ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32870/0 (socket says 172.21.15.83:32870) 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.646+0000 7fc416ffd700 1 -- 172.21.15.83:0/2562459396 learned_addr learned my addr 172.21.15.83:0/2562459396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.646+0000 7fc416ffd700 1 -- 172.21.15.83:0/2562459396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c009550 con 0x7fc4181080b0 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.646+0000 7fc416ffd700 1 --2- 172.21.15.83:0/2562459396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41819b3b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc40c003f20 tx=0x7fc40c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.647+0000 7fc3fffff700 1 -- 172.21.15.83:0/2562459396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc40c0039f0 con 0x7fc4181080b0 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.647+0000 7fc41d6bf700 1 -- 172.21.15.83:0/2562459396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41819baf0 con 0x7fc4181080b0 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.647+0000 7fc3fffff700 1 -- 172.21.15.83:0/2562459396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc40c003b50 con 0x7fc4181080b0 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.647+0000 7fc3fffff700 1 -- 172.21.15.83:0/2562459396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc40c01a6d0 con 0x7fc4181080b0 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.647+0000 7fc41d6bf700 1 -- 172.21.15.83:0/2562459396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41819bf90 con 0x7fc4181080b0 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.647+0000 7fc3fffff700 1 -- 172.21.15.83:0/2562459396 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 11) v1 ==== 45676+0+0 (secure 0 0 0) 0x7fc40c01a990 con 0x7fc4181080b0 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.648+0000 7fc41d6bf700 1 -- 172.21.15.83:0/2562459396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41810b820 con 0x7fc4181080b0 2024-03-11T19:30:11.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.648+0000 7fc3fffff700 1 --2- 172.21.15.83:0/2562459396 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc400038ba0 0x7fc40003b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.648+0000 7fc3fffff700 1 -- 172.21.15.83:0/2562459396 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc40c04cec0 con 0x7fc4181080b0 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.648+0000 7fc4167fc700 1 --2- 172.21.15.83:0/2562459396 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc400038ba0 0x7fc40003b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.648+0000 7fc4167fc700 1 --2- 172.21.15.83:0/2562459396 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc400038ba0 0x7fc40003b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc408006fd0 tx=0x7fc408006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.653+0000 7fc3fffff700 1 -- 172.21.15.83:0/2562459396 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc40c022740 con 0x7fc4181080b0 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:10.862+0000 7fc41d6bf700 1 -- 172.21.15.83:0/2562459396 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc418102cc0 con 0x7fc400038ba0 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.021+0000 7fc3fffff700 1 -- 172.21.15.83:0/2562459396 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc418102cc0 con 0x7fc400038ba0 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.023+0000 7fc41d6bf700 1 -- 172.21.15.83:0/2562459396 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc400038ba0 msgr2=0x7fc40003b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.023+0000 7fc41d6bf700 1 --2- 172.21.15.83:0/2562459396 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc400038ba0 0x7fc40003b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc408006fd0 tx=0x7fc408006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.023+0000 7fc41d6bf700 1 -- 172.21.15.83:0/2562459396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 msgr2=0x7fc41819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.023+0000 7fc41d6bf700 1 --2- 172.21.15.83:0/2562459396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41819b3b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc40c003f20 tx=0x7fc40c011b00 comp rx=0 tx=0).stop 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.023+0000 7fc41d6bf700 1 -- 172.21.15.83:0/2562459396 shutdown_connections 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.023+0000 7fc41d6bf700 1 --2- 172.21.15.83:0/2562459396 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc400038ba0 0x7fc40003b060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.023+0000 7fc41d6bf700 1 --2- 172.21.15.83:0/2562459396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4181080b0 0x7fc41819b3b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.023+0000 7fc41d6bf700 1 -- 172.21.15.83:0/2562459396 >> 172.21.15.83:0/2562459396 conn(0x7fc418101ac0 msgr2=0x7fc4181025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:11.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.024+0000 7fc41d6bf700 1 -- 172.21.15.83:0/2562459396 shutdown_connections 2024-03-11T19:30:11.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.024+0000 7fc41d6bf700 1 -- 172.21.15.83:0/2562459396 wait complete. 2024-03-11T19:30:11.546 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial admin user... 2024-03-11T19:30:12.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:12 smithi083 ceph-mon[31864]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:30:12.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:12.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:12.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:12 smithi083 ceph-mon[31864]: mgrmap e12: x(active, since 2s) 2024-03-11T19:30:12.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:13.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:13 smithi083 ceph-mon[31864]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:30:13.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:13 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:13.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$14uAHKeFhFvsxXXoscZFrO3rZL4AD3LMIm6g/gb4CQyIZKyIW/OWK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710185412, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-11T19:30:13.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.939+0000 7fede574e700 1 Processor -- start 2024-03-11T19:30:13.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.939+0000 7fede574e700 1 -- start start 2024-03-11T19:30:13.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.940+0000 7fede574e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede010a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:13.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.940+0000 7fede574e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede00754f0 con 0x7fede01080b0 2024-03-11T19:30:13.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.940+0000 7feddeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede010a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:13.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.940+0000 7feddeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede010a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32880/0 (socket says 172.21.15.83:32880) 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.940+0000 7feddeffd700 1 -- 172.21.15.83:0/3424349188 learned_addr learned my addr 172.21.15.83:0/3424349188 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.940+0000 7feddeffd700 1 -- 172.21.15.83:0/3424349188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede010a9e0 con 0x7fede01080b0 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.940+0000 7feddeffd700 1 --2- 172.21.15.83:0/3424349188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede010a4a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fedd00098a0 tx=0x7fedd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc0ec21469ccbc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.940+0000 7fedddffb700 1 -- 172.21.15.83:0/3424349188 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedd0004160 con 0x7fede01080b0 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.940+0000 7fedddffb700 1 -- 172.21.15.83:0/3424349188 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd00042c0 con 0x7fede01080b0 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.940+0000 7fedddffb700 1 -- 172.21.15.83:0/3424349188 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedd0011a80 con 0x7fede01080b0 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.941+0000 7fede574e700 1 -- 172.21.15.83:0/3424349188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 msgr2=0x7fede010a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.941+0000 7fede574e700 1 --2- 172.21.15.83:0/3424349188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede010a4a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fedd00098a0 tx=0x7fedd0009bb0 comp rx=0 tx=0).stop 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.941+0000 7fede574e700 1 -- 172.21.15.83:0/3424349188 shutdown_connections 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.941+0000 7fede574e700 1 --2- 172.21.15.83:0/3424349188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede010a4a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.941+0000 7fede574e700 1 -- 172.21.15.83:0/3424349188 >> 172.21.15.83:0/3424349188 conn(0x7fede0101ac0 msgr2=0x7fede0103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.941+0000 7fede574e700 1 -- 172.21.15.83:0/3424349188 shutdown_connections 2024-03-11T19:30:13.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.941+0000 7fede574e700 1 -- 172.21.15.83:0/3424349188 wait complete. 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.942+0000 7fede574e700 1 Processor -- start 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.942+0000 7fede574e700 1 -- start start 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.942+0000 7fede574e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.942+0000 7fede574e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede019b8c0 con 0x7fede01080b0 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.942+0000 7feddeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.942+0000 7feddeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32890/0 (socket says 172.21.15.83:32890) 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.942+0000 7feddeffd700 1 -- 172.21.15.83:0/2567063010 learned_addr learned my addr 172.21.15.83:0/2567063010 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.943+0000 7feddeffd700 1 -- 172.21.15.83:0/2567063010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd0009550 con 0x7fede01080b0 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.943+0000 7feddeffd700 1 --2- 172.21.15.83:0/2567063010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede019b380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fedd0003f20 tx=0x7fedd0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.943+0000 7fedc7fff700 1 -- 172.21.15.83:0/2567063010 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedd00039f0 con 0x7fede01080b0 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.943+0000 7fede574e700 1 -- 172.21.15.83:0/2567063010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede019bac0 con 0x7fede01080b0 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.943+0000 7fedc7fff700 1 -- 172.21.15.83:0/2567063010 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd0003b50 con 0x7fede01080b0 2024-03-11T19:30:13.943 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.943+0000 7fedc7fff700 1 -- 172.21.15.83:0/2567063010 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedd001a6d0 con 0x7fede01080b0 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.943+0000 7fede574e700 1 -- 172.21.15.83:0/2567063010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede019bf60 con 0x7fede01080b0 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.944+0000 7fedc7fff700 1 -- 172.21.15.83:0/2567063010 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7fedd001aa10 con 0x7fede01080b0 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.944+0000 7fedc7fff700 1 --2- 172.21.15.83:0/2567063010 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc8038cb0 0x7fedc803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.944+0000 7fedc7fff700 1 -- 172.21.15.83:0/2567063010 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fedd004d1b0 con 0x7fede01080b0 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.944+0000 7fedde7fc700 1 --2- 172.21.15.83:0/2567063010 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc8038cb0 0x7fedc803b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.944+0000 7fede574e700 1 -- 172.21.15.83:0/2567063010 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede010b820 con 0x7fede01080b0 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.944+0000 7fedde7fc700 1 --2- 172.21.15.83:0/2567063010 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc8038cb0 0x7fedc803b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fedd4006fd0 tx=0x7fedd4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:11.953+0000 7fedc7fff700 1 -- 172.21.15.83:0/2567063010 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fedd001ad10 con 0x7fede01080b0 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.165+0000 7fede574e700 1 -- 172.21.15.83:0/2567063010 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fede010ba00 con 0x7fedc8038cb0 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.393+0000 7fedc7fff700 1 -- 172.21.15.83:0/2567063010 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fede010ba00 con 0x7fedc8038cb0 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.395+0000 7fede574e700 1 -- 172.21.15.83:0/2567063010 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc8038cb0 msgr2=0x7fedc803b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.395+0000 7fede574e700 1 --2- 172.21.15.83:0/2567063010 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc8038cb0 0x7fedc803b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fedd4006fd0 tx=0x7fedd4006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:13.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.395+0000 7fede574e700 1 -- 172.21.15.83:0/2567063010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 msgr2=0x7fede019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:13.945 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.395+0000 7fede574e700 1 --2- 172.21.15.83:0/2567063010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede019b380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fedd0003f20 tx=0x7fedd0011b00 comp rx=0 tx=0).stop 2024-03-11T19:30:13.945 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.395+0000 7fede574e700 1 -- 172.21.15.83:0/2567063010 shutdown_connections 2024-03-11T19:30:13.945 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.395+0000 7fede574e700 1 --2- 172.21.15.83:0/2567063010 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc8038cb0 0x7fedc803b170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:13.945 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.395+0000 7fede574e700 1 --2- 172.21.15.83:0/2567063010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fede01080b0 0x7fede019b380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:13.945 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.395+0000 7fede574e700 1 -- 172.21.15.83:0/2567063010 >> 172.21.15.83:0/2567063010 conn(0x7fede0101ac0 msgr2=0x7fede01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:13.945 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.395+0000 7fede574e700 1 -- 172.21.15.83:0/2567063010 shutdown_connections 2024-03-11T19:30:13.945 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:12.395+0000 7fede574e700 1 -- 172.21.15.83:0/2567063010 wait complete. 2024-03-11T19:30:13.945 INFO:teuthology.orchestra.run.smithi083.stdout:Fetching dashboard port number... 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 8443 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.399+0000 7f8d16012700 1 Processor -- start 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.399+0000 7f8d16012700 1 -- start start 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.399+0000 7f8d16012700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1010a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.399+0000 7f8d16012700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1007ae60 con 0x7f8d101080d0 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.399+0000 7f8d0f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1010a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.399+0000 7f8d0f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1010a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32892/0 (socket says 172.21.15.83:32892) 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.399+0000 7f8d0f7fe700 1 -- 172.21.15.83:0/1339169885 learned_addr learned my addr 172.21.15.83:0/1339169885 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.400+0000 7f8d0f7fe700 1 -- 172.21.15.83:0/1339169885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1010aa00 con 0x7f8d101080d0 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.400+0000 7f8d0f7fe700 1 --2- 172.21.15.83:0/1339169885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1010a4c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8d000098a0 tx=0x7f8d00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3898f38520d47272 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.400+0000 7f8d0e7fc700 1 -- 172.21.15.83:0/1339169885 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d00004160 con 0x7f8d101080d0 2024-03-11T19:30:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.400+0000 7f8d0e7fc700 1 -- 172.21.15.83:0/1339169885 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d000042c0 con 0x7f8d101080d0 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.400+0000 7f8d0e7fc700 1 -- 172.21.15.83:0/1339169885 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d00011a80 con 0x7f8d101080d0 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.400+0000 7f8d16012700 1 -- 172.21.15.83:0/1339169885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 msgr2=0x7f8d1010a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.400+0000 7f8d16012700 1 --2- 172.21.15.83:0/1339169885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1010a4c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8d000098a0 tx=0x7f8d00009bb0 comp rx=0 tx=0).stop 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.400+0000 7f8d16012700 1 -- 172.21.15.83:0/1339169885 shutdown_connections 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.400+0000 7f8d16012700 1 --2- 172.21.15.83:0/1339169885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1010a4c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.401+0000 7f8d16012700 1 -- 172.21.15.83:0/1339169885 >> 172.21.15.83:0/1339169885 conn(0x7f8d10101b40 msgr2=0x7f8d10103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.401+0000 7f8d16012700 1 -- 172.21.15.83:0/1339169885 shutdown_connections 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.401+0000 7f8d16012700 1 -- 172.21.15.83:0/1339169885 wait complete. 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.401+0000 7f8d16012700 1 Processor -- start 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.401+0000 7f8d16012700 1 -- start start 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.401+0000 7f8d16012700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.401+0000 7f8d16012700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1019b820 con 0x7f8d101080d0 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.402+0000 7f8d0f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:15.249 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.402+0000 7f8d0f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32896/0 (socket says 172.21.15.83:32896) 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.402+0000 7f8d0f7fe700 1 -- 172.21.15.83:0/300805215 learned_addr learned my addr 172.21.15.83:0/300805215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.402+0000 7f8d0f7fe700 1 -- 172.21.15.83:0/300805215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d00009550 con 0x7f8d101080d0 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.402+0000 7f8d0f7fe700 1 --2- 172.21.15.83:0/300805215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1019b2e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8d00011740 tx=0x7f8d00011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.402+0000 7f8d0cff9700 1 -- 172.21.15.83:0/300805215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d00003b20 con 0x7f8d101080d0 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.402+0000 7f8d16012700 1 -- 172.21.15.83:0/300805215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1019ba20 con 0x7f8d101080d0 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.402+0000 7f8d0cff9700 1 -- 172.21.15.83:0/300805215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d0001a910 con 0x7f8d101080d0 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.402+0000 7f8d0cff9700 1 -- 172.21.15.83:0/300805215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d00024900 con 0x7f8d101080d0 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.402+0000 7f8d16012700 1 -- 172.21.15.83:0/300805215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1019bec0 con 0x7f8d101080d0 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.403+0000 7f8d0cff9700 1 -- 172.21.15.83:0/300805215 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) v1 ==== 45782+0+0 (secure 0 0 0) 0x7f8d00024b70 con 0x7f8d101080d0 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.403+0000 7f8d0cff9700 1 --2- 172.21.15.83:0/300805215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cf8038cb0 0x7f8cf803b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.403+0000 7f8d16012700 1 -- 172.21.15.83:0/300805215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1010b840 con 0x7f8d101080d0 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.403+0000 7f8d0cff9700 1 -- 172.21.15.83:0/300805215 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8d0004d0b0 con 0x7f8d101080d0 2024-03-11T19:30:15.250 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.403+0000 7f8d0effd700 1 --2- 172.21.15.83:0/300805215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cf8038cb0 0x7f8cf803b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.404+0000 7f8d0effd700 1 --2- 172.21.15.83:0/300805215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cf8038cb0 0x7f8cf803b170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8d04006fd0 tx=0x7f8d04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.408+0000 7f8d0cff9700 1 -- 172.21.15.83:0/300805215 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d0002d3f0 con 0x7f8d101080d0 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.516+0000 7f8d0cff9700 1 -- 172.21.15.83:0/300805215 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f8d0002ad60 con 0x7f8d101080d0 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.601+0000 7f8d16012700 1 -- 172.21.15.83:0/300805215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f8d1019e7a0 con 0x7f8d101080d0 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.602+0000 7f8d0cff9700 1 -- 172.21.15.83:0/300805215 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8d0002d3f0 con 0x7f8d101080d0 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.603+0000 7f8d16012700 1 -- 172.21.15.83:0/300805215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cf8038cb0 msgr2=0x7f8cf803b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.603+0000 7f8d16012700 1 --2- 172.21.15.83:0/300805215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cf8038cb0 0x7f8cf803b170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8d04006fd0 tx=0x7f8d04006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.604+0000 7f8d16012700 1 -- 172.21.15.83:0/300805215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 msgr2=0x7f8d1019b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.604+0000 7f8d16012700 1 --2- 172.21.15.83:0/300805215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1019b2e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8d00011740 tx=0x7f8d00011a50 comp rx=0 tx=0).stop 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.604+0000 7f8d16012700 1 -- 172.21.15.83:0/300805215 shutdown_connections 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.604+0000 7f8d16012700 1 --2- 172.21.15.83:0/300805215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cf8038cb0 0x7f8cf803b170 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:15.251 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.604+0000 7f8d16012700 1 --2- 172.21.15.83:0/300805215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d101080d0 0x7f8d1019b2e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:15.252 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.604+0000 7f8d16012700 1 -- 172.21.15.83:0/300805215 >> 172.21.15.83:0/300805215 conn(0x7f8d10101b40 msgr2=0x7f8d10102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:15.252 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.604+0000 7f8d16012700 1 -- 172.21.15.83:0/300805215 shutdown_connections 2024-03-11T19:30:15.252 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:14.604+0000 7f8d16012700 1 -- 172.21.15.83:0/300805215 wait complete. 2024-03-11T19:30:15.255 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-11T19:30:15.255 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-03-11T19:30:15.261 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-11T19:30:15.261 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-03-11T19:30:15.261 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-03-11T19:30:15.261 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-11T19:30:15.262 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph Dashboard is now available at: 2024-03-11T19:30:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: URL: https://smithi083.front.sepia.ceph.com:8443/ 2024-03-11T19:30:15.262 INFO:teuthology.orchestra.run.smithi083.stdout: User: admin 2024-03-11T19:30:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: Password: m4vy15v6xe 2024-03-11T19:30:15.263 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:15.263 INFO:teuthology.orchestra.run.smithi083.stdout:Saving cluster configuration to /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config directory 2024-03-11T19:30:15.263 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling autotune for osd_memory_target 2024-03-11T19:30:15.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:15 smithi083 ceph-mon[31864]: mgrmap e13: x(active, since 6s) 2024-03-11T19:30:15.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:15 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/300805215' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-11T19:30:16.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.724+0000 7fedd921c700 1 Processor -- start 2024-03-11T19:30:16.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.724+0000 7fedd921c700 1 -- start start 2024-03-11T19:30:16.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.724+0000 7fedd921c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd4105d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:16.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.724+0000 7fedd921c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd41062d0 con 0x7fedd41058e0 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.724+0000 7fedd2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd4105d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.724+0000 7fedd2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd4105d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32912/0 (socket says 172.21.15.83:32912) 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.725+0000 7fedd2d9d700 1 -- 172.21.15.83:0/3761659999 learned_addr learned my addr 172.21.15.83:0/3761659999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.725+0000 7fedd2d9d700 1 -- 172.21.15.83:0/3761659999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd4106410 con 0x7fedd41058e0 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.725+0000 7fedd2d9d700 1 --2- 172.21.15.83:0/3761659999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd4105d00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fedc40098a0 tx=0x7fedc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3777edec67a32dd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.725+0000 7fedd259c700 1 -- 172.21.15.83:0/3761659999 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedc4004160 con 0x7fedd41058e0 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.725+0000 7fedd259c700 1 -- 172.21.15.83:0/3761659999 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedc40042c0 con 0x7fedd41058e0 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.725+0000 7fedd259c700 1 -- 172.21.15.83:0/3761659999 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedc4011a80 con 0x7fedd41058e0 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.726+0000 7fedd921c700 1 -- 172.21.15.83:0/3761659999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 msgr2=0x7fedd4105d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.726+0000 7fedd921c700 1 --2- 172.21.15.83:0/3761659999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd4105d00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fedc40098a0 tx=0x7fedc4009bb0 comp rx=0 tx=0).stop 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.726+0000 7fedd921c700 1 -- 172.21.15.83:0/3761659999 shutdown_connections 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.726+0000 7fedd921c700 1 --2- 172.21.15.83:0/3761659999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd4105d00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:16.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.726+0000 7fedd921c700 1 -- 172.21.15.83:0/3761659999 >> 172.21.15.83:0/3761659999 conn(0x7fedd4101260 msgr2=0x7fedd4103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.726+0000 7fedd921c700 1 -- 172.21.15.83:0/3761659999 shutdown_connections 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.726+0000 7fedd921c700 1 -- 172.21.15.83:0/3761659999 wait complete. 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.726+0000 7fedd921c700 1 Processor -- start 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.726+0000 7fedd921c700 1 -- start start 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd921c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd921c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd419b850 con 0x7fedd41058e0 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32916/0 (socket says 172.21.15.83:32916) 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd2d9d700 1 -- 172.21.15.83:0/3243005429 learned_addr learned my addr 172.21.15.83:0/3243005429 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd2d9d700 1 -- 172.21.15.83:0/3243005429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc4009550 con 0x7fedd41058e0 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd2d9d700 1 --2- 172.21.15.83:0/3243005429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd419b310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fedc4011740 tx=0x7fedc4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd0d99700 1 -- 172.21.15.83:0/3243005429 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedc4003af0 con 0x7fedd41058e0 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd0d99700 1 -- 172.21.15.83:0/3243005429 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedc401a910 con 0x7fedd41058e0 2024-03-11T19:30:16.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd0d99700 1 -- 172.21.15.83:0/3243005429 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedc4024870 con 0x7fedd41058e0 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd921c700 1 -- 172.21.15.83:0/3243005429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd419bab0 con 0x7fedd41058e0 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.727+0000 7fedd921c700 1 -- 172.21.15.83:0/3243005429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd419e7d0 con 0x7fedd41058e0 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.728+0000 7fedd0d99700 1 -- 172.21.15.83:0/3243005429 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fedc4024af0 con 0x7fedd41058e0 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.728+0000 7fedd0d99700 1 --2- 172.21.15.83:0/3243005429 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc0038d50 0x7fedc003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.728+0000 7fedd0d99700 1 -- 172.21.15.83:0/3243005429 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fedc404d180 con 0x7fedd41058e0 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.728+0000 7fedcbfff700 1 --2- 172.21.15.83:0/3243005429 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc0038d50 0x7fedc003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.728+0000 7fedd921c700 1 -- 172.21.15.83:0/3243005429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd410bb90 con 0x7fedd41058e0 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.729+0000 7fedcbfff700 1 --2- 172.21.15.83:0/3243005429 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc0038d50 0x7fedc003b210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fedbc006fd0 tx=0x7fedbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.733+0000 7fedd0d99700 1 -- 172.21.15.83:0/3243005429 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fedc4022310 con 0x7fedd41058e0 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.923+0000 7fedd921c700 1 -- 172.21.15.83:0/3243005429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fedd40632a0 con 0x7fedd41058e0 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.924+0000 7fedd0d99700 1 -- 172.21.15.83:0/3243005429 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fedc4018be0 con 0x7fedd41058e0 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.926+0000 7fedd921c700 1 -- 172.21.15.83:0/3243005429 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc0038d50 msgr2=0x7fedc003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:16.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.926+0000 7fedd921c700 1 --2- 172.21.15.83:0/3243005429 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc0038d50 0x7fedc003b210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fedbc006fd0 tx=0x7fedbc006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.926+0000 7fedd921c700 1 -- 172.21.15.83:0/3243005429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 msgr2=0x7fedd419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.926+0000 7fedd921c700 1 --2- 172.21.15.83:0/3243005429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd419b310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fedc4011740 tx=0x7fedc4011a50 comp rx=0 tx=0).stop 2024-03-11T19:30:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.926+0000 7fedd921c700 1 -- 172.21.15.83:0/3243005429 shutdown_connections 2024-03-11T19:30:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.926+0000 7fedd921c700 1 --2- 172.21.15.83:0/3243005429 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fedc0038d50 0x7fedc003b210 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.926+0000 7fedd921c700 1 --2- 172.21.15.83:0/3243005429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedd41058e0 0x7fedd419b310 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.926+0000 7fedd921c700 1 -- 172.21.15.83:0/3243005429 >> 172.21.15.83:0/3243005429 conn(0x7fedd4101260 msgr2=0x7fedd4101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.926+0000 7fedd921c700 1 -- 172.21.15.83:0/3243005429 shutdown_connections 2024-03-11T19:30:16.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:15.926+0000 7fedd921c700 1 -- 172.21.15.83:0/3243005429 wait complete. 2024-03-11T19:30:17.855 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.978+0000 7f146e27a700 1 Processor -- start 2024-03-11T19:30:17.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.978+0000 7f146e27a700 1 -- start start 2024-03-11T19:30:17.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.979+0000 7f146e27a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f146810a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:17.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.979+0000 7f146e27a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14680754f0 con 0x7f14681080b0 2024-03-11T19:30:17.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.979+0000 7f1467fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f146810a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:17.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.979+0000 7f1467fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f146810a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32924/0 (socket says 172.21.15.83:32924) 2024-03-11T19:30:17.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.979+0000 7f1467fff700 1 -- 172.21.15.83:0/1898081143 learned_addr learned my addr 172.21.15.83:0/1898081143 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:17.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.979+0000 7f1467fff700 1 -- 172.21.15.83:0/1898081143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146810a9e0 con 0x7f14681080b0 2024-03-11T19:30:17.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.979+0000 7f1467fff700 1 --2- 172.21.15.83:0/1898081143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f146810a4a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14580098a0 tx=0x7f1458009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc20f2abbdf3be9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:17.856 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.979+0000 7f1466ffd700 1 -- 172.21.15.83:0/1898081143 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1458004160 con 0x7f14681080b0 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.979+0000 7f1466ffd700 1 -- 172.21.15.83:0/1898081143 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14580042c0 con 0x7f14681080b0 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.979+0000 7f1466ffd700 1 -- 172.21.15.83:0/1898081143 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1458011a80 con 0x7f14681080b0 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.980+0000 7f146e27a700 1 -- 172.21.15.83:0/1898081143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 msgr2=0x7f146810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.980+0000 7f146e27a700 1 --2- 172.21.15.83:0/1898081143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f146810a4a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14580098a0 tx=0x7f1458009bb0 comp rx=0 tx=0).stop 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.980+0000 7f146e27a700 1 -- 172.21.15.83:0/1898081143 shutdown_connections 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.980+0000 7f146e27a700 1 --2- 172.21.15.83:0/1898081143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f146810a4a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.980+0000 7f146e27a700 1 -- 172.21.15.83:0/1898081143 >> 172.21.15.83:0/1898081143 conn(0x7f1468101ac0 msgr2=0x7f1468103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.980+0000 7f146e27a700 1 -- 172.21.15.83:0/1898081143 shutdown_connections 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.980+0000 7f146e27a700 1 -- 172.21.15.83:0/1898081143 wait complete. 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.981+0000 7f146e27a700 1 Processor -- start 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.981+0000 7f146e27a700 1 -- start start 2024-03-11T19:30:17.857 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.981+0000 7f146e27a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f1468107930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.981+0000 7f146e27a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1468107e70 con 0x7f14681080b0 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.981+0000 7f1467fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f1468107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.981+0000 7f1467fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f1468107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32928/0 (socket says 172.21.15.83:32928) 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.981+0000 7f1467fff700 1 -- 172.21.15.83:0/2260825084 learned_addr learned my addr 172.21.15.83:0/2260825084 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.981+0000 7f1467fff700 1 -- 172.21.15.83:0/2260825084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1458009550 con 0x7f14681080b0 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.981+0000 7f1467fff700 1 --2- 172.21.15.83:0/2260825084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f1468107930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1458009130 tx=0x7f1458011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.982+0000 7f14657fa700 1 -- 172.21.15.83:0/2260825084 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1458003aa0 con 0x7f14681080b0 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.982+0000 7f14657fa700 1 -- 172.21.15.83:0/2260825084 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1458003c00 con 0x7f14681080b0 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.982+0000 7f146e27a700 1 -- 172.21.15.83:0/2260825084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1468106060 con 0x7f14681080b0 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.982+0000 7f14657fa700 1 -- 172.21.15.83:0/2260825084 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14580248a0 con 0x7f14681080b0 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.982+0000 7f146e27a700 1 -- 172.21.15.83:0/2260825084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1468106500 con 0x7f14681080b0 2024-03-11T19:30:17.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.982+0000 7f14657fa700 1 -- 172.21.15.83:0/2260825084 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f1458024a40 con 0x7f14681080b0 2024-03-11T19:30:17.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.983+0000 7f14657fa700 1 --2- 172.21.15.83:0/2260825084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1450038e00 0x7f145003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:17.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.983+0000 7f14657fa700 1 -- 172.21.15.83:0/2260825084 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f145804d050 con 0x7f14681080b0 2024-03-11T19:30:17.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.983+0000 7f146e27a700 1 -- 172.21.15.83:0/2260825084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146810b820 con 0x7f14681080b0 2024-03-11T19:30:17.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.983+0000 7f14677fe700 1 --2- 172.21.15.83:0/2260825084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1450038e00 0x7f145003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:17.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.983+0000 7f14677fe700 1 --2- 172.21.15.83:0/2260825084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1450038e00 0x7f145003b2c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f145c006fd0 tx=0x7f145c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:17.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:16.988+0000 7f14657fa700 1 -- 172.21.15.83:0/2260825084 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1458024cf0 con 0x7f14681080b0 2024-03-11T19:30:17.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.289+0000 7f146e27a700 1 -- 172.21.15.83:0/2260825084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f146810b9e0 con 0x7f14681080b0 2024-03-11T19:30:17.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.292+0000 7f14657fa700 1 -- 172.21.15.83:0/2260825084 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f145802c350 con 0x7f14681080b0 2024-03-11T19:30:17.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.294+0000 7f146e27a700 1 -- 172.21.15.83:0/2260825084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1450038e00 msgr2=0x7f145003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:17.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.294+0000 7f146e27a700 1 --2- 172.21.15.83:0/2260825084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1450038e00 0x7f145003b2c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f145c006fd0 tx=0x7f145c006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:17.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.294+0000 7f146e27a700 1 -- 172.21.15.83:0/2260825084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 msgr2=0x7f1468107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:17.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.294+0000 7f146e27a700 1 --2- 172.21.15.83:0/2260825084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f1468107930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1458009130 tx=0x7f1458011b00 comp rx=0 tx=0).stop 2024-03-11T19:30:17.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.294+0000 7f146e27a700 1 -- 172.21.15.83:0/2260825084 shutdown_connections 2024-03-11T19:30:17.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.294+0000 7f146e27a700 1 --2- 172.21.15.83:0/2260825084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1450038e00 0x7f145003b2c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:17.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.294+0000 7f146e27a700 1 --2- 172.21.15.83:0/2260825084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14681080b0 0x7f1468107930 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:17.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.294+0000 7f146e27a700 1 -- 172.21.15.83:0/2260825084 >> 172.21.15.83:0/2260825084 conn(0x7f1468101ac0 msgr2=0x7f14681025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:17.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.294+0000 7f146e27a700 1 -- 172.21.15.83:0/2260825084 shutdown_connections 2024-03-11T19:30:17.860 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-03-11T19:30:17.294+0000 7f146e27a700 1 -- 172.21.15.83:0/2260825084 wait complete. 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout:Or, if you are only running a single cluster on this host: 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:17.861 INFO:teuthology.orchestra.run.smithi083.stdout: ceph telemetry on 2024-03-11T19:30:17.862 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:17.862 INFO:teuthology.orchestra.run.smithi083.stdout:For more information see: 2024-03-11T19:30:17.862 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:17.862 INFO:teuthology.orchestra.run.smithi083.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-11T19:30:17.862 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:17.862 INFO:teuthology.orchestra.run.smithi083.stdout:Bootstrap complete. 2024-03-11T19:30:17.892 INFO:tasks.cephadm:Fetching config... 2024-03-11T19:30:17.893 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:30:17.893 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-11T19:30:17.908 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-11T19:30:17.908 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:30:17.908 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-11T19:30:17.963 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-11T19:30:17.963 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:30:17.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-11T19:30:18.030 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-11T19:30:18.030 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:30:18.030 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-11T19:30:18.085 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-11T19:30:18.086 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDalrRLnJ6uujR52Yhg//lOQLVxM5vMVfa4TgVGNTbPLWijvX11jW9SNDfiZZSzUwVByQ7csArIOOxA5298Wl4P1Mq+IND5LkhG7hH/UqBKzkZ+12n61+SblAoKOEC/L+GSv+gOYKDSkwCYJoR8cq3mu67dx6e7nEX/0rBY6uz0fPAqi8hNuRjylVe5XP46jzyLeAiiTh2Os5D6pGt6flDK7TceawNOR+fR/h61gb8B6DZ0vdMczaqv3UF43vWiERjmBAjVXUzSFzEgEC14/hQzn0w9z4BwtaPNZ8WKR6Ony/vOfEhwbz1JF1MOIZDGE0vbc2idpQ3PWWABGK8HwouHHt5RGrfbIa8NC1gqRYzFfEDNf7XbICzXM7BAv5wx8/gxKn+vEUC0fUcdT2yMRWqpkLtjPbYk6g+0rLIQSRtxRDIom3qhswj/QvDwXJ1Pe+yYNJx1R0HHBXvacIkVHuI7vJG36tNMPjEEyKTUoItZ7GIcLMOrDA5V8+T4J0ps+B8= ceph-839f93c8-dfdd-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-11T19:30:18.170 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDalrRLnJ6uujR52Yhg//lOQLVxM5vMVfa4TgVGNTbPLWijvX11jW9SNDfiZZSzUwVByQ7csArIOOxA5298Wl4P1Mq+IND5LkhG7hH/UqBKzkZ+12n61+SblAoKOEC/L+GSv+gOYKDSkwCYJoR8cq3mu67dx6e7nEX/0rBY6uz0fPAqi8hNuRjylVe5XP46jzyLeAiiTh2Os5D6pGt6flDK7TceawNOR+fR/h61gb8B6DZ0vdMczaqv3UF43vWiERjmBAjVXUzSFzEgEC14/hQzn0w9z4BwtaPNZ8WKR6Ony/vOfEhwbz1JF1MOIZDGE0vbc2idpQ3PWWABGK8HwouHHt5RGrfbIa8NC1gqRYzFfEDNf7XbICzXM7BAv5wx8/gxKn+vEUC0fUcdT2yMRWqpkLtjPbYk6g+0rLIQSRtxRDIom3qhswj/QvDwXJ1Pe+yYNJx1R0HHBXvacIkVHuI7vJG36tNMPjEEyKTUoItZ7GIcLMOrDA5V8+T4J0ps+B8= ceph-839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:30:18.186 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDalrRLnJ6uujR52Yhg//lOQLVxM5vMVfa4TgVGNTbPLWijvX11jW9SNDfiZZSzUwVByQ7csArIOOxA5298Wl4P1Mq+IND5LkhG7hH/UqBKzkZ+12n61+SblAoKOEC/L+GSv+gOYKDSkwCYJoR8cq3mu67dx6e7nEX/0rBY6uz0fPAqi8hNuRjylVe5XP46jzyLeAiiTh2Os5D6pGt6flDK7TceawNOR+fR/h61gb8B6DZ0vdMczaqv3UF43vWiERjmBAjVXUzSFzEgEC14/hQzn0w9z4BwtaPNZ8WKR6Ony/vOfEhwbz1JF1MOIZDGE0vbc2idpQ3PWWABGK8HwouHHt5RGrfbIa8NC1gqRYzFfEDNf7XbICzXM7BAv5wx8/gxKn+vEUC0fUcdT2yMRWqpkLtjPbYk6g+0rLIQSRtxRDIom3qhswj/QvDwXJ1Pe+yYNJx1R0HHBXvacIkVHuI7vJG36tNMPjEEyKTUoItZ7GIcLMOrDA5V8+T4J0ps+B8= ceph-839f93c8-dfdd-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-11T19:30:18.234 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDalrRLnJ6uujR52Yhg//lOQLVxM5vMVfa4TgVGNTbPLWijvX11jW9SNDfiZZSzUwVByQ7csArIOOxA5298Wl4P1Mq+IND5LkhG7hH/UqBKzkZ+12n61+SblAoKOEC/L+GSv+gOYKDSkwCYJoR8cq3mu67dx6e7nEX/0rBY6uz0fPAqi8hNuRjylVe5XP46jzyLeAiiTh2Os5D6pGt6flDK7TceawNOR+fR/h61gb8B6DZ0vdMczaqv3UF43vWiERjmBAjVXUzSFzEgEC14/hQzn0w9z4BwtaPNZ8WKR6Ony/vOfEhwbz1JF1MOIZDGE0vbc2idpQ3PWWABGK8HwouHHt5RGrfbIa8NC1gqRYzFfEDNf7XbICzXM7BAv5wx8/gxKn+vEUC0fUcdT2yMRWqpkLtjPbYk6g+0rLIQSRtxRDIom3qhswj/QvDwXJ1Pe+yYNJx1R0HHBXvacIkVHuI7vJG36tNMPjEEyKTUoItZ7GIcLMOrDA5V8+T4J0ps+B8= ceph-839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:30:18.372 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDalrRLnJ6uujR52Yhg//lOQLVxM5vMVfa4TgVGNTbPLWijvX11jW9SNDfiZZSzUwVByQ7csArIOOxA5298Wl4P1Mq+IND5LkhG7hH/UqBKzkZ+12n61+SblAoKOEC/L+GSv+gOYKDSkwCYJoR8cq3mu67dx6e7nEX/0rBY6uz0fPAqi8hNuRjylVe5XP46jzyLeAiiTh2Os5D6pGt6flDK7TceawNOR+fR/h61gb8B6DZ0vdMczaqv3UF43vWiERjmBAjVXUzSFzEgEC14/hQzn0w9z4BwtaPNZ8WKR6Ony/vOfEhwbz1JF1MOIZDGE0vbc2idpQ3PWWABGK8HwouHHt5RGrfbIa8NC1gqRYzFfEDNf7XbICzXM7BAv5wx8/gxKn+vEUC0fUcdT2yMRWqpkLtjPbYk6g+0rLIQSRtxRDIom3qhswj/QvDwXJ1Pe+yYNJx1R0HHBXvacIkVHuI7vJG36tNMPjEEyKTUoItZ7GIcLMOrDA5V8+T4J0ps+B8= ceph-839f93c8-dfdd-11ee-95c8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-11T19:30:18.421 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDalrRLnJ6uujR52Yhg//lOQLVxM5vMVfa4TgVGNTbPLWijvX11jW9SNDfiZZSzUwVByQ7csArIOOxA5298Wl4P1Mq+IND5LkhG7hH/UqBKzkZ+12n61+SblAoKOEC/L+GSv+gOYKDSkwCYJoR8cq3mu67dx6e7nEX/0rBY6uz0fPAqi8hNuRjylVe5XP46jzyLeAiiTh2Os5D6pGt6flDK7TceawNOR+fR/h61gb8B6DZ0vdMczaqv3UF43vWiERjmBAjVXUzSFzEgEC14/hQzn0w9z4BwtaPNZ8WKR6Ony/vOfEhwbz1JF1MOIZDGE0vbc2idpQ3PWWABGK8HwouHHt5RGrfbIa8NC1gqRYzFfEDNf7XbICzXM7BAv5wx8/gxKn+vEUC0fUcdT2yMRWqpkLtjPbYk6g+0rLIQSRtxRDIom3qhswj/QvDwXJ1Pe+yYNJx1R0HHBXvacIkVHuI7vJG36tNMPjEEyKTUoItZ7GIcLMOrDA5V8+T4J0ps+B8= ceph-839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:30:18.544 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-11T19:30:18.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:18 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2260825084' entity='client.admin' 2024-03-11T19:30:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:30:20.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:20 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:20.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:20 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:20.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:20 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:30:20.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:20 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:20.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:20 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:30:20.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:20 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.063+0000 7f6005a0a700 1 -- 172.21.15.83:0/3202377031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60001016c0 msgr2=0x7f6000101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.063+0000 7f6005a0a700 1 --2- 172.21.15.83:0/3202377031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60001016c0 0x7f6000101ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009910 tx=0x7f5fe8009c20 comp rx=0 tx=0).stop 2024-03-11T19:30:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.063+0000 7f6005a0a700 1 -- 172.21.15.83:0/3202377031 shutdown_connections 2024-03-11T19:30:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.063+0000 7f6005a0a700 1 --2- 172.21.15.83:0/3202377031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60001016c0 0x7f6000101ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.063+0000 7f6005a0a700 1 -- 172.21.15.83:0/3202377031 >> 172.21.15.83:0/3202377031 conn(0x7f60000fcc60 msgr2=0x7f60000ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.063+0000 7f6005a0a700 1 -- 172.21.15.83:0/3202377031 shutdown_connections 2024-03-11T19:30:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.063+0000 7f6005a0a700 1 -- 172.21.15.83:0/3202377031 wait complete. 2024-03-11T19:30:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.064+0000 7f6005a0a700 1 Processor -- start 2024-03-11T19:30:21.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.064+0000 7f6005a0a700 1 -- start start 2024-03-11T19:30:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.064+0000 7f6005a0a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60001016c0 0x7f6000196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.064+0000 7f6005a0a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000197120 con 0x7f60001016c0 2024-03-11T19:30:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.065+0000 7f5ffeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60001016c0 0x7f6000196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.065+0000 7f5ffeffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60001016c0 0x7f6000196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36440/0 (socket says 172.21.15.83:36440) 2024-03-11T19:30:21.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.065+0000 7f5ffeffd700 1 -- 172.21.15.83:0/2216016176 learned_addr learned my addr 172.21.15.83:0/2216016176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.065+0000 7f5ffeffd700 1 -- 172.21.15.83:0/2216016176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe80095f0 con 0x7f60001016c0 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.065+0000 7f5ffeffd700 1 --2- 172.21.15.83:0/2216016176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60001016c0 0x7f6000196be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8005850 tx=0x7f5fe8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.065+0000 7f6004a08700 1 -- 172.21.15.83:0/2216016176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fe801d070 con 0x7f60001016c0 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.065+0000 7f6004a08700 1 -- 172.21.15.83:0/2216016176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fe800be40 con 0x7f60001016c0 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.065+0000 7f6004a08700 1 -- 172.21.15.83:0/2216016176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fe8022ea0 con 0x7f60001016c0 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.065+0000 7f6005a0a700 1 -- 172.21.15.83:0/2216016176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6000197320 con 0x7f60001016c0 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.065+0000 7f6005a0a700 1 -- 172.21.15.83:0/2216016176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60001977c0 con 0x7f60001016c0 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.066+0000 7f6004a08700 1 -- 172.21.15.83:0/2216016176 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f5fe800b960 con 0x7f60001016c0 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.066+0000 7f6005a0a700 1 -- 172.21.15.83:0/2216016176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6000190520 con 0x7f60001016c0 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.066+0000 7f6004a08700 1 --2- 172.21.15.83:0/2216016176 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fec038db0 0x7f5fec03b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.066+0000 7f6004a08700 1 -- 172.21.15.83:0/2216016176 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5fe804cda0 con 0x7f60001016c0 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.066+0000 7f5ffe7fc700 1 --2- 172.21.15.83:0/2216016176 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fec038db0 0x7f5fec03b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:21.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.067+0000 7f5ffe7fc700 1 --2- 172.21.15.83:0/2216016176 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fec038db0 0x7f5fec03b270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0006fd0 tx=0x7f5ff0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:21.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.071+0000 7f6004a08700 1 -- 172.21.15.83:0/2216016176 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5fe8022710 con 0x7f60001016c0 2024-03-11T19:30:21.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.263+0000 7f6005a0a700 1 -- 172.21.15.83:0/2216016176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f60000632a0 con 0x7f60001016c0 2024-03-11T19:30:21.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.266+0000 7f6004a08700 1 -- 172.21.15.83:0/2216016176 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5fe8027030 con 0x7f60001016c0 2024-03-11T19:30:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.268+0000 7f6005a0a700 1 -- 172.21.15.83:0/2216016176 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fec038db0 msgr2=0x7f5fec03b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.268+0000 7f6005a0a700 1 --2- 172.21.15.83:0/2216016176 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fec038db0 0x7f5fec03b270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0006fd0 tx=0x7f5ff0006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.268+0000 7f6005a0a700 1 -- 172.21.15.83:0/2216016176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60001016c0 msgr2=0x7f6000196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.268+0000 7f6005a0a700 1 --2- 172.21.15.83:0/2216016176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60001016c0 0x7f6000196be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8005850 tx=0x7f5fe8004ff0 comp rx=0 tx=0).stop 2024-03-11T19:30:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.268+0000 7f6005a0a700 1 -- 172.21.15.83:0/2216016176 shutdown_connections 2024-03-11T19:30:21.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.268+0000 7f6005a0a700 1 --2- 172.21.15.83:0/2216016176 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fec038db0 0x7f5fec03b270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:21.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.268+0000 7f6005a0a700 1 --2- 172.21.15.83:0/2216016176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f60001016c0 0x7f6000196be0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:21.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.268+0000 7f6005a0a700 1 -- 172.21.15.83:0/2216016176 >> 172.21.15.83:0/2216016176 conn(0x7f60000fcc60 msgr2=0x7f60000fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:21.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.268+0000 7f6005a0a700 1 -- 172.21.15.83:0/2216016176 shutdown_connections 2024-03-11T19:30:21.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:21.268+0000 7f6005a0a700 1 -- 172.21.15.83:0/2216016176 wait complete. 2024-03-11T19:30:21.854 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-11T19:30:21.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-11T19:30:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:30:22.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:22 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2216016176' entity='client.admin' 2024-03-11T19:30:22.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:22 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:30:22.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:22 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.687+0000 7fb710dfb700 1 -- 172.21.15.83:0/1958447149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70c1016c0 msgr2=0x7fb70c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.687+0000 7fb710dfb700 1 --2- 172.21.15.83:0/1958447149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70c1016c0 0x7fb70c101ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009910 tx=0x7fb6fc009c20 comp rx=0 tx=0).stop 2024-03-11T19:30:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.687+0000 7fb710dfb700 1 -- 172.21.15.83:0/1958447149 shutdown_connections 2024-03-11T19:30:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.687+0000 7fb710dfb700 1 --2- 172.21.15.83:0/1958447149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70c1016c0 0x7fb70c101ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.687+0000 7fb710dfb700 1 -- 172.21.15.83:0/1958447149 >> 172.21.15.83:0/1958447149 conn(0x7fb70c0fcc60 msgr2=0x7fb70c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.687+0000 7fb710dfb700 1 -- 172.21.15.83:0/1958447149 shutdown_connections 2024-03-11T19:30:23.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.687+0000 7fb710dfb700 1 -- 172.21.15.83:0/1958447149 wait complete. 2024-03-11T19:30:23.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.688+0000 7fb710dfb700 1 Processor -- start 2024-03-11T19:30:23.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.688+0000 7fb710dfb700 1 -- start start 2024-03-11T19:30:23.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.688+0000 7fb710dfb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70c1016c0 0x7fb70c196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:23.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.688+0000 7fb710dfb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c1971a0 con 0x7fb70c1016c0 2024-03-11T19:30:23.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.688+0000 7fb70a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70c1016c0 0x7fb70c196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.688+0000 7fb70a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70c1016c0 0x7fb70c196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36472/0 (socket says 172.21.15.83:36472) 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.688+0000 7fb70a59c700 1 -- 172.21.15.83:0/952655399 learned_addr learned my addr 172.21.15.83:0/952655399 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.688+0000 7fb70a59c700 1 -- 172.21.15.83:0/952655399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc0095f0 con 0x7fb70c1016c0 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.688+0000 7fb70a59c700 1 --2- 172.21.15.83:0/952655399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70c1016c0 0x7fb70c196c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc005850 tx=0x7fb6fc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.688+0000 7fb6fb7fe700 1 -- 172.21.15.83:0/952655399 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6fc01d070 con 0x7fb70c1016c0 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.689+0000 7fb710dfb700 1 -- 172.21.15.83:0/952655399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70c1973a0 con 0x7fb70c1016c0 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.689+0000 7fb6fb7fe700 1 -- 172.21.15.83:0/952655399 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6fc00be40 con 0x7fb70c1016c0 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.689+0000 7fb6fb7fe700 1 -- 172.21.15.83:0/952655399 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6fc022ea0 con 0x7fb70c1016c0 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.689+0000 7fb710dfb700 1 -- 172.21.15.83:0/952655399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c197840 con 0x7fb70c1016c0 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.689+0000 7fb6fb7fe700 1 -- 172.21.15.83:0/952655399 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fb6fc00f500 con 0x7fb70c1016c0 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.689+0000 7fb710dfb700 1 -- 172.21.15.83:0/952655399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70c190520 con 0x7fb70c1016c0 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.690+0000 7fb6fb7fe700 1 --2- 172.21.15.83:0/952655399 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb6f4038d50 0x7fb6f403b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.690+0000 7fb6fb7fe700 1 -- 172.21.15.83:0/952655399 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb6fc04d090 con 0x7fb70c1016c0 2024-03-11T19:30:23.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.690+0000 7fb709d9b700 1 --2- 172.21.15.83:0/952655399 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb6f4038d50 0x7fb6f403b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.690+0000 7fb709d9b700 1 --2- 172.21.15.83:0/952655399 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb6f4038d50 0x7fb6f403b210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb700006fd0 tx=0x7fb700006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:23.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.694+0000 7fb6fb7fe700 1 -- 172.21.15.83:0/952655399 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb6fc017440 con 0x7fb70c1016c0 2024-03-11T19:30:23.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.892+0000 7fb710dfb700 1 -- 172.21.15.83:0/952655399 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fb70c0620b0 con 0x7fb6f4038d50 2024-03-11T19:30:23.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.895+0000 7fb6fb7fe700 1 -- 172.21.15.83:0/952655399 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb70c0620b0 con 0x7fb6f4038d50 2024-03-11T19:30:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.897+0000 7fb710dfb700 1 -- 172.21.15.83:0/952655399 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb6f4038d50 msgr2=0x7fb6f403b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.897+0000 7fb710dfb700 1 --2- 172.21.15.83:0/952655399 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb6f4038d50 0x7fb6f403b210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb700006fd0 tx=0x7fb700006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.897+0000 7fb710dfb700 1 -- 172.21.15.83:0/952655399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70c1016c0 msgr2=0x7fb70c196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.897+0000 7fb710dfb700 1 --2- 172.21.15.83:0/952655399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70c1016c0 0x7fb70c196c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc005850 tx=0x7fb6fc004ff0 comp rx=0 tx=0).stop 2024-03-11T19:30:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.898+0000 7fb710dfb700 1 -- 172.21.15.83:0/952655399 shutdown_connections 2024-03-11T19:30:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.898+0000 7fb710dfb700 1 --2- 172.21.15.83:0/952655399 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb6f4038d50 0x7fb6f403b210 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.898+0000 7fb710dfb700 1 --2- 172.21.15.83:0/952655399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb70c1016c0 0x7fb70c196c60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.898+0000 7fb710dfb700 1 -- 172.21.15.83:0/952655399 >> 172.21.15.83:0/952655399 conn(0x7fb70c0fcc60 msgr2=0x7fb70c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.898+0000 7fb710dfb700 1 -- 172.21.15.83:0/952655399 shutdown_connections 2024-03-11T19:30:23.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:23.898+0000 7fb710dfb700 1 -- 172.21.15.83:0/952655399 wait complete. 2024-03-11T19:30:24.676 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi137 2024-03-11T19:30:24.677 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-11T19:30:24.677 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.conf 2024-03-11T19:30:24.696 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-11T19:30:24.696 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:30:24.754 INFO:tasks.cephadm:Adding host smithi137 to orchestrator... 2024-03-11T19:30:24.754 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch host add smithi137 2024-03-11T19:30:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:30:25.197 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:24 smithi083 ceph-mon[31864]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:30:25.197 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:25.197 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:30:25.197 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:30:25.197 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:30:25.197 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:24 smithi083 ceph-mon[31864]: Updating smithi083:/etc/ceph/ceph.conf 2024-03-11T19:30:25.197 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:24 smithi083 ceph-mon[31864]: Updating smithi083:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:30:25.197 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:24 smithi083 ceph-mon[31864]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:30:25.197 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:25.197 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:25.198 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:26.196 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:25 smithi083 ceph-mon[31864]: Updating smithi083:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:30:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.384+0000 7f15be9d5700 1 -- 172.21.15.83:0/1231056701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15b8106620 msgr2=0x7f15b8106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.384+0000 7f15be9d5700 1 --2- 172.21.15.83:0/1231056701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15b8106620 0x7f15b8106a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009910 tx=0x7f15ac009c20 comp rx=0 tx=0).stop 2024-03-11T19:30:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.384+0000 7f15be9d5700 1 -- 172.21.15.83:0/1231056701 shutdown_connections 2024-03-11T19:30:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.384+0000 7f15be9d5700 1 --2- 172.21.15.83:0/1231056701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15b8106620 0x7f15b8106a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.384+0000 7f15be9d5700 1 -- 172.21.15.83:0/1231056701 >> 172.21.15.83:0/1231056701 conn(0x7f15b80fc060 msgr2=0x7f15b80fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.384+0000 7f15be9d5700 1 -- 172.21.15.83:0/1231056701 shutdown_connections 2024-03-11T19:30:26.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.384+0000 7f15be9d5700 1 -- 172.21.15.83:0/1231056701 wait complete. 2024-03-11T19:30:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.385+0000 7f15be9d5700 1 Processor -- start 2024-03-11T19:30:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.385+0000 7f15be9d5700 1 -- start start 2024-03-11T19:30:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.385+0000 7f15be9d5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15b8106620 0x7f15b8195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.385+0000 7f15be9d5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b806a8f0 con 0x7f15b8106620 2024-03-11T19:30:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.385+0000 7f15b7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15b8106620 0x7f15b8195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.385+0000 7f15b7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15b8106620 0x7f15b8195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36490/0 (socket says 172.21.15.83:36490) 2024-03-11T19:30:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.385+0000 7f15b7fff700 1 -- 172.21.15.83:0/953169996 learned_addr learned my addr 172.21.15.83:0/953169996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.385+0000 7f15b7fff700 1 -- 172.21.15.83:0/953169996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac0095f0 con 0x7f15b8106620 2024-03-11T19:30:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.385+0000 7f15b7fff700 1 --2- 172.21.15.83:0/953169996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15b8106620 0x7f15b8195ab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f15ac004f40 tx=0x7f15ac005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.386+0000 7f15b57fa700 1 -- 172.21.15.83:0/953169996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15ac01d070 con 0x7f15b8106620 2024-03-11T19:30:26.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.386+0000 7f15b57fa700 1 -- 172.21.15.83:0/953169996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15ac00bc30 con 0x7f15b8106620 2024-03-11T19:30:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.386+0000 7f15be9d5700 1 -- 172.21.15.83:0/953169996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b8195ff0 con 0x7f15b8106620 2024-03-11T19:30:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.386+0000 7f15b57fa700 1 -- 172.21.15.83:0/953169996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15ac003970 con 0x7f15b8106620 2024-03-11T19:30:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.386+0000 7f15be9d5700 1 -- 172.21.15.83:0/953169996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b81964f0 con 0x7f15b8106620 2024-03-11T19:30:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.386+0000 7f15b57fa700 1 -- 172.21.15.83:0/953169996 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f15ac003c30 con 0x7f15b8106620 2024-03-11T19:30:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.386+0000 7f15be9d5700 1 -- 172.21.15.83:0/953169996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b8101c90 con 0x7f15b8106620 2024-03-11T19:30:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.387+0000 7f15b57fa700 1 --2- 172.21.15.83:0/953169996 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f15a0038d50 0x7f15a003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.387+0000 7f15b57fa700 1 -- 172.21.15.83:0/953169996 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f15ac04cb20 con 0x7f15b8106620 2024-03-11T19:30:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.387+0000 7f15b77fe700 1 --2- 172.21.15.83:0/953169996 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f15a0038d50 0x7f15a003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:26.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.387+0000 7f15b77fe700 1 --2- 172.21.15.83:0/953169996 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f15a0038d50 0x7f15a003b210 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f15a8006fd0 tx=0x7f15a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:26.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.391+0000 7f15b57fa700 1 -- 172.21.15.83:0/953169996 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f15ac029950 con 0x7f15b8106620 2024-03-11T19:30:26.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:26.590+0000 7f15be9d5700 1 -- 172.21.15.83:0/953169996 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}) v1 -- 0x7f15b81967b0 con 0x7f15a0038d50 2024-03-11T19:30:28.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:27 smithi083 ceph-mon[31864]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:30:28.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:27 smithi083 ceph-mon[31864]: Deploying cephadm binary to smithi137 2024-03-11T19:30:29.722 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:29 smithi083 ceph-mon[31864]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:30.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:30.628+0000 7f15b57fa700 1 -- 172.21.15.83:0/953169996 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f15b81967b0 con 0x7f15a0038d50 2024-03-11T19:30:30.632 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi137' with addr '172.21.15.137' 2024-03-11T19:30:30.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:30.630+0000 7f15be9d5700 1 -- 172.21.15.83:0/953169996 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f15a0038d50 msgr2=0x7f15a003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:30.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:30.630+0000 7f15be9d5700 1 --2- 172.21.15.83:0/953169996 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f15a0038d50 0x7f15a003b210 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f15a8006fd0 tx=0x7f15a8006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:30.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:30.630+0000 7f15be9d5700 1 -- 172.21.15.83:0/953169996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15b8106620 msgr2=0x7f15b8195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:30.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:30.630+0000 7f15be9d5700 1 --2- 172.21.15.83:0/953169996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15b8106620 0x7f15b8195ab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f15ac004f40 tx=0x7f15ac005020 comp rx=0 tx=0).stop 2024-03-11T19:30:30.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:30.631+0000 7f15be9d5700 1 -- 172.21.15.83:0/953169996 shutdown_connections 2024-03-11T19:30:30.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:30.631+0000 7f15be9d5700 1 --2- 172.21.15.83:0/953169996 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f15a0038d50 0x7f15a003b210 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:30.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:30.631+0000 7f15be9d5700 1 --2- 172.21.15.83:0/953169996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15b8106620 0x7f15b8195ab0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:30.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:30.631+0000 7f15be9d5700 1 -- 172.21.15.83:0/953169996 >> 172.21.15.83:0/953169996 conn(0x7f15b80fc060 msgr2=0x7f15b80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:30.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:30.631+0000 7f15be9d5700 1 -- 172.21.15.83:0/953169996 shutdown_connections 2024-03-11T19:30:30.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:30.631+0000 7f15be9d5700 1 -- 172.21.15.83:0/953169996 wait complete. 2024-03-11T19:30:31.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch host ls --format=json 2024-03-11T19:30:31.519 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:30:31.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:31 smithi083 ceph-mon[31864]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:31.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:31.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:30:31.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:32.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:32 smithi083 ceph-mon[31864]: Added host smithi137 2024-03-11T19:30:32.572 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:32.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.987+0000 7f2c60b7d700 1 -- 172.21.15.83:0/1375674357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c102810 msgr2=0x7f2c5c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:32.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.987+0000 7f2c60b7d700 1 --2- 172.21.15.83:0/1375674357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c102810 0x7f2c5c102bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009910 tx=0x7f2c54009c20 comp rx=0 tx=0).stop 2024-03-11T19:30:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.987+0000 7f2c60b7d700 1 -- 172.21.15.83:0/1375674357 shutdown_connections 2024-03-11T19:30:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.987+0000 7f2c60b7d700 1 --2- 172.21.15.83:0/1375674357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c102810 0x7f2c5c102bf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.987+0000 7f2c60b7d700 1 -- 172.21.15.83:0/1375674357 >> 172.21.15.83:0/1375674357 conn(0x7f2c5c0fc060 msgr2=0x7f2c5c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.988+0000 7f2c60b7d700 1 -- 172.21.15.83:0/1375674357 shutdown_connections 2024-03-11T19:30:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.988+0000 7f2c60b7d700 1 -- 172.21.15.83:0/1375674357 wait complete. 2024-03-11T19:30:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.988+0000 7f2c60b7d700 1 Processor -- start 2024-03-11T19:30:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.988+0000 7f2c60b7d700 1 -- start start 2024-03-11T19:30:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.988+0000 7f2c60b7d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c102810 0x7f2c5c197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.988+0000 7f2c60b7d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c1975b0 con 0x7f2c5c102810 2024-03-11T19:30:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.989+0000 7f2c5a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c102810 0x7f2c5c197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.989+0000 7f2c5a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c102810 0x7f2c5c197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38740/0 (socket says 172.21.15.83:38740) 2024-03-11T19:30:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.989+0000 7f2c5a59c700 1 -- 172.21.15.83:0/641906298 learned_addr learned my addr 172.21.15.83:0/641906298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.989+0000 7f2c5a59c700 1 -- 172.21.15.83:0/641906298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c540095f0 con 0x7f2c5c102810 2024-03-11T19:30:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.989+0000 7f2c5a59c700 1 --2- 172.21.15.83:0/641906298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c102810 0x7f2c5c197070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c54006010 tx=0x7f2c54004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.989+0000 7f2c4b7fe700 1 -- 172.21.15.83:0/641906298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c5401d070 con 0x7f2c5c102810 2024-03-11T19:30:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.989+0000 7f2c4b7fe700 1 -- 172.21.15.83:0/641906298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c5400bc30 con 0x7f2c5c102810 2024-03-11T19:30:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.989+0000 7f2c4b7fe700 1 -- 172.21.15.83:0/641906298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2c540039a0 con 0x7f2c5c102810 2024-03-11T19:30:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.989+0000 7f2c60b7d700 1 -- 172.21.15.83:0/641906298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c1977b0 con 0x7f2c5c102810 2024-03-11T19:30:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.989+0000 7f2c60b7d700 1 -- 172.21.15.83:0/641906298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c197bd0 con 0x7f2c5c102810 2024-03-11T19:30:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.990+0000 7f2c4b7fe700 1 -- 172.21.15.83:0/641906298 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f2c54003c20 con 0x7f2c5c102810 2024-03-11T19:30:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.990+0000 7f2c60b7d700 1 -- 172.21.15.83:0/641906298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5c06a3d0 con 0x7f2c5c102810 2024-03-11T19:30:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.990+0000 7f2c4b7fe700 1 --2- 172.21.15.83:0/641906298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c44038d50 0x7f2c4403b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.991+0000 7f2c4b7fe700 1 -- 172.21.15.83:0/641906298 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2c5404cb20 con 0x7f2c5c102810 2024-03-11T19:30:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.991+0000 7f2c59d9b700 1 --2- 172.21.15.83:0/641906298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c44038d50 0x7f2c4403b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.991+0000 7f2c59d9b700 1 --2- 172.21.15.83:0/641906298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c44038d50 0x7f2c4403b210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2c50006fd0 tx=0x7f2c50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:32.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:32.995+0000 7f2c4b7fe700 1 -- 172.21.15.83:0/641906298 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2c5400ba70 con 0x7f2c5c102810 2024-03-11T19:30:33.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.192+0000 7f2c60b7d700 1 -- 172.21.15.83:0/641906298 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c5c10bb60 con 0x7f2c44038d50 2024-03-11T19:30:33.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.193+0000 7f2c4b7fe700 1 -- 172.21.15.83:0/641906298 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f2c5c10bb60 con 0x7f2c44038d50 2024-03-11T19:30:33.197 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:33.197 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}] 2024-03-11T19:30:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.196+0000 7f2c60b7d700 1 -- 172.21.15.83:0/641906298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c44038d50 msgr2=0x7f2c4403b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.196+0000 7f2c60b7d700 1 --2- 172.21.15.83:0/641906298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c44038d50 0x7f2c4403b210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2c50006fd0 tx=0x7f2c50006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.196+0000 7f2c60b7d700 1 -- 172.21.15.83:0/641906298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c102810 msgr2=0x7f2c5c197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.196+0000 7f2c60b7d700 1 --2- 172.21.15.83:0/641906298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c102810 0x7f2c5c197070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c54006010 tx=0x7f2c54004ff0 comp rx=0 tx=0).stop 2024-03-11T19:30:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.196+0000 7f2c60b7d700 1 -- 172.21.15.83:0/641906298 shutdown_connections 2024-03-11T19:30:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.196+0000 7f2c60b7d700 1 --2- 172.21.15.83:0/641906298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c44038d50 0x7f2c4403b210 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.196+0000 7f2c60b7d700 1 --2- 172.21.15.83:0/641906298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c5c102810 0x7f2c5c197070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.196+0000 7f2c60b7d700 1 -- 172.21.15.83:0/641906298 >> 172.21.15.83:0/641906298 conn(0x7f2c5c0fc060 msgr2=0x7f2c5c1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.196+0000 7f2c60b7d700 1 -- 172.21.15.83:0/641906298 shutdown_connections 2024-03-11T19:30:33.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:33.196+0000 7f2c60b7d700 1 -- 172.21.15.83:0/641906298 wait complete. 2024-03-11T19:30:33.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:33 smithi083 ceph-mon[31864]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:33.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:33.838 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-03-11T19:30:33.838 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-11T19:30:33.838 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-03-11T19:30:33.857 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-11T19:30:33.857 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:30:33.915 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-03-11T19:30:33.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch host add smithi204 2024-03-11T19:30:34.147 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:30:34.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:34 smithi083 ceph-mon[31864]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:30:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:35 smithi083 ceph-mon[31864]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:35.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.603+0000 7f5edef45700 1 -- 172.21.15.83:0/2749057457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed8107650 msgr2=0x7f5ed8107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:35.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.603+0000 7f5edef45700 1 --2- 172.21.15.83:0/2749057457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed8107650 0x7f5ed8107a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009910 tx=0x7f5ecc009c20 comp rx=0 tx=0).stop 2024-03-11T19:30:35.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.603+0000 7f5edef45700 1 -- 172.21.15.83:0/2749057457 shutdown_connections 2024-03-11T19:30:35.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.603+0000 7f5edef45700 1 --2- 172.21.15.83:0/2749057457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed8107650 0x7f5ed8107a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:35.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.603+0000 7f5edef45700 1 -- 172.21.15.83:0/2749057457 >> 172.21.15.83:0/2749057457 conn(0x7f5ed80fd180 msgr2=0x7f5ed80ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:35.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.603+0000 7f5edef45700 1 -- 172.21.15.83:0/2749057457 shutdown_connections 2024-03-11T19:30:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.603+0000 7f5edef45700 1 -- 172.21.15.83:0/2749057457 wait complete. 2024-03-11T19:30:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.603+0000 7f5edef45700 1 Processor -- start 2024-03-11T19:30:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.603+0000 7f5edef45700 1 -- start start 2024-03-11T19:30:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.603+0000 7f5edef45700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed8107650 0x7f5ed8196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.603+0000 7f5edef45700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed81974e0 con 0x7f5ed8107650 2024-03-11T19:30:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.604+0000 7f5edcce1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed8107650 0x7f5ed8196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.604+0000 7f5edcce1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed8107650 0x7f5ed8196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38746/0 (socket says 172.21.15.83:38746) 2024-03-11T19:30:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.604+0000 7f5edcce1700 1 -- 172.21.15.83:0/4189030199 learned_addr learned my addr 172.21.15.83:0/4189030199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.604+0000 7f5edcce1700 1 -- 172.21.15.83:0/4189030199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc0095f0 con 0x7f5ed8107650 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.604+0000 7f5edcce1700 1 --2- 172.21.15.83:0/4189030199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed8107650 0x7f5ed8196fa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc004f40 tx=0x7f5ecc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.604+0000 7f5ed5ffb700 1 -- 172.21.15.83:0/4189030199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ecc01d070 con 0x7f5ed8107650 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.604+0000 7f5ed5ffb700 1 -- 172.21.15.83:0/4189030199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ecc00bbd0 con 0x7f5ed8107650 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.604+0000 7f5edef45700 1 -- 172.21.15.83:0/4189030199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed8197740 con 0x7f5ed8107650 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.604+0000 7f5ed5ffb700 1 -- 172.21.15.83:0/4189030199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ecc003900 con 0x7f5ed8107650 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.604+0000 7f5edef45700 1 -- 172.21.15.83:0/4189030199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed8197bc0 con 0x7f5ed8107650 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.605+0000 7f5ed5ffb700 1 -- 172.21.15.83:0/4189030199 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f5ecc003bc0 con 0x7f5ed8107650 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.605+0000 7f5edef45700 1 -- 172.21.15.83:0/4189030199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed8102d90 con 0x7f5ed8107650 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.605+0000 7f5ed5ffb700 1 --2- 172.21.15.83:0/4189030199 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ec8038d50 0x7f5ec803b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.605+0000 7f5ed5ffb700 1 -- 172.21.15.83:0/4189030199 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5ecc04cbe0 con 0x7f5ed8107650 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.606+0000 7f5ed7fff700 1 --2- 172.21.15.83:0/4189030199 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ec8038d50 0x7f5ec803b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.606+0000 7f5ed7fff700 1 --2- 172.21.15.83:0/4189030199 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ec8038d50 0x7f5ec803b210 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4006fd0 tx=0x7f5ec4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.610+0000 7f5ed5ffb700 1 -- 172.21.15.83:0/4189030199 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ecc02ae20 con 0x7f5ed8107650 2024-03-11T19:30:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:35.812+0000 7f5edef45700 1 -- 172.21.15.83:0/4189030199 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ed810bba0 con 0x7f5ec8038d50 2024-03-11T19:30:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:30:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:30:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:30:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-11T19:30:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:30:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: Updating smithi137:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:30:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:30:37.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: Updating smithi137:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:30:37.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: Deploying cephadm binary to smithi204 2024-03-11T19:30:37.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:37.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:37.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:37.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:39.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:39 smithi083 ceph-mon[31864]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:39.859+0000 7f5ed5ffb700 1 -- 172.21.15.83:0/4189030199 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5ed810bba0 con 0x7f5ec8038d50 2024-03-11T19:30:39.864 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-03-11T19:30:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:39.862+0000 7f5edef45700 1 -- 172.21.15.83:0/4189030199 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ec8038d50 msgr2=0x7f5ec803b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:39.862+0000 7f5edef45700 1 --2- 172.21.15.83:0/4189030199 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ec8038d50 0x7f5ec803b210 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4006fd0 tx=0x7f5ec4006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:39.862+0000 7f5edef45700 1 -- 172.21.15.83:0/4189030199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed8107650 msgr2=0x7f5ed8196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:39.862+0000 7f5edef45700 1 --2- 172.21.15.83:0/4189030199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed8107650 0x7f5ed8196fa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc004f40 tx=0x7f5ecc005020 comp rx=0 tx=0).stop 2024-03-11T19:30:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:39.862+0000 7f5edef45700 1 -- 172.21.15.83:0/4189030199 shutdown_connections 2024-03-11T19:30:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:39.862+0000 7f5edef45700 1 --2- 172.21.15.83:0/4189030199 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ec8038d50 0x7f5ec803b210 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:39.862+0000 7f5edef45700 1 --2- 172.21.15.83:0/4189030199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed8107650 0x7f5ed8196fa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:39.862+0000 7f5edef45700 1 -- 172.21.15.83:0/4189030199 >> 172.21.15.83:0/4189030199 conn(0x7f5ed80fd180 msgr2=0x7f5ed8109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:39.862+0000 7f5edef45700 1 -- 172.21.15.83:0/4189030199 shutdown_connections 2024-03-11T19:30:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:39.862+0000 7f5edef45700 1 -- 172.21.15.83:0/4189030199 wait complete. 2024-03-11T19:30:40.494 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch host ls --format=json 2024-03-11T19:30:40.727 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:30:41.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:40 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:41.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:40 smithi083 ceph-mon[31864]: Added host smithi204 2024-03-11T19:30:41.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:40 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:30:41.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:40 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:41.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:40 smithi083 ceph-mon[31864]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:41.979 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.182+0000 7f74da6dc700 1 -- 172.21.15.83:0/2348750954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d4106600 msgr2=0x7f74d41069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.182+0000 7f74da6dc700 1 --2- 172.21.15.83:0/2348750954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d4106600 0x7f74d41069e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009910 tx=0x7f74c4009c20 comp rx=0 tx=0).stop 2024-03-11T19:30:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.182+0000 7f74da6dc700 1 -- 172.21.15.83:0/2348750954 shutdown_connections 2024-03-11T19:30:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.182+0000 7f74da6dc700 1 --2- 172.21.15.83:0/2348750954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d4106600 0x7f74d41069e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.182+0000 7f74da6dc700 1 -- 172.21.15.83:0/2348750954 >> 172.21.15.83:0/2348750954 conn(0x7f74d40fc040 msgr2=0x7f74d40fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:42.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.182+0000 7f74da6dc700 1 -- 172.21.15.83:0/2348750954 shutdown_connections 2024-03-11T19:30:42.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.182+0000 7f74da6dc700 1 -- 172.21.15.83:0/2348750954 wait complete. 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.183+0000 7f74da6dc700 1 Processor -- start 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.183+0000 7f74da6dc700 1 -- start start 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.183+0000 7f74da6dc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d4106600 0x7f74d40fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.183+0000 7f74da6dc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d40fe390 con 0x7f74d4106600 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.183+0000 7f74d3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d4106600 0x7f74d40fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.183+0000 7f74d3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d4106600 0x7f74d40fc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44338/0 (socket says 172.21.15.83:44338) 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.183+0000 7f74d3fff700 1 -- 172.21.15.83:0/1807861310 learned_addr learned my addr 172.21.15.83:0/1807861310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.184+0000 7f74d3fff700 1 -- 172.21.15.83:0/1807861310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c40095f0 con 0x7f74d4106600 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.184+0000 7f74d3fff700 1 --2- 172.21.15.83:0/1807861310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d4106600 0x7f74d40fc4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f74c4004f40 tx=0x7f74c4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.184+0000 7f74d17fa700 1 -- 172.21.15.83:0/1807861310 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74c401d070 con 0x7f74d4106600 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.184+0000 7f74da6dc700 1 -- 172.21.15.83:0/1807861310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d40fca30 con 0x7f74d4106600 2024-03-11T19:30:42.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.184+0000 7f74d17fa700 1 -- 172.21.15.83:0/1807861310 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74c400bc30 con 0x7f74d4106600 2024-03-11T19:30:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.184+0000 7f74d17fa700 1 -- 172.21.15.83:0/1807861310 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74c4003970 con 0x7f74d4106600 2024-03-11T19:30:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.184+0000 7f74da6dc700 1 -- 172.21.15.83:0/1807861310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d40fceb0 con 0x7f74d4106600 2024-03-11T19:30:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.185+0000 7f74d17fa700 1 -- 172.21.15.83:0/1807861310 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f74c4003c30 con 0x7f74d4106600 2024-03-11T19:30:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.185+0000 7f74da6dc700 1 -- 172.21.15.83:0/1807861310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d4101c70 con 0x7f74d4106600 2024-03-11T19:30:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.185+0000 7f74d17fa700 1 --2- 172.21.15.83:0/1807861310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f74bc038d50 0x7f74bc03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.185+0000 7f74d17fa700 1 -- 172.21.15.83:0/1807861310 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f74c404cab0 con 0x7f74d4106600 2024-03-11T19:30:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.185+0000 7f74d37fe700 1 --2- 172.21.15.83:0/1807861310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f74bc038d50 0x7f74bc03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:42.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.185+0000 7f74d37fe700 1 --2- 172.21.15.83:0/1807861310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f74bc038d50 0x7f74bc03b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f74c8006fd0 tx=0x7f74c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:42.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.190+0000 7f74d17fa700 1 -- 172.21.15.83:0/1807861310 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74c4029950 con 0x7f74d4106600 2024-03-11T19:30:42.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.385+0000 7f74da6dc700 1 -- 172.21.15.83:0/1807861310 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74d40008d0 con 0x7f74bc038d50 2024-03-11T19:30:42.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.386+0000 7f74d17fa700 1 -- 172.21.15.83:0/1807861310 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f74d40008d0 con 0x7f74bc038d50 2024-03-11T19:30:42.390 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:30:42.390 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-03-11T19:30:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.388+0000 7f74da6dc700 1 -- 172.21.15.83:0/1807861310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f74bc038d50 msgr2=0x7f74bc03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.388+0000 7f74da6dc700 1 --2- 172.21.15.83:0/1807861310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f74bc038d50 0x7f74bc03b210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f74c8006fd0 tx=0x7f74c8006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.388+0000 7f74da6dc700 1 -- 172.21.15.83:0/1807861310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d4106600 msgr2=0x7f74d40fc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.388+0000 7f74da6dc700 1 --2- 172.21.15.83:0/1807861310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d4106600 0x7f74d40fc4f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f74c4004f40 tx=0x7f74c4005020 comp rx=0 tx=0).stop 2024-03-11T19:30:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.388+0000 7f74da6dc700 1 -- 172.21.15.83:0/1807861310 shutdown_connections 2024-03-11T19:30:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.388+0000 7f74da6dc700 1 --2- 172.21.15.83:0/1807861310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f74bc038d50 0x7f74bc03b210 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.388+0000 7f74da6dc700 1 --2- 172.21.15.83:0/1807861310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d4106600 0x7f74d40fc4f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:42.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.388+0000 7f74da6dc700 1 -- 172.21.15.83:0/1807861310 >> 172.21.15.83:0/1807861310 conn(0x7f74d40fc040 msgr2=0x7f74d4069a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:42.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.388+0000 7f74da6dc700 1 -- 172.21.15.83:0/1807861310 shutdown_connections 2024-03-11T19:30:42.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:42.388+0000 7f74da6dc700 1 -- 172.21.15.83:0/1807861310 wait complete. 2024-03-11T19:30:42.981 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:42 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:42.981 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:42 smithi083 ceph-mon[31864]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:30:42.981 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:42 smithi083 ceph-mon[31864]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:43.007 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-11T19:30:43.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd crush tunables default 2024-03-11T19:30:43.236 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:30:44.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.796+0000 7f07c6b8c700 1 -- 172.21.15.83:0/87490924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0106620 msgr2=0x7f07c0106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:44.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.796+0000 7f07c6b8c700 1 --2- 172.21.15.83:0/87490924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0106620 0x7f07c0106a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009910 tx=0x7f07b4009c20 comp rx=0 tx=0).stop 2024-03-11T19:30:44.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.796+0000 7f07c6b8c700 1 -- 172.21.15.83:0/87490924 shutdown_connections 2024-03-11T19:30:44.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.796+0000 7f07c6b8c700 1 --2- 172.21.15.83:0/87490924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0106620 0x7f07c0106a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:44.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.796+0000 7f07c6b8c700 1 -- 172.21.15.83:0/87490924 >> 172.21.15.83:0/87490924 conn(0x7f07c00fc060 msgr2=0x7f07c00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:44.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.796+0000 7f07c6b8c700 1 -- 172.21.15.83:0/87490924 shutdown_connections 2024-03-11T19:30:44.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.796+0000 7f07c6b8c700 1 -- 172.21.15.83:0/87490924 wait complete. 2024-03-11T19:30:44.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.797+0000 7f07c6b8c700 1 Processor -- start 2024-03-11T19:30:44.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.797+0000 7f07c6b8c700 1 -- start start 2024-03-11T19:30:44.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.797+0000 7f07c6b8c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0106620 0x7f07c01021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:44.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.797+0000 7f07c6b8c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c01040d0 con 0x7f07c0106620 2024-03-11T19:30:44.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.797+0000 7f07c4928700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0106620 0x7f07c01021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:44.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.797+0000 7f07c4928700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0106620 0x7f07c01021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44348/0 (socket says 172.21.15.83:44348) 2024-03-11T19:30:44.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.797+0000 7f07c4928700 1 -- 172.21.15.83:0/2662206832 learned_addr learned my addr 172.21.15.83:0/2662206832 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:30:44.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.798+0000 7f07c4928700 1 -- 172.21.15.83:0/2662206832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b40095f0 con 0x7f07c0106620 2024-03-11T19:30:44.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.798+0000 7f07c4928700 1 --2- 172.21.15.83:0/2662206832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0106620 0x7f07c01021e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f07b4004f40 tx=0x7f07b4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.798+0000 7f07bdffb700 1 -- 172.21.15.83:0/2662206832 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07b401d070 con 0x7f07c0106620 2024-03-11T19:30:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.798+0000 7f07bdffb700 1 -- 172.21.15.83:0/2662206832 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07b400bc30 con 0x7f07c0106620 2024-03-11T19:30:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.798+0000 7f07c6b8c700 1 -- 172.21.15.83:0/2662206832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c0102720 con 0x7f07c0106620 2024-03-11T19:30:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.798+0000 7f07bdffb700 1 -- 172.21.15.83:0/2662206832 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07b4003970 con 0x7f07c0106620 2024-03-11T19:30:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.798+0000 7f07c6b8c700 1 -- 172.21.15.83:0/2662206832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c0102ba0 con 0x7f07c0106620 2024-03-11T19:30:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.799+0000 7f07bdffb700 1 -- 172.21.15.83:0/2662206832 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f07b4003c30 con 0x7f07c0106620 2024-03-11T19:30:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.799+0000 7f07c6b8c700 1 -- 172.21.15.83:0/2662206832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c00632a0 con 0x7f07c0106620 2024-03-11T19:30:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.799+0000 7f07bdffb700 1 --2- 172.21.15.83:0/2662206832 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f07b0038d50 0x7f07b003b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.799+0000 7f07bdffb700 1 -- 172.21.15.83:0/2662206832 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f07b404cc50 con 0x7f07c0106620 2024-03-11T19:30:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.799+0000 7f07bffff700 1 --2- 172.21.15.83:0/2662206832 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f07b0038d50 0x7f07b003b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:44.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.799+0000 7f07bffff700 1 --2- 172.21.15.83:0/2662206832 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f07b0038d50 0x7f07b003b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f07ac006fd0 tx=0x7f07ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:44.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:44.804+0000 7f07bdffb700 1 -- 172.21.15.83:0/2662206832 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f07b4052020 con 0x7f07c0106620 2024-03-11T19:30:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.008+0000 7f07c6b8c700 1 -- 172.21.15.83:0/2662206832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f07c0102df0 con 0x7f07c0106620 2024-03-11T19:30:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:45 smithi083 ceph-mon[31864]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:30:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:30:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:30:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:45 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2662206832' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-11T19:30:45.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.974+0000 7f07bdffb700 1 -- 172.21.15.83:0/2662206832 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f07b4026070 con 0x7f07c0106620 2024-03-11T19:30:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.976+0000 7f07c6b8c700 1 -- 172.21.15.83:0/2662206832 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f07b0038d50 msgr2=0x7f07b003b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.976+0000 7f07c6b8c700 1 --2- 172.21.15.83:0/2662206832 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f07b0038d50 0x7f07b003b210 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f07ac006fd0 tx=0x7f07ac006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.976+0000 7f07c6b8c700 1 -- 172.21.15.83:0/2662206832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0106620 msgr2=0x7f07c01021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.976+0000 7f07c6b8c700 1 --2- 172.21.15.83:0/2662206832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0106620 0x7f07c01021e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f07b4004f40 tx=0x7f07b4005020 comp rx=0 tx=0).stop 2024-03-11T19:30:45.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.977+0000 7f07c6b8c700 1 -- 172.21.15.83:0/2662206832 shutdown_connections 2024-03-11T19:30:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.977+0000 7f07c6b8c700 1 --2- 172.21.15.83:0/2662206832 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f07b0038d50 0x7f07b003b210 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.977+0000 7f07c6b8c700 1 --2- 172.21.15.83:0/2662206832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0106620 0x7f07c01021e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.977+0000 7f07c6b8c700 1 -- 172.21.15.83:0/2662206832 >> 172.21.15.83:0/2662206832 conn(0x7f07c00fc060 msgr2=0x7f07c00fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.977+0000 7f07c6b8c700 1 -- 172.21.15.83:0/2662206832 shutdown_connections 2024-03-11T19:30:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:30:45.977+0000 7f07c6b8c700 1 -- 172.21.15.83:0/2662206832 wait complete. 2024-03-11T19:30:45.982 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-03-11T19:30:46.587 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:46 smithi083 ceph-mon[31864]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-11T19:30:46.587 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:46 smithi083 ceph-mon[31864]: Updating smithi204:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:30:46.587 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:46 smithi083 ceph-mon[31864]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:30:46.587 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:46.587 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:46.587 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:46.587 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:46 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2662206832' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-11T19:30:46.587 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:46 smithi083 ceph-mon[31864]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:30:46.612 INFO:tasks.cephadm:Adding mon.a on smithi083 2024-03-11T19:30:46.613 INFO:tasks.cephadm:Adding mon.b on smithi137 2024-03-11T19:30:46.613 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-03-11T19:30:46.613 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch apply mon '3;smithi083:172.21.15.83=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c' 2024-03-11T19:30:46.846 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:30:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:47 smithi083 ceph-mon[31864]: Updating smithi204:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.client.admin.keyring 2024-03-11T19:30:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:47 smithi083 ceph-mon[31864]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:48.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.320+0000 7fe0c1185700 1 -- 172.21.15.204:0/1684671397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0bc106620 msgr2=0x7fe0bc106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.320+0000 7fe0c1185700 1 --2- 172.21.15.204:0/1684671397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0bc106620 0x7fe0bc106a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009910 tx=0x7fe0ac009c20 comp rx=0 tx=0).stop 2024-03-11T19:30:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.320+0000 7fe0c1185700 1 -- 172.21.15.204:0/1684671397 shutdown_connections 2024-03-11T19:30:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.320+0000 7fe0c1185700 1 --2- 172.21.15.204:0/1684671397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0bc106620 0x7fe0bc106a00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.320+0000 7fe0c1185700 1 -- 172.21.15.204:0/1684671397 >> 172.21.15.204:0/1684671397 conn(0x7fe0bc0fc060 msgr2=0x7fe0bc0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.320+0000 7fe0c1185700 1 -- 172.21.15.204:0/1684671397 shutdown_connections 2024-03-11T19:30:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.320+0000 7fe0c1185700 1 -- 172.21.15.204:0/1684671397 wait complete. 2024-03-11T19:30:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.321+0000 7fe0c1185700 1 Processor -- start 2024-03-11T19:30:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.321+0000 7fe0c1185700 1 -- start start 2024-03-11T19:30:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.321+0000 7fe0c1185700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0bc106620 0x7fe0bc19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:48.325 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.321+0000 7fe0c1185700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc19ae10 con 0x7fe0bc106620 2024-03-11T19:30:48.327 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.321+0000 7fe0bad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0bc106620 0x7fe0bc19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:48.327 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.321+0000 7fe0bad9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0bc106620 0x7fe0bc19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.204:34720/0 (socket says 172.21.15.204:34720) 2024-03-11T19:30:48.327 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.321+0000 7fe0bad9d700 1 -- 172.21.15.204:0/3001364229 learned_addr learned my addr 172.21.15.204:0/3001364229 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-11T19:30:48.327 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.322+0000 7fe0bad9d700 1 -- 172.21.15.204:0/3001364229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac0095f0 con 0x7fe0bc106620 2024-03-11T19:30:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.322+0000 7fe0bad9d700 1 --2- 172.21.15.204:0/3001364229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0bc106620 0x7fe0bc19a8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac004f40 tx=0x7fe0ac005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.322+0000 7fe0a3fff700 1 -- 172.21.15.204:0/3001364229 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0ac01d070 con 0x7fe0bc106620 2024-03-11T19:30:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.322+0000 7fe0c1185700 1 -- 172.21.15.204:0/3001364229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0bc19b070 con 0x7fe0bc106620 2024-03-11T19:30:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.322+0000 7fe0a3fff700 1 -- 172.21.15.204:0/3001364229 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac00bc30 con 0x7fe0bc106620 2024-03-11T19:30:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.322+0000 7fe0c1185700 1 -- 172.21.15.204:0/3001364229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc194a90 con 0x7fe0bc106620 2024-03-11T19:30:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.322+0000 7fe0a3fff700 1 -- 172.21.15.204:0/3001364229 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0ac003960 con 0x7fe0bc106620 2024-03-11T19:30:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.323+0000 7fe0a3fff700 1 -- 172.21.15.204:0/3001364229 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fe0ac00b6f0 con 0x7fe0bc106620 2024-03-11T19:30:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.323+0000 7fe0c1185700 1 -- 172.21.15.204:0/3001364229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0bc101c90 con 0x7fe0bc106620 2024-03-11T19:30:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.323+0000 7fe0a3fff700 1 --2- 172.21.15.204:0/3001364229 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe0a403afb0 0x7fe0a403d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.323+0000 7fe0a3fff700 1 -- 172.21.15.204:0/3001364229 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe0ac04ca70 con 0x7fe0bc106620 2024-03-11T19:30:48.328 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.324+0000 7fe0ba59c700 1 --2- 172.21.15.204:0/3001364229 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe0a403afb0 0x7fe0a403d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:48.329 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.324+0000 7fe0ba59c700 1 --2- 172.21.15.204:0/3001364229 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe0a403afb0 0x7fe0a403d470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0006fd0 tx=0x7fe0b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:48.332 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.329+0000 7fe0a3fff700 1 -- 172.21.15.204:0/3001364229 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe0ac004390 con 0x7fe0bc106620 2024-03-11T19:30:48.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.533+0000 7fe0c1185700 1 -- 172.21.15.204:0/3001364229 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0bc109890 con 0x7fe0a403afb0 2024-03-11T19:30:48.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.540+0000 7fe0a3fff700 1 -- 172.21.15.204:0/3001364229 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe0bc109890 con 0x7fe0a403afb0 2024-03-11T19:30:48.544 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-03-11T19:30:48.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.542+0000 7fe0c1185700 1 -- 172.21.15.204:0/3001364229 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe0a403afb0 msgr2=0x7fe0a403d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:48.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.542+0000 7fe0c1185700 1 --2- 172.21.15.204:0/3001364229 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe0a403afb0 0x7fe0a403d470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0006fd0 tx=0x7fe0b0006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:48.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.542+0000 7fe0c1185700 1 -- 172.21.15.204:0/3001364229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0bc106620 msgr2=0x7fe0bc19a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:48.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.542+0000 7fe0c1185700 1 --2- 172.21.15.204:0/3001364229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0bc106620 0x7fe0bc19a8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac004f40 tx=0x7fe0ac005020 comp rx=0 tx=0).stop 2024-03-11T19:30:48.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.542+0000 7fe0c1185700 1 -- 172.21.15.204:0/3001364229 shutdown_connections 2024-03-11T19:30:48.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.542+0000 7fe0c1185700 1 --2- 172.21.15.204:0/3001364229 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe0a403afb0 0x7fe0a403d470 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:48.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.542+0000 7fe0c1185700 1 --2- 172.21.15.204:0/3001364229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0bc106620 0x7fe0bc19a8d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:48.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.542+0000 7fe0c1185700 1 -- 172.21.15.204:0/3001364229 >> 172.21.15.204:0/3001364229 conn(0x7fe0bc0fc060 msgr2=0x7fe0bc0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:48.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.542+0000 7fe0c1185700 1 -- 172.21.15.204:0/3001364229 shutdown_connections 2024-03-11T19:30:48.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:48.542+0000 7fe0c1185700 1 -- 172.21.15.204:0/3001364229 wait complete. 2024-03-11T19:30:49.202 DEBUG:teuthology.orchestra.run.smithi137:mon.b> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.b.service 2024-03-11T19:30:49.205 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.c.service 2024-03-11T19:30:49.209 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-11T19:30:49.209 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-11T19:30:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:49 smithi083 ceph-mon[31864]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:49 smithi083 ceph-mon[31864]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:30:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:49 smithi083 ceph-mon[31864]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c;count:3 2024-03-11T19:30:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:30:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:30:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:30:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:30:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:30:50.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:50 smithi083 ceph-mon[31864]: Deploying daemon mon.c on smithi204 2024-03-11T19:30:50.889 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:30:51.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:51 smithi083 ceph-mon[31864]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:52.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:52 smithi083 ceph-mon[31864]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:53.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.040+0000 7f0dda484700 1 -- 172.21.15.204:0/1310202461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd4106600 msgr2=0x7f0dd41069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:53.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.040+0000 7f0dda484700 1 --2- 172.21.15.204:0/1310202461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd4106600 0x7f0dd41069e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009910 tx=0x7f0dc4009c20 comp rx=0 tx=0).stop 2024-03-11T19:30:53.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.040+0000 7f0dda484700 1 -- 172.21.15.204:0/1310202461 shutdown_connections 2024-03-11T19:30:53.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.040+0000 7f0dda484700 1 --2- 172.21.15.204:0/1310202461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd4106600 0x7f0dd41069e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:53.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.040+0000 7f0dda484700 1 -- 172.21.15.204:0/1310202461 >> 172.21.15.204:0/1310202461 conn(0x7f0dd40fc020 msgr2=0x7f0dd40fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:53.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.040+0000 7f0dda484700 1 -- 172.21.15.204:0/1310202461 shutdown_connections 2024-03-11T19:30:53.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.040+0000 7f0dda484700 1 -- 172.21.15.204:0/1310202461 wait complete. 2024-03-11T19:30:53.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.041+0000 7f0dda484700 1 Processor -- start 2024-03-11T19:30:53.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.041+0000 7f0dda484700 1 -- start start 2024-03-11T19:30:53.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.041+0000 7f0dda484700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd4106600 0x7f0dd4196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:53.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.041+0000 7f0dda484700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4197370 con 0x7f0dd4106600 2024-03-11T19:30:53.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.042+0000 7f0dd3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd4106600 0x7f0dd4196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:53.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.042+0000 7f0dd3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd4106600 0x7f0dd4196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.204:34736/0 (socket says 172.21.15.204:34736) 2024-03-11T19:30:53.047 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.042+0000 7f0dd3fff700 1 -- 172.21.15.204:0/2996584094 learned_addr learned my addr 172.21.15.204:0/2996584094 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-11T19:30:53.047 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.042+0000 7f0dd3fff700 1 -- 172.21.15.204:0/2996584094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc40095f0 con 0x7f0dd4106600 2024-03-11T19:30:53.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.042+0000 7f0dd3fff700 1 --2- 172.21.15.204:0/2996584094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd4106600 0x7f0dd4196e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4004f40 tx=0x7f0dc4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:53.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.042+0000 7f0dd17fa700 1 -- 172.21.15.204:0/2996584094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0dc401d070 con 0x7f0dd4106600 2024-03-11T19:30:53.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.042+0000 7f0dd17fa700 1 -- 172.21.15.204:0/2996584094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0dc400bc30 con 0x7f0dd4106600 2024-03-11T19:30:53.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.042+0000 7f0dda484700 1 -- 172.21.15.204:0/2996584094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd41975d0 con 0x7f0dd4106600 2024-03-11T19:30:53.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.042+0000 7f0dd17fa700 1 -- 172.21.15.204:0/2996584094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0dc4003970 con 0x7f0dd4106600 2024-03-11T19:30:53.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.042+0000 7f0dda484700 1 -- 172.21.15.204:0/2996584094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd4197ad0 con 0x7f0dd4106600 2024-03-11T19:30:53.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.043+0000 7f0dd17fa700 1 -- 172.21.15.204:0/2996584094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f0dc4003c30 con 0x7f0dd4106600 2024-03-11T19:30:53.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.043+0000 7f0dda484700 1 -- 172.21.15.204:0/2996584094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd40632a0 con 0x7f0dd4106600 2024-03-11T19:30:53.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.044+0000 7f0dd17fa700 1 --2- 172.21.15.204:0/2996584094 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0dbc038d50 0x7f0dbc03b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:53.048 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.044+0000 7f0dd17fa700 1 -- 172.21.15.204:0/2996584094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0dc404c9d0 con 0x7f0dd4106600 2024-03-11T19:30:53.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.044+0000 7f0dd37fe700 1 --2- 172.21.15.204:0/2996584094 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0dbc038d50 0x7f0dbc03b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:53.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.044+0000 7f0dd37fe700 1 --2- 172.21.15.204:0/2996584094 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0dbc038d50 0x7f0dbc03b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8006fd0 tx=0x7f0dc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:53.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.049+0000 7f0dd17fa700 1 -- 172.21.15.204:0/2996584094 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0dc402ae20 con 0x7f0dd4106600 2024-03-11T19:30:53.331 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.326+0000 7f0dda484700 1 -- 172.21.15.204:0/2996584094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0dd4198190 con 0x7f0dd4106600 2024-03-11T19:30:53.331 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.327+0000 7f0dd17fa700 1 -- 172.21.15.204:0/2996584094 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f0dc402a730 con 0x7f0dd4106600 2024-03-11T19:30:53.331 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:30:53.331 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","modified":"2024-03-11T19:29:01.189729Z","created":"2024-03-11T19:29:01.189729Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-11T19:30:53.333 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.329+0000 7f0dda484700 1 -- 172.21.15.204:0/2996584094 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0dbc038d50 msgr2=0x7f0dbc03b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:53.333 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.329+0000 7f0dda484700 1 --2- 172.21.15.204:0/2996584094 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0dbc038d50 0x7f0dbc03b210 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8006fd0 tx=0x7f0dc8006e40 comp rx=0 tx=0).stop 2024-03-11T19:30:53.333 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.329+0000 7f0dda484700 1 -- 172.21.15.204:0/2996584094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd4106600 msgr2=0x7f0dd4196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:53.333 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.329+0000 7f0dda484700 1 --2- 172.21.15.204:0/2996584094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd4106600 0x7f0dd4196e30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4004f40 tx=0x7f0dc4005020 comp rx=0 tx=0).stop 2024-03-11T19:30:53.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.329+0000 7f0dda484700 1 -- 172.21.15.204:0/2996584094 shutdown_connections 2024-03-11T19:30:53.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.329+0000 7f0dda484700 1 --2- 172.21.15.204:0/2996584094 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0dbc038d50 0x7f0dbc03b210 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:53.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.329+0000 7f0dda484700 1 --2- 172.21.15.204:0/2996584094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd4106600 0x7f0dd4196e30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:53.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.329+0000 7f0dda484700 1 -- 172.21.15.204:0/2996584094 >> 172.21.15.204:0/2996584094 conn(0x7f0dd40fc020 msgr2=0x7f0dd40698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:53.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.330+0000 7f0dda484700 1 -- 172.21.15.204:0/2996584094 shutdown_connections 2024-03-11T19:30:53.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:53.330+0000 7f0dda484700 1 -- 172.21.15.204:0/2996584094 wait complete. 2024-03-11T19:30:53.334 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-03-11T19:30:55.120 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-11T19:30:55.120 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-11T19:30:55.366 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:30:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:30:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: mon.a calling monitor election 2024-03-11T19:30:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:30:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:30:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:30:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: mon.c calling monitor election 2024-03-11T19:30:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: monmap e2: 2 mons at {a=[v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: fsmap 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: mgrmap e13: x(active, since 50s) 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: overall HEALTH_OK 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:30:58.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:30:58.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:30:58 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:30:59.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:59 smithi083 ceph-mon[31864]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:59 smithi083 ceph-mon[31864]: Deploying daemon mon.b on smithi137 2024-03-11T19:30:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:30:59 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:30:59.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:30:59 smithi204 ceph-mon[36739]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:30:59.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:30:59 smithi204 ceph-mon[36739]: Deploying daemon mon.b on smithi137 2024-03-11T19:30:59.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:30:59 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:30:59.879 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.872+0000 7fa6e81da700 1 -- 172.21.15.204:0/2462916665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6cc0057d0 msgr2=0x7fa6cc007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:30:59.879 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.872+0000 7fa6e81da700 1 --2- 172.21.15.204:0/2462916665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6cc0057d0 0x7fa6cc007c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0100560 tx=0x7fa6dc00feb0 comp rx=0 tx=0).stop 2024-03-11T19:30:59.879 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.872+0000 7fa6e81da700 1 -- 172.21.15.204:0/2462916665 shutdown_connections 2024-03-11T19:30:59.879 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.872+0000 7fa6e81da700 1 --2- 172.21.15.204:0/2462916665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6cc0057d0 0x7fa6cc007c60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.872+0000 7fa6e81da700 1 -- 172.21.15.204:0/2462916665 >> 172.21.15.204:0/2462916665 conn(0x7fa6e00fc120 msgr2=0x7fa6e00fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.872+0000 7fa6e81da700 1 -- 172.21.15.204:0/2462916665 shutdown_connections 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.872+0000 7fa6e81da700 1 -- 172.21.15.204:0/2462916665 wait complete. 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e81da700 1 Processor -- start 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e81da700 1 -- start start 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e81da700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6cc0057d0 0x7fa6e0194f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e81da700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0195450 0x7fa6e01997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e81da700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e01959e0 con 0x7fa6cc0057d0 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e81da700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0195b50 con 0x7fa6e0195450 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e5775700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0195450 0x7fa6e01997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e5775700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0195450 0x7fa6e01997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41308/0 (socket says 172.21.15.204:41308) 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e5775700 1 -- 172.21.15.204:0/1997215102 learned_addr learned my addr 172.21.15.204:0/1997215102 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-11T19:30:59.880 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e5775700 1 -- 172.21.15.204:0/1997215102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0195450 msgr2=0x7fa6e01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e5775700 1 -- 172.21.15.204:0/1997215102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0195450 msgr2=0x7fa6e01997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e5775700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0195450 0x7fa6e01997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e5775700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0195450 0x7fa6e01997e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.873+0000 7fa6e5f76700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6cc0057d0 0x7fa6e0194f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.874+0000 7fa6e5f76700 1 -- 172.21.15.204:0/1997215102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0195450 msgr2=0x7fa6e01997e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.874+0000 7fa6e5f76700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0195450 0x7fa6e01997e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.874+0000 7fa6e5f76700 1 -- 172.21.15.204:0/1997215102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc00f900 con 0x7fa6cc0057d0 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.874+0000 7fa6e5f76700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6cc0057d0 0x7fa6e0194f10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc010010 tx=0x7fa6dc0056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.874+0000 7fa6d6ffd700 1 -- 172.21.15.204:0/1997215102 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa6dc004aa0 con 0x7fa6cc0057d0 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.874+0000 7fa6d6ffd700 1 -- 172.21.15.204:0/1997215102 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa6dc013e40 con 0x7fa6cc0057d0 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.874+0000 7fa6e81da700 1 -- 172.21.15.204:0/1997215102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e0199d80 con 0x7fa6cc0057d0 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.874+0000 7fa6d6ffd700 1 -- 172.21.15.204:0/1997215102 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa6dc011b30 con 0x7fa6cc0057d0 2024-03-11T19:30:59.881 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.874+0000 7fa6e81da700 1 -- 172.21.15.204:0/1997215102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e019a2d0 con 0x7fa6cc0057d0 2024-03-11T19:30:59.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.875+0000 7fa6d6ffd700 1 -- 172.21.15.204:0/1997215102 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fa6dc011cb0 con 0x7fa6cc0057d0 2024-03-11T19:30:59.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.876+0000 7fa6d6ffd700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa6c4038b40 0x7fa6c403b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:30:59.882 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.876+0000 7fa6d6ffd700 1 -- 172.21.15.204:0/1997215102 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa6dc04f2a0 con 0x7fa6cc0057d0 2024-03-11T19:30:59.883 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.876+0000 7fa6e5775700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa6c4038b40 0x7fa6c403b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:30:59.883 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.876+0000 7fa6e81da700 1 -- 172.21.15.204:0/1997215102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e018edc0 con 0x7fa6cc0057d0 2024-03-11T19:30:59.883 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.876+0000 7fa6e5775700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa6c4038b40 0x7fa6c403b000 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009180 tx=0x7fa6d0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:30:59.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:30:59.881+0000 7fa6d6ffd700 1 -- 172.21.15.204:0/1997215102 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa6dc0109c0 con 0x7fa6cc0057d0 2024-03-11T19:31:00.167 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.161+0000 7fa6e81da700 1 -- 172.21.15.204:0/1997215102 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa6e00632a0 con 0x7fa6cc0057d0 2024-03-11T19:31:00.167 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.162+0000 7fa6d6ffd700 1 -- 172.21.15.204:0/1997215102 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fa6dc0103d0 con 0x7fa6cc0057d0 2024-03-11T19:31:00.168 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:31:00.168 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","modified":"2024-03-11T19:30:53.408638Z","created":"2024-03-11T19:29:01.189729Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-11T19:31:00.169 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.164+0000 7fa6e81da700 1 -- 172.21.15.204:0/1997215102 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa6c4038b40 msgr2=0x7fa6c403b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:00.169 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.164+0000 7fa6e81da700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa6c4038b40 0x7fa6c403b000 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009180 tx=0x7fa6d0008040 comp rx=0 tx=0).stop 2024-03-11T19:31:00.169 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.164+0000 7fa6e81da700 1 -- 172.21.15.204:0/1997215102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6cc0057d0 msgr2=0x7fa6e0194f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.164+0000 7fa6e81da700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6cc0057d0 0x7fa6e0194f10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc010010 tx=0x7fa6dc0056f0 comp rx=0 tx=0).stop 2024-03-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.164+0000 7fa6e81da700 1 -- 172.21.15.204:0/1997215102 shutdown_connections 2024-03-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.164+0000 7fa6e81da700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa6c4038b40 0x7fa6c403b000 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.164+0000 7fa6e81da700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6cc0057d0 0x7fa6e0194f10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.164+0000 7fa6e81da700 1 --2- 172.21.15.204:0/1997215102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6e0195450 0x7fa6e01997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.164+0000 7fa6e81da700 1 -- 172.21.15.204:0/1997215102 >> 172.21.15.204:0/1997215102 conn(0x7fa6e00fc120 msgr2=0x7fa6e00fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.164+0000 7fa6e81da700 1 -- 172.21.15.204:0/1997215102 shutdown_connections 2024-03-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:00.164+0000 7fa6e81da700 1 -- 172.21.15.204:0/1997215102 wait complete. 2024-03-11T19:31:00.170 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-03-11T19:31:00.770 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:00 smithi204 ceph-mon[36739]: from='client.? 172.21.15.204:0/1997215102' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:31:00.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:00 smithi083 ceph-mon[31864]: from='client.? 172.21.15.204:0/1997215102' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:31:00.819 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:31:00 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:31:00.408+0000 7f8522979700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-11T19:31:01.797 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-11T19:31:01.797 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph mon dump -f json 2024-03-11T19:31:01.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:01 smithi083 ceph-mon[31864]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:01.827 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:01 smithi204 ceph-mon[36739]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:02.041 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:31:02.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:02 smithi137 systemd[1]: Started Ceph mon.b for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:31:07.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:31:07.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: mon.c calling monitor election 2024-03-11T19:31:07.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: mon.a calling monitor election 2024-03-11T19:31:07.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: monmap e3: 3 mons at {a=[v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: fsmap 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:31:07.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: mgrmap e13: x(active, since 58s) 2024-03-11T19:31:07.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-11T19:31:07.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-11T19:31:07.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:31:07.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: mon.b (rank 2) addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] is down (out of quorum) 2024-03-11T19:31:07.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:07.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:07.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:07.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:07.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:31:07.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: mon.c calling monitor election 2024-03-11T19:31:07.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: mon.a calling monitor election 2024-03-11T19:31:07.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:31:07.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:07.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: monmap e3: 3 mons at {a=[v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: fsmap 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: mgrmap e13: x(active, since 58s) 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-11T19:31:07.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:31:07.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: mon.b (rank 2) addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] is down (out of quorum) 2024-03-11T19:31:07.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:07.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:07.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:07.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:07.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.752+0000 7f3259949700 1 -- 172.21.15.204:0/1292052216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c0035c0 msgr2=0x7f323c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:07.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.752+0000 7f3259949700 1 --2- 172.21.15.204:0/1292052216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c0035c0 0x7f323c005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3254101680 tx=0x7f324400fe60 comp rx=0 tx=0).stop 2024-03-11T19:31:07.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.752+0000 7f3259949700 1 -- 172.21.15.204:0/1292052216 shutdown_connections 2024-03-11T19:31:07.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.752+0000 7f3259949700 1 --2- 172.21.15.204:0/1292052216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f323c0035c0 0x7f323c005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:07.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.752+0000 7f3259949700 1 -- 172.21.15.204:0/1292052216 >> 172.21.15.204:0/1292052216 conn(0x7f32540fd240 msgr2=0x7f32540ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:31:07.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.752+0000 7f3259949700 1 -- 172.21.15.204:0/1292052216 shutdown_connections 2024-03-11T19:31:07.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.752+0000 7f3259949700 1 -- 172.21.15.204:0/1292052216 wait complete. 2024-03-11T19:31:07.760 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.752+0000 7f3259949700 1 Processor -- start 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3259949700 1 -- start start 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3259949700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f323c0035c0 0x7f32541971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3259949700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32541976e0 0x7f325419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3259949700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f325419bfb0 0x7f325419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3259949700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3254197dd0 con 0x7f32541976e0 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3259949700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32541a3d20 con 0x7f323c0035c0 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3259949700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32541a3e60 con 0x7f325419bfb0 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32537fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f325419bfb0 0x7f325419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32537fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f325419bfb0 0x7f325419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56482/0 (socket says 172.21.15.204:56482) 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32537fe700 1 -- 172.21.15.204:0/3532732742 learned_addr learned my addr 172.21.15.204:0/3532732742 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32527fc700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32541976e0 0x7f325419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3252ffd700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f323c0035c0 0x7f32541971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:07.761 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3252ffd700 1 -- 172.21.15.204:0/3532732742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f323c0035c0 msgr2=0x7f32541971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3252ffd700 1 -- 172.21.15.204:0/3532732742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f323c0035c0 msgr2=0x7f32541971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32537fe700 1 -- 172.21.15.204:0/3532732742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f325419bfb0 msgr2=0x7f325419c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3252ffd700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f323c0035c0 0x7f32541971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32537fe700 1 -- 172.21.15.204:0/3532732742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f325419bfb0 msgr2=0x7f325419c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32537fe700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f325419bfb0 0x7f325419c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f3252ffd700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f323c0035c0 0x7f32541971a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32537fe700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f325419bfb0 0x7f325419c460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32527fc700 1 -- 172.21.15.204:0/3532732742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f325419bfb0 msgr2=0x7f325419c460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32527fc700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f325419bfb0 0x7f325419c460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32527fc700 1 -- 172.21.15.204:0/3532732742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f323c0035c0 msgr2=0x7f32541971a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32527fc700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f323c0035c0 0x7f32541971a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:07.762 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.753+0000 7f32527fc700 1 -- 172.21.15.204:0/3532732742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324400f8b0 con 0x7f32541976e0 2024-03-11T19:31:07.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.754+0000 7f32527fc700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32541976e0 0x7f325419ba70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f324800b550 tx=0x7f324800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:31:07.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.754+0000 7f3258947700 1 -- 172.21.15.204:0/3532732742 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248004230 con 0x7f32541976e0 2024-03-11T19:31:07.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.754+0000 7f3258947700 1 -- 172.21.15.204:0/3532732742 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3248004390 con 0x7f32541976e0 2024-03-11T19:31:07.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.754+0000 7f3259949700 1 -- 172.21.15.204:0/3532732742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32541a4140 con 0x7f32541976e0 2024-03-11T19:31:07.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.754+0000 7f3258947700 1 -- 172.21.15.204:0/3532732742 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248011b90 con 0x7f32541976e0 2024-03-11T19:31:07.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.754+0000 7f3259949700 1 -- 172.21.15.204:0/3532732742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32541a4690 con 0x7f32541976e0 2024-03-11T19:31:07.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.755+0000 7f3258947700 1 -- 172.21.15.204:0/3532732742 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f3248011cf0 con 0x7f32541976e0 2024-03-11T19:31:07.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.755+0000 7f3259949700 1 -- 172.21.15.204:0/3532732742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32540632a0 con 0x7f32541976e0 2024-03-11T19:31:07.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.755+0000 7f3258947700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32400391e0 0x7f324003b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:07.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.755+0000 7f3258947700 1 -- 172.21.15.204:0/3532732742 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f324804b450 con 0x7f32541976e0 2024-03-11T19:31:07.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.755+0000 7f3252ffd700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32400391e0 0x7f324003b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:07.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.756+0000 7f3252ffd700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32400391e0 0x7f324003b6a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3244005850 tx=0x7f3244001e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:31:07.765 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:07.760+0000 7f3258947700 1 -- 172.21.15.204:0/3532732742 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f324801a400 con 0x7f32541976e0 2024-03-11T19:31:08.052 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.047+0000 7f3259949700 1 -- 172.21.15.204:0/3532732742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3254109600 con 0x7f32541976e0 2024-03-11T19:31:08.053 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.048+0000 7f3258947700 1 -- 172.21.15.204:0/3532732742 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7f324800c150 con 0x7f32541976e0 2024-03-11T19:31:08.053 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-11T19:31:08.053 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","modified":"2024-03-11T19:31:02.209971Z","created":"2024-03-11T19:29:01.189729Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-11T19:31:08.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 -- 172.21.15.204:0/3532732742 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32400391e0 msgr2=0x7f324003b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:08.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32400391e0 0x7f324003b6a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3244005850 tx=0x7f3244001e50 comp rx=0 tx=0).stop 2024-03-11T19:31:08.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 -- 172.21.15.204:0/3532732742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32541976e0 msgr2=0x7f325419ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:08.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32541976e0 0x7f325419ba70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f324800b550 tx=0x7f324800b910 comp rx=0 tx=0).stop 2024-03-11T19:31:08.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 -- 172.21.15.204:0/3532732742 shutdown_connections 2024-03-11T19:31:08.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32400391e0 0x7f324003b6a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:08.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f325419bfb0 0x7f325419c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:08.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f323c0035c0 0x7f32541971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:08.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 --2- 172.21.15.204:0/3532732742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32541976e0 0x7f325419ba70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:08.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 -- 172.21.15.204:0/3532732742 >> 172.21.15.204:0/3532732742 conn(0x7f32540fd240 msgr2=0x7f32540ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:31:08.057 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 -- 172.21.15.204:0/3532732742 shutdown_connections 2024-03-11T19:31:08.058 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:08.051+0000 7f3259949700 1 -- 172.21.15.204:0/3532732742 wait complete. 2024-03-11T19:31:08.058 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-03-11T19:31:08.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:08.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:08 smithi083 ceph-mon[31864]: from='client.? 172.21.15.204:0/3532732742' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:31:08.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:08.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:08 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:08.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:08 smithi204 ceph-mon[36739]: from='client.? 172.21.15.204:0/3532732742' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:31:08.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:08 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:08.818 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-11T19:31:08.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph config generate-minimal-conf 2024-03-11T19:31:09.066 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:31:09.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:31:09.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: mon.c calling monitor election 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: mon.a calling monitor election 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-11T19:31:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: monmap e3: 3 mons at {a=[v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: fsmap 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: mgrmap e13: x(active, since 58s) 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: mon.b (rank 2) addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] is down (out of quorum) 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='client.? 172.21.15.204:0/3532732742' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-11T19:31:09.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: mon.b calling monitor election 2024-03-11T19:31:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: mon.b calling monitor election 2024-03-11T19:31:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: mon.a calling monitor election 2024-03-11T19:31:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: mon.c calling monitor election 2024-03-11T19:31:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-11T19:31:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: monmap e3: 3 mons at {a=[v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:31:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: fsmap 2024-03-11T19:31:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: mgrmap e13: x(active, since 60s) 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: Cluster is now healthy 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: overall HEALTH_OK 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: Updating smithi083:/etc/ceph/ceph.conf 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: Updating smithi083:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:31:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: Updating smithi204:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:31:09.820 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: Updating smithi137:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:31:09.820 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:09.820 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:10.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: mon.b calling monitor election 2024-03-11T19:31:10.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: mon.b calling monitor election 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: mon.a calling monitor election 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: mon.c calling monitor election 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: monmap e3: 3 mons at {a=[v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: fsmap 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: mgrmap e13: x(active, since 60s) 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: Cluster is now healthy 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: overall HEALTH_OK 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:10.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:10.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: Updating smithi083:/etc/ceph/ceph.conf 2024-03-11T19:31:10.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-11T19:31:10.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-11T19:31:10.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: Updating smithi083:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:31:10.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: Updating smithi204:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:31:10.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: Updating smithi137:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:31:10.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:10.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: mon.b calling monitor election 2024-03-11T19:31:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: mon.b calling monitor election 2024-03-11T19:31:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: mon.a calling monitor election 2024-03-11T19:31:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: mon.c calling monitor election 2024-03-11T19:31:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-11T19:31:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: monmap e3: 3 mons at {a=[v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-11T19:31:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: fsmap 2024-03-11T19:31:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: osdmap e4: 0 total, 0 up, 0 in 2024-03-11T19:31:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: mgrmap e13: x(active, since 60s) 2024-03-11T19:31:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: Cluster is now healthy 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: overall HEALTH_OK 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: Updating smithi083:/etc/ceph/ceph.conf 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: Updating smithi083:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: Updating smithi204:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: Updating smithi137:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/config/ceph.conf 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:10.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:10.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.772+0000 7f55c743b700 1 -- 172.21.15.83:0/1630889209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55c0107650 msgr2=0x7f55c0107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:10.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.772+0000 7f55c743b700 1 --2- 172.21.15.83:0/1630889209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55c0107650 0x7f55c0107a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009910 tx=0x7f55bc009c20 comp rx=0 tx=0).stop 2024-03-11T19:31:10.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 -- 172.21.15.83:0/1630889209 shutdown_connections 2024-03-11T19:31:10.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 --2- 172.21.15.83:0/1630889209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55c0107650 0x7f55c0107a30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:10.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 -- 172.21.15.83:0/1630889209 >> 172.21.15.83:0/1630889209 conn(0x7f55c00fd180 msgr2=0x7f55c00ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:31:10.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 -- 172.21.15.83:0/1630889209 shutdown_connections 2024-03-11T19:31:10.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 -- 172.21.15.83:0/1630889209 wait complete. 2024-03-11T19:31:10.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 Processor -- start 2024-03-11T19:31:10.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 -- start start 2024-03-11T19:31:10.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55c0107650 0x7f55c01971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55c0197730 0x7f55c019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55c019c000 0x7f55c019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c0197e20 con 0x7f55c019c000 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01a3d70 con 0x7f55c0107650 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c743b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01a3eb0 con 0x7f55c0197730 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.773+0000 7f55c59d8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55c019c000 0x7f55c019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55c59d8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55c019c000 0x7f55c019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33294/0 (socket says 172.21.15.83:33294) 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55c59d8700 1 -- 172.21.15.83:0/3548444805 learned_addr learned my addr 172.21.15.83:0/3548444805 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55c51d7700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55c0107650 0x7f55c01971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55c59d8700 1 -- 172.21.15.83:0/3548444805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55c0197730 msgr2=0x7f55c019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55c59d8700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55c0197730 0x7f55c019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55c59d8700 1 -- 172.21.15.83:0/3548444805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55c0107650 msgr2=0x7f55c01971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55c59d8700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55c0107650 0x7f55c01971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:10.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55c59d8700 1 -- 172.21.15.83:0/3548444805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0095f0 con 0x7f55c019c000 2024-03-11T19:31:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55c59d8700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55c019c000 0x7f55c019c4b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f55b800ba50 tx=0x7f55b800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:31:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55b67fc700 1 -- 172.21.15.83:0/3548444805 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800cab0 con 0x7f55c019c000 2024-03-11T19:31:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55b67fc700 1 -- 172.21.15.83:0/3548444805 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55b800cc10 con 0x7f55c019c000 2024-03-11T19:31:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55c743b700 1 -- 172.21.15.83:0/3548444805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c01a4190 con 0x7f55c019c000 2024-03-11T19:31:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55b67fc700 1 -- 172.21.15.83:0/3548444805 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80075d0 con 0x7f55c019c000 2024-03-11T19:31:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.774+0000 7f55c743b700 1 -- 172.21.15.83:0/3548444805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c01a4690 con 0x7f55c019c000 2024-03-11T19:31:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.775+0000 7f55b67fc700 1 -- 172.21.15.83:0/3548444805 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f55b8007790 con 0x7f55c019c000 2024-03-11T19:31:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.775+0000 7f55c743b700 1 -- 172.21.15.83:0/3548444805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c00632a0 con 0x7f55c019c000 2024-03-11T19:31:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.775+0000 7f55b67fc700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55ac0391e0 0x7f55ac03b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.775+0000 7f55b67fc700 1 -- 172.21.15.83:0/3548444805 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f55b804b270 con 0x7f55c019c000 2024-03-11T19:31:10.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.775+0000 7f55c51d7700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55ac0391e0 0x7f55ac03b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:10.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.775+0000 7f55c51d7700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55ac0391e0 0x7f55ac03b6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00b5c0 tx=0x7f55bc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:31:10.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.780+0000 7f55b67fc700 1 -- 172.21.15.83:0/3548444805 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f55b8013c30 con 0x7f55c019c000 2024-03-11T19:31:10.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.980+0000 7f55c743b700 1 -- 172.21.15.83:0/3548444805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f55c010bbe0 con 0x7f55c019c000 2024-03-11T19:31:10.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.980+0000 7f55b67fc700 1 -- 172.21.15.83:0/3548444805 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f55b8019070 con 0x7f55c019c000 2024-03-11T19:31:10.984 INFO:teuthology.orchestra.run.smithi083.stdout:# minimal ceph.conf for 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:31:10.984 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-03-11T19:31:10.984 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:31:10.984 INFO:teuthology.orchestra.run.smithi083.stdout: mon_host = [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-03-11T19:31:10.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 -- 172.21.15.83:0/3548444805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55ac0391e0 msgr2=0x7f55ac03b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:10.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55ac0391e0 0x7f55ac03b6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00b5c0 tx=0x7f55bc005a90 comp rx=0 tx=0).stop 2024-03-11T19:31:10.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 -- 172.21.15.83:0/3548444805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55c019c000 msgr2=0x7f55c019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:10.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55c019c000 0x7f55c019c4b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f55b800ba50 tx=0x7f55b800be10 comp rx=0 tx=0).stop 2024-03-11T19:31:10.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 -- 172.21.15.83:0/3548444805 shutdown_connections 2024-03-11T19:31:10.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55ac0391e0 0x7f55ac03b6a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:10.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55c019c000 0x7f55c019c4b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:10.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55c0107650 0x7f55c01971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:10.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 --2- 172.21.15.83:0/3548444805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55c0197730 0x7f55c019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:10.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 -- 172.21.15.83:0/3548444805 >> 172.21.15.83:0/3548444805 conn(0x7f55c00fd180 msgr2=0x7f55c0103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:31:10.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 -- 172.21.15.83:0/3548444805 shutdown_connections 2024-03-11T19:31:10.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:10.982+0000 7f55c743b700 1 -- 172.21.15.83:0/3548444805 wait complete. 2024-03-11T19:31:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: Reconfiguring mon.a (unknown last config time)... 2024-03-11T19:31:11.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:31:11.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:31:11.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:11.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: Reconfiguring daemon mon.a on smithi083 2024-03-11T19:31:11.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:11.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:10 smithi083 ceph-mon[31864]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: Reconfiguring mon.a (unknown last config time)... 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: Reconfiguring daemon mon.a on smithi083 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:10 smithi137 ceph-mon[36220]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:11.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: Reconfiguring mon.a (unknown last config time)... 2024-03-11T19:31:11.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:31:11.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:31:11.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:11.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: Reconfiguring daemon mon.a on smithi083 2024-03-11T19:31:11.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:31:11.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:10 smithi204 ceph-mon[36739]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:11.599 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-11T19:31:11.599 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:31:11.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-03-11T19:31:11.630 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:31:11.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:31:11.698 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-11T19:31:11.698 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-03-11T19:31:11.733 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-11T19:31:11.733 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:31:11.804 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-11T19:31:11.804 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-03-11T19:31:11.838 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-11T19:31:11.838 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-11T19:31:11.908 INFO:tasks.cephadm:Adding mgr.x on smithi083 2024-03-11T19:31:11.908 INFO:tasks.cephadm:Adding mgr.y on smithi137 2024-03-11T19:31:11.908 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-03-11T19:31:11.909 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch apply mgr '3;smithi083=x;smithi137=y;smithi204=z' 2024-03-11T19:31:12.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:11 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3548444805' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:12.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:11 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3548444805' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:12.184 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:31:12.216 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:11 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3548444805' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:13.233 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:12 smithi204 ceph-mon[36739]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:13.234 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:13.234 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:13.234 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:31:13.234 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:31:13.234 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:13.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:12 smithi137 ceph-mon[36220]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:13.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:13.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:13.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:31:13.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:31:13.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:13.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:12 smithi083 ceph-mon[31864]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:13.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:13.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:13.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:31:13.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:31:13.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:13.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.684+0000 7fc1be4f2700 1 -- 172.21.15.204:0/26156492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b81006d0 msgr2=0x7fc1b8100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.684+0000 7fc1be4f2700 1 --2- 172.21.15.204:0/26156492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b81006d0 0x7fc1b8100af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009910 tx=0x7fc1ac009c20 comp rx=0 tx=0).stop 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.684+0000 7fc1be4f2700 1 -- 172.21.15.204:0/26156492 shutdown_connections 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.684+0000 7fc1be4f2700 1 --2- 172.21.15.204:0/26156492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b81006d0 0x7fc1b8100af0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.684+0000 7fc1be4f2700 1 -- 172.21.15.204:0/26156492 >> 172.21.15.204:0/26156492 conn(0x7fc1b80fbc70 msgr2=0x7fc1b80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.684+0000 7fc1be4f2700 1 -- 172.21.15.204:0/26156492 shutdown_connections 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.684+0000 7fc1be4f2700 1 -- 172.21.15.204:0/26156492 wait complete. 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.684+0000 7fc1be4f2700 1 Processor -- start 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.684+0000 7fc1be4f2700 1 -- start start 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1be4f2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b81006d0 0x7fc1b8195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1be4f2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8196260 0x7fc1b819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1be4f2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b81966e0 0x7fc1b819d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1be4f2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b819e010 con 0x7fc1b8196260 2024-03-11T19:31:13.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1be4f2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b819e180 con 0x7fc1b81006d0 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1be4f2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b819e340 con 0x7fc1b81966e0 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1bca8f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b81966e0 0x7fc1b819d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1bca8f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b81966e0 0x7fc1b819d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:56492/0 (socket says 172.21.15.204:56492) 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1bca8f700 1 -- 172.21.15.204:0/2215172080 learned_addr learned my addr 172.21.15.204:0/2215172080 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1b7fff700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b81006d0 0x7fc1b8195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1b77fe700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8196260 0x7fc1b819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1bca8f700 1 -- 172.21.15.204:0/2215172080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b81006d0 msgr2=0x7fc1b8195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1bca8f700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b81006d0 0x7fc1b8195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1bca8f700 1 -- 172.21.15.204:0/2215172080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8196260 msgr2=0x7fc1b819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1bca8f700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8196260 0x7fc1b819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1bca8f700 1 -- 172.21.15.204:0/2215172080 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac0095f0 con 0x7fc1b81966e0 2024-03-11T19:31:13.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1b77fe700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8196260 0x7fc1b819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:31:13.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.685+0000 7fc1bca8f700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b81966e0 0x7fc1b819d840 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000df40 tx=0x7fc1b000df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:31:13.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.686+0000 7fc1b57fa700 1 -- 172.21.15.204:0/2215172080 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b000e7d0 con 0x7fc1b81966e0 2024-03-11T19:31:13.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.686+0000 7fc1b57fa700 1 -- 172.21.15.204:0/2215172080 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1b000bb40 con 0x7fc1b81966e0 2024-03-11T19:31:13.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.686+0000 7fc1b57fa700 1 -- 172.21.15.204:0/2215172080 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0016650 con 0x7fc1b81966e0 2024-03-11T19:31:13.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.686+0000 7fc1be4f2700 1 -- 172.21.15.204:0/2215172080 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b8069e50 con 0x7fc1b81966e0 2024-03-11T19:31:13.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.686+0000 7fc1be4f2700 1 -- 172.21.15.204:0/2215172080 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b806a380 con 0x7fc1b81966e0 2024-03-11T19:31:13.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.686+0000 7fc1b57fa700 1 -- 172.21.15.204:0/2215172080 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7fc1b00167b0 con 0x7fc1b81966e0 2024-03-11T19:31:13.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.687+0000 7fc1be4f2700 1 -- 172.21.15.204:0/2215172080 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b80632a0 con 0x7fc1b81966e0 2024-03-11T19:31:13.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.687+0000 7fc1b57fa700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1a00391e0 0x7fc1a003b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:13.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.687+0000 7fc1b57fa700 1 -- 172.21.15.204:0/2215172080 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc1b004d0e0 con 0x7fc1b81966e0 2024-03-11T19:31:13.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.687+0000 7fc1b7fff700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1a00391e0 0x7fc1a003b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:13.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.687+0000 7fc1b7fff700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1a00391e0 0x7fc1a003b6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8197200 tx=0x7fc1ac005300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:31:13.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.692+0000 7fc1b57fa700 1 -- 172.21.15.204:0/2215172080 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1b00148f0 con 0x7fc1b81966e0 2024-03-11T19:31:13.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.892+0000 7fc1be4f2700 1 -- 172.21.15.204:0/2215172080 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi137=y;smithi204=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1b818f0b0 con 0x7fc1a00391e0 2024-03-11T19:31:13.902 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.898+0000 7fc1b57fa700 1 -- 172.21.15.204:0/2215172080 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc1b818f0b0 con 0x7fc1a00391e0 2024-03-11T19:31:13.902 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-03-11T19:31:13.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 -- 172.21.15.204:0/2215172080 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1a00391e0 msgr2=0x7fc1a003b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:13.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1a00391e0 0x7fc1a003b6a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8197200 tx=0x7fc1ac005300 comp rx=0 tx=0).stop 2024-03-11T19:31:13.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 -- 172.21.15.204:0/2215172080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b81966e0 msgr2=0x7fc1b819d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:13.904 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b81966e0 0x7fc1b819d840 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000df40 tx=0x7fc1b000df70 comp rx=0 tx=0).stop 2024-03-11T19:31:13.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 -- 172.21.15.204:0/2215172080 shutdown_connections 2024-03-11T19:31:13.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1a00391e0 0x7fc1a003b6a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:13.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1b81966e0 0x7fc1b819d840 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:13.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b81006d0 0x7fc1b8195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:13.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 --2- 172.21.15.204:0/2215172080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1b8196260 0x7fc1b819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:13.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 -- 172.21.15.204:0/2215172080 >> 172.21.15.204:0/2215172080 conn(0x7fc1b80fbc70 msgr2=0x7fc1b80fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:31:13.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 -- 172.21.15.204:0/2215172080 shutdown_connections 2024-03-11T19:31:13.905 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:31:13.900+0000 7fc1be4f2700 1 -- 172.21.15.204:0/2215172080 wait complete. 2024-03-11T19:31:14.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:13 smithi204 ceph-mon[36739]: Reconfiguring mon.b (monmap changed)... 2024-03-11T19:31:14.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:13 smithi204 ceph-mon[36739]: Reconfiguring daemon mon.b on smithi137 2024-03-11T19:31:14.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:13 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:14.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:13 smithi083 ceph-mon[31864]: Reconfiguring mon.b (monmap changed)... 2024-03-11T19:31:14.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:13 smithi083 ceph-mon[31864]: Reconfiguring daemon mon.b on smithi137 2024-03-11T19:31:14.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:13 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:14.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:13 smithi137 ceph-mon[36220]: Reconfiguring mon.b (monmap changed)... 2024-03-11T19:31:14.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:13 smithi137 ceph-mon[36220]: Reconfiguring daemon mon.b on smithi137 2024-03-11T19:31:14.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:13 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:14.510 DEBUG:teuthology.orchestra.run.smithi137:mgr.y> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.y.service 2024-03-11T19:31:14.512 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.z.service 2024-03-11T19:31:14.515 INFO:tasks.cephadm:Deploying OSDs... 2024-03-11T19:31:14.515 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:31:14.515 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-03-11T19:31:14.534 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-11T19:31:14.535 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-03-11T19:31:14.592 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-11T19:31:14.592 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:14.593 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-11T19:31:14.593 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:14.593 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:14.593 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-03-11 19:30:18.799030503 +0000 2024-03-11T19:31:14.593 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-03-11 19:25:01.660255868 +0000 2024-03-11T19:31:14.593 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-03-11 19:25:01.660255868 +0000 2024-03-11T19:31:14.593 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-03-11 19:25:01.660255868 +0000 2024-03-11T19:31:14.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-11T19:31:14.662 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-03-11T19:31:14.662 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-03-11T19:31:14.662 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000224702 s, 2.3 MB/s 2024-03-11T19:31:14.663 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-11T19:31:14.721 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-03-11T19:31:14.777 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-11T19:31:14.778 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:14.778 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-11T19:31:14.778 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:14.778 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:14.778 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-03-11 19:30:18.826029845 +0000 2024-03-11T19:31:14.778 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-03-11 19:25:02.017247224 +0000 2024-03-11T19:31:14.778 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-03-11 19:25:02.017247224 +0000 2024-03-11T19:31:14.778 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-03-11 19:25:02.017247224 +0000 2024-03-11T19:31:14.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-11T19:31:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-03-11T19:31:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-03-11T19:31:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000203046 s, 2.5 MB/s 2024-03-11T19:31:14.846 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-11T19:31:14.902 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-03-11T19:31:14.958 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-11T19:31:14.958 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:14.958 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-11T19:31:14.959 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:14.959 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:14.959 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-03-11 19:30:18.851029235 +0000 2024-03-11T19:31:14.959 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-03-11 19:25:02.341239380 +0000 2024-03-11T19:31:14.959 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-03-11 19:25:02.341239380 +0000 2024-03-11T19:31:14.959 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-03-11 19:25:02.341239380 +0000 2024-03-11T19:31:14.959 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-11T19:31:15.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-03-11T19:31:15.025 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-03-11T19:31:15.025 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000212008 s, 2.4 MB/s 2024-03-11T19:31:15.026 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-11T19:31:15.082 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-03-11T19:31:15.138 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-11T19:31:15.138 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:15.138 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-11T19:31:15.138 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:15.138 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:15.138 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-03-11 19:30:18.875028649 +0000 2024-03-11T19:31:15.138 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-03-11 19:25:02.700230688 +0000 2024-03-11T19:31:15.138 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-03-11 19:25:02.700230688 +0000 2024-03-11T19:31:15.138 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-03-11 19:25:02.700230688 +0000 2024-03-11T19:31:15.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-11T19:31:15.204 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-03-11T19:31:15.204 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-03-11T19:31:15.204 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000282733 s, 1.8 MB/s 2024-03-11T19:31:15.206 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-11T19:31:15.262 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-11T19:31:15.262 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-03-11T19:31:15.281 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-11T19:31:15.281 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-03-11T19:31:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:15 smithi083 ceph-mon[31864]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi137=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:31:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:15 smithi083 ceph-mon[31864]: Saving service mgr spec with placement smithi083=x;smithi137=y;smithi204=z;count:3 2024-03-11T19:31:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:15 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:15 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:15 smithi083 ceph-mon[31864]: Reconfiguring mon.c (monmap changed)... 2024-03-11T19:31:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:15 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:31:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:15 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:31:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:15 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:15 smithi083 ceph-mon[31864]: Reconfiguring daemon mon.c on smithi204 2024-03-11T19:31:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:15 smithi083 ceph-mon[31864]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:15.338 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-11T19:31:15.339 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:15.339 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-03-11T19:31:15.339 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:15.339 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:15.339 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-11 19:31:01.682668212 +0000 2024-03-11T19:31:15.339 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-11 19:25:38.067623069 +0000 2024-03-11T19:31:15.339 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-11 19:25:38.067623069 +0000 2024-03-11T19:31:15.339 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-11 19:25:38.067623069 +0000 2024-03-11T19:31:15.339 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-11T19:31:15.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:15 smithi137 ceph-mon[36220]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi137=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:31:15.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:15 smithi137 ceph-mon[36220]: Saving service mgr spec with placement smithi083=x;smithi137=y;smithi204=z;count:3 2024-03-11T19:31:15.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:15 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:15.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:15 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:15.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:15 smithi137 ceph-mon[36220]: Reconfiguring mon.c (monmap changed)... 2024-03-11T19:31:15.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:15 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:31:15.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:15 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:31:15.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:15 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:15.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:15 smithi137 ceph-mon[36220]: Reconfiguring daemon mon.c on smithi204 2024-03-11T19:31:15.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:15 smithi137 ceph-mon[36220]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:15.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:15 smithi204 ceph-mon[36739]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi137=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:31:15.403 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:15 smithi204 ceph-mon[36739]: Saving service mgr spec with placement smithi083=x;smithi137=y;smithi204=z;count:3 2024-03-11T19:31:15.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:15 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:15.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:15 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:15.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:15 smithi204 ceph-mon[36739]: Reconfiguring mon.c (monmap changed)... 2024-03-11T19:31:15.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:15 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-11T19:31:15.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:15 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-11T19:31:15.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:15 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:15.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:15 smithi204 ceph-mon[36739]: Reconfiguring daemon mon.c on smithi204 2024-03-11T19:31:15.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:15 smithi204 ceph-mon[36739]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:15.408 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-11T19:31:15.409 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-11T19:31:15.409 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000296078 s, 1.7 MB/s 2024-03-11T19:31:15.410 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-11T19:31:15.469 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-03-11T19:31:15.525 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-11T19:31:15.525 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:15.525 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 679 Links: 1 2024-03-11T19:31:15.525 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:15.525 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:15.525 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-11 19:31:01.683668194 +0000 2024-03-11T19:31:15.525 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-11 19:25:38.411616928 +0000 2024-03-11T19:31:15.526 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-11 19:25:38.411616928 +0000 2024-03-11T19:31:15.526 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-11 19:25:38.411616928 +0000 2024-03-11T19:31:15.526 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-11T19:31:15.592 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-11T19:31:15.593 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-11T19:31:15.593 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000231043 s, 2.2 MB/s 2024-03-11T19:31:15.594 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-11T19:31:15.654 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-03-11T19:31:15.711 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-11T19:31:15.711 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:15.711 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-11T19:31:15.711 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:15.711 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:15.711 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-11 19:31:01.683668194 +0000 2024-03-11T19:31:15.711 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-11 19:25:38.750610876 +0000 2024-03-11T19:31:15.711 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-11 19:25:38.750610876 +0000 2024-03-11T19:31:15.711 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-11 19:25:38.750610876 +0000 2024-03-11T19:31:15.712 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-11T19:31:15.778 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-11T19:31:15.779 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-11T19:31:15.779 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000208536 s, 2.5 MB/s 2024-03-11T19:31:15.780 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-11T19:31:15.836 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-03-11T19:31:15.892 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-11T19:31:15.893 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:15.893 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-03-11T19:31:15.893 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:15.893 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:15.893 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-11 19:31:01.683668194 +0000 2024-03-11T19:31:15.893 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-11 19:25:39.072605128 +0000 2024-03-11T19:31:15.893 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-11 19:25:39.072605128 +0000 2024-03-11T19:31:15.893 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-11 19:25:39.072605128 +0000 2024-03-11T19:31:15.893 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-11T19:31:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-11T19:31:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-11T19:31:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000205554 s, 2.5 MB/s 2024-03-11T19:31:15.960 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-11T19:31:16.017 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-11T19:31:16.017 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-03-11T19:31:16.034 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-11T19:31:16.034 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-03-11T19:31:16.089 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-11T19:31:16.089 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:16.090 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-03-11T19:31:16.090 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:16.090 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:16.090 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-11 19:30:52.302023819 +0000 2024-03-11T19:31:16.090 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-11 19:25:30.971991978 +0000 2024-03-11T19:31:16.090 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-11 19:25:30.971991978 +0000 2024-03-11T19:31:16.090 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-11 19:25:30.971991978 +0000 2024-03-11T19:31:16.090 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-11T19:31:16.156 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-11T19:31:16.156 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-11T19:31:16.156 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000273668 s, 1.9 MB/s 2024-03-11T19:31:16.157 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-11T19:31:16.213 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-03-11T19:31:16.270 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-11T19:31:16.271 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:16.271 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-11T19:31:16.271 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:16.271 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:16.271 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-11 19:30:52.302023819 +0000 2024-03-11T19:31:16.271 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-11 19:25:31.336986496 +0000 2024-03-11T19:31:16.271 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-11 19:25:31.336986496 +0000 2024-03-11T19:31:16.271 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-11 19:25:31.336986496 +0000 2024-03-11T19:31:16.272 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-11T19:31:16.335 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-11T19:31:16.335 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-11T19:31:16.336 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000215158 s, 2.4 MB/s 2024-03-11T19:31:16.337 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-11T19:31:16.392 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-03-11T19:31:16.447 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-11T19:31:16.447 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:16.447 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-03-11T19:31:16.447 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:16.448 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:16.448 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-11 19:30:52.303023804 +0000 2024-03-11T19:31:16.448 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-11 19:25:31.676981389 +0000 2024-03-11T19:31:16.448 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-11 19:25:31.676981389 +0000 2024-03-11T19:31:16.448 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-11 19:25:31.676981389 +0000 2024-03-11T19:31:16.448 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-11T19:31:16.513 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-11T19:31:16.513 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-11T19:31:16.513 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000220442 s, 2.3 MB/s 2024-03-11T19:31:16.514 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-11T19:31:16.570 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-03-11T19:31:16.624 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-11T19:31:16.624 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T19:31:16.624 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-03-11T19:31:16.624 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T19:31:16.624 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-11T19:31:16.624 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-11 19:30:52.303023804 +0000 2024-03-11T19:31:16.624 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-11 19:25:32.014976312 +0000 2024-03-11T19:31:16.624 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-11 19:25:32.014976312 +0000 2024-03-11T19:31:16.624 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-11 19:25:32.014976312 +0000 2024-03-11T19:31:16.625 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-11T19:31:16.692 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-11T19:31:16.692 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-11T19:31:16.693 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000201654 s, 2.5 MB/s 2024-03-11T19:31:16.694 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-11T19:31:16.751 INFO:tasks.cephadm:Deploying osd.0 on smithi083 with /dev/vg_nvme/lv_4... 2024-03-11T19:31:16.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-11T19:31:16.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:16.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:16.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:16.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:16.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:16.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:16.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:31:16.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:31:16.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:31:16.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:16.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: Deploying daemon mgr.z on smithi204 2024-03-11T19:31:16.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:16 smithi204 ceph-mon[36739]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:16.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:16.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:16.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:16.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:16.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:16.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:16.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:31:16.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:31:16.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:31:16.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:16.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: Deploying daemon mgr.z on smithi204 2024-03-11T19:31:16.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:16 smithi137 ceph-mon[36220]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:16.983 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:31:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:31:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:31:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:31:17.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:17.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: Deploying daemon mgr.z on smithi204 2024-03-11T19:31:17.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:16 smithi083 ceph-mon[31864]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:18.406 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:18 smithi204 systemd[1]: Starting Ceph mgr.z for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:31:18.734 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:18 smithi204 podman[38581]: 2024-03-11 19:31:18.332658974 +0000 UTC m=+0.018157359 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:31:18.734 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:18 smithi204 podman[38581]: 2024-03-11 19:31:18.43680297 +0000 UTC m=+0.122301315 container create 154856ab6423e87e1b0890feeab0e0da53869a5fb5871bce8a602053d44f5040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, io.buildah.version=1.33.5) 2024-03-11T19:31:18.734 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:18 smithi204 podman[38581]: 2024-03-11 19:31:18.646259002 +0000 UTC m=+0.331757351 container init 154856ab6423e87e1b0890feeab0e0da53869a5fb5871bce8a602053d44f5040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=) 2024-03-11T19:31:18.734 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:18 smithi204 podman[38581]: 2024-03-11 19:31:18.648815696 +0000 UTC m=+0.334314043 container start 154856ab6423e87e1b0890feeab0e0da53869a5fb5871bce8a602053d44f5040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-94a5397) 2024-03-11T19:31:18.734 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:18 smithi204 ceph-mgr[38600]: -- 172.21.15.204:0/4101479255 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563552db81a0 con 0x563552d8e800 2024-03-11T19:31:18.734 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:18 smithi204 bash[38581]: 154856ab6423e87e1b0890feeab0e0da53869a5fb5871bce8a602053d44f5040 2024-03-11T19:31:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-11T19:31:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-11T19:31:18.857 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-03-11T19:31:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-03-11T19:31:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011286 s, 929 MB/s 2024-03-11T19:31:18.857 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 2024-03-11T19:31:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-03-11T19:31:19.156 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:18 smithi204 systemd[1]: Started Ceph mgr.z for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:31:19.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-03-11T19:31:19.499 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:19.193+0000 7f9e836a5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:31:19.499 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:19.286+0000 7f9e836a5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:31:19.718 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:31:19.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:19 smithi137 ceph-mon[36220]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:19.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:19.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:19.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:19.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:31:19.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:31:19.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:31:19.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:19 smithi083 ceph-mon[31864]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:31:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:31:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:31:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:19 smithi204 ceph-mon[36739]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:31:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-11T19:31:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:31:19.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:20.268 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:19.931+0000 7f9e836a5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:31:20.268 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:20.026+0000 7f9e836a5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:31:20.268 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:20.169+0000 7f9e836a5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:31:20.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:20 smithi204 ceph-mon[36739]: Deploying daemon mgr.y on smithi137 2024-03-11T19:31:20.614 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:20.264+0000 7f9e836a5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:31:20.614 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:20.346+0000 7f9e836a5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:31:20.614 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:20.610+0000 7f9e836a5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:31:20.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:20 smithi137 ceph-mon[36220]: Deploying daemon mgr.y on smithi137 2024-03-11T19:31:20.755 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:20 smithi083 ceph-mon[31864]: Deploying daemon mgr.y on smithi137 2024-03-11T19:31:20.906 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:20.719+0000 7f9e836a5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:31:20.906 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:20.807+0000 7f9e836a5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:31:21.132 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:20 smithi137 systemd[1]: Starting Ceph mgr.y for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:31:21.235 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:20.906+0000 7f9e836a5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:31:21.235 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:21 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:21.089+0000 7f9e836a5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:31:21.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.285+0000 7f66eafb0700 1 -- 172.21.15.83:0/3654005052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4101da0 msgr2=0x7f66e4102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:21.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.285+0000 7f66eafb0700 1 --2- 172.21.15.83:0/3654005052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4101da0 0x7f66e4102200 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f66d00099c0 tx=0x7f66d0009cd0 comp rx=0 tx=0).stop 2024-03-11T19:31:21.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.285+0000 7f66eafb0700 1 -- 172.21.15.83:0/3654005052 shutdown_connections 2024-03-11T19:31:21.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.285+0000 7f66eafb0700 1 --2- 172.21.15.83:0/3654005052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e4102740 0x7f66e4108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.285+0000 7f66eafb0700 1 --2- 172.21.15.83:0/3654005052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e4100ba0 0x7f66e4100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.285+0000 7f66eafb0700 1 --2- 172.21.15.83:0/3654005052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4101da0 0x7f66e4102200 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.285+0000 7f66eafb0700 1 -- 172.21.15.83:0/3654005052 >> 172.21.15.83:0/3654005052 conn(0x7f66e40fc140 msgr2=0x7f66e40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.285+0000 7f66eafb0700 1 -- 172.21.15.83:0/3654005052 shutdown_connections 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.285+0000 7f66eafb0700 1 -- 172.21.15.83:0/3654005052 wait complete. 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66eafb0700 1 Processor -- start 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66eafb0700 1 -- start start 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66eafb0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e4100ba0 0x7f66e4197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66eafb0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e4101da0 0x7f66e4198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66eafb0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4102740 0x7f66e419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66eafb0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e419f830 con 0x7f66e4102740 2024-03-11T19:31:21.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66eafb0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e419f970 con 0x7f66e4100ba0 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66eafb0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e419fb30 con 0x7f66e4101da0 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66e8d4c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e4100ba0 0x7f66e4197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66e954d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4102740 0x7f66e419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66e8d4c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e4100ba0 0x7f66e4197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39554/0 (socket says 172.21.15.83:39554) 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66e954d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4102740 0x7f66e419f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49828/0 (socket says 172.21.15.83:49828) 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66e8d4c700 1 -- 172.21.15.83:0/2541998155 learned_addr learned my addr 172.21.15.83:0/2541998155 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66e3fff700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e4101da0 0x7f66e4198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66e8d4c700 1 -- 172.21.15.83:0/2541998155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e4101da0 msgr2=0x7f66e4198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66e8d4c700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e4101da0 0x7f66e4198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66e8d4c700 1 -- 172.21.15.83:0/2541998155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4102740 msgr2=0x7f66e419f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66e8d4c700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4102740 0x7f66e419f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.286+0000 7f66e8d4c700 1 -- 172.21.15.83:0/2541998155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d8009520 con 0x7f66e4100ba0 2024-03-11T19:31:21.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.287+0000 7f66e954d700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4102740 0x7f66e419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:31:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.287+0000 7f66e8d4c700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e4100ba0 0x7f66e4197ce0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f66d800ea50 tx=0x7f66d800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:31:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.287+0000 7f66e1ffb700 1 -- 172.21.15.83:0/2541998155 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d800cbd0 con 0x7f66e4100ba0 2024-03-11T19:31:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.287+0000 7f66e1ffb700 1 -- 172.21.15.83:0/2541998155 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66d800cd30 con 0x7f66e4100ba0 2024-03-11T19:31:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.287+0000 7f66eafb0700 1 -- 172.21.15.83:0/2541998155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d00095f0 con 0x7f66e4100ba0 2024-03-11T19:31:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.287+0000 7f66e1ffb700 1 -- 172.21.15.83:0/2541998155 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d80186a0 con 0x7f66e4100ba0 2024-03-11T19:31:21.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.287+0000 7f66eafb0700 1 -- 172.21.15.83:0/2541998155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e41a0180 con 0x7f66e4100ba0 2024-03-11T19:31:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.288+0000 7f66e1ffb700 1 -- 172.21.15.83:0/2541998155 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45825+0+0 (secure 0 0 0) 0x7f66d8018800 con 0x7f66e4100ba0 2024-03-11T19:31:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.288+0000 7f66e1ffb700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66d4038f90 0x7f66d403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.288+0000 7f66e1ffb700 1 -- 172.21.15.83:0/2541998155 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f66d8014070 con 0x7f66e4100ba0 2024-03-11T19:31:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.288+0000 7f66eafb0700 1 -- 172.21.15.83:0/2541998155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e4068930 con 0x7f66e4100ba0 2024-03-11T19:31:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.288+0000 7f66e3fff700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66d4038f90 0x7f66d403b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:21.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.289+0000 7f66e3fff700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66d4038f90 0x7f66d403b450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66d0000c00 tx=0x7f66d0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:31:21.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.294+0000 7f66e1ffb700 1 -- 172.21.15.83:0/2541998155 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f66d8018ba0 con 0x7f66e4100ba0 2024-03-11T19:31:21.390 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:21 smithi137 podman[37598]: 2024-03-11 19:31:21.128772632 +0000 UTC m=+0.107443365 container create 24e5dc9168b0a5c03548f10462c6f9627febf255c82e65e514e93edce9b9582f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-11T19:31:21.390 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:21 smithi137 podman[37598]: 2024-03-11 19:31:21.039077411 +0000 UTC m=+0.017748163 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:31:21.390 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:21 smithi137 podman[37598]: 2024-03-11 19:31:21.314454474 +0000 UTC m=+0.293125204 container init 24e5dc9168b0a5c03548f10462c6f9627febf255c82e65e514e93edce9b9582f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-03-11T19:31:21.393 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:21 smithi137 podman[37598]: 2024-03-11 19:31:21.317229353 +0000 UTC m=+0.295900087 container start 24e5dc9168b0a5c03548f10462c6f9627febf255c82e65e514e93edce9b9582f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-11T19:31:21.393 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:21 smithi137 ceph-mgr[37617]: -- 172.21.15.137:0/4117064233 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562283c481a0 con 0x562283c1e800 2024-03-11T19:31:21.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:21.502+0000 7f66eafb0700 1 -- 172.21.15.83:0/2541998155 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f66e406a030 con 0x7f66d4038f90 2024-03-11T19:31:21.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:21 smithi083 ceph-mon[31864]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:21.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:21.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:21.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:21.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:21.870 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:21 smithi137 bash[37598]: 24e5dc9168b0a5c03548f10462c6f9627febf255c82e65e514e93edce9b9582f 2024-03-11T19:31:21.870 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:21 smithi137 systemd[1]: Started Ceph mgr.y for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:31:21.870 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:21 smithi137 ceph-mon[36220]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:21.870 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:21.870 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:21.871 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:21.871 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:21 smithi204 ceph-mon[36739]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:22.139 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:21 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:21.865+0000 7fd9756f1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T19:31:22.139 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:21 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:21.962+0000 7fd9756f1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T19:31:22.406 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:22 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:22.130+0000 7f9e836a5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:31:22.406 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:22 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:22.282+0000 7f9e836a5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:31:22.687 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:22 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:22.673+0000 7fd9756f1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T19:31:22.688 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:22 smithi137 ceph-mon[36220]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:31:22.688 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:22 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:31:22.688 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:22 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:31:22.688 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:22 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:22.689 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:22 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:22.715 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:22 smithi204 ceph-mon[36739]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:31:22.715 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:22 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:31:22.715 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:22 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:31:22.715 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:22 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:22.715 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:22 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:22.716 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:22 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:22.622+0000 7f9e836a5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:31:22.716 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:22 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:22.711+0000 7f9e836a5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:31:22.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:22 smithi083 ceph-mon[31864]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:31:22.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:22 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:31:22.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:22 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:31:22.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:22 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:22.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:22 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:23.020 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:22 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:22.767+0000 7fd9756f1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T19:31:23.020 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:22 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:22.915+0000 7fd9756f1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T19:31:23.020 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:23.016+0000 7fd9756f1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T19:31:23.107 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:22 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:22.917+0000 7f9e836a5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:31:23.107 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:23.102+0000 7f9e836a5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:31:23.364 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:23.098+0000 7fd9756f1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T19:31:23.364 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:23.360+0000 7fd9756f1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-11T19:31:23.639 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:23.467+0000 7fd9756f1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T19:31:23.639 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:23.554+0000 7fd9756f1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T19:31:23.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:23.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:23.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:23.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:31:23.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:31:23.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:23.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.642 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:23.379+0000 7f9e836a5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:31:23.642 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:23.638+0000 7f9e836a5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:31:23.642 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:23.642 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.642 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.642 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:31:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:31:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:23.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-11T19:31:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-11T19:31:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:23.906 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:23 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:23.868+0000 7f9e836a5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:31:24.139 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:23.646+0000 7fd9756f1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T19:31:24.139 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:23 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:23.821+0000 7fd9756f1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T19:31:24.783 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:24 smithi137 ceph-mon[36220]: Reconfiguring mgr.x (unknown last config time)... 2024-03-11T19:31:24.783 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:24 smithi137 ceph-mon[36220]: Reconfiguring daemon mgr.x on smithi083 2024-03-11T19:31:24.783 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:24 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4000997437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4626dde-b940-4692-9e2d-2a2f3f59a70f"}]: dispatch 2024-03-11T19:31:24.783 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:24 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4000997437' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4626dde-b940-4692-9e2d-2a2f3f59a70f"}]': finished 2024-03-11T19:31:24.783 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:24 smithi137 ceph-mon[36220]: osdmap e5: 1 total, 0 up, 1 in 2024-03-11T19:31:24.784 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:24 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:24 smithi083 ceph-mon[31864]: Reconfiguring mgr.x (unknown last config time)... 2024-03-11T19:31:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:24 smithi083 ceph-mon[31864]: Reconfiguring daemon mgr.x on smithi083 2024-03-11T19:31:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:24 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4000997437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4626dde-b940-4692-9e2d-2a2f3f59a70f"}]: dispatch 2024-03-11T19:31:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:24 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4000997437' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4626dde-b940-4692-9e2d-2a2f3f59a70f"}]': finished 2024-03-11T19:31:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:24 smithi083 ceph-mon[31864]: osdmap e5: 1 total, 0 up, 1 in 2024-03-11T19:31:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:24 smithi204 ceph-mon[36739]: Reconfiguring mgr.x (unknown last config time)... 2024-03-11T19:31:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:24 smithi204 ceph-mon[36739]: Reconfiguring daemon mgr.x on smithi083 2024-03-11T19:31:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:24 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4000997437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4626dde-b940-4692-9e2d-2a2f3f59a70f"}]: dispatch 2024-03-11T19:31:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:24 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4000997437' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4626dde-b940-4692-9e2d-2a2f3f59a70f"}]': finished 2024-03-11T19:31:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:24 smithi204 ceph-mon[36739]: osdmap e5: 1 total, 0 up, 1 in 2024-03-11T19:31:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:24 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:24.907 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:24 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:24.654+0000 7f9e836a5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:31:25.139 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:24 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:24.779+0000 7fd9756f1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-11T19:31:25.139 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:24 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:24.908+0000 7fd9756f1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T19:31:25.511 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:25 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:25.255+0000 7fd9756f1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T19:31:25.512 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:25 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:25.343+0000 7fd9756f1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T19:31:25.513 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:25 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:25.220+0000 7f9e836a5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:31:25.513 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:25 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:25.316+0000 7f9e836a5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:31:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:25 smithi083 ceph-mon[31864]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:25 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3068215823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:31:25.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:25 smithi137 ceph-mon[36220]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:25.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:25 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3068215823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:31:25.889 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:25 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:25.534+0000 7fd9756f1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:31:25.890 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:25 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:25.716+0000 7fd9756f1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T19:31:25.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:25 smithi204 ceph-mon[36739]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:25.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:25 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3068215823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:31:25.906 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:31:25 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z[38596]: 2024-03-11T19:31:25.526+0000 7f9e836a5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:31:26.260 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:25 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:25.995+0000 7fd9756f1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T19:31:26.260 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:26 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:26.256+0000 7fd9756f1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T19:31:26.513 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:26 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:26.489+0000 7fd9756f1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T19:31:26.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:26.518+0000 7f66e1ffb700 1 -- 172.21.15.83:0/2541998155 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 14) v1 ==== 91024+0+0 (secure 0 0 0) 0x7f66d8021a10 con 0x7f66e4100ba0 2024-03-11T19:31:26.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:26 smithi083 ceph-mon[31864]: Standby manager daemon z started 2024-03-11T19:31:26.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:26 smithi083 ceph-mon[31864]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:31:26.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:26 smithi083 ceph-mon[31864]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:31:26.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:26 smithi083 ceph-mon[31864]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:31:26.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:26 smithi083 ceph-mon[31864]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:31:26.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:26 smithi137 ceph-mon[36220]: Standby manager daemon z started 2024-03-11T19:31:26.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:26 smithi137 ceph-mon[36220]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:31:26.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:26 smithi137 ceph-mon[36220]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:31:26.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:26 smithi137 ceph-mon[36220]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:31:26.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:26 smithi137 ceph-mon[36220]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:31:26.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:26 smithi204 ceph-mon[36739]: Standby manager daemon z started 2024-03-11T19:31:26.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:26 smithi204 ceph-mon[36739]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-11T19:31:26.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:26 smithi204 ceph-mon[36739]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:31:26.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:26 smithi204 ceph-mon[36739]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-11T19:31:26.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:26 smithi204 ceph-mon[36739]: from='mgr.? 172.21.15.204:0/1323794856' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:31:27.640 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:27 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:27.274+0000 7fd9756f1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T19:31:27.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:27 smithi137 ceph-mon[36220]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:27.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:27 smithi137 ceph-mon[36220]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-11T19:31:27.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:31:27.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:27 smithi083 ceph-mon[31864]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:27 smithi083 ceph-mon[31864]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-11T19:31:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:31:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:27 smithi204 ceph-mon[36739]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:27 smithi204 ceph-mon[36739]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-11T19:31:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-11T19:31:28.139 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:27 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:27.840+0000 7fd9756f1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T19:31:28.139 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:27 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:27.937+0000 7fd9756f1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T19:31:28.522 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:31:28 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y[37613]: 2024-03-11T19:31:28.149+0000 7fd9756f1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T19:31:28.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:28.528+0000 7f66e1ffb700 1 -- 172.21.15.83:0/2541998155 <== mon.2 v2:172.21.15.137:3300/0 8 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7f66d80108b0 con 0x7f66e4100ba0 2024-03-11T19:31:28.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:28 smithi083 ceph-mon[31864]: Standby manager daemon y started 2024-03-11T19:31:28.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:28 smithi083 ceph-mon[31864]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:31:28.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:28 smithi083 ceph-mon[31864]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:31:28.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:28 smithi083 ceph-mon[31864]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:31:28.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:28 smithi083 ceph-mon[31864]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:31:28.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:28 smithi137 ceph-mon[36220]: Standby manager daemon y started 2024-03-11T19:31:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:28 smithi137 ceph-mon[36220]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:31:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:28 smithi137 ceph-mon[36220]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:31:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:28 smithi137 ceph-mon[36220]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:31:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:28 smithi137 ceph-mon[36220]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:31:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:28 smithi204 ceph-mon[36739]: Standby manager daemon y started 2024-03-11T19:31:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:28 smithi204 ceph-mon[36739]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-11T19:31:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:28 smithi204 ceph-mon[36739]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-11T19:31:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:28 smithi204 ceph-mon[36739]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-11T19:31:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:28 smithi204 ceph-mon[36739]: from='mgr.? 172.21.15.137:0/2590613652' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-11T19:31:29.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:29 smithi083 ceph-mon[31864]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:29.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:29 smithi083 ceph-mon[31864]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-11T19:31:29.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:31:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:29 smithi137 ceph-mon[36220]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:29 smithi137 ceph-mon[36220]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-11T19:31:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:31:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:29 smithi204 ceph-mon[36739]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:29 smithi204 ceph-mon[36739]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-11T19:31:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-11T19:31:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:30 smithi083 ceph-mon[31864]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:31.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:31.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:31.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:31.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:31.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:30 smithi137 ceph-mon[36220]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:30 smithi204 ceph-mon[36739]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:32.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-11T19:31:32.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:32.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-11T19:31:32.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:32.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-11T19:31:32.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:33.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:33 smithi083 ceph-mon[31864]: Deploying daemon osd.0 on smithi083 2024-03-11T19:31:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:33 smithi083 ceph-mon[31864]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:33.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:33 smithi137 ceph-mon[36220]: Deploying daemon osd.0 on smithi083 2024-03-11T19:31:33.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:33 smithi137 ceph-mon[36220]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:33.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:33 smithi204 ceph-mon[36739]: Deploying daemon osd.0 on smithi083 2024-03-11T19:31:33.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:33 smithi204 ceph-mon[36739]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:35.770 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:35 smithi083 ceph-mon[31864]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:35.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:35 smithi137 ceph-mon[36220]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:35.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:35 smithi204 ceph-mon[36739]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:37.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:37 smithi083 ceph-mon[31864]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:37.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:37.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:37.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:37 smithi137 ceph-mon[36220]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:37.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:37.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:37 smithi204 ceph-mon[36739]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:38.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:38.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:38.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:39.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:39 smithi083 ceph-mon[31864]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:39.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:39 smithi137 ceph-mon[36220]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:39.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:39 smithi204 ceph-mon[36739]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:40.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.906+0000 7f66e1ffb700 1 -- 172.21.15.83:0/2541998155 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f66e406a030 con 0x7f66d4038f90 2024-03-11T19:31:40.911 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 0 on host 'smithi083' 2024-03-11T19:31:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 -- 172.21.15.83:0/2541998155 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66d4038f90 msgr2=0x7f66d403b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66d4038f90 0x7f66d403b450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66d0000c00 tx=0x7f66d0005fb0 comp rx=0 tx=0).stop 2024-03-11T19:31:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 -- 172.21.15.83:0/2541998155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e4100ba0 msgr2=0x7f66e4197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e4100ba0 0x7f66e4197ce0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f66d800ea50 tx=0x7f66d800ed60 comp rx=0 tx=0).stop 2024-03-11T19:31:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 -- 172.21.15.83:0/2541998155 shutdown_connections 2024-03-11T19:31:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66d4038f90 0x7f66d403b450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4102740 0x7f66e419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e4100ba0 0x7f66e4197ce0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 --2- 172.21.15.83:0/2541998155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66e4101da0 0x7f66e4198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 -- 172.21.15.83:0/2541998155 >> 172.21.15.83:0/2541998155 conn(0x7f66e40fc140 msgr2=0x7f66e4069920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:31:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 -- 172.21.15.83:0/2541998155 shutdown_connections 2024-03-11T19:31:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:31:40.909+0000 7f66eafb0700 1 -- 172.21.15.83:0/2541998155 wait complete. 2024-03-11T19:31:41.562 DEBUG:teuthology.orchestra.run.smithi083:osd.0> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.0.service 2024-03-11T19:31:41.564 INFO:tasks.cephadm:Deploying osd.1 on smithi137 with /dev/vg_nvme/lv_4... 2024-03-11T19:31:41.564 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-11T19:31:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:31:41.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:41 smithi083 ceph-mon[31864]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:41.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:41 smithi083 ceph-mon[31864]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-11T19:31:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:41 smithi137 ceph-mon[36220]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:41 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:41 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:41 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:41 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:41 smithi137 ceph-mon[36220]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-11T19:31:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:41 smithi204 ceph-mon[36739]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:41 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:41 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:41 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:41 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:41 smithi204 ceph-mon[36739]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-11T19:31:43.063 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:42 smithi137 ceph-mon[36220]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-11T19:31:43.063 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:42 smithi137 ceph-mon[36220]: osdmap e6: 1 total, 0 up, 1 in 2024-03-11T19:31:43.064 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:42 smithi137 ceph-mon[36220]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:31:43.064 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:42 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:43.064 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:42 smithi137 ceph-mon[36220]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:43.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:42 smithi083 ceph-mon[31864]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-11T19:31:43.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:42 smithi083 ceph-mon[31864]: osdmap e6: 1 total, 0 up, 1 in 2024-03-11T19:31:43.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:42 smithi083 ceph-mon[31864]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:31:43.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:42 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:43.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:42 smithi083 ceph-mon[31864]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:42 smithi204 ceph-mon[36739]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-11T19:31:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:42 smithi204 ceph-mon[36739]: osdmap e6: 1 total, 0 up, 1 in 2024-03-11T19:31:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:42 smithi204 ceph-mon[36739]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:31:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:42 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:42 smithi204 ceph-mon[36739]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:43.609 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-11T19:31:43.609 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-11T19:31:43.610 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-11T19:31:43.610 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-11T19:31:43.610 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126696 s, 828 MB/s 2024-03-11T19:31:43.610 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-11T19:31:43.923 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:31:43 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0[43998]: 2024-03-11T19:31:43.801+0000 7f7b0e9a0700 -1 osd.0 0 waiting for initial osdmap 2024-03-11T19:31:43.923 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:31:43 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0[43998]: 2024-03-11T19:31:43.806+0000 7f7b05789700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:31:44.189 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:43 smithi137 ceph-mon[36220]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:31:44.190 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:43 smithi137 ceph-mon[36220]: osdmap e7: 1 total, 0 up, 1 in 2024-03-11T19:31:44.190 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:43 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:44.190 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:43 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:44.190 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:43 smithi137 ceph-mon[36220]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' 2024-03-11T19:31:44.216 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-03-11T19:31:44.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:43 smithi083 ceph-mon[31864]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:31:44.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:43 smithi083 ceph-mon[31864]: osdmap e7: 1 total, 0 up, 1 in 2024-03-11T19:31:44.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:43 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:44.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:43 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:44.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:43 smithi083 ceph-mon[31864]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' 2024-03-11T19:31:44.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:43 smithi204 ceph-mon[36739]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:31:44.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:43 smithi204 ceph-mon[36739]: osdmap e7: 1 total, 0 up, 1 in 2024-03-11T19:31:44.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:43 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:44.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:43 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:44.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:43 smithi204 ceph-mon[36739]: from='osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285]' entity='osd.0' 2024-03-11T19:31:44.457 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:31:45.095 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285] boot 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: osdmap e8: 1 total, 1 up, 1 in 2024-03-11T19:31:45.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:45.097 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:45.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:31:45.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:31:45.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:45.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:45.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:31:45.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:45.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:45.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285] boot 2024-03-11T19:31:45.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: osdmap e8: 1 total, 1 up, 1 in 2024-03-11T19:31:45.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:45.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:45.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:31:45.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:31:45.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:45.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-11T19:31:45.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:31:45.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:45.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:45.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:45.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: osd.0 [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285] boot 2024-03-11T19:31:45.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: osdmap e8: 1 total, 1 up, 1 in 2024-03-11T19:31:45.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-11T19:31:45.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:31:45.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.864+0000 7fc69605c700 1 -- 172.21.15.137:0/768111232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc690101ca0 msgr2=0x7fc6901020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:45.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.864+0000 7fc69605c700 1 --2- 172.21.15.137:0/768111232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc690101ca0 0x7fc6901020c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc680009920 tx=0x7fc680009c30 comp rx=0 tx=0).stop 2024-03-11T19:31:45.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.864+0000 7fc69605c700 1 -- 172.21.15.137:0/768111232 shutdown_connections 2024-03-11T19:31:45.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.864+0000 7fc69605c700 1 --2- 172.21.15.137:0/768111232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc690103840 0x7fc69010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:45.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.864+0000 7fc69605c700 1 --2- 172.21.15.137:0/768111232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc690101ca0 0x7fc6901020c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:45.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.864+0000 7fc69605c700 1 --2- 172.21.15.137:0/768111232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc690102ea0 0x7fc690103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:45.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.864+0000 7fc69605c700 1 -- 172.21.15.137:0/768111232 >> 172.21.15.137:0/768111232 conn(0x7fc6900fd280 msgr2=0x7fc6900ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:31:45.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.864+0000 7fc69605c700 1 -- 172.21.15.137:0/768111232 shutdown_connections 2024-03-11T19:31:45.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.864+0000 7fc69605c700 1 -- 172.21.15.137:0/768111232 wait complete. 2024-03-11T19:31:45.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc69605c700 1 Processor -- start 2024-03-11T19:31:45.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc69605c700 1 -- start start 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc69605c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc690101ca0 0x7fc690197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc69605c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc690102ea0 0x7fc690198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc69605c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc690103840 0x7fc69019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc69605c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69019f840 con 0x7fc690102ea0 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc69605c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69019f980 con 0x7fc690103840 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc69605c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69019fb40 con 0x7fc690101ca0 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc68f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc690101ca0 0x7fc690197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc68ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc690103840 0x7fc69019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc68f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc690101ca0 0x7fc690197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.137:50882/0 (socket says 172.21.15.137:50882) 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc68f7fe700 1 -- 172.21.15.137:0/650671785 learned_addr learned my addr 172.21.15.137:0/650671785 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc68effd700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc690102ea0 0x7fc690198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:45.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc68f7fe700 1 -- 172.21.15.137:0/650671785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc690103840 msgr2=0x7fc69019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:45.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc68f7fe700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc690103840 0x7fc69019f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:45.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc68f7fe700 1 -- 172.21.15.137:0/650671785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc690102ea0 msgr2=0x7fc690198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:31:45.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.865+0000 7fc68f7fe700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc690102ea0 0x7fc690198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:31:45.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.866+0000 7fc68f7fe700 1 -- 172.21.15.137:0/650671785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc680009520 con 0x7fc690101ca0 2024-03-11T19:31:45.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.866+0000 7fc68ffff700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc690103840 0x7fc69019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:31:45.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.866+0000 7fc68f7fe700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc690101ca0 0x7fc690197d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc680009230 tx=0x7fc680005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:31:45.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.866+0000 7fc68cff9700 1 -- 172.21.15.137:0/650671785 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68001c070 con 0x7fc690101ca0 2024-03-11T19:31:45.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.866+0000 7fc69605c700 1 -- 172.21.15.137:0/650671785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69019fdc0 con 0x7fc690101ca0 2024-03-11T19:31:45.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.866+0000 7fc68cff9700 1 -- 172.21.15.137:0/650671785 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc68000f910 con 0x7fc690101ca0 2024-03-11T19:31:45.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.866+0000 7fc68cff9700 1 -- 172.21.15.137:0/650671785 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680003a00 con 0x7fc690101ca0 2024-03-11T19:31:45.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.866+0000 7fc69605c700 1 -- 172.21.15.137:0/650671785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6901a0180 con 0x7fc690101ca0 2024-03-11T19:31:45.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.867+0000 7fc69605c700 1 -- 172.21.15.137:0/650671785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc690109180 con 0x7fc690101ca0 2024-03-11T19:31:45.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.868+0000 7fc68cff9700 1 -- 172.21.15.137:0/650671785 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7fc680020030 con 0x7fc690101ca0 2024-03-11T19:31:45.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.868+0000 7fc68cff9700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6780a1a30 0x7fc6780a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:31:45.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.869+0000 7fc68cff9700 1 -- 172.21.15.137:0/650671785 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fc6800cc890 con 0x7fc690101ca0 2024-03-11T19:31:45.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.869+0000 7fc68effd700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6780a1a30 0x7fc6780a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:31:45.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.869+0000 7fc68effd700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6780a1a30 0x7fc6780a3ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc684005fd0 tx=0x7fc684009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:31:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:45.872+0000 7fc68cff9700 1 -- 172.21.15.137:0/650671785 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc680097500 con 0x7fc690101ca0 2024-03-11T19:31:46.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:31:46.073+0000 7fc69605c700 1 -- 172.21.15.137:0/650671785 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc69002d990 con 0x7fc6780a1a30 2024-03-11T19:31:46.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:45 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi083 2024-03-11T19:31:46.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:45 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi083 to 17088M 2024-03-11T19:31:46.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:46.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:46.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:46.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:45 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi083 2024-03-11T19:31:46.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:45 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi083 to 17088M 2024-03-11T19:31:46.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:46.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:46.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:46.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:45 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi083 2024-03-11T19:31:46.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:45 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi083 to 17088M 2024-03-11T19:31:46.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:46.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:31:46.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:31:47.188 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:46 smithi137 ceph-mon[36220]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:31:47.188 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:31:47.188 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:31:47.188 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:47.188 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:46 smithi137 ceph-mon[36220]: osdmap e9: 1 total, 1 up, 1 in 2024-03-11T19:31:47.188 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:46 smithi137 ceph-mon[36220]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:47.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:46 smithi083 ceph-mon[31864]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:31:47.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:31:47.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:31:47.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:47.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:46 smithi083 ceph-mon[31864]: osdmap e9: 1 total, 1 up, 1 in 2024-03-11T19:31:47.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:46 smithi083 ceph-mon[31864]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:46 smithi204 ceph-mon[36739]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:31:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:31:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:31:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:46 smithi204 ceph-mon[36739]: osdmap e9: 1 total, 1 up, 1 in 2024-03-11T19:31:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:46 smithi204 ceph-mon[36739]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:49 smithi083 ceph-mon[31864]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:49 smithi083 ceph-mon[31864]: from='client.? 172.21.15.137:0/2180943651' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb07056c-aceb-4fd5-8b42-c1e33fede2b3"}]: dispatch 2024-03-11T19:31:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:49 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb07056c-aceb-4fd5-8b42-c1e33fede2b3"}]: dispatch 2024-03-11T19:31:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:49 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb07056c-aceb-4fd5-8b42-c1e33fede2b3"}]': finished 2024-03-11T19:31:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:49 smithi083 ceph-mon[31864]: osdmap e10: 2 total, 1 up, 2 in 2024-03-11T19:31:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:31:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:49 smithi083 ceph-mon[31864]: from='client.? 172.21.15.137:0/4191576386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:31:49.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:49 smithi137 ceph-mon[36220]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:49 smithi137 ceph-mon[36220]: from='client.? 172.21.15.137:0/2180943651' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb07056c-aceb-4fd5-8b42-c1e33fede2b3"}]: dispatch 2024-03-11T19:31:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:49 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb07056c-aceb-4fd5-8b42-c1e33fede2b3"}]: dispatch 2024-03-11T19:31:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:49 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb07056c-aceb-4fd5-8b42-c1e33fede2b3"}]': finished 2024-03-11T19:31:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:49 smithi137 ceph-mon[36220]: osdmap e10: 2 total, 1 up, 2 in 2024-03-11T19:31:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:31:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:49 smithi137 ceph-mon[36220]: from='client.? 172.21.15.137:0/4191576386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:31:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:49 smithi204 ceph-mon[36739]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:49 smithi204 ceph-mon[36739]: from='client.? 172.21.15.137:0/2180943651' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb07056c-aceb-4fd5-8b42-c1e33fede2b3"}]: dispatch 2024-03-11T19:31:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:49 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb07056c-aceb-4fd5-8b42-c1e33fede2b3"}]: dispatch 2024-03-11T19:31:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:49 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb07056c-aceb-4fd5-8b42-c1e33fede2b3"}]': finished 2024-03-11T19:31:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:49 smithi204 ceph-mon[36739]: osdmap e10: 2 total, 1 up, 2 in 2024-03-11T19:31:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:31:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:49 smithi204 ceph-mon[36739]: from='client.? 172.21.15.137:0/4191576386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:31:51.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:51 smithi083 ceph-mon[31864]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:51.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:51 smithi137 ceph-mon[36220]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:51.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:51 smithi204 ceph-mon[36739]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:53.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:53 smithi137 ceph-mon[36220]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:53.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:53 smithi083 ceph-mon[31864]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:53 smithi204 ceph-mon[36739]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:55.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:55 smithi137 ceph-mon[36220]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:55.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:55 smithi083 ceph-mon[31864]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:55 smithi204 ceph-mon[36739]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:56.677 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:56 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-11T19:31:56.678 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:56 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:56.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:56 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-11T19:31:56.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:56 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:56 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-11T19:31:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:56 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:31:57.774 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:57 smithi137 ceph-mon[36220]: Deploying daemon osd.1 on smithi137 2024-03-11T19:31:57.774 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:57 smithi137 ceph-mon[36220]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:57.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:57 smithi083 ceph-mon[31864]: Deploying daemon osd.1 on smithi137 2024-03-11T19:31:57.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:57 smithi083 ceph-mon[31864]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:57.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:57 smithi204 ceph-mon[36739]: Deploying daemon osd.1 on smithi137 2024-03-11T19:31:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:57 smithi204 ceph-mon[36739]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:59.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:31:59 smithi137 ceph-mon[36220]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:31:59 smithi083 ceph-mon[31864]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:31:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:31:59 smithi204 ceph-mon[36739]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:01.727 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:01 smithi137 ceph-mon[36220]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:01.727 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:01.727 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:01.727 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:01.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:01 smithi083 ceph-mon[31864]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:01.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:01.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:01.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:01 smithi204 ceph-mon[36739]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:03.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:03 smithi083 ceph-mon[31864]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:03.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:03 smithi137 ceph-mon[36220]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:03.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:03 smithi204 ceph-mon[36739]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:04.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.336+0000 7fc68cff9700 1 -- 172.21.15.137:0/650671785 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc69002d990 con 0x7fc6780a1a30 2024-03-11T19:32:04.341 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 1 on host 'smithi137' 2024-03-11T19:32:04.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.338+0000 7fc69605c700 1 -- 172.21.15.137:0/650671785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6780a1a30 msgr2=0x7fc6780a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:04.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.338+0000 7fc69605c700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6780a1a30 0x7fc6780a3ef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc684005fd0 tx=0x7fc684009310 comp rx=0 tx=0).stop 2024-03-11T19:32:04.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.338+0000 7fc69605c700 1 -- 172.21.15.137:0/650671785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc690101ca0 msgr2=0x7fc690197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:04.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.338+0000 7fc69605c700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc690101ca0 0x7fc690197d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc680009230 tx=0x7fc680005250 comp rx=0 tx=0).stop 2024-03-11T19:32:04.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.338+0000 7fc69605c700 1 -- 172.21.15.137:0/650671785 shutdown_connections 2024-03-11T19:32:04.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.338+0000 7fc69605c700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6780a1a30 0x7fc6780a3ef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:04.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.338+0000 7fc69605c700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc690101ca0 0x7fc690197d40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:04.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.338+0000 7fc69605c700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc690103840 0x7fc69019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:04.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.338+0000 7fc69605c700 1 --2- 172.21.15.137:0/650671785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc690102ea0 0x7fc690198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:04.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.338+0000 7fc69605c700 1 -- 172.21.15.137:0/650671785 >> 172.21.15.137:0/650671785 conn(0x7fc6900fd280 msgr2=0x7fc6901062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:32:04.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.339+0000 7fc69605c700 1 -- 172.21.15.137:0/650671785 shutdown_connections 2024-03-11T19:32:04.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:04.339+0000 7fc69605c700 1 -- 172.21.15.137:0/650671785 wait complete. 2024-03-11T19:32:04.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:04.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:04.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:04.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:04.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:04.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:04.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:04.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:04.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:04 smithi137 ceph-mon[36220]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:04.908 DEBUG:teuthology.orchestra.run.smithi137:osd.1> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.1.service 2024-03-11T19:32:04.910 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-03-11T19:32:04.910 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-11T19:32:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:04 smithi083 ceph-mon[31864]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:05.155 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:32:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:05.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:04 smithi204 ceph-mon[36739]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:05 smithi204 ceph-mon[36739]: from='osd.1 [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:32:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:05 smithi204 ceph-mon[36739]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:32:05.964 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:05 smithi137 ceph-mon[36220]: from='osd.1 [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:32:05.964 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:05 smithi137 ceph-mon[36220]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:32:06.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:05 smithi083 ceph-mon[31864]: from='osd.1 [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:32:06.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:05 smithi083 ceph-mon[31864]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-11T19:32:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:06 smithi204 ceph-mon[36739]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-11T19:32:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:06 smithi204 ceph-mon[36739]: from='osd.1 [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:32:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:06 smithi204 ceph-mon[36739]: osdmap e11: 2 total, 1 up, 2 in 2024-03-11T19:32:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:06 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:06.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:06 smithi204 ceph-mon[36739]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:32:06.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:06 smithi204 ceph-mon[36739]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:07.049 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:06 smithi137 ceph-mon[36220]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-11T19:32:07.049 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:06 smithi137 ceph-mon[36220]: from='osd.1 [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:32:07.049 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:06 smithi137 ceph-mon[36220]: osdmap e11: 2 total, 1 up, 2 in 2024-03-11T19:32:07.050 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:06 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:07.050 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:06 smithi137 ceph-mon[36220]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:32:07.050 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:06 smithi137 ceph-mon[36220]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:06 smithi083 ceph-mon[31864]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-11T19:32:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:06 smithi083 ceph-mon[31864]: from='osd.1 [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:32:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:06 smithi083 ceph-mon[31864]: osdmap e11: 2 total, 1 up, 2 in 2024-03-11T19:32:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:06 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:06 smithi083 ceph-mon[31864]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:32:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:06 smithi083 ceph-mon[31864]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110702 s, 947 MB/s 2024-03-11T19:32:07.134 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-11T19:32:07.640 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1[40817]: 2024-03-11T19:32:07.557+0000 7f2e0904c700 -1 osd.1 0 waiting for initial osdmap 2024-03-11T19:32:07.640 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1[40817]: 2024-03-11T19:32:07.566+0000 7f2dffe35700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:32:07.787 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-03-11T19:32:08.036 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:32:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:32:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: osdmap e12: 2 total, 1 up, 2 in 2024-03-11T19:32:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: from='osd.1 ' entity='osd.1' 2024-03-11T19:32:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:08.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:08.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:08.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:32:08.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: osdmap e12: 2 total, 1 up, 2 in 2024-03-11T19:32:08.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: from='osd.1 ' entity='osd.1' 2024-03-11T19:32:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:32:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: osdmap e12: 2 total, 1 up, 2 in 2024-03-11T19:32:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: from='osd.1 ' entity='osd.1' 2024-03-11T19:32:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:08.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:08.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:08.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:08.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:08 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:32:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:08 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:32:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:08 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi137 2024-03-11T19:32:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:08 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi137 to 17080M 2024-03-11T19:32:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:08 smithi083 ceph-mon[31864]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:08 smithi083 ceph-mon[31864]: osd.1 [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205] boot 2024-03-11T19:32:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:08 smithi083 ceph-mon[31864]: osdmap e13: 2 total, 2 up, 2 in 2024-03-11T19:32:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:09.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:08 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:32:09.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:08 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:32:09.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:08 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi137 2024-03-11T19:32:09.087 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:08 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi137 to 17080M 2024-03-11T19:32:09.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:08 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:09.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:08 smithi204 ceph-mon[36739]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:09.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:08 smithi204 ceph-mon[36739]: osd.1 [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205] boot 2024-03-11T19:32:09.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:08 smithi204 ceph-mon[36739]: osdmap e13: 2 total, 2 up, 2 in 2024-03-11T19:32:09.088 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:08 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:09.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:08 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:32:09.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:08 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:32:09.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:08 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi137 2024-03-11T19:32:09.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:08 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi137 to 17080M 2024-03-11T19:32:09.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:08 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:09.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:08 smithi137 ceph-mon[36220]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-11T19:32:09.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:08 smithi137 ceph-mon[36220]: osd.1 [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205] boot 2024-03-11T19:32:09.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:08 smithi137 ceph-mon[36220]: osdmap e13: 2 total, 2 up, 2 in 2024-03-11T19:32:09.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:08 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-11T19:32:09.563 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.558+0000 7f33cda1a700 1 -- 172.21.15.204:0/1553463687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33c8102ea0 msgr2=0x7f33c8103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:09.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.558+0000 7f33cda1a700 1 --2- 172.21.15.204:0/1553463687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33c8102ea0 0x7f33c8103300 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009990 tx=0x7f33b0009ca0 comp rx=0 tx=0).stop 2024-03-11T19:32:09.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.558+0000 7f33cda1a700 1 -- 172.21.15.204:0/1553463687 shutdown_connections 2024-03-11T19:32:09.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.558+0000 7f33cda1a700 1 --2- 172.21.15.204:0/1553463687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33c8103840 0x7f33c810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:09.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.558+0000 7f33cda1a700 1 --2- 172.21.15.204:0/1553463687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8101ca0 0x7f33c81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:09.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.558+0000 7f33cda1a700 1 --2- 172.21.15.204:0/1553463687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33c8102ea0 0x7f33c8103300 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:09.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.558+0000 7f33cda1a700 1 -- 172.21.15.204:0/1553463687 >> 172.21.15.204:0/1553463687 conn(0x7f33c80fd280 msgr2=0x7f33c80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:32:09.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.558+0000 7f33cda1a700 1 -- 172.21.15.204:0/1553463687 shutdown_connections 2024-03-11T19:32:09.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.558+0000 7f33cda1a700 1 -- 172.21.15.204:0/1553463687 wait complete. 2024-03-11T19:32:09.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33cda1a700 1 Processor -- start 2024-03-11T19:32:09.564 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33cda1a700 1 -- start start 2024-03-11T19:32:09.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33cda1a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33c8101ca0 0x7f33c8197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:09.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33cda1a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8102ea0 0x7f33c8198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:09.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33cda1a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33c8103840 0x7f33c819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:09.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33cda1a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c819f840 con 0x7f33c8103840 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33cda1a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c819f980 con 0x7f33c8102ea0 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33cda1a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c819fb40 con 0x7f33c8101ca0 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33c6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33c8101ca0 0x7f33c8197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33c6ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33c8101ca0 0x7f33c8197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:53204/0 (socket says 172.21.15.204:53204) 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33c6ffd700 1 -- 172.21.15.204:0/2052763825 learned_addr learned my addr 172.21.15.204:0/2052763825 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33be7fc700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8102ea0 0x7f33c8198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33c77fe700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33c8103840 0x7f33c819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33c6ffd700 1 -- 172.21.15.204:0/2052763825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8102ea0 msgr2=0x7f33c8198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33c6ffd700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8102ea0 0x7f33c8198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33c6ffd700 1 -- 172.21.15.204:0/2052763825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33c8103840 msgr2=0x7f33c819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33c6ffd700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33c8103840 0x7f33c819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33c6ffd700 1 -- 172.21.15.204:0/2052763825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b00095f0 con 0x7f33c8101ca0 2024-03-11T19:32:09.567 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.559+0000 7f33c77fe700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33c8103840 0x7f33c819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:32:09.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.560+0000 7f33c6ffd700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33c8101ca0 0x7f33c8197cf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f33b800b500 tx=0x7f33b800b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:32:09.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.560+0000 7f33cca18700 1 -- 172.21.15.204:0/2052763825 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8011830 con 0x7f33c8101ca0 2024-03-11T19:32:09.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.560+0000 7f33cda1a700 1 -- 172.21.15.204:0/2052763825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c819fe20 con 0x7f33c8101ca0 2024-03-11T19:32:09.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.560+0000 7f33cca18700 1 -- 172.21.15.204:0/2052763825 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33b8011e70 con 0x7f33c8101ca0 2024-03-11T19:32:09.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.560+0000 7f33cca18700 1 -- 172.21.15.204:0/2052763825 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b800f2e0 con 0x7f33c8101ca0 2024-03-11T19:32:09.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.560+0000 7f33cda1a700 1 -- 172.21.15.204:0/2052763825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c81a02b0 con 0x7f33c8101ca0 2024-03-11T19:32:09.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.561+0000 7f33cda1a700 1 -- 172.21.15.204:0/2052763825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c8109060 con 0x7f33c8101ca0 2024-03-11T19:32:09.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.562+0000 7f33cca18700 1 -- 172.21.15.204:0/2052763825 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7f33b80105f0 con 0x7f33c8101ca0 2024-03-11T19:32:09.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.562+0000 7f33cca18700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33b40a1a30 0x7f33b40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:09.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.563+0000 7f33cca18700 1 -- 172.21.15.204:0/2052763825 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f33b80cba00 con 0x7f33c8101ca0 2024-03-11T19:32:09.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.563+0000 7f33be7fc700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33b40a1a30 0x7f33b40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:09.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.563+0000 7f33be7fc700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33b40a1a30 0x7f33b40a3ef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33b000b5c0 tx=0x7f33b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:32:09.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.566+0000 7f33cca18700 1 -- 172.21.15.204:0/2052763825 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33b8096460 con 0x7f33c8101ca0 2024-03-11T19:32:09.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:09.766+0000 7f33cda1a700 1 -- 172.21.15.204:0/2052763825 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f33c802da20 con 0x7f33b40a1a30 2024-03-11T19:32:10.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:32:10.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:32:10.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:10.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:32:10.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:32:10.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:32:10.229 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:32:10.229 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:11.067 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:10 smithi083 ceph-mon[31864]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:32:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:10 smithi083 ceph-mon[31864]: osdmap e14: 2 total, 2 up, 2 in 2024-03-11T19:32:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:10 smithi083 ceph-mon[31864]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:10 smithi137 ceph-mon[36220]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:32:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:10 smithi137 ceph-mon[36220]: osdmap e14: 2 total, 2 up, 2 in 2024-03-11T19:32:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:10 smithi137 ceph-mon[36220]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:10 smithi204 ceph-mon[36739]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:32:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:10 smithi204 ceph-mon[36739]: osdmap e14: 2 total, 2 up, 2 in 2024-03-11T19:32:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:10 smithi204 ceph-mon[36739]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:12.701 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:12 smithi204 ceph-mon[36739]: from='client.? 172.21.15.204:0/2792635840' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4f2c4-3f48-4c0f-a438-de6d8f151438"}]: dispatch 2024-03-11T19:32:12.701 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:12 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4f2c4-3f48-4c0f-a438-de6d8f151438"}]: dispatch 2024-03-11T19:32:12.701 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:12 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0f4f2c4-3f48-4c0f-a438-de6d8f151438"}]': finished 2024-03-11T19:32:12.701 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:12 smithi204 ceph-mon[36739]: osdmap e15: 3 total, 2 up, 3 in 2024-03-11T19:32:12.701 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:12.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:12 smithi083 ceph-mon[31864]: from='client.? 172.21.15.204:0/2792635840' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4f2c4-3f48-4c0f-a438-de6d8f151438"}]: dispatch 2024-03-11T19:32:12.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:12 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4f2c4-3f48-4c0f-a438-de6d8f151438"}]: dispatch 2024-03-11T19:32:12.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:12 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0f4f2c4-3f48-4c0f-a438-de6d8f151438"}]': finished 2024-03-11T19:32:12.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:12 smithi083 ceph-mon[31864]: osdmap e15: 3 total, 2 up, 3 in 2024-03-11T19:32:12.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:12.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:12 smithi137 ceph-mon[36220]: from='client.? 172.21.15.204:0/2792635840' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4f2c4-3f48-4c0f-a438-de6d8f151438"}]: dispatch 2024-03-11T19:32:12.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:12 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0f4f2c4-3f48-4c0f-a438-de6d8f151438"}]: dispatch 2024-03-11T19:32:12.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:12 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0f4f2c4-3f48-4c0f-a438-de6d8f151438"}]': finished 2024-03-11T19:32:12.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:12 smithi137 ceph-mon[36220]: osdmap e15: 3 total, 2 up, 3 in 2024-03-11T19:32:12.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:13.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:13 smithi083 ceph-mon[31864]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:13.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:13 smithi083 ceph-mon[31864]: from='client.? 172.21.15.204:0/3055394809' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:32:13.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:13 smithi137 ceph-mon[36220]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:13.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:13 smithi137 ceph-mon[36220]: from='client.? 172.21.15.204:0/3055394809' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:32:13.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:13 smithi204 ceph-mon[36739]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:13.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:13 smithi204 ceph-mon[36739]: from='client.? 172.21.15.204:0/3055394809' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:32:15.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:15 smithi083 ceph-mon[31864]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:15.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:15 smithi137 ceph-mon[36220]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:15.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:15 smithi204 ceph-mon[36739]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:17.758 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:17 smithi204 ceph-mon[36739]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:17.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:17 smithi083 ceph-mon[31864]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:17.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:17 smithi137 ceph-mon[36220]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:19.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:19 smithi204 ceph-mon[36739]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:19.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:19 smithi083 ceph-mon[31864]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:19.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:19 smithi137 ceph-mon[36220]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:20.621 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:20 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-11T19:32:20.621 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:20 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:20.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:20 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-11T19:32:20.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:20 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:20.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:20 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-11T19:32:20.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:20 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:21.617 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:21 smithi204 ceph-mon[36739]: Deploying daemon osd.2 on smithi204 2024-03-11T19:32:21.618 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:21 smithi204 ceph-mon[36739]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:21.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:21 smithi083 ceph-mon[31864]: Deploying daemon osd.2 on smithi204 2024-03-11T19:32:21.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:21 smithi083 ceph-mon[31864]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:21.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:21 smithi137 ceph-mon[36220]: Deploying daemon osd.2 on smithi204 2024-03-11T19:32:21.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:21 smithi137 ceph-mon[36220]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:23.620 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:23 smithi204 ceph-mon[36739]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:23.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:23 smithi083 ceph-mon[31864]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:23.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:23 smithi137 ceph-mon[36220]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:25.800 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:25 smithi204 ceph-mon[36739]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:25.800 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:25 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:25.800 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:25 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:25.800 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:25 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:25 smithi083 ceph-mon[31864]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:25 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:25 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:25 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:25.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:25 smithi137 ceph-mon[36220]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:25.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:25 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:25.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:25 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:25.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:25 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:27.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:27 smithi204 ceph-mon[36739]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:27 smithi083 ceph-mon[31864]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:27.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:27 smithi137 ceph-mon[36220]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:28.235 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.230+0000 7f33cca18700 1 -- 172.21.15.204:0/2052763825 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f33c802da20 con 0x7f33b40a1a30 2024-03-11T19:32:28.235 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-03-11T19:32:28.236 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 -- 172.21.15.204:0/2052763825 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33b40a1a30 msgr2=0x7f33b40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:28.237 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33b40a1a30 0x7f33b40a3ef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33b000b5c0 tx=0x7f33b0005fd0 comp rx=0 tx=0).stop 2024-03-11T19:32:28.237 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 -- 172.21.15.204:0/2052763825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33c8101ca0 msgr2=0x7f33c8197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:28.237 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33c8101ca0 0x7f33c8197cf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f33b800b500 tx=0x7f33b800b810 comp rx=0 tx=0).stop 2024-03-11T19:32:28.237 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 -- 172.21.15.204:0/2052763825 shutdown_connections 2024-03-11T19:32:28.237 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33b40a1a30 0x7f33b40a3ef0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:28.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33c8101ca0 0x7f33c8197cf0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:28.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33c8103840 0x7f33c819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:28.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 --2- 172.21.15.204:0/2052763825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8102ea0 0x7f33c8198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:28.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 -- 172.21.15.204:0/2052763825 >> 172.21.15.204:0/2052763825 conn(0x7f33c80fd280 msgr2=0x7f33c81061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:32:28.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 -- 172.21.15.204:0/2052763825 shutdown_connections 2024-03-11T19:32:28.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:32:28.232+0000 7f33cda1a700 1 -- 172.21.15.204:0/2052763825 wait complete. 2024-03-11T19:32:29.196 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.2.service 2024-03-11T19:32:29.199 INFO:tasks.cephadm:Deploying osd.3 on smithi083 with /dev/vg_nvme/lv_3... 2024-03-11T19:32:29.199 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-11T19:32:29.444 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:32:29.479 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.479 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.479 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:29 smithi204 ceph-mon[36739]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:29.480 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.480 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.480 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:29 smithi204 ceph-mon[36739]: from='osd.2 [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:32:29.480 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:29 smithi204 ceph-mon[36739]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:32:29.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.565 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:29 smithi083 ceph-mon[31864]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:29.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:29 smithi083 ceph-mon[31864]: from='osd.2 [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:32:29.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:29 smithi083 ceph-mon[31864]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:32:29.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:29 smithi137 ceph-mon[36220]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:29 smithi137 ceph-mon[36220]: from='osd.2 [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:32:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:29 smithi137 ceph-mon[36220]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-11T19:32:31.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:30 smithi137 ceph-mon[36220]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-11T19:32:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:30 smithi137 ceph-mon[36220]: osdmap e16: 3 total, 2 up, 3 in 2024-03-11T19:32:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:30 smithi137 ceph-mon[36220]: from='osd.2 [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:32:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:30 smithi137 ceph-mon[36220]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:32:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:30 smithi137 ceph-mon[36220]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:30 smithi204 ceph-mon[36739]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-11T19:32:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:30 smithi204 ceph-mon[36739]: osdmap e16: 3 total, 2 up, 3 in 2024-03-11T19:32:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:30 smithi204 ceph-mon[36739]: from='osd.2 [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:32:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:30 smithi204 ceph-mon[36739]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:32:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:30 smithi204 ceph-mon[36739]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:31.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:30 smithi083 ceph-mon[31864]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-11T19:32:31.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:30 smithi083 ceph-mon[31864]: osdmap e16: 3 total, 2 up, 3 in 2024-03-11T19:32:31.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:30 smithi083 ceph-mon[31864]: from='osd.2 [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:32:31.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:31.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:30 smithi083 ceph-mon[31864]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:32:31.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:30 smithi083 ceph-mon[31864]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-11T19:32:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-11T19:32:31.408 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-03-11T19:32:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-03-11T19:32:31.408 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115421 s, 908 MB/s 2024-03-11T19:32:31.408 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-03-11T19:32:32.005 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:31 smithi083 ceph-mon[31864]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:32:32.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:31 smithi083 ceph-mon[31864]: osdmap e17: 3 total, 2 up, 3 in 2024-03-11T19:32:32.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:32.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:32.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:31 smithi083 ceph-mon[31864]: from='osd.2 ' entity='osd.2' 2024-03-11T19:32:32.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:32.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:31 smithi204 ceph-mon[36739]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:32:32.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:31 smithi204 ceph-mon[36739]: osdmap e17: 3 total, 2 up, 3 in 2024-03-11T19:32:32.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:31 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:32.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:31 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:32.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:31 smithi204 ceph-mon[36739]: from='osd.2 ' entity='osd.2' 2024-03-11T19:32:32.011 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:31 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:32.011 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:32:31 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2[41904]: 2024-03-11T19:32:31.719+0000 7f9064b92700 -1 osd.2 0 waiting for initial osdmap 2024-03-11T19:32:32.012 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:32:31 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2[41904]: 2024-03-11T19:32:31.727+0000 7f905d17e700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:32:32.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-03-11T19:32:32.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:31 smithi137 ceph-mon[36220]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:32:32.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:31 smithi137 ceph-mon[36220]: osdmap e17: 3 total, 2 up, 3 in 2024-03-11T19:32:32.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:31 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:32.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:31 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:32.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:31 smithi137 ceph-mon[36220]: from='osd.2 ' entity='osd.2' 2024-03-11T19:32:32.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:31 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:32:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:32:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:32:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi204 2024-03-11T19:32:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi204 to 17080M 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: osd.2 [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727] boot 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: osdmap e18: 3 total, 3 up, 3 in 2024-03-11T19:32:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:33.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi204 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi204 to 17080M 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:33.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:33.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:33.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: osd.2 [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727] boot 2024-03-11T19:32:33.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: osdmap e18: 3 total, 3 up, 3 in 2024-03-11T19:32:33.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:32:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:32:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi204 2024-03-11T19:32:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi204 to 17080M 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: osd.2 [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727] boot 2024-03-11T19:32:33.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: osdmap e18: 3 total, 3 up, 3 in 2024-03-11T19:32:33.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-11T19:32:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.840+0000 7eff0c777700 1 -- 172.21.15.83:0/3309431727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff04102790 msgr2=0x7eff04109040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.840+0000 7eff0c777700 1 --2- 172.21.15.83:0/3309431727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff04102790 0x7eff04109040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff000099c0 tx=0x7eff00009cd0 comp rx=0 tx=0).stop 2024-03-11T19:32:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.841+0000 7eff0c777700 1 -- 172.21.15.83:0/3309431727 shutdown_connections 2024-03-11T19:32:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.841+0000 7eff0c777700 1 --2- 172.21.15.83:0/3309431727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff04102790 0x7eff04109040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.841+0000 7eff0c777700 1 --2- 172.21.15.83:0/3309431727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff04100bf0 0x7eff04101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.841+0000 7eff0c777700 1 --2- 172.21.15.83:0/3309431727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff04101df0 0x7eff04102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.841+0000 7eff0c777700 1 -- 172.21.15.83:0/3309431727 >> 172.21.15.83:0/3309431727 conn(0x7eff040fc190 msgr2=0x7eff040fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:32:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.841+0000 7eff0c777700 1 -- 172.21.15.83:0/3309431727 shutdown_connections 2024-03-11T19:32:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.841+0000 7eff0c777700 1 -- 172.21.15.83:0/3309431727 wait complete. 2024-03-11T19:32:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.841+0000 7eff0c777700 1 Processor -- start 2024-03-11T19:32:33.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.841+0000 7eff0c777700 1 -- start start 2024-03-11T19:32:33.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff0c777700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff04100bf0 0x7eff04197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:33.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff0c777700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff04101df0 0x7eff04198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:33.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff0c777700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff04102790 0x7eff0419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:33.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff0c777700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0419f750 con 0x7eff04101df0 2024-03-11T19:32:33.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff0c777700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0419f890 con 0x7eff04100bf0 2024-03-11T19:32:33.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff0c777700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0419fa20 con 0x7eff04102790 2024-03-11T19:32:33.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff09d12700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff04101df0 0x7eff04198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:33.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff09d12700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff04101df0 0x7eff04198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44754/0 (socket says 172.21.15.83:44754) 2024-03-11T19:32:33.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff09d12700 1 -- 172.21.15.83:0/2352534585 learned_addr learned my addr 172.21.15.83:0/2352534585 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:32:33.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff0a513700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff04100bf0 0x7eff04197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:33.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff0ad14700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff04102790 0x7eff0419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:33.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff09d12700 1 -- 172.21.15.83:0/2352534585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff04102790 msgr2=0x7eff0419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:33.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff09d12700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff04102790 0x7eff0419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:33.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff09d12700 1 -- 172.21.15.83:0/2352534585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff04100bf0 msgr2=0x7eff04197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:33.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff09d12700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff04100bf0 0x7eff04197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff09d12700 1 -- 172.21.15.83:0/2352534585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff000095f0 con 0x7eff04101df0 2024-03-11T19:32:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff0a513700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff04100bf0 0x7eff04197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:32:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff09d12700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff04101df0 0x7eff04198100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efef400d980 tx=0x7efef400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:32:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7efefb7fe700 1 -- 172.21.15.83:0/2352534585 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4009b40 con 0x7eff04101df0 2024-03-11T19:32:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7efefb7fe700 1 -- 172.21.15.83:0/2352534585 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efef400b6e0 con 0x7eff04101df0 2024-03-11T19:32:33.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff0c777700 1 -- 172.21.15.83:0/2352534585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0419fd00 con 0x7eff04101df0 2024-03-11T19:32:33.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7efefb7fe700 1 -- 172.21.15.83:0/2352534585 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400f620 con 0x7eff04101df0 2024-03-11T19:32:33.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.842+0000 7eff0c777700 1 -- 172.21.15.83:0/2352534585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff041a01f0 con 0x7eff04101df0 2024-03-11T19:32:33.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.843+0000 7eff0c777700 1 -- 172.21.15.83:0/2352534585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff04068930 con 0x7eff04101df0 2024-03-11T19:32:33.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.844+0000 7efefb7fe700 1 -- 172.21.15.83:0/2352534585 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 15) v1 ==== 136223+0+0 (secure 0 0 0) 0x7efef4010460 con 0x7eff04101df0 2024-03-11T19:32:33.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.845+0000 7efefb7fe700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efef00a1ab0 0x7efef00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:33.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.845+0000 7eff0a513700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efef00a1ab0 0x7efef00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:33.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.845+0000 7efefb7fe700 1 -- 172.21.15.83:0/2352534585 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7efef40cbf60 con 0x7eff04101df0 2024-03-11T19:32:33.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.845+0000 7eff0a513700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efef00a1ab0 0x7efef00a3f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff04101c50 tx=0x7efefc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:32:33.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:33.849+0000 7efefb7fe700 1 -- 172.21.15.83:0/2352534585 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efef40967a0 con 0x7eff04101df0 2024-03-11T19:32:34.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:34.050+0000 7eff0c777700 1 -- 172.21.15.83:0/2352534585 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7eff040fe250 con 0x7efef00a1ab0 2024-03-11T19:32:34.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:34 smithi083 ceph-mon[31864]: osdmap e19: 3 total, 3 up, 3 in 2024-03-11T19:32:34.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:34 smithi083 ceph-mon[31864]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:32:34.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:34 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:32:34.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:34 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:32:34.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:34 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:34.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:34 smithi083 ceph-mon[31864]: pgmap v82: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:34.944 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:34 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:32:35.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:34 smithi137 ceph-mon[36220]: osdmap e19: 3 total, 3 up, 3 in 2024-03-11T19:32:35.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:34 smithi137 ceph-mon[36220]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:32:35.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:34 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:32:35.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:34 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:32:35.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:34 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:35.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:34 smithi137 ceph-mon[36220]: pgmap v82: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:35.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:34 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:32:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:34 smithi204 ceph-mon[36739]: osdmap e19: 3 total, 3 up, 3 in 2024-03-11T19:32:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:34 smithi204 ceph-mon[36739]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:32:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:34 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:32:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:34 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:32:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:34 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:34 smithi204 ceph-mon[36739]: pgmap v82: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:34 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:32:36.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:35 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-11T19:32:36.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:35 smithi204 ceph-mon[36739]: osdmap e20: 3 total, 3 up, 3 in 2024-03-11T19:32:36.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:35 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:32:36.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:35 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-11T19:32:36.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:35 smithi083 ceph-mon[31864]: osdmap e20: 3 total, 3 up, 3 in 2024-03-11T19:32:36.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:35 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:32:36.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:35 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-11T19:32:36.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:35 smithi137 ceph-mon[36220]: osdmap e20: 3 total, 3 up, 3 in 2024-03-11T19:32:36.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:35 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:32:36.643 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:32:36 smithi083 sudo[48927]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:32:36.643 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:32:36 smithi083 sudo[48927]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:36.643 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:32:36 smithi083 sudo[48927]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:36.643 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:32:36 smithi083 sudo[48927]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:36.896 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:32:36 smithi083 sudo[48950]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:32:36.896 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:32:36 smithi083 sudo[48950]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:36.896 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:32:36 smithi083 sudo[48950]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:36.896 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:32:36 smithi083 sudo[48950]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:32:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:36 smithi204 ceph-mon[36739]: osdmap e21: 3 total, 3 up, 3 in 2024-03-11T19:32:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:36 smithi204 ceph-mon[36739]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:36 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/740508805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12c251e9-dcee-4d62-b731-9a63cd9815fe"}]: dispatch 2024-03-11T19:32:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:36 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/740508805' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12c251e9-dcee-4d62-b731-9a63cd9815fe"}]': finished 2024-03-11T19:32:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:36 smithi204 ceph-mon[36739]: osdmap e22: 4 total, 3 up, 4 in 2024-03-11T19:32:37.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:37.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:37.168 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:32:36 smithi137 sudo[44692]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:32:37.168 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:32:36 smithi137 sudo[44692]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:37.168 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:32:36 smithi137 sudo[44692]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:37.168 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:32:36 smithi137 sudo[44692]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:37.168 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:32:37.169 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:36 smithi137 ceph-mon[36220]: osdmap e21: 3 total, 3 up, 3 in 2024-03-11T19:32:37.169 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:36 smithi137 ceph-mon[36220]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:37.169 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:36 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/740508805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12c251e9-dcee-4d62-b731-9a63cd9815fe"}]: dispatch 2024-03-11T19:32:37.169 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:36 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/740508805' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12c251e9-dcee-4d62-b731-9a63cd9815fe"}]': finished 2024-03-11T19:32:37.169 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:36 smithi137 ceph-mon[36220]: osdmap e22: 4 total, 3 up, 4 in 2024-03-11T19:32:37.169 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:37.169 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:37.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:32:37.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:36 smithi083 ceph-mon[31864]: osdmap e21: 3 total, 3 up, 3 in 2024-03-11T19:32:37.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:36 smithi083 ceph-mon[31864]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:37.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:36 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/740508805' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12c251e9-dcee-4d62-b731-9a63cd9815fe"}]: dispatch 2024-03-11T19:32:37.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:36 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/740508805' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12c251e9-dcee-4d62-b731-9a63cd9815fe"}]': finished 2024-03-11T19:32:37.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:36 smithi083 ceph-mon[31864]: osdmap e22: 4 total, 3 up, 4 in 2024-03-11T19:32:37.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:37.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:37.639 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:32:37 smithi137 sudo[44695]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:32:37.639 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:32:37 smithi137 sudo[44695]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:37.639 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:32:37 smithi137 sudo[44695]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:37.639 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:32:37 smithi137 sudo[44695]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:37.684 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:32:37 smithi204 sudo[45593]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:32:37.684 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:32:37 smithi204 sudo[45593]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:37.684 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:32:37 smithi204 sudo[45593]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:37.684 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:32:37 smithi204 sudo[45593]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:38.156 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:32:37 smithi204 sudo[45596]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:32:38.156 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:32:37 smithi204 sudo[45596]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:38.156 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:32:37 smithi204 sudo[45596]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:38.156 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:32:37 smithi204 sudo[45596]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:38.246 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:37 smithi083 sudo[49036]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:32:38.246 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:37 smithi083 sudo[49036]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:38.246 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:37 smithi083 sudo[49036]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:38.246 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:37 smithi083 sudo[49036]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:38.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:38 smithi083 sudo[49226]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:32:38.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:38 smithi083 sudo[49226]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:38.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:38 smithi083 sudo[49226]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:38.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:38 smithi083 sudo[49226]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:38.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:38.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:38 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1697707268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:32:38.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:38.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:38 smithi083 ceph-mon[31864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:32:38.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:38.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:38 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1697707268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:32:38.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:38.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:38 smithi137 ceph-mon[36220]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:32:38.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 sudo[45599]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:32:38.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 sudo[45599]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:38.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 sudo[45599]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:38.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 sudo[45599]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:38.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 sudo[45602]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:32:38.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 sudo[45602]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:38.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 sudo[45602]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:38.739 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 sudo[45602]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:38.746+0000 7efefb7fe700 1 -- 172.21.15.83:0/2352534585 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7efef4096b70 con 0x7eff04101df0 2024-03-11T19:32:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1697707268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:32:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:38 smithi204 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:32:39.229 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:38 smithi137 sudo[44698]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-11T19:32:39.229 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:38 smithi137 sudo[44698]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:39.229 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:38 smithi137 sudo[44698]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:39.229 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:38 smithi137 sudo[44698]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:32:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:32:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:32:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:32:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:32:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:32:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:32:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:32:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:39.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:32:39.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:32:39.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:32:39.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:32:39.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:32:39.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-11T19:32:39.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:39 smithi083 ceph-mon[31864]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:32:39.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 sudo[44701]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 sudo[44701]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 sudo[44701]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 sudo[44701]: pam_unix(sudo:session): session closed for user root 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:32:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:32:39.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:32:39.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:32:39.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:32:39.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-11T19:32:39.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:39 smithi137 ceph-mon[36220]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:32:39.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:32:39.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:32:39.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:32:39.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:32:39.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:32:39.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:32:39.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:32:39.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:32:39.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:39.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:32:39.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-11T19:32:39.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-11T19:32:39.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-11T19:32:39.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-11T19:32:39.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-11T19:32:39.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:39 smithi204 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-11T19:32:41.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:41 smithi083 ceph-mon[31864]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:41 smithi137 ceph-mon[36220]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:41 smithi204 ceph-mon[36739]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:43.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:43 smithi083 ceph-mon[31864]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:43.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:43 smithi137 ceph-mon[36220]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:43.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:43 smithi204 ceph-mon[36739]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:44.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-11T19:32:44.773 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:44.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-11T19:32:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:44.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-11T19:32:44.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:45.776 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:45 smithi083 ceph-mon[31864]: Deploying daemon osd.3 on smithi083 2024-03-11T19:32:45.776 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:45 smithi083 ceph-mon[31864]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-03-11T19:32:45.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:45 smithi137 ceph-mon[36220]: Deploying daemon osd.3 on smithi083 2024-03-11T19:32:45.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:45 smithi137 ceph-mon[36220]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-03-11T19:32:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:45 smithi204 ceph-mon[36739]: Deploying daemon osd.3 on smithi083 2024-03-11T19:32:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:45 smithi204 ceph-mon[36739]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-03-11T19:32:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:47 smithi083 ceph-mon[31864]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-11T19:32:47.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:47 smithi137 ceph-mon[36220]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-11T19:32:47.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:47 smithi204 ceph-mon[36739]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-11T19:32:49.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:49 smithi083 ceph-mon[31864]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-11T19:32:49.878 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:49 smithi137 ceph-mon[36220]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-11T19:32:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:49 smithi204 ceph-mon[36739]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-11T19:32:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:50 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:50 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:50 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:50 smithi083 ceph-mon[31864]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-11T19:32:51.078 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:50 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:51.078 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:50 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:51.078 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:50 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:51.078 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:50 smithi137 ceph-mon[36220]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-11T19:32:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:50 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:50 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:50 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:50 smithi204 ceph-mon[36739]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-11T19:32:53.650 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:53 smithi083 ceph-mon[31864]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-11T19:32:53.650 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:53.650 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:53.650 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:53.651 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:53.651 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:53.718 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 3 on host 'smithi083' 2024-03-11T19:32:53.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.714+0000 7efefb7fe700 1 -- 172.21.15.83:0/2352534585 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7eff040fe250 con 0x7efef00a1ab0 2024-03-11T19:32:53.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 -- 172.21.15.83:0/2352534585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efef00a1ab0 msgr2=0x7efef00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:53.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efef00a1ab0 0x7efef00a3f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff04101c50 tx=0x7efefc009040 comp rx=0 tx=0).stop 2024-03-11T19:32:53.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 -- 172.21.15.83:0/2352534585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff04101df0 msgr2=0x7eff04198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:53.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff04101df0 0x7eff04198100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efef400d980 tx=0x7efef400dd40 comp rx=0 tx=0).stop 2024-03-11T19:32:53.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 -- 172.21.15.83:0/2352534585 shutdown_connections 2024-03-11T19:32:53.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efef00a1ab0 0x7efef00a3f70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:53.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff04102790 0x7eff0419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:53.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff04100bf0 0x7eff04197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:53.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 --2- 172.21.15.83:0/2352534585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff04101df0 0x7eff04198100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:53.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 -- 172.21.15.83:0/2352534585 >> 172.21.15.83:0/2352534585 conn(0x7eff040fc190 msgr2=0x7eff040fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:32:53.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 -- 172.21.15.83:0/2352534585 shutdown_connections 2024-03-11T19:32:53.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:32:53.716+0000 7eff0c777700 1 -- 172.21.15.83:0/2352534585 wait complete. 2024-03-11T19:32:53.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:53 smithi137 ceph-mon[36220]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-11T19:32:53.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:53.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:53.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:53.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:53.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:53 smithi204 ceph-mon[36739]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-11T19:32:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:54.311 DEBUG:teuthology.orchestra.run.smithi083:osd.3> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.3.service 2024-03-11T19:32:54.313 INFO:tasks.cephadm:Deploying osd.4 on smithi137 with /dev/vg_nvme/lv_3... 2024-03-11T19:32:54.314 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-11T19:32:54.559 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:32:54.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:54 smithi083 ceph-mon[31864]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-11T19:32:54.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:54 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:54.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:54 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:54.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:54 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:54.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:54 smithi137 ceph-mon[36220]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-11T19:32:54.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:54 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:54.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:54 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:54.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:54 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:54.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:54 smithi204 ceph-mon[36739]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-11T19:32:54.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:54 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:54.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:54 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:54.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:54 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:32:55.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:55 smithi083 ceph-mon[31864]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-11T19:32:55.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:55 smithi083 ceph-mon[31864]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-11T19:32:55.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:55 smithi083 ceph-mon[31864]: osdmap e23: 4 total, 3 up, 4 in 2024-03-11T19:32:55.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:55 smithi083 ceph-mon[31864]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:32:55.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:55 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:55.659 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:32:55 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3[51002]: 2024-03-11T19:32:55.523+0000 7f2ec52b6700 -1 osd.3 0 waiting for initial osdmap 2024-03-11T19:32:55.659 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:32:55 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3[51002]: 2024-03-11T19:32:55.530+0000 7f2ec18aa700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:32:55.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:55 smithi137 ceph-mon[36220]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-11T19:32:55.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:55 smithi137 ceph-mon[36220]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-11T19:32:55.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:55 smithi137 ceph-mon[36220]: osdmap e23: 4 total, 3 up, 4 in 2024-03-11T19:32:55.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:55 smithi137 ceph-mon[36220]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:32:55.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:55 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:55 smithi204 ceph-mon[36739]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-11T19:32:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:55 smithi204 ceph-mon[36739]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-11T19:32:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:55 smithi204 ceph-mon[36739]: osdmap e23: 4 total, 3 up, 4 in 2024-03-11T19:32:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:55 smithi204 ceph-mon[36739]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:32:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:55 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:56.435 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-11T19:32:56.436 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-11T19:32:56.436 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-11T19:32:56.436 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-11T19:32:56.436 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113983 s, 920 MB/s 2024-03-11T19:32:56.436 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-11T19:32:56.771 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:56 smithi137 ceph-mon[36220]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:32:56.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:56 smithi137 ceph-mon[36220]: osdmap e24: 4 total, 3 up, 4 in 2024-03-11T19:32:56.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:56 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:56.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:56 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:56.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:56 smithi083 ceph-mon[31864]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:32:56.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:56 smithi083 ceph-mon[31864]: osdmap e24: 4 total, 3 up, 4 in 2024-03-11T19:32:56.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:56 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:56.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:56 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:56 smithi204 ceph-mon[36739]: from='osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:32:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:56 smithi204 ceph-mon[36739]: osdmap e24: 4 total, 3 up, 4 in 2024-03-11T19:32:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:56 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:56 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:57.048 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-03-11T19:32:57.283 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:32:57.699 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:32:57.699 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:32:57.699 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:57.699 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:57.700 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155] boot 2024-03-11T19:32:57.700 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: osdmap e25: 4 total, 4 up, 4 in 2024-03-11T19:32:57.700 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:57.700 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:57.700 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:57.700 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:57.700 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:57.700 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:57.700 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:57.700 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:57.700 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:57.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:32:57.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:32:57.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:57.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:57.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155] boot 2024-03-11T19:32:57.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: osdmap e25: 4 total, 4 up, 4 in 2024-03-11T19:32:57.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:57.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:32:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:32:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-03-11T19:32:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: osd.3 [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155] boot 2024-03-11T19:32:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: osdmap e25: 4 total, 4 up, 4 in 2024-03-11T19:32:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-11T19:32:57.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:57.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:57.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:57.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:32:57.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:57.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:57.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:32:57.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:32:58.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:58 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi083 2024-03-11T19:32:58.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:58 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi083 to 8544M 2024-03-11T19:32:58.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:58 smithi137 ceph-mon[36220]: osdmap e26: 4 total, 4 up, 4 in 2024-03-11T19:32:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:58 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi083 2024-03-11T19:32:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:58 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi083 to 8544M 2024-03-11T19:32:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:58 smithi083 ceph-mon[31864]: osdmap e26: 4 total, 4 up, 4 in 2024-03-11T19:32:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.889+0000 7fa93a79d700 1 -- 172.21.15.137:0/3704337069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934105060 msgr2=0x7fa934107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.889+0000 7fa93a79d700 1 --2- 172.21.15.137:0/3704337069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934105060 0x7fa934107450 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa928009990 tx=0x7fa928009ca0 comp rx=0 tx=0).stop 2024-03-11T19:32:58.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.889+0000 7fa93a79d700 1 -- 172.21.15.137:0/3704337069 shutdown_connections 2024-03-11T19:32:58.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.889+0000 7fa93a79d700 1 --2- 172.21.15.137:0/3704337069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934107990 0x7fa934109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:58.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.889+0000 7fa93a79d700 1 --2- 172.21.15.137:0/3704337069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa934102730 0x7fa934104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:58.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.889+0000 7fa93a79d700 1 --2- 172.21.15.137:0/3704337069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934105060 0x7fa934107450 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:58.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.889+0000 7fa93a79d700 1 -- 172.21.15.137:0/3704337069 >> 172.21.15.137:0/3704337069 conn(0x7fa9340fc140 msgr2=0x7fa9340fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:32:58.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.889+0000 7fa93a79d700 1 -- 172.21.15.137:0/3704337069 shutdown_connections 2024-03-11T19:32:58.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.889+0000 7fa93a79d700 1 -- 172.21.15.137:0/3704337069 wait complete. 2024-03-11T19:32:58.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.889+0000 7fa93a79d700 1 Processor -- start 2024-03-11T19:32:58.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.889+0000 7fa93a79d700 1 -- start start 2024-03-11T19:32:58.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa93a79d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa934102730 0x7fa934197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:58.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa93a79d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934105060 0x7fa934198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa93a79d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934107990 0x7fa93419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa93a79d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93419f860 con 0x7fa934102730 2024-03-11T19:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa93a79d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93419f9a0 con 0x7fa934105060 2024-03-11T19:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa93a79d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93419fb60 con 0x7fa934107990 2024-03-11T19:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa9337fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934105060 0x7fa934198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa9337fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934105060 0x7fa934198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45696/0 (socket says 172.21.15.137:45696) 2024-03-11T19:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa9337fe700 1 -- 172.21.15.137:0/1064953162 learned_addr learned my addr 172.21.15.137:0/1064953162 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-11T19:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa938d3a700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934107990 0x7fa93419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa933fff700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa934102730 0x7fa934197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:58.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa9337fe700 1 -- 172.21.15.137:0/1064953162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934107990 msgr2=0x7fa93419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa9337fe700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934107990 0x7fa93419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa9337fe700 1 -- 172.21.15.137:0/1064953162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa934102730 msgr2=0x7fa934197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa9337fe700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa934102730 0x7fa934197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa9337fe700 1 -- 172.21.15.137:0/1064953162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9280095f0 con 0x7fa934105060 2024-03-11T19:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.890+0000 7fa9337fe700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934105060 0x7fa934198250 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa9280095c0 tx=0x7fa92800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.891+0000 7fa9317fa700 1 -- 172.21.15.137:0/1064953162 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92801d070 con 0x7fa934105060 2024-03-11T19:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.891+0000 7fa93a79d700 1 -- 172.21.15.137:0/1064953162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93419fde0 con 0x7fa934105060 2024-03-11T19:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.891+0000 7fa9317fa700 1 -- 172.21.15.137:0/1064953162 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa928005030 con 0x7fa934105060 2024-03-11T19:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.891+0000 7fa93a79d700 1 -- 172.21.15.137:0/1064953162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9341a01a0 con 0x7fa934105060 2024-03-11T19:32:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.891+0000 7fa9317fa700 1 -- 172.21.15.137:0/1064953162 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928004ae0 con 0x7fa934105060 2024-03-11T19:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.892+0000 7fa93a79d700 1 -- 172.21.15.137:0/1064953162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa920005360 con 0x7fa934105060 2024-03-11T19:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.893+0000 7fa9317fa700 1 -- 172.21.15.137:0/1064953162 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa928004c40 con 0x7fa934105060 2024-03-11T19:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.893+0000 7fa9317fa700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1b40 0x7fa91c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:32:58.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.893+0000 7fa9317fa700 1 -- 172.21.15.137:0/1064953162 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fa9280cd200 con 0x7fa934105060 2024-03-11T19:32:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.893+0000 7fa933fff700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1b40 0x7fa91c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:32:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.894+0000 7fa933fff700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1b40 0x7fa91c0a4000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9341991f0 tx=0x7fa924009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:32:58.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:58.897+0000 7fa9317fa700 1 -- 172.21.15.137:0/1064953162 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa9280977c0 con 0x7fa934105060 2024-03-11T19:32:58.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:58 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi083 2024-03-11T19:32:58.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:58 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi083 to 8544M 2024-03-11T19:32:58.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:58 smithi204 ceph-mon[36739]: osdmap e26: 4 total, 4 up, 4 in 2024-03-11T19:32:59.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:32:59.097+0000 7fa93a79d700 1 -- 172.21.15.137:0/1064953162 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa920000bf0 con 0x7fa91c0a1b40 2024-03-11T19:32:59.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:59 smithi137 ceph-mon[36220]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:32:59.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:59 smithi137 ceph-mon[36220]: osdmap e27: 4 total, 4 up, 4 in 2024-03-11T19:32:59.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:59 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:32:59.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:59 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:32:59.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:32:59 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:59 smithi083 ceph-mon[31864]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:32:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:59 smithi083 ceph-mon[31864]: osdmap e27: 4 total, 4 up, 4 in 2024-03-11T19:32:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:59 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:32:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:59 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:32:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:32:59 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:32:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:59 smithi204 ceph-mon[36739]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:32:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:59 smithi204 ceph-mon[36739]: osdmap e27: 4 total, 4 up, 4 in 2024-03-11T19:32:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:59 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:32:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:59 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:32:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:32:59 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:00.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:00 smithi083 ceph-mon[31864]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:00.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:00 smithi083 ceph-mon[31864]: osdmap e28: 4 total, 4 up, 4 in 2024-03-11T19:33:00.853 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:00 smithi137 ceph-mon[36220]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:00.853 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:00 smithi137 ceph-mon[36220]: osdmap e28: 4 total, 4 up, 4 in 2024-03-11T19:33:00.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:00 smithi204 ceph-mon[36739]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:00.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:00 smithi204 ceph-mon[36739]: osdmap e28: 4 total, 4 up, 4 in 2024-03-11T19:33:01.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:01 smithi083 ceph-mon[31864]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:33:01.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:01 smithi083 ceph-mon[31864]: osdmap e29: 4 total, 4 up, 4 in 2024-03-11T19:33:01.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:01 smithi137 ceph-mon[36220]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:33:01.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:01 smithi137 ceph-mon[36220]: osdmap e29: 4 total, 4 up, 4 in 2024-03-11T19:33:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:01 smithi204 ceph-mon[36739]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:33:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:01 smithi204 ceph-mon[36739]: osdmap e29: 4 total, 4 up, 4 in 2024-03-11T19:33:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:02 smithi083 ceph-mon[31864]: from='client.? 172.21.15.137:0/357668622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "438ba247-da85-4ac4-967f-3ded2137b07d"}]: dispatch 2024-03-11T19:33:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:02 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "438ba247-da85-4ac4-967f-3ded2137b07d"}]: dispatch 2024-03-11T19:33:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:02 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "438ba247-da85-4ac4-967f-3ded2137b07d"}]': finished 2024-03-11T19:33:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:02 smithi083 ceph-mon[31864]: osdmap e30: 5 total, 4 up, 5 in 2024-03-11T19:33:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:02 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:02 smithi083 ceph-mon[31864]: from='client.? 172.21.15.137:0/3016687066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:33:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:02 smithi137 ceph-mon[36220]: from='client.? 172.21.15.137:0/357668622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "438ba247-da85-4ac4-967f-3ded2137b07d"}]: dispatch 2024-03-11T19:33:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:02 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "438ba247-da85-4ac4-967f-3ded2137b07d"}]: dispatch 2024-03-11T19:33:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:02 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "438ba247-da85-4ac4-967f-3ded2137b07d"}]': finished 2024-03-11T19:33:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:02 smithi137 ceph-mon[36220]: osdmap e30: 5 total, 4 up, 5 in 2024-03-11T19:33:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:02 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:02 smithi137 ceph-mon[36220]: from='client.? 172.21.15.137:0/3016687066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:33:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:02 smithi204 ceph-mon[36739]: from='client.? 172.21.15.137:0/357668622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "438ba247-da85-4ac4-967f-3ded2137b07d"}]: dispatch 2024-03-11T19:33:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:02 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "438ba247-da85-4ac4-967f-3ded2137b07d"}]: dispatch 2024-03-11T19:33:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:02 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "438ba247-da85-4ac4-967f-3ded2137b07d"}]': finished 2024-03-11T19:33:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:02 smithi204 ceph-mon[36739]: osdmap e30: 5 total, 4 up, 5 in 2024-03-11T19:33:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:02 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:02 smithi204 ceph-mon[36739]: from='client.? 172.21.15.137:0/3016687066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:33:03.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:03 smithi083 ceph-mon[31864]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:33:03.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:03 smithi137 ceph-mon[36220]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:33:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:03 smithi204 ceph-mon[36739]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:33:05.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:05 smithi083 ceph-mon[31864]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-11T19:33:05.843 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:05 smithi137 ceph-mon[36220]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-11T19:33:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:05 smithi204 ceph-mon[36739]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-11T19:33:06.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:06 smithi137 ceph-mon[36220]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:06 smithi204 ceph-mon[36739]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:06 smithi083 ceph-mon[31864]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:09.775 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:09 smithi137 ceph-mon[36220]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:09.776 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-11T19:33:09.776 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:09.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:09 smithi083 ceph-mon[31864]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-11T19:33:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:09 smithi204 ceph-mon[36739]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-11T19:33:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:10.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:10 smithi137 ceph-mon[36220]: Deploying daemon osd.4 on smithi137 2024-03-11T19:33:10.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:10 smithi083 ceph-mon[31864]: Deploying daemon osd.4 on smithi137 2024-03-11T19:33:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:10 smithi204 ceph-mon[36739]: Deploying daemon osd.4 on smithi137 2024-03-11T19:33:11.635 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:11 smithi137 ceph-mon[36220]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:11.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:11 smithi083 ceph-mon[31864]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:11 smithi204 ceph-mon[36739]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:13.683 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:13 smithi137 ceph-mon[36220]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:13.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:13 smithi083 ceph-mon[31864]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:13.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:13 smithi204 ceph-mon[36739]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:15.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:15 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:15.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:15 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:15.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:15 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:15.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:15 smithi083 ceph-mon[31864]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:15.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:15 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:15.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:15 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:15.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:15 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:15.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:15 smithi137 ceph-mon[36220]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:15.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:15 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:15.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:15 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:15.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:15 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:15.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:15 smithi204 ceph-mon[36739]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:17.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:17 smithi137 ceph-mon[36220]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:17.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:17 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:17.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:17 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:17.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:17 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:17.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:17 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:17.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:17 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:17.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.813+0000 7fa9317fa700 1 -- 172.21.15.137:0/1064953162 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa920000bf0 con 0x7fa91c0a1b40 2024-03-11T19:33:17.818 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 4 on host 'smithi137' 2024-03-11T19:33:17.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:17 smithi083 ceph-mon[31864]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:17.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:17 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:17.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:17 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:17.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:17 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:17.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:17 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:17.820 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:17 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:17.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 -- 172.21.15.137:0/1064953162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1b40 msgr2=0x7fa91c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:17.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1b40 0x7fa91c0a4000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9341991f0 tx=0x7fa924009190 comp rx=0 tx=0).stop 2024-03-11T19:33:17.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 -- 172.21.15.137:0/1064953162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934105060 msgr2=0x7fa934198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:17.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934105060 0x7fa934198250 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa9280095c0 tx=0x7fa92800f690 comp rx=0 tx=0).stop 2024-03-11T19:33:17.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 -- 172.21.15.137:0/1064953162 shutdown_connections 2024-03-11T19:33:17.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1b40 0x7fa91c0a4000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:17.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934107990 0x7fa93419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:17.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa934102730 0x7fa934197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:17.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 --2- 172.21.15.137:0/1064953162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934105060 0x7fa934198250 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:17.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 -- 172.21.15.137:0/1064953162 >> 172.21.15.137:0/1064953162 conn(0x7fa9340fc140 msgr2=0x7fa9341031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:17.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 -- 172.21.15.137:0/1064953162 shutdown_connections 2024-03-11T19:33:17.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:33:17.815+0000 7fa93a79d700 1 -- 172.21.15.137:0/1064953162 wait complete. 2024-03-11T19:33:17.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:17 smithi204 ceph-mon[36739]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:17.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:17 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:17.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:17 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:17.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:17 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:17.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:17 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:17.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:17 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:18.567 DEBUG:teuthology.orchestra.run.smithi137:osd.4> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.4.service 2024-03-11T19:33:18.569 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-03-11T19:33:18.569 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-11T19:33:18.819 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:33:18.852 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:18 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:18.852 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:18 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:18.852 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:18 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:18.852 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:18 smithi137 ceph-mon[36220]: from='osd.4 [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:33:18.852 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:18 smithi137 ceph-mon[36220]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:33:18.852 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:18 smithi137 ceph-mon[36220]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:18 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:18 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:18 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:18 smithi204 ceph-mon[36739]: from='osd.4 [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:33:18.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:18 smithi204 ceph-mon[36739]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:33:18.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:18 smithi204 ceph-mon[36739]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:18 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:18 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:18 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:18 smithi083 ceph-mon[31864]: from='osd.4 [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:33:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:18 smithi083 ceph-mon[31864]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-11T19:33:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:18 smithi083 ceph-mon[31864]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:19.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:19 smithi204 ceph-mon[36739]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-11T19:33:19.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:19 smithi204 ceph-mon[36739]: from='osd.4 [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:33:19.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:19 smithi204 ceph-mon[36739]: osdmap e31: 5 total, 4 up, 5 in 2024-03-11T19:33:19.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:19.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:19 smithi204 ceph-mon[36739]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:33:19.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:19 smithi137 ceph-mon[36220]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-11T19:33:19.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:19 smithi137 ceph-mon[36220]: from='osd.4 [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:33:19.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:19 smithi137 ceph-mon[36220]: osdmap e31: 5 total, 4 up, 5 in 2024-03-11T19:33:19.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:19.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:19 smithi137 ceph-mon[36220]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:33:20.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:19 smithi083 ceph-mon[31864]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-11T19:33:20.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:19 smithi083 ceph-mon[31864]: from='osd.4 [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:33:20.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:19 smithi083 ceph-mon[31864]: osdmap e31: 5 total, 4 up, 5 in 2024-03-11T19:33:20.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:20.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:19 smithi083 ceph-mon[31864]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:33:20.706 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-11T19:33:20.707 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-11T19:33:20.707 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-11T19:33:20.707 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-11T19:33:20.707 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114201 s, 918 MB/s 2024-03-11T19:33:20.707 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-11T19:33:20.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:20 smithi137 ceph-mon[36220]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:33:20.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:20 smithi137 ceph-mon[36220]: osdmap e32: 5 total, 4 up, 5 in 2024-03-11T19:33:20.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:20 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:20.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:20 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:20.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:20 smithi137 ceph-mon[36220]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:20.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:20 smithi137 ceph-mon[36220]: from='osd.4 ' entity='osd.4' 2024-03-11T19:33:20.891 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:33:20 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4[47521]: 2024-03-11T19:33:20.581+0000 7fe21f6ee700 -1 osd.4 0 waiting for initial osdmap 2024-03-11T19:33:20.891 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:33:20 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4[47521]: 2024-03-11T19:33:20.589+0000 7fe217cda700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:33:21.062 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:20 smithi204 ceph-mon[36739]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:33:21.062 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:20 smithi204 ceph-mon[36739]: osdmap e32: 5 total, 4 up, 5 in 2024-03-11T19:33:21.062 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:20 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:21.062 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:20 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:21.062 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:20 smithi204 ceph-mon[36739]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:21.062 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:20 smithi204 ceph-mon[36739]: from='osd.4 ' entity='osd.4' 2024-03-11T19:33:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:20 smithi083 ceph-mon[31864]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:33:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:20 smithi083 ceph-mon[31864]: osdmap e32: 5 total, 4 up, 5 in 2024-03-11T19:33:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:20 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:20 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:20 smithi083 ceph-mon[31864]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-11T19:33:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:20 smithi083 ceph-mon[31864]: from='osd.4 ' entity='osd.4' 2024-03-11T19:33:21.326 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-03-11T19:33:21.578 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:33:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi137 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi137 to 8540M 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: osd.4 [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703] boot 2024-03-11T19:33:21.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: osdmap e33: 5 total, 5 up, 5 in 2024-03-11T19:33:21.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:33:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:33:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi137 2024-03-11T19:33:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:22.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi137 to 8540M 2024-03-11T19:33:22.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:22.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:22.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:22.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:22.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: osd.4 [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703] boot 2024-03-11T19:33:22.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: osdmap e33: 5 total, 5 up, 5 in 2024-03-11T19:33:22.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:22.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:33:22.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:33:22.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:22.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi137 2024-03-11T19:33:22.125 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:22.125 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:22.125 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:22.125 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:22.125 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi137 to 8540M 2024-03-11T19:33:22.125 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:22.125 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:22.125 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:22.125 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:22.125 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: osd.4 [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703] boot 2024-03-11T19:33:22.126 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: osdmap e33: 5 total, 5 up, 5 in 2024-03-11T19:33:22.126 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-11T19:33:22.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:22 smithi204 ceph-mon[36739]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:22 smithi083 ceph-mon[31864]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:23.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:22 smithi137 ceph-mon[36220]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:23.179 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.172+0000 7f2ca0177700 1 -- 172.21.15.204:0/2587551370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98102ea0 msgr2=0x7f2c98103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:23.179 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.172+0000 7f2ca0177700 1 --2- 172.21.15.204:0/2587551370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98102ea0 0x7f2c98103300 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009990 tx=0x7f2c94009ca0 comp rx=0 tx=0).stop 2024-03-11T19:33:23.179 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.172+0000 7f2ca0177700 1 -- 172.21.15.204:0/2587551370 shutdown_connections 2024-03-11T19:33:23.179 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.172+0000 7f2ca0177700 1 --2- 172.21.15.204:0/2587551370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98103840 0x7f2c9810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:23.179 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.172+0000 7f2ca0177700 1 --2- 172.21.15.204:0/2587551370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98101ca0 0x7f2c981020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.172+0000 7f2ca0177700 1 --2- 172.21.15.204:0/2587551370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98102ea0 0x7f2c98103300 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.172+0000 7f2ca0177700 1 -- 172.21.15.204:0/2587551370 >> 172.21.15.204:0/2587551370 conn(0x7f2c980fd280 msgr2=0x7f2c980ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.172+0000 7f2ca0177700 1 -- 172.21.15.204:0/2587551370 shutdown_connections 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.172+0000 7f2ca0177700 1 -- 172.21.15.204:0/2587551370 wait complete. 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2ca0177700 1 Processor -- start 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2ca0177700 1 -- start start 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2ca0177700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98101ca0 0x7f2c98075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2ca0177700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98102ea0 0x7f2c98074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2ca0177700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98103840 0x7f2c980745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2ca0177700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c98074d20 con 0x7f2c98101ca0 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2ca0177700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c98074e90 con 0x7f2c98102ea0 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2ca0177700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c981a1cf0 con 0x7f2c98103840 2024-03-11T19:33:23.180 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2c9e714700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98103840 0x7f2c980745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2c9e714700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98103840 0x7f2c980745a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:46500/0 (socket says 172.21.15.204:46500) 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2c9e714700 1 -- 172.21.15.204:0/2820485898 learned_addr learned my addr 172.21.15.204:0/2820485898 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2c9d712700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98102ea0 0x7f2c98074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.173+0000 7f2c9df13700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98101ca0 0x7f2c98075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2c9e714700 1 -- 172.21.15.204:0/2820485898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98102ea0 msgr2=0x7f2c98074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2c9e714700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98102ea0 0x7f2c98074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2c9e714700 1 -- 172.21.15.204:0/2820485898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98101ca0 msgr2=0x7f2c98075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2c9e714700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98101ca0 0x7f2c98075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2c9e714700 1 -- 172.21.15.204:0/2820485898 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940095f0 con 0x7f2c98103840 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2c9d712700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98102ea0 0x7f2c98074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2c9df13700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98101ca0 0x7f2c98075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2c9e714700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98103840 0x7f2c980745a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c90007ee0 tx=0x7f2c9000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:23.181 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2c8effd700 1 -- 172.21.15.204:0/2820485898 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90017070 con 0x7f2c98103840 2024-03-11T19:33:23.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2ca0177700 1 -- 172.21.15.204:0/2820485898 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c981a1fd0 con 0x7f2c98103840 2024-03-11T19:33:23.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2c8effd700 1 -- 172.21.15.204:0/2820485898 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c9000f040 con 0x7f2c98103840 2024-03-11T19:33:23.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2c8effd700 1 -- 172.21.15.204:0/2820485898 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c900057f0 con 0x7f2c98103840 2024-03-11T19:33:23.182 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.174+0000 7f2ca0177700 1 -- 172.21.15.204:0/2820485898 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c981a2440 con 0x7f2c98103840 2024-03-11T19:33:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.175+0000 7f2ca0177700 1 -- 172.21.15.204:0/2820485898 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c98109060 con 0x7f2c98103840 2024-03-11T19:33:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.176+0000 7f2c8effd700 1 -- 172.21.15.204:0/2820485898 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2c90005950 con 0x7f2c98103840 2024-03-11T19:33:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.177+0000 7f2c8effd700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c840a1ac0 0x7f2c840a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.177+0000 7f2c9df13700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c840a1ac0 0x7f2c840a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.177+0000 7f2c8effd700 1 -- 172.21.15.204:0/2820485898 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f2c900cbc20 con 0x7f2c98103840 2024-03-11T19:33:23.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.177+0000 7f2c9df13700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c840a1ac0 0x7f2c840a3f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2c88005fd0 tx=0x7f2c88005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:23.186 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.181+0000 7f2c8effd700 1 -- 172.21.15.204:0/2820485898 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2c90095ea0 con 0x7f2c98103840 2024-03-11T19:33:23.388 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:23.383+0000 7f2ca0177700 1 -- 172.21.15.204:0/2820485898 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c9802d8e0 con 0x7f2c840a1ac0 2024-03-11T19:33:24.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:23 smithi083 ceph-mon[31864]: osdmap e34: 5 total, 5 up, 5 in 2024-03-11T19:33:24.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:23 smithi083 ceph-mon[31864]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:24.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:33:24.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:33:24.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:24.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:23 smithi204 ceph-mon[36739]: osdmap e34: 5 total, 5 up, 5 in 2024-03-11T19:33:24.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:23 smithi204 ceph-mon[36739]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:24.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:33:24.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:33:24.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:24.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:23 smithi137 ceph-mon[36220]: osdmap e34: 5 total, 5 up, 5 in 2024-03-11T19:33:24.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:23 smithi137 ceph-mon[36220]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:24.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:33:24.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:33:24.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:24 smithi204 ceph-mon[36739]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:24 smithi083 ceph-mon[31864]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:25.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:24 smithi137 ceph-mon[36220]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:26.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:26 smithi137 ceph-mon[36220]: from='client.? 172.21.15.204:0/3581254817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c57d7878-189e-48d6-8d9a-a4b105dea801"}]: dispatch 2024-03-11T19:33:26.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:26 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c57d7878-189e-48d6-8d9a-a4b105dea801"}]: dispatch 2024-03-11T19:33:26.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:26 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c57d7878-189e-48d6-8d9a-a4b105dea801"}]': finished 2024-03-11T19:33:26.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:26 smithi137 ceph-mon[36220]: osdmap e35: 6 total, 5 up, 6 in 2024-03-11T19:33:26.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:26 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:26.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:26 smithi204 ceph-mon[36739]: from='client.? 172.21.15.204:0/3581254817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c57d7878-189e-48d6-8d9a-a4b105dea801"}]: dispatch 2024-03-11T19:33:26.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:26 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c57d7878-189e-48d6-8d9a-a4b105dea801"}]: dispatch 2024-03-11T19:33:26.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:26 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c57d7878-189e-48d6-8d9a-a4b105dea801"}]': finished 2024-03-11T19:33:26.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:26 smithi204 ceph-mon[36739]: osdmap e35: 6 total, 5 up, 6 in 2024-03-11T19:33:26.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:26 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:26.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:26 smithi083 ceph-mon[31864]: from='client.? 172.21.15.204:0/3581254817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c57d7878-189e-48d6-8d9a-a4b105dea801"}]: dispatch 2024-03-11T19:33:26.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:26 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c57d7878-189e-48d6-8d9a-a4b105dea801"}]: dispatch 2024-03-11T19:33:26.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:26 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c57d7878-189e-48d6-8d9a-a4b105dea801"}]': finished 2024-03-11T19:33:26.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:26 smithi083 ceph-mon[31864]: osdmap e35: 6 total, 5 up, 6 in 2024-03-11T19:33:26.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:26 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:27 smithi204 ceph-mon[36739]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:27 smithi204 ceph-mon[36739]: from='client.? 172.21.15.204:0/3364051875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:33:27.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:27 smithi083 ceph-mon[31864]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:27.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:27 smithi083 ceph-mon[31864]: from='client.? 172.21.15.204:0/3364051875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:33:27.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:27 smithi137 ceph-mon[36220]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:27.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:27 smithi137 ceph-mon[36220]: from='client.? 172.21.15.204:0/3364051875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:33:29.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:29 smithi083 ceph-mon[31864]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:29 smithi137 ceph-mon[36220]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:29.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:29 smithi204 ceph-mon[36739]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:31.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:31 smithi204 ceph-mon[36739]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:31.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:31 smithi083 ceph-mon[31864]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:31.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:31 smithi137 ceph-mon[36220]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:33.562 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:33 smithi204 ceph-mon[36739]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:33.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:33 smithi083 ceph-mon[31864]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:33.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:33 smithi137 ceph-mon[36220]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:34.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:34 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-11T19:33:34.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:34 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:34.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:34 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-11T19:33:34.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:34 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:34.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:34 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-11T19:33:34.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:34 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:35.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:35 smithi083 ceph-mon[31864]: Deploying daemon osd.5 on smithi204 2024-03-11T19:33:35.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:35 smithi083 ceph-mon[31864]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:35.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:35 smithi204 ceph-mon[36739]: Deploying daemon osd.5 on smithi204 2024-03-11T19:33:35.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:35 smithi204 ceph-mon[36739]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:35.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:35 smithi137 ceph-mon[36220]: Deploying daemon osd.5 on smithi204 2024-03-11T19:33:35.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:35 smithi137 ceph-mon[36220]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:37.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:37 smithi204 ceph-mon[36739]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:37.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:37 smithi083 ceph-mon[31864]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:37.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:37 smithi137 ceph-mon[36220]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:39.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:39 smithi083 ceph-mon[31864]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:39.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:39.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:39.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:39.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:39 smithi204 ceph-mon[36739]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:39.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:39.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:39.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:39.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:39 smithi137 ceph-mon[36220]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:39.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:39.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:39.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:41.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:41 smithi083 ceph-mon[31864]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:41.838 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:41 smithi204 ceph-mon[36739]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:41 smithi137 ceph-mon[36220]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:42.454 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.450+0000 7f2c8effd700 1 -- 172.21.15.204:0/2820485898 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2c9802d8e0 con 0x7f2c840a1ac0 2024-03-11T19:33:42.454 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-03-11T19:33:42.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 -- 172.21.15.204:0/2820485898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c840a1ac0 msgr2=0x7f2c840a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:42.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c840a1ac0 0x7f2c840a3f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2c88005fd0 tx=0x7f2c88005f00 comp rx=0 tx=0).stop 2024-03-11T19:33:42.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 -- 172.21.15.204:0/2820485898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98103840 msgr2=0x7f2c980745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:42.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98103840 0x7f2c980745a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c90007ee0 tx=0x7f2c9000d3b0 comp rx=0 tx=0).stop 2024-03-11T19:33:42.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 -- 172.21.15.204:0/2820485898 shutdown_connections 2024-03-11T19:33:42.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2c840a1ac0 0x7f2c840a3f80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:42.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c98103840 0x7f2c980745a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:42.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c98101ca0 0x7f2c98075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:42.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 --2- 172.21.15.204:0/2820485898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98102ea0 0x7f2c98074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:42.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 -- 172.21.15.204:0/2820485898 >> 172.21.15.204:0/2820485898 conn(0x7f2c980fd280 msgr2=0x7f2c981061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:42.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 -- 172.21.15.204:0/2820485898 shutdown_connections 2024-03-11T19:33:42.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:33:42.452+0000 7f2ca0177700 1 -- 172.21.15.204:0/2820485898 wait complete. 2024-03-11T19:33:43.134 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.5.service 2024-03-11T19:33:43.136 INFO:tasks.cephadm:Deploying osd.6 on smithi083 with /dev/vg_nvme/lv_2... 2024-03-11T19:33:43.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-11T19:33:43.385 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:33:43.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:43 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:43 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:43 smithi204 ceph-mon[36739]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:43.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:43 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:43 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:43 smithi204 ceph-mon[36739]: from='osd.5 [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:33:43.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:43 smithi204 ceph-mon[36739]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:33:43.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:43 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:43 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:43 smithi083 ceph-mon[31864]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:43.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:43 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:43 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:43 smithi083 ceph-mon[31864]: from='osd.5 [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:33:43.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:43 smithi083 ceph-mon[31864]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:33:43.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:43 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:43 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:43 smithi137 ceph-mon[36220]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:43.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:43 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:43 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:43.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:43 smithi137 ceph-mon[36220]: from='osd.5 [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:33:43.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:43 smithi137 ceph-mon[36220]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-11T19:33:44.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:44 smithi204 ceph-mon[36739]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-11T19:33:44.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:44 smithi204 ceph-mon[36739]: from='osd.5 [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:33:44.782 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:44 smithi204 ceph-mon[36739]: osdmap e36: 6 total, 5 up, 6 in 2024-03-11T19:33:44.783 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:44.783 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:44 smithi204 ceph-mon[36739]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:33:44.783 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:33:44 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:33:44.520+0000 7fc628be0700 -1 osd.5 0 waiting for initial osdmap 2024-03-11T19:33:44.783 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:33:44 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:33:44.527+0000 7fc6251d4700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:33:44.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:44 smithi083 ceph-mon[31864]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-11T19:33:44.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:44 smithi083 ceph-mon[31864]: from='osd.5 [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:33:44.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:44 smithi083 ceph-mon[31864]: osdmap e36: 6 total, 5 up, 6 in 2024-03-11T19:33:44.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:44.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:44 smithi083 ceph-mon[31864]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:33:44.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:44 smithi137 ceph-mon[36220]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-11T19:33:44.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:44 smithi137 ceph-mon[36220]: from='osd.5 [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:33:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:44 smithi137 ceph-mon[36220]: osdmap e36: 6 total, 5 up, 6 in 2024-03-11T19:33:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:44 smithi137 ceph-mon[36220]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:33:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-11T19:33:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-11T19:33:45.519 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-03-11T19:33:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-03-11T19:33:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112415 s, 933 MB/s 2024-03-11T19:33:45.519 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 2024-03-11T19:33:45.519 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-03-11T19:33:45.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:45 smithi083 ceph-mon[31864]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:45.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:45 smithi083 ceph-mon[31864]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:33:45.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:45 smithi083 ceph-mon[31864]: osdmap e37: 6 total, 5 up, 6 in 2024-03-11T19:33:45.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:45.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:45.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:45 smithi083 ceph-mon[31864]: osd.5 [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517] boot 2024-03-11T19:33:45.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:45 smithi083 ceph-mon[31864]: osdmap e38: 6 total, 6 up, 6 in 2024-03-11T19:33:45.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:45.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:45 smithi204 ceph-mon[36739]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:45.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:45 smithi204 ceph-mon[36739]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:33:45.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:45 smithi204 ceph-mon[36739]: osdmap e37: 6 total, 5 up, 6 in 2024-03-11T19:33:45.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:45.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:45.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:45 smithi204 ceph-mon[36739]: osd.5 [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517] boot 2024-03-11T19:33:45.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:45 smithi204 ceph-mon[36739]: osdmap e38: 6 total, 6 up, 6 in 2024-03-11T19:33:45.752 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:45.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:45 smithi137 ceph-mon[36220]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-11T19:33:45.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:45 smithi137 ceph-mon[36220]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:33:45.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:45 smithi137 ceph-mon[36220]: osdmap e37: 6 total, 5 up, 6 in 2024-03-11T19:33:45.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:45.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:45.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:45 smithi137 ceph-mon[36220]: osd.5 [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517] boot 2024-03-11T19:33:45.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:45 smithi137 ceph-mon[36220]: osdmap e38: 6 total, 6 up, 6 in 2024-03-11T19:33:45.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-11T19:33:46.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-03-11T19:33:46.382 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:33:46.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:33:46.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:33:46.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:46.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:46.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:46.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: osdmap e39: 6 total, 6 up, 6 in 2024-03-11T19:33:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:46 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:33:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:33:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: osdmap e39: 6 total, 6 up, 6 in 2024-03-11T19:33:46.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:46.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:46.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:46.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:46 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:33:46.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:33:46.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:46.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:33:46.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:46.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:46.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:46.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: osdmap e39: 6 total, 6 up, 6 in 2024-03-11T19:33:46.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:33:46.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:46.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:33:46.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:46 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:33:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:47 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi204 2024-03-11T19:33:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:47 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi204 to 8540M 2024-03-11T19:33:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:47 smithi083 ceph-mon[31864]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:33:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:47 smithi083 ceph-mon[31864]: osdmap e40: 6 total, 6 up, 6 in 2024-03-11T19:33:47.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:47 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi204 2024-03-11T19:33:47.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:47 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi204 to 8540M 2024-03-11T19:33:47.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:47 smithi137 ceph-mon[36220]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:33:47.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:47 smithi137 ceph-mon[36220]: osdmap e40: 6 total, 6 up, 6 in 2024-03-11T19:33:47.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:47 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi204 2024-03-11T19:33:47.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:47 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi204 to 8540M 2024-03-11T19:33:47.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:47 smithi204 ceph-mon[36739]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:33:47.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:47 smithi204 ceph-mon[36739]: osdmap e40: 6 total, 6 up, 6 in 2024-03-11T19:33:48.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.099+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1280764703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a4100b90 msgr2=0x7f61a4100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:48.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.099+0000 7f61a8ba3700 1 --2- 172.21.15.83:0/1280764703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a4100b90 0x7f61a4100fb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6194009990 tx=0x7f6194009ca0 comp rx=0 tx=0).stop 2024-03-11T19:33:48.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.099+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1280764703 shutdown_connections 2024-03-11T19:33:48.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.099+0000 7f61a8ba3700 1 --2- 172.21.15.83:0/1280764703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61a4102730 0x7f61a4108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:48.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.099+0000 7f61a8ba3700 1 --2- 172.21.15.83:0/1280764703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a4100b90 0x7f61a4100fb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:48.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.099+0000 7f61a8ba3700 1 --2- 172.21.15.83:0/1280764703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a4101d90 0x7f61a41021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:48.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.099+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1280764703 >> 172.21.15.83:0/1280764703 conn(0x7f61a40fc130 msgr2=0x7f61a40fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:33:48.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.099+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1280764703 shutdown_connections 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.099+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1280764703 wait complete. 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.099+0000 7f61a8ba3700 1 Processor -- start 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a8ba3700 1 -- start start 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a8ba3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61a4100b90 0x7f61a4197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a8ba3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a4101d90 0x7f61a41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a8ba3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a4102730 0x7f61a419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a8ba3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a419f7c0 con 0x7f61a4101d90 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a8ba3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a419f900 con 0x7f61a4102730 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a8ba3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a419fac0 con 0x7f61a4100b90 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a1d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a4101d90 0x7f61a41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a1d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a4101d90 0x7f61a41981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37634/0 (socket says 172.21.15.83:37634) 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a1d9b700 1 -- 172.21.15.83:0/1192133459 learned_addr learned my addr 172.21.15.83:0/1192133459 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a259c700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61a4100b90 0x7f61a4197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:48.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a2d9d700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a4102730 0x7f61a419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a1d9b700 1 -- 172.21.15.83:0/1192133459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61a4100b90 msgr2=0x7f61a4197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a1d9b700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61a4100b90 0x7f61a4197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a1d9b700 1 -- 172.21.15.83:0/1192133459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a4102730 msgr2=0x7f61a419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a1d9b700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a4102730 0x7f61a419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a1d9b700 1 -- 172.21.15.83:0/1192133459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61940095f0 con 0x7f61a4101d90 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a259c700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61a4100b90 0x7f61a4197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61a1d9b700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a4101d90 0x7f61a41981b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f619800d950 tx=0x7f619800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.100+0000 7f61937fe700 1 -- 172.21.15.83:0/1192133459 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198009b40 con 0x7f61a4101d90 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.101+0000 7f61937fe700 1 -- 172.21.15.83:0/1192133459 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6198009ca0 con 0x7f61a4101d90 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.101+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1192133459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a419fda0 con 0x7f61a4101d90 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.101+0000 7f61937fe700 1 -- 172.21.15.83:0/1192133459 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619800f6b0 con 0x7f61a4101d90 2024-03-11T19:33:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.101+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1192133459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a41a01e0 con 0x7f61a4101d90 2024-03-11T19:33:48.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.102+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1192133459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a4107fd0 con 0x7f61a4101d90 2024-03-11T19:33:48.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.103+0000 7f61937fe700 1 -- 172.21.15.83:0/1192133459 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6198010460 con 0x7f61a4101d90 2024-03-11T19:33:48.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.103+0000 7f61937fe700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f618c0a1ac0 0x7f618c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:33:48.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.103+0000 7f61a259c700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f618c0a1ac0 0x7f618c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:33:48.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.103+0000 7f61937fe700 1 -- 172.21.15.83:0/1192133459 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f61980cc880 con 0x7f61a4101d90 2024-03-11T19:33:48.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.104+0000 7f61a259c700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f618c0a1ac0 0x7f618c0a3f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6194003790 tx=0x7f6194005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:33:48.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.107+0000 7f61937fe700 1 -- 172.21.15.83:0/1192133459 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6198096930 con 0x7f61a4101d90 2024-03-11T19:33:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:33:48.308+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1192133459 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f61a402da20 con 0x7f618c0a1ac0 2024-03-11T19:33:49.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:49 smithi083 ceph-mon[31864]: osdmap e41: 6 total, 6 up, 6 in 2024-03-11T19:33:49.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:49 smithi083 ceph-mon[31864]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:49.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:33:49.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:33:49.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:49.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:49 smithi083 ceph-mon[31864]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:33:49.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:49 smithi137 ceph-mon[36220]: osdmap e41: 6 total, 6 up, 6 in 2024-03-11T19:33:49.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:49 smithi137 ceph-mon[36220]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:33:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:33:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:49 smithi137 ceph-mon[36220]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:33:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:49 smithi204 ceph-mon[36739]: osdmap e41: 6 total, 6 up, 6 in 2024-03-11T19:33:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:49 smithi204 ceph-mon[36739]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:33:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:33:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:33:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:49 smithi204 ceph-mon[36739]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:33:50.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:50 smithi083 ceph-mon[31864]: osdmap e42: 6 total, 6 up, 6 in 2024-03-11T19:33:50.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:50 smithi137 ceph-mon[36220]: osdmap e42: 6 total, 6 up, 6 in 2024-03-11T19:33:50.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:50 smithi204 ceph-mon[36739]: osdmap e42: 6 total, 6 up, 6 in 2024-03-11T19:33:51.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:51 smithi083 ceph-mon[31864]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-11T19:33:51.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:51 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3355382034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "829db8a4-7f58-4770-8bd7-8cd258b2b1be"}]: dispatch 2024-03-11T19:33:51.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:51 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3355382034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "829db8a4-7f58-4770-8bd7-8cd258b2b1be"}]': finished 2024-03-11T19:33:51.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:51 smithi083 ceph-mon[31864]: osdmap e43: 7 total, 6 up, 7 in 2024-03-11T19:33:51.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:51 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:33:51.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:51 smithi137 ceph-mon[36220]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-11T19:33:51.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:51 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3355382034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "829db8a4-7f58-4770-8bd7-8cd258b2b1be"}]: dispatch 2024-03-11T19:33:51.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:51 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3355382034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "829db8a4-7f58-4770-8bd7-8cd258b2b1be"}]': finished 2024-03-11T19:33:51.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:51 smithi137 ceph-mon[36220]: osdmap e43: 7 total, 6 up, 7 in 2024-03-11T19:33:51.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:51 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:33:51.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:51 smithi204 ceph-mon[36739]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-03-11T19:33:51.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:51 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3355382034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "829db8a4-7f58-4770-8bd7-8cd258b2b1be"}]: dispatch 2024-03-11T19:33:51.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:51 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3355382034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "829db8a4-7f58-4770-8bd7-8cd258b2b1be"}]': finished 2024-03-11T19:33:51.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:51 smithi204 ceph-mon[36739]: osdmap e43: 7 total, 6 up, 7 in 2024-03-11T19:33:51.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:51 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:33:52.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:52 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2109923011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:33:52.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:52 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2109923011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:33:52.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:52 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2109923011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:33:53.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:53 smithi083 ceph-mon[31864]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-11T19:33:53.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:53 smithi137 ceph-mon[36220]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-11T19:33:53.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:53 smithi204 ceph-mon[36739]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-11T19:33:55.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:55 smithi083 ceph-mon[31864]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-03-11T19:33:55.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:55 smithi137 ceph-mon[36220]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-03-11T19:33:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:55 smithi204 ceph-mon[36739]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-03-11T19:33:57.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:57 smithi083 ceph-mon[31864]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:33:57.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:57 smithi137 ceph-mon[36220]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:33:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:57 smithi204 ceph-mon[36739]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:33:59.653 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:59 smithi083 ceph-mon[31864]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:33:59.653 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:59 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-11T19:33:59.653 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:33:59 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:59.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:59 smithi137 ceph-mon[36220]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:33:59.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:59 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-11T19:33:59.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:33:59 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:33:59.905 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:59 smithi204 ceph-mon[36739]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:33:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:59 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-11T19:33:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:33:59 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:00.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:00 smithi137 ceph-mon[36220]: Deploying daemon osd.6 on smithi083 2024-03-11T19:34:00.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:00 smithi204 ceph-mon[36739]: Deploying daemon osd.6 on smithi083 2024-03-11T19:34:00.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:00 smithi083 ceph-mon[31864]: Deploying daemon osd.6 on smithi083 2024-03-11T19:34:01.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:01 smithi083 ceph-mon[31864]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:01.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:01 smithi137 ceph-mon[36220]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:01 smithi204 ceph-mon[36739]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:03.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:03 smithi083 ceph-mon[31864]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:03.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:03 smithi137 ceph-mon[36220]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:03 smithi204 ceph-mon[36739]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:05.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:05 smithi083 ceph-mon[31864]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:05.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:05 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:05.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:05 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:05.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:05 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:05 smithi137 ceph-mon[36220]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:05 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:05 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:05 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:05 smithi204 ceph-mon[36739]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:05 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:05 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:05 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:07.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:07 smithi083 ceph-mon[31864]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:07.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:07 smithi137 ceph-mon[36220]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:07 smithi204 ceph-mon[36739]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:07.936 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 6 on host 'smithi083' 2024-03-11T19:34:07.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.931+0000 7f61937fe700 1 -- 172.21.15.83:0/1192133459 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f61a402da20 con 0x7f618c0a1ac0 2024-03-11T19:34:07.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1192133459 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f618c0a1ac0 msgr2=0x7f618c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f618c0a1ac0 0x7f618c0a3f80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6194003790 tx=0x7f6194005fb0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1192133459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a4101d90 msgr2=0x7f61a41981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a4101d90 0x7f61a41981b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f619800d950 tx=0x7f619800dd10 comp rx=0 tx=0).stop 2024-03-11T19:34:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1192133459 shutdown_connections 2024-03-11T19:34:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f618c0a1ac0 0x7f618c0a3f80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61a4100b90 0x7f61a4197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a4102730 0x7f61a419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 --2- 172.21.15.83:0/1192133459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61a4101d90 0x7f61a41981b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1192133459 >> 172.21.15.83:0/1192133459 conn(0x7f61a40fc130 msgr2=0x7f61a4105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1192133459 shutdown_connections 2024-03-11T19:34:07.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:34:07.934+0000 7f61a8ba3700 1 -- 172.21.15.83:0/1192133459 wait complete. 2024-03-11T19:34:08.690 DEBUG:teuthology.orchestra.run.smithi083:osd.6> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.6.service 2024-03-11T19:34:08.692 INFO:tasks.cephadm:Deploying osd.7 on smithi137 with /dev/vg_nvme/lv_2... 2024-03-11T19:34:08.693 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-11T19:34:08.940 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:34:09.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:08 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:08 smithi083 ceph-mon[31864]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:09.022 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:08 smithi083 ceph-mon[31864]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-11T19:34:09.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:08 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:08 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:08 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:08 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:08 smithi137 ceph-mon[36220]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:09.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:08 smithi137 ceph-mon[36220]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-11T19:34:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:08 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:08 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:08 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:08 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:08 smithi204 ceph-mon[36739]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:08 smithi204 ceph-mon[36739]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-11T19:34:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:10 smithi083 ceph-mon[31864]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-11T19:34:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:10 smithi083 ceph-mon[31864]: osdmap e44: 7 total, 6 up, 7 in 2024-03-11T19:34:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:10 smithi083 ceph-mon[31864]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:34:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:10.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:10 smithi137 ceph-mon[36220]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-11T19:34:10.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:10 smithi137 ceph-mon[36220]: osdmap e44: 7 total, 6 up, 7 in 2024-03-11T19:34:10.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:10 smithi137 ceph-mon[36220]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:34:10.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:10.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:10 smithi204 ceph-mon[36739]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-11T19:34:10.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:10 smithi204 ceph-mon[36739]: osdmap e44: 7 total, 6 up, 7 in 2024-03-11T19:34:10.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:10 smithi204 ceph-mon[36739]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:34:10.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:10.919 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-11T19:34:10.919 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-11T19:34:10.919 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-11T19:34:10.919 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-11T19:34:10.919 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113722 s, 922 MB/s 2024-03-11T19:34:10.919 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-11T19:34:11.434 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:11 smithi083 ceph-mon[31864]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:34:11.434 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:11 smithi083 ceph-mon[31864]: osdmap e45: 7 total, 6 up, 7 in 2024-03-11T19:34:11.435 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:11 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:11.435 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:11 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:11.435 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:11 smithi083 ceph-mon[31864]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:11.435 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:11 smithi083 ceph-mon[31864]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' 2024-03-11T19:34:11.435 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:11 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:11.435 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:34:11 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:34:11.128+0000 7fbbbb2b2700 -1 osd.6 0 waiting for initial osdmap 2024-03-11T19:34:11.435 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:34:11 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:34:11.139+0000 7fbbb389e700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:34:11.513 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:11 smithi137 ceph-mon[36220]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:34:11.513 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:11 smithi137 ceph-mon[36220]: osdmap e45: 7 total, 6 up, 7 in 2024-03-11T19:34:11.513 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:11 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:11.513 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:11 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:11.513 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:11 smithi137 ceph-mon[36220]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:11.513 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:11 smithi137 ceph-mon[36220]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' 2024-03-11T19:34:11.513 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:11 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:11.541 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-03-11T19:34:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:11 smithi204 ceph-mon[36739]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:34:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:11 smithi204 ceph-mon[36739]: osdmap e45: 7 total, 6 up, 7 in 2024-03-11T19:34:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:11 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:11 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:11 smithi204 ceph-mon[36739]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-03-11T19:34:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:11 smithi204 ceph-mon[36739]: from='osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907]' entity='osd.6' 2024-03-11T19:34:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:11 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:11.782 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:34:12.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:34:12.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:34:12.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:12.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:12.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:12.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:12.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:12.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:12.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:12.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:12.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907] boot 2024-03-11T19:34:12.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: osdmap e46: 7 total, 7 up, 7 in 2024-03-11T19:34:12.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:12 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:12.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:12.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:12.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907] boot 2024-03-11T19:34:12.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: osdmap e46: 7 total, 7 up, 7 in 2024-03-11T19:34:12.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:12 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:12.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:34:12.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:34:12.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:12.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:12.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:12.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:12.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:12.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:12.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:12.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:12.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:12.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: osd.6 [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907] boot 2024-03-11T19:34:12.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: osdmap e46: 7 total, 7 up, 7 in 2024-03-11T19:34:12.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:12 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-11T19:34:13.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.520+0000 7f4559e26700 1 -- 172.21.15.137:0/4208618022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102740 msgr2=0x7f4554108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.520+0000 7f4559e26700 1 --2- 172.21.15.137:0/4208618022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102740 0x7f4554108ff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f45440099c0 tx=0x7f4544009cd0 comp rx=0 tx=0).stop 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.520+0000 7f4559e26700 1 -- 172.21.15.137:0/4208618022 shutdown_connections 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.520+0000 7f4559e26700 1 --2- 172.21.15.137:0/4208618022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102740 0x7f4554108ff0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.520+0000 7f4559e26700 1 --2- 172.21.15.137:0/4208618022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4554100ba0 0x7f4554100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.520+0000 7f4559e26700 1 --2- 172.21.15.137:0/4208618022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554101da0 0x7f4554102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.520+0000 7f4559e26700 1 -- 172.21.15.137:0/4208618022 >> 172.21.15.137:0/4208618022 conn(0x7f45540fc140 msgr2=0x7f45540fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.520+0000 7f4559e26700 1 -- 172.21.15.137:0/4208618022 shutdown_connections 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.520+0000 7f4559e26700 1 -- 172.21.15.137:0/4208618022 wait complete. 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4559e26700 1 Processor -- start 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4559e26700 1 -- start start 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4559e26700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554100ba0 0x7f4554197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:13.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4559e26700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4554101da0 0x7f45541981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4559e26700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102740 0x7f455419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4559e26700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455419f810 con 0x7f4554101da0 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4559e26700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455419f950 con 0x7f4554102740 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4559e26700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455419fb10 con 0x7f4554100ba0 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4553fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102740 0x7f455419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4553fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102740 0x7f455419f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59036/0 (socket says 172.21.15.137:59036) 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4553fff700 1 -- 172.21.15.137:0/2667856471 learned_addr learned my addr 172.21.15.137:0/2667856471 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4552ffd700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4554101da0 0x7f45541981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f45537fe700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554100ba0 0x7f4554197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4553fff700 1 -- 172.21.15.137:0/2667856471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554100ba0 msgr2=0x7f4554197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4553fff700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554100ba0 0x7f4554197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4553fff700 1 -- 172.21.15.137:0/2667856471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4554101da0 msgr2=0x7f45541981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:13.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4553fff700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4554101da0 0x7f45541981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:13.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.521+0000 7f4553fff700 1 -- 172.21.15.137:0/2667856471 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440095f0 con 0x7f4554102740 2024-03-11T19:34:13.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.522+0000 7f45537fe700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554100ba0 0x7f4554197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:34:13.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.522+0000 7f4553fff700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102740 0x7f455419f240 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4544005f50 tx=0x7f4544004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:13.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.522+0000 7f4550ff9700 1 -- 172.21.15.137:0/2667856471 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454401d070 con 0x7f4554102740 2024-03-11T19:34:13.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.522+0000 7f4550ff9700 1 -- 172.21.15.137:0/2667856471 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4544004190 con 0x7f4554102740 2024-03-11T19:34:13.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.522+0000 7f4559e26700 1 -- 172.21.15.137:0/2667856471 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455419fd90 con 0x7f4554102740 2024-03-11T19:34:13.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.522+0000 7f4550ff9700 1 -- 172.21.15.137:0/2667856471 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544005310 con 0x7f4554102740 2024-03-11T19:34:13.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.522+0000 7f4559e26700 1 -- 172.21.15.137:0/2667856471 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45541a0140 con 0x7f4554102740 2024-03-11T19:34:13.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.523+0000 7f4559e26700 1 -- 172.21.15.137:0/2667856471 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4554107f00 con 0x7f4554102740 2024-03-11T19:34:13.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.524+0000 7f4550ff9700 1 -- 172.21.15.137:0/2667856471 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4544004a10 con 0x7f4554102740 2024-03-11T19:34:13.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.524+0000 7f4550ff9700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45400a1ac0 0x7f45400a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:13.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.525+0000 7f4550ff9700 1 -- 172.21.15.137:0/2667856471 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f45440cdac0 con 0x7f4554102740 2024-03-11T19:34:13.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.525+0000 7f45537fe700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45400a1ac0 0x7f45400a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:13.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.525+0000 7f45537fe700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45400a1ac0 0x7f45400a3f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4554198b70 tx=0x7f454c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:13.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.528+0000 7f4550ff9700 1 -- 172.21.15.137:0/2667856471 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4544097a70 con 0x7f4554102740 2024-03-11T19:34:13.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:13 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi083 2024-03-11T19:34:13.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:13 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi083 to 5696M 2024-03-11T19:34:13.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:13 smithi137 ceph-mon[36220]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:13.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:13 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi083 2024-03-11T19:34:13.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:13 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi083 to 5696M 2024-03-11T19:34:13.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:13 smithi204 ceph-mon[36739]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:13.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:13.729+0000 7f4559e26700 1 -- 172.21.15.137:0/2667856471 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f455402da20 con 0x7f45400a1ac0 2024-03-11T19:34:13.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:13 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi083 2024-03-11T19:34:13.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:13 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi083 to 5696M 2024-03-11T19:34:13.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:13 smithi083 ceph-mon[31864]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:14.616 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:14 smithi137 ceph-mon[36220]: osdmap e47: 7 total, 7 up, 7 in 2024-03-11T19:34:14.617 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:34:14.617 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:34:14.617 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:14.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:14 smithi204 ceph-mon[36739]: osdmap e47: 7 total, 7 up, 7 in 2024-03-11T19:34:14.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:34:14.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:34:14.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:14.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:14 smithi083 ceph-mon[31864]: osdmap e47: 7 total, 7 up, 7 in 2024-03-11T19:34:14.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:34:14.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:34:14.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:15.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:15 smithi137 ceph-mon[36220]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:34:15.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:15 smithi137 ceph-mon[36220]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:15.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:15 smithi204 ceph-mon[36739]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:34:15.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:15 smithi204 ceph-mon[36739]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:15.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:15 smithi083 ceph-mon[31864]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:34:15.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:15 smithi083 ceph-mon[31864]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:16.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:16 smithi137 ceph-mon[36220]: from='client.? 172.21.15.137:0/4198691167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cf6f477-f5c5-4764-9110-98898779c9f9"}]: dispatch 2024-03-11T19:34:16.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:16 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cf6f477-f5c5-4764-9110-98898779c9f9"}]: dispatch 2024-03-11T19:34:16.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:16 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cf6f477-f5c5-4764-9110-98898779c9f9"}]': finished 2024-03-11T19:34:16.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:16 smithi137 ceph-mon[36220]: osdmap e48: 8 total, 7 up, 8 in 2024-03-11T19:34:16.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:16 smithi204 ceph-mon[36739]: from='client.? 172.21.15.137:0/4198691167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cf6f477-f5c5-4764-9110-98898779c9f9"}]: dispatch 2024-03-11T19:34:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:16 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cf6f477-f5c5-4764-9110-98898779c9f9"}]: dispatch 2024-03-11T19:34:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:16 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cf6f477-f5c5-4764-9110-98898779c9f9"}]': finished 2024-03-11T19:34:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:16 smithi204 ceph-mon[36739]: osdmap e48: 8 total, 7 up, 8 in 2024-03-11T19:34:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:16 smithi083 ceph-mon[31864]: from='client.? 172.21.15.137:0/4198691167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cf6f477-f5c5-4764-9110-98898779c9f9"}]: dispatch 2024-03-11T19:34:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:16 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cf6f477-f5c5-4764-9110-98898779c9f9"}]: dispatch 2024-03-11T19:34:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:16 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cf6f477-f5c5-4764-9110-98898779c9f9"}]': finished 2024-03-11T19:34:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:16 smithi083 ceph-mon[31864]: osdmap e48: 8 total, 7 up, 8 in 2024-03-11T19:34:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:17.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:17 smithi137 ceph-mon[36220]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:17.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:17 smithi137 ceph-mon[36220]: from='client.? 172.21.15.137:0/947778290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:34:17.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:17 smithi204 ceph-mon[36739]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:17.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:17 smithi204 ceph-mon[36739]: from='client.? 172.21.15.137:0/947778290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:34:17.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:17 smithi083 ceph-mon[31864]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:17.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:17 smithi083 ceph-mon[31864]: from='client.? 172.21.15.137:0/947778290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:34:19.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:19 smithi083 ceph-mon[31864]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:19.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:19 smithi137 ceph-mon[36220]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:19 smithi204 ceph-mon[36739]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:21.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:21 smithi137 ceph-mon[36220]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:21.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:21 smithi083 ceph-mon[31864]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:21 smithi204 ceph-mon[36739]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:23.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:23 smithi137 ceph-mon[36220]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:23.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:23 smithi083 ceph-mon[31864]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:23 smithi204 ceph-mon[36739]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:24.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:24 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-11T19:34:24.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:24 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-11T19:34:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:24 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-11T19:34:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:24 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:25.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:25 smithi137 ceph-mon[36220]: Deploying daemon osd.7 on smithi137 2024-03-11T19:34:25.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:25 smithi137 ceph-mon[36220]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:25 smithi083 ceph-mon[31864]: Deploying daemon osd.7 on smithi137 2024-03-11T19:34:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:25 smithi083 ceph-mon[31864]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:25.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:25 smithi204 ceph-mon[36739]: Deploying daemon osd.7 on smithi137 2024-03-11T19:34:25.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:25 smithi204 ceph-mon[36739]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:27.625 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:27 smithi137 ceph-mon[36220]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:27 smithi083 ceph-mon[31864]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:27 smithi204 ceph-mon[36739]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:29.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:29 smithi083 ceph-mon[31864]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:29.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:29.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:29.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:29 smithi137 ceph-mon[36220]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:29.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:29.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:29 smithi204 ceph-mon[36739]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:31.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:31 smithi083 ceph-mon[31864]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:31.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:31 smithi137 ceph-mon[36220]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:31 smithi204 ceph-mon[36739]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:33.000 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 7 on host 'smithi137' 2024-03-11T19:34:33.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.995+0000 7f4550ff9700 1 -- 172.21.15.137:0/2667856471 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f455402da20 con 0x7f45400a1ac0 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 -- 172.21.15.137:0/2667856471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45400a1ac0 msgr2=0x7f45400a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45400a1ac0 0x7f45400a3f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4554198b70 tx=0x7f454c009040 comp rx=0 tx=0).stop 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 -- 172.21.15.137:0/2667856471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102740 msgr2=0x7f455419f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102740 0x7f455419f240 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4544005f50 tx=0x7f4544004730 comp rx=0 tx=0).stop 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 -- 172.21.15.137:0/2667856471 shutdown_connections 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45400a1ac0 0x7f45400a3f80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4554100ba0 0x7f4554197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554102740 0x7f455419f240 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 --2- 172.21.15.137:0/2667856471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4554101da0 0x7f45541981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 -- 172.21.15.137:0/2667856471 >> 172.21.15.137:0/2667856471 conn(0x7f45540fc140 msgr2=0x7f4554105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 -- 172.21.15.137:0/2667856471 shutdown_connections 2024-03-11T19:34:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:34:32.998+0000 7f4559e26700 1 -- 172.21.15.137:0/2667856471 wait complete. 2024-03-11T19:34:33.218 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.218 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.218 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:33.218 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:33.218 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.218 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:33 smithi137 ceph-mon[36220]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:33.218 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.219 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:33.219 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:33 smithi083 ceph-mon[31864]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.611 DEBUG:teuthology.orchestra.run.smithi137:osd.7> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.7.service 2024-03-11T19:34:33.613 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-03-11T19:34:33.613 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-11T19:34:33.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:33 smithi204 ceph-mon[36739]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:33.859 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:34:34.354 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:34 smithi204 ceph-mon[36739]: from='osd.7 [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:34:34.354 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:34 smithi204 ceph-mon[36739]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:34:34.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:34 smithi137 ceph-mon[36220]: from='osd.7 [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:34:34.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:34 smithi137 ceph-mon[36220]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:34:34.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:34 smithi083 ceph-mon[31864]: from='osd.7 [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:34:34.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:34 smithi083 ceph-mon[31864]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-11T19:34:35.319 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:35 smithi204 ceph-mon[36739]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-11T19:34:35.319 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:35 smithi204 ceph-mon[36739]: osdmap e49: 8 total, 7 up, 8 in 2024-03-11T19:34:35.319 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:35 smithi204 ceph-mon[36739]: from='osd.7 [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:34:35.319 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:35 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:35.319 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:35 smithi204 ceph-mon[36739]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:34:35.319 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:35 smithi204 ceph-mon[36739]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:35.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:35 smithi137 ceph-mon[36220]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-11T19:34:35.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:35 smithi137 ceph-mon[36220]: osdmap e49: 8 total, 7 up, 8 in 2024-03-11T19:34:35.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:35 smithi137 ceph-mon[36220]: from='osd.7 [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:34:35.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:35 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:35.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:35 smithi137 ceph-mon[36220]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:34:35.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:35 smithi137 ceph-mon[36220]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:35.390 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:34:35 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:34:35.186+0000 7f56fca67700 -1 osd.7 0 waiting for initial osdmap 2024-03-11T19:34:35.390 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:34:35 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:34:35.197+0000 7f56f905b700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:34:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:35 smithi083 ceph-mon[31864]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-11T19:34:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:35 smithi083 ceph-mon[31864]: osdmap e49: 8 total, 7 up, 8 in 2024-03-11T19:34:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:35 smithi083 ceph-mon[31864]: from='osd.7 [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:34:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:35 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:35 smithi083 ceph-mon[31864]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:34:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:35 smithi083 ceph-mon[31864]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-03-11T19:34:35.880 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-11T19:34:35.880 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-11T19:34:35.880 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-11T19:34:35.880 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-11T19:34:35.880 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011164 s, 939 MB/s 2024-03-11T19:34:35.880 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-11T19:34:36.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:36 smithi137 ceph-mon[36220]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:34:36.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:36 smithi137 ceph-mon[36220]: osdmap e50: 8 total, 7 up, 8 in 2024-03-11T19:34:36.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:36.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:36.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:36 smithi137 ceph-mon[36220]: osd.7 [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017] boot 2024-03-11T19:34:36.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:36 smithi137 ceph-mon[36220]: osdmap e51: 8 total, 8 up, 8 in 2024-03-11T19:34:36.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:36.498 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:36 smithi204 ceph-mon[36739]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:34:36.498 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:36 smithi204 ceph-mon[36739]: osdmap e50: 8 total, 7 up, 8 in 2024-03-11T19:34:36.498 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:36.499 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:36.499 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:36 smithi204 ceph-mon[36739]: osd.7 [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017] boot 2024-03-11T19:34:36.499 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:36 smithi204 ceph-mon[36739]: osdmap e51: 8 total, 8 up, 8 in 2024-03-11T19:34:36.499 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:36.526 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-03-11T19:34:36.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:36 smithi083 ceph-mon[31864]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:34:36.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:36 smithi083 ceph-mon[31864]: osdmap e50: 8 total, 7 up, 8 in 2024-03-11T19:34:36.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:36.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:36.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:36 smithi083 ceph-mon[31864]: osd.7 [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017] boot 2024-03-11T19:34:36.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:36 smithi083 ceph-mon[31864]: osdmap e51: 8 total, 8 up, 8 in 2024-03-11T19:34:36.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-11T19:34:36.811 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:34:37.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:34:37.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: osdmap e52: 8 total, 8 up, 8 in 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi137 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi137 to 5693M 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:37 smithi204 ceph-mon[36739]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:37.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:34:37.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:34:37.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: osdmap e52: 8 total, 8 up, 8 in 2024-03-11T19:34:37.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi137 2024-03-11T19:34:37.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:37.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:37.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:37.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi137 to 5693M 2024-03-11T19:34:37.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:37.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:37.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:37 smithi083 ceph-mon[31864]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:37.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:34:37.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: osdmap e52: 8 total, 8 up, 8 in 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi137 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi137 to 5693M 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:37.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:37 smithi137 ceph-mon[36220]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:38.357 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.351+0000 7f1df17f9700 1 -- 172.21.15.204:0/3511007694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec103840 msgr2=0x7f1dec10a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:38.357 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.351+0000 7f1df17f9700 1 --2- 172.21.15.204:0/3511007694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec103840 0x7f1dec10a0f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1de400b0a0 tx=0x7f1de400b3b0 comp rx=0 tx=0).stop 2024-03-11T19:34:38.357 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.351+0000 7f1df17f9700 1 -- 172.21.15.204:0/3511007694 shutdown_connections 2024-03-11T19:34:38.357 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.351+0000 7f1df17f9700 1 --2- 172.21.15.204:0/3511007694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec103840 0x7f1dec10a0f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:38.357 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.351+0000 7f1df17f9700 1 --2- 172.21.15.204:0/3511007694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dec101ca0 0x7f1dec1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:38.357 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.351+0000 7f1df17f9700 1 --2- 172.21.15.204:0/3511007694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dec102ea0 0x7f1dec103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.351+0000 7f1df17f9700 1 -- 172.21.15.204:0/3511007694 >> 172.21.15.204:0/3511007694 conn(0x7f1dec0fd280 msgr2=0x7f1dec0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.351+0000 7f1df17f9700 1 -- 172.21.15.204:0/3511007694 shutdown_connections 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.351+0000 7f1df17f9700 1 -- 172.21.15.204:0/3511007694 wait complete. 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.352+0000 7f1df17f9700 1 Processor -- start 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.352+0000 7f1df17f9700 1 -- start start 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.352+0000 7f1df17f9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec101ca0 0x7f1dec197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.352+0000 7f1df17f9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dec102ea0 0x7f1dec198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.352+0000 7f1df17f9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dec103840 0x7f1dec19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.352+0000 7f1df17f9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec19f8c0 con 0x7f1dec103840 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.352+0000 7f1df17f9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec19fa00 con 0x7f1dec102ea0 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.352+0000 7f1df17f9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec19fb90 con 0x7f1dec101ca0 2024-03-11T19:34:38.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.352+0000 7f1deaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec101ca0 0x7f1dec197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.352+0000 7f1deaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec101ca0 0x7f1dec197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47746/0 (socket says 172.21.15.204:47746) 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1deaffd700 1 -- 172.21.15.204:0/2590959339 learned_addr learned my addr 172.21.15.204:0/2590959339 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1deb7fe700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dec103840 0x7f1dec19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1dea7fc700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dec102ea0 0x7f1dec198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1deaffd700 1 -- 172.21.15.204:0/2590959339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dec102ea0 msgr2=0x7f1dec198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1deaffd700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dec102ea0 0x7f1dec198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1deaffd700 1 -- 172.21.15.204:0/2590959339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dec103840 msgr2=0x7f1dec19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1deaffd700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dec103840 0x7f1dec19f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1deaffd700 1 -- 172.21.15.204:0/2590959339 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de4009d00 con 0x7f1dec101ca0 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1deaffd700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec101ca0 0x7f1dec197d30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00b530 tx=0x7f1ddc00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1dd3fff700 1 -- 172.21.15.204:0/2590959339 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc011830 con 0x7f1dec101ca0 2024-03-11T19:34:38.359 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1dd3fff700 1 -- 172.21.15.204:0/2590959339 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ddc011e70 con 0x7f1dec101ca0 2024-03-11T19:34:38.360 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1df17f9700 1 -- 172.21.15.204:0/2590959339 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dec19fe70 con 0x7f1dec101ca0 2024-03-11T19:34:38.360 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1dd3fff700 1 -- 172.21.15.204:0/2590959339 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc00f2e0 con 0x7f1dec101ca0 2024-03-11T19:34:38.360 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.353+0000 7f1df17f9700 1 -- 172.21.15.204:0/2590959339 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dec1a0300 con 0x7f1dec101ca0 2024-03-11T19:34:38.361 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.354+0000 7f1df17f9700 1 -- 172.21.15.204:0/2590959339 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dec109060 con 0x7f1dec101ca0 2024-03-11T19:34:38.361 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.355+0000 7f1dd3fff700 1 -- 172.21.15.204:0/2590959339 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1ddc0105f0 con 0x7f1dec101ca0 2024-03-11T19:34:38.361 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.356+0000 7f1dd3fff700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1dd40a1ac0 0x7f1dd40a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:34:38.361 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.356+0000 7f1dd3fff700 1 -- 172.21.15.204:0/2590959339 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f1ddc0cc790 con 0x7f1dec101ca0 2024-03-11T19:34:38.361 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.356+0000 7f1dea7fc700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1dd40a1ac0 0x7f1dd40a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:34:38.361 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.357+0000 7f1dea7fc700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1dd40a1ac0 0x7f1dd40a3f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1de0005ea0 tx=0x7f1de0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:34:38.363 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.359+0000 7f1dd3fff700 1 -- 172.21.15.204:0/2590959339 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ddc096500 con 0x7f1dec101ca0 2024-03-11T19:34:38.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:38 smithi204 ceph-mon[36739]: osdmap e53: 8 total, 8 up, 8 in 2024-03-11T19:34:38.566 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:38.562+0000 7f1df17f9700 1 -- 172.21.15.204:0/2590959339 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dec02da20 con 0x7f1dd40a1ac0 2024-03-11T19:34:38.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:38 smithi083 ceph-mon[31864]: osdmap e53: 8 total, 8 up, 8 in 2024-03-11T19:34:38.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:38 smithi137 ceph-mon[36220]: osdmap e53: 8 total, 8 up, 8 in 2024-03-11T19:34:39.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:39 smithi204 ceph-mon[36739]: osdmap e54: 8 total, 8 up, 8 in 2024-03-11T19:34:39.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:39 smithi204 ceph-mon[36739]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:39.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:39 smithi204 ceph-mon[36739]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:34:39.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:34:39.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:34:39.478 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:39 smithi083 ceph-mon[31864]: osdmap e54: 8 total, 8 up, 8 in 2024-03-11T19:34:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:39 smithi083 ceph-mon[31864]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:39 smithi083 ceph-mon[31864]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:34:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:34:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:34:39.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:39.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:39 smithi137 ceph-mon[36220]: osdmap e54: 8 total, 8 up, 8 in 2024-03-11T19:34:39.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:39 smithi137 ceph-mon[36220]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:39 smithi137 ceph-mon[36220]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:34:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:34:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:34:39.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:40.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:40 smithi204 ceph-mon[36739]: osdmap e55: 8 total, 8 up, 8 in 2024-03-11T19:34:40.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:40 smithi083 ceph-mon[31864]: osdmap e55: 8 total, 8 up, 8 in 2024-03-11T19:34:40.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:40 smithi137 ceph-mon[36220]: osdmap e55: 8 total, 8 up, 8 in 2024-03-11T19:34:41.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:41 smithi204 ceph-mon[36739]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:41.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:41 smithi204 ceph-mon[36739]: from='client.? 172.21.15.204:0/3164627644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e808c31-b428-45c6-896d-42eba00a25f5"}]: dispatch 2024-03-11T19:34:41.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:41 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e808c31-b428-45c6-896d-42eba00a25f5"}]: dispatch 2024-03-11T19:34:41.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:41 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e808c31-b428-45c6-896d-42eba00a25f5"}]': finished 2024-03-11T19:34:41.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:41 smithi204 ceph-mon[36739]: osdmap e56: 9 total, 8 up, 9 in 2024-03-11T19:34:41.567 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:41 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:34:41.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:41 smithi083 ceph-mon[31864]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:41.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:41 smithi083 ceph-mon[31864]: from='client.? 172.21.15.204:0/3164627644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e808c31-b428-45c6-896d-42eba00a25f5"}]: dispatch 2024-03-11T19:34:41.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:41 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e808c31-b428-45c6-896d-42eba00a25f5"}]: dispatch 2024-03-11T19:34:41.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:41 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e808c31-b428-45c6-896d-42eba00a25f5"}]': finished 2024-03-11T19:34:41.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:41 smithi083 ceph-mon[31864]: osdmap e56: 9 total, 8 up, 9 in 2024-03-11T19:34:41.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:34:41.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:41 smithi137 ceph-mon[36220]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:41.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:41 smithi137 ceph-mon[36220]: from='client.? 172.21.15.204:0/3164627644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e808c31-b428-45c6-896d-42eba00a25f5"}]: dispatch 2024-03-11T19:34:41.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:41 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e808c31-b428-45c6-896d-42eba00a25f5"}]: dispatch 2024-03-11T19:34:41.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:41 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e808c31-b428-45c6-896d-42eba00a25f5"}]': finished 2024-03-11T19:34:41.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:41 smithi137 ceph-mon[36220]: osdmap e56: 9 total, 8 up, 9 in 2024-03-11T19:34:41.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:41 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:34:42.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:42 smithi083 ceph-mon[31864]: from='client.? 172.21.15.204:0/440399064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:34:42.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:42 smithi137 ceph-mon[36220]: from='client.? 172.21.15.204:0/440399064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:34:42.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:42 smithi204 ceph-mon[36739]: from='client.? 172.21.15.204:0/440399064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:34:43.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:43 smithi137 ceph-mon[36220]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:43.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:43 smithi204 ceph-mon[36739]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:43.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:43 smithi083 ceph-mon[31864]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:45.766 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:45 smithi204 ceph-mon[36739]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:45 smithi083 ceph-mon[31864]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:45.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:45 smithi137 ceph-mon[36220]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:47.640 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:47 smithi204 ceph-mon[36739]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:47 smithi083 ceph-mon[31864]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:47.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:47 smithi137 ceph-mon[36220]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:49.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:49 smithi204 ceph-mon[36739]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:49.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-11T19:34:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:49 smithi083 ceph-mon[31864]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-11T19:34:49.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:49.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:49 smithi137 ceph-mon[36220]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:49.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-11T19:34:49.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:50.731 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:50 smithi204 ceph-mon[36739]: Deploying daemon osd.8 on smithi204 2024-03-11T19:34:50.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:50 smithi083 ceph-mon[31864]: Deploying daemon osd.8 on smithi204 2024-03-11T19:34:50.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:50 smithi137 ceph-mon[36220]: Deploying daemon osd.8 on smithi204 2024-03-11T19:34:51.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:51 smithi083 ceph-mon[31864]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:51.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:51 smithi204 ceph-mon[36739]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:51.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:51 smithi137 ceph-mon[36220]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:53.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:53 smithi083 ceph-mon[31864]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:53.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:53 smithi204 ceph-mon[36739]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:53.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:53 smithi137 ceph-mon[36220]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:55.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:55 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:55.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:55 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:55.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:55 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:55.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:55 smithi137 ceph-mon[36220]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:55.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:55 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:55.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:55 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:55.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:55 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:55.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:55 smithi204 ceph-mon[36739]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:55.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:55 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:55.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:55 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:55.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:55 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:55.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:55 smithi083 ceph-mon[31864]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:57.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:57 smithi083 ceph-mon[31864]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:57.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:57.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:57 smithi137 ceph-mon[36220]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:57.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:57.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:57.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:57.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:57.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:57 smithi204 ceph-mon[36739]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:34:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:34:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:58.038 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-03-11T19:34:58.038 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.033+0000 7f1dd3fff700 1 -- 172.21.15.204:0/2590959339 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1dec02da20 con 0x7f1dd40a1ac0 2024-03-11T19:34:58.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 -- 172.21.15.204:0/2590959339 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1dd40a1ac0 msgr2=0x7f1dd40a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:58.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1dd40a1ac0 0x7f1dd40a3f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1de0005ea0 tx=0x7f1de0005e30 comp rx=0 tx=0).stop 2024-03-11T19:34:58.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 -- 172.21.15.204:0/2590959339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec101ca0 msgr2=0x7f1dec197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:34:58.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec101ca0 0x7f1dec197d30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00b530 tx=0x7f1ddc00b840 comp rx=0 tx=0).stop 2024-03-11T19:34:58.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 -- 172.21.15.204:0/2590959339 shutdown_connections 2024-03-11T19:34:58.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1dd40a1ac0 0x7f1dd40a3f80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:58.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dec101ca0 0x7f1dec197d30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:58.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dec103840 0x7f1dec19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:58.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 --2- 172.21.15.204:0/2590959339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dec102ea0 0x7f1dec198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:34:58.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 -- 172.21.15.204:0/2590959339 >> 172.21.15.204:0/2590959339 conn(0x7f1dec0fd280 msgr2=0x7f1dec1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:34:58.041 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 -- 172.21.15.204:0/2590959339 shutdown_connections 2024-03-11T19:34:58.041 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:34:58.036+0000 7f1df17f9700 1 -- 172.21.15.204:0/2590959339 wait complete. 2024-03-11T19:34:58.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:58 smithi204 ceph-mon[36739]: from='osd.8 [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:34:58.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:58 smithi204 ceph-mon[36739]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:34:58.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:58 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:58.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:58 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:58.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:58 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:58.695 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.8.service 2024-03-11T19:34:58.697 INFO:tasks.cephadm:Deploying osd.9 on smithi083 with /dev/vg_nvme/lv_1... 2024-03-11T19:34:58.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-11T19:34:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:58 smithi083 ceph-mon[31864]: from='osd.8 [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:34:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:58 smithi083 ceph-mon[31864]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:34:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:58.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:58 smithi137 ceph-mon[36220]: from='osd.8 [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:34:58.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:58 smithi137 ceph-mon[36220]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-11T19:34:58.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:58 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:58.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:58 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:34:58.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:58 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:34:58.947 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:34:59.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:59 smithi204 ceph-mon[36739]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:59.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:59 smithi204 ceph-mon[36739]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-11T19:34:59.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:59 smithi204 ceph-mon[36739]: from='osd.8 [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:34:59.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:59 smithi204 ceph-mon[36739]: osdmap e57: 9 total, 8 up, 9 in 2024-03-11T19:34:59.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:59 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:34:59.774 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:34:59 smithi204 ceph-mon[36739]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:34:59.775 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:34:59 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:34:59.553+0000 7ff98b446700 -1 osd.8 0 waiting for initial osdmap 2024-03-11T19:34:59.775 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:34:59 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:34:59.564+0000 7ff987a3a700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:34:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:59 smithi083 ceph-mon[31864]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:59 smithi083 ceph-mon[31864]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-11T19:34:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:59 smithi083 ceph-mon[31864]: from='osd.8 [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:34:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:59 smithi083 ceph-mon[31864]: osdmap e57: 9 total, 8 up, 9 in 2024-03-11T19:34:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:59 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:34:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:34:59 smithi083 ceph-mon[31864]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:34:59.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:59 smithi137 ceph-mon[36220]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:34:59.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:59 smithi137 ceph-mon[36220]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-11T19:34:59.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:59 smithi137 ceph-mon[36220]: from='osd.8 [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:34:59.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:59 smithi137 ceph-mon[36220]: osdmap e57: 9 total, 8 up, 9 in 2024-03-11T19:34:59.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:59 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:34:59.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:34:59 smithi137 ceph-mon[36220]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:35:00.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:00 smithi083 ceph-mon[31864]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:35:00.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:00 smithi083 ceph-mon[31864]: osdmap e58: 9 total, 8 up, 9 in 2024-03-11T19:35:00.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:00 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:35:00.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:00 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:35:00.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:00 smithi083 ceph-mon[31864]: osd.8 [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508] boot 2024-03-11T19:35:00.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:00 smithi083 ceph-mon[31864]: osdmap e59: 9 total, 9 up, 9 in 2024-03-11T19:35:00.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:00 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:35:00.825 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:00 smithi204 ceph-mon[36739]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:35:00.825 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:00 smithi204 ceph-mon[36739]: osdmap e58: 9 total, 8 up, 9 in 2024-03-11T19:35:00.826 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:00 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:35:00.826 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:00 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:35:00.826 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:00 smithi204 ceph-mon[36739]: osd.8 [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508] boot 2024-03-11T19:35:00.826 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:00 smithi204 ceph-mon[36739]: osdmap e59: 9 total, 9 up, 9 in 2024-03-11T19:35:00.826 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:00 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:35:00.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:00 smithi137 ceph-mon[36220]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:35:00.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:00 smithi137 ceph-mon[36220]: osdmap e58: 9 total, 8 up, 9 in 2024-03-11T19:35:00.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:00 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:35:00.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:00 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:35:00.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:00 smithi137 ceph-mon[36220]: osd.8 [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508] boot 2024-03-11T19:35:00.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:00 smithi137 ceph-mon[36220]: osdmap e59: 9 total, 9 up, 9 in 2024-03-11T19:35:00.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:00 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-11T19:35:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-11T19:35:01.020 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-11T19:35:01.020 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-03-11T19:35:01.021 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-03-11T19:35:01.021 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114064 s, 919 MB/s 2024-03-11T19:35:01.021 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-03-11T19:35:01.684 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:35:01.685 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:35:01.685 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:35:01.685 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: osdmap e60: 9 total, 9 up, 9 in 2024-03-11T19:35:01.685 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.685 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.685 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:01.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:01.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:01.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:01.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:01.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-03-11T19:35:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:35:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:35:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:35:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: osdmap e60: 9 total, 9 up, 9 in 2024-03-11T19:35:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:01.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:01.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:35:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:35:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-03-11T19:35:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: osdmap e60: 9 total, 9 up, 9 in 2024-03-11T19:35:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:01.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:01.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:01.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:01.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:01.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:01.960 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:35:02.749 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:02 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi204 2024-03-11T19:35:02.749 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:02 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi204 to 5693M 2024-03-11T19:35:02.749 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:02 smithi083 ceph-mon[31864]: osdmap e61: 9 total, 9 up, 9 in 2024-03-11T19:35:02.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:02 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi204 2024-03-11T19:35:02.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:02 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi204 to 5693M 2024-03-11T19:35:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:02 smithi137 ceph-mon[36220]: osdmap e61: 9 total, 9 up, 9 in 2024-03-11T19:35:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:02 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi204 2024-03-11T19:35:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:02 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi204 to 5693M 2024-03-11T19:35:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:02 smithi204 ceph-mon[36739]: osdmap e61: 9 total, 9 up, 9 in 2024-03-11T19:35:03.720 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:03 smithi083 ceph-mon[31864]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:03.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.721+0000 7f389aa00700 1 -- 172.21.15.83:0/825803354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38941039f0 msgr2=0x7f389410a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:03.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.721+0000 7f389aa00700 1 --2- 172.21.15.83:0/825803354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38941039f0 0x7f389410a2a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f388c00b0a0 tx=0x7f388c00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:35:03.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.721+0000 7f389aa00700 1 -- 172.21.15.83:0/825803354 shutdown_connections 2024-03-11T19:35:03.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.721+0000 7f389aa00700 1 --2- 172.21.15.83:0/825803354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38941039f0 0x7f389410a2a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:03.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.721+0000 7f389aa00700 1 --2- 172.21.15.83:0/825803354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38940fffd0 0x7f38941003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:03.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.721+0000 7f389aa00700 1 --2- 172.21.15.83:0/825803354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3894103050 0x7f38941034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:03.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.721+0000 7f389aa00700 1 -- 172.21.15.83:0/825803354 >> 172.21.15.83:0/825803354 conn(0x7f38940fb910 msgr2=0x7f38940fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:35:03.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.721+0000 7f389aa00700 1 -- 172.21.15.83:0/825803354 shutdown_connections 2024-03-11T19:35:03.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.721+0000 7f389aa00700 1 -- 172.21.15.83:0/825803354 wait complete. 2024-03-11T19:35:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f389aa00700 1 Processor -- start 2024-03-11T19:35:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f389aa00700 1 -- start start 2024-03-11T19:35:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f389aa00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38940fffd0 0x7f38941029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f389aa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3894103050 0x7f3894101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f389aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38941039f0 0x7f3894101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f389aa00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3894101cc0 con 0x7f38941039f0 2024-03-11T19:35:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f389aa00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3894101e30 con 0x7f3894103050 2024-03-11T19:35:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f389aa00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38941a1e00 con 0x7f38940fffd0 2024-03-11T19:35:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f3898f9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38941039f0 0x7f3894101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f3898f9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38941039f0 0x7f3894101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53992/0 (socket says 172.21.15.83:53992) 2024-03-11T19:35:03.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f3898f9d700 1 -- 172.21.15.83:0/1841893369 learned_addr learned my addr 172.21.15.83:0/1841893369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f3893fff700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38940fffd0 0x7f38941029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f38937fe700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3894103050 0x7f3894101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.722+0000 7f3898f9d700 1 -- 172.21.15.83:0/1841893369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38940fffd0 msgr2=0x7f38941029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.723+0000 7f3898f9d700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38940fffd0 0x7f38941029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.723+0000 7f3898f9d700 1 -- 172.21.15.83:0/1841893369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3894103050 msgr2=0x7f3894101000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.723+0000 7f3898f9d700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3894103050 0x7f3894101000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.723+0000 7f3898f9d700 1 -- 172.21.15.83:0/1841893369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c009d00 con 0x7f38941039f0 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.723+0000 7f3898f9d700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38941039f0 0x7f3894101540 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f388c00b070 tx=0x7f388c0093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.723+0000 7f38917fa700 1 -- 172.21.15.83:0/1841893369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c00e040 con 0x7f38941039f0 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.723+0000 7f38917fa700 1 -- 172.21.15.83:0/1841893369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f388c01d960 con 0x7f38941039f0 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.723+0000 7f389aa00700 1 -- 172.21.15.83:0/1841893369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38941a2080 con 0x7f38941039f0 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.723+0000 7f38917fa700 1 -- 172.21.15.83:0/1841893369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c01ba80 con 0x7f38941039f0 2024-03-11T19:35:03.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.723+0000 7f389aa00700 1 -- 172.21.15.83:0/1841893369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38941a2520 con 0x7f38941039f0 2024-03-11T19:35:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.724+0000 7f389aa00700 1 -- 172.21.15.83:0/1841893369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3894109140 con 0x7f38941039f0 2024-03-11T19:35:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.725+0000 7f38917fa700 1 -- 172.21.15.83:0/1841893369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f388c01dad0 con 0x7f38941039f0 2024-03-11T19:35:03.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.725+0000 7f38917fa700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38800a1ac0 0x7f38800a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.726+0000 7f3893fff700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38800a1ac0 0x7f38800a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.726+0000 7f38917fa700 1 -- 172.21.15.83:0/1841893369 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f388c01de40 con 0x7f38941039f0 2024-03-11T19:35:03.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.726+0000 7f3893fff700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38800a1ac0 0x7f38800a3f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f389406a090 tx=0x7f3884009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:35:03.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.729+0000 7f38917fa700 1 -- 172.21.15.83:0/1841893369 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f388c098a30 con 0x7f38941039f0 2024-03-11T19:35:03.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:03 smithi137 ceph-mon[36220]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:03 smithi204 ceph-mon[36739]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:03.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:03.930+0000 7f389aa00700 1 -- 172.21.15.83:0/1841893369 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f389402d7d0 con 0x7f38800a1ac0 2024-03-11T19:35:04.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:04 smithi083 ceph-mon[31864]: osdmap e62: 9 total, 9 up, 9 in 2024-03-11T19:35:04.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:35:04.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:35:04.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:04.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:04 smithi137 ceph-mon[36220]: osdmap e62: 9 total, 9 up, 9 in 2024-03-11T19:35:04.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:35:04.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:35:04.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:04 smithi204 ceph-mon[36739]: osdmap e62: 9 total, 9 up, 9 in 2024-03-11T19:35:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:35:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:35:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:05.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:05 smithi083 ceph-mon[31864]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:35:05.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:05 smithi083 ceph-mon[31864]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:05.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:05 smithi083 ceph-mon[31864]: osdmap e63: 9 total, 9 up, 9 in 2024-03-11T19:35:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:05 smithi137 ceph-mon[36220]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:35:05.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:05 smithi137 ceph-mon[36220]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:05.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:05 smithi137 ceph-mon[36220]: osdmap e63: 9 total, 9 up, 9 in 2024-03-11T19:35:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:05 smithi204 ceph-mon[36739]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:35:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:05 smithi204 ceph-mon[36739]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:05 smithi204 ceph-mon[36739]: osdmap e63: 9 total, 9 up, 9 in 2024-03-11T19:35:07.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:07 smithi137 ceph-mon[36220]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-11T19:35:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:07 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/539733329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78fabab9-c022-4e3a-a912-aab73696f4b6"}]: dispatch 2024-03-11T19:35:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:07 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/539733329' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78fabab9-c022-4e3a-a912-aab73696f4b6"}]': finished 2024-03-11T19:35:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:07 smithi137 ceph-mon[36220]: osdmap e64: 10 total, 9 up, 10 in 2024-03-11T19:35:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:07 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4033342001' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:35:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:07 smithi204 ceph-mon[36739]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-11T19:35:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:07 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/539733329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78fabab9-c022-4e3a-a912-aab73696f4b6"}]: dispatch 2024-03-11T19:35:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:07 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/539733329' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78fabab9-c022-4e3a-a912-aab73696f4b6"}]': finished 2024-03-11T19:35:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:07 smithi204 ceph-mon[36739]: osdmap e64: 10 total, 9 up, 10 in 2024-03-11T19:35:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:07 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4033342001' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:35:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:07 smithi083 ceph-mon[31864]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-03-11T19:35:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:07 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/539733329' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78fabab9-c022-4e3a-a912-aab73696f4b6"}]: dispatch 2024-03-11T19:35:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:07 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/539733329' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78fabab9-c022-4e3a-a912-aab73696f4b6"}]': finished 2024-03-11T19:35:08.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:07 smithi083 ceph-mon[31864]: osdmap e64: 10 total, 9 up, 10 in 2024-03-11T19:35:08.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:08.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:07 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4033342001' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:35:08.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:08 smithi137 ceph-mon[36220]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:35:08.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:08 smithi204 ceph-mon[36739]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:35:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:08 smithi083 ceph-mon[31864]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:35:11.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:11 smithi083 ceph-mon[31864]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-11T19:35:11.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:11 smithi137 ceph-mon[36220]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-11T19:35:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:11 smithi204 ceph-mon[36739]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-11T19:35:13.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:13 smithi083 ceph-mon[31864]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:35:13.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:13 smithi137 ceph-mon[36220]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:35:13.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:13 smithi204 ceph-mon[36739]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-11T19:35:14.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-11T19:35:14.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:14.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-11T19:35:14.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:14.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-11T19:35:14.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:15.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:15 smithi083 ceph-mon[31864]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:35:15.625 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:15 smithi083 ceph-mon[31864]: Deploying daemon osd.9 on smithi083 2024-03-11T19:35:15.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:15 smithi137 ceph-mon[36220]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:35:15.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:15 smithi137 ceph-mon[36220]: Deploying daemon osd.9 on smithi083 2024-03-11T19:35:15.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:15 smithi204 ceph-mon[36739]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-11T19:35:15.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:15 smithi204 ceph-mon[36739]: Deploying daemon osd.9 on smithi083 2024-03-11T19:35:17.634 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:17 smithi083 ceph-mon[31864]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:17.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:17 smithi137 ceph-mon[36220]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:17.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:17 smithi204 ceph-mon[36739]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:19 smithi083 ceph-mon[31864]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:19.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:19 smithi137 ceph-mon[36220]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:19 smithi204 ceph-mon[36739]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:21.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:21.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:21.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:21.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:21 smithi083 ceph-mon[31864]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:21.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:21.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:21.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:21.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:21 smithi137 ceph-mon[36220]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:21.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:21.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:21.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:21.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:21 smithi204 ceph-mon[36739]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:23.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:23 smithi083 ceph-mon[31864]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:23.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:23.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:23.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:23.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:23.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:23.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:23 smithi137 ceph-mon[36220]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:23.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:23.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:23.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:23.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:23.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:23 smithi204 ceph-mon[36739]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:23.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:24.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.091+0000 7f38917fa700 1 -- 172.21.15.83:0/1841893369 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f389402d7d0 con 0x7f38800a1ac0 2024-03-11T19:35:24.095 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 9 on host 'smithi083' 2024-03-11T19:35:24.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 -- 172.21.15.83:0/1841893369 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38800a1ac0 msgr2=0x7f38800a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:24.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38800a1ac0 0x7f38800a3f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f389406a090 tx=0x7f3884009190 comp rx=0 tx=0).stop 2024-03-11T19:35:24.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 -- 172.21.15.83:0/1841893369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38941039f0 msgr2=0x7f3894101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:24.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38941039f0 0x7f3894101540 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f388c00b070 tx=0x7f388c0093f0 comp rx=0 tx=0).stop 2024-03-11T19:35:24.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 -- 172.21.15.83:0/1841893369 shutdown_connections 2024-03-11T19:35:24.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38800a1ac0 0x7f38800a3f80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:24.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38940fffd0 0x7f38941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:24.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38941039f0 0x7f3894101540 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:24.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 --2- 172.21.15.83:0/1841893369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3894103050 0x7f3894101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:24.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 -- 172.21.15.83:0/1841893369 >> 172.21.15.83:0/1841893369 conn(0x7f38940fb910 msgr2=0x7f3894106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:35:24.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 -- 172.21.15.83:0/1841893369 shutdown_connections 2024-03-11T19:35:24.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:35:24.094+0000 7f389aa00700 1 -- 172.21.15.83:0/1841893369 wait complete. 2024-03-11T19:35:24.695 DEBUG:teuthology.orchestra.run.smithi083:osd.9> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.9.service 2024-03-11T19:35:24.698 INFO:tasks.cephadm:Deploying osd.10 on smithi137 with /dev/vg_nvme/lv_1... 2024-03-11T19:35:24.698 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-11T19:35:24.942 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:35:25.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:25 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:25.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:25 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:25.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:25 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:25.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:25 smithi083 ceph-mon[31864]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:35:25.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:25 smithi083 ceph-mon[31864]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:25.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:25 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:25.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:25 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:25.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:25 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:25.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:25 smithi137 ceph-mon[36220]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:35:25.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:25 smithi137 ceph-mon[36220]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:25.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:25 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:25.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:25 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:25.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:25 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:25.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:25 smithi204 ceph-mon[36739]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-11T19:35:25.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:25 smithi204 ceph-mon[36739]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:26.139 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:26 smithi083 ceph-mon[31864]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-11T19:35:26.139 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:26 smithi083 ceph-mon[31864]: osdmap e65: 10 total, 9 up, 10 in 2024-03-11T19:35:26.139 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:26 smithi083 ceph-mon[31864]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:35:26.140 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:26 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:26.140 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:35:26 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:35:26.094+0000 7faf8bc4b700 -1 osd.9 0 waiting for initial osdmap 2024-03-11T19:35:26.140 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:35:26 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:35:26.109+0000 7faf8823f700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:35:26.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:26 smithi137 ceph-mon[36220]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-11T19:35:26.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:26 smithi137 ceph-mon[36220]: osdmap e65: 10 total, 9 up, 10 in 2024-03-11T19:35:26.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:26 smithi137 ceph-mon[36220]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:35:26.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:26 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:26.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:26 smithi204 ceph-mon[36739]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-11T19:35:26.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:26 smithi204 ceph-mon[36739]: osdmap e65: 10 total, 9 up, 10 in 2024-03-11T19:35:26.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:26 smithi204 ceph-mon[36739]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-03-11T19:35:26.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:26 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113015 s, 928 MB/s 2024-03-11T19:35:26.833 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-11T19:35:27.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:27 smithi083 ceph-mon[31864]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:35:27.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:27 smithi083 ceph-mon[31864]: osdmap e66: 10 total, 9 up, 10 in 2024-03-11T19:35:27.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:27.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:27.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:27 smithi083 ceph-mon[31864]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:27.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:27 smithi137 ceph-mon[36220]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:35:27.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:27 smithi137 ceph-mon[36220]: osdmap e66: 10 total, 9 up, 10 in 2024-03-11T19:35:27.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:27.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:27.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:27 smithi137 ceph-mon[36220]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:27 smithi204 ceph-mon[36739]: from='osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-03-11T19:35:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:27 smithi204 ceph-mon[36739]: osdmap e66: 10 total, 9 up, 10 in 2024-03-11T19:35:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:27.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:27 smithi204 ceph-mon[36739]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-03-11T19:35:27.436 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2024-03-11T19:35:27.683 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:35:28.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:35:28.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190] boot 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: osdmap e67: 10 total, 10 up, 10 in 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:28.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:28.391 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:28.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:35:28.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:35:28.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:28.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190] boot 2024-03-11T19:35:28.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: osdmap e67: 10 total, 10 up, 10 in 2024-03-11T19:35:28.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:28.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:28.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:28.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:28.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:28.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:28.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:28.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:35:28.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:35:28.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:28.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: osd.9 [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190] boot 2024-03-11T19:35:28.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: osdmap e67: 10 total, 10 up, 10 in 2024-03-11T19:35:28.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-11T19:35:28.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:28.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:28.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:28.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:28.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:28.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:28.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:29.316 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:29 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi083 2024-03-11T19:35:29.317 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:29 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi083 to 4272M 2024-03-11T19:35:29.317 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:29 smithi137 ceph-mon[36220]: osdmap e68: 10 total, 10 up, 10 in 2024-03-11T19:35:29.317 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:29 smithi137 ceph-mon[36220]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:29.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.316+0000 7f14a4d6c700 1 -- 172.21.15.137:0/3975405681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0101d90 msgr2=0x7f14a01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:29.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.316+0000 7f14a4d6c700 1 --2- 172.21.15.137:0/3975405681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0101d90 0x7f14a01021f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f14880099c0 tx=0x7f1488009cd0 comp rx=0 tx=0).stop 2024-03-11T19:35:29.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.316+0000 7f14a4d6c700 1 -- 172.21.15.137:0/3975405681 shutdown_connections 2024-03-11T19:35:29.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.316+0000 7f14a4d6c700 1 --2- 172.21.15.137:0/3975405681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a0100b90 0x7f14a0100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:29.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.316+0000 7f14a4d6c700 1 --2- 172.21.15.137:0/3975405681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14a0102730 0x7f14a0108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:29.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.316+0000 7f14a4d6c700 1 --2- 172.21.15.137:0/3975405681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0101d90 0x7f14a01021f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:29.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.316+0000 7f14a4d6c700 1 -- 172.21.15.137:0/3975405681 >> 172.21.15.137:0/3975405681 conn(0x7f14a00fc130 msgr2=0x7f14a00fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:35:29.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.316+0000 7f14a4d6c700 1 -- 172.21.15.137:0/3975405681 shutdown_connections 2024-03-11T19:35:29.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.316+0000 7f14a4d6c700 1 -- 172.21.15.137:0/3975405681 wait complete. 2024-03-11T19:35:29.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f14a4d6c700 1 Processor -- start 2024-03-11T19:35:29.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f14a4d6c700 1 -- start start 2024-03-11T19:35:29.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f14a4d6c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0100b90 0x7f14a0196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:29.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f14a4d6c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a0101d90 0x7f14a0197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:29.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f14a4d6c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14a0102730 0x7f14a019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:29.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f14a4d6c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a019e6a0 con 0x7f14a0102730 2024-03-11T19:35:29.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f14a4d6c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a019e7e0 con 0x7f14a0100b90 2024-03-11T19:35:29.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f14a4d6c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a006a230 con 0x7f14a0101d90 2024-03-11T19:35:29.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f149e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0100b90 0x7f14a0196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:29.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f149e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0100b90 0x7f14a0196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44938/0 (socket says 172.21.15.137:44938) 2024-03-11T19:35:29.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f149e59c700 1 -- 172.21.15.137:0/2965044889 learned_addr learned my addr 172.21.15.137:0/2965044889 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-11T19:35:29.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f149dd9b700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a0101d90 0x7f14a0197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:29.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f149ed9d700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14a0102730 0x7f14a019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:29.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f149e59c700 1 -- 172.21.15.137:0/2965044889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a0101d90 msgr2=0x7f14a0197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:29.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f149e59c700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a0101d90 0x7f14a0197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:29.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f149e59c700 1 -- 172.21.15.137:0/2965044889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14a0102730 msgr2=0x7f14a019e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:29.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f149e59c700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14a0102730 0x7f14a019e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:29.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.317+0000 7f149e59c700 1 -- 172.21.15.137:0/2965044889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14880095f0 con 0x7f14a0100b90 2024-03-11T19:35:29.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.318+0000 7f149e59c700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0100b90 0x7f14a0196b10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f149800b9c0 tx=0x7f149800bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:35:29.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.318+0000 7f14977fe700 1 -- 172.21.15.137:0/2965044889 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149800ca30 con 0x7f14a0100b90 2024-03-11T19:35:29.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.318+0000 7f14a4d6c700 1 -- 172.21.15.137:0/2965044889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a006a510 con 0x7f14a0100b90 2024-03-11T19:35:29.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.318+0000 7f14977fe700 1 -- 172.21.15.137:0/2965044889 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f149800cb90 con 0x7f14a0100b90 2024-03-11T19:35:29.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.318+0000 7f14977fe700 1 -- 172.21.15.137:0/2965044889 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14980125a0 con 0x7f14a0100b90 2024-03-11T19:35:29.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.318+0000 7f14a4d6c700 1 -- 172.21.15.137:0/2965044889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a006aa60 con 0x7f14a0100b90 2024-03-11T19:35:29.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.319+0000 7f14a4d6c700 1 -- 172.21.15.137:0/2965044889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a0107ef0 con 0x7f14a0100b90 2024-03-11T19:35:29.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.320+0000 7f14977fe700 1 -- 172.21.15.137:0/2965044889 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f149800cd00 con 0x7f14a0100b90 2024-03-11T19:35:29.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.320+0000 7f14977fe700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f148c0a1ac0 0x7f148c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:29.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.321+0000 7f14977fe700 1 -- 172.21.15.137:0/2965044889 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f1498006e40 con 0x7f14a0100b90 2024-03-11T19:35:29.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.321+0000 7f149dd9b700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f148c0a1ac0 0x7f148c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:29.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.321+0000 7f149dd9b700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f148c0a1ac0 0x7f148c0a3f80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1488000c00 tx=0x7f1488005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:35:29.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.324+0000 7f14977fe700 1 -- 172.21.15.137:0/2965044889 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1498096200 con 0x7f14a0100b90 2024-03-11T19:35:29.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:29 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi083 2024-03-11T19:35:29.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:29 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi083 to 4272M 2024-03-11T19:35:29.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:29 smithi204 ceph-mon[36739]: osdmap e68: 10 total, 10 up, 10 in 2024-03-11T19:35:29.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:29 smithi204 ceph-mon[36739]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:29.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:29.523+0000 7f14a4d6c700 1 -- 172.21.15.137:0/2965044889 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f14a006adf0 con 0x7f148c0a1ac0 2024-03-11T19:35:29.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:29 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi083 2024-03-11T19:35:29.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:29 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi083 to 4272M 2024-03-11T19:35:29.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:29 smithi083 ceph-mon[31864]: osdmap e68: 10 total, 10 up, 10 in 2024-03-11T19:35:29.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:29 smithi083 ceph-mon[31864]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:30.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:30 smithi137 ceph-mon[36220]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:35:30.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:35:30.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:35:30.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:30.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:30 smithi204 ceph-mon[36739]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:35:30.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:35:30.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:35:30.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:30.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:30 smithi083 ceph-mon[31864]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:35:30.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:35:30.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:35:30.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:31.344 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:31 smithi137 ceph-mon[36220]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:31.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:31 smithi083 ceph-mon[31864]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:31 smithi204 ceph-mon[36739]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:32.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:32 smithi083 ceph-mon[31864]: from='client.? 172.21.15.137:0/2447584056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f76fae-ccd2-4b2b-8885-fb008d99ad9b"}]: dispatch 2024-03-11T19:35:32.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:32 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f76fae-ccd2-4b2b-8885-fb008d99ad9b"}]: dispatch 2024-03-11T19:35:32.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:32 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7f76fae-ccd2-4b2b-8885-fb008d99ad9b"}]': finished 2024-03-11T19:35:32.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:32 smithi083 ceph-mon[31864]: osdmap e69: 11 total, 10 up, 11 in 2024-03-11T19:35:32.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:32.569 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:32 smithi137 ceph-mon[36220]: from='client.? 172.21.15.137:0/2447584056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f76fae-ccd2-4b2b-8885-fb008d99ad9b"}]: dispatch 2024-03-11T19:35:32.569 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:32 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f76fae-ccd2-4b2b-8885-fb008d99ad9b"}]: dispatch 2024-03-11T19:35:32.569 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:32 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7f76fae-ccd2-4b2b-8885-fb008d99ad9b"}]': finished 2024-03-11T19:35:32.569 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:32 smithi137 ceph-mon[36220]: osdmap e69: 11 total, 10 up, 11 in 2024-03-11T19:35:32.569 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:32.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:32 smithi204 ceph-mon[36739]: from='client.? 172.21.15.137:0/2447584056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f76fae-ccd2-4b2b-8885-fb008d99ad9b"}]: dispatch 2024-03-11T19:35:32.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:32 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f76fae-ccd2-4b2b-8885-fb008d99ad9b"}]: dispatch 2024-03-11T19:35:32.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:32 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7f76fae-ccd2-4b2b-8885-fb008d99ad9b"}]': finished 2024-03-11T19:35:32.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:32 smithi204 ceph-mon[36739]: osdmap e69: 11 total, 10 up, 11 in 2024-03-11T19:35:32.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:33 smithi083 ceph-mon[31864]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:33 smithi083 ceph-mon[31864]: from='client.? 172.21.15.137:0/2864476460' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:35:33.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:33 smithi137 ceph-mon[36220]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:33.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:33 smithi137 ceph-mon[36220]: from='client.? 172.21.15.137:0/2864476460' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:35:33.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:33 smithi204 ceph-mon[36739]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:33.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:33 smithi204 ceph-mon[36739]: from='client.? 172.21.15.137:0/2864476460' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:35:35.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:35 smithi083 ceph-mon[31864]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:35.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:35 smithi137 ceph-mon[36220]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:35.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:35 smithi204 ceph-mon[36739]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:37.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:37 smithi083 ceph-mon[31864]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:37.868 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:37 smithi137 ceph-mon[36220]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:37 smithi204 ceph-mon[36739]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:39.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:39 smithi083 ceph-mon[31864]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:39.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:39 smithi137 ceph-mon[36220]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:39 smithi204 ceph-mon[36739]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:40.766 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:40 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-11T19:35:40.766 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:40 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:40.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:40 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-11T19:35:40.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:40 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:40.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:40 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-11T19:35:40.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:40 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:41.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:41 smithi137 ceph-mon[36220]: Deploying daemon osd.10 on smithi137 2024-03-11T19:35:41.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:41 smithi137 ceph-mon[36220]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:41.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:41 smithi083 ceph-mon[31864]: Deploying daemon osd.10 on smithi137 2024-03-11T19:35:41.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:41 smithi083 ceph-mon[31864]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:41 smithi204 ceph-mon[36739]: Deploying daemon osd.10 on smithi137 2024-03-11T19:35:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:41 smithi204 ceph-mon[36739]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:43.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:43 smithi083 ceph-mon[31864]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:43.845 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:43 smithi137 ceph-mon[36220]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:43.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:43 smithi204 ceph-mon[36739]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:45.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:45 smithi137 ceph-mon[36220]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:45.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:45.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:45.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:45 smithi083 ceph-mon[31864]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:45.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:45 smithi204 ceph-mon[36739]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:47.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:47 smithi137 ceph-mon[36220]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:47.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:47 smithi204 ceph-mon[36739]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:48.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:47 smithi083 ceph-mon[31864]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:48.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.788+0000 7f14977fe700 1 -- 172.21.15.137:0/2965044889 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f14a006adf0 con 0x7f148c0a1ac0 2024-03-11T19:35:48.793 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 10 on host 'smithi137' 2024-03-11T19:35:48.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 -- 172.21.15.137:0/2965044889 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f148c0a1ac0 msgr2=0x7f148c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:48.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f148c0a1ac0 0x7f148c0a3f80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1488000c00 tx=0x7f1488005fb0 comp rx=0 tx=0).stop 2024-03-11T19:35:48.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 -- 172.21.15.137:0/2965044889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0100b90 msgr2=0x7f14a0196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:48.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0100b90 0x7f14a0196b10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f149800b9c0 tx=0x7f149800bd80 comp rx=0 tx=0).stop 2024-03-11T19:35:48.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 -- 172.21.15.137:0/2965044889 shutdown_connections 2024-03-11T19:35:48.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f148c0a1ac0 0x7f148c0a3f80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:48.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14a0102730 0x7f14a019e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:48.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0100b90 0x7f14a0196b10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:48.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 --2- 172.21.15.137:0/2965044889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a0101d90 0x7f14a0197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:48.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 -- 172.21.15.137:0/2965044889 >> 172.21.15.137:0/2965044889 conn(0x7f14a00fc130 msgr2=0x7f14a0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:35:48.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 -- 172.21.15.137:0/2965044889 shutdown_connections 2024-03-11T19:35:48.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-11T19:35:48.791+0000 7f14a4d6c700 1 -- 172.21.15.137:0/2965044889 wait complete. 2024-03-11T19:35:49.423 DEBUG:teuthology.orchestra.run.smithi137:osd.10> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.10.service 2024-03-11T19:35:49.426 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-03-11T19:35:49.426 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-11T19:35:49.678 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:35:49.708 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:49 smithi204 ceph-mon[36739]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:49.708 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:49.708 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:49.708 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:49 smithi204 ceph-mon[36739]: from='osd.10 [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:35:49.708 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:49 smithi204 ceph-mon[36739]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:35:49.708 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:49.709 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:49.870 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:49 smithi083 ceph-mon[31864]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:49.870 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:49.870 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:49.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:49 smithi083 ceph-mon[31864]: from='osd.10 [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:35:49.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:49 smithi083 ceph-mon[31864]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:35:49.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:49.871 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:49.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:49 smithi137 ceph-mon[36220]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:49 smithi137 ceph-mon[36220]: from='osd.10 [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:35:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:49 smithi137 ceph-mon[36220]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-11T19:35:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:50 smithi083 ceph-mon[31864]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-11T19:35:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:50 smithi083 ceph-mon[31864]: osdmap e70: 11 total, 10 up, 11 in 2024-03-11T19:35:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:50 smithi083 ceph-mon[31864]: from='osd.10 [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:35:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:50 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:50 smithi083 ceph-mon[31864]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:35:51.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:50 smithi083 ceph-mon[31864]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:51.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:50 smithi137 ceph-mon[36220]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-11T19:35:51.096 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:50 smithi137 ceph-mon[36220]: osdmap e70: 11 total, 10 up, 11 in 2024-03-11T19:35:51.097 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:50 smithi137 ceph-mon[36220]: from='osd.10 [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:35:51.097 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:50 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:51.097 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:50 smithi137 ceph-mon[36220]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:35:51.097 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:50 smithi137 ceph-mon[36220]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:50 smithi204 ceph-mon[36739]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-11T19:35:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:50 smithi204 ceph-mon[36739]: osdmap e70: 11 total, 10 up, 11 in 2024-03-11T19:35:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:50 smithi204 ceph-mon[36739]: from='osd.10 [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:35:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:50 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:50 smithi204 ceph-mon[36739]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-11T19:35:51.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:50 smithi204 ceph-mon[36739]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:51.685 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:35:51 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:35:51.614+0000 7f36e7375700 -1 osd.10 0 waiting for initial osdmap 2024-03-11T19:35:51.686 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:35:51 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:35:51.639+0000 7f36df961700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:35:51.824 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-11T19:35:51.824 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-11T19:35:51.824 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-11T19:35:51.824 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-11T19:35:51.824 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112528 s, 932 MB/s 2024-03-11T19:35:51.824 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-11T19:35:52.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:51 smithi083 ceph-mon[31864]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:35:52.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:51 smithi083 ceph-mon[31864]: osdmap e71: 11 total, 10 up, 11 in 2024-03-11T19:35:52.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:51 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:52.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:51 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:52.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:51 smithi083 ceph-mon[31864]: from='osd.10 ' entity='osd.10' 2024-03-11T19:35:52.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:51 smithi137 ceph-mon[36220]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:35:52.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:51 smithi137 ceph-mon[36220]: osdmap e71: 11 total, 10 up, 11 in 2024-03-11T19:35:52.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:51 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:52.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:51 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:52.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:51 smithi137 ceph-mon[36220]: from='osd.10 ' entity='osd.10' 2024-03-11T19:35:52.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:51 smithi204 ceph-mon[36739]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-11T19:35:52.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:51 smithi204 ceph-mon[36739]: osdmap e71: 11 total, 10 up, 11 in 2024-03-11T19:35:52.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:51 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:52.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:51 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:52.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:51 smithi204 ceph-mon[36739]: from='osd.10 ' entity='osd.10' 2024-03-11T19:35:52.528 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-03-11T19:35:52.767 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:35:52.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:52 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:35:52.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:52 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:35:52.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:52 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:52.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:52 smithi137 ceph-mon[36220]: osd.10 [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423] boot 2024-03-11T19:35:52.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:52 smithi137 ceph-mon[36220]: osdmap e72: 11 total, 11 up, 11 in 2024-03-11T19:35:52.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:52 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:52.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:52 smithi137 ceph-mon[36220]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:52 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:35:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:52 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:35:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:52 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:52 smithi083 ceph-mon[31864]: osd.10 [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423] boot 2024-03-11T19:35:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:52 smithi083 ceph-mon[31864]: osdmap e72: 11 total, 11 up, 11 in 2024-03-11T19:35:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:52 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:52 smithi083 ceph-mon[31864]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:53.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:52 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:35:53.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:52 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:35:53.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:52 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:53.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:52 smithi204 ceph-mon[36739]: osd.10 [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423] boot 2024-03-11T19:35:53.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:52 smithi204 ceph-mon[36739]: osdmap e72: 11 total, 11 up, 11 in 2024-03-11T19:35:53.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:52 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-11T19:35:53.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:52 smithi204 ceph-mon[36739]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-11T19:35:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: osdmap e73: 11 total, 11 up, 11 in 2024-03-11T19:35:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi137 2024-03-11T19:35:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi137 to 4270M 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:53.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:54.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: osdmap e73: 11 total, 11 up, 11 in 2024-03-11T19:35:54.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi137 2024-03-11T19:35:54.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:54.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:54.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:54.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:54.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:54.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:54.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi137 to 4270M 2024-03-11T19:35:54.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:54.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:54.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:54.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:54.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:54.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:54.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:54.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:54.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: osdmap e73: 11 total, 11 up, 11 in 2024-03-11T19:35:54.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi137 2024-03-11T19:35:54.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:54.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:54.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:54.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:54.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:54.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:35:54.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi137 to 4270M 2024-03-11T19:35:54.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:54.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:54.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:54.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:54.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:35:54.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:54.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:35:54.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:35:54.432 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.426+0000 7eff5ba4b700 1 -- 172.21.15.204:0/2725923622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff541027b0 msgr2=0x7eff54109060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.426+0000 7eff5ba4b700 1 --2- 172.21.15.204:0/2725923622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff541027b0 0x7eff54109060 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b0a0 tx=0x7eff4c00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 -- 172.21.15.204:0/2725923622 shutdown_connections 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 --2- 172.21.15.204:0/2725923622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff541027b0 0x7eff54109060 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 --2- 172.21.15.204:0/2725923622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff54100c10 0x7eff54101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 --2- 172.21.15.204:0/2725923622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff54101e10 0x7eff54102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 -- 172.21.15.204:0/2725923622 >> 172.21.15.204:0/2725923622 conn(0x7eff540fc1d0 msgr2=0x7eff540fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 -- 172.21.15.204:0/2725923622 shutdown_connections 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 -- 172.21.15.204:0/2725923622 wait complete. 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 Processor -- start 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 -- start start 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff54100c10 0x7eff54199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff54101e10 0x7eff5419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:54.433 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff541027b0 0x7eff541a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff541a1a60 con 0x7eff54100c10 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff541a1ba0 con 0x7eff54101e10 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.427+0000 7eff5ba4b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff541a1d30 con 0x7eff541027b0 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff59fe8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff541027b0 0x7eff541a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff59fe8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff541027b0 0x7eff541a1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:40602/0 (socket says 172.21.15.204:40602) 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff59fe8700 1 -- 172.21.15.204:0/629021164 learned_addr learned my addr 172.21.15.204:0/629021164 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff597e7700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff54100c10 0x7eff54199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff58fe6700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff54101e10 0x7eff5419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff59fe8700 1 -- 172.21.15.204:0/629021164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff54101e10 msgr2=0x7eff5419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff59fe8700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff54101e10 0x7eff5419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff59fe8700 1 -- 172.21.15.204:0/629021164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff54100c10 msgr2=0x7eff54199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff59fe8700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff54100c10 0x7eff54199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:35:54.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff59fe8700 1 -- 172.21.15.204:0/629021164 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c009d00 con 0x7eff541027b0 2024-03-11T19:35:54.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff59fe8700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff541027b0 0x7eff541a1490 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0094c0 tx=0x7eff4c0094f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:35:54.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff4a7fc700 1 -- 172.21.15.204:0/629021164 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00e040 con 0x7eff541027b0 2024-03-11T19:35:54.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff4a7fc700 1 -- 172.21.15.204:0/629021164 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff4c004de0 con 0x7eff541027b0 2024-03-11T19:35:54.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff4a7fc700 1 -- 172.21.15.204:0/629021164 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c01b980 con 0x7eff541027b0 2024-03-11T19:35:54.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff5ba4b700 1 -- 172.21.15.204:0/629021164 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff541a1fb0 con 0x7eff541027b0 2024-03-11T19:35:54.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.428+0000 7eff5ba4b700 1 -- 172.21.15.204:0/629021164 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff541a2390 con 0x7eff541027b0 2024-03-11T19:35:54.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.429+0000 7eff5ba4b700 1 -- 172.21.15.204:0/629021164 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff54107f70 con 0x7eff541027b0 2024-03-11T19:35:54.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.430+0000 7eff4a7fc700 1 -- 172.21.15.204:0/629021164 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7eff4c01dc10 con 0x7eff541027b0 2024-03-11T19:35:54.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.431+0000 7eff4a7fc700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff400a1b40 0x7eff400a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:35:54.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.431+0000 7eff4a7fc700 1 -- 172.21.15.204:0/629021164 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7eff4c0cf220 con 0x7eff541027b0 2024-03-11T19:35:54.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.431+0000 7eff597e7700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff400a1b40 0x7eff400a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:35:54.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.432+0000 7eff597e7700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff400a1b40 0x7eff400a4000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7eff54101c70 tx=0x7eff5000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:35:54.439 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.435+0000 7eff4a7fc700 1 -- 172.21.15.204:0/629021164 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff4c098a50 con 0x7eff541027b0 2024-03-11T19:35:54.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:35:54.638+0000 7eff5ba4b700 1 -- 172.21.15.204:0/629021164 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7eff5402d8d0 con 0x7eff400a1b40 2024-03-11T19:35:54.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:54 smithi204 ceph-mon[36739]: osdmap e74: 11 total, 11 up, 11 in 2024-03-11T19:35:54.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:54 smithi204 ceph-mon[36739]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:35:54.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:54 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:35:54.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:54 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:35:54.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:54 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:54 smithi083 ceph-mon[31864]: osdmap e74: 11 total, 11 up, 11 in 2024-03-11T19:35:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:54 smithi083 ceph-mon[31864]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:35:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:54 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:35:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:54 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:35:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:54 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:55.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:54 smithi137 ceph-mon[36220]: osdmap e74: 11 total, 11 up, 11 in 2024-03-11T19:35:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:54 smithi137 ceph-mon[36220]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:35:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:54 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-11T19:35:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:54 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-11T19:35:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:54 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:35:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:55 smithi204 ceph-mon[36739]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:35:55.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:55 smithi204 ceph-mon[36739]: osdmap e75: 11 total, 11 up, 11 in 2024-03-11T19:35:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:55 smithi083 ceph-mon[31864]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:35:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:55 smithi083 ceph-mon[31864]: osdmap e75: 11 total, 11 up, 11 in 2024-03-11T19:35:56.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:55 smithi137 ceph-mon[36220]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:35:56.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:55 smithi137 ceph-mon[36220]: osdmap e75: 11 total, 11 up, 11 in 2024-03-11T19:35:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:56 smithi204 ceph-mon[36739]: osdmap e76: 11 total, 11 up, 11 in 2024-03-11T19:35:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:56 smithi204 ceph-mon[36739]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:35:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:56 smithi083 ceph-mon[31864]: osdmap e76: 11 total, 11 up, 11 in 2024-03-11T19:35:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:56 smithi083 ceph-mon[31864]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:35:57.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:56 smithi137 ceph-mon[36220]: osdmap e76: 11 total, 11 up, 11 in 2024-03-11T19:35:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:56 smithi137 ceph-mon[36220]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-11T19:35:58.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:57 smithi137 ceph-mon[36220]: from='client.? 172.21.15.204:0/572774553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7778999c-58d4-487e-a8ae-eb317e078f58"}]: dispatch 2024-03-11T19:35:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:57 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7778999c-58d4-487e-a8ae-eb317e078f58"}]: dispatch 2024-03-11T19:35:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:57 smithi137 ceph-mon[36220]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7778999c-58d4-487e-a8ae-eb317e078f58"}]': finished 2024-03-11T19:35:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:57 smithi137 ceph-mon[36220]: osdmap e77: 12 total, 11 up, 12 in 2024-03-11T19:35:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:57 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:35:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:57 smithi204 ceph-mon[36739]: from='client.? 172.21.15.204:0/572774553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7778999c-58d4-487e-a8ae-eb317e078f58"}]: dispatch 2024-03-11T19:35:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:57 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7778999c-58d4-487e-a8ae-eb317e078f58"}]: dispatch 2024-03-11T19:35:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:57 smithi204 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7778999c-58d4-487e-a8ae-eb317e078f58"}]': finished 2024-03-11T19:35:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:57 smithi204 ceph-mon[36739]: osdmap e77: 12 total, 11 up, 12 in 2024-03-11T19:35:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:57 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:35:58.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:57 smithi083 ceph-mon[31864]: from='client.? 172.21.15.204:0/572774553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7778999c-58d4-487e-a8ae-eb317e078f58"}]: dispatch 2024-03-11T19:35:58.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:57 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7778999c-58d4-487e-a8ae-eb317e078f58"}]: dispatch 2024-03-11T19:35:58.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:57 smithi083 ceph-mon[31864]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7778999c-58d4-487e-a8ae-eb317e078f58"}]': finished 2024-03-11T19:35:58.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:57 smithi083 ceph-mon[31864]: osdmap e77: 12 total, 11 up, 12 in 2024-03-11T19:35:58.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:57 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:35:59.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:58 smithi137 ceph-mon[36220]: from='client.? 172.21.15.204:0/110668592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:35:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:35:58 smithi137 ceph-mon[36220]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-11T19:35:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:58 smithi204 ceph-mon[36739]: from='client.? 172.21.15.204:0/110668592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:35:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:35:58 smithi204 ceph-mon[36739]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-11T19:35:59.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:58 smithi083 ceph-mon[31864]: from='client.? 172.21.15.204:0/110668592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-11T19:35:59.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:35:58 smithi083 ceph-mon[31864]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-11T19:36:01.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:01 smithi204 ceph-mon[36739]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:36:01.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:01 smithi083 ceph-mon[31864]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:36:01.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:01 smithi137 ceph-mon[36220]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-11T19:36:03.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:03 smithi083 ceph-mon[31864]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-11T19:36:03.837 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:03 smithi204 ceph-mon[36739]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-11T19:36:03.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:03 smithi137 ceph-mon[36220]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-11T19:36:05.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:05 smithi083 ceph-mon[31864]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-11T19:36:05.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:05 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-11T19:36:05.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:05 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:05.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:05 smithi204 ceph-mon[36739]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-11T19:36:05.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:05 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-11T19:36:05.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:05 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:05 smithi137 ceph-mon[36220]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-11T19:36:05.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:05 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-11T19:36:05.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:05 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:06.776 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:06 smithi204 ceph-mon[36739]: Deploying daemon osd.11 on smithi204 2024-03-11T19:36:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:06 smithi083 ceph-mon[31864]: Deploying daemon osd.11 on smithi204 2024-03-11T19:36:06.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:06 smithi137 ceph-mon[36220]: Deploying daemon osd.11 on smithi204 2024-03-11T19:36:07.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:07 smithi083 ceph-mon[31864]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:07.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:07 smithi204 ceph-mon[36739]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:07 smithi137 ceph-mon[36220]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:09 smithi083 ceph-mon[31864]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:09.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:09 smithi137 ceph-mon[36220]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:09 smithi204 ceph-mon[36739]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:11 smithi204 ceph-mon[36739]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:11 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:11 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:11 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:36:11.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:11 smithi137 ceph-mon[36220]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:11.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:11 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:11.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:11 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:11.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:11 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:36:12.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:11 smithi083 ceph-mon[31864]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:12.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:11 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:12.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:11 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:12.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:11 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:36:13.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:13 smithi137 ceph-mon[36220]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:13.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:13 smithi204 ceph-mon[36739]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:14.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:13 smithi083 ceph-mon[31864]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:14.960 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.956+0000 7eff4a7fc700 1 -- 172.21.15.204:0/629021164 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7eff5402d8d0 con 0x7eff400a1b40 2024-03-11T19:36:14.960 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-03-11T19:36:14.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 -- 172.21.15.204:0/629021164 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff400a1b40 msgr2=0x7eff400a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:14.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff400a1b40 0x7eff400a4000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7eff54101c70 tx=0x7eff5000a3b0 comp rx=0 tx=0).stop 2024-03-11T19:36:14.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 -- 172.21.15.204:0/629021164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff541027b0 msgr2=0x7eff541a1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:14.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff541027b0 0x7eff541a1490 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0094c0 tx=0x7eff4c0094f0 comp rx=0 tx=0).stop 2024-03-11T19:36:14.962 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 -- 172.21.15.204:0/629021164 shutdown_connections 2024-03-11T19:36:14.963 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff400a1b40 0x7eff400a4000 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:14.963 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff541027b0 0x7eff541a1490 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:14.963 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff54100c10 0x7eff54199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:14.963 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 --2- 172.21.15.204:0/629021164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff54101e10 0x7eff5419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:14.963 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 -- 172.21.15.204:0/629021164 >> 172.21.15.204:0/629021164 conn(0x7eff540fc1d0 msgr2=0x7eff541050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:14.963 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 -- 172.21.15.204:0/629021164 shutdown_connections 2024-03-11T19:36:14.963 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:14.958+0000 7eff5ba4b700 1 -- 172.21.15.204:0/629021164 wait complete. 2024-03-11T19:36:15.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:15.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:36:15.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:14 smithi204 ceph-mon[36739]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:15.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:14 smithi204 ceph-mon[36739]: from='osd.11 [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:36:15.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:14 smithi204 ceph-mon[36739]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:36:15.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:14 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:36:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:36:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:14 smithi083 ceph-mon[31864]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:15.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:14 smithi083 ceph-mon[31864]: from='osd.11 [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:36:15.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:14 smithi083 ceph-mon[31864]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:36:15.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:14 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:36:15.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:15.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:36:15.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:14 smithi137 ceph-mon[36220]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:15.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:14 smithi137 ceph-mon[36220]: from='osd.11 [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:36:15.391 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:14 smithi137 ceph-mon[36220]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-11T19:36:15.391 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.391 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:15.391 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:14 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:36:15.677 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.11.service 2024-03-11T19:36:15.680 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-11T19:36:15.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd stat -f json 2024-03-11T19:36:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:36:16.246 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:16 smithi204 ceph-mon[36739]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-11T19:36:16.246 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:16 smithi204 ceph-mon[36739]: osdmap e78: 12 total, 11 up, 12 in 2024-03-11T19:36:16.246 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:16 smithi204 ceph-mon[36739]: from='osd.11 [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:36:16.246 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:16 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:16.246 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:16 smithi204 ceph-mon[36739]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:36:16.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:16 smithi083 ceph-mon[31864]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-11T19:36:16.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:16 smithi083 ceph-mon[31864]: osdmap e78: 12 total, 11 up, 12 in 2024-03-11T19:36:16.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:16 smithi083 ceph-mon[31864]: from='osd.11 [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:36:16.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:16 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:16.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:16 smithi083 ceph-mon[31864]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:36:16.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:16 smithi137 ceph-mon[36220]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-11T19:36:16.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:16 smithi137 ceph-mon[36220]: osdmap e78: 12 total, 11 up, 12 in 2024-03-11T19:36:16.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:16 smithi137 ceph-mon[36220]: from='osd.11 [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:36:16.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:16 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:16.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:16 smithi137 ceph-mon[36220]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-11T19:36:17.109 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:17 smithi204 ceph-mon[36739]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:36:17.109 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:17 smithi204 ceph-mon[36739]: osdmap e79: 12 total, 11 up, 12 in 2024-03-11T19:36:17.109 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:17 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:17.109 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:17 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:17.110 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:17 smithi204 ceph-mon[36739]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:17.110 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:17 smithi204 ceph-mon[36739]: from='osd.11 ' entity='osd.11' 2024-03-11T19:36:17.110 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:36:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:36:16.853+0000 7f0e35740700 -1 osd.11 0 waiting for initial osdmap 2024-03-11T19:36:17.110 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:36:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:36:16.873+0000 7f0e2dd2c700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T19:36:17.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:17 smithi083 ceph-mon[31864]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:36:17.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:17 smithi083 ceph-mon[31864]: osdmap e79: 12 total, 11 up, 12 in 2024-03-11T19:36:17.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:17 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:17.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:17 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:17.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:17 smithi083 ceph-mon[31864]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:17.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:17 smithi083 ceph-mon[31864]: from='osd.11 ' entity='osd.11' 2024-03-11T19:36:17.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:17 smithi137 ceph-mon[36220]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-11T19:36:17.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:17 smithi137 ceph-mon[36220]: osdmap e79: 12 total, 11 up, 12 in 2024-03-11T19:36:17.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:17 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:17.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:17 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:17.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:17 smithi137 ceph-mon[36220]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-11T19:36:17.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:17 smithi137 ceph-mon[36220]: from='osd.11 ' entity='osd.11' 2024-03-11T19:36:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.748+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/3304511372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d081027a0 msgr2=0x7f4d08104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.748+0000 7f4d0ce7a700 1 --2- 172.21.15.83:0/3304511372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d081027a0 0x7f4d08104b90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009990 tx=0x7f4cf8009ca0 comp rx=0 tx=0).stop 2024-03-11T19:36:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.748+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/3304511372 shutdown_connections 2024-03-11T19:36:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.748+0000 7f4d0ce7a700 1 --2- 172.21.15.83:0/3304511372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d08107a00 0x7f4d08109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.748+0000 7f4d0ce7a700 1 --2- 172.21.15.83:0/3304511372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d081027a0 0x7f4d08104b90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.748+0000 7f4d0ce7a700 1 --2- 172.21.15.83:0/3304511372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d081050d0 0x7f4d081074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.748+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/3304511372 >> 172.21.15.83:0/3304511372 conn(0x7f4d080fc170 msgr2=0x7f4d080fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.748+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/3304511372 shutdown_connections 2024-03-11T19:36:17.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.748+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/3304511372 wait complete. 2024-03-11T19:36:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.748+0000 7f4d0ce7a700 1 Processor -- start 2024-03-11T19:36:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.748+0000 7f4d0ce7a700 1 -- start start 2024-03-11T19:36:17.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0ce7a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d081027a0 0x7f4d08102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0ce7a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d081050d0 0x7f4d08100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0ce7a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d08107a00 0x7f4d08100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0ce7a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08101410 con 0x7f4d081027a0 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0ce7a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08101580 con 0x7f4d08107a00 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0ce7a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d081a1d50 con 0x7f4d081050d0 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d081027a0 0x7f4d08102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d081027a0 0x7f4d08102100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46564/0 (socket says 172.21.15.83:46564) 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0659c700 1 -- 172.21.15.83:0/849113204 learned_addr learned my addr 172.21.15.83:0/849113204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d05d9b700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d081050d0 0x7f4d08100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d06d9d700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d08107a00 0x7f4d08100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0659c700 1 -- 172.21.15.83:0/849113204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d081050d0 msgr2=0x7f4d08100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0659c700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d081050d0 0x7f4d08100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0659c700 1 -- 172.21.15.83:0/849113204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d08107a00 msgr2=0x7f4d08100c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:17.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0659c700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d08107a00 0x7f4d08100c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:17.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0659c700 1 -- 172.21.15.83:0/849113204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80095f0 con 0x7f4d081027a0 2024-03-11T19:36:17.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0659c700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d081027a0 0x7f4d08102100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8006010 tx=0x7f4cf800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:17.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4cf77fe700 1 -- 172.21.15.83:0/849113204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf801d070 con 0x7f4d081027a0 2024-03-11T19:36:17.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4cf77fe700 1 -- 172.21.15.83:0/849113204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4cf800fb80 con 0x7f4d081027a0 2024-03-11T19:36:17.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/849113204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d081a1fd0 con 0x7f4d081027a0 2024-03-11T19:36:17.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.749+0000 7f4cf77fe700 1 -- 172.21.15.83:0/849113204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf80173f0 con 0x7f4d081027a0 2024-03-11T19:36:17.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.750+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/849113204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d081a2380 con 0x7f4d081027a0 2024-03-11T19:36:17.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.750+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/849113204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d080fdcf0 con 0x7f4d081027a0 2024-03-11T19:36:17.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.751+0000 7f4cf77fe700 1 -- 172.21.15.83:0/849113204 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4cf800fcf0 con 0x7f4d081027a0 2024-03-11T19:36:17.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.752+0000 7f4cf77fe700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cf00a1b40 0x7f4cf00a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:17.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.752+0000 7f4d05d9b700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cf00a1b40 0x7f4cf00a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:17.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.752+0000 7f4cf77fe700 1 -- 172.21.15.83:0/849113204 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f4cf80ce040 con 0x7f4d081027a0 2024-03-11T19:36:17.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.752+0000 7f4d05d9b700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cf00a1b40 0x7f4cf00a4000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005950 tx=0x7f4cfc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.756+0000 7f4cf77fe700 1 -- 172.21.15.83:0/849113204 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4cf805dd20 con 0x7f4d081027a0 2024-03-11T19:36:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.954+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/849113204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f4d0810d660 con 0x7f4d081027a0 2024-03-11T19:36:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.954+0000 7f4cf77fe700 1 -- 172.21.15.83:0/849113204 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f4cf80979c0 con 0x7f4d081027a0 2024-03-11T19:36:17.959 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:17.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.957+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/849113204 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cf00a1b40 msgr2=0x7f4cf00a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.957+0000 7f4d0ce7a700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cf00a1b40 0x7f4cf00a4000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005950 tx=0x7f4cfc00b410 comp rx=0 tx=0).stop 2024-03-11T19:36:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.957+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/849113204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d081027a0 msgr2=0x7f4d08102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.957+0000 7f4d0ce7a700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d081027a0 0x7f4d08102100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8006010 tx=0x7f4cf800f690 comp rx=0 tx=0).stop 2024-03-11T19:36:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.957+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/849113204 shutdown_connections 2024-03-11T19:36:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.957+0000 7f4d0ce7a700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cf00a1b40 0x7f4cf00a4000 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.958+0000 7f4d0ce7a700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d08107a00 0x7f4d08100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.958+0000 7f4d0ce7a700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d081027a0 0x7f4d08102100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.958+0000 7f4d0ce7a700 1 --2- 172.21.15.83:0/849113204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d081050d0 0x7f4d08100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.958+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/849113204 >> 172.21.15.83:0/849113204 conn(0x7f4d080fc170 msgr2=0x7f4d081030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.958+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/849113204 shutdown_connections 2024-03-11T19:36:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:17.958+0000 7f4d0ce7a700 1 -- 172.21.15.83:0/849113204 wait complete. 2024-03-11T19:36:18.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:18 smithi083 ceph-mon[31864]: purged_snaps scrub starts 2024-03-11T19:36:18.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:18 smithi083 ceph-mon[31864]: purged_snaps scrub ok 2024-03-11T19:36:18.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:18 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:18.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:18 smithi083 ceph-mon[31864]: osd.11 [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819] boot 2024-03-11T19:36:18.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:18 smithi083 ceph-mon[31864]: osdmap e80: 12 total, 12 up, 12 in 2024-03-11T19:36:18.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:18 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:18.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:18 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/849113204' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-11T19:36:18.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:18 smithi137 ceph-mon[36220]: purged_snaps scrub starts 2024-03-11T19:36:18.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:18 smithi137 ceph-mon[36220]: purged_snaps scrub ok 2024-03-11T19:36:18.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:18 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:18.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:18 smithi137 ceph-mon[36220]: osd.11 [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819] boot 2024-03-11T19:36:18.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:18 smithi137 ceph-mon[36220]: osdmap e80: 12 total, 12 up, 12 in 2024-03-11T19:36:18.391 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:18 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:18.391 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:18 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/849113204' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-11T19:36:18.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:18 smithi204 ceph-mon[36739]: purged_snaps scrub starts 2024-03-11T19:36:18.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:18 smithi204 ceph-mon[36739]: purged_snaps scrub ok 2024-03-11T19:36:18.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:18 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:18.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:18 smithi204 ceph-mon[36739]: osd.11 [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819] boot 2024-03-11T19:36:18.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:18 smithi204 ceph-mon[36739]: osdmap e80: 12 total, 12 up, 12 in 2024-03-11T19:36:18.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:18 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-11T19:36:18.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:18 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/849113204' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-11T19:36:18.716 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1710185777,"num_in_osds":12,"osd_in_since":1710185757,"num_remapped_pgs":0} 2024-03-11T19:36:18.717 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-11T19:36:18.974 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:36:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:19 smithi204 ceph-mon[36739]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:36:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:19.291 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:19 smithi137 ceph-mon[36220]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:19.291 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:19.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:19.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:19.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:19.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:36:19.292 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:19.308 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:19 smithi083 ceph-mon[31864]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:19.308 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:19.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:19.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:36:19.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:19.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:19.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:36:19.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:20 smithi083 ceph-mon[31864]: Detected new or changed devices on smithi204 2024-03-11T19:36:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:20 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi204 to 4270M 2024-03-11T19:36:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:20 smithi083 ceph-mon[31864]: osdmap e81: 12 total, 12 up, 12 in 2024-03-11T19:36:20.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:20 smithi137 ceph-mon[36220]: Detected new or changed devices on smithi204 2024-03-11T19:36:20.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:20 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi204 to 4270M 2024-03-11T19:36:20.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:20 smithi137 ceph-mon[36220]: osdmap e81: 12 total, 12 up, 12 in 2024-03-11T19:36:20.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:20 smithi204 ceph-mon[36739]: Detected new or changed devices on smithi204 2024-03-11T19:36:20.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:20 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi204 to 4270M 2024-03-11T19:36:20.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:20 smithi204 ceph-mon[36739]: osdmap e81: 12 total, 12 up, 12 in 2024-03-11T19:36:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.692+0000 7fc61635b700 1 -- 172.21.15.83:0/680555866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc610105e80 msgr2=0x7fc61010da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.692+0000 7fc61635b700 1 --2- 172.21.15.83:0/680555866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc610105e80 0x7fc61010da50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b0a0 tx=0x7fc60800b3b0 comp rx=0 tx=0).stop 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.692+0000 7fc61635b700 1 -- 172.21.15.83:0/680555866 shutdown_connections 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.692+0000 7fc61635b700 1 --2- 172.21.15.83:0/680555866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc610100dc0 0x7fc6101011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.692+0000 7fc61635b700 1 --2- 172.21.15.83:0/680555866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc610105e80 0x7fc61010da50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.692+0000 7fc61635b700 1 --2- 172.21.15.83:0/680555866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc610101770 0x7fc6101057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.692+0000 7fc61635b700 1 -- 172.21.15.83:0/680555866 >> 172.21.15.83:0/680555866 conn(0x7fc610077ae0 msgr2=0x7fc610077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.692+0000 7fc61635b700 1 -- 172.21.15.83:0/680555866 shutdown_connections 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.692+0000 7fc61635b700 1 -- 172.21.15.83:0/680555866 wait complete. 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.692+0000 7fc61635b700 1 Processor -- start 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.692+0000 7fc61635b700 1 -- start start 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc61635b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc610100dc0 0x7fc6101990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc61635b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc610101770 0x7fc610199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc61635b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc610105e80 0x7fc6101a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc61635b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc610199ec0 con 0x7fc610100dc0 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc61635b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6101a4680 con 0x7fc610101770 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc61635b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6101a47c0 con 0x7fc610105e80 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc610100dc0 0x7fc6101990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc610100dc0 0x7fc6101990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51822/0 (socket says 172.21.15.83:51822) 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60ffff700 1 -- 172.21.15.83:0/4009390475 learned_addr learned my addr 172.21.15.83:0/4009390475 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc6148f8700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc610105e80 0x7fc6101a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60ffff700 1 -- 172.21.15.83:0/4009390475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc610105e80 msgr2=0x7fc6101a4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60f7fe700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc610101770 0x7fc610199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60ffff700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc610105e80 0x7fc6101a4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60ffff700 1 -- 172.21.15.83:0/4009390475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc610101770 msgr2=0x7fc610199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:20.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60ffff700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc610101770 0x7fc610199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:20.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60ffff700 1 -- 172.21.15.83:0/4009390475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc608009d00 con 0x7fc610100dc0 2024-03-11T19:36:20.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60ffff700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc610100dc0 0x7fc6101990c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc60000e9a0 tx=0x7fc60000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:20.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60d7fa700 1 -- 172.21.15.83:0/4009390475 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600009790 con 0x7fc610100dc0 2024-03-11T19:36:20.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60d7fa700 1 -- 172.21.15.83:0/4009390475 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc60000cbd0 con 0x7fc610100dc0 2024-03-11T19:36:20.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc61635b700 1 -- 172.21.15.83:0/4009390475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6101a4aa0 con 0x7fc610100dc0 2024-03-11T19:36:20.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc60d7fa700 1 -- 172.21.15.83:0/4009390475 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600018680 con 0x7fc610100dc0 2024-03-11T19:36:20.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.693+0000 7fc61635b700 1 -- 172.21.15.83:0/4009390475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6101a4fa0 con 0x7fc610100dc0 2024-03-11T19:36:20.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.694+0000 7fc61635b700 1 -- 172.21.15.83:0/4009390475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc610109110 con 0x7fc610100dc0 2024-03-11T19:36:20.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.695+0000 7fc60d7fa700 1 -- 172.21.15.83:0/4009390475 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc60000cd40 con 0x7fc610100dc0 2024-03-11T19:36:20.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.696+0000 7fc60d7fa700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5f80a1ac0 0x7fc5f80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:20.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.696+0000 7fc60f7fe700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5f80a1ac0 0x7fc5f80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:20.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.696+0000 7fc60d7fa700 1 -- 172.21.15.83:0/4009390475 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc600014070 con 0x7fc610100dc0 2024-03-11T19:36:20.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.696+0000 7fc60f7fe700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5f80a1ac0 0x7fc5f80a3f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc604005fd0 tx=0x7fc604005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:20.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.700+0000 7fc60d7fa700 1 -- 172.21.15.83:0/4009390475 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc600096d50 con 0x7fc610100dc0 2024-03-11T19:36:20.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.892+0000 7fc61635b700 1 -- 172.21.15.83:0/4009390475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc610102280 con 0x7fc610100dc0 2024-03-11T19:36:20.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.893+0000 7fc60d7fa700 1 -- 172.21.15.83:0/4009390475 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18469 (secure 0 0 0) 0x7fc60009a410 con 0x7fc610100dc0 2024-03-11T19:36:20.897 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:20.897 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:36:19.029213+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:36:20.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 -- 172.21.15.83:0/4009390475 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5f80a1ac0 msgr2=0x7fc5f80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:20.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5f80a1ac0 0x7fc5f80a3f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc604005fd0 tx=0x7fc604005f60 comp rx=0 tx=0).stop 2024-03-11T19:36:20.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 -- 172.21.15.83:0/4009390475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc610100dc0 msgr2=0x7fc6101990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:20.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc610100dc0 0x7fc6101990c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc60000e9a0 tx=0x7fc60000ecb0 comp rx=0 tx=0).stop 2024-03-11T19:36:20.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 -- 172.21.15.83:0/4009390475 shutdown_connections 2024-03-11T19:36:20.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5f80a1ac0 0x7fc5f80a3f80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:20.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc610105e80 0x7fc6101a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:20.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc610100dc0 0x7fc6101990c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:20.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 --2- 172.21.15.83:0/4009390475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc610101770 0x7fc610199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:20.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 -- 172.21.15.83:0/4009390475 >> 172.21.15.83:0/4009390475 conn(0x7fc610077ae0 msgr2=0x7fc6100febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:20.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 -- 172.21.15.83:0/4009390475 shutdown_connections 2024-03-11T19:36:20.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:20.895+0000 7fc61635b700 1 -- 172.21.15.83:0/4009390475 wait complete. 2024-03-11T19:36:21.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:21 smithi083 ceph-mon[31864]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:21.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:21 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4009390475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:36:21.360 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:21 smithi204 ceph-mon[36739]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:21.360 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:21 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4009390475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:36:21.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:21 smithi137 ceph-mon[36220]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:21.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:21 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4009390475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:36:21.636 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:36:21.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-11T19:36:21.876 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:36:23.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:23 smithi083 ceph-mon[31864]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.576+0000 7f4938a38700 1 -- 172.21.15.83:0/1056156130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49341026e0 msgr2=0x7f493410ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.576+0000 7f4938a38700 1 --2- 172.21.15.83:0/1056156130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49341026e0 0x7f493410ead0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4924009990 tx=0x7f4924009ca0 comp rx=0 tx=0).stop 2024-03-11T19:36:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.576+0000 7f4938a38700 1 -- 172.21.15.83:0/1056156130 shutdown_connections 2024-03-11T19:36:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.576+0000 7f4938a38700 1 --2- 172.21.15.83:0/1056156130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4934107d90 0x7f4934108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.576+0000 7f4938a38700 1 --2- 172.21.15.83:0/1056156130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49341026e0 0x7f493410ead0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.576+0000 7f4938a38700 1 --2- 172.21.15.83:0/1056156130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4934101d20 0x7f49341021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.576+0000 7f4938a38700 1 -- 172.21.15.83:0/1056156130 >> 172.21.15.83:0/1056156130 conn(0x7f4934078d40 msgr2=0x7f4934079150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.576+0000 7f4938a38700 1 -- 172.21.15.83:0/1056156130 shutdown_connections 2024-03-11T19:36:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.576+0000 7f4938a38700 1 -- 172.21.15.83:0/1056156130 wait complete. 2024-03-11T19:36:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4938a38700 1 Processor -- start 2024-03-11T19:36:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4938a38700 1 -- start start 2024-03-11T19:36:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4938a38700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4934101d20 0x7f493419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4938a38700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49341026e0 0x7f493419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4938a38700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4934107d90 0x7f493419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4938a38700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493419f1c0 con 0x7f4934107d90 2024-03-11T19:36:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4938a38700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493419f330 con 0x7f49341026e0 2024-03-11T19:36:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4938a38700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493419f4f0 con 0x7f4934101d20 2024-03-11T19:36:23.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4932d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4934107d90 0x7f493419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4932d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4934107d90 0x7f493419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51832/0 (socket says 172.21.15.83:51832) 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4932d9d700 1 -- 172.21.15.83:0/1849246979 learned_addr learned my addr 172.21.15.83:0/1849246979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.577+0000 7f4931d9b700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49341026e0 0x7f493419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f493259c700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4934101d20 0x7f493419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f4932d9d700 1 -- 172.21.15.83:0/1849246979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4934101d20 msgr2=0x7f493419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f4932d9d700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4934101d20 0x7f493419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f4932d9d700 1 -- 172.21.15.83:0/1849246979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49341026e0 msgr2=0x7f493419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f4932d9d700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49341026e0 0x7f493419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f4932d9d700 1 -- 172.21.15.83:0/1849246979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49240095f0 con 0x7f4934107d90 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f493259c700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4934101d20 0x7f493419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f4932d9d700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4934107d90 0x7f493419ebb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49240095c0 tx=0x7f492400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f49237fe700 1 -- 172.21.15.83:0/1849246979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492401d070 con 0x7f4934107d90 2024-03-11T19:36:23.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f4938a38700 1 -- 172.21.15.83:0/1849246979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493419f770 con 0x7f4934107d90 2024-03-11T19:36:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f49237fe700 1 -- 172.21.15.83:0/1849246979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4924003680 con 0x7f4934107d90 2024-03-11T19:36:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f49237fe700 1 -- 172.21.15.83:0/1849246979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924004bd0 con 0x7f4934107d90 2024-03-11T19:36:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.578+0000 7f4938a38700 1 -- 172.21.15.83:0/1849246979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49341a7030 con 0x7f4934107d90 2024-03-11T19:36:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.579+0000 7f4938a38700 1 -- 172.21.15.83:0/1849246979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4934068930 con 0x7f4934107d90 2024-03-11T19:36:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.580+0000 7f49237fe700 1 -- 172.21.15.83:0/1849246979 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4924004d30 con 0x7f4934107d90 2024-03-11T19:36:23.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.580+0000 7f49237fe700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f491c0a1ac0 0x7f491c0a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:23.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.581+0000 7f493259c700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f491c0a1ac0 0x7f491c0a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:23.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.581+0000 7f49237fe700 1 -- 172.21.15.83:0/1849246979 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f49240cdde0 con 0x7f4934107d90 2024-03-11T19:36:23.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.581+0000 7f493259c700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f491c0a1ac0 0x7f491c0a3f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f492c009690 tx=0x7f492c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.585+0000 7f49237fe700 1 -- 172.21.15.83:0/1849246979 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4924097520 con 0x7f4934107d90 2024-03-11T19:36:23.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.775+0000 7f4938a38700 1 -- 172.21.15.83:0/1849246979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f493419f9e0 con 0x7f4934107d90 2024-03-11T19:36:23.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.775+0000 7f49237fe700 1 -- 172.21.15.83:0/1849246979 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f492409abe0 con 0x7f4934107d90 2024-03-11T19:36:23.779 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:36:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 -- 172.21.15.83:0/1849246979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f491c0a1ac0 msgr2=0x7f491c0a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f491c0a1ac0 0x7f491c0a3f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f492c009690 tx=0x7f492c009040 comp rx=0 tx=0).stop 2024-03-11T19:36:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 -- 172.21.15.83:0/1849246979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4934107d90 msgr2=0x7f493419ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4934107d90 0x7f493419ebb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49240095c0 tx=0x7f492400f690 comp rx=0 tx=0).stop 2024-03-11T19:36:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 -- 172.21.15.83:0/1849246979 shutdown_connections 2024-03-11T19:36:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f491c0a1ac0 0x7f491c0a3f80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4934101d20 0x7f493419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4934107d90 0x7f493419ebb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 --2- 172.21.15.83:0/1849246979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49341026e0 0x7f493419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 -- 172.21.15.83:0/1849246979 >> 172.21.15.83:0/1849246979 conn(0x7f4934078d40 msgr2=0x7f493410d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 -- 172.21.15.83:0/1849246979 shutdown_connections 2024-03-11T19:36:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:23.777+0000 7f4938a38700 1 -- 172.21.15.83:0/1849246979 wait complete. 2024-03-11T19:36:23.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:23 smithi137 ceph-mon[36220]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:23 smithi204 ceph-mon[36739]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:24.491 INFO:tasks.cephadm:Adding mds.a on smithi083 2024-03-11T19:36:24.491 INFO:tasks.cephadm:Adding mds.d on smithi083 2024-03-11T19:36:24.492 INFO:tasks.cephadm:Adding mds.g on smithi083 2024-03-11T19:36:24.492 INFO:tasks.cephadm:Adding mds.j on smithi083 2024-03-11T19:36:24.492 INFO:tasks.cephadm:Adding mds.b on smithi137 2024-03-11T19:36:24.492 INFO:tasks.cephadm:Adding mds.e on smithi137 2024-03-11T19:36:24.492 INFO:tasks.cephadm:Adding mds.h on smithi137 2024-03-11T19:36:24.492 INFO:tasks.cephadm:Adding mds.k on smithi137 2024-03-11T19:36:24.492 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-03-11T19:36:24.492 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-03-11T19:36:24.492 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-03-11T19:36:24.492 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-03-11T19:36:24.493 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph orch apply mds all '12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-03-11T19:36:24.746 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:36:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:24 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1849246979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:36:24.867 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:24 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1849246979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:36:24.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:24 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1849246979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:36:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:25 smithi083 ceph-mon[31864]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:25.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:25 smithi137 ceph-mon[36220]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:25.897 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:25 smithi204 ceph-mon[36739]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:26.390 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.384+0000 7f2861bc5700 1 -- 172.21.15.204:0/2749131415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f285c1050d0 msgr2=0x7f285c1074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:26.390 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.384+0000 7f2861bc5700 1 --2- 172.21.15.204:0/2749131415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f285c1050d0 0x7f285c1074c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2850009990 tx=0x7f2850009ca0 comp rx=0 tx=0).stop 2024-03-11T19:36:26.390 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.384+0000 7f2861bc5700 1 -- 172.21.15.204:0/2749131415 shutdown_connections 2024-03-11T19:36:26.390 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.384+0000 7f2861bc5700 1 --2- 172.21.15.204:0/2749131415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f285c107a00 0x7f285c109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:26.390 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.384+0000 7f2861bc5700 1 --2- 172.21.15.204:0/2749131415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f285c1027a0 0x7f285c104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:26.390 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.384+0000 7f2861bc5700 1 --2- 172.21.15.204:0/2749131415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f285c1050d0 0x7f285c1074c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:26.390 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.384+0000 7f2861bc5700 1 -- 172.21.15.204:0/2749131415 >> 172.21.15.204:0/2749131415 conn(0x7f285c0fc1d0 msgr2=0x7f285c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:26.390 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.384+0000 7f2861bc5700 1 -- 172.21.15.204:0/2749131415 shutdown_connections 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.384+0000 7f2861bc5700 1 -- 172.21.15.204:0/2749131415 wait complete. 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.384+0000 7f2861bc5700 1 Processor -- start 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f2861bc5700 1 -- start start 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f2861bc5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f285c1027a0 0x7f285c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f2861bc5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f285c1050d0 0x7f285c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f2861bc5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f285c107a00 0x7f285c19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f2861bc5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c19f780 con 0x7f285c1050d0 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f2861bc5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c19f8c0 con 0x7f285c1027a0 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f2861bc5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c19fa80 con 0x7f285c107a00 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f285bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f285c107a00 0x7f285c19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f285bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f285c107a00 0x7f285c19f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:42718/0 (socket says 172.21.15.204:42718) 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f285bfff700 1 -- 172.21.15.204:0/2215149528 learned_addr learned my addr 172.21.15.204:0/2215149528 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-11T19:36:26.391 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f285b7fe700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f285c1027a0 0x7f285c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f285affd700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f285c1050d0 0x7f285c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f285bfff700 1 -- 172.21.15.204:0/2215149528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f285c1027a0 msgr2=0x7f285c197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f285bfff700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f285c1027a0 0x7f285c197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f285bfff700 1 -- 172.21.15.204:0/2215149528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f285c1050d0 msgr2=0x7f285c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f285bfff700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f285c1050d0 0x7f285c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f285bfff700 1 -- 172.21.15.204:0/2215149528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28500095f0 con 0x7f285c107a00 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.385+0000 7f285bfff700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f285c107a00 0x7f285c19f1b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f28540094f0 tx=0x7f285400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.386+0000 7f2858ff9700 1 -- 172.21.15.204:0/2215149528 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285400ddd0 con 0x7f285c107a00 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.386+0000 7f2861bc5700 1 -- 172.21.15.204:0/2215149528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285c19fd60 con 0x7f285c107a00 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.386+0000 7f2858ff9700 1 -- 172.21.15.204:0/2215149528 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f285400f040 con 0x7f285c107a00 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.386+0000 7f2858ff9700 1 -- 172.21.15.204:0/2215149528 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854015400 con 0x7f285c107a00 2024-03-11T19:36:26.392 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.386+0000 7f2861bc5700 1 -- 172.21.15.204:0/2215149528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285c1a01b0 con 0x7f285c107a00 2024-03-11T19:36:26.394 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.387+0000 7f2861bc5700 1 -- 172.21.15.204:0/2215149528 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285c0fdd50 con 0x7f285c107a00 2024-03-11T19:36:26.394 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.388+0000 7f2858ff9700 1 -- 172.21.15.204:0/2215149528 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2854004cd0 con 0x7f285c107a00 2024-03-11T19:36:26.395 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.388+0000 7f2858ff9700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f28440a1ac0 0x7f28440a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:26.395 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.389+0000 7f2858ff9700 1 -- 172.21.15.204:0/2215149528 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f28540cc8e0 con 0x7f285c107a00 2024-03-11T19:36:26.395 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.389+0000 7f285b7fe700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f28440a1ac0 0x7f28440a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:26.395 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.389+0000 7f285b7fe700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f28440a1ac0 0x7f28440a3f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f285c199110 tx=0x7f284c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:26.397 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.393+0000 7f2858ff9700 1 -- 172.21.15.204:0/2215149528 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2854095f90 con 0x7f285c107a00 2024-03-11T19:36:26.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.597+0000 7f2861bc5700 1 -- 172.21.15.204:0/2215149528 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f285c02dab0 con 0x7f28440a1ac0 2024-03-11T19:36:26.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.604+0000 7f2858ff9700 1 -- 172.21.15.204:0/2215149528 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f285c02dab0 con 0x7f28440a1ac0 2024-03-11T19:36:26.608 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-03-11T19:36:26.610 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 -- 172.21.15.204:0/2215149528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f28440a1ac0 msgr2=0x7f28440a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:26.610 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f28440a1ac0 0x7f28440a3f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f285c199110 tx=0x7f284c009260 comp rx=0 tx=0).stop 2024-03-11T19:36:26.610 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 -- 172.21.15.204:0/2215149528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f285c107a00 msgr2=0x7f285c19f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:26.611 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f285c107a00 0x7f285c19f1b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f28540094f0 tx=0x7f285400d3b0 comp rx=0 tx=0).stop 2024-03-11T19:36:26.611 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 -- 172.21.15.204:0/2215149528 shutdown_connections 2024-03-11T19:36:26.611 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f28440a1ac0 0x7f28440a3f80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:26.611 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f285c107a00 0x7f285c19f1b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:26.611 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f285c1027a0 0x7f285c197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:26.611 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 --2- 172.21.15.204:0/2215149528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f285c1050d0 0x7f285c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:26.611 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 -- 172.21.15.204:0/2215149528 >> 172.21.15.204:0/2215149528 conn(0x7f285c0fc1d0 msgr2=0x7f285c103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:26.611 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 -- 172.21.15.204:0/2215149528 shutdown_connections 2024-03-11T19:36:26.611 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-11T19:36:26.606+0000 7f2861bc5700 1 -- 172.21.15.204:0/2215149528 wait complete. 2024-03-11T19:36:27.232 DEBUG:teuthology.orchestra.run.smithi083:mds.a> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.a.service 2024-03-11T19:36:27.234 DEBUG:teuthology.orchestra.run.smithi083:mds.d> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.d.service 2024-03-11T19:36:27.277 DEBUG:teuthology.orchestra.run.smithi083:mds.g> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.g.service 2024-03-11T19:36:27.319 DEBUG:teuthology.orchestra.run.smithi083:mds.j> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.j.service 2024-03-11T19:36:27.363 DEBUG:teuthology.orchestra.run.smithi137:mds.b> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.b.service 2024-03-11T19:36:27.365 DEBUG:teuthology.orchestra.run.smithi137:mds.e> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.e.service 2024-03-11T19:36:27.408 DEBUG:teuthology.orchestra.run.smithi137:mds.h> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.h.service 2024-03-11T19:36:27.452 DEBUG:teuthology.orchestra.run.smithi137:mds.k> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.k.service 2024-03-11T19:36:27.495 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.c.service 2024-03-11T19:36:27.497 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.f.service 2024-03-11T19:36:27.540 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.i.service 2024-03-11T19:36:27.583 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.l.service 2024-03-11T19:36:27.626 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-11T19:36:27.626 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:36:27.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:27 smithi137 ceph-mon[36220]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:27.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:27.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:36:27.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:27.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:36:27.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:27.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:27.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:27.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.729+0000 7fb475f3a640 1 -- 172.21.15.83:0/3373637469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4701011a0 msgr2=0x7fb4701015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.729+0000 7fb475f3a640 1 --2- 172.21.15.83:0/3373637469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4701011a0 0x7fb4701015c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb460009900 tx=0x7fb46002bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.729+0000 7fb475f3a640 1 -- 172.21.15.83:0/3373637469 shutdown_connections 2024-03-11T19:36:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.729+0000 7fb475f3a640 1 --2- 172.21.15.83:0/3373637469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb470102d50 0x7fb4701095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.729+0000 7fb475f3a640 1 --2- 172.21.15.83:0/3373637469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb470102390 0x7fb470102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.729+0000 7fb475f3a640 1 --2- 172.21.15.83:0/3373637469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4701011a0 0x7fb4701015c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.729+0000 7fb475f3a640 1 -- 172.21.15.83:0/3373637469 >> 172.21.15.83:0/3373637469 conn(0x7fb4700fc730 msgr2=0x7fb4700feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.729+0000 7fb475f3a640 1 -- 172.21.15.83:0/3373637469 shutdown_connections 2024-03-11T19:36:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.729+0000 7fb475f3a640 1 -- 172.21.15.83:0/3373637469 wait complete. 2024-03-11T19:36:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb475f3a640 1 Processor -- start 2024-03-11T19:36:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb475f3a640 1 -- start start 2024-03-11T19:36:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb475f3a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4701011a0 0x7fb470197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb475f3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb470102390 0x7fb4701977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb475f3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb470102d50 0x7fb47019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb475f3a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47019ed50 con 0x7fb470102d50 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb475f3a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47019eec0 con 0x7fb470102390 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb475f3a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47019f080 con 0x7fb4701011a0 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb470102d50 0x7fb47019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb470102d50 0x7fb47019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51842/0 (socket says 172.21.15.83:51842) 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb46ffff640 1 -- 172.21.15.83:0/3148566345 learned_addr learned my addr 172.21.15.83:0/3148566345 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb46f7fe640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4701011a0 0x7fb470197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb46effd640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb470102390 0x7fb4701977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb46ffff640 1 -- 172.21.15.83:0/3148566345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4701011a0 msgr2=0x7fb470197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb46ffff640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4701011a0 0x7fb470197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb46ffff640 1 -- 172.21.15.83:0/3148566345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb470102390 msgr2=0x7fb4701977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:27.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb46ffff640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb470102390 0x7fb4701977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.730+0000 7fb46ffff640 1 -- 172.21.15.83:0/3148566345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb460009560 con 0x7fb470102d50 2024-03-11T19:36:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.731+0000 7fb46ffff640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb470102d50 0x7fb47019e810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb46800eb10 tx=0x7fb46800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.731+0000 7fb46cff9640 1 -- 172.21.15.83:0/3148566345 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468004b20 con 0x7fb470102d50 2024-03-11T19:36:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.731+0000 7fb46cff9640 1 -- 172.21.15.83:0/3148566345 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb468004c80 con 0x7fb470102d50 2024-03-11T19:36:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.731+0000 7fb475f3a640 1 -- 172.21.15.83:0/3148566345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47019f360 con 0x7fb470102d50 2024-03-11T19:36:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.731+0000 7fb46cff9640 1 -- 172.21.15.83:0/3148566345 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468010650 con 0x7fb470102d50 2024-03-11T19:36:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.731+0000 7fb475f3a640 1 -- 172.21.15.83:0/3148566345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47019f870 con 0x7fb470102d50 2024-03-11T19:36:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.732+0000 7fb475f3a640 1 -- 172.21.15.83:0/3148566345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4701084c0 con 0x7fb470102d50 2024-03-11T19:36:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.733+0000 7fb46cff9640 1 -- 172.21.15.83:0/3148566345 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fb468005020 con 0x7fb470102d50 2024-03-11T19:36:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.733+0000 7fb46cff9640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4580a1a30 0x7fb4580a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.734+0000 7fb46f7fe640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4580a1a30 0x7fb4580a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.734+0000 7fb46cff9640 1 -- 172.21.15.83:0/3148566345 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb468014070 con 0x7fb470102d50 2024-03-11T19:36:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.734+0000 7fb46f7fe640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4580a1a30 0x7fb4580a3ef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb46002c240 tx=0x7fb46002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.738+0000 7fb46cff9640 1 -- 172.21.15.83:0/3148566345 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb468096ba0 con 0x7fb470102d50 2024-03-11T19:36:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:27 smithi083 ceph-mon[31864]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:36:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:36:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:27.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:27 smithi204 ceph-mon[36739]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:36:27.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:27.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:36:27.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:27.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:27.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:27.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:27.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.914+0000 7fb475f3a640 1 -- 172.21.15.83:0/3148566345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb47002ce40 con 0x7fb470102d50 2024-03-11T19:36:27.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.915+0000 7fb46cff9640 1 -- 172.21.15.83:0/3148566345 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18469 (secure 0 0 0) 0x7fb46809a260 con 0x7fb470102d50 2024-03-11T19:36:27.920 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:27.920 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:36:19.029213+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:36:27.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.917+0000 7fb475f3a640 1 -- 172.21.15.83:0/3148566345 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4580a1a30 msgr2=0x7fb4580a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.917+0000 7fb475f3a640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4580a1a30 0x7fb4580a3ef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb46002c240 tx=0x7fb46002c7e0 comp rx=0 tx=0).stop 2024-03-11T19:36:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.917+0000 7fb475f3a640 1 -- 172.21.15.83:0/3148566345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb470102d50 msgr2=0x7fb47019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.917+0000 7fb475f3a640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb470102d50 0x7fb47019e810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb46800eb10 tx=0x7fb46800c2d0 comp rx=0 tx=0).stop 2024-03-11T19:36:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.917+0000 7fb475f3a640 1 -- 172.21.15.83:0/3148566345 shutdown_connections 2024-03-11T19:36:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.917+0000 7fb475f3a640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4580a1a30 0x7fb4580a3ef0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.917+0000 7fb475f3a640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb470102d50 0x7fb47019e810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.917+0000 7fb475f3a640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb470102390 0x7fb4701977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.917+0000 7fb475f3a640 1 --2- 172.21.15.83:0/3148566345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4701011a0 0x7fb470197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.917+0000 7fb475f3a640 1 -- 172.21.15.83:0/3148566345 >> 172.21.15.83:0/3148566345 conn(0x7fb4700fc730 msgr2=0x7fb470105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.918+0000 7fb475f3a640 1 -- 172.21.15.83:0/3148566345 shutdown_connections 2024-03-11T19:36:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:27.918+0000 7fb475f3a640 1 -- 172.21.15.83:0/3148566345 wait complete. 2024-03-11T19:36:27.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:36:27.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:36:28.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.034+0000 7ff92fadf640 1 -- 172.21.15.83:0/1905348376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff928102d40 msgr2=0x7ff9281095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.034+0000 7ff92fadf640 1 --2- 172.21.15.83:0/1905348376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff928102d40 0x7ff9281095d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff920009900 tx=0x7ff92002bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.034+0000 7ff92fadf640 1 -- 172.21.15.83:0/1905348376 shutdown_connections 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.034+0000 7ff92fadf640 1 --2- 172.21.15.83:0/1905348376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff928102d40 0x7ff9281095d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.034+0000 7ff92fadf640 1 --2- 172.21.15.83:0/1905348376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff928102380 0x7ff928102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.034+0000 7ff92fadf640 1 --2- 172.21.15.83:0/1905348376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff928101190 0x7ff9281015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.034+0000 7ff92fadf640 1 -- 172.21.15.83:0/1905348376 >> 172.21.15.83:0/1905348376 conn(0x7ff9280fc720 msgr2=0x7ff9280feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.034+0000 7ff92fadf640 1 -- 172.21.15.83:0/1905348376 shutdown_connections 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.034+0000 7ff92fadf640 1 -- 172.21.15.83:0/1905348376 wait complete. 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92fadf640 1 Processor -- start 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92fadf640 1 -- start start 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92fadf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff928101190 0x7ff9281971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92fadf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff928102380 0x7ff928197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92fadf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff928102d40 0x7ff92819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92fadf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819ecd0 con 0x7ff928102380 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92fadf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819ee40 con 0x7ff928101190 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92fadf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92819f000 con 0x7ff928102d40 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92d07a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff928102380 0x7ff928197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92d07a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff928102380 0x7ff928197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51854/0 (socket says 172.21.15.83:51854) 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92d07a640 1 -- 172.21.15.83:0/280336273 learned_addr learned my addr 172.21.15.83:0/280336273 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92e07c640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff928102d40 0x7ff92819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92d87b640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff928101190 0x7ff9281971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92d07a640 1 -- 172.21.15.83:0/280336273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff928102d40 msgr2=0x7ff92819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92d07a640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff928102d40 0x7ff92819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92d07a640 1 -- 172.21.15.83:0/280336273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff928101190 msgr2=0x7ff9281971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92d07a640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff928101190 0x7ff9281971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92d07a640 1 -- 172.21.15.83:0/280336273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920009560 con 0x7ff928102380 2024-03-11T19:36:28.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92e07c640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff928102d40 0x7ff92819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:36:28.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92d07a640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff928102380 0x7ff928197720 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e9f0 tx=0x7ff91800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:28.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff91effd640 1 -- 172.21.15.83:0/280336273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918009700 con 0x7ff928102380 2024-03-11T19:36:28.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff91effd640 1 -- 172.21.15.83:0/280336273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff91800cd80 con 0x7ff928102380 2024-03-11T19:36:28.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff91effd640 1 -- 172.21.15.83:0/280336273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918018920 con 0x7ff928102380 2024-03-11T19:36:28.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92fadf640 1 -- 172.21.15.83:0/280336273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92819f2e0 con 0x7ff928102380 2024-03-11T19:36:28.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.035+0000 7ff92fadf640 1 -- 172.21.15.83:0/280336273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92819f840 con 0x7ff928102380 2024-03-11T19:36:28.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.037+0000 7ff92fadf640 1 -- 172.21.15.83:0/280336273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff928108530 con 0x7ff928102380 2024-03-11T19:36:28.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.038+0000 7ff91effd640 1 -- 172.21.15.83:0/280336273 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff918018a80 con 0x7ff928102380 2024-03-11T19:36:28.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.038+0000 7ff91effd640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9140a1ab0 0x7ff9140a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:28.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.038+0000 7ff92d87b640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9140a1ab0 0x7ff9140a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:28.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.038+0000 7ff91effd640 1 -- 172.21.15.83:0/280336273 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff918014070 con 0x7ff928102380 2024-03-11T19:36:28.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.039+0000 7ff92d87b640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9140a1ab0 0x7ff9140a3f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff9281021e0 tx=0x7ff9240098f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.043+0000 7ff91effd640 1 -- 172.21.15.83:0/280336273 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff918052d20 con 0x7ff928102380 2024-03-11T19:36:28.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.213+0000 7ff92fadf640 1 -- 172.21.15.83:0/280336273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff92802ce40 con 0x7ff928102380 2024-03-11T19:36:28.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.213+0000 7ff91effd640 1 -- 172.21.15.83:0/280336273 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9180969e0 con 0x7ff928102380 2024-03-11T19:36:28.217 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:36:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 -- 172.21.15.83:0/280336273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9140a1ab0 msgr2=0x7ff9140a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9140a1ab0 0x7ff9140a3f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff9281021e0 tx=0x7ff9240098f0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 -- 172.21.15.83:0/280336273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff928102380 msgr2=0x7ff928197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff928102380 0x7ff928197720 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff91800e9f0 tx=0x7ff91800eeb0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 -- 172.21.15.83:0/280336273 shutdown_connections 2024-03-11T19:36:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9140a1ab0 0x7ff9140a3f70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff928102d40 0x7ff92819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff928102380 0x7ff928197720 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 --2- 172.21.15.83:0/280336273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff928101190 0x7ff9281971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 -- 172.21.15.83:0/280336273 >> 172.21.15.83:0/280336273 conn(0x7ff9280fc720 msgr2=0x7ff928105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 -- 172.21.15.83:0/280336273 shutdown_connections 2024-03-11T19:36:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.215+0000 7ff92fadf640 1 -- 172.21.15.83:0/280336273 wait complete. 2024-03-11T19:36:28.230 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:28.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.327+0000 7f57a909d640 1 -- 172.21.15.83:0/1910979649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57a4106e80 msgr2=0x7f57a41092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.327+0000 7f57a909d640 1 --2- 172.21.15.83:0/1910979649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57a4106e80 0x7f57a41092d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f579c009ca0 tx=0x7f579c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.327+0000 7f57a909d640 1 -- 172.21.15.83:0/1910979649 shutdown_connections 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.327+0000 7f57a909d640 1 --2- 172.21.15.83:0/1910979649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57a4106e80 0x7f57a41092d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.327+0000 7f57a909d640 1 --2- 172.21.15.83:0/1910979649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57a4104550 0x7f57a4106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.327+0000 7f57a909d640 1 --2- 172.21.15.83:0/1910979649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a4101c20 0x7f57a4104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.327+0000 7f57a909d640 1 -- 172.21.15.83:0/1910979649 >> 172.21.15.83:0/1910979649 conn(0x7f57a40fb610 msgr2=0x7f57a40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.327+0000 7f57a909d640 1 -- 172.21.15.83:0/1910979649 shutdown_connections 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.327+0000 7f57a909d640 1 -- 172.21.15.83:0/1910979649 wait complete. 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a909d640 1 Processor -- start 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a909d640 1 -- start start 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a909d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57a4101c20 0x7f57a41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:28.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a909d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57a4104550 0x7f57a4197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a909d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a4106e80 0x7f57a419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a909d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a419ece0 con 0x7f57a4101c20 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a909d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a419ee50 con 0x7f57a4104550 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a909d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a419f010 con 0x7f57a4106e80 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57a4101c20 0x7f57a41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57a4101c20 0x7f57a41971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51864/0 (socket says 172.21.15.83:51864) 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a2d9d640 1 -- 172.21.15.83:0/2413729834 learned_addr learned my addr 172.21.15.83:0/2413729834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a359e640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a4106e80 0x7f57a419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f579bfff640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57a4104550 0x7f57a4197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a2d9d640 1 -- 172.21.15.83:0/2413729834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a4106e80 msgr2=0x7f57a419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a2d9d640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a4106e80 0x7f57a419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a2d9d640 1 -- 172.21.15.83:0/2413729834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57a4104550 msgr2=0x7f57a4197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:28.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a2d9d640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57a4104550 0x7f57a4197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f57a2d9d640 1 -- 172.21.15.83:0/2413729834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c009900 con 0x7f57a4101c20 2024-03-11T19:36:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.328+0000 7f579bfff640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57a4104550 0x7f57a4197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:36:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.329+0000 7f57a2d9d640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57a4101c20 0x7f57a41971f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f579400e9f0 tx=0x7f579400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.329+0000 7f57a0d99640 1 -- 172.21.15.83:0/2413729834 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794009720 con 0x7f57a4101c20 2024-03-11T19:36:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.329+0000 7f57a0d99640 1 -- 172.21.15.83:0/2413729834 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f579400cd80 con 0x7f57a4101c20 2024-03-11T19:36:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.329+0000 7f57a0d99640 1 -- 172.21.15.83:0/2413729834 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794010690 con 0x7f57a4101c20 2024-03-11T19:36:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.329+0000 7f57a909d640 1 -- 172.21.15.83:0/2413729834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a419f2f0 con 0x7f57a4101c20 2024-03-11T19:36:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.329+0000 7f57a909d640 1 -- 172.21.15.83:0/2413729834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a419f800 con 0x7f57a4101c20 2024-03-11T19:36:28.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.330+0000 7f57a909d640 1 -- 172.21.15.83:0/2413729834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a40fd100 con 0x7f57a4101c20 2024-03-11T19:36:28.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.331+0000 7f57a0d99640 1 -- 172.21.15.83:0/2413729834 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f57940046c0 con 0x7f57a4101c20 2024-03-11T19:36:28.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.331+0000 7f57a0d99640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57900a1ab0 0x7f57900a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:28.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.332+0000 7f579bfff640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57900a1ab0 0x7f57900a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:28.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.332+0000 7f57a0d99640 1 -- 172.21.15.83:0/2413729834 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5794014070 con 0x7f57a4101c20 2024-03-11T19:36:28.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.332+0000 7f579bfff640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57900a1ab0 0x7f57900a3f70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57a4198700 tx=0x7f578c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:28.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.336+0000 7f57a0d99640 1 -- 172.21.15.83:0/2413729834 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5794096a60 con 0x7f57a4101c20 2024-03-11T19:36:28.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.543+0000 7f57a909d640 1 -- 172.21.15.83:0/2413729834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57a410cbb0 con 0x7f57a4101c20 2024-03-11T19:36:28.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.544+0000 7f57a0d99640 1 -- 172.21.15.83:0/2413729834 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f579409a120 con 0x7f57a4101c20 2024-03-11T19:36:28.548 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:28.548 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-11T19:36:28.548 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 1 2024-03-11T19:36:28.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.545+0000 7f57a909d640 1 -- 172.21.15.83:0/2413729834 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57900a1ab0 msgr2=0x7f57900a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.545+0000 7f57a909d640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57900a1ab0 0x7f57900a3f70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57a4198700 tx=0x7f578c005f90 comp rx=0 tx=0).stop 2024-03-11T19:36:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.545+0000 7f57a909d640 1 -- 172.21.15.83:0/2413729834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57a4101c20 msgr2=0x7f57a41971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.545+0000 7f57a909d640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57a4101c20 0x7f57a41971f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f579400e9f0 tx=0x7f579400edf0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.546+0000 7f57a909d640 1 -- 172.21.15.83:0/2413729834 shutdown_connections 2024-03-11T19:36:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.546+0000 7f57a909d640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57900a1ab0 0x7f57900a3f70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.546+0000 7f57a909d640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57a4106e80 0x7f57a419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.546+0000 7f57a909d640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57a4104550 0x7f57a4197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.546+0000 7f57a909d640 1 --2- 172.21.15.83:0/2413729834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57a4101c20 0x7f57a41971f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.546+0000 7f57a909d640 1 -- 172.21.15.83:0/2413729834 >> 172.21.15.83:0/2413729834 conn(0x7f57a40fb610 msgr2=0x7f57a4102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.546+0000 7f57a909d640 1 -- 172.21.15.83:0/2413729834 shutdown_connections 2024-03-11T19:36:28.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:28.546+0000 7f57a909d640 1 -- 172.21.15.83:0/2413729834 wait complete. 2024-03-11T19:36:28.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:28 smithi083 ceph-mon[31864]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:36:28.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:28 smithi083 ceph-mon[31864]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-11T19:36:28.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:28 smithi083 ceph-mon[31864]: Deploying daemon mds.l on smithi204 2024-03-11T19:36:28.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:28 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3148566345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:36:28.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:28 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/280336273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:36:28.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:28 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2413729834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:28.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:28 smithi137 ceph-mon[36220]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:36:28.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:28 smithi137 ceph-mon[36220]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-11T19:36:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:28 smithi137 ceph-mon[36220]: Deploying daemon mds.l on smithi204 2024-03-11T19:36:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:28 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3148566345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:36:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:28 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/280336273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:36:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:28 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2413729834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:28 smithi204 ceph-mon[36739]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:36:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:28 smithi204 ceph-mon[36739]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-11T19:36:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:28 smithi204 ceph-mon[36739]: Deploying daemon mds.l on smithi204 2024-03-11T19:36:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:28 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3148566345' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:36:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:28 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/280336273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:36:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:28 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2413729834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:29.573 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:36:29 smithi204 systemd[1]: Starting Ceph mds.l for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:36:29.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:29 smithi083 ceph-mon[31864]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:29.866 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:29 smithi204 ceph-mon[36739]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:29.866 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:36:29 smithi204 podman[67260]: 2024-03-11 19:36:29.51801674 +0000 UTC m=+0.017895978 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:36:29.866 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:36:29 smithi204 podman[67260]: 2024-03-11 19:36:29.634913071 +0000 UTC m=+0.134792333 container create 582f1dd65852207c6a876dadd34ef3c2b479e8f7ffaa88b4bcedaff4973e5082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-l, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0) 2024-03-11T19:36:29.866 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:36:29 smithi204 podman[67260]: 2024-03-11 19:36:29.862379403 +0000 UTC m=+0.362258619 container init 582f1dd65852207c6a876dadd34ef3c2b479e8f7ffaa88b4bcedaff4973e5082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-l, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-03-11T19:36:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:29 smithi137 ceph-mon[36220]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:30.156 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:36:29 smithi204 podman[67260]: 2024-03-11 19:36:29.864974458 +0000 UTC m=+0.364853679 container start 582f1dd65852207c6a876dadd34ef3c2b479e8f7ffaa88b4bcedaff4973e5082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-l, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-11T19:36:30.156 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:36:29 smithi204 ceph-mds[67280]: -- 172.21.15.204:0/3780611159 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ff88bbe1a0 con 0x55ff88b94c00 2024-03-11T19:36:30.156 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:36:29 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-l[67276]: starting mds.l at 2024-03-11T19:36:30.156 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:36:29 smithi204 bash[67260]: 582f1dd65852207c6a876dadd34ef3c2b479e8f7ffaa88b4bcedaff4973e5082 2024-03-11T19:36:30.156 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:36:29 smithi204 systemd[1]: Started Ceph mds.l for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:36:30.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:30.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.668+0000 7fed62815640 1 -- 172.21.15.83:0/3792499690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed5c1011a0 msgr2=0x7fed5c1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:30.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.668+0000 7fed62815640 1 --2- 172.21.15.83:0/3792499690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed5c1011a0 0x7fed5c1015c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009900 tx=0x7fed4c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:30.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.668+0000 7fed62815640 1 -- 172.21.15.83:0/3792499690 shutdown_connections 2024-03-11T19:36:30.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.668+0000 7fed62815640 1 --2- 172.21.15.83:0/3792499690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed5c102d50 0x7fed5c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:30.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.668+0000 7fed62815640 1 --2- 172.21.15.83:0/3792499690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c102390 0x7fed5c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:30.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.668+0000 7fed62815640 1 --2- 172.21.15.83:0/3792499690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed5c1011a0 0x7fed5c1015c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:30.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.668+0000 7fed62815640 1 -- 172.21.15.83:0/3792499690 >> 172.21.15.83:0/3792499690 conn(0x7fed5c0fc730 msgr2=0x7fed5c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.668+0000 7fed62815640 1 -- 172.21.15.83:0/3792499690 shutdown_connections 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.668+0000 7fed62815640 1 -- 172.21.15.83:0/3792499690 wait complete. 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed62815640 1 Processor -- start 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed62815640 1 -- start start 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed62815640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed5c1011a0 0x7fed5c074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed62815640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c102390 0x7fed5c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed62815640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed5c102d50 0x7fed5c073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed62815640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c074210 con 0x7fed5c1011a0 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed62815640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c074380 con 0x7fed5c102d50 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed62815640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c1a12c0 con 0x7fed5c102390 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed5bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed5c1011a0 0x7fed5c074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:30.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed5bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed5c1011a0 0x7fed5c074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42364/0 (socket says 172.21.15.83:42364) 2024-03-11T19:36:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed5bfff640 1 -- 172.21.15.83:0/1879810755 learned_addr learned my addr 172.21.15.83:0/1879810755 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed5b7fe640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c102390 0x7fed5c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed60db2640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed5c102d50 0x7fed5c073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed5bfff640 1 -- 172.21.15.83:0/1879810755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c102390 msgr2=0x7fed5c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed5bfff640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c102390 0x7fed5c073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed5bfff640 1 -- 172.21.15.83:0/1879810755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed5c102d50 msgr2=0x7fed5c073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed5bfff640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed5c102d50 0x7fed5c073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed5bfff640 1 -- 172.21.15.83:0/1879810755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c009560 con 0x7fed5c1011a0 2024-03-11T19:36:30.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.669+0000 7fed5b7fe640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c102390 0x7fed5c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:36:30.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.670+0000 7fed5bfff640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed5c1011a0 0x7fed5c074f30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fed4c036040 tx=0x7fed4c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:30.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.670+0000 7fed597fa640 1 -- 172.21.15.83:0/1879810755 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c004df0 con 0x7fed5c1011a0 2024-03-11T19:36:30.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.670+0000 7fed597fa640 1 -- 172.21.15.83:0/1879810755 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed4c005430 con 0x7fed5c1011a0 2024-03-11T19:36:30.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.670+0000 7fed62815640 1 -- 172.21.15.83:0/1879810755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5c1a1510 con 0x7fed5c1011a0 2024-03-11T19:36:30.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.670+0000 7fed597fa640 1 -- 172.21.15.83:0/1879810755 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c03e450 con 0x7fed5c1011a0 2024-03-11T19:36:30.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.670+0000 7fed62815640 1 -- 172.21.15.83:0/1879810755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5c1a1a20 con 0x7fed5c1011a0 2024-03-11T19:36:30.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.671+0000 7fed62815640 1 -- 172.21.15.83:0/1879810755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5c1084c0 con 0x7fed5c1011a0 2024-03-11T19:36:30.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.672+0000 7fed597fa640 1 -- 172.21.15.83:0/1879810755 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fed4c003680 con 0x7fed5c1011a0 2024-03-11T19:36:30.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.672+0000 7fed597fa640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fed440a1a30 0x7fed440a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:30.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.673+0000 7fed5b7fe640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fed440a1a30 0x7fed440a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:30.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.673+0000 7fed597fa640 1 -- 172.21.15.83:0/1879810755 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fed4c0efae0 con 0x7fed5c1011a0 2024-03-11T19:36:30.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.673+0000 7fed5b7fe640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fed440a1a30 0x7fed440a3ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed500058c0 tx=0x7fed50005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:30.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.676+0000 7fed597fa640 1 -- 172.21.15.83:0/1879810755 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed4c0b9190 con 0x7fed5c1011a0 2024-03-11T19:36:30.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.885+0000 7fed62815640 1 -- 172.21.15.83:0/1879810755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed5c10cbb0 con 0x7fed5c1011a0 2024-03-11T19:36:30.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.886+0000 7fed597fa640 1 -- 172.21.15.83:0/1879810755 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fed4c0bc850 con 0x7fed5c1011a0 2024-03-11T19:36:30.891 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:30.891 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-11T19:36:30.892 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-03-11T19:36:30.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 -- 172.21.15.83:0/1879810755 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fed440a1a30 msgr2=0x7fed440a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:30.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fed440a1a30 0x7fed440a3ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed500058c0 tx=0x7fed50005850 comp rx=0 tx=0).stop 2024-03-11T19:36:30.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 -- 172.21.15.83:0/1879810755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed5c1011a0 msgr2=0x7fed5c074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:30.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed5c1011a0 0x7fed5c074f30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fed4c036040 tx=0x7fed4c004270 comp rx=0 tx=0).stop 2024-03-11T19:36:30.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 -- 172.21.15.83:0/1879810755 shutdown_connections 2024-03-11T19:36:30.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fed440a1a30 0x7fed440a3ef0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:30.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed5c102d50 0x7fed5c073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c102390 0x7fed5c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 --2- 172.21.15.83:0/1879810755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed5c1011a0 0x7fed5c074f30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 -- 172.21.15.83:0/1879810755 >> 172.21.15.83:0/1879810755 conn(0x7fed5c0fc730 msgr2=0x7fed5c105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 -- 172.21.15.83:0/1879810755 shutdown_connections 2024-03-11T19:36:30.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:30.888+0000 7fed62815640 1 -- 172.21.15.83:0/1879810755 wait complete. 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.204:6832/1242693918,v1:172.21.15.204:6833/1242693918] up:boot 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: fsmap 1 up:standby 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: Deploying daemon mds.k on smithi137 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:30 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1879810755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:31.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.204:6832/1242693918,v1:172.21.15.204:6833/1242693918] up:boot 2024-03-11T19:36:31.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: fsmap 1 up:standby 2024-03-11T19:36:31.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:36:31.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:31.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:31.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:31.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:31.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:31.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:31.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: Deploying daemon mds.k on smithi137 2024-03-11T19:36:31.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:31.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:30 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1879810755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.204:6832/1242693918,v1:172.21.15.204:6833/1242693918] up:boot 2024-03-11T19:36:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: fsmap 1 up:standby 2024-03-11T19:36:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-11T19:36:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:31.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:31.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:31.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:31.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:31.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:31.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: Deploying daemon mds.k on smithi137 2024-03-11T19:36:31.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:31.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:30 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1879810755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:32.390 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:36:32 smithi137 systemd[1]: Starting Ceph mds.k for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:36:32.697 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:36:32 smithi137 podman[65860]: 2024-03-11 19:36:32.300650652 +0000 UTC m=+0.017896908 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:36:32.697 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:36:32 smithi137 podman[65860]: 2024-03-11 19:36:32.399238917 +0000 UTC m=+0.116485164 container create c2a41a73f12acdb7a94039b2b32c37474fe894207df67113ed09e70e720e1d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-k, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0) 2024-03-11T19:36:32.697 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:36:32 smithi137 podman[65860]: 2024-03-11 19:36:32.610413304 +0000 UTC m=+0.327659561 container init c2a41a73f12acdb7a94039b2b32c37474fe894207df67113ed09e70e720e1d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-11T19:36:32.698 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:36:32 smithi137 podman[65860]: 2024-03-11 19:36:32.613213372 +0000 UTC m=+0.330459630 container start c2a41a73f12acdb7a94039b2b32c37474fe894207df67113ed09e70e720e1d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2) 2024-03-11T19:36:32.698 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:36:32 smithi137 ceph-mds[65879]: -- 172.21.15.137:0/3817860109 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563e3f7e21a0 con 0x563e3f7b9000 2024-03-11T19:36:32.698 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:36:32 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-k[65875]: starting mds.k at 2024-03-11T19:36:32.698 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:36:32 smithi137 bash[65860]: c2a41a73f12acdb7a94039b2b32c37474fe894207df67113ed09e70e720e1d96 2024-03-11T19:36:32.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:33.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.006+0000 7fd626464640 1 -- 172.21.15.83:0/184824378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6201000a0 msgr2=0x7fd6201004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.006+0000 7fd626464640 1 --2- 172.21.15.83:0/184824378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6201000a0 0x7fd6201004c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd614009900 tx=0x7fd61402bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.006+0000 7fd626464640 1 -- 172.21.15.83:0/184824378 shutdown_connections 2024-03-11T19:36:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.006+0000 7fd626464640 1 --2- 172.21.15.83:0/184824378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd620101c50 0x7fd6201084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.006+0000 7fd626464640 1 --2- 172.21.15.83:0/184824378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd620101290 0x7fd620101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.006+0000 7fd626464640 1 --2- 172.21.15.83:0/184824378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6201000a0 0x7fd6201004c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.006+0000 7fd626464640 1 -- 172.21.15.83:0/184824378 >> 172.21.15.83:0/184824378 conn(0x7fd6200fb610 msgr2=0x7fd6200fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.006+0000 7fd626464640 1 -- 172.21.15.83:0/184824378 shutdown_connections 2024-03-11T19:36:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.006+0000 7fd626464640 1 -- 172.21.15.83:0/184824378 wait complete. 2024-03-11T19:36:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.006+0000 7fd626464640 1 Processor -- start 2024-03-11T19:36:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.006+0000 7fd626464640 1 -- start start 2024-03-11T19:36:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd626464640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6201000a0 0x7fd6201961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd626464640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd620101290 0x7fd6201966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd626464640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd620101c50 0x7fd62019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd626464640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62019dc90 con 0x7fd620101290 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd626464640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62019de00 con 0x7fd6201000a0 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd626464640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62019dfc0 con 0x7fd620101c50 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd620101290 0x7fd6201966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd620101290 0x7fd6201966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42382/0 (socket says 172.21.15.83:42382) 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61f7fe640 1 -- 172.21.15.83:0/808587880 learned_addr learned my addr 172.21.15.83:0/808587880 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61ffff640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6201000a0 0x7fd6201961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd624a01640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd620101c50 0x7fd62019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61f7fe640 1 -- 172.21.15.83:0/808587880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd620101c50 msgr2=0x7fd62019d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:33.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61f7fe640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd620101c50 0x7fd62019d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:33.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61f7fe640 1 -- 172.21.15.83:0/808587880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6201000a0 msgr2=0x7fd6201961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:33.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61f7fe640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6201000a0 0x7fd6201961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:33.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61f7fe640 1 -- 172.21.15.83:0/808587880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd614009560 con 0x7fd620101290 2024-03-11T19:36:33.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd624a01640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd620101c50 0x7fd62019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:36:33.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61f7fe640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd620101290 0x7fd6201966e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d770 tx=0x7fd61000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:33.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61d7fa640 1 -- 172.21.15.83:0/808587880 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610009c30 con 0x7fd620101290 2024-03-11T19:36:33.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61d7fa640 1 -- 172.21.15.83:0/808587880 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd61000de80 con 0x7fd620101290 2024-03-11T19:36:33.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd626464640 1 -- 172.21.15.83:0/808587880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62019e2a0 con 0x7fd620101290 2024-03-11T19:36:33.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd61d7fa640 1 -- 172.21.15.83:0/808587880 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610004e80 con 0x7fd620101290 2024-03-11T19:36:33.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.007+0000 7fd626464640 1 -- 172.21.15.83:0/808587880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62019e760 con 0x7fd620101290 2024-03-11T19:36:33.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.008+0000 7fd626464640 1 -- 172.21.15.83:0/808587880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd620107440 con 0x7fd620101290 2024-03-11T19:36:33.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.009+0000 7fd61d7fa640 1 -- 172.21.15.83:0/808587880 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd610005000 con 0x7fd620101290 2024-03-11T19:36:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.010+0000 7fd61d7fa640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6080a1a30 0x7fd6080a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.010+0000 7fd61ffff640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6080a1a30 0x7fd6080a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.010+0000 7fd61d7fa640 1 -- 172.21.15.83:0/808587880 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd6100cce10 con 0x7fd620101290 2024-03-11T19:36:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.010+0000 7fd61ffff640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6080a1a30 0x7fd6080a3ef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd614005ab0 tx=0x7fd614005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:33.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.018+0000 7fd61d7fa640 1 -- 172.21.15.83:0/808587880 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd6100964c0 con 0x7fd620101290 2024-03-11T19:36:33.139 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:36:32 smithi137 systemd[1]: Started Ceph mds.k for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:36:33.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.237+0000 7fd626464640 1 -- 172.21.15.83:0/808587880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd62010bab0 con 0x7fd620101290 2024-03-11T19:36:33.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.238+0000 7fd61d7fa640 1 -- 172.21.15.83:0/808587880 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fd610017090 con 0x7fd620101290 2024-03-11T19:36:33.243 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:33.243 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-11T19:36:33.243 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-03-11T19:36:33.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 -- 172.21.15.83:0/808587880 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6080a1a30 msgr2=0x7fd6080a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:33.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6080a1a30 0x7fd6080a3ef0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd614005ab0 tx=0x7fd614005fd0 comp rx=0 tx=0).stop 2024-03-11T19:36:33.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 -- 172.21.15.83:0/808587880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd620101290 msgr2=0x7fd6201966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:33.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd620101290 0x7fd6201966e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d770 tx=0x7fd61000dc30 comp rx=0 tx=0).stop 2024-03-11T19:36:33.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 -- 172.21.15.83:0/808587880 shutdown_connections 2024-03-11T19:36:33.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6080a1a30 0x7fd6080a3ef0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:33.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd620101c50 0x7fd62019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:33.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd620101290 0x7fd6201966e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:33.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 --2- 172.21.15.83:0/808587880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6201000a0 0x7fd6201961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:33.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 -- 172.21.15.83:0/808587880 >> 172.21.15.83:0/808587880 conn(0x7fd6200fb610 msgr2=0x7fd620104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:33.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 -- 172.21.15.83:0/808587880 shutdown_connections 2024-03-11T19:36:33.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:33.240+0000 7fd626464640 1 -- 172.21.15.83:0/808587880 wait complete. 2024-03-11T19:36:33.561 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:33 smithi083 ceph-mon[31864]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:33.561 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:33.562 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:33.562 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:33.562 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:33.562 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:33.562 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:33.562 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:33 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/808587880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:33.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:33 smithi137 ceph-mon[36220]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:33.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:33.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:33.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:33.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:33.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:33.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:33.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:33 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/808587880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:33 smithi204 ceph-mon[36739]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:33.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:33.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:33 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/808587880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:34.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:34 smithi083 ceph-mon[31864]: Deploying daemon mds.j on smithi083 2024-03-11T19:36:34.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:34 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.137:6832/1480144108,v1:172.21.15.137:6833/1480144108] up:boot 2024-03-11T19:36:34.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:34 smithi083 ceph-mon[31864]: fsmap 2 up:standby 2024-03-11T19:36:34.736 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:34 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:36:34.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:34 smithi137 ceph-mon[36220]: Deploying daemon mds.j on smithi083 2024-03-11T19:36:34.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:34 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.137:6832/1480144108,v1:172.21.15.137:6833/1480144108] up:boot 2024-03-11T19:36:34.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:34 smithi137 ceph-mon[36220]: fsmap 2 up:standby 2024-03-11T19:36:34.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:34 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:36:34.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:34 smithi204 ceph-mon[36739]: Deploying daemon mds.j on smithi083 2024-03-11T19:36:34.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:34 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.137:6832/1480144108,v1:172.21.15.137:6833/1480144108] up:boot 2024-03-11T19:36:34.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:34 smithi204 ceph-mon[36739]: fsmap 2 up:standby 2024-03-11T19:36:34.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:34 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-11T19:36:35.068 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:36:34 smithi083 systemd[1]: Starting Ceph mds.j for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:36:35.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:35.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.360+0000 7fe91c4c0640 1 -- 172.21.15.83:0/4061403573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9140ff490 msgr2=0x7fe9140ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:35.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.360+0000 7fe91c4c0640 1 --2- 172.21.15.83:0/4061403573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9140ff490 0x7fe9140ff8b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe910009900 tx=0x7fe91002bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 -- 172.21.15.83:0/4061403573 shutdown_connections 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 --2- 172.21.15.83:0/4061403573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914102e20 0x7fe9141096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 --2- 172.21.15.83:0/4061403573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe914102460 0x7fe9141028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 --2- 172.21.15.83:0/4061403573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9140ff490 0x7fe9140ff8b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 -- 172.21.15.83:0/4061403573 >> 172.21.15.83:0/4061403573 conn(0x7fe9140fadc0 msgr2=0x7fe9140fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 -- 172.21.15.83:0/4061403573 shutdown_connections 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 -- 172.21.15.83:0/4061403573 wait complete. 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 Processor -- start 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 -- start start 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9140ff490 0x7fe914197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914102460 0x7fe9141977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914102e20 0x7fe91419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91419ed80 con 0x7fe914102e20 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91419eef0 con 0x7fe9140ff490 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91c4c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91419f0b0 con 0x7fe914102460 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91aa5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914102e20 0x7fe91419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91aa5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914102e20 0x7fe91419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42408/0 (socket says 172.21.15.83:42408) 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91aa5d640 1 -- 172.21.15.83:0/158716628 learned_addr learned my addr 172.21.15.83:0/158716628 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.361+0000 7fe91a25c640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9140ff490 0x7fe914197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe919a5b640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914102460 0x7fe9141977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe91aa5d640 1 -- 172.21.15.83:0/158716628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914102460 msgr2=0x7fe9141977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe91aa5d640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914102460 0x7fe9141977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe91aa5d640 1 -- 172.21.15.83:0/158716628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9140ff490 msgr2=0x7fe914197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe91aa5d640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9140ff490 0x7fe914197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:35.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe91aa5d640 1 -- 172.21.15.83:0/158716628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe910009560 con 0x7fe914102e20 2024-03-11T19:36:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe91aa5d640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914102e20 0x7fe91419e840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00cad0 tx=0x7fe90c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe90b7fe640 1 -- 172.21.15.83:0/158716628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c00ae20 con 0x7fe914102e20 2024-03-11T19:36:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe90b7fe640 1 -- 172.21.15.83:0/158716628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe90c005020 con 0x7fe914102e20 2024-03-11T19:36:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe91c4c0640 1 -- 172.21.15.83:0/158716628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91419f390 con 0x7fe914102e20 2024-03-11T19:36:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe90b7fe640 1 -- 172.21.15.83:0/158716628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c00f6b0 con 0x7fe914102e20 2024-03-11T19:36:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.362+0000 7fe91c4c0640 1 -- 172.21.15.83:0/158716628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91419f8f0 con 0x7fe914102e20 2024-03-11T19:36:35.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.363+0000 7fe91c4c0640 1 -- 172.21.15.83:0/158716628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe914108590 con 0x7fe914102e20 2024-03-11T19:36:35.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.364+0000 7fe90b7fe640 1 -- 172.21.15.83:0/158716628 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe90c005520 con 0x7fe914102e20 2024-03-11T19:36:35.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.364+0000 7fe90b7fe640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe9000a1ab0 0x7fe9000a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:35.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.364+0000 7fe91a25c640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe9000a1ab0 0x7fe9000a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:35.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.365+0000 7fe90b7fe640 1 -- 172.21.15.83:0/158716628 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe90c0cd280 con 0x7fe914102e20 2024-03-11T19:36:35.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.365+0000 7fe91a25c640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe9000a1ab0 0x7fe9000a3f70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe91002c240 tx=0x7fe91002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.369+0000 7fe90b7fe640 1 -- 172.21.15.83:0/158716628 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe90c096930 con 0x7fe914102e20 2024-03-11T19:36:35.384 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:36:35 smithi083 podman[69713]: 2024-03-11 19:36:35.021995052 +0000 UTC m=+0.017307724 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:36:35.385 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:36:35 smithi083 podman[69713]: 2024-03-11 19:36:35.120418902 +0000 UTC m=+0.115731565 container create f6dc6086f4c5784d592032a70e8905e82960bec6016f6ceef2d3ab9f2b10c9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:36:35.385 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:36:35 smithi083 podman[69713]: 2024-03-11 19:36:35.381394532 +0000 UTC m=+0.376707190 container init f6dc6086f4c5784d592032a70e8905e82960bec6016f6ceef2d3ab9f2b10c9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-11T19:36:35.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.591+0000 7fe91c4c0640 1 -- 172.21.15.83:0/158716628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe91402ce40 con 0x7fe914102e20 2024-03-11T19:36:35.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.592+0000 7fe90b7fe640 1 -- 172.21.15.83:0/158716628 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7fe90c083e60 con 0x7fe914102e20 2024-03-11T19:36:35.597 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:35.597 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-11T19:36:35.597 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 4 2024-03-11T19:36:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 -- 172.21.15.83:0/158716628 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe9000a1ab0 msgr2=0x7fe9000a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe9000a1ab0 0x7fe9000a3f70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe91002c240 tx=0x7fe91002c7e0 comp rx=0 tx=0).stop 2024-03-11T19:36:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 -- 172.21.15.83:0/158716628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914102e20 msgr2=0x7fe91419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914102e20 0x7fe91419e840 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00cad0 tx=0x7fe90c00cf90 comp rx=0 tx=0).stop 2024-03-11T19:36:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 -- 172.21.15.83:0/158716628 shutdown_connections 2024-03-11T19:36:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe9000a1ab0 0x7fe9000a3f70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe914102e20 0x7fe91419e840 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe914102460 0x7fe9141977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 --2- 172.21.15.83:0/158716628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9140ff490 0x7fe914197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 -- 172.21.15.83:0/158716628 >> 172.21.15.83:0/158716628 conn(0x7fe9140fadc0 msgr2=0x7fe914105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 -- 172.21.15.83:0/158716628 shutdown_connections 2024-03-11T19:36:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:35.594+0000 7fe91c4c0640 1 -- 172.21.15.83:0/158716628 wait complete. 2024-03-11T19:36:35.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:35 smithi083 ceph-mon[31864]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:35.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:35 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:35.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:35 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:35.818 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:36:35 smithi083 podman[69713]: 2024-03-11 19:36:35.384704819 +0000 UTC m=+0.380017484 container start f6dc6086f4c5784d592032a70e8905e82960bec6016f6ceef2d3ab9f2b10c9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-11T19:36:35.818 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:36:35 smithi083 ceph-mds[69772]: -- 172.21.15.83:0/3233979135 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fa257dc1a0 con 0x55fa257b2c00 2024-03-11T19:36:35.818 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:36:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-j[69729]: starting mds.j at 2024-03-11T19:36:35.819 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:36:35 smithi083 bash[69713]: f6dc6086f4c5784d592032a70e8905e82960bec6016f6ceef2d3ab9f2b10c9b2 2024-03-11T19:36:35.819 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:36:35 smithi083 systemd[1]: Started Ceph mds.j for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:36:35.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:35 smithi137 ceph-mon[36220]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:35.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:35 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:35.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:35 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:35.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:35 smithi204 ceph-mon[36739]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:35.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:35 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:35.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:35 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:36.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:36 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.83:6834/4213430163,v1:172.21.15.83:6835/4213430163] up:boot 2024-03-11T19:36:36.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:36 smithi083 ceph-mon[31864]: fsmap 3 up:standby 2024-03-11T19:36:36.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:36:36.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:36.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:36.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:36.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:36.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:36 smithi083 ceph-mon[31864]: Deploying daemon mds.d on smithi083 2024-03-11T19:36:36.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:36 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/158716628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:36.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:36 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.83:6834/4213430163,v1:172.21.15.83:6835/4213430163] up:boot 2024-03-11T19:36:36.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:36 smithi137 ceph-mon[36220]: fsmap 3 up:standby 2024-03-11T19:36:36.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:36:36.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:36.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:36.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:36.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:36.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:36 smithi137 ceph-mon[36220]: Deploying daemon mds.d on smithi083 2024-03-11T19:36:36.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:36 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/158716628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:36.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:36 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.83:6834/4213430163,v1:172.21.15.83:6835/4213430163] up:boot 2024-03-11T19:36:36.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:36 smithi204 ceph-mon[36739]: fsmap 3 up:standby 2024-03-11T19:36:36.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-11T19:36:36.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:36.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:36.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:36.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:36.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:36 smithi204 ceph-mon[36739]: Deploying daemon mds.d on smithi083 2024-03-11T19:36:36.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:36 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/158716628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:37.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:37.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.729+0000 7ff553c23640 1 -- 172.21.15.83:0/1854540270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c102390 msgr2=0x7ff54c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:37.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.729+0000 7ff553c23640 1 --2- 172.21.15.83:0/1854540270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c102390 0x7ff54c102810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff534009ca0 tx=0x7ff53402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:36:37.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.729+0000 7ff553c23640 1 -- 172.21.15.83:0/1854540270 shutdown_connections 2024-03-11T19:36:37.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.729+0000 7ff553c23640 1 --2- 172.21.15.83:0/1854540270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff54c102d50 0x7ff54c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:37.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.729+0000 7ff553c23640 1 --2- 172.21.15.83:0/1854540270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c102390 0x7ff54c102810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:37.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.729+0000 7ff553c23640 1 --2- 172.21.15.83:0/1854540270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c1011a0 0x7ff54c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:37.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.729+0000 7ff553c23640 1 -- 172.21.15.83:0/1854540270 >> 172.21.15.83:0/1854540270 conn(0x7ff54c0fc730 msgr2=0x7ff54c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:37.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.729+0000 7ff553c23640 1 -- 172.21.15.83:0/1854540270 shutdown_connections 2024-03-11T19:36:37.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.729+0000 7ff553c23640 1 -- 172.21.15.83:0/1854540270 wait complete. 2024-03-11T19:36:37.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff553c23640 1 Processor -- start 2024-03-11T19:36:37.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff553c23640 1 -- start start 2024-03-11T19:36:37.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff553c23640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff54c1011a0 0x7ff54c197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:37.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff553c23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c102390 0x7ff54c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:37.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff553c23640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c102d50 0x7ff54c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:37.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff553c23640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19ed10 con 0x7ff54c102390 2024-03-11T19:36:37.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff553c23640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19ee80 con 0x7ff54c1011a0 2024-03-11T19:36:37.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff553c23640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19f040 con 0x7ff54c102d50 2024-03-11T19:36:37.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff5511be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c102390 0x7ff54c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:37.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff5519bf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff54c1011a0 0x7ff54c197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:37.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff5511be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c102390 0x7ff54c197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42446/0 (socket says 172.21.15.83:42446) 2024-03-11T19:36:37.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff5511be640 1 -- 172.21.15.83:0/2125678184 learned_addr learned my addr 172.21.15.83:0/2125678184 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:37.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff5521c0640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c102d50 0x7ff54c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:37.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff5511be640 1 -- 172.21.15.83:0/2125678184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c102d50 msgr2=0x7ff54c19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:37.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff5511be640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c102d50 0x7ff54c19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:37.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff5511be640 1 -- 172.21.15.83:0/2125678184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff54c1011a0 msgr2=0x7ff54c197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:37.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff5511be640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff54c1011a0 0x7ff54c197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:37.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff5511be640 1 -- 172.21.15.83:0/2125678184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff534009900 con 0x7ff54c102390 2024-03-11T19:36:37.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.730+0000 7ff5511be640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c102390 0x7ff54c197760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff534032710 tx=0x7ff534034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:37.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.731+0000 7ff54affd640 1 -- 172.21.15.83:0/2125678184 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534032e40 con 0x7ff54c102390 2024-03-11T19:36:37.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.731+0000 7ff54affd640 1 -- 172.21.15.83:0/2125678184 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff534003e50 con 0x7ff54c102390 2024-03-11T19:36:37.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.731+0000 7ff553c23640 1 -- 172.21.15.83:0/2125678184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c19f2c0 con 0x7ff54c102390 2024-03-11T19:36:37.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.731+0000 7ff54affd640 1 -- 172.21.15.83:0/2125678184 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534040450 con 0x7ff54c102390 2024-03-11T19:36:37.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.731+0000 7ff553c23640 1 -- 172.21.15.83:0/2125678184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c19f740 con 0x7ff54c102390 2024-03-11T19:36:37.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.732+0000 7ff553c23640 1 -- 172.21.15.83:0/2125678184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c108540 con 0x7ff54c102390 2024-03-11T19:36:37.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.733+0000 7ff54affd640 1 -- 172.21.15.83:0/2125678184 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff53400a040 con 0x7ff54c102390 2024-03-11T19:36:37.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.733+0000 7ff54affd640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff5380a1a30 0x7ff5380a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:37.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.734+0000 7ff5519bf640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff5380a1a30 0x7ff5380a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:37.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.734+0000 7ff54affd640 1 -- 172.21.15.83:0/2125678184 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5340efda0 con 0x7ff54c102390 2024-03-11T19:36:37.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.734+0000 7ff5519bf640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff5380a1a30 0x7ff5380a3ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff54c1021f0 tx=0x7ff53c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:37.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.741+0000 7ff54affd640 1 -- 172.21.15.83:0/2125678184 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff5340b9510 con 0x7ff54c102390 2024-03-11T19:36:37.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:37 smithi083 ceph-mon[31864]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:37.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:37 smithi137 ceph-mon[36220]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:37 smithi204 ceph-mon[36739]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:37.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.964+0000 7ff553c23640 1 -- 172.21.15.83:0/2125678184 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff54c10ce00 con 0x7ff54c102390 2024-03-11T19:36:37.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.964+0000 7ff54affd640 1 -- 172.21.15.83:0/2125678184 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7ff534004160 con 0x7ff54c102390 2024-03-11T19:36:37.968 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:37.968 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-11T19:36:37.969 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 4 2024-03-11T19:36:37.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 -- 172.21.15.83:0/2125678184 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff5380a1a30 msgr2=0x7ff5380a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:37.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff5380a1a30 0x7ff5380a3ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff54c1021f0 tx=0x7ff53c005f90 comp rx=0 tx=0).stop 2024-03-11T19:36:37.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 -- 172.21.15.83:0/2125678184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c102390 msgr2=0x7ff54c197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:37.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c102390 0x7ff54c197760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff534032710 tx=0x7ff534034cc0 comp rx=0 tx=0).stop 2024-03-11T19:36:37.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 -- 172.21.15.83:0/2125678184 shutdown_connections 2024-03-11T19:36:37.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff5380a1a30 0x7ff5380a3ef0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:37.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff54c102d50 0x7ff54c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:37.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54c102390 0x7ff54c197760 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:37.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 --2- 172.21.15.83:0/2125678184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff54c1011a0 0x7ff54c197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:37.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 -- 172.21.15.83:0/2125678184 >> 172.21.15.83:0/2125678184 conn(0x7ff54c0fc730 msgr2=0x7ff54c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:37.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 -- 172.21.15.83:0/2125678184 shutdown_connections 2024-03-11T19:36:37.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:37.966+0000 7ff553c23640 1 -- 172.21.15.83:0/2125678184 wait complete. 2024-03-11T19:36:38.568 INFO:journalctl@ceph.mds.d.smithi083.stdout:Mar 11 19:36:38 smithi083 systemd[1]: Started Ceph mds.d for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:36:38.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:38 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2125678184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:38.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:38.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:38.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:38.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:38.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:38.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:38.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:38 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2125678184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:38.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:38.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:38.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:38.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:38.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:38.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:38 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2125678184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:39.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:39.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:39.861 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:39 smithi137 ceph-mon[36220]: Deploying daemon mds.e on smithi137 2024-03-11T19:36:39.862 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:39 smithi137 ceph-mon[36220]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:39.862 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:39 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.83:6836/1285695998,v1:172.21.15.83:6837/1285695998] up:boot 2024-03-11T19:36:39.862 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:39 smithi137 ceph-mon[36220]: fsmap 4 up:standby 2024-03-11T19:36:39.862 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:36:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:39 smithi204 ceph-mon[36739]: Deploying daemon mds.e on smithi137 2024-03-11T19:36:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:39 smithi204 ceph-mon[36739]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:39 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.83:6836/1285695998,v1:172.21.15.83:6837/1285695998] up:boot 2024-03-11T19:36:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:39 smithi204 ceph-mon[36739]: fsmap 4 up:standby 2024-03-11T19:36:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:36:39.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:40.012 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:39 smithi083 ceph-mon[31864]: Deploying daemon mds.e on smithi137 2024-03-11T19:36:40.012 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:39 smithi083 ceph-mon[31864]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:40.012 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:39 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.83:6836/1285695998,v1:172.21.15.83:6837/1285695998] up:boot 2024-03-11T19:36:40.012 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:39 smithi083 ceph-mon[31864]: fsmap 4 up:standby 2024-03-11T19:36:40.013 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-11T19:36:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.093+0000 7f83ab39a640 1 -- 172.21.15.83:0/1108046612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a41000a0 msgr2=0x7f83a41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.093+0000 7f83ab39a640 1 --2- 172.21.15.83:0/1108046612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a41000a0 0x7f83a41004c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8394009900 tx=0x7f839402bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.093+0000 7f83ab39a640 1 -- 172.21.15.83:0/1108046612 shutdown_connections 2024-03-11T19:36:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.093+0000 7f83ab39a640 1 --2- 172.21.15.83:0/1108046612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83a4101c50 0x7f83a41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.093+0000 7f83ab39a640 1 --2- 172.21.15.83:0/1108046612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a4101290 0x7f83a4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.093+0000 7f83ab39a640 1 --2- 172.21.15.83:0/1108046612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a41000a0 0x7f83a41004c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.093+0000 7f83ab39a640 1 -- 172.21.15.83:0/1108046612 >> 172.21.15.83:0/1108046612 conn(0x7f83a40fb610 msgr2=0x7f83a40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.093+0000 7f83ab39a640 1 -- 172.21.15.83:0/1108046612 shutdown_connections 2024-03-11T19:36:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.093+0000 7f83ab39a640 1 -- 172.21.15.83:0/1108046612 wait complete. 2024-03-11T19:36:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83ab39a640 1 Processor -- start 2024-03-11T19:36:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83ab39a640 1 -- start start 2024-03-11T19:36:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83ab39a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a41000a0 0x7f83a4197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83ab39a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83a4101290 0x7f83a4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83ab39a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a4101c50 0x7f83a419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83ab39a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a419ed00 con 0x7f83a41000a0 2024-03-11T19:36:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83ab39a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a419ee70 con 0x7f83a4101c50 2024-03-11T19:36:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83ab39a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a419f030 con 0x7f83a4101290 2024-03-11T19:36:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83a9136640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a41000a0 0x7f83a4197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83a9136640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a41000a0 0x7f83a4197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33356/0 (socket says 172.21.15.83:33356) 2024-03-11T19:36:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83a9136640 1 -- 172.21.15.83:0/2019520327 learned_addr learned my addr 172.21.15.83:0/2019520327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83a9937640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a4101c50 0x7f83a419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83a8935640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83a4101290 0x7f83a4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83a9136640 1 -- 172.21.15.83:0/2019520327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83a4101290 msgr2=0x7f83a4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83a9136640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83a4101290 0x7f83a4197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83a9136640 1 -- 172.21.15.83:0/2019520327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a4101c50 msgr2=0x7f83a419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83a9136640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a4101c50 0x7f83a419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83a9136640 1 -- 172.21.15.83:0/2019520327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8394009560 con 0x7f83a41000a0 2024-03-11T19:36:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f83a9136640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a41000a0 0x7f83a4197210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8394005ab0 tx=0x7f8394004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.094+0000 7f839a7fc640 1 -- 172.21.15.83:0/2019520327 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394004aa0 con 0x7f83a41000a0 2024-03-11T19:36:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.095+0000 7f839a7fc640 1 -- 172.21.15.83:0/2019520327 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83940052d0 con 0x7f83a41000a0 2024-03-11T19:36:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.095+0000 7f83ab39a640 1 -- 172.21.15.83:0/2019520327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a419f2b0 con 0x7f83a41000a0 2024-03-11T19:36:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.095+0000 7f839a7fc640 1 -- 172.21.15.83:0/2019520327 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394003e80 con 0x7f83a41000a0 2024-03-11T19:36:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.095+0000 7f83ab39a640 1 -- 172.21.15.83:0/2019520327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a419f7c0 con 0x7f83a41000a0 2024-03-11T19:36:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.096+0000 7f83ab39a640 1 -- 172.21.15.83:0/2019520327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a410ba70 con 0x7f83a41000a0 2024-03-11T19:36:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.097+0000 7f839a7fc640 1 -- 172.21.15.83:0/2019520327 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8394005440 con 0x7f83a41000a0 2024-03-11T19:36:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.097+0000 7f839a7fc640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f83900a1a30 0x7f83900a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.097+0000 7f83a8935640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f83900a1a30 0x7f83900a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.097+0000 7f839a7fc640 1 -- 172.21.15.83:0/2019520327 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f83940ef460 con 0x7f83a41000a0 2024-03-11T19:36:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.098+0000 7f83a8935640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f83900a1a30 0x7f83900a3ef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f83a4198720 tx=0x7f83a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.102+0000 7f839a7fc640 1 -- 172.21.15.83:0/2019520327 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f83940b8b10 con 0x7f83a41000a0 2024-03-11T19:36:40.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.311+0000 7f83ab39a640 1 -- 172.21.15.83:0/2019520327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83a410bc50 con 0x7f83a41000a0 2024-03-11T19:36:40.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.311+0000 7f839a7fc640 1 -- 172.21.15.83:0/2019520327 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f8394043e90 con 0x7f83a41000a0 2024-03-11T19:36:40.315 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:40.315 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-11T19:36:40.316 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 5 2024-03-11T19:36:40.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 -- 172.21.15.83:0/2019520327 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f83900a1a30 msgr2=0x7f83900a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:40.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f83900a1a30 0x7f83900a3ef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f83a4198720 tx=0x7f83a0005f90 comp rx=0 tx=0).stop 2024-03-11T19:36:40.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 -- 172.21.15.83:0/2019520327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a41000a0 msgr2=0x7f83a4197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:40.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a41000a0 0x7f83a4197210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8394005ab0 tx=0x7f8394004270 comp rx=0 tx=0).stop 2024-03-11T19:36:40.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 -- 172.21.15.83:0/2019520327 shutdown_connections 2024-03-11T19:36:40.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f83900a1a30 0x7f83900a3ef0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:40.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83a4101c50 0x7f83a419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:40.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83a4101290 0x7f83a4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:40.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 --2- 172.21.15.83:0/2019520327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83a41000a0 0x7f83a4197210 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:40.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 -- 172.21.15.83:0/2019520327 >> 172.21.15.83:0/2019520327 conn(0x7f83a40fb610 msgr2=0x7f83a40697f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:40.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 -- 172.21.15.83:0/2019520327 shutdown_connections 2024-03-11T19:36:40.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:40.313+0000 7f83ab39a640 1 -- 172.21.15.83:0/2019520327 wait complete. 2024-03-11T19:36:40.560 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:36:40 smithi137 systemd[1]: Starting Ceph mds.e for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:36:40.815 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:40 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2019520327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:40.816 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:36:40 smithi137 podman[66182]: 2024-03-11 19:36:40.459176853 +0000 UTC m=+0.017757940 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:36:40.816 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:36:40 smithi137 podman[66182]: 2024-03-11 19:36:40.568232171 +0000 UTC m=+0.126813242 container create 3c3a611e31be5c47f60daca5d7f357df9f867964d8e85243c0e933fbfa443372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True) 2024-03-11T19:36:40.816 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:36:40 smithi137 podman[66182]: 2024-03-11 19:36:40.787141133 +0000 UTC m=+0.345722212 container init 3c3a611e31be5c47f60daca5d7f357df9f867964d8e85243c0e933fbfa443372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:36:40.816 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:36:40 smithi137 podman[66182]: 2024-03-11 19:36:40.789630696 +0000 UTC m=+0.348211784 container start 3c3a611e31be5c47f60daca5d7f357df9f867964d8e85243c0e933fbfa443372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-11T19:36:40.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:40 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2019520327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:41.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:40 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2019520327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:41.140 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:36:40 smithi137 ceph-mds[66201]: -- 172.21.15.137:0/3612266035 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5603f6bc61a0 con 0x5603f6b9d000 2024-03-11T19:36:41.140 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:36:40 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e[66197]: starting mds.e at 2024-03-11T19:36:41.140 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:36:40 smithi137 bash[66182]: 3c3a611e31be5c47f60daca5d7f357df9f867964d8e85243c0e933fbfa443372 2024-03-11T19:36:41.140 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:36:40 smithi137 systemd[1]: Started Ceph mds.e for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:36:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:41 smithi137 ceph-mon[36220]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:41.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:41 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:41.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:41 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:41.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:41 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:41.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:41 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:41.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:41 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:41.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:41 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:41 smithi204 ceph-mon[36739]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:41 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:41 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:41 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:41 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:41 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:41.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:41 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:41 smithi083 ceph-mon[31864]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:42.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:42.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:42.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:42.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:42.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:41 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:42.329 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.439+0000 7fc4b5972640 1 -- 172.21.15.83:0/2394354116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b0101c50 msgr2=0x7fc4b01084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.439+0000 7fc4b5972640 1 --2- 172.21.15.83:0/2394354116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b0101c50 0x7fc4b01084e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009900 tx=0x7fc4a802bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.440+0000 7fc4b5972640 1 -- 172.21.15.83:0/2394354116 shutdown_connections 2024-03-11T19:36:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.440+0000 7fc4b5972640 1 --2- 172.21.15.83:0/2394354116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b0101c50 0x7fc4b01084e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.440+0000 7fc4b5972640 1 --2- 172.21.15.83:0/2394354116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0101290 0x7fc4b0101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.440+0000 7fc4b5972640 1 --2- 172.21.15.83:0/2394354116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4b01000a0 0x7fc4b01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.440+0000 7fc4b5972640 1 -- 172.21.15.83:0/2394354116 >> 172.21.15.83:0/2394354116 conn(0x7fc4b00fb610 msgr2=0x7fc4b00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.440+0000 7fc4b5972640 1 -- 172.21.15.83:0/2394354116 shutdown_connections 2024-03-11T19:36:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.440+0000 7fc4b5972640 1 -- 172.21.15.83:0/2394354116 wait complete. 2024-03-11T19:36:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.440+0000 7fc4b5972640 1 Processor -- start 2024-03-11T19:36:42.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.440+0000 7fc4b5972640 1 -- start start 2024-03-11T19:36:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.440+0000 7fc4b5972640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4b01000a0 0x7fc4b01971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4b5972640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b0101290 0x7fc4b0197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4b5972640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0101c50 0x7fc4b019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4b5972640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b019ecd0 con 0x7fc4b0101290 2024-03-11T19:36:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4b5972640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b019ee40 con 0x7fc4b01000a0 2024-03-11T19:36:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4b5972640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b019f000 con 0x7fc4b0101c50 2024-03-11T19:36:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4ae7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b0101290 0x7fc4b0197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:42.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4ae7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b0101290 0x7fc4b0197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33382/0 (socket says 172.21.15.83:33382) 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4ae7fc640 1 -- 172.21.15.83:0/1627735320 learned_addr learned my addr 172.21.15.83:0/1627735320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4af7fe640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0101c50 0x7fc4b019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4aeffd640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4b01000a0 0x7fc4b01971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4ae7fc640 1 -- 172.21.15.83:0/1627735320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0101c50 msgr2=0x7fc4b019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4ae7fc640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0101c50 0x7fc4b019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4ae7fc640 1 -- 172.21.15.83:0/1627735320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4b01000a0 msgr2=0x7fc4b01971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4ae7fc640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4b01000a0 0x7fc4b01971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4ae7fc640 1 -- 172.21.15.83:0/1627735320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a8009560 con 0x7fc4b0101290 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4aeffd640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4b01000a0 0x7fc4b01971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4ae7fc640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b0101290 0x7fc4b0197720 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400d950 tx=0x7fc4a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4b4970640 1 -- 172.21.15.83:0/1627735320 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4009a30 con 0x7fc4b0101290 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4b4970640 1 -- 172.21.15.83:0/1627735320 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4a4009b90 con 0x7fc4b0101290 2024-03-11T19:36:42.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4b5972640 1 -- 172.21.15.83:0/1627735320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b019f2e0 con 0x7fc4b0101290 2024-03-11T19:36:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4b4970640 1 -- 172.21.15.83:0/1627735320 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4004a00 con 0x7fc4b0101290 2024-03-11T19:36:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.441+0000 7fc4b5972640 1 -- 172.21.15.83:0/1627735320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b019f7f0 con 0x7fc4b0101290 2024-03-11T19:36:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.442+0000 7fc4b5972640 1 -- 172.21.15.83:0/1627735320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b0107360 con 0x7fc4b0101290 2024-03-11T19:36:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.443+0000 7fc4b4970640 1 -- 172.21.15.83:0/1627735320 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc4a4009d00 con 0x7fc4b0101290 2024-03-11T19:36:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.444+0000 7fc4b4970640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc4980a1a30 0x7fc4980a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:42.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.444+0000 7fc4aeffd640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc4980a1a30 0x7fc4980a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:42.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.444+0000 7fc4b4970640 1 -- 172.21.15.83:0/1627735320 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc4a40cc990 con 0x7fc4b0101290 2024-03-11T19:36:42.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.444+0000 7fc4aeffd640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc4980a1a30 0x7fc4980a3ef0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc4b01010f0 tx=0x7fc4a0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:42.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.448+0000 7fc4b4970640 1 -- 172.21.15.83:0/1627735320 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc4a4096040 con 0x7fc4b0101290 2024-03-11T19:36:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.660+0000 7fc4b5972640 1 -- 172.21.15.83:0/1627735320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4b010cbb0 con 0x7fc4b0101290 2024-03-11T19:36:42.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.660+0000 7fc4b4970640 1 -- 172.21.15.83:0/1627735320 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4498 (secure 0 0 0) 0x7fc4a4099700 con 0x7fc4b0101290 2024-03-11T19:36:42.664 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:42.664 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-11T19:36:42.665 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-03-11T19:36:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 -- 172.21.15.83:0/1627735320 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc4980a1a30 msgr2=0x7fc4980a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc4980a1a30 0x7fc4980a3ef0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc4b01010f0 tx=0x7fc4a0009500 comp rx=0 tx=0).stop 2024-03-11T19:36:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 -- 172.21.15.83:0/1627735320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b0101290 msgr2=0x7fc4b0197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b0101290 0x7fc4b0197720 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400d950 tx=0x7fc4a400de10 comp rx=0 tx=0).stop 2024-03-11T19:36:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 -- 172.21.15.83:0/1627735320 shutdown_connections 2024-03-11T19:36:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc4980a1a30 0x7fc4980a3ef0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:42.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4b0101c50 0x7fc4b019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:42.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b0101290 0x7fc4b0197720 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:42.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 --2- 172.21.15.83:0/1627735320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4b01000a0 0x7fc4b01971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:42.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 -- 172.21.15.83:0/1627735320 >> 172.21.15.83:0/1627735320 conn(0x7fc4b00fb610 msgr2=0x7fc4b0104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:42.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 -- 172.21.15.83:0/1627735320 shutdown_connections 2024-03-11T19:36:42.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:42.662+0000 7fc4b5972640 1 -- 172.21.15.83:0/1627735320 wait complete. 2024-03-11T19:36:42.676 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:42 smithi083 ceph-mon[31864]: Deploying daemon mds.f on smithi204 2024-03-11T19:36:42.676 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:42 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.137:6834/1924725912,v1:172.21.15.137:6835/1924725912] up:boot 2024-03-11T19:36:42.676 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:42 smithi083 ceph-mon[31864]: fsmap 5 up:standby 2024-03-11T19:36:42.676 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:42 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:36:42.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:42 smithi204 ceph-mon[36739]: Deploying daemon mds.f on smithi204 2024-03-11T19:36:42.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:42 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.137:6834/1924725912,v1:172.21.15.137:6835/1924725912] up:boot 2024-03-11T19:36:42.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:42 smithi204 ceph-mon[36739]: fsmap 5 up:standby 2024-03-11T19:36:42.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:42 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:36:42.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:42 smithi137 ceph-mon[36220]: Deploying daemon mds.f on smithi204 2024-03-11T19:36:42.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:42 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.137:6834/1924725912,v1:172.21.15.137:6835/1924725912] up:boot 2024-03-11T19:36:42.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:42 smithi137 ceph-mon[36220]: fsmap 5 up:standby 2024-03-11T19:36:42.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:42 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:36:43.156 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:36:42 smithi204 systemd[1]: Starting Ceph mds.f for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:36:43.471 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:36:43 smithi204 podman[67570]: 2024-03-11 19:36:43.204289088 +0000 UTC m=+0.110905924 container create 055abf3673c8bc86cc20f21c0f02fdb4db373b050e1045dc6ec518aedef48b95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-11T19:36:43.471 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:36:43 smithi204 podman[67570]: 2024-03-11 19:36:43.111615437 +0000 UTC m=+0.018232284 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:36:43.471 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:36:43 smithi204 podman[67570]: 2024-03-11 19:36:43.441816545 +0000 UTC m=+0.348433378 container init 055abf3673c8bc86cc20f21c0f02fdb4db373b050e1045dc6ec518aedef48b95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-11T19:36:43.472 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:36:43 smithi204 podman[67570]: 2024-03-11 19:36:43.44453803 +0000 UTC m=+0.351154878 container start 055abf3673c8bc86cc20f21c0f02fdb4db373b050e1045dc6ec518aedef48b95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:36:43.472 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:36:43 smithi204 ceph-mds[67589]: -- 172.21.15.204:0/189310043 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5619c69881a0 con 0x5619c695ec00 2024-03-11T19:36:43.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:43 smithi137 ceph-mon[36220]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:43.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:43 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1627735320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:43.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:43 smithi204 ceph-mon[36739]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:43.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:43 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1627735320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:43.906 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:36:43 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f[67585]: starting mds.f at 2024-03-11T19:36:43.906 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:36:43 smithi204 bash[67570]: 055abf3673c8bc86cc20f21c0f02fdb4db373b050e1045dc6ec518aedef48b95 2024-03-11T19:36:43.906 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:36:43 smithi204 systemd[1]: Started Ceph mds.f for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:36:44.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:43 smithi083 ceph-mon[31864]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:44.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:43 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1627735320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:44.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.788+0000 7eff1ccf1640 1 -- 172.21.15.83:0/1884079254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff18101c50 msgr2=0x7eff181084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.788+0000 7eff1ccf1640 1 --2- 172.21.15.83:0/1884079254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff18101c50 0x7eff181084e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7eff08009900 tx=0x7eff0802bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.788+0000 7eff1ccf1640 1 -- 172.21.15.83:0/1884079254 shutdown_connections 2024-03-11T19:36:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.788+0000 7eff1ccf1640 1 --2- 172.21.15.83:0/1884079254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff18101c50 0x7eff181084e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.788+0000 7eff1ccf1640 1 --2- 172.21.15.83:0/1884079254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff18101290 0x7eff18101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.788+0000 7eff1ccf1640 1 --2- 172.21.15.83:0/1884079254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff181000a0 0x7eff181004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.788+0000 7eff1ccf1640 1 -- 172.21.15.83:0/1884079254 >> 172.21.15.83:0/1884079254 conn(0x7eff180fb610 msgr2=0x7eff180fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.788+0000 7eff1ccf1640 1 -- 172.21.15.83:0/1884079254 shutdown_connections 2024-03-11T19:36:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.788+0000 7eff1ccf1640 1 -- 172.21.15.83:0/1884079254 wait complete. 2024-03-11T19:36:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.788+0000 7eff1ccf1640 1 Processor -- start 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.788+0000 7eff1ccf1640 1 -- start start 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff1ccf1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff181000a0 0x7eff18104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff1ccf1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff18101290 0x7eff18102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff1ccf1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff18101c50 0x7eff18102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff1ccf1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff18103310 con 0x7eff18101290 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff1ccf1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff18103480 con 0x7eff18101c50 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff1ccf1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181a1220 con 0x7eff181000a0 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff15d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff18101290 0x7eff18102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff15d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff18101290 0x7eff18102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33406/0 (socket says 172.21.15.83:33406) 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff15d9b640 1 -- 172.21.15.83:0/389066783 learned_addr learned my addr 172.21.15.83:0/389066783 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff16d9d640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff18101c50 0x7eff18102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff1659c640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff181000a0 0x7eff18104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff15d9b640 1 -- 172.21.15.83:0/389066783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff181000a0 msgr2=0x7eff18104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff15d9b640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff181000a0 0x7eff18104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff15d9b640 1 -- 172.21.15.83:0/389066783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff18101c50 msgr2=0x7eff18102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff15d9b640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff18101c50 0x7eff18102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff15d9b640 1 -- 172.21.15.83:0/389066783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff08009560 con 0x7eff18101290 2024-03-11T19:36:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff15d9b640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff18101290 0x7eff18102670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00eab0 tx=0x7eff0c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff077fe640 1 -- 172.21.15.83:0/389066783 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00ce40 con 0x7eff18101290 2024-03-11T19:36:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff077fe640 1 -- 172.21.15.83:0/389066783 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff0c005020 con 0x7eff18101290 2024-03-11T19:36:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff1ccf1640 1 -- 172.21.15.83:0/389066783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff181a14d0 con 0x7eff18101290 2024-03-11T19:36:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff077fe640 1 -- 172.21.15.83:0/389066783 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0116b0 con 0x7eff18101290 2024-03-11T19:36:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.789+0000 7eff1ccf1640 1 -- 172.21.15.83:0/389066783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff181a1990 con 0x7eff18101290 2024-03-11T19:36:44.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.790+0000 7eff1ccf1640 1 -- 172.21.15.83:0/389066783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1810bab0 con 0x7eff18101290 2024-03-11T19:36:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.792+0000 7eff077fe640 1 -- 172.21.15.83:0/389066783 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7eff0c005520 con 0x7eff18101290 2024-03-11T19:36:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.792+0000 7eff077fe640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff000a1a30 0x7eff000a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.792+0000 7eff1659c640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff000a1a30 0x7eff000a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.792+0000 7eff077fe640 1 -- 172.21.15.83:0/389066783 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7eff0c0cd630 con 0x7eff18101290 2024-03-11T19:36:44.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.793+0000 7eff1659c640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff000a1a30 0x7eff000a3ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff100058c0 tx=0x7eff10005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:44.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:44.796+0000 7eff077fe640 1 -- 172.21.15.83:0/389066783 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff0c096ce0 con 0x7eff18101290 2024-03-11T19:36:44.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:44 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.204:6834/2604685946,v1:172.21.15.204:6835/2604685946] up:boot 2024-03-11T19:36:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:44 smithi137 ceph-mon[36220]: fsmap 6 up:standby 2024-03-11T19:36:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:36:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:44 smithi137 ceph-mon[36220]: Deploying daemon mds.c on smithi204 2024-03-11T19:36:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:44 smithi137 ceph-mon[36220]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:44.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:44.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:44 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.204:6834/2604685946,v1:172.21.15.204:6835/2604685946] up:boot 2024-03-11T19:36:44.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:44 smithi204 ceph-mon[36739]: fsmap 6 up:standby 2024-03-11T19:36:44.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:36:44.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:44.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:44.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:44.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:44.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:44.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:44 smithi204 ceph-mon[36739]: Deploying daemon mds.c on smithi204 2024-03-11T19:36:44.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:44 smithi204 ceph-mon[36739]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:45.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.007+0000 7eff1ccf1640 1 -- 172.21.15.83:0/389066783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff181a1c10 con 0x7eff18101290 2024-03-11T19:36:45.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.007+0000 7eff077fe640 1 -- 172.21.15.83:0/389066783 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7eff0c09a3a0 con 0x7eff18101290 2024-03-11T19:36:45.012 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:45.012 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-11T19:36:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-03-11T19:36:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.009+0000 7eff1ccf1640 1 -- 172.21.15.83:0/389066783 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff000a1a30 msgr2=0x7eff000a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.009+0000 7eff1ccf1640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff000a1a30 0x7eff000a3ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff100058c0 tx=0x7eff10005850 comp rx=0 tx=0).stop 2024-03-11T19:36:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.009+0000 7eff1ccf1640 1 -- 172.21.15.83:0/389066783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff18101290 msgr2=0x7eff18102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.009+0000 7eff1ccf1640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff18101290 0x7eff18102670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00eab0 tx=0x7eff0c00ef70 comp rx=0 tx=0).stop 2024-03-11T19:36:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.009+0000 7eff1ccf1640 1 -- 172.21.15.83:0/389066783 shutdown_connections 2024-03-11T19:36:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.009+0000 7eff1ccf1640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff000a1a30 0x7eff000a3ef0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.009+0000 7eff1ccf1640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff18101c50 0x7eff18102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.009+0000 7eff1ccf1640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff18101290 0x7eff18102670 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.009+0000 7eff1ccf1640 1 --2- 172.21.15.83:0/389066783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff181000a0 0x7eff18104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.010+0000 7eff1ccf1640 1 -- 172.21.15.83:0/389066783 >> 172.21.15.83:0/389066783 conn(0x7eff180fb610 msgr2=0x7eff180fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:45.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.010+0000 7eff1ccf1640 1 -- 172.21.15.83:0/389066783 shutdown_connections 2024-03-11T19:36:45.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:45.010+0000 7eff1ccf1640 1 -- 172.21.15.83:0/389066783 wait complete. 2024-03-11T19:36:45.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:45.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:44 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.204:6834/2604685946,v1:172.21.15.204:6835/2604685946] up:boot 2024-03-11T19:36:45.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:44 smithi083 ceph-mon[31864]: fsmap 6 up:standby 2024-03-11T19:36:45.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:36:45.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:45.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:45.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:45.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:45.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:45.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:44 smithi083 ceph-mon[31864]: Deploying daemon mds.c on smithi204 2024-03-11T19:36:45.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:44 smithi083 ceph-mon[31864]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:45.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:45 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/389066783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:45 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/389066783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:45.906 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:36:45 smithi204 systemd[1]: Starting Ceph mds.c for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:36:46.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:45 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/389066783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:46.259 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:36:45 smithi204 podman[67879]: 2024-03-11 19:36:45.882819314 +0000 UTC m=+0.017926075 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:36:46.259 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:36:46 smithi204 podman[67879]: 2024-03-11 19:36:46.00801658 +0000 UTC m=+0.143123335 container create 2ad65201b1a866be0434652ed74e91b1e110caff903de84f5381aaf2a0ccb5fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-11T19:36:46.259 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:36:46 smithi204 podman[67879]: 2024-03-11 19:36:46.227385316 +0000 UTC m=+0.362492082 container init 2ad65201b1a866be0434652ed74e91b1e110caff903de84f5381aaf2a0ccb5fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-11T19:36:46.259 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:36:46 smithi204 podman[67879]: 2024-03-11 19:36:46.229993515 +0000 UTC m=+0.365100277 container start 2ad65201b1a866be0434652ed74e91b1e110caff903de84f5381aaf2a0ccb5fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-11T19:36:46.259 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:36:46 smithi204 ceph-mds[67898]: -- 172.21.15.204:0/1328649457 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aeff7fc1a0 con 0x55aeff7d2c00 2024-03-11T19:36:46.656 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:36:46 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c[67894]: starting mds.c at 2024-03-11T19:36:46.656 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:36:46 smithi204 bash[67879]: 2ad65201b1a866be0434652ed74e91b1e110caff903de84f5381aaf2a0ccb5fd 2024-03-11T19:36:46.656 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:36:46 smithi204 systemd[1]: Started Ceph mds.c for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:36:47.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:47.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.134+0000 7facf514f640 1 -- 172.21.15.83:0/1299278070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facf0102390 msgr2=0x7facf0102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:47.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.134+0000 7facf514f640 1 --2- 172.21.15.83:0/1299278070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facf0102390 0x7facf0102810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7face4009900 tx=0x7face402bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:47.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.134+0000 7facf514f640 1 -- 172.21.15.83:0/1299278070 shutdown_connections 2024-03-11T19:36:47.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.134+0000 7facf514f640 1 --2- 172.21.15.83:0/1299278070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf0102d50 0x7facf01095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.134+0000 7facf514f640 1 --2- 172.21.15.83:0/1299278070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facf0102390 0x7facf0102810 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.134+0000 7facf514f640 1 --2- 172.21.15.83:0/1299278070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facf01011a0 0x7facf01015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.134+0000 7facf514f640 1 -- 172.21.15.83:0/1299278070 >> 172.21.15.83:0/1299278070 conn(0x7facf00fc730 msgr2=0x7facf00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.134+0000 7facf514f640 1 -- 172.21.15.83:0/1299278070 shutdown_connections 2024-03-11T19:36:47.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.134+0000 7facf514f640 1 -- 172.21.15.83:0/1299278070 wait complete. 2024-03-11T19:36:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facf514f640 1 Processor -- start 2024-03-11T19:36:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facf514f640 1 -- start start 2024-03-11T19:36:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facf514f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facf01011a0 0x7facf0197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facf514f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facf0102390 0x7facf0197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facf514f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf0102d50 0x7facf019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facf514f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf019ed00 con 0x7facf0102390 2024-03-11T19:36:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facf514f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf019ee70 con 0x7facf01011a0 2024-03-11T19:36:47.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facf514f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf019f030 con 0x7facf0102d50 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facee59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facf0102390 0x7facf0197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facee59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facf0102390 0x7facf0197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33422/0 (socket says 172.21.15.83:33422) 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facee59c640 1 -- 172.21.15.83:0/2694542999 learned_addr learned my addr 172.21.15.83:0/2694542999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7faceed9d640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facf01011a0 0x7facf0197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.135+0000 7facef59e640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf0102d50 0x7facf019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7facee59c640 1 -- 172.21.15.83:0/2694542999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf0102d50 msgr2=0x7facf019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7facee59c640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf0102d50 0x7facf019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7facee59c640 1 -- 172.21.15.83:0/2694542999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facf01011a0 msgr2=0x7facf0197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7facee59c640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facf01011a0 0x7facf0197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7facee59c640 1 -- 172.21.15.83:0/2694542999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face4009560 con 0x7facf0102390 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7faceed9d640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facf01011a0 0x7facf0197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:36:47.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7facee59c640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facf0102390 0x7facf0197750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7face4006010 tx=0x7face40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:47.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7facd7fff640 1 -- 172.21.15.83:0/2694542999 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40044c0 con 0x7facf0102390 2024-03-11T19:36:47.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7facd7fff640 1 -- 172.21.15.83:0/2694542999 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7face40052d0 con 0x7facf0102390 2024-03-11T19:36:47.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7facf514f640 1 -- 172.21.15.83:0/2694542999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf019f2b0 con 0x7facf0102390 2024-03-11T19:36:47.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7facd7fff640 1 -- 172.21.15.83:0/2694542999 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4003e10 con 0x7facf0102390 2024-03-11T19:36:47.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.136+0000 7facf514f640 1 -- 172.21.15.83:0/2694542999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf019f7c0 con 0x7facf0102390 2024-03-11T19:36:47.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.137+0000 7facf514f640 1 -- 172.21.15.83:0/2694542999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf01084c0 con 0x7facf0102390 2024-03-11T19:36:47.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.138+0000 7facd7fff640 1 -- 172.21.15.83:0/2694542999 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7face4005440 con 0x7facf0102390 2024-03-11T19:36:47.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.139+0000 7facd7fff640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7facd80a1a30 0x7facd80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:47.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.139+0000 7faceed9d640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7facd80a1a30 0x7facd80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:47.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.139+0000 7facd7fff640 1 -- 172.21.15.83:0/2694542999 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7face40ef320 con 0x7facf0102390 2024-03-11T19:36:47.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.139+0000 7faceed9d640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7facd80a1a30 0x7facd80a3ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7face00058c0 tx=0x7face0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:47.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.143+0000 7facd7fff640 1 -- 172.21.15.83:0/2694542999 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7face40b89d0 con 0x7facf0102390 2024-03-11T19:36:47.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.357+0000 7facf514f640 1 -- 172.21.15.83:0/2694542999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facf010ce00 con 0x7facf0102390 2024-03-11T19:36:47.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.358+0000 7facd7fff640 1 -- 172.21.15.83:0/2694542999 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6086 (secure 0 0 0) 0x7face40bc090 con 0x7facf0102390 2024-03-11T19:36:47.362 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:47.362 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-11T19:36:47.362 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2024-03-11T19:36:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 -- 172.21.15.83:0/2694542999 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7facd80a1a30 msgr2=0x7facd80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7facd80a1a30 0x7facd80a3ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7face00058c0 tx=0x7face0005850 comp rx=0 tx=0).stop 2024-03-11T19:36:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 -- 172.21.15.83:0/2694542999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facf0102390 msgr2=0x7facf0197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facf0102390 0x7facf0197750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7face4006010 tx=0x7face40040c0 comp rx=0 tx=0).stop 2024-03-11T19:36:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 -- 172.21.15.83:0/2694542999 shutdown_connections 2024-03-11T19:36:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7facd80a1a30 0x7facd80a3ef0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facf0102d50 0x7facf019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facf0102390 0x7facf0197750 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 --2- 172.21.15.83:0/2694542999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facf01011a0 0x7facf0197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 -- 172.21.15.83:0/2694542999 >> 172.21.15.83:0/2694542999 conn(0x7facf00fc730 msgr2=0x7facf0105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 -- 172.21.15.83:0/2694542999 shutdown_connections 2024-03-11T19:36:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:47.361+0000 7facf514f640 1 -- 172.21.15.83:0/2694542999 wait complete. 2024-03-11T19:36:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:47 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:47 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:47 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:47 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:47 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:47 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:47 smithi137 ceph-mon[36220]: Deploying daemon mds.h on smithi137 2024-03-11T19:36:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:47 smithi137 ceph-mon[36220]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:47 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.204:6836/2654992874,v1:172.21.15.204:6837/2654992874] up:boot 2024-03-11T19:36:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:47 smithi137 ceph-mon[36220]: fsmap 7 up:standby 2024-03-11T19:36:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:47 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:36:47.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:47 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:47.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:47 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:47.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:47 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:47.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:47 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:47.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:47 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:47.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:47 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:47.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:47 smithi204 ceph-mon[36739]: Deploying daemon mds.h on smithi137 2024-03-11T19:36:47.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:47 smithi204 ceph-mon[36739]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:47.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:47 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.204:6836/2654992874,v1:172.21.15.204:6837/2654992874] up:boot 2024-03-11T19:36:47.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:47 smithi204 ceph-mon[36739]: fsmap 7 up:standby 2024-03-11T19:36:47.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:47 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:36:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:47 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:47 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:47 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:47 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:47 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:47 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:47 smithi083 ceph-mon[31864]: Deploying daemon mds.h on smithi137 2024-03-11T19:36:47.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:47 smithi083 ceph-mon[31864]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:47.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:47 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.204:6836/2654992874,v1:172.21.15.204:6837/2654992874] up:boot 2024-03-11T19:36:47.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:47 smithi083 ceph-mon[31864]: fsmap 7 up:standby 2024-03-11T19:36:47.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:47 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:36:48.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:48 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2694542999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:48.640 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:36:48 smithi137 systemd[1]: Starting Ceph mds.h for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:36:48.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:48 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2694542999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:48.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:48 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2694542999' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:49.112 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:36:48 smithi137 podman[66490]: 2024-03-11 19:36:48.628244069 +0000 UTC m=+0.017945088 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:36:49.113 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:36:48 smithi137 podman[66490]: 2024-03-11 19:36:48.746451694 +0000 UTC m=+0.136152749 container create 047e588e571b84798d5df649d2611a1f54231238252004ce0713fd349725b088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-11T19:36:49.371 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:36:49 smithi137 podman[66490]: 2024-03-11 19:36:49.109470148 +0000 UTC m=+0.499171163 container init 047e588e571b84798d5df649d2611a1f54231238252004ce0713fd349725b088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-03-11T19:36:49.371 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:36:49 smithi137 podman[66490]: 2024-03-11 19:36:49.112246669 +0000 UTC m=+0.501947677 container start 047e588e571b84798d5df649d2611a1f54231238252004ce0713fd349725b088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-11T19:36:49.371 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:36:49 smithi137 ceph-mds[66509]: -- 172.21.15.137:0/2753819035 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55650ff541a0 con 0x55650ff2b000 2024-03-11T19:36:49.371 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:36:49 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h[66505]: starting mds.h at 2024-03-11T19:36:49.371 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:36:49 smithi137 bash[66490]: 047e588e571b84798d5df649d2611a1f54231238252004ce0713fd349725b088 2024-03-11T19:36:49.371 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:36:49 smithi137 systemd[1]: Started Ceph mds.h for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:36:49.378 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.483+0000 7f9df8356640 1 -- 172.21.15.83:0/955245508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df0101c50 msgr2=0x7f9df01084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.483+0000 7f9df8356640 1 --2- 172.21.15.83:0/955245508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df0101c50 0x7f9df01084e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009ca0 tx=0x7f9de002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:36:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.483+0000 7f9df8356640 1 -- 172.21.15.83:0/955245508 shutdown_connections 2024-03-11T19:36:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.483+0000 7f9df8356640 1 --2- 172.21.15.83:0/955245508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df0101c50 0x7f9df01084e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.483+0000 7f9df8356640 1 --2- 172.21.15.83:0/955245508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9df0101290 0x7f9df0101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.483+0000 7f9df8356640 1 --2- 172.21.15.83:0/955245508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df01000a0 0x7f9df01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.483+0000 7f9df8356640 1 -- 172.21.15.83:0/955245508 >> 172.21.15.83:0/955245508 conn(0x7f9df00fb610 msgr2=0x7f9df00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.483+0000 7f9df8356640 1 -- 172.21.15.83:0/955245508 shutdown_connections 2024-03-11T19:36:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.483+0000 7f9df8356640 1 -- 172.21.15.83:0/955245508 wait complete. 2024-03-11T19:36:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.483+0000 7f9df8356640 1 Processor -- start 2024-03-11T19:36:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df8356640 1 -- start start 2024-03-11T19:36:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df8356640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df01000a0 0x7f9df0197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df8356640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df0101290 0x7f9df0197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df8356640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9df0101c50 0x7f9df019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df8356640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df019ec40 con 0x7f9df0101290 2024-03-11T19:36:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df8356640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df019edb0 con 0x7f9df0101c50 2024-03-11T19:36:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df8356640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df019ef70 con 0x7f9df01000a0 2024-03-11T19:36:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df58f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df0101290 0x7f9df0197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df58f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df0101290 0x7f9df0197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33434/0 (socket says 172.21.15.83:33434) 2024-03-11T19:36:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df58f1640 1 -- 172.21.15.83:0/3298620028 learned_addr learned my addr 172.21.15.83:0/3298620028 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df60f2640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df01000a0 0x7f9df0197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df68f3640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9df0101c50 0x7f9df019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df58f1640 1 -- 172.21.15.83:0/3298620028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df01000a0 msgr2=0x7f9df0197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df58f1640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df01000a0 0x7f9df0197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df58f1640 1 -- 172.21.15.83:0/3298620028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9df0101c50 msgr2=0x7f9df019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df58f1640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9df0101c50 0x7f9df019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df58f1640 1 -- 172.21.15.83:0/3298620028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de0009900 con 0x7f9df0101290 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df58f1640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df0101290 0x7f9df0197690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c920 tx=0x7f9de400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9def7fe640 1 -- 172.21.15.83:0/3298620028 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4007c70 con 0x7f9df0101290 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9df8356640 1 -- 172.21.15.83:0/3298620028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df019f250 con 0x7f9df0101290 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9def7fe640 1 -- 172.21.15.83:0/3298620028 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9de4007dd0 con 0x7f9df0101290 2024-03-11T19:36:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.484+0000 7f9def7fe640 1 -- 172.21.15.83:0/3298620028 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4018860 con 0x7f9df0101290 2024-03-11T19:36:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.485+0000 7f9df8356640 1 -- 172.21.15.83:0/3298620028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df019f760 con 0x7f9df0101290 2024-03-11T19:36:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.485+0000 7f9df8356640 1 -- 172.21.15.83:0/3298620028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df010bab0 con 0x7f9df0101290 2024-03-11T19:36:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.487+0000 7f9def7fe640 1 -- 172.21.15.83:0/3298620028 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9de400fac0 con 0x7f9df0101290 2024-03-11T19:36:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.487+0000 7f9def7fe640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9dd80a1a30 0x7f9dd80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.488+0000 7f9df60f2640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9dd80a1a30 0x7f9dd80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.488+0000 7f9def7fe640 1 -- 172.21.15.83:0/3298620028 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9de40cd310 con 0x7f9df0101290 2024-03-11T19:36:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.488+0000 7f9df60f2640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9dd80a1a30 0x7f9dd80a3ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9de8005fd0 tx=0x7f9de800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.491+0000 7f9def7fe640 1 -- 172.21.15.83:0/3298620028 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9de4096a30 con 0x7f9df0101290 2024-03-11T19:36:49.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:49 smithi137 ceph-mon[36220]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:49 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:49 smithi204 ceph-mon[36739]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:49.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:49 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.700+0000 7f9df8356640 1 -- 172.21.15.83:0/3298620028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9df019f9e0 con 0x7f9df0101290 2024-03-11T19:36:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.700+0000 7f9def7fe640 1 -- 172.21.15.83:0/3298620028 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6880 (secure 0 0 0) 0x7f9de409a0f0 con 0x7f9df0101290 2024-03-11T19:36:49.704 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:49.705 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-11T19:36:49.705 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-03-11T19:36:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 -- 172.21.15.83:0/3298620028 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9dd80a1a30 msgr2=0x7f9dd80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9dd80a1a30 0x7f9dd80a3ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9de8005fd0 tx=0x7f9de800b390 comp rx=0 tx=0).stop 2024-03-11T19:36:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 -- 172.21.15.83:0/3298620028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df0101290 msgr2=0x7f9df0197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df0101290 0x7f9df0197690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c920 tx=0x7f9de400cde0 comp rx=0 tx=0).stop 2024-03-11T19:36:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 -- 172.21.15.83:0/3298620028 shutdown_connections 2024-03-11T19:36:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9dd80a1a30 0x7f9dd80a3ef0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9df0101c50 0x7f9df019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9df0101290 0x7f9df0197690 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 --2- 172.21.15.83:0/3298620028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df01000a0 0x7f9df0197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 -- 172.21.15.83:0/3298620028 >> 172.21.15.83:0/3298620028 conn(0x7f9df00fb610 msgr2=0x7f9df00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 -- 172.21.15.83:0/3298620028 shutdown_connections 2024-03-11T19:36:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:49.702+0000 7f9df8356640 1 -- 172.21.15.83:0/3298620028 wait complete. 2024-03-11T19:36:49.716 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:49 smithi083 ceph-mon[31864]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:49.716 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:49.716 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:49.716 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:49.716 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:49.716 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:49.716 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:49 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:50.570 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:50 smithi137 ceph-mon[36220]: Deploying daemon mds.i on smithi204 2024-03-11T19:36:50.571 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:50 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] up:boot 2024-03-11T19:36:50.571 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:50 smithi137 ceph-mon[36220]: fsmap 8 up:standby 2024-03-11T19:36:50.571 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:50 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:36:50.571 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:50 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3298620028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:50.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:50 smithi204 ceph-mon[36739]: Deploying daemon mds.i on smithi204 2024-03-11T19:36:50.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:50 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] up:boot 2024-03-11T19:36:50.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:50 smithi204 ceph-mon[36739]: fsmap 8 up:standby 2024-03-11T19:36:50.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:50 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:36:50.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:50 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3298620028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:50.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:50 smithi083 ceph-mon[31864]: Deploying daemon mds.i on smithi204 2024-03-11T19:36:50.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:50 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] up:boot 2024-03-11T19:36:50.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:50 smithi083 ceph-mon[31864]: fsmap 8 up:standby 2024-03-11T19:36:50.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:50 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:36:50.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:50 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3298620028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:51.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:51 smithi137 ceph-mon[36220]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:51.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:51 smithi204 ceph-mon[36739]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:51.657 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:36:51 smithi204 systemd[1]: Starting Ceph mds.i for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:36:51.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:51.749 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:51 smithi083 ceph-mon[31864]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.830+0000 7f14d529d640 1 -- 172.21.15.83:0/1102053169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0100120 msgr2=0x7f14d0100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.830+0000 7f14d529d640 1 --2- 172.21.15.83:0/1102053169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0100120 0x7f14d0100540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009900 tx=0x7f14c002bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.830+0000 7f14d529d640 1 -- 172.21.15.83:0/1102053169 shutdown_connections 2024-03-11T19:36:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.830+0000 7f14d529d640 1 --2- 172.21.15.83:0/1102053169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0101cd0 0x7f14d0108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.830+0000 7f14d529d640 1 --2- 172.21.15.83:0/1102053169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d0101310 0x7f14d0101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.830+0000 7f14d529d640 1 --2- 172.21.15.83:0/1102053169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0100120 0x7f14d0100540 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.830+0000 7f14d529d640 1 -- 172.21.15.83:0/1102053169 >> 172.21.15.83:0/1102053169 conn(0x7f14d00fb6b0 msgr2=0x7f14d00fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.830+0000 7f14d529d640 1 -- 172.21.15.83:0/1102053169 shutdown_connections 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.830+0000 7f14d529d640 1 -- 172.21.15.83:0/1102053169 wait complete. 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.830+0000 7f14d529d640 1 Processor -- start 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.830+0000 7f14d529d640 1 -- start start 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14d529d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0100120 0x7f14d0197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14d529d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d0101310 0x7f14d01978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14d529d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0101cd0 0x7f14d019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14d529d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d019ee00 con 0x7f14d0100120 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14d529d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d019ef70 con 0x7f14d0101cd0 2024-03-11T19:36:51.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14d529d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d019f130 con 0x7f14d0101310 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ceffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0100120 0x7f14d0197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ceffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0100120 0x7f14d0197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45996/0 (socket says 172.21.15.83:45996) 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ceffd640 1 -- 172.21.15.83:0/2705459802 learned_addr learned my addr 172.21.15.83:0/2705459802 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14c7fff640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d0101310 0x7f14d01978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14cf7fe640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0101cd0 0x7f14d019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ceffd640 1 -- 172.21.15.83:0/2705459802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d0101310 msgr2=0x7f14d01978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ceffd640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d0101310 0x7f14d01978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ceffd640 1 -- 172.21.15.83:0/2705459802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0101cd0 msgr2=0x7f14d019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ceffd640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0101cd0 0x7f14d019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ceffd640 1 -- 172.21.15.83:0/2705459802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c0009560 con 0x7f14d0100120 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ceffd640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0100120 0x7f14d0197380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14c0036040 tx=0x7f14c0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ccff9640 1 -- 172.21.15.83:0/2705459802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0004440 con 0x7f14d0100120 2024-03-11T19:36:51.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ccff9640 1 -- 172.21.15.83:0/2705459802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14c00052d0 con 0x7f14d0100120 2024-03-11T19:36:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14d529d640 1 -- 172.21.15.83:0/2705459802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d019f3b0 con 0x7f14d0100120 2024-03-11T19:36:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14ccff9640 1 -- 172.21.15.83:0/2705459802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0003d80 con 0x7f14d0100120 2024-03-11T19:36:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.831+0000 7f14d529d640 1 -- 172.21.15.83:0/2705459802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d019f8c0 con 0x7f14d0100120 2024-03-11T19:36:51.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.832+0000 7f14d529d640 1 -- 172.21.15.83:0/2705459802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d010bac0 con 0x7f14d0100120 2024-03-11T19:36:51.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.833+0000 7f14ccff9640 1 -- 172.21.15.83:0/2705459802 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f14c0005440 con 0x7f14d0100120 2024-03-11T19:36:51.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.834+0000 7f14ccff9640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14bc0a1a30 0x7f14bc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:51.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.834+0000 7f14c7fff640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14bc0a1a30 0x7f14bc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:51.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.834+0000 7f14ccff9640 1 -- 172.21.15.83:0/2705459802 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f14c00ef6e0 con 0x7f14d0100120 2024-03-11T19:36:51.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.835+0000 7f14c7fff640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14bc0a1a30 0x7f14bc0a3ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f14d0198820 tx=0x7f14b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:51.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:51.838+0000 7f14ccff9640 1 -- 172.21.15.83:0/2705459802 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f14c00b8d90 con 0x7f14d0100120 2024-03-11T19:36:51.967 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:36:51 smithi204 podman[68205]: 2024-03-11 19:36:51.573946848 +0000 UTC m=+0.017844729 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:36:51.968 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:36:51 smithi204 podman[68205]: 2024-03-11 19:36:51.695136893 +0000 UTC m=+0.139034800 container create 9db8ac59f9805054e1da7bc414980ccd77e88cd1a17f7804517e8e3de687169f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397) 2024-03-11T19:36:51.968 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:36:51 smithi204 podman[68205]: 2024-03-11 19:36:51.964195971 +0000 UTC m=+0.408093838 container init 9db8ac59f9805054e1da7bc414980ccd77e88cd1a17f7804517e8e3de687169f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-11T19:36:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.045+0000 7f14d529d640 1 -- 172.21.15.83:0/2705459802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14d010bca0 con 0x7f14d0100120 2024-03-11T19:36:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.045+0000 7f14ccff9640 1 -- 172.21.15.83:0/2705459802 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6880 (secure 0 0 0) 0x7f14c00bc450 con 0x7f14d0100120 2024-03-11T19:36:52.050 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:52.050 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-11T19:36:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-03-11T19:36:52.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.047+0000 7f14d529d640 1 -- 172.21.15.83:0/2705459802 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14bc0a1a30 msgr2=0x7f14bc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.047+0000 7f14d529d640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14bc0a1a30 0x7f14bc0a3ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f14d0198820 tx=0x7f14b8005f90 comp rx=0 tx=0).stop 2024-03-11T19:36:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.047+0000 7f14d529d640 1 -- 172.21.15.83:0/2705459802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0100120 msgr2=0x7f14d0197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.047+0000 7f14d529d640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0100120 0x7f14d0197380 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14c0036040 tx=0x7f14c0004040 comp rx=0 tx=0).stop 2024-03-11T19:36:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.047+0000 7f14d529d640 1 -- 172.21.15.83:0/2705459802 shutdown_connections 2024-03-11T19:36:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.048+0000 7f14d529d640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14bc0a1a30 0x7f14bc0a3ef0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.048+0000 7f14d529d640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14d0101cd0 0x7f14d019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.048+0000 7f14d529d640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14d0101310 0x7f14d01978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.048+0000 7f14d529d640 1 --2- 172.21.15.83:0/2705459802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14d0100120 0x7f14d0197380 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.048+0000 7f14d529d640 1 -- 172.21.15.83:0/2705459802 >> 172.21.15.83:0/2705459802 conn(0x7f14d00fb6b0 msgr2=0x7f14d00fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.048+0000 7f14d529d640 1 -- 172.21.15.83:0/2705459802 shutdown_connections 2024-03-11T19:36:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:52.048+0000 7f14d529d640 1 -- 172.21.15.83:0/2705459802 wait complete. 2024-03-11T19:36:52.380 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:36:51 smithi204 podman[68205]: 2024-03-11 19:36:51.966903159 +0000 UTC m=+0.410801028 container start 9db8ac59f9805054e1da7bc414980ccd77e88cd1a17f7804517e8e3de687169f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-11T19:36:52.380 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:36:51 smithi204 ceph-mds[68224]: -- 172.21.15.204:0/3530504759 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56468ffa41a0 con 0x56468ff7ac00 2024-03-11T19:36:52.380 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:36:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i[68220]: starting mds.i at 2024-03-11T19:36:52.380 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:36:52 smithi204 bash[68205]: 9db8ac59f9805054e1da7bc414980ccd77e88cd1a17f7804517e8e3de687169f 2024-03-11T19:36:52.380 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:36:52 smithi204 systemd[1]: Started Ceph mds.i for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:36:52.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:52 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2705459802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:52.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:52 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:52.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:52 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:52.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:52 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:52.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:52 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:52.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:52 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:52.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:52 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:52.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:52 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2705459802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:52.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:52 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:52.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:52 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:52.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:52 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:52.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:52 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:52.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:52 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:52.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:52 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:52.765 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:52 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2705459802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:52.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:52 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:52.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:52 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:52.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:52 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:52.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:52 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:52.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:52 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:52.766 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:52 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:53.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:53 smithi204 ceph-mon[36739]: Deploying daemon mds.g on smithi083 2024-03-11T19:36:53.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:53 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] up:boot 2024-03-11T19:36:53.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:53 smithi204 ceph-mon[36739]: fsmap 9 up:standby 2024-03-11T19:36:53.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:53 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:36:53.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:53 smithi204 ceph-mon[36739]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:53.769 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:53 smithi083 ceph-mon[31864]: Deploying daemon mds.g on smithi083 2024-03-11T19:36:53.769 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:53 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] up:boot 2024-03-11T19:36:53.769 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:53 smithi083 ceph-mon[31864]: fsmap 9 up:standby 2024-03-11T19:36:53.769 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:53 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:36:53.769 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:53 smithi083 ceph-mon[31864]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:53.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:53 smithi137 ceph-mon[36220]: Deploying daemon mds.g on smithi083 2024-03-11T19:36:53.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:53 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] up:boot 2024-03-11T19:36:53.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:53 smithi137 ceph-mon[36220]: fsmap 9 up:standby 2024-03-11T19:36:53.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:53 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:36:53.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:53 smithi137 ceph-mon[36220]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:54.063 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.175+0000 7f0ec81b5640 1 -- 172.21.15.83:0/3540070912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0101290 msgr2=0x7f0ec0101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.175+0000 7f0ec81b5640 1 --2- 172.21.15.83:0/3540070912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0101290 0x7f0ec0101710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009900 tx=0x7f0ebc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.175+0000 7f0ec81b5640 1 -- 172.21.15.83:0/3540070912 shutdown_connections 2024-03-11T19:36:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.175+0000 7f0ec81b5640 1 --2- 172.21.15.83:0/3540070912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ec0101c50 0x7f0ec01084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.175+0000 7f0ec81b5640 1 --2- 172.21.15.83:0/3540070912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0101290 0x7f0ec0101710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.175+0000 7f0ec81b5640 1 --2- 172.21.15.83:0/3540070912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec01000a0 0x7f0ec01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.175+0000 7f0ec81b5640 1 -- 172.21.15.83:0/3540070912 >> 172.21.15.83:0/3540070912 conn(0x7f0ec00fb610 msgr2=0x7f0ec00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec81b5640 1 -- 172.21.15.83:0/3540070912 shutdown_connections 2024-03-11T19:36:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec81b5640 1 -- 172.21.15.83:0/3540070912 wait complete. 2024-03-11T19:36:54.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec81b5640 1 Processor -- start 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec81b5640 1 -- start start 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec81b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ec01000a0 0x7f0ec0068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec81b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0101290 0x7f0ec019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec81b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101c50 0x7f0ec019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec81b5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec019ed30 con 0x7f0ec0101290 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec81b5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec019eea0 con 0x7f0ec0101c50 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec81b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec019f060 con 0x7f0ec01000a0 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec5750640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0101290 0x7f0ec019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec5750640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0101290 0x7f0ec019bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46018/0 (socket says 172.21.15.83:46018) 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec5750640 1 -- 172.21.15.83:0/2761603930 learned_addr learned my addr 172.21.15.83:0/2761603930 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec5f51640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ec01000a0 0x7f0ec0068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.176+0000 7f0ec6752640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101c50 0x7f0ec019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:54.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0ec5750640 1 -- 172.21.15.83:0/2761603930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ec01000a0 msgr2=0x7f0ec0068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0ec5750640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ec01000a0 0x7f0ec0068e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0ec5750640 1 -- 172.21.15.83:0/2761603930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101c50 msgr2=0x7f0ec019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0ec5750640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101c50 0x7f0ec019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0ec5750640 1 -- 172.21.15.83:0/2761603930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc009560 con 0x7f0ec0101290 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0ec6752640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101c50 0x7f0ec019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0ec5750640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0101290 0x7f0ec019bf60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc02c6a0 tx=0x7f0ebc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0eb6ffd640 1 -- 172.21.15.83:0/2761603930 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0044e0 con 0x7f0ec0101290 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0ec81b5640 1 -- 172.21.15.83:0/2761603930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec019f2e0 con 0x7f0ec0101290 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0eb6ffd640 1 -- 172.21.15.83:0/2761603930 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ebc0054e0 con 0x7f0ec0101290 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0eb6ffd640 1 -- 172.21.15.83:0/2761603930 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc03e450 con 0x7f0ec0101290 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.177+0000 7f0ec81b5640 1 -- 172.21.15.83:0/2761603930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec019f7f0 con 0x7f0ec0101290 2024-03-11T19:36:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.178+0000 7f0ec81b5640 1 -- 172.21.15.83:0/2761603930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec01073c0 con 0x7f0ec0101290 2024-03-11T19:36:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.180+0000 7f0eb6ffd640 1 -- 172.21.15.83:0/2761603930 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0ebc003710 con 0x7f0ec0101290 2024-03-11T19:36:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.180+0000 7f0eb6ffd640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0eac0a1a30 0x7f0eac0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.180+0000 7f0ec5f51640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0eac0a1a30 0x7f0eac0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.180+0000 7f0eb6ffd640 1 -- 172.21.15.83:0/2761603930 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0ebc0efdd0 con 0x7f0ec0101290 2024-03-11T19:36:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.181+0000 7f0ec5f51640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0eac0a1a30 0x7f0eac0a3ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0005fd0 tx=0x7f0eb000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.184+0000 7f0eb6ffd640 1 -- 172.21.15.83:0/2761603930 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ebc0b95a0 con 0x7f0ec0101290 2024-03-11T19:36:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.406+0000 7f0ec81b5640 1 -- 172.21.15.83:0/2761603930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ec010bb00 con 0x7f0ec0101290 2024-03-11T19:36:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.407+0000 7f0eb6ffd640 1 -- 172.21.15.83:0/2761603930 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7f0ebc043020 con 0x7f0ec0101290 2024-03-11T19:36:54.411 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:54.411 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-11T19:36:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 10 2024-03-11T19:36:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 -- 172.21.15.83:0/2761603930 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0eac0a1a30 msgr2=0x7f0eac0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0eac0a1a30 0x7f0eac0a3ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0005fd0 tx=0x7f0eb000b410 comp rx=0 tx=0).stop 2024-03-11T19:36:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 -- 172.21.15.83:0/2761603930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0101290 msgr2=0x7f0ec019bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0101290 0x7f0ec019bf60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc02c6a0 tx=0x7f0ebc0042f0 comp rx=0 tx=0).stop 2024-03-11T19:36:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 -- 172.21.15.83:0/2761603930 shutdown_connections 2024-03-11T19:36:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0eac0a1a30 0x7f0eac0a3ef0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101c50 0x7f0ec019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0101290 0x7f0ec019bf60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 --2- 172.21.15.83:0/2761603930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ec01000a0 0x7f0ec0068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 -- 172.21.15.83:0/2761603930 >> 172.21.15.83:0/2761603930 conn(0x7f0ec00fb610 msgr2=0x7f0ec0104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 -- 172.21.15.83:0/2761603930 shutdown_connections 2024-03-11T19:36:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:54.409+0000 7f0ec81b5640 1 -- 172.21.15.83:0/2761603930 wait complete. 2024-03-11T19:36:54.697 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:54 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2761603930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:54.697 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:36:54 smithi083 systemd[1]: Starting Ceph mds.g for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:36:54.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:54 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2761603930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:54.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:54 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2761603930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:55.020 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:36:54 smithi083 podman[70711]: 2024-03-11 19:36:54.595925746 +0000 UTC m=+0.017595445 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:36:55.020 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:36:54 smithi083 podman[70711]: 2024-03-11 19:36:54.697172744 +0000 UTC m=+0.118842431 container create 672aa82f8972085d906f86e0fc9fc75a95c0b2bb6e22367a3fe3601ab431c4e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True) 2024-03-11T19:36:55.020 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:36:54 smithi083 podman[70711]: 2024-03-11 19:36:54.908460986 +0000 UTC m=+0.330130668 container init 672aa82f8972085d906f86e0fc9fc75a95c0b2bb6e22367a3fe3601ab431c4e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-g, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-11T19:36:55.021 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:36:54 smithi083 podman[70711]: 2024-03-11 19:36:54.911101443 +0000 UTC m=+0.332771127 container start 672aa82f8972085d906f86e0fc9fc75a95c0b2bb6e22367a3fe3601ab431c4e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:36:55.021 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:36:54 smithi083 ceph-mds[70731]: -- 172.21.15.83:0/1060700122 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556d0711e1a0 con 0x556d070f5000 2024-03-11T19:36:55.021 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:36:54 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-g[70727]: starting mds.g at 2024-03-11T19:36:55.021 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:36:55 smithi083 bash[70711]: 672aa82f8972085d906f86e0fc9fc75a95c0b2bb6e22367a3fe3601ab431c4e1 2024-03-11T19:36:55.318 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:36:55 smithi083 systemd[1]: Started Ceph mds.g for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:36:55.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:55 smithi083 ceph-mon[31864]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:55.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:55 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:55.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:55 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:55.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:55 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:55.801 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:55 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:55.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:55 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:55.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:55 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:55.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:55 smithi137 ceph-mon[36220]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:55.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:55 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:55.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:55 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:55.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:55 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:55.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:55 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:55.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:55 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:55.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:55 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:55 smithi204 ceph-mon[36739]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:55 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:55 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:55 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:55 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:55 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:55 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:56.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:56.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 -- 172.21.15.83:0/3290031916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e41023a0 msgr2=0x7f22e4102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:56.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 --2- 172.21.15.83:0/3290031916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e41023a0 0x7f22e4102820 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009900 tx=0x7f22d402bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:56.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 -- 172.21.15.83:0/3290031916 shutdown_connections 2024-03-11T19:36:56.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 --2- 172.21.15.83:0/3290031916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e4102d60 0x7f22e41095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:56.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 --2- 172.21.15.83:0/3290031916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e41023a0 0x7f22e4102820 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 --2- 172.21.15.83:0/3290031916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e41011b0 0x7f22e41015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 -- 172.21.15.83:0/3290031916 >> 172.21.15.83:0/3290031916 conn(0x7f22e40fc720 msgr2=0x7f22e40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 -- 172.21.15.83:0/3290031916 shutdown_connections 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 -- 172.21.15.83:0/3290031916 wait complete. 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 Processor -- start 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 -- start start 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.530+0000 7f22ec5ca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e41011b0 0x7f22e41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22ec5ca640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e41023a0 0x7f22e4197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22ec5ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e4102d60 0x7f22e419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22ec5ca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e419ece0 con 0x7f22e4102d60 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22ec5ca640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e419ee50 con 0x7f22e41011b0 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22ec5ca640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e419f010 con 0x7f22e41023a0 2024-03-11T19:36:56.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22eab67640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e4102d60 0x7f22e419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22eab67640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e4102d60 0x7f22e419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46052/0 (socket says 172.21.15.83:46052) 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22eab67640 1 -- 172.21.15.83:0/1785498100 learned_addr learned my addr 172.21.15.83:0/1785498100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22e9b65640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e41023a0 0x7f22e4197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22ea366640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e41011b0 0x7f22e41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22eab67640 1 -- 172.21.15.83:0/1785498100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e41023a0 msgr2=0x7f22e4197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22eab67640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e41023a0 0x7f22e4197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22eab67640 1 -- 172.21.15.83:0/1785498100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e41011b0 msgr2=0x7f22e41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22eab67640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e41011b0 0x7f22e41971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22eab67640 1 -- 172.21.15.83:0/1785498100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4009560 con 0x7f22e4102d60 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22eab67640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e4102d60 0x7f22e419e7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00eaa0 tx=0x7f22dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22db7fe640 1 -- 172.21.15.83:0/1785498100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00cdb0 con 0x7f22e4102d60 2024-03-11T19:36:56.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22db7fe640 1 -- 172.21.15.83:0/1785498100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22dc005020 con 0x7f22e4102d60 2024-03-11T19:36:56.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22ec5ca640 1 -- 172.21.15.83:0/1785498100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e419f2f0 con 0x7f22e4102d60 2024-03-11T19:36:56.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22db7fe640 1 -- 172.21.15.83:0/1785498100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0116b0 con 0x7f22e4102d60 2024-03-11T19:36:56.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.531+0000 7f22ec5ca640 1 -- 172.21.15.83:0/1785498100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e419f800 con 0x7f22e4102d60 2024-03-11T19:36:56.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.532+0000 7f22ec5ca640 1 -- 172.21.15.83:0/1785498100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e41084d0 con 0x7f22e4102d60 2024-03-11T19:36:56.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.533+0000 7f22db7fe640 1 -- 172.21.15.83:0/1785498100 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f22dc005520 con 0x7f22e4102d60 2024-03-11T19:36:56.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.534+0000 7f22db7fe640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22d00a1a30 0x7f22d00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:56.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.534+0000 7f22ea366640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22d00a1a30 0x7f22d00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:56.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.534+0000 7f22db7fe640 1 -- 172.21.15.83:0/1785498100 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f22dc0cd1c0 con 0x7f22e4102d60 2024-03-11T19:36:56.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.534+0000 7f22ea366640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22d00a1a30 0x7f22d00a3ef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005fd0 tx=0x7f22e0005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:56.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.538+0000 7f22db7fe640 1 -- 172.21.15.83:0/1785498100 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22dc096870 con 0x7f22e4102d60 2024-03-11T19:36:56.595 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:56 smithi083 ceph-mon[31864]: Deploying daemon mds.a on smithi083 2024-03-11T19:36:56.595 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:56 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.83:6838/1413247052,v1:172.21.15.83:6839/1413247052] up:boot 2024-03-11T19:36:56.595 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:56 smithi083 ceph-mon[31864]: fsmap 10 up:standby 2024-03-11T19:36:56.595 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:56 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:36:56.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.760+0000 7f22ec5ca640 1 -- 172.21.15.83:0/1785498100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22e410cbc0 con 0x7f22e4102d60 2024-03-11T19:36:56.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.760+0000 7f22db7fe640 1 -- 172.21.15.83:0/1785498100 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8465 (secure 0 0 0) 0x7f22dc099f30 con 0x7f22e4102d60 2024-03-11T19:36:56.764 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:56.765 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-11T19:36:56.765 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 11 2024-03-11T19:36:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 -- 172.21.15.83:0/1785498100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22d00a1a30 msgr2=0x7f22d00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22d00a1a30 0x7f22d00a3ef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f22e0005fd0 tx=0x7f22e0005c70 comp rx=0 tx=0).stop 2024-03-11T19:36:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 -- 172.21.15.83:0/1785498100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e4102d60 msgr2=0x7f22e419e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:56.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e4102d60 0x7f22e419e7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00eaa0 tx=0x7f22dc00ef60 comp rx=0 tx=0).stop 2024-03-11T19:36:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 -- 172.21.15.83:0/1785498100 shutdown_connections 2024-03-11T19:36:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22d00a1a30 0x7f22d00a3ef0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22e4102d60 0x7f22e419e7a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22e41023a0 0x7f22e4197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 --2- 172.21.15.83:0/1785498100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22e41011b0 0x7f22e41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 -- 172.21.15.83:0/1785498100 >> 172.21.15.83:0/1785498100 conn(0x7f22e40fc720 msgr2=0x7f22e41056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 -- 172.21.15.83:0/1785498100 shutdown_connections 2024-03-11T19:36:56.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:56.762+0000 7f22ec5ca640 1 -- 172.21.15.83:0/1785498100 wait complete. 2024-03-11T19:36:56.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:56 smithi137 ceph-mon[36220]: Deploying daemon mds.a on smithi083 2024-03-11T19:36:56.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:56 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.83:6838/1413247052,v1:172.21.15.83:6839/1413247052] up:boot 2024-03-11T19:36:56.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:56 smithi137 ceph-mon[36220]: fsmap 10 up:standby 2024-03-11T19:36:56.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:56 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:36:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:56 smithi204 ceph-mon[36739]: Deploying daemon mds.a on smithi083 2024-03-11T19:36:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:56 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.83:6838/1413247052,v1:172.21.15.83:6839/1413247052] up:boot 2024-03-11T19:36:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:56 smithi204 ceph-mon[36739]: fsmap 10 up:standby 2024-03-11T19:36:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:56 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-11T19:36:57.294 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:36:57 smithi083 systemd[1]: Starting Ceph mds.a for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:36:57.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:57 smithi083 ceph-mon[31864]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:57.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:57 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1785498100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:57.567 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:36:57 smithi083 podman[71054]: 2024-03-11 19:36:57.194170218 +0000 UTC m=+0.018185534 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:36:57.567 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:36:57 smithi083 podman[71054]: 2024-03-11 19:36:57.293433944 +0000 UTC m=+0.117449297 container create 5d7b2f08a83b4e13d4232a7c4547947ac20fd698fb4e7945688d9d9af6f33d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-a, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-11T19:36:57.567 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:36:57 smithi083 podman[71054]: 2024-03-11 19:36:57.53792971 +0000 UTC m=+0.361945022 container init 5d7b2f08a83b4e13d4232a7c4547947ac20fd698fb4e7945688d9d9af6f33d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-11T19:36:57.567 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:36:57 smithi083 podman[71054]: 2024-03-11 19:36:57.540603326 +0000 UTC m=+0.364618638 container start 5d7b2f08a83b4e13d4232a7c4547947ac20fd698fb4e7945688d9d9af6f33d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-11T19:36:57.818 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:36:57 smithi083 ceph-mds[71073]: -- 172.21.15.83:0/2875422853 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5568db79e1a0 con 0x5568db775000 2024-03-11T19:36:57.818 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:36:57 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-a[71069]: starting mds.a at 2024-03-11T19:36:57.818 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:36:57 smithi083 bash[71054]: 5d7b2f08a83b4e13d4232a7c4547947ac20fd698fb4e7945688d9d9af6f33d9d 2024-03-11T19:36:57.819 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:36:57 smithi083 systemd[1]: Started Ceph mds.a for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:36:57.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:57 smithi137 ceph-mon[36220]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:57.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:57 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1785498100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:57 smithi204 ceph-mon[36739]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:57 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1785498100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:36:58.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:36:58.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.890+0000 7f5caa90f640 1 -- 172.21.15.83:0/179992157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104670 msgr2=0x7f5ca4106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.890+0000 7f5caa90f640 1 --2- 172.21.15.83:0/179992157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104670 0x7f5ca4106a60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009900 tx=0x7f5c9802bc30 comp rx=0 tx=0).stop 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.890+0000 7f5caa90f640 1 -- 172.21.15.83:0/179992157 shutdown_connections 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.890+0000 7f5caa90f640 1 --2- 172.21.15.83:0/179992157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ca4106fa0 0x7f5ca4109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.890+0000 7f5caa90f640 1 --2- 172.21.15.83:0/179992157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104670 0x7f5ca4106a60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.890+0000 7f5caa90f640 1 --2- 172.21.15.83:0/179992157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ca4101d40 0x7f5ca4104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.890+0000 7f5caa90f640 1 -- 172.21.15.83:0/179992157 >> 172.21.15.83:0/179992157 conn(0x7f5ca40fb6b0 msgr2=0x7f5ca40fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.890+0000 7f5caa90f640 1 -- 172.21.15.83:0/179992157 shutdown_connections 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.890+0000 7f5caa90f640 1 -- 172.21.15.83:0/179992157 wait complete. 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.890+0000 7f5caa90f640 1 Processor -- start 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5caa90f640 1 -- start start 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5caa90f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ca4101d40 0x7f5ca4197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5caa90f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104670 0x7f5ca4197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5caa90f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ca4106fa0 0x7f5ca419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5caa90f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca419ed20 con 0x7f5ca4104670 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5caa90f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca419ee90 con 0x7f5ca4101d40 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5caa90f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca419f050 con 0x7f5ca4106fa0 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5ca37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104670 0x7f5ca4197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5ca37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104670 0x7f5ca4197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46080/0 (socket says 172.21.15.83:46080) 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5ca37fe640 1 -- 172.21.15.83:0/3576451134 learned_addr learned my addr 172.21.15.83:0/3576451134 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5ca3fff640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ca4101d40 0x7f5ca4197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5ca8eac640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ca4106fa0 0x7f5ca419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5ca37fe640 1 -- 172.21.15.83:0/3576451134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ca4106fa0 msgr2=0x7f5ca419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5ca37fe640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ca4106fa0 0x7f5ca419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5ca37fe640 1 -- 172.21.15.83:0/3576451134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ca4101d40 msgr2=0x7f5ca4197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5ca37fe640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ca4101d40 0x7f5ca4197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.891+0000 7f5ca37fe640 1 -- 172.21.15.83:0/3576451134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c98009560 con 0x7f5ca4104670 2024-03-11T19:36:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.892+0000 7f5ca37fe640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104670 0x7f5ca4197770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c98000c00 tx=0x7f5c98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.892+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3576451134 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98004df0 con 0x7f5ca4104670 2024-03-11T19:36:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.892+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3576451134 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c98005430 con 0x7f5ca4104670 2024-03-11T19:36:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.892+0000 7f5caa90f640 1 -- 172.21.15.83:0/3576451134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca419f2d0 con 0x7f5ca4104670 2024-03-11T19:36:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.892+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3576451134 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9803e450 con 0x7f5ca4104670 2024-03-11T19:36:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.892+0000 7f5caa90f640 1 -- 172.21.15.83:0/3576451134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca419f7e0 con 0x7f5ca4104670 2024-03-11T19:36:58.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.893+0000 7f5caa90f640 1 -- 172.21.15.83:0/3576451134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca40fd1a0 con 0x7f5ca4104670 2024-03-11T19:36:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.894+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3576451134 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5c98003680 con 0x7f5ca4104670 2024-03-11T19:36:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.894+0000 7f5ca17fa640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c8c0a1a30 0x7f5c8c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:36:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.894+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3576451134 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5c980efa90 con 0x7f5ca4104670 2024-03-11T19:36:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.894+0000 7f5ca3fff640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c8c0a1a30 0x7f5c8c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:36:58.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.895+0000 7f5ca3fff640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c8c0a1a30 0x7f5c8c0a3ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400bbe0 tx=0x7f5c9400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:36:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:58.898+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3576451134 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c980b9140 con 0x7f5ca4104670 2024-03-11T19:36:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:59.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:58 smithi083 ceph-mon[31864]: Deploying daemon mds.b on smithi137 2024-03-11T19:36:59.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:58 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.83:6840/1703400254,v1:172.21.15.83:6841/1703400254] up:boot 2024-03-11T19:36:59.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:58 smithi083 ceph-mon[31864]: fsmap 11 up:standby 2024-03-11T19:36:59.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:58 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:36:59.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:58 smithi083 ceph-mon[31864]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.107+0000 7f5caa90f640 1 -- 172.21.15.83:0/3576451134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ca410cbe0 con 0x7f5ca4104670 2024-03-11T19:36:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.108+0000 7f5ca17fa640 1 -- 172.21.15.83:0/3576451134 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9257 (secure 0 0 0) 0x7f5c980bc800 con 0x7f5ca4104670 2024-03-11T19:36:59.112 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:36:59.112 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-11T19:36:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 12 2024-03-11T19:36:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 -- 172.21.15.83:0/3576451134 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c8c0a1a30 msgr2=0x7f5c8c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c8c0a1a30 0x7f5c8c0a3ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400bbe0 tx=0x7f5c9400b390 comp rx=0 tx=0).stop 2024-03-11T19:36:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 -- 172.21.15.83:0/3576451134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104670 msgr2=0x7f5ca4197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:36:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104670 0x7f5ca4197770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c98000c00 tx=0x7f5c98004270 comp rx=0 tx=0).stop 2024-03-11T19:36:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 -- 172.21.15.83:0/3576451134 shutdown_connections 2024-03-11T19:36:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c8c0a1a30 0x7f5c8c0a3ef0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ca4106fa0 0x7f5ca419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104670 0x7f5ca4197770 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 --2- 172.21.15.83:0/3576451134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ca4101d40 0x7f5ca4197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:36:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 -- 172.21.15.83:0/3576451134 >> 172.21.15.83:0/3576451134 conn(0x7f5ca40fb6b0 msgr2=0x7f5ca4102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:36:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 -- 172.21.15.83:0/3576451134 shutdown_connections 2024-03-11T19:36:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:36:59.111+0000 7f5caa90f640 1 -- 172.21.15.83:0/3576451134 wait complete. 2024-03-11T19:36:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:58 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:58 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:58 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:58 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:58 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:58 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:58 smithi137 ceph-mon[36220]: Deploying daemon mds.b on smithi137 2024-03-11T19:36:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:58 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.83:6840/1703400254,v1:172.21.15.83:6841/1703400254] up:boot 2024-03-11T19:36:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:58 smithi137 ceph-mon[36220]: fsmap 11 up:standby 2024-03-11T19:36:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:58 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:36:59.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:58 smithi137 ceph-mon[36220]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:58 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:58 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:58 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:36:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:58 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-11T19:36:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:58 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-11T19:36:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:58 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:36:59.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:58 smithi204 ceph-mon[36739]: Deploying daemon mds.b on smithi137 2024-03-11T19:36:59.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:58 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.83:6840/1703400254,v1:172.21.15.83:6841/1703400254] up:boot 2024-03-11T19:36:59.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:58 smithi204 ceph-mon[36739]: fsmap 11 up:standby 2024-03-11T19:36:59.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:58 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-11T19:36:59.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:58 smithi204 ceph-mon[36739]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:36:59.970 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:36:59 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3576451134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:37:00.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:36:59 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3576451134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:37:00.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:36:59 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3576451134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:37:00.639 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:37:00 smithi137 systemd[1]: Started Ceph mds.b for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:37:01.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:37:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.225+0000 7f0a1700e640 1 -- 172.21.15.83:0/1956739980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a10101c50 msgr2=0x7f0a101084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.225+0000 7f0a1700e640 1 --2- 172.21.15.83:0/1956739980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a10101c50 0x7f0a101084e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009ca0 tx=0x7f0a0802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.225+0000 7f0a1700e640 1 -- 172.21.15.83:0/1956739980 shutdown_connections 2024-03-11T19:37:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.225+0000 7f0a1700e640 1 --2- 172.21.15.83:0/1956739980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a10101c50 0x7f0a101084e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.225+0000 7f0a1700e640 1 --2- 172.21.15.83:0/1956739980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a10101290 0x7f0a10101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.225+0000 7f0a1700e640 1 --2- 172.21.15.83:0/1956739980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a101000a0 0x7f0a101004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.225+0000 7f0a1700e640 1 -- 172.21.15.83:0/1956739980 >> 172.21.15.83:0/1956739980 conn(0x7f0a100fb610 msgr2=0x7f0a100fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.225+0000 7f0a1700e640 1 -- 172.21.15.83:0/1956739980 shutdown_connections 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.225+0000 7f0a1700e640 1 -- 172.21.15.83:0/1956739980 wait complete. 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a1700e640 1 Processor -- start 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a1700e640 1 -- start start 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a1700e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a101000a0 0x7f0a101040a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a1700e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a10101290 0x7f0a101026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a1700e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a10101c50 0x7f0a10102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a1700e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a10103380 con 0x7f0a10101c50 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a1700e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a101034f0 con 0x7f0a10101290 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a1700e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a101a1230 con 0x7f0a101000a0 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a155ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a10101c50 0x7f0a10102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a155ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a10101c50 0x7f0a10102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43988/0 (socket says 172.21.15.83:43988) 2024-03-11T19:37:01.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a155ab640 1 -- 172.21.15.83:0/1976103933 learned_addr learned my addr 172.21.15.83:0/1976103933 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a14daa640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a101000a0 0x7f0a101040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a0ffff640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a10101290 0x7f0a101026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a155ab640 1 -- 172.21.15.83:0/1976103933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a101000a0 msgr2=0x7f0a101040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a155ab640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a101000a0 0x7f0a101040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.226+0000 7f0a155ab640 1 -- 172.21.15.83:0/1976103933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a10101290 msgr2=0x7f0a101026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.227+0000 7f0a155ab640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a10101290 0x7f0a101026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.227+0000 7f0a155ab640 1 -- 172.21.15.83:0/1976103933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a08009900 con 0x7f0a10101c50 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.227+0000 7f0a0ffff640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a10101290 0x7f0a101026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.227+0000 7f0a14daa640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a101000a0 0x7f0a101040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.227+0000 7f0a155ab640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a10101c50 0x7f0a10102c20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0a08033000 tx=0x7f0a08034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.227+0000 7f0a0dffb640 1 -- 172.21.15.83:0/1976103933 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a080329b0 con 0x7f0a10101c50 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.227+0000 7f0a1700e640 1 -- 172.21.15.83:0/1976103933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a101a1480 con 0x7f0a10101c50 2024-03-11T19:37:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.227+0000 7f0a0dffb640 1 -- 172.21.15.83:0/1976103933 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a08003e50 con 0x7f0a10101c50 2024-03-11T19:37:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.227+0000 7f0a0dffb640 1 -- 172.21.15.83:0/1976103933 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08040450 con 0x7f0a10101c50 2024-03-11T19:37:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.227+0000 7f0a1700e640 1 -- 172.21.15.83:0/1976103933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a101a1930 con 0x7f0a10101c50 2024-03-11T19:37:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.228+0000 7f0a1700e640 1 -- 172.21.15.83:0/1976103933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1010ba70 con 0x7f0a10101c50 2024-03-11T19:37:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.229+0000 7f0a0dffb640 1 -- 172.21.15.83:0/1976103933 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0a0800a040 con 0x7f0a10101c50 2024-03-11T19:37:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.230+0000 7f0a0dffb640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a000a1a30 0x7f0a000a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.230+0000 7f0a0dffb640 1 -- 172.21.15.83:0/1976103933 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0a080f02b0 con 0x7f0a10101c50 2024-03-11T19:37:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.230+0000 7f0a14daa640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a000a1a30 0x7f0a000a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.230+0000 7f0a14daa640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a000a1a30 0x7f0a000a3ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0a04005d00 tx=0x7f0a04005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:01.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.237+0000 7f0a0dffb640 1 -- 172.21.15.83:0/1976103933 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0a080b9960 con 0x7f0a10101c50 2024-03-11T19:37:01.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.468+0000 7f0a1700e640 1 -- 172.21.15.83:0/1976103933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a1010bc00 con 0x7f0a10101c50 2024-03-11T19:37:01.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.469+0000 7f0a0dffb640 1 -- 172.21.15.83:0/1976103933 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10049 (secure 0 0 0) 0x7f0a080bd020 con 0x7f0a10101c50 2024-03-11T19:37:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:01.474 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24470,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-11T19:37:01.474 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 13 2024-03-11T19:37:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 -- 172.21.15.83:0/1976103933 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a000a1a30 msgr2=0x7f0a000a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a000a1a30 0x7f0a000a3ef0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0a04005d00 tx=0x7f0a04005c90 comp rx=0 tx=0).stop 2024-03-11T19:37:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 -- 172.21.15.83:0/1976103933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a10101c50 msgr2=0x7f0a10102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a10101c50 0x7f0a10102c20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0a08033000 tx=0x7f0a08034a90 comp rx=0 tx=0).stop 2024-03-11T19:37:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 -- 172.21.15.83:0/1976103933 shutdown_connections 2024-03-11T19:37:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a000a1a30 0x7f0a000a3ef0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a10101c50 0x7f0a10102c20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a10101290 0x7f0a101026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 --2- 172.21.15.83:0/1976103933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a101000a0 0x7f0a101040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 -- 172.21.15.83:0/1976103933 >> 172.21.15.83:0/1976103933 conn(0x7f0a100fb610 msgr2=0x7f0a100682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:01.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 -- 172.21.15.83:0/1976103933 shutdown_connections 2024-03-11T19:37:01.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.471+0000 7f0a1700e640 1 -- 172.21.15.83:0/1976103933 wait complete. 2024-03-11T19:37:01.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:37:01.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.588+0000 7feff1881640 1 -- 172.21.15.83:0/2822072965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec101290 msgr2=0x7fefec101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.588+0000 7feff1881640 1 --2- 172.21.15.83:0/2822072965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec101290 0x7fefec101710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009900 tx=0x7fefdc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:01.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.588+0000 7feff1881640 1 -- 172.21.15.83:0/2822072965 shutdown_connections 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.588+0000 7feff1881640 1 --2- 172.21.15.83:0/2822072965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefec101c50 0x7fefec1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.588+0000 7feff1881640 1 --2- 172.21.15.83:0/2822072965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec101290 0x7fefec101710 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.588+0000 7feff1881640 1 --2- 172.21.15.83:0/2822072965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec1000a0 0x7fefec1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.588+0000 7feff1881640 1 -- 172.21.15.83:0/2822072965 >> 172.21.15.83:0/2822072965 conn(0x7fefec0fb610 msgr2=0x7fefec0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.588+0000 7feff1881640 1 -- 172.21.15.83:0/2822072965 shutdown_connections 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.588+0000 7feff1881640 1 -- 172.21.15.83:0/2822072965 wait complete. 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7feff1881640 1 Processor -- start 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7feff1881640 1 -- start start 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7feff1881640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec1000a0 0x7fefec104030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7feff1881640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefec101290 0x7fefec102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7feff1881640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec101c50 0x7fefec102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7feff1881640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec103310 con 0x7fefec101c50 2024-03-11T19:37:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7feff1881640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec103480 con 0x7fefec101290 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7feff1881640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec1a1260 con 0x7fefec1000a0 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7fefeb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec101c50 0x7fefec102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7fefeb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec101c50 0x7fefec102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44010/0 (socket says 172.21.15.83:44010) 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7fefeb7fe640 1 -- 172.21.15.83:0/999668854 learned_addr learned my addr 172.21.15.83:0/999668854 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7fefeaffd640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec1000a0 0x7fefec104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.589+0000 7fefe27fc640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefec101290 0x7fefec102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.590+0000 7fefeb7fe640 1 -- 172.21.15.83:0/999668854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec1000a0 msgr2=0x7fefec104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.590+0000 7fefeb7fe640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec1000a0 0x7fefec104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.590+0000 7fefeb7fe640 1 -- 172.21.15.83:0/999668854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefec101290 msgr2=0x7fefec102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.590+0000 7fefeb7fe640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefec101290 0x7fefec102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.590+0000 7fefeb7fe640 1 -- 172.21.15.83:0/999668854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc009560 con 0x7fefec101c50 2024-03-11T19:37:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.590+0000 7fefeb7fe640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec101c50 0x7fefec102bb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fefe400eb40 tx=0x7fefe400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:01.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.590+0000 7feff087f640 1 -- 172.21.15.83:0/999668854 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4004b20 con 0x7fefec101c50 2024-03-11T19:37:01.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.590+0000 7feff087f640 1 -- 172.21.15.83:0/999668854 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefe4004c80 con 0x7fefec101c50 2024-03-11T19:37:01.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.590+0000 7feff087f640 1 -- 172.21.15.83:0/999668854 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4010670 con 0x7fefec101c50 2024-03-11T19:37:01.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.590+0000 7feff1881640 1 -- 172.21.15.83:0/999668854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefec1a1510 con 0x7fefec101c50 2024-03-11T19:37:01.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.590+0000 7feff1881640 1 -- 172.21.15.83:0/999668854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec1a19a0 con 0x7fefec101c50 2024-03-11T19:37:01.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.591+0000 7feff1881640 1 -- 172.21.15.83:0/999668854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefec10bab0 con 0x7fefec101c50 2024-03-11T19:37:01.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.592+0000 7feff087f640 1 -- 172.21.15.83:0/999668854 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fefe4005020 con 0x7fefec101c50 2024-03-11T19:37:01.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.593+0000 7feff087f640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefcc0a1a30 0x7fefcc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.593+0000 7fefeaffd640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefcc0a1a30 0x7fefcc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.593+0000 7feff087f640 1 -- 172.21.15.83:0/999668854 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fefe4014070 con 0x7fefec101c50 2024-03-11T19:37:01.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.593+0000 7fefeaffd640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefcc0a1a30 0x7fefcc0a3ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fefec1010f0 tx=0x7fefd400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:01.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.597+0000 7feff087f640 1 -- 172.21.15.83:0/999668854 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fefe4096bd0 con 0x7fefec101c50 2024-03-11T19:37:01.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.779+0000 7feff1881640 1 -- 172.21.15.83:0/999668854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefec10bc40 con 0x7fefec101c50 2024-03-11T19:37:01.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.780+0000 7feff087f640 1 -- 172.21.15.83:0/999668854 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18469 (secure 0 0 0) 0x7fefe409a290 con 0x7fefec101c50 2024-03-11T19:37:01.784 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:01.784 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:36:19.029213+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:37:01.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 -- 172.21.15.83:0/999668854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefcc0a1a30 msgr2=0x7fefcc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefcc0a1a30 0x7fefcc0a3ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fefec1010f0 tx=0x7fefd400b390 comp rx=0 tx=0).stop 2024-03-11T19:37:01.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 -- 172.21.15.83:0/999668854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec101c50 msgr2=0x7fefec102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec101c50 0x7fefec102bb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fefe400eb40 tx=0x7fefe400c2d0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 -- 172.21.15.83:0/999668854 shutdown_connections 2024-03-11T19:37:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefcc0a1a30 0x7fefcc0a3ef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefec101c50 0x7fefec102bb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefec101290 0x7fefec102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 --2- 172.21.15.83:0/999668854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec1000a0 0x7fefec104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 -- 172.21.15.83:0/999668854 >> 172.21.15.83:0/999668854 conn(0x7fefec0fb610 msgr2=0x7fefec0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 -- 172.21.15.83:0/999668854 shutdown_connections 2024-03-11T19:37:01.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.782+0000 7feff1881640 1 -- 172.21.15.83:0/999668854 wait complete. 2024-03-11T19:37:01.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:01 smithi083 ceph-mon[31864]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:01.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:01 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] up:boot 2024-03-11T19:37:01.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:01 smithi083 ceph-mon[31864]: fsmap 12 up:standby 2024-03-11T19:37:01.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:37:01.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:01 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:37:01.796 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:01 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1976103933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:37:01.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:37:01.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:37:01.810 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:01 smithi137 ceph-mon[36220]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:01.810 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.810 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.810 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.810 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.810 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.810 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:01 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] up:boot 2024-03-11T19:37:01.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:01 smithi137 ceph-mon[36220]: fsmap 12 up:standby 2024-03-11T19:37:01.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:37:01.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:01 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:37:01.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:01 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1976103933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:37:01.828 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:01 smithi204 ceph-mon[36739]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:01.828 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:01.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:01 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] up:boot 2024-03-11T19:37:01.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:01 smithi204 ceph-mon[36739]: fsmap 12 up:standby 2024-03-11T19:37:01.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:37:01.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:01 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:37:01.829 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:01 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1976103933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:37:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.899+0000 7f835910d640 1 -- 172.21.15.83:0/3549069216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8354101190 msgr2=0x7f83541015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.899+0000 7f835910d640 1 --2- 172.21.15.83:0/3549069216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8354101190 0x7f83541015b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f834c009900 tx=0x7f834c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.899+0000 7f835910d640 1 -- 172.21.15.83:0/3549069216 shutdown_connections 2024-03-11T19:37:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.899+0000 7f835910d640 1 --2- 172.21.15.83:0/3549069216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354102d40 0x7f83541095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.899+0000 7f835910d640 1 --2- 172.21.15.83:0/3549069216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8354102380 0x7f8354102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.899+0000 7f835910d640 1 --2- 172.21.15.83:0/3549069216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8354101190 0x7f83541015b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.899+0000 7f835910d640 1 -- 172.21.15.83:0/3549069216 >> 172.21.15.83:0/3549069216 conn(0x7f83540fc720 msgr2=0x7f83540feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.899+0000 7f835910d640 1 -- 172.21.15.83:0/3549069216 shutdown_connections 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.899+0000 7f835910d640 1 -- 172.21.15.83:0/3549069216 wait complete. 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.900+0000 7f835910d640 1 Processor -- start 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.900+0000 7f835910d640 1 -- start start 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.900+0000 7f835910d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8354101190 0x7f8354197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.900+0000 7f835910d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8354102380 0x7f8354197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.900+0000 7f835910d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354102d40 0x7f835419e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.900+0000 7f835910d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835419ef10 con 0x7f8354102380 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.900+0000 7f835910d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835419f080 con 0x7f8354101190 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.900+0000 7f835910d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835419f240 con 0x7f8354102d40 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.900+0000 7f835259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8354102380 0x7f8354197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.900+0000 7f835259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8354102380 0x7f8354197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44024/0 (socket says 172.21.15.83:44024) 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.900+0000 7f835259c640 1 -- 172.21.15.83:0/2488754717 learned_addr learned my addr 172.21.15.83:0/2488754717 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f835359e640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354102d40 0x7f835419e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f8352d9d640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8354101190 0x7f8354197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f835259c640 1 -- 172.21.15.83:0/2488754717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354102d40 msgr2=0x7f835419e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f835259c640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354102d40 0x7f835419e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f835259c640 1 -- 172.21.15.83:0/2488754717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8354101190 msgr2=0x7f8354197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f835259c640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8354101190 0x7f8354197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f835259c640 1 -- 172.21.15.83:0/2488754717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c009560 con 0x7f8354102380 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f835359e640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354102d40 0x7f835419e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f835259c640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8354102380 0x7f8354197960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f834800ba50 tx=0x7f834800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f833bfff640 1 -- 172.21.15.83:0/2488754717 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800cbe0 con 0x7f8354102380 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f835910d640 1 -- 172.21.15.83:0/2488754717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835419f520 con 0x7f8354102380 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f833bfff640 1 -- 172.21.15.83:0/2488754717 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f834800cd40 con 0x7f8354102380 2024-03-11T19:37:01.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f833bfff640 1 -- 172.21.15.83:0/2488754717 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480077d0 con 0x7f8354102380 2024-03-11T19:37:01.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.901+0000 7f835910d640 1 -- 172.21.15.83:0/2488754717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835419fab0 con 0x7f8354102380 2024-03-11T19:37:01.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.902+0000 7f835910d640 1 -- 172.21.15.83:0/2488754717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83541085a0 con 0x7f8354102380 2024-03-11T19:37:01.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.903+0000 7f833bfff640 1 -- 172.21.15.83:0/2488754717 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8348014440 con 0x7f8354102380 2024-03-11T19:37:01.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.904+0000 7f833bfff640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f833c0a1a30 0x7f833c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.904+0000 7f8352d9d640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f833c0a1a30 0x7f833c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.904+0000 7f833bfff640 1 -- 172.21.15.83:0/2488754717 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f83480ccdb0 con 0x7f8354102380 2024-03-11T19:37:01.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.904+0000 7f8352d9d640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f833c0a1a30 0x7f833c0a3ef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f834c02c6a0 tx=0x7f834c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:01.907+0000 7f833bfff640 1 -- 172.21.15.83:0/2488754717 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8348096460 con 0x7f8354102380 2024-03-11T19:37:02.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.091+0000 7f835910d640 1 -- 172.21.15.83:0/2488754717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f835410cdf0 con 0x7f8354102380 2024-03-11T19:37:02.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.092+0000 7f833bfff640 1 -- 172.21.15.83:0/2488754717 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83480230a0 con 0x7f8354102380 2024-03-11T19:37:02.096 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:37:02.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 -- 172.21.15.83:0/2488754717 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f833c0a1a30 msgr2=0x7f833c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:02.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f833c0a1a30 0x7f833c0a3ef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f834c02c6a0 tx=0x7f834c005fd0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 -- 172.21.15.83:0/2488754717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8354102380 msgr2=0x7f8354197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:02.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8354102380 0x7f8354197960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f834800ba50 tx=0x7f834800bf10 comp rx=0 tx=0).stop 2024-03-11T19:37:02.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 -- 172.21.15.83:0/2488754717 shutdown_connections 2024-03-11T19:37:02.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f833c0a1a30 0x7f833c0a3ef0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8354102d40 0x7f835419e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8354102380 0x7f8354197960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 --2- 172.21.15.83:0/2488754717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8354101190 0x7f8354197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 -- 172.21.15.83:0/2488754717 >> 172.21.15.83:0/2488754717 conn(0x7f83540fc720 msgr2=0x7f8354105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 -- 172.21.15.83:0/2488754717 shutdown_connections 2024-03-11T19:37:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.094+0000 7f835910d640 1 -- 172.21.15.83:0/2488754717 wait complete. 2024-03-11T19:37:02.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-11T19:37:02.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.219+0000 7f22d9965640 1 -- 172.21.15.83:0/1389476975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d41045d0 msgr2=0x7f22d41069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:02.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.219+0000 7f22d9965640 1 --2- 172.21.15.83:0/1389476975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d41045d0 0x7f22d41069c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009900 tx=0x7f22bc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:02.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.219+0000 7f22d9965640 1 -- 172.21.15.83:0/1389476975 shutdown_connections 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.219+0000 7f22d9965640 1 --2- 172.21.15.83:0/1389476975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d4106f00 0x7f22d41092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.219+0000 7f22d9965640 1 --2- 172.21.15.83:0/1389476975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d41045d0 0x7f22d41069c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.219+0000 7f22d9965640 1 --2- 172.21.15.83:0/1389476975 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22d4101ca0 0x7f22d4104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.219+0000 7f22d9965640 1 -- 172.21.15.83:0/1389476975 >> 172.21.15.83:0/1389476975 conn(0x7f22d40fb630 msgr2=0x7f22d40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.219+0000 7f22d9965640 1 -- 172.21.15.83:0/1389476975 shutdown_connections 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.219+0000 7f22d9965640 1 -- 172.21.15.83:0/1389476975 wait complete. 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d9965640 1 Processor -- start 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d9965640 1 -- start start 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d9965640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d4101ca0 0x7f22d41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d9965640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d41045d0 0x7f22d4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d9965640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22d4106f00 0x7f22d419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:02.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d9965640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d419edc0 con 0x7f22d4101ca0 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d9965640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d419ef30 con 0x7f22d4106f00 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d9965640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d419f0f0 con 0x7f22d41045d0 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d4101ca0 0x7f22d41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d4101ca0 0x7f22d41972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44030/0 (socket says 172.21.15.83:44030) 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d2ffd640 1 -- 172.21.15.83:0/3386597242 learned_addr learned my addr 172.21.15.83:0/3386597242 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22cbfff640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d41045d0 0x7f22d4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.220+0000 7f22d37fe640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22d4106f00 0x7f22d419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d2ffd640 1 -- 172.21.15.83:0/3386597242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d41045d0 msgr2=0x7f22d4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d2ffd640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d41045d0 0x7f22d4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d2ffd640 1 -- 172.21.15.83:0/3386597242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22d4106f00 msgr2=0x7f22d419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d2ffd640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22d4106f00 0x7f22d419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d2ffd640 1 -- 172.21.15.83:0/3386597242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc009560 con 0x7f22d4101ca0 2024-03-11T19:37:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d37fe640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22d4106f00 0x7f22d419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:37:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d2ffd640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d4101ca0 0x7f22d41972d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f22c400d950 tx=0x7f22c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d8963640 1 -- 172.21.15.83:0/3386597242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4009c30 con 0x7f22d4101ca0 2024-03-11T19:37:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d8963640 1 -- 172.21.15.83:0/3386597242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22c4009d90 con 0x7f22d4101ca0 2024-03-11T19:37:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d9965640 1 -- 172.21.15.83:0/3386597242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d419f3d0 con 0x7f22d4101ca0 2024-03-11T19:37:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d8963640 1 -- 172.21.15.83:0/3386597242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c400f7e0 con 0x7f22d4101ca0 2024-03-11T19:37:02.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.221+0000 7f22d9965640 1 -- 172.21.15.83:0/3386597242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d419f900 con 0x7f22d4101ca0 2024-03-11T19:37:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.222+0000 7f22d9965640 1 -- 172.21.15.83:0/3386597242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d40fd120 con 0x7f22d4101ca0 2024-03-11T19:37:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.223+0000 7f22d8963640 1 -- 172.21.15.83:0/3386597242 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f22c4010ad0 con 0x7f22d4101ca0 2024-03-11T19:37:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.224+0000 7f22d8963640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22c00a1ab0 0x7f22c00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.224+0000 7f22cbfff640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22c00a1ab0 0x7f22c00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.224+0000 7f22d8963640 1 -- 172.21.15.83:0/3386597242 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f22c40ccab0 con 0x7f22d4101ca0 2024-03-11T19:37:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.224+0000 7f22cbfff640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22c00a1ab0 0x7f22c00a3f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f22bc005ab0 tx=0x7f22bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.229+0000 7f22d8963640 1 -- 172.21.15.83:0/3386597242 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22c4096210 con 0x7f22d4101ca0 2024-03-11T19:37:02.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.450+0000 7f22d9965640 1 -- 172.21.15.83:0/3386597242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f22d410cbd0 con 0x7f22d4101ca0 2024-03-11T19:37:02.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.451+0000 7f22d8963640 1 -- 172.21.15.83:0/3386597242 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f22c40998d0 con 0x7f22d4101ca0 2024-03-11T19:37:02.455 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:02.455 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-03-11T19:37:02.455 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:37:02.455 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-03-11T19:37:02.455 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-03-11T19:37:02.455 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:37:02.455 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-03-11T19:37:02.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 -- 172.21.15.83:0/3386597242 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22c00a1ab0 msgr2=0x7f22c00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:02.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22c00a1ab0 0x7f22c00a3f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f22bc005ab0 tx=0x7f22bc005fd0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 -- 172.21.15.83:0/3386597242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d4101ca0 msgr2=0x7f22d41972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:02.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d4101ca0 0x7f22d41972d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f22c400d950 tx=0x7f22c400de10 comp rx=0 tx=0).stop 2024-03-11T19:37:02.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 -- 172.21.15.83:0/3386597242 shutdown_connections 2024-03-11T19:37:02.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22c00a1ab0 0x7f22c00a3f70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22d4106f00 0x7f22d419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d41045d0 0x7f22d4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 --2- 172.21.15.83:0/3386597242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22d4101ca0 0x7f22d41972d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 -- 172.21.15.83:0/3386597242 >> 172.21.15.83:0/3386597242 conn(0x7f22d40fb630 msgr2=0x7f22d41027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:02.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 -- 172.21.15.83:0/3386597242 shutdown_connections 2024-03-11T19:37:02.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.453+0000 7f22d9965640 1 -- 172.21.15.83:0/3386597242 wait complete. 2024-03-11T19:37:02.467 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-11T19:37:02.468 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-11T19:37:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.579+0000 7f4aceb93640 1 -- 172.21.15.83:0/41659320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac8101290 msgr2=0x7f4ac8101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.579+0000 7f4aceb93640 1 --2- 172.21.15.83:0/41659320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac8101290 0x7f4ac8101710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009900 tx=0x7f4ab402bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.579+0000 7f4aceb93640 1 -- 172.21.15.83:0/41659320 shutdown_connections 2024-03-11T19:37:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.579+0000 7f4aceb93640 1 --2- 172.21.15.83:0/41659320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac8101c50 0x7f4ac81084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.579+0000 7f4aceb93640 1 --2- 172.21.15.83:0/41659320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac8101290 0x7f4ac8101710 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.579+0000 7f4aceb93640 1 --2- 172.21.15.83:0/41659320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac81000a0 0x7f4ac81004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.579+0000 7f4aceb93640 1 -- 172.21.15.83:0/41659320 >> 172.21.15.83:0/41659320 conn(0x7f4ac80fb630 msgr2=0x7f4ac80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.579+0000 7f4aceb93640 1 -- 172.21.15.83:0/41659320 shutdown_connections 2024-03-11T19:37:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.579+0000 7f4aceb93640 1 -- 172.21.15.83:0/41659320 wait complete. 2024-03-11T19:37:02.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.579+0000 7f4aceb93640 1 Processor -- start 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4aceb93640 1 -- start start 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4aceb93640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac81000a0 0x7f4ac8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4aceb93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac8101290 0x7f4ac8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4aceb93640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac8101c50 0x7f4ac819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4aceb93640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac819ed20 con 0x7f4ac8101290 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4aceb93640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac819ee90 con 0x7f4ac81000a0 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4aceb93640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac819f050 con 0x7f4ac8101c50 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac8101290 0x7f4ac8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac8101290 0x7f4ac8197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44060/0 (socket says 172.21.15.83:44060) 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac7fff640 1 -- 172.21.15.83:0/2861632528 learned_addr learned my addr 172.21.15.83:0/2861632528 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4acc92f640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac81000a0 0x7f4ac8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:02.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4acd130640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac8101c50 0x7f4ac819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:02.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac7fff640 1 -- 172.21.15.83:0/2861632528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac8101c50 msgr2=0x7f4ac819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:02.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac7fff640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac8101c50 0x7f4ac819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac7fff640 1 -- 172.21.15.83:0/2861632528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac81000a0 msgr2=0x7f4ac8197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:02.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac7fff640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac81000a0 0x7f4ac8197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:02.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac7fff640 1 -- 172.21.15.83:0/2861632528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab4009560 con 0x7f4ac8101290 2024-03-11T19:37:02.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4acc92f640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac81000a0 0x7f4ac8197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:37:02.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac7fff640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac8101290 0x7f4ac8197770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4006010 tx=0x7f4ab4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:02.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac5ffb640 1 -- 172.21.15.83:0/2861632528 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab40043d0 con 0x7f4ac8101290 2024-03-11T19:37:02.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac5ffb640 1 -- 172.21.15.83:0/2861632528 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ab40052d0 con 0x7f4ac8101290 2024-03-11T19:37:02.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4aceb93640 1 -- 172.21.15.83:0/2861632528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac819f2d0 con 0x7f4ac8101290 2024-03-11T19:37:02.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.580+0000 7f4ac5ffb640 1 -- 172.21.15.83:0/2861632528 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4003cc0 con 0x7f4ac8101290 2024-03-11T19:37:02.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.581+0000 7f4aceb93640 1 -- 172.21.15.83:0/2861632528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac819f7e0 con 0x7f4ac8101290 2024-03-11T19:37:02.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.582+0000 7f4aceb93640 1 -- 172.21.15.83:0/2861632528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac81073c0 con 0x7f4ac8101290 2024-03-11T19:37:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.583+0000 7f4ac5ffb640 1 -- 172.21.15.83:0/2861632528 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4ab4005440 con 0x7f4ac8101290 2024-03-11T19:37:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.584+0000 7f4ac5ffb640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4ab80a1a30 0x7f4ab80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.584+0000 7f4acc92f640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4ab80a1a30 0x7f4ab80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.584+0000 7f4ac5ffb640 1 -- 172.21.15.83:0/2861632528 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4ab40ef730 con 0x7f4ac8101290 2024-03-11T19:37:02.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.584+0000 7f4acc92f640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4ab80a1a30 0x7f4ab80a3ef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4ac81010f0 tx=0x7f4abc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:02.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.587+0000 7f4ac5ffb640 1 -- 172.21.15.83:0/2861632528 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ab40b8de0 con 0x7f4ac8101290 2024-03-11T19:37:02.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:02.789+0000 7f4aceb93640 1 -- 172.21.15.83:0/2861632528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4ac810cbb0 con 0x7f4ac8101290 2024-03-11T19:37:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/999668854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2488754717' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:37:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3386597242' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-11T19:37:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:37:02.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:37:02.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:02 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/999668854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2488754717' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3386597242' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:37:02.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:02 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/999668854' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2488754717' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:37:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3386597242' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-11T19:37:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:02.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:37:02.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:37:02.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:02 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.561+0000 7f4ac5ffb640 1 -- 172.21.15.83:0/2861632528 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4ab40bc4a0 con 0x7f4ac8101290 2024-03-11T19:37:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-03-11T19:37:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.563+0000 7f4aceb93640 1 -- 172.21.15.83:0/2861632528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4ab80a1a30 msgr2=0x7f4ab80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.563+0000 7f4aceb93640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4ab80a1a30 0x7f4ab80a3ef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4ac81010f0 tx=0x7f4abc005f90 comp rx=0 tx=0).stop 2024-03-11T19:37:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.563+0000 7f4aceb93640 1 -- 172.21.15.83:0/2861632528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac8101290 msgr2=0x7f4ac8197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:03.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.563+0000 7f4aceb93640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac8101290 0x7f4ac8197770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4006010 tx=0x7f4ab4005e80 comp rx=0 tx=0).stop 2024-03-11T19:37:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.563+0000 7f4aceb93640 1 -- 172.21.15.83:0/2861632528 shutdown_connections 2024-03-11T19:37:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.563+0000 7f4aceb93640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4ab80a1a30 0x7f4ab80a3ef0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.563+0000 7f4aceb93640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac8101c50 0x7f4ac819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.563+0000 7f4aceb93640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac8101290 0x7f4ac8197770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.563+0000 7f4aceb93640 1 --2- 172.21.15.83:0/2861632528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac81000a0 0x7f4ac8197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.564+0000 7f4aceb93640 1 -- 172.21.15.83:0/2861632528 >> 172.21.15.83:0/2861632528 conn(0x7f4ac80fb630 msgr2=0x7f4ac8104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.564+0000 7f4aceb93640 1 -- 172.21.15.83:0/2861632528 shutdown_connections 2024-03-11T19:37:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.564+0000 7f4aceb93640 1 -- 172.21.15.83:0/2861632528 wait complete. 2024-03-11T19:37:03.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-11T19:37:03.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.689+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1869702002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5af0101270 msgr2=0x7f5af01016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:03.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.689+0000 7f5af4e8d640 1 --2- 172.21.15.83:0/1869702002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5af0101270 0x7f5af01016f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009900 tx=0x7f5ae402bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:03.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.689+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1869702002 shutdown_connections 2024-03-11T19:37:03.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.689+0000 7f5af4e8d640 1 --2- 172.21.15.83:0/1869702002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0101c30 0x7f5af01084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.689+0000 7f5af4e8d640 1 --2- 172.21.15.83:0/1869702002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5af0101270 0x7f5af01016f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.689+0000 7f5af4e8d640 1 --2- 172.21.15.83:0/1869702002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0100080 0x7f5af01004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.689+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1869702002 >> 172.21.15.83:0/1869702002 conn(0x7f5af00fb610 msgr2=0x7f5af00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.689+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1869702002 shutdown_connections 2024-03-11T19:37:03.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.689+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1869702002 wait complete. 2024-03-11T19:37:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5af4e8d640 1 Processor -- start 2024-03-11T19:37:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5af4e8d640 1 -- start start 2024-03-11T19:37:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5af4e8d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5af0100080 0x7f5af01960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5af4e8d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101270 0x7f5af01965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:03.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5af4e8d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0101c30 0x7f5af019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5af4e8d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af019dba0 con 0x7f5af0100080 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5af4e8d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af019dd10 con 0x7f5af0101270 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5af4e8d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af019ded0 con 0x7f5af0101c30 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5aee59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5af0100080 0x7f5af01960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5aee59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5af0100080 0x7f5af01960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44078/0 (socket says 172.21.15.83:44078) 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5aee59c640 1 -- 172.21.15.83:0/1839909990 learned_addr learned my addr 172.21.15.83:0/1839909990 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5aedd9b640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101270 0x7f5af01965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5aeed9d640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0101c30 0x7f5af019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5aee59c640 1 -- 172.21.15.83:0/1839909990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0101c30 msgr2=0x7f5af019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5aee59c640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0101c30 0x7f5af019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5aee59c640 1 -- 172.21.15.83:0/1839909990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101270 msgr2=0x7f5af01965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5aee59c640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101270 0x7f5af01965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5aee59c640 1 -- 172.21.15.83:0/1839909990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4009560 con 0x7f5af0100080 2024-03-11T19:37:03.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.690+0000 7f5aee59c640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5af0100080 0x7f5af01960b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800ecd0 tx=0x7f5ae800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:03.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.691+0000 7f5adf7fe640 1 -- 172.21.15.83:0/1839909990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8013070 con 0x7f5af0100080 2024-03-11T19:37:03.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.691+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1839909990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af0069810 con 0x7f5af0100080 2024-03-11T19:37:03.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.691+0000 7f5adf7fe640 1 -- 172.21.15.83:0/1839909990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ae8003e50 con 0x7f5af0100080 2024-03-11T19:37:03.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.691+0000 7f5adf7fe640 1 -- 172.21.15.83:0/1839909990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8017670 con 0x7f5af0100080 2024-03-11T19:37:03.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.691+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1839909990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af0069cf0 con 0x7f5af0100080 2024-03-11T19:37:03.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.692+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1839909990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af00683c0 con 0x7f5af0100080 2024-03-11T19:37:03.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.693+0000 7f5adf7fe640 1 -- 172.21.15.83:0/1839909990 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5ae80042a0 con 0x7f5af0100080 2024-03-11T19:37:03.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.693+0000 7f5adf7fe640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ad80a1a30 0x7f5ad80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:03.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.693+0000 7f5aedd9b640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ad80a1a30 0x7f5ad80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:03.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.693+0000 7f5adf7fe640 1 -- 172.21.15.83:0/1839909990 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f5ae80ceb50 con 0x7f5af0100080 2024-03-11T19:37:03.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.694+0000 7f5aedd9b640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ad80a1a30 0x7f5ad80a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009230 tx=0x7f5ae4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:03.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.698+0000 7f5adf7fe640 1 -- 172.21.15.83:0/1839909990 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ae8098140 con 0x7f5af0100080 2024-03-11T19:37:03.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:03 smithi083 ceph-mon[31864]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:03.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:03 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:37:03.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:03 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2861632528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:37:03.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:03 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:03.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:03.875+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1839909990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5af010bd00 con 0x7f5af0100080 2024-03-11T19:37:03.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:03 smithi137 ceph-mon[36220]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:03.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:03 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:37:03.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:03 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2861632528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:37:03.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:03 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:03 smithi204 ceph-mon[36739]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:03 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:37:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:03 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2861632528' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-11T19:37:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:03 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.567+0000 7f5adf7fe640 1 -- 172.21.15.83:0/1839909990 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5ae809b800 con 0x7f5af0100080 2024-03-11T19:37:04.571 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-03-11T19:37:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1839909990 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ad80a1a30 msgr2=0x7f5ad80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ad80a1a30 0x7f5ad80a3ef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009230 tx=0x7f5ae4005fd0 comp rx=0 tx=0).stop 2024-03-11T19:37:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1839909990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5af0100080 msgr2=0x7f5af01960b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5af0100080 0x7f5af01960b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800ecd0 tx=0x7f5ae800b690 comp rx=0 tx=0).stop 2024-03-11T19:37:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1839909990 shutdown_connections 2024-03-11T19:37:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5ad80a1a30 0x7f5ad80a3ef0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5af0101c30 0x7f5af019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101270 0x7f5af01965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 --2- 172.21.15.83:0/1839909990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5af0100080 0x7f5af01960b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:04.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1839909990 >> 172.21.15.83:0/1839909990 conn(0x7f5af00fb610 msgr2=0x7f5af0104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1839909990 shutdown_connections 2024-03-11T19:37:04.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.569+0000 7f5af4e8d640 1 -- 172.21.15.83:0/1839909990 wait complete. 2024-03-11T19:37:04.593 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-11T19:37:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.694+0000 7f27cd563640 1 -- 172.21.15.83:0/3795198748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8102430 msgr2=0x7f27c81028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.694+0000 7f27cd563640 1 --2- 172.21.15.83:0/3795198748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8102430 0x7f27c81028b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009900 tx=0x7f27b002bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.694+0000 7f27cd563640 1 -- 172.21.15.83:0/3795198748 shutdown_connections 2024-03-11T19:37:04.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.694+0000 7f27cd563640 1 --2- 172.21.15.83:0/3795198748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c8102df0 0x7f27c8109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.694+0000 7f27cd563640 1 --2- 172.21.15.83:0/3795198748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8102430 0x7f27c81028b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.694+0000 7f27cd563640 1 --2- 172.21.15.83:0/3795198748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27c80ff460 0x7f27c80ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.694+0000 7f27cd563640 1 -- 172.21.15.83:0/3795198748 >> 172.21.15.83:0/3795198748 conn(0x7f27c80fad90 msgr2=0x7f27c80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.694+0000 7f27cd563640 1 -- 172.21.15.83:0/3795198748 shutdown_connections 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.694+0000 7f27cd563640 1 -- 172.21.15.83:0/3795198748 wait complete. 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27cd563640 1 Processor -- start 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27cd563640 1 -- start start 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27cd563640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c80ff460 0x7f27c8101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27cd563640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27c8102430 0x7f27c81003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27cd563640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8102df0 0x7f27c8100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27cd563640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8101080 con 0x7f27c8102df0 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27cd563640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81011f0 con 0x7f27c8102430 2024-03-11T19:37:04.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27cd563640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81a1230 con 0x7f27c80ff460 2024-03-11T19:37:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27c77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8102df0 0x7f27c8100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27c77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8102df0 0x7f27c8100920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44100/0 (socket says 172.21.15.83:44100) 2024-03-11T19:37:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27c77fe640 1 -- 172.21.15.83:0/950576948 learned_addr learned my addr 172.21.15.83:0/950576948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27bffff640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27c8102430 0x7f27c81003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27c6ffd640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c80ff460 0x7f27c8101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27c77fe640 1 -- 172.21.15.83:0/950576948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c80ff460 msgr2=0x7f27c8101da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27c77fe640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c80ff460 0x7f27c8101da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27c77fe640 1 -- 172.21.15.83:0/950576948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27c8102430 msgr2=0x7f27c81003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27c77fe640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27c8102430 0x7f27c81003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27c77fe640 1 -- 172.21.15.83:0/950576948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0009560 con 0x7f27c8102df0 2024-03-11T19:37:04.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.695+0000 7f27c77fe640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8102df0 0x7f27c8100920 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f27c000eb40 tx=0x7f27c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.696+0000 7f27c4ff9640 1 -- 172.21.15.83:0/950576948 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0004b20 con 0x7f27c8102df0 2024-03-11T19:37:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.696+0000 7f27cd563640 1 -- 172.21.15.83:0/950576948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c81a14e0 con 0x7f27c8102df0 2024-03-11T19:37:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.696+0000 7f27c4ff9640 1 -- 172.21.15.83:0/950576948 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27c0004c80 con 0x7f27c8102df0 2024-03-11T19:37:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.696+0000 7f27c4ff9640 1 -- 172.21.15.83:0/950576948 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0010690 con 0x7f27c8102df0 2024-03-11T19:37:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.696+0000 7f27cd563640 1 -- 172.21.15.83:0/950576948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c81a1a10 con 0x7f27c8102df0 2024-03-11T19:37:04.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.697+0000 7f27cd563640 1 -- 172.21.15.83:0/950576948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c8108560 con 0x7f27c8102df0 2024-03-11T19:37:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.698+0000 7f27c4ff9640 1 -- 172.21.15.83:0/950576948 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f27c0004df0 con 0x7f27c8102df0 2024-03-11T19:37:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.698+0000 7f27c4ff9640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f27b40a1a30 0x7f27b40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.698+0000 7f27c6ffd640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f27b40a1a30 0x7f27b40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.698+0000 7f27c4ff9640 1 -- 172.21.15.83:0/950576948 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f27c0014070 con 0x7f27c8102df0 2024-03-11T19:37:04.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.699+0000 7f27c6ffd640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f27b40a1a30 0x7f27b40a3ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f27c8069570 tx=0x7f27b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:04.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.702+0000 7f27c4ff9640 1 -- 172.21.15.83:0/950576948 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27c0097040 con 0x7f27c8102df0 2024-03-11T19:37:04.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2861632528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-11T19:37:04.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: osdmap e82: 12 total, 12 up, 12 in 2024-03-11T19:37:04.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1839909990' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:37:04.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:37:04.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:37:04.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2861632528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-11T19:37:04.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: osdmap e82: 12 total, 12 up, 12 in 2024-03-11T19:37:04.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1839909990' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:37:04.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:37:04.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:37:04.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2861632528' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-11T19:37:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: osdmap e82: 12 total, 12 up, 12 in 2024-03-11T19:37:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1839909990' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-11T19:37:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:37:04.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:37:04.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:37:04.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:04.915+0000 7f27cd563640 1 -- 172.21.15.83:0/950576948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f27c810cbc0 con 0x7f27c8102df0 2024-03-11T19:37:05.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.568+0000 7f27c4ff9640 1 -- 172.21.15.83:0/950576948 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f27c009a700 con 0x7f27c8102df0 2024-03-11T19:37:05.573 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 2 and data pool 3 2024-03-11T19:37:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 -- 172.21.15.83:0/950576948 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f27b40a1a30 msgr2=0x7f27b40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f27b40a1a30 0x7f27b40a3ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f27c8069570 tx=0x7f27b8005f90 comp rx=0 tx=0).stop 2024-03-11T19:37:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 -- 172.21.15.83:0/950576948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8102df0 msgr2=0x7f27c8100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8102df0 0x7f27c8100920 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f27c000eb40 tx=0x7f27c000c2d0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 -- 172.21.15.83:0/950576948 shutdown_connections 2024-03-11T19:37:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f27b40a1a30 0x7f27b40a3ef0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27c8102df0 0x7f27c8100920 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27c8102430 0x7f27c81003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 --2- 172.21.15.83:0/950576948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c80ff460 0x7f27c8101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 -- 172.21.15.83:0/950576948 >> 172.21.15.83:0/950576948 conn(0x7f27c80fad90 msgr2=0x7f27c8105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 -- 172.21.15.83:0/950576948 shutdown_connections 2024-03-11T19:37:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.571+0000 7f27cd563640 1 -- 172.21.15.83:0/950576948 wait complete. 2024-03-11T19:37:05.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:37:05.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.685+0000 7f729ab8c640 1 -- 172.21.15.83:0/3894868391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72941000a0 msgr2=0x7f72941004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:05.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.685+0000 7f729ab8c640 1 --2- 172.21.15.83:0/3894868391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72941000a0 0x7f72941004c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7288009900 tx=0x7f728802bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:05.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.685+0000 7f729ab8c640 1 -- 172.21.15.83:0/3894868391 shutdown_connections 2024-03-11T19:37:05.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.685+0000 7f729ab8c640 1 --2- 172.21.15.83:0/3894868391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7294101c50 0x7f72941084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.685+0000 7f729ab8c640 1 --2- 172.21.15.83:0/3894868391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7294101290 0x7f7294101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.685+0000 7f729ab8c640 1 --2- 172.21.15.83:0/3894868391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72941000a0 0x7f72941004c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.685+0000 7f729ab8c640 1 -- 172.21.15.83:0/3894868391 >> 172.21.15.83:0/3894868391 conn(0x7f72940fb610 msgr2=0x7f72940fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.685+0000 7f729ab8c640 1 -- 172.21.15.83:0/3894868391 shutdown_connections 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.685+0000 7f729ab8c640 1 -- 172.21.15.83:0/3894868391 wait complete. 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f729ab8c640 1 Processor -- start 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f729ab8c640 1 -- start start 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f729ab8c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72941000a0 0x7f7294068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f729ab8c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7294101290 0x7f729419bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f729ab8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294101c50 0x7f729419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f729ab8c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729419edc0 con 0x7f7294101c50 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f729ab8c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729419ef30 con 0x7f7294101290 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f729ab8c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729419f0f0 con 0x7f72941000a0 2024-03-11T19:37:05.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f7299129640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294101c50 0x7f729419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f7299129640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294101c50 0x7f729419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44124/0 (socket says 172.21.15.83:44124) 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f7299129640 1 -- 172.21.15.83:0/380420531 learned_addr learned my addr 172.21.15.83:0/380420531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f7293fff640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7294101290 0x7f729419bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f7299129640 1 -- 172.21.15.83:0/380420531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72941000a0 msgr2=0x7f7294068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f7298928640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72941000a0 0x7f7294068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f7299129640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72941000a0 0x7f7294068e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f7299129640 1 -- 172.21.15.83:0/380420531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7294101290 msgr2=0x7f729419bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f7299129640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7294101290 0x7f729419bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.686+0000 7f7299129640 1 -- 172.21.15.83:0/380420531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7288009560 con 0x7f7294101c50 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.687+0000 7f7299129640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294101c50 0x7f729419e630 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f728c00eaa0 tx=0x7f728c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.687+0000 7f7291ffb640 1 -- 172.21.15.83:0/380420531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c004d20 con 0x7f7294101c50 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.687+0000 7f729ab8c640 1 -- 172.21.15.83:0/380420531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729419f3d0 con 0x7f7294101c50 2024-03-11T19:37:05.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.687+0000 7f7291ffb640 1 -- 172.21.15.83:0/380420531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f728c004e80 con 0x7f7294101c50 2024-03-11T19:37:05.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.687+0000 7f7291ffb640 1 -- 172.21.15.83:0/380420531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c005a10 con 0x7f7294101c50 2024-03-11T19:37:05.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.687+0000 7f729ab8c640 1 -- 172.21.15.83:0/380420531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729419f910 con 0x7f7294101c50 2024-03-11T19:37:05.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.688+0000 7f729ab8c640 1 -- 172.21.15.83:0/380420531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72941073c0 con 0x7f7294101c50 2024-03-11T19:37:05.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.689+0000 7f7291ffb640 1 -- 172.21.15.83:0/380420531 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f728c005020 con 0x7f7294101c50 2024-03-11T19:37:05.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.689+0000 7f7291ffb640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72840a1a30 0x7f72840a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:05.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.689+0000 7f7298928640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72840a1a30 0x7f72840a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:05.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.689+0000 7f7291ffb640 1 -- 172.21.15.83:0/380420531 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f728c0cd580 con 0x7f7294101c50 2024-03-11T19:37:05.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.690+0000 7f7298928640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72840a1a30 0x7f72840a3ef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f728802c6a0 tx=0x7f7288005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:05.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.694+0000 7f7291ffb640 1 -- 172.21.15.83:0/380420531 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f728c0968e0 con 0x7f7294101c50 2024-03-11T19:37:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.867+0000 7f729ab8c640 1 -- 172.21.15.83:0/380420531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f729410bd40 con 0x7f7294101c50 2024-03-11T19:37:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.868+0000 7f7291ffb640 1 -- 172.21.15.83:0/380420531 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22245 (secure 0 0 0) 0x7f728c099fa0 con 0x7f7294101c50 2024-03-11T19:37:05.873 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:05.873 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":84,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:37:04.916427+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:37:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.870+0000 7f729ab8c640 1 -- 172.21.15.83:0/380420531 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72840a1a30 msgr2=0x7f72840a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.870+0000 7f729ab8c640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72840a1a30 0x7f72840a3ef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f728802c6a0 tx=0x7f7288005fd0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.870+0000 7f729ab8c640 1 -- 172.21.15.83:0/380420531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294101c50 msgr2=0x7f729419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.870+0000 7f729ab8c640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294101c50 0x7f729419e630 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f728c00eaa0 tx=0x7f728c00eea0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.871+0000 7f729ab8c640 1 -- 172.21.15.83:0/380420531 shutdown_connections 2024-03-11T19:37:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.871+0000 7f729ab8c640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72840a1a30 0x7f72840a3ef0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.871+0000 7f729ab8c640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7294101c50 0x7f729419e630 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.871+0000 7f729ab8c640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7294101290 0x7f729419bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.871+0000 7f729ab8c640 1 --2- 172.21.15.83:0/380420531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72941000a0 0x7f7294068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.871+0000 7f729ab8c640 1 -- 172.21.15.83:0/380420531 >> 172.21.15.83:0/380420531 conn(0x7f72940fb610 msgr2=0x7f7294104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.871+0000 7f729ab8c640 1 -- 172.21.15.83:0/380420531 shutdown_connections 2024-03-11T19:37:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.871+0000 7f729ab8c640 1 -- 172.21.15.83:0/380420531 wait complete. 2024-03-11T19:37:05.884 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:05 smithi083 ceph-mon[31864]: pgmap v280: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:05.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:05 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1839909990' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-11T19:37:05.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:05 smithi083 ceph-mon[31864]: osdmap e83: 12 total, 12 up, 12 in 2024-03-11T19:37:05.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:05 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/950576948' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:37:05.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:05 smithi083 ceph-mon[31864]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-11T19:37:05.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:05 smithi083 ceph-mon[31864]: osdmap e84: 12 total, 12 up, 12 in 2024-03-11T19:37:05.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:05 smithi083 ceph-mon[31864]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:37:05.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:37:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:05 smithi137 ceph-mon[36220]: pgmap v280: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:05 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1839909990' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-11T19:37:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:05 smithi137 ceph-mon[36220]: osdmap e83: 12 total, 12 up, 12 in 2024-03-11T19:37:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:05 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/950576948' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:37:05.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:05 smithi137 ceph-mon[36220]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-11T19:37:05.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:05 smithi137 ceph-mon[36220]: osdmap e84: 12 total, 12 up, 12 in 2024-03-11T19:37:05.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:05 smithi137 ceph-mon[36220]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:37:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:05 smithi204 ceph-mon[36739]: pgmap v280: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:05 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1839909990' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-11T19:37:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:05 smithi204 ceph-mon[36739]: osdmap e83: 12 total, 12 up, 12 in 2024-03-11T19:37:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:05 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/950576948' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-11T19:37:05.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:05 smithi204 ceph-mon[36739]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-11T19:37:05.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:05 smithi204 ceph-mon[36739]: osdmap e84: 12 total, 12 up, 12 in 2024-03-11T19:37:05.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:05 smithi204 ceph-mon[36739]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-11T19:37:05.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.982+0000 7f25d7e15640 1 -- 172.21.15.83:0/4151132700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d0100120 msgr2=0x7f25d0100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.982+0000 7f25d7e15640 1 --2- 172.21.15.83:0/4151132700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d0100120 0x7f25d0100540 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009900 tx=0x7f25c002bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.982+0000 7f25d7e15640 1 -- 172.21.15.83:0/4151132700 shutdown_connections 2024-03-11T19:37:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.982+0000 7f25d7e15640 1 --2- 172.21.15.83:0/4151132700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25d0101cd0 0x7f25d0108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.982+0000 7f25d7e15640 1 --2- 172.21.15.83:0/4151132700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d0101310 0x7f25d0101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.982+0000 7f25d7e15640 1 --2- 172.21.15.83:0/4151132700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d0100120 0x7f25d0100540 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.982+0000 7f25d7e15640 1 -- 172.21.15.83:0/4151132700 >> 172.21.15.83:0/4151132700 conn(0x7f25d00fb6b0 msgr2=0x7f25d00fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d7e15640 1 -- 172.21.15.83:0/4151132700 shutdown_connections 2024-03-11T19:37:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d7e15640 1 -- 172.21.15.83:0/4151132700 wait complete. 2024-03-11T19:37:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d7e15640 1 Processor -- start 2024-03-11T19:37:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d7e15640 1 -- start start 2024-03-11T19:37:05.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d7e15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d0100120 0x7f25d0196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d7e15640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d0101310 0x7f25d0196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d7e15640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25d0101cd0 0x7f25d019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d7e15640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d019dc30 con 0x7f25d0101310 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d7e15640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d019dda0 con 0x7f25d0100120 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d7e15640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d019df60 con 0x7f25d0101cd0 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d53b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d0101310 0x7f25d0196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d53b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d0101310 0x7f25d0196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44136/0 (socket says 172.21.15.83:44136) 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d53b0640 1 -- 172.21.15.83:0/2080054710 learned_addr learned my addr 172.21.15.83:0/2080054710 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.983+0000 7f25d63b2640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25d0101cd0 0x7f25d019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.984+0000 7f25d53b0640 1 -- 172.21.15.83:0/2080054710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25d0101cd0 msgr2=0x7f25d019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.984+0000 7f25d53b0640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25d0101cd0 0x7f25d019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.984+0000 7f25d53b0640 1 -- 172.21.15.83:0/2080054710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d0100120 msgr2=0x7f25d0196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:05.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.984+0000 7f25d53b0640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d0100120 0x7f25d0196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.984+0000 7f25d53b0640 1 -- 172.21.15.83:0/2080054710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c0009560 con 0x7f25d0101310 2024-03-11T19:37:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.984+0000 7f25d53b0640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d0101310 0x7f25d0196680 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00d950 tx=0x7f25cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.984+0000 7f25c6ffd640 1 -- 172.21.15.83:0/2080054710 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc009c30 con 0x7f25d0101310 2024-03-11T19:37:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.984+0000 7f25c6ffd640 1 -- 172.21.15.83:0/2080054710 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25cc009d90 con 0x7f25d0101310 2024-03-11T19:37:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.984+0000 7f25c6ffd640 1 -- 172.21.15.83:0/2080054710 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00f7e0 con 0x7f25d0101310 2024-03-11T19:37:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.984+0000 7f25d7e15640 1 -- 172.21.15.83:0/2080054710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d019e240 con 0x7f25d0101310 2024-03-11T19:37:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.984+0000 7f25d7e15640 1 -- 172.21.15.83:0/2080054710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d019e750 con 0x7f25d0101310 2024-03-11T19:37:05.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.985+0000 7f25d7e15640 1 -- 172.21.15.83:0/2080054710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d0107440 con 0x7f25d0101310 2024-03-11T19:37:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.986+0000 7f25c6ffd640 1 -- 172.21.15.83:0/2080054710 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f25cc010ad0 con 0x7f25d0101310 2024-03-11T19:37:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.986+0000 7f25c6ffd640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f25bc0a1a30 0x7f25bc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.987+0000 7f25d5bb1640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f25bc0a1a30 0x7f25bc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.987+0000 7f25c6ffd640 1 -- 172.21.15.83:0/2080054710 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f25cc0ccfa0 con 0x7f25d0101310 2024-03-11T19:37:05.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.987+0000 7f25d5bb1640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f25bc0a1a30 0x7f25bc0a3ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f25c002c6a0 tx=0x7f25c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:05.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:05.990+0000 7f25c6ffd640 1 -- 172.21.15.83:0/2080054710 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f25cc096300 con 0x7f25d0101310 2024-03-11T19:37:06.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.165+0000 7f25d7e15640 1 -- 172.21.15.83:0/2080054710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25d010bc90 con 0x7f25d0101310 2024-03-11T19:37:06.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.167+0000 7f25c6ffd640 1 -- 172.21.15.83:0/2080054710 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22245 (secure 0 0 0) 0x7f25d010bc90 con 0x7f25d0101310 2024-03-11T19:37:06.171 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:06.171 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":84,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:37:04.916427+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:37:06.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 -- 172.21.15.83:0/2080054710 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f25bc0a1a30 msgr2=0x7f25bc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:06.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f25bc0a1a30 0x7f25bc0a3ef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f25c002c6a0 tx=0x7f25c0000bc0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 -- 172.21.15.83:0/2080054710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d0101310 msgr2=0x7f25d0196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:06.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d0101310 0x7f25d0196680 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00d950 tx=0x7f25cc00de10 comp rx=0 tx=0).stop 2024-03-11T19:37:06.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 -- 172.21.15.83:0/2080054710 shutdown_connections 2024-03-11T19:37:06.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f25bc0a1a30 0x7f25bc0a3ef0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25d0101cd0 0x7f25d019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25d0101310 0x7f25d0196680 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 --2- 172.21.15.83:0/2080054710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25d0100120 0x7f25d0196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 -- 172.21.15.83:0/2080054710 >> 172.21.15.83:0/2080054710 conn(0x7f25d00fb6b0 msgr2=0x7f25d0104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:06.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 -- 172.21.15.83:0/2080054710 shutdown_connections 2024-03-11T19:37:06.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.169+0000 7f25d7e15640 1 -- 172.21.15.83:0/2080054710 wait complete. 2024-03-11T19:37:06.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-11T19:37:06.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.284+0000 7f05045d0640 1 -- 172.21.15.83:0/216998892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04fc102d60 msgr2=0x7f04fc1095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:06.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.284+0000 7f05045d0640 1 --2- 172.21.15.83:0/216998892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04fc102d60 0x7f04fc1095f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009ca0 tx=0x7f04f402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.284+0000 7f05045d0640 1 -- 172.21.15.83:0/216998892 shutdown_connections 2024-03-11T19:37:06.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.284+0000 7f05045d0640 1 --2- 172.21.15.83:0/216998892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04fc102d60 0x7f04fc1095f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.284+0000 7f05045d0640 1 --2- 172.21.15.83:0/216998892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04fc1023a0 0x7f04fc102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.284+0000 7f05045d0640 1 --2- 172.21.15.83:0/216998892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04fc1011b0 0x7f04fc1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.284+0000 7f05045d0640 1 -- 172.21.15.83:0/216998892 >> 172.21.15.83:0/216998892 conn(0x7f04fc0fc720 msgr2=0x7f04fc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:06.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.284+0000 7f05045d0640 1 -- 172.21.15.83:0/216998892 shutdown_connections 2024-03-11T19:37:06.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.284+0000 7f05045d0640 1 -- 172.21.15.83:0/216998892 wait complete. 2024-03-11T19:37:06.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f05045d0640 1 Processor -- start 2024-03-11T19:37:06.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f05045d0640 1 -- start start 2024-03-11T19:37:06.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f05045d0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04fc1011b0 0x7f04fc197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f05045d0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04fc1023a0 0x7f04fc197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f05045d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04fc102d60 0x7f04fc19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f05045d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc19ecf0 con 0x7f04fc102d60 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f05045d0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc19ee60 con 0x7f04fc1023a0 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f05045d0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc19f020 con 0x7f04fc1011b0 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f0502b6d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04fc102d60 0x7f04fc19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f0502b6d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04fc102d60 0x7f04fc19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44152/0 (socket says 172.21.15.83:44152) 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f0502b6d640 1 -- 172.21.15.83:0/4047296501 learned_addr learned my addr 172.21.15.83:0/4047296501 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f050236c640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04fc1011b0 0x7f04fc197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f0501b6b640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04fc1023a0 0x7f04fc197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f0502b6d640 1 -- 172.21.15.83:0/4047296501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04fc1011b0 msgr2=0x7f04fc197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f0502b6d640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04fc1011b0 0x7f04fc197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f0502b6d640 1 -- 172.21.15.83:0/4047296501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04fc1023a0 msgr2=0x7f04fc197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:06.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f0502b6d640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04fc1023a0 0x7f04fc197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f0502b6d640 1 -- 172.21.15.83:0/4047296501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f4009900 con 0x7f04fc102d60 2024-03-11T19:37:06.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.285+0000 7f050236c640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04fc1011b0 0x7f04fc197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:06.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.286+0000 7f0502b6d640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04fc102d60 0x7f04fc19e7b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f04f4037040 tx=0x7f04f4034d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:06.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.286+0000 7f04f37fe640 1 -- 172.21.15.83:0/4047296501 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f4003970 con 0x7f04fc102d60 2024-03-11T19:37:06.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.286+0000 7f04f37fe640 1 -- 172.21.15.83:0/4047296501 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04f4004040 con 0x7f04fc102d60 2024-03-11T19:37:06.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.286+0000 7f05045d0640 1 -- 172.21.15.83:0/4047296501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04fc19f2a0 con 0x7f04fc102d60 2024-03-11T19:37:06.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.286+0000 7f04f37fe640 1 -- 172.21.15.83:0/4047296501 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f40406b0 con 0x7f04fc102d60 2024-03-11T19:37:06.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.286+0000 7f05045d0640 1 -- 172.21.15.83:0/4047296501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04fc19f750 con 0x7f04fc102d60 2024-03-11T19:37:06.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.287+0000 7f05045d0640 1 -- 172.21.15.83:0/4047296501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04fc1084d0 con 0x7f04fc102d60 2024-03-11T19:37:06.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.288+0000 7f04f37fe640 1 -- 172.21.15.83:0/4047296501 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f04f400a040 con 0x7f04fc102d60 2024-03-11T19:37:06.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.289+0000 7f04f37fe640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f04e80a1a30 0x7f04e80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:06.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.289+0000 7f050236c640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f04e80a1a30 0x7f04e80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:06.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.289+0000 7f04f37fe640 1 -- 172.21.15.83:0/4047296501 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f04f40f0960 con 0x7f04fc102d60 2024-03-11T19:37:06.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.289+0000 7f050236c640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f04e80a1a30 0x7f04e80a3ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f04fc102200 tx=0x7f04f800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:06.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.292+0000 7f04f37fe640 1 -- 172.21.15.83:0/4047296501 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f04f40b9c60 con 0x7f04fc102d60 2024-03-11T19:37:06.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.533+0000 7f05045d0640 1 -- 172.21.15.83:0/4047296501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f04fc10cb80 con 0x7f04fc102d60 2024-03-11T19:37:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.585+0000 7f04f37fe640 1 -- 172.21.15.83:0/4047296501 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f04f40bd320 con 0x7f04fc102d60 2024-03-11T19:37:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 -- 172.21.15.83:0/4047296501 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f04e80a1a30 msgr2=0x7f04e80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f04e80a1a30 0x7f04e80a3ef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f04fc102200 tx=0x7f04f800a560 comp rx=0 tx=0).stop 2024-03-11T19:37:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 -- 172.21.15.83:0/4047296501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04fc102d60 msgr2=0x7f04fc19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04fc102d60 0x7f04fc19e7b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f04f4037040 tx=0x7f04f4034d20 comp rx=0 tx=0).stop 2024-03-11T19:37:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 -- 172.21.15.83:0/4047296501 shutdown_connections 2024-03-11T19:37:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f04e80a1a30 0x7f04e80a3ef0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04fc102d60 0x7f04fc19e7b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04fc1023a0 0x7f04fc197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 --2- 172.21.15.83:0/4047296501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04fc1011b0 0x7f04fc197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 -- 172.21.15.83:0/4047296501 >> 172.21.15.83:0/4047296501 conn(0x7f04fc0fc720 msgr2=0x7f04fc1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 -- 172.21.15.83:0/4047296501 shutdown_connections 2024-03-11T19:37:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.587+0000 7f05045d0640 1 -- 172.21.15.83:0/4047296501 wait complete. 2024-03-11T19:37:06.602 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-03-11T19:37:06.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-11T19:37:06.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.704+0000 7f3de7e39640 1 -- 172.21.15.83:0/1910774225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00ffbb0 msgr2=0x7f3de0109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:06.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.704+0000 7f3de7e39640 1 --2- 172.21.15.83:0/1910774225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00ffbb0 0x7f3de0109760 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009ca0 tx=0x7f3dd802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 -- 172.21.15.83:0/1910774225 shutdown_connections 2024-03-11T19:37:06.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 --2- 172.21.15.83:0/1910774225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00ffbb0 0x7f3de0109760 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 --2- 172.21.15.83:0/1910774225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de00ff1f0 0x7f3de00ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 --2- 172.21.15.83:0/1910774225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de0102780 0x7f3de0102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 -- 172.21.15.83:0/1910774225 >> 172.21.15.83:0/1910774225 conn(0x7f3de00fadb0 msgr2=0x7f3de00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:06.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 -- 172.21.15.83:0/1910774225 shutdown_connections 2024-03-11T19:37:06.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 -- 172.21.15.83:0/1910774225 wait complete. 2024-03-11T19:37:06.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 Processor -- start 2024-03-11T19:37:06.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 -- start start 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00ff1f0 0x7f3de01972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de00ffbb0 0x7f3de01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de0102780 0x7f3de019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de019eda0 con 0x7f3de00ff1f0 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de019ef10 con 0x7f3de0102780 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.705+0000 7f3de7e39640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de019f0d0 con 0x7f3de00ffbb0 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de5bd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00ff1f0 0x7f3de01972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de5bd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00ff1f0 0x7f3de01972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44170/0 (socket says 172.21.15.83:44170) 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de5bd5640 1 -- 172.21.15.83:0/740055944 learned_addr learned my addr 172.21.15.83:0/740055944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de53d4640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de00ffbb0 0x7f3de01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de5bd5640 1 -- 172.21.15.83:0/740055944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de00ffbb0 msgr2=0x7f3de01977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de5bd5640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de00ffbb0 0x7f3de01977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de5bd5640 1 -- 172.21.15.83:0/740055944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de0102780 msgr2=0x7f3de019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:06.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de5bd5640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de0102780 0x7f3de019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de5bd5640 1 -- 172.21.15.83:0/740055944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd8009900 con 0x7f3de00ff1f0 2024-03-11T19:37:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de53d4640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de00ffbb0 0x7f3de01977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de5bd5640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00ff1f0 0x7f3de01972b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00b450 tx=0x7f3ddc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3dd6ffd640 1 -- 172.21.15.83:0/740055944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc011890 con 0x7f3de00ff1f0 2024-03-11T19:37:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3dd6ffd640 1 -- 172.21.15.83:0/740055944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ddc0103e0 con 0x7f3de00ff1f0 2024-03-11T19:37:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de7e39640 1 -- 172.21.15.83:0/740055944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de019f3b0 con 0x7f3de00ff1f0 2024-03-11T19:37:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3dd6ffd640 1 -- 172.21.15.83:0/740055944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00f580 con 0x7f3de00ff1f0 2024-03-11T19:37:06.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.706+0000 7f3de7e39640 1 -- 172.21.15.83:0/740055944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de019f940 con 0x7f3de00ff1f0 2024-03-11T19:37:06.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.707+0000 7f3de7e39640 1 -- 172.21.15.83:0/740055944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de01085e0 con 0x7f3de00ff1f0 2024-03-11T19:37:06.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.708+0000 7f3dd6ffd640 1 -- 172.21.15.83:0/740055944 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3ddc0108e0 con 0x7f3de00ff1f0 2024-03-11T19:37:06.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.709+0000 7f3dd6ffd640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3dcc0a1a30 0x7f3dcc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:06.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.709+0000 7f3de53d4640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3dcc0a1a30 0x7f3dcc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:06.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.709+0000 7f3dd6ffd640 1 -- 172.21.15.83:0/740055944 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3ddc0cd0f0 con 0x7f3de00ff1f0 2024-03-11T19:37:06.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.709+0000 7f3de53d4640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3dcc0a1a30 0x7f3dcc0a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3de01987c0 tx=0x7f3dd0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:06.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.713+0000 7f3dd6ffd640 1 -- 172.21.15.83:0/740055944 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3ddc096450 con 0x7f3de00ff1f0 2024-03-11T19:37:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:06 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/950576948' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-11T19:37:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:06 smithi083 ceph-mon[31864]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-11T19:37:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:06 smithi083 ceph-mon[31864]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-11T19:37:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:06 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/380420531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:06 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2080054710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:06 smithi083 ceph-mon[31864]: osdmap e85: 12 total, 12 up, 12 in 2024-03-11T19:37:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:06 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4047296501' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:37:06.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:06 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/950576948' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-11T19:37:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:06 smithi137 ceph-mon[36220]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-11T19:37:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:06 smithi137 ceph-mon[36220]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-11T19:37:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:06 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/380420531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:06 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2080054710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:06 smithi137 ceph-mon[36220]: osdmap e85: 12 total, 12 up, 12 in 2024-03-11T19:37:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:06 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4047296501' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:37:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:06 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/950576948' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-11T19:37:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:06 smithi204 ceph-mon[36739]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-11T19:37:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:06 smithi204 ceph-mon[36739]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-11T19:37:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:06 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/380420531' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:06 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2080054710' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:06 smithi204 ceph-mon[36739]: osdmap e85: 12 total, 12 up, 12 in 2024-03-11T19:37:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:06 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4047296501' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-11T19:37:06.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:06.924+0000 7f3de7e39640 1 -- 172.21.15.83:0/740055944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f3de002cea0 con 0x7f3de00ff1f0 2024-03-11T19:37:07.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.595+0000 7f3dd6ffd640 1 -- 172.21.15.83:0/740055944 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3ddc099b10 con 0x7f3de00ff1f0 2024-03-11T19:37:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.597+0000 7f3de7e39640 1 -- 172.21.15.83:0/740055944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3dcc0a1a30 msgr2=0x7f3dcc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.597+0000 7f3de7e39640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3dcc0a1a30 0x7f3dcc0a3ef0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3de01987c0 tx=0x7f3dd0008040 comp rx=0 tx=0).stop 2024-03-11T19:37:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.597+0000 7f3de7e39640 1 -- 172.21.15.83:0/740055944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00ff1f0 msgr2=0x7f3de01972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.597+0000 7f3de7e39640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00ff1f0 0x7f3de01972b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00b450 tx=0x7f3ddc00b910 comp rx=0 tx=0).stop 2024-03-11T19:37:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.597+0000 7f3de7e39640 1 -- 172.21.15.83:0/740055944 shutdown_connections 2024-03-11T19:37:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.597+0000 7f3de7e39640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3dcc0a1a30 0x7f3dcc0a3ef0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.597+0000 7f3de7e39640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3de0102780 0x7f3de019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.597+0000 7f3de7e39640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3de00ffbb0 0x7f3de01977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.597+0000 7f3de7e39640 1 --2- 172.21.15.83:0/740055944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3de00ff1f0 0x7f3de01972b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.597+0000 7f3de7e39640 1 -- 172.21.15.83:0/740055944 >> 172.21.15.83:0/740055944 conn(0x7f3de00fadb0 msgr2=0x7f3de01057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.597+0000 7f3de7e39640 1 -- 172.21.15.83:0/740055944 shutdown_connections 2024-03-11T19:37:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.598+0000 7f3de7e39640 1 -- 172.21.15.83:0/740055944 wait complete. 2024-03-11T19:37:07.617 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-11T19:37:07.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.721+0000 7fad63b48640 1 -- 172.21.15.83:0/348191624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5c0ff460 msgr2=0x7fad5c0ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:07.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.721+0000 7fad63b48640 1 --2- 172.21.15.83:0/348191624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5c0ff460 0x7fad5c0ff880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fad58009900 tx=0x7fad5802bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:07.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.721+0000 7fad63b48640 1 -- 172.21.15.83:0/348191624 shutdown_connections 2024-03-11T19:37:07.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.721+0000 7fad63b48640 1 --2- 172.21.15.83:0/348191624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad5c102df0 0x7fad5c109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:07.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.721+0000 7fad63b48640 1 --2- 172.21.15.83:0/348191624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad5c102430 0x7fad5c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:07.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.721+0000 7fad63b48640 1 --2- 172.21.15.83:0/348191624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5c0ff460 0x7fad5c0ff880 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:07.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.721+0000 7fad63b48640 1 -- 172.21.15.83:0/348191624 >> 172.21.15.83:0/348191624 conn(0x7fad5c0fad70 msgr2=0x7fad5c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:07.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.721+0000 7fad63b48640 1 -- 172.21.15.83:0/348191624 shutdown_connections 2024-03-11T19:37:07.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.721+0000 7fad63b48640 1 -- 172.21.15.83:0/348191624 wait complete. 2024-03-11T19:37:07.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad63b48640 1 Processor -- start 2024-03-11T19:37:07.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad63b48640 1 -- start start 2024-03-11T19:37:07.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad63b48640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5c0ff460 0x7fad5c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad63b48640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad5c102430 0x7fad5c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad63b48640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad5c102df0 0x7fad5c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad63b48640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c19eda0 con 0x7fad5c0ff460 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad63b48640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c19ef10 con 0x7fad5c102df0 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad63b48640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c19f0d0 con 0x7fad5c102430 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad618e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5c0ff460 0x7fad5c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad618e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5c0ff460 0x7fad5c1972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44190/0 (socket says 172.21.15.83:44190) 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad618e4640 1 -- 172.21.15.83:0/3183701508 learned_addr learned my addr 172.21.15.83:0/3183701508 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad610e3640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad5c102430 0x7fad5c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad620e5640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad5c102df0 0x7fad5c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad618e4640 1 -- 172.21.15.83:0/3183701508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad5c102430 msgr2=0x7fad5c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad618e4640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad5c102430 0x7fad5c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:07.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad618e4640 1 -- 172.21.15.83:0/3183701508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad5c102df0 msgr2=0x7fad5c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:07.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad618e4640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad5c102df0 0x7fad5c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:07.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad618e4640 1 -- 172.21.15.83:0/3183701508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad58009560 con 0x7fad5c0ff460 2024-03-11T19:37:07.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad620e5640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad5c102df0 0x7fad5c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:37:07.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad618e4640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5c0ff460 0x7fad5c1972b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fad58000c00 tx=0x7fad58004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:07.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad52ffd640 1 -- 172.21.15.83:0/3183701508 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58004df0 con 0x7fad5c0ff460 2024-03-11T19:37:07.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.722+0000 7fad52ffd640 1 -- 172.21.15.83:0/3183701508 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad58005430 con 0x7fad5c0ff460 2024-03-11T19:37:07.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.723+0000 7fad63b48640 1 -- 172.21.15.83:0/3183701508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c19f350 con 0x7fad5c0ff460 2024-03-11T19:37:07.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.723+0000 7fad52ffd640 1 -- 172.21.15.83:0/3183701508 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5803e450 con 0x7fad5c0ff460 2024-03-11T19:37:07.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.723+0000 7fad63b48640 1 -- 172.21.15.83:0/3183701508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c19f8e0 con 0x7fad5c0ff460 2024-03-11T19:37:07.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.723+0000 7fad63b48640 1 -- 172.21.15.83:0/3183701508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c108580 con 0x7fad5c0ff460 2024-03-11T19:37:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.725+0000 7fad52ffd640 1 -- 172.21.15.83:0/3183701508 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fad580055a0 con 0x7fad5c0ff460 2024-03-11T19:37:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.725+0000 7fad52ffd640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad480a1a30 0x7fad480a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.725+0000 7fad610e3640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad480a1a30 0x7fad480a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.725+0000 7fad52ffd640 1 -- 172.21.15.83:0/3183701508 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fad580efd10 con 0x7fad5c0ff460 2024-03-11T19:37:07.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.726+0000 7fad610e3640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad480a1a30 0x7fad480a3ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad5c1987c0 tx=0x7fad4c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:07.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.729+0000 7fad52ffd640 1 -- 172.21.15.83:0/3183701508 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad580b9070 con 0x7fad5c0ff460 2024-03-11T19:37:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:07 smithi137 ceph-mon[36220]: pgmap v284: 97 pgs: 32 active+clean, 9 creating+peering, 56 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:07 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4047296501' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-11T19:37:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:07 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] up:active 2024-03-11T19:37:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:07 smithi137 ceph-mon[36220]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:37:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:07 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/740055944' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:37:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:07 smithi137 ceph-mon[36220]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:37:07.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:07 smithi137 ceph-mon[36220]: Cluster is now healthy 2024-03-11T19:37:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:07 smithi204 ceph-mon[36739]: pgmap v284: 97 pgs: 32 active+clean, 9 creating+peering, 56 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:07 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4047296501' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-11T19:37:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:07 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] up:active 2024-03-11T19:37:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:07 smithi204 ceph-mon[36739]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:37:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:07 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/740055944' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:37:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:07 smithi204 ceph-mon[36739]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:37:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:07 smithi204 ceph-mon[36739]: Cluster is now healthy 2024-03-11T19:37:07.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:07.939+0000 7fad63b48640 1 -- 172.21.15.83:0/3183701508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fad5c10cc00 con 0x7fad5c0ff460 2024-03-11T19:37:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:07 smithi083 ceph-mon[31864]: pgmap v284: 97 pgs: 32 active+clean, 9 creating+peering, 56 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-11T19:37:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:07 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4047296501' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-11T19:37:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:07 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] up:active 2024-03-11T19:37:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:07 smithi083 ceph-mon[31864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:37:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:07 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/740055944' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-11T19:37:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:07 smithi083 ceph-mon[31864]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-11T19:37:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:07 smithi083 ceph-mon[31864]: Cluster is now healthy 2024-03-11T19:37:08.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.610+0000 7fad52ffd640 1 -- 172.21.15.83:0/3183701508 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7fad580bc730 con 0x7fad5c0ff460 2024-03-11T19:37:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 -- 172.21.15.83:0/3183701508 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad480a1a30 msgr2=0x7fad480a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad480a1a30 0x7fad480a3ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad5c1987c0 tx=0x7fad4c008040 comp rx=0 tx=0).stop 2024-03-11T19:37:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 -- 172.21.15.83:0/3183701508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5c0ff460 msgr2=0x7fad5c1972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5c0ff460 0x7fad5c1972b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fad58000c00 tx=0x7fad58004270 comp rx=0 tx=0).stop 2024-03-11T19:37:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 -- 172.21.15.83:0/3183701508 shutdown_connections 2024-03-11T19:37:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad480a1a30 0x7fad480a3ef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad5c102df0 0x7fad5c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad5c102430 0x7fad5c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 --2- 172.21.15.83:0/3183701508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad5c0ff460 0x7fad5c1972b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 -- 172.21.15.83:0/3183701508 >> 172.21.15.83:0/3183701508 conn(0x7fad5c0fad70 msgr2=0x7fad5c105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 -- 172.21.15.83:0/3183701508 shutdown_connections 2024-03-11T19:37:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.613+0000 7fad63b48640 1 -- 172.21.15.83:0/3183701508 wait complete. 2024-03-11T19:37:08.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-11T19:37:08.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.739+0000 7f842c676640 1 -- 172.21.15.83:0/2962245069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8424101c20 msgr2=0x7f8424104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.739+0000 7f842c676640 1 --2- 172.21.15.83:0/2962245069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8424101c20 0x7f8424104010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8420009900 tx=0x7f842002bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.739+0000 7f842c676640 1 -- 172.21.15.83:0/2962245069 shutdown_connections 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.739+0000 7f842c676640 1 --2- 172.21.15.83:0/2962245069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8424106e80 0x7f84241092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.739+0000 7f842c676640 1 --2- 172.21.15.83:0/2962245069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8424104550 0x7f8424106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.739+0000 7f842c676640 1 --2- 172.21.15.83:0/2962245069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8424101c20 0x7f8424104010 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.739+0000 7f842c676640 1 -- 172.21.15.83:0/2962245069 >> 172.21.15.83:0/2962245069 conn(0x7f84240fb630 msgr2=0x7f84240fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.739+0000 7f842c676640 1 -- 172.21.15.83:0/2962245069 shutdown_connections 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.739+0000 7f842c676640 1 -- 172.21.15.83:0/2962245069 wait complete. 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842c676640 1 Processor -- start 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842c676640 1 -- start start 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842c676640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8424101c20 0x7f8424197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:08.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842c676640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8424104550 0x7f8424197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842c676640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8424106e80 0x7f842419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842c676640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842419eef0 con 0x7f8424101c20 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842c676640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842419f060 con 0x7f8424104550 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842c676640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842419f220 con 0x7f8424106e80 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842a412640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8424101c20 0x7f8424197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842a412640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8424101c20 0x7f8424197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44212/0 (socket says 172.21.15.83:44212) 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842a412640 1 -- 172.21.15.83:0/3206528478 learned_addr learned my addr 172.21.15.83:0/3206528478 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f8429c11640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8424104550 0x7f8424197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842ac13640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8424106e80 0x7f842419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842a412640 1 -- 172.21.15.83:0/3206528478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8424106e80 msgr2=0x7f842419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842a412640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8424106e80 0x7f842419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842a412640 1 -- 172.21.15.83:0/3206528478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8424104550 msgr2=0x7f8424197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:08.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842a412640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8424104550 0x7f8424197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f842a412640 1 -- 172.21.15.83:0/3206528478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8420009560 con 0x7f8424101c20 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.740+0000 7f8429c11640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8424104550 0x7f8424197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.741+0000 7f842a412640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8424101c20 0x7f8424197450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8420005ab0 tx=0x7f84200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.741+0000 7f841b7fe640 1 -- 172.21.15.83:0/3206528478 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84200044e0 con 0x7f8424101c20 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.741+0000 7f841b7fe640 1 -- 172.21.15.83:0/3206528478 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84200054e0 con 0x7f8424101c20 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.741+0000 7f842c676640 1 -- 172.21.15.83:0/3206528478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842419f4a0 con 0x7f8424101c20 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.741+0000 7f841b7fe640 1 -- 172.21.15.83:0/3206528478 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842003e450 con 0x7f8424101c20 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.741+0000 7f842c676640 1 -- 172.21.15.83:0/3206528478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842419fa30 con 0x7f8424101c20 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.742+0000 7f842c676640 1 -- 172.21.15.83:0/3206528478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84240fd120 con 0x7f8424101c20 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.743+0000 7f841b7fe640 1 -- 172.21.15.83:0/3206528478 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f84200035f0 con 0x7f8424101c20 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.744+0000 7f841b7fe640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f84100a1a30 0x7f84100a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.744+0000 7f8429c11640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f84100a1a30 0x7f84100a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.744+0000 7f841b7fe640 1 -- 172.21.15.83:0/3206528478 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f84200ef8d0 con 0x7f8424101c20 2024-03-11T19:37:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.744+0000 7f8429c11640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f84100a1a30 0x7f84100a3ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8424198910 tx=0x7f8414008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:08.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.748+0000 7f841b7fe640 1 -- 172.21.15.83:0/3206528478 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f84200b8c30 con 0x7f8424101c20 2024-03-11T19:37:08.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:08 smithi137 ceph-mon[36220]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-11T19:37:08.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:08 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/740055944' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-11T19:37:08.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:08 smithi137 ceph-mon[36220]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-11T19:37:08.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:08 smithi137 ceph-mon[36220]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:37:08.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:08 smithi137 ceph-mon[36220]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-11T19:37:08.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:08 smithi137 ceph-mon[36220]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-11T19:37:08.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:08 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3183701508' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:37:08.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:08 smithi204 ceph-mon[36739]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-11T19:37:08.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:08 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/740055944' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-11T19:37:08.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:08 smithi204 ceph-mon[36739]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-11T19:37:08.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:08 smithi204 ceph-mon[36739]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:37:08.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:08 smithi204 ceph-mon[36739]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-11T19:37:08.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:08 smithi204 ceph-mon[36739]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-11T19:37:08.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:08 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3183701508' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:37:08.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:08.958+0000 7f842c676640 1 -- 172.21.15.83:0/3206528478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f842402cdd0 con 0x7f8424101c20 2024-03-11T19:37:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:08 smithi083 ceph-mon[31864]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-11T19:37:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:08 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/740055944' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-11T19:37:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:08 smithi083 ceph-mon[31864]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-11T19:37:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:08 smithi083 ceph-mon[31864]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-11T19:37:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:08 smithi083 ceph-mon[31864]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-11T19:37:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:08 smithi083 ceph-mon[31864]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-11T19:37:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:08 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3183701508' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-11T19:37:09.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.616+0000 7f841b7fe640 1 -- 172.21.15.83:0/3206528478 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7f84200bc2f0 con 0x7f8424101c20 2024-03-11T19:37:09.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 -- 172.21.15.83:0/3206528478 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f84100a1a30 msgr2=0x7f84100a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:09.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f84100a1a30 0x7f84100a3ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8424198910 tx=0x7f8414008040 comp rx=0 tx=0).stop 2024-03-11T19:37:09.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 -- 172.21.15.83:0/3206528478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8424101c20 msgr2=0x7f8424197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:09.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8424101c20 0x7f8424197450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8420005ab0 tx=0x7f84200042f0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 -- 172.21.15.83:0/3206528478 shutdown_connections 2024-03-11T19:37:09.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f84100a1a30 0x7f84100a3ef0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8424106e80 0x7f842419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8424104550 0x7f8424197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 --2- 172.21.15.83:0/3206528478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8424101c20 0x7f8424197450 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 -- 172.21.15.83:0/3206528478 >> 172.21.15.83:0/3206528478 conn(0x7f84240fb630 msgr2=0x7f8424102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 -- 172.21.15.83:0/3206528478 shutdown_connections 2024-03-11T19:37:09.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.618+0000 7f842c676640 1 -- 172.21.15.83:0/3206528478 wait complete. 2024-03-11T19:37:09.634 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-03-11T19:37:09.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-11T19:37:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.735+0000 7fc7828d4640 1 -- 172.21.15.83:0/3055536831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c0ffbb0 msgr2=0x7fc77c109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.735+0000 7fc7828d4640 1 --2- 172.21.15.83:0/3055536831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c0ffbb0 0x7fc77c109760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc774009ca0 tx=0x7fc77402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.735+0000 7fc7828d4640 1 -- 172.21.15.83:0/3055536831 shutdown_connections 2024-03-11T19:37:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.735+0000 7fc7828d4640 1 --2- 172.21.15.83:0/3055536831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c0ffbb0 0x7fc77c109760 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.735+0000 7fc7828d4640 1 --2- 172.21.15.83:0/3055536831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c0ff1f0 0x7fc77c0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.735+0000 7fc7828d4640 1 --2- 172.21.15.83:0/3055536831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c102780 0x7fc77c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.735+0000 7fc7828d4640 1 -- 172.21.15.83:0/3055536831 >> 172.21.15.83:0/3055536831 conn(0x7fc77c0fad90 msgr2=0x7fc77c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc7828d4640 1 -- 172.21.15.83:0/3055536831 shutdown_connections 2024-03-11T19:37:09.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc7828d4640 1 -- 172.21.15.83:0/3055536831 wait complete. 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc7828d4640 1 Processor -- start 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc7828d4640 1 -- start start 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc7828d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c0ff1f0 0x7fc77c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc7828d4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c0ffbb0 0x7fc77c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc7828d4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c102780 0x7fc77c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc7828d4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c19ee00 con 0x7fc77c0ff1f0 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc7828d4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c19ef70 con 0x7fc77c0ffbb0 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc7828d4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c19f130 con 0x7fc77c102780 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc77bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c0ff1f0 0x7fc77c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.736+0000 7fc77bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c0ff1f0 0x7fc77c197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60492/0 (socket says 172.21.15.83:60492) 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc77bfff640 1 -- 172.21.15.83:0/1383246846 learned_addr learned my addr 172.21.15.83:0/1383246846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:09.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc780e71640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c102780 0x7fc77c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc77bfff640 1 -- 172.21.15.83:0/1383246846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c102780 msgr2=0x7fc77c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc77bfff640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c102780 0x7fc77c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc77bfff640 1 -- 172.21.15.83:0/1383246846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c0ffbb0 msgr2=0x7fc77c197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc77bfff640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c0ffbb0 0x7fc77c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc77bfff640 1 -- 172.21.15.83:0/1383246846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc774009900 con 0x7fc77c0ff1f0 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc780e71640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c102780 0x7fc77c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc77bfff640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c0ff1f0 0x7fc77c197310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc77000b450 tx=0x7fc77000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc7797fa640 1 -- 172.21.15.83:0/1383246846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770011890 con 0x7fc77c0ff1f0 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc7828d4640 1 -- 172.21.15.83:0/1383246846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c19f410 con 0x7fc77c0ff1f0 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc7797fa640 1 -- 172.21.15.83:0/1383246846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7700103e0 con 0x7fc77c0ff1f0 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc7797fa640 1 -- 172.21.15.83:0/1383246846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77000f580 con 0x7fc77c0ff1f0 2024-03-11T19:37:09.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.737+0000 7fc7828d4640 1 -- 172.21.15.83:0/1383246846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c19f950 con 0x7fc77c0ff1f0 2024-03-11T19:37:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.738+0000 7fc7828d4640 1 -- 172.21.15.83:0/1383246846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c1085e0 con 0x7fc77c0ff1f0 2024-03-11T19:37:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.739+0000 7fc7797fa640 1 -- 172.21.15.83:0/1383246846 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc7700108e0 con 0x7fc77c0ff1f0 2024-03-11T19:37:09.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.739+0000 7fc7797fa640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7640a1a30 0x7fc7640a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:09.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.740+0000 7fc77b7fe640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7640a1a30 0x7fc7640a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:09.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.740+0000 7fc7797fa640 1 -- 172.21.15.83:0/1383246846 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc7700cd150 con 0x7fc77c0ff1f0 2024-03-11T19:37:09.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.740+0000 7fc77b7fe640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7640a1a30 0x7fc7640a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc77c198820 tx=0x7fc76c006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:09.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.744+0000 7fc7797fa640 1 -- 172.21.15.83:0/1383246846 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc7700964b0 con 0x7fc77c0ff1f0 2024-03-11T19:37:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:09 smithi137 ceph-mon[36220]: pgmap v285: 97 pgs: 82 active+clean, 9 creating+peering, 6 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-03-11T19:37:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:09 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3183701508' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-11T19:37:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:09 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] up:active 2024-03-11T19:37:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:09 smithi137 ceph-mon[36220]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-11T19:37:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:09 smithi137 ceph-mon[36220]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-11T19:37:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:09 smithi137 ceph-mon[36220]: Cluster is now healthy 2024-03-11T19:37:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:09 smithi137 ceph-mon[36220]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-11T19:37:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:09 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-11T19:37:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:09 smithi137 ceph-mon[36220]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-11T19:37:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:09 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3206528478' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:37:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:09 smithi204 ceph-mon[36739]: pgmap v285: 97 pgs: 82 active+clean, 9 creating+peering, 6 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-03-11T19:37:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:09 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3183701508' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-11T19:37:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:09 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] up:active 2024-03-11T19:37:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:09 smithi204 ceph-mon[36739]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-11T19:37:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:09 smithi204 ceph-mon[36739]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-11T19:37:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:09 smithi204 ceph-mon[36739]: Cluster is now healthy 2024-03-11T19:37:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:09 smithi204 ceph-mon[36739]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-11T19:37:09.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:09 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-11T19:37:09.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:09 smithi204 ceph-mon[36739]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-11T19:37:09.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:09 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3206528478' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:37:09.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.940+0000 7fc7828d4640 1 -- 172.21.15.83:0/1383246846 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fc77c02ceb0 con 0x7fc7640a1a30 2024-03-11T19:37:09.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-mon[31864]: pgmap v285: 97 pgs: 82 active+clean, 9 creating+peering, 6 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-03-11T19:37:09.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3183701508' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-11T19:37:09.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] up:active 2024-03-11T19:37:09.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-mon[31864]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-11T19:37:09.950 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-mon[31864]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-11T19:37:09.951 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-mon[31864]: Cluster is now healthy 2024-03-11T19:37:09.951 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-mon[31864]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-11T19:37:09.951 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-11T19:37:09.951 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-mon[31864]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-11T19:37:09.951 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3206528478' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-11T19:37:09.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.966+0000 7fc7797fa640 1 -- 172.21.15.83:0/1383246846 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc77c02ceb0 con 0x7fc7640a1a30 2024-03-11T19:37:09.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 -- 172.21.15.83:0/1383246846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7640a1a30 msgr2=0x7fc7640a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:09.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7640a1a30 0x7fc7640a3ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc77c198820 tx=0x7fc76c006c90 comp rx=0 tx=0).stop 2024-03-11T19:37:09.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 -- 172.21.15.83:0/1383246846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c0ff1f0 msgr2=0x7fc77c197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:09.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c0ff1f0 0x7fc77c197310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc77000b450 tx=0x7fc77000b910 comp rx=0 tx=0).stop 2024-03-11T19:37:09.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 -- 172.21.15.83:0/1383246846 shutdown_connections 2024-03-11T19:37:09.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7640a1a30 0x7fc7640a3ef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc77c102780 0x7fc77c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c0ffbb0 0x7fc77c197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 --2- 172.21.15.83:0/1383246846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c0ff1f0 0x7fc77c197310 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:09.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 -- 172.21.15.83:0/1383246846 >> 172.21.15.83:0/1383246846 conn(0x7fc77c0fad90 msgr2=0x7fc77c1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:09.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 -- 172.21.15.83:0/1383246846 shutdown_connections 2024-03-11T19:37:09.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:09.968+0000 7fc7828d4640 1 -- 172.21.15.83:0/1383246846 wait complete. 2024-03-11T19:37:09.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-11T19:37:10.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.084+0000 7f550420e640 1 -- 172.21.15.83:0/1162765093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54fc101190 msgr2=0x7f54fc1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.084+0000 7f550420e640 1 --2- 172.21.15.83:0/1162765093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54fc101190 0x7f54fc1015b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009900 tx=0x7f54f802bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.084+0000 7f550420e640 1 -- 172.21.15.83:0/1162765093 shutdown_connections 2024-03-11T19:37:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.084+0000 7f550420e640 1 --2- 172.21.15.83:0/1162765093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc102d40 0x7f54fc1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.084+0000 7f550420e640 1 --2- 172.21.15.83:0/1162765093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54fc102380 0x7f54fc102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.084+0000 7f550420e640 1 --2- 172.21.15.83:0/1162765093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54fc101190 0x7f54fc1015b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.084+0000 7f550420e640 1 -- 172.21.15.83:0/1162765093 >> 172.21.15.83:0/1162765093 conn(0x7f54fc0fc720 msgr2=0x7f54fc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.084+0000 7f550420e640 1 -- 172.21.15.83:0/1162765093 shutdown_connections 2024-03-11T19:37:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.084+0000 7f550420e640 1 -- 172.21.15.83:0/1162765093 wait complete. 2024-03-11T19:37:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f550420e640 1 Processor -- start 2024-03-11T19:37:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f550420e640 1 -- start start 2024-03-11T19:37:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f550420e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54fc101190 0x7f54fc197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f550420e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc102380 0x7f54fc197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f550420e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54fc102d40 0x7f54fc19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f550420e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc19ed20 con 0x7f54fc102d40 2024-03-11T19:37:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f550420e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc19ee90 con 0x7f54fc101190 2024-03-11T19:37:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f550420e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc19f050 con 0x7f54fc102380 2024-03-11T19:37:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f55027ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54fc102d40 0x7f54fc19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f55027ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54fc102d40 0x7f54fc19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60520/0 (socket says 172.21.15.83:60520) 2024-03-11T19:37:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f55027ab640 1 -- 172.21.15.83:0/1909460090 learned_addr learned my addr 172.21.15.83:0/1909460090 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f55017a9640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc102380 0x7f54fc197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f5501faa640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54fc101190 0x7f54fc197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f55027ab640 1 -- 172.21.15.83:0/1909460090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc102380 msgr2=0x7f54fc197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f55027ab640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc102380 0x7f54fc197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f55027ab640 1 -- 172.21.15.83:0/1909460090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54fc101190 msgr2=0x7f54fc197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f55027ab640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54fc101190 0x7f54fc197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f55027ab640 1 -- 172.21.15.83:0/1909460090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f8009560 con 0x7f54fc102d40 2024-03-11T19:37:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.085+0000 7f5501faa640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54fc101190 0x7f54fc197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:37:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.086+0000 7f55027ab640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54fc102d40 0x7f54fc19e7e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f54f400eaa0 tx=0x7f54f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.086+0000 7f54f2ffd640 1 -- 172.21.15.83:0/1909460090 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f400cdb0 con 0x7f54fc102d40 2024-03-11T19:37:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.086+0000 7f550420e640 1 -- 172.21.15.83:0/1909460090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc19f330 con 0x7f54fc102d40 2024-03-11T19:37:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.086+0000 7f54f2ffd640 1 -- 172.21.15.83:0/1909460090 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54f4005020 con 0x7f54fc102d40 2024-03-11T19:37:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.086+0000 7f54f2ffd640 1 -- 172.21.15.83:0/1909460090 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f40116b0 con 0x7f54fc102d40 2024-03-11T19:37:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.086+0000 7f550420e640 1 -- 172.21.15.83:0/1909460090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc19f840 con 0x7f54fc102d40 2024-03-11T19:37:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.087+0000 7f550420e640 1 -- 172.21.15.83:0/1909460090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc108530 con 0x7f54fc102d40 2024-03-11T19:37:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.088+0000 7f54f2ffd640 1 -- 172.21.15.83:0/1909460090 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f54f4005520 con 0x7f54fc102d40 2024-03-11T19:37:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.088+0000 7f54f2ffd640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f54e80a1a30 0x7f54e80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.088+0000 7f5501faa640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f54e80a1a30 0x7f54e80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.088+0000 7f54f2ffd640 1 -- 172.21.15.83:0/1909460090 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f54f40cd8a0 con 0x7f54fc102d40 2024-03-11T19:37:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.089+0000 7f5501faa640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f54e80a1a30 0x7f54e80a3ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f54f8036040 tx=0x7f54f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.092+0000 7f54f2ffd640 1 -- 172.21.15.83:0/1909460090 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f54f4096c00 con 0x7f54fc102d40 2024-03-11T19:37:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.291+0000 7f550420e640 1 -- 172.21.15.83:0/1909460090 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f54fc02ceb0 con 0x7f54e80a1a30 2024-03-11T19:37:10.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.312+0000 7f54f2ffd640 1 -- 172.21.15.83:0/1909460090 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f54fc02ceb0 con 0x7f54e80a1a30 2024-03-11T19:37:10.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:37:09.945+0000 7f84cecab700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:37:10.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:37:09.945+0000 7f84cecab700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:37:10.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:37:09.945+0000 7f84cecab700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:37:10.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:37:09.945+0000 7f84cecab700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:37:10.318 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:37:09 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:37:09.945+0000 7f84cecab700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-11T19:37:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 -- 172.21.15.83:0/1909460090 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f54e80a1a30 msgr2=0x7f54e80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f54e80a1a30 0x7f54e80a3ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f54f8036040 tx=0x7f54f8000bc0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 -- 172.21.15.83:0/1909460090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54fc102d40 msgr2=0x7f54fc19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54fc102d40 0x7f54fc19e7e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f54f400eaa0 tx=0x7f54f400ef60 comp rx=0 tx=0).stop 2024-03-11T19:37:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 -- 172.21.15.83:0/1909460090 shutdown_connections 2024-03-11T19:37:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f54e80a1a30 0x7f54e80a3ef0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54fc102d40 0x7f54fc19e7e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc102380 0x7f54fc197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 --2- 172.21.15.83:0/1909460090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54fc101190 0x7f54fc197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 -- 172.21.15.83:0/1909460090 >> 172.21.15.83:0/1909460090 conn(0x7f54fc0fc720 msgr2=0x7f54fc105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 -- 172.21.15.83:0/1909460090 shutdown_connections 2024-03-11T19:37:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.314+0000 7f550420e640 1 -- 172.21.15.83:0/1909460090 wait complete. 2024-03-11T19:37:10.329 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-11T19:37:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/958522881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb04100360 msgr2=0x7ffb041007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 --2- 172.21.15.83:0/958522881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb04100360 0x7ffb041007e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009930 tx=0x7ffaf402bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/958522881 shutdown_connections 2024-03-11T19:37:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 --2- 172.21.15.83:0/958522881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04100d20 0x7ffb041085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 --2- 172.21.15.83:0/958522881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb04100360 0x7ffb041007e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 --2- 172.21.15.83:0/958522881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb04101630 0x7ffb04101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/958522881 >> 172.21.15.83:0/958522881 conn(0x7ffb040fadb0 msgr2=0x7ffb040fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/958522881 shutdown_connections 2024-03-11T19:37:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/958522881 wait complete. 2024-03-11T19:37:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 Processor -- start 2024-03-11T19:37:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 -- start start 2024-03-11T19:37:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb04100360 0x7ffb041961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb04100d20 0x7ffb041966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04101630 0x7ffb0419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0419dca0 con 0x7ffb04100360 2024-03-11T19:37:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0419de10 con 0x7ffb04100d20 2024-03-11T19:37:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb0bc9a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0419dfd0 con 0x7ffb04101630 2024-03-11T19:37:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb09a36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb04100360 0x7ffb041961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb09a36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb04100360 0x7ffb041961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60538/0 (socket says 172.21.15.83:60538) 2024-03-11T19:37:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.431+0000 7ffb09a36640 1 -- 172.21.15.83:0/1660570236 learned_addr learned my addr 172.21.15.83:0/1660570236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffb09235640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb04100d20 0x7ffb041966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffb0a237640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04101630 0x7ffb0419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffb09a36640 1 -- 172.21.15.83:0/1660570236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04101630 msgr2=0x7ffb0419d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffb09a36640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04101630 0x7ffb0419d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffb09a36640 1 -- 172.21.15.83:0/1660570236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb04100d20 msgr2=0x7ffb041966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffb09a36640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb04100d20 0x7ffb041966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffb09a36640 1 -- 172.21.15.83:0/1660570236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4009560 con 0x7ffb04100360 2024-03-11T19:37:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffb0a237640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04101630 0x7ffb0419d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:37:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffb09a36640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb04100360 0x7ffb041961b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c970 tx=0x7ffb0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:10.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffafaffd640 1 -- 172.21.15.83:0/1660570236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00004220 con 0x7ffb04100360 2024-03-11T19:37:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffafaffd640 1 -- 172.21.15.83:0/1660570236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb00007e90 con 0x7ffb04100360 2024-03-11T19:37:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/1660570236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0419e2b0 con 0x7ffb04100360 2024-03-11T19:37:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffafaffd640 1 -- 172.21.15.83:0/1660570236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0000f450 con 0x7ffb04100360 2024-03-11T19:37:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.432+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/1660570236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0419e7e0 con 0x7ffb04100360 2024-03-11T19:37:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.433+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/1660570236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb041074a0 con 0x7ffb04100360 2024-03-11T19:37:10.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.434+0000 7ffafaffd640 1 -- 172.21.15.83:0/1660570236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ffb00003610 con 0x7ffb04100360 2024-03-11T19:37:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.434+0000 7ffafaffd640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffaf00a1ab0 0x7ffaf00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.435+0000 7ffb09235640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffaf00a1ab0 0x7ffaf00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.435+0000 7ffafaffd640 1 -- 172.21.15.83:0/1660570236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffb000ccfd0 con 0x7ffb04100360 2024-03-11T19:37:10.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.435+0000 7ffb09235640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffaf00a1ab0 0x7ffaf00a3f70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009900 tx=0x7ffaf4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:10.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.439+0000 7ffafaffd640 1 -- 172.21.15.83:0/1660570236 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb00096330 con 0x7ffb04100360 2024-03-11T19:37:10.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.642+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/1660570236 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb0402ceb0 con 0x7ffaf00a1ab0 2024-03-11T19:37:10.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.646+0000 7ffafaffd640 1 -- 172.21.15.83:0/1660570236 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7ffb0402ceb0 con 0x7ffaf00a1ab0 2024-03-11T19:37:10.650 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_0/cf99f58b-bdc4-4a25-b6f4-8d6b1b8a55a8 2024-03-11T19:37:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/1660570236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffaf00a1ab0 msgr2=0x7ffaf00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffaf00a1ab0 0x7ffaf00a3f70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009900 tx=0x7ffaf4000bc0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/1660570236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb04100360 msgr2=0x7ffb041961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb04100360 0x7ffb041961b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c970 tx=0x7ffb0000ce30 comp rx=0 tx=0).stop 2024-03-11T19:37:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/1660570236 shutdown_connections 2024-03-11T19:37:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffaf00a1ab0 0x7ffaf00a3f70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb04101630 0x7ffb0419d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb04100d20 0x7ffb041966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 --2- 172.21.15.83:0/1660570236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb04100360 0x7ffb041961b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/1660570236 >> 172.21.15.83:0/1660570236 conn(0x7ffb040fadb0 msgr2=0x7ffb04104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/1660570236 shutdown_connections 2024-03-11T19:37:10.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.648+0000 7ffb0bc9a640 1 -- 172.21.15.83:0/1660570236 wait complete. 2024-03-11T19:37:10.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-11T19:37:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 -- 172.21.15.83:0/3740578978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f8102eb0 msgr2=0x7f29f8109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 --2- 172.21.15.83:0/3740578978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f8102eb0 0x7f29f8109740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009ca0 tx=0x7f29f002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 -- 172.21.15.83:0/3740578978 shutdown_connections 2024-03-11T19:37:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 --2- 172.21.15.83:0/3740578978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f8102eb0 0x7f29f8109740 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 --2- 172.21.15.83:0/3740578978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29f81024f0 0x7f29f8102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 --2- 172.21.15.83:0/3740578978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29f80ff480 0x7f29f80ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 -- 172.21.15.83:0/3740578978 >> 172.21.15.83:0/3740578978 conn(0x7f29f80fadb0 msgr2=0x7f29f80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 -- 172.21.15.83:0/3740578978 shutdown_connections 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 -- 172.21.15.83:0/3740578978 wait complete. 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 Processor -- start 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 -- start start 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29f80ff480 0x7f29f81971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f81024f0 0x7f29f81976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29f8102eb0 0x7f29f819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f819eca0 con 0x7f29f81024f0 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f819ee10 con 0x7f29f80ff480 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.765+0000 7f29fed77640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f819efd0 con 0x7f29f8102eb0 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f81024f0 0x7f29f81976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f81024f0 0x7f29f81976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60556/0 (socket says 172.21.15.83:60556) 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f7fff640 1 -- 172.21.15.83:0/1766982893 learned_addr learned my addr 172.21.15.83:0/1766982893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29fcb13640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29f80ff480 0x7f29f81971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f7fff640 1 -- 172.21.15.83:0/1766982893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29f8102eb0 msgr2=0x7f29f819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f7fff640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29f8102eb0 0x7f29f819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f7fff640 1 -- 172.21.15.83:0/1766982893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29f80ff480 msgr2=0x7f29f81971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f7fff640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29f80ff480 0x7f29f81971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f7fff640 1 -- 172.21.15.83:0/1766982893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f0009900 con 0x7f29f81024f0 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29fcb13640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29f80ff480 0x7f29f81971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f7fff640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f81024f0 0x7f29f81976f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f29e400b4a0 tx=0x7f29e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f5ffb640 1 -- 172.21.15.83:0/1766982893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4010610 con 0x7f29f81024f0 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29fed77640 1 -- 172.21.15.83:0/1766982893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f819f2b0 con 0x7f29f81024f0 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f5ffb640 1 -- 172.21.15.83:0/1766982893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29e4010c50 con 0x7f29f81024f0 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29f5ffb640 1 -- 172.21.15.83:0/1766982893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4017e20 con 0x7f29f81024f0 2024-03-11T19:37:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.766+0000 7f29fed77640 1 -- 172.21.15.83:0/1766982893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f819f7c0 con 0x7f29f81024f0 2024-03-11T19:37:10.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.767+0000 7f29fed77640 1 -- 172.21.15.83:0/1766982893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f8108550 con 0x7f29f81024f0 2024-03-11T19:37:10.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.768+0000 7f29f5ffb640 1 -- 172.21.15.83:0/1766982893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f29e4017480 con 0x7f29f81024f0 2024-03-11T19:37:10.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.769+0000 7f29f5ffb640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f29e80a1ab0 0x7f29e80a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:10.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.769+0000 7f29fcb13640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f29e80a1ab0 0x7f29e80a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:10.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.769+0000 7f29f5ffb640 1 -- 172.21.15.83:0/1766982893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f29e40cc950 con 0x7f29f81024f0 2024-03-11T19:37:10.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.770+0000 7f29fcb13640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f29e80a1ab0 0x7f29e80a3f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f29f8069510 tx=0x7f29ec005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:10.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.774+0000 7f29f5ffb640 1 -- 172.21.15.83:0/1766982893 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f29e4095cb0 con 0x7f29f81024f0 2024-03-11T19:37:10.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:10 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3206528478' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-11T19:37:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:10 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] up:active 2024-03-11T19:37:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:10 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-11T19:37:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:10 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:37:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:10 smithi137 ceph-mon[36220]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:10 smithi137 ceph-mon[36220]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:10 smithi137 ceph-mon[36220]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-03-11T19:37:10.890 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:37:10 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-b[66824]: 2024-03-11T19:37:10.705+0000 7f92a4647700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-11T19:37:10.890 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:37:10 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-b[66824]: 2024-03-11T19:37:10.705+0000 7f92a4647700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-11T19:37:10.890 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:37:10 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-b[66824]: 2024-03-11T19:37:10.705+0000 7f92a4647700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-11T19:37:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:10 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3206528478' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-11T19:37:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:10 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] up:active 2024-03-11T19:37:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:10 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-11T19:37:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:10 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:37:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:10 smithi204 ceph-mon[36739]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:10 smithi204 ceph-mon[36739]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:10 smithi204 ceph-mon[36739]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-03-11T19:37:10.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.973+0000 7f29fed77640 1 -- 172.21.15.83:0/1766982893 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f29f802cf10 con 0x7f29e80a1ab0 2024-03-11T19:37:10.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.995+0000 7f29f5ffb640 1 -- 172.21.15.83:0/1766982893 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f29f802cf10 con 0x7f29e80a1ab0 2024-03-11T19:37:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 -- 172.21.15.83:0/1766982893 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f29e80a1ab0 msgr2=0x7f29e80a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f29e80a1ab0 0x7f29e80a3f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f29f8069510 tx=0x7f29ec005db0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 -- 172.21.15.83:0/1766982893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f81024f0 msgr2=0x7f29f81976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f81024f0 0x7f29f81976f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f29e400b4a0 tx=0x7f29e400b960 comp rx=0 tx=0).stop 2024-03-11T19:37:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 -- 172.21.15.83:0/1766982893 shutdown_connections 2024-03-11T19:37:11.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f29e80a1ab0 0x7f29e80a3f70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29f8102eb0 0x7f29f819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29f81024f0 0x7f29f81976f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 --2- 172.21.15.83:0/1766982893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29f80ff480 0x7f29f81971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 -- 172.21.15.83:0/1766982893 >> 172.21.15.83:0/1766982893 conn(0x7f29f80fadb0 msgr2=0x7f29f8105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 -- 172.21.15.83:0/1766982893 shutdown_connections 2024-03-11T19:37:11.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:10.997+0000 7f29fed77640 1 -- 172.21.15.83:0/1766982893 wait complete. 2024-03-11T19:37:11.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:10 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3206528478' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-11T19:37:11.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:10 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] up:active 2024-03-11T19:37:11.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:10 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-11T19:37:11.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:10 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:37:11.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:10 smithi083 ceph-mon[31864]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:11.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:10 smithi083 ceph-mon[31864]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:11.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:10 smithi083 ceph-mon[31864]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-03-11T19:37:11.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-11T19:37:11.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.114+0000 7fbc5b423640 1 -- 172.21.15.83:0/3215548829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc54101270 msgr2=0x7fbc541016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.114+0000 7fbc5b423640 1 --2- 172.21.15.83:0/3215548829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc54101270 0x7fbc541016f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009900 tx=0x7fbc4802bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.114+0000 7fbc5b423640 1 -- 172.21.15.83:0/3215548829 shutdown_connections 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.114+0000 7fbc5b423640 1 --2- 172.21.15.83:0/3215548829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc54101c30 0x7fbc541084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.114+0000 7fbc5b423640 1 --2- 172.21.15.83:0/3215548829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc54101270 0x7fbc541016f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.114+0000 7fbc5b423640 1 --2- 172.21.15.83:0/3215548829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc54100080 0x7fbc541004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.114+0000 7fbc5b423640 1 -- 172.21.15.83:0/3215548829 >> 172.21.15.83:0/3215548829 conn(0x7fbc540fb610 msgr2=0x7fbc540fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc5b423640 1 -- 172.21.15.83:0/3215548829 shutdown_connections 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc5b423640 1 -- 172.21.15.83:0/3215548829 wait complete. 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc5b423640 1 Processor -- start 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc5b423640 1 -- start start 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc5b423640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc54100080 0x7fbc54068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc5b423640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc54101270 0x7fbc5419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc5b423640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc54101c30 0x7fbc5419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc5b423640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419eed0 con 0x7fbc54101c30 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc5b423640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419f040 con 0x7fbc54101270 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc5b423640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419f200 con 0x7fbc54100080 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc599c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc54101c30 0x7fbc5419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc599c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc54101c30 0x7fbc5419e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60580/0 (socket says 172.21.15.83:60580) 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.115+0000 7fbc599c0640 1 -- 172.21.15.83:0/1462800471 learned_addr learned my addr 172.21.15.83:0/1462800471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc589be640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc54101270 0x7fbc5419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc591bf640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc54100080 0x7fbc54068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc599c0640 1 -- 172.21.15.83:0/1462800471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc54100080 msgr2=0x7fbc54068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc599c0640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc54100080 0x7fbc54068e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc599c0640 1 -- 172.21.15.83:0/1462800471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc54101270 msgr2=0x7fbc5419c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc599c0640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc54101270 0x7fbc5419c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc599c0640 1 -- 172.21.15.83:0/1462800471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc48009560 con 0x7fbc54101c30 2024-03-11T19:37:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc599c0640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc54101c30 0x7fbc5419e740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009fd0 tx=0x7fbc4400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc527fc640 1 -- 172.21.15.83:0/1462800471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44011830 con 0x7fbc54101c30 2024-03-11T19:37:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc527fc640 1 -- 172.21.15.83:0/1462800471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc44011e70 con 0x7fbc54101c30 2024-03-11T19:37:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc5b423640 1 -- 172.21.15.83:0/1462800471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5419f4e0 con 0x7fbc54101c30 2024-03-11T19:37:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc527fc640 1 -- 172.21.15.83:0/1462800471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400f2e0 con 0x7fbc54101c30 2024-03-11T19:37:11.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.116+0000 7fbc5b423640 1 -- 172.21.15.83:0/1462800471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5419fa70 con 0x7fbc54101c30 2024-03-11T19:37:11.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.117+0000 7fbc5b423640 1 -- 172.21.15.83:0/1462800471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc54107470 con 0x7fbc54101c30 2024-03-11T19:37:11.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.119+0000 7fbc527fc640 1 -- 172.21.15.83:0/1462800471 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fbc440105f0 con 0x7fbc54101c30 2024-03-11T19:37:11.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.119+0000 7fbc527fc640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbc3c0a1ab0 0x7fbc3c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.119+0000 7fbc591bf640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbc3c0a1ab0 0x7fbc3c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:11.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.119+0000 7fbc527fc640 1 -- 172.21.15.83:0/1462800471 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbc440cd1b0 con 0x7fbc54101c30 2024-03-11T19:37:11.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.120+0000 7fbc591bf640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbc3c0a1ab0 0x7fbc3c0a3f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc541010d0 tx=0x7fbc4c0072f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:11.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.123+0000 7fbc527fc640 1 -- 172.21.15.83:0/1462800471 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc44096510 con 0x7fbc54101c30 2024-03-11T19:37:11.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.324+0000 7fbc5b423640 1 -- 172.21.15.83:0/1462800471 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc5402ceb0 con 0x7fbc3c0a1ab0 2024-03-11T19:37:11.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.328+0000 7fbc527fc640 1 -- 172.21.15.83:0/1462800471 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fbc5402ceb0 con 0x7fbc3c0a1ab0 2024-03-11T19:37:11.333 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_1/bddea3eb-992d-47ca-8e3a-c7ffc2fc4e6f 2024-03-11T19:37:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.329+0000 7fbc5b423640 1 -- 172.21.15.83:0/1462800471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbc3c0a1ab0 msgr2=0x7fbc3c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.329+0000 7fbc5b423640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbc3c0a1ab0 0x7fbc3c0a3f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc541010d0 tx=0x7fbc4c0072f0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.330+0000 7fbc5b423640 1 -- 172.21.15.83:0/1462800471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc54101c30 msgr2=0x7fbc5419e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.330+0000 7fbc5b423640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc54101c30 0x7fbc5419e740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009fd0 tx=0x7fbc4400b8e0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.330+0000 7fbc5b423640 1 -- 172.21.15.83:0/1462800471 shutdown_connections 2024-03-11T19:37:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.330+0000 7fbc5b423640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbc3c0a1ab0 0x7fbc3c0a3f70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.330+0000 7fbc5b423640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc54101c30 0x7fbc5419e740 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.330+0000 7fbc5b423640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc54101270 0x7fbc5419c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.330+0000 7fbc5b423640 1 --2- 172.21.15.83:0/1462800471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc54100080 0x7fbc54068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.330+0000 7fbc5b423640 1 -- 172.21.15.83:0/1462800471 >> 172.21.15.83:0/1462800471 conn(0x7fbc540fb610 msgr2=0x7fbc54104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:11.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.330+0000 7fbc5b423640 1 -- 172.21.15.83:0/1462800471 shutdown_connections 2024-03-11T19:37:11.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.330+0000 7fbc5b423640 1 -- 172.21.15.83:0/1462800471 wait complete. 2024-03-11T19:37:11.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:37:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.443+0000 7f8530e76640 1 -- 172.21.15.83:0/1054653585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f852c1000a0 msgr2=0x7f852c1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.443+0000 7f8530e76640 1 --2- 172.21.15.83:0/1054653585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f852c1000a0 0x7f852c1004c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f851c009900 tx=0x7f851c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.443+0000 7f8530e76640 1 -- 172.21.15.83:0/1054653585 shutdown_connections 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.443+0000 7f8530e76640 1 --2- 172.21.15.83:0/1054653585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f852c101c50 0x7f852c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.443+0000 7f8530e76640 1 --2- 172.21.15.83:0/1054653585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f852c101290 0x7f852c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.443+0000 7f8530e76640 1 --2- 172.21.15.83:0/1054653585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f852c1000a0 0x7f852c1004c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.443+0000 7f8530e76640 1 -- 172.21.15.83:0/1054653585 >> 172.21.15.83:0/1054653585 conn(0x7f852c0fb610 msgr2=0x7f852c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.443+0000 7f8530e76640 1 -- 172.21.15.83:0/1054653585 shutdown_connections 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.443+0000 7f8530e76640 1 -- 172.21.15.83:0/1054653585 wait complete. 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f8530e76640 1 Processor -- start 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f8530e76640 1 -- start start 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f8530e76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f852c1000a0 0x7f852c06a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f8530e76640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f852c101290 0x7f852c19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f8530e76640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f852c101c50 0x7f852c19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f8530e76640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c19ed00 con 0x7f852c1000a0 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f8530e76640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c19ee70 con 0x7f852c101290 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f8530e76640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c19f030 con 0x7f852c101c50 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f852a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f852c1000a0 0x7f852c06a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f852a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f852c1000a0 0x7f852c06a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60602/0 (socket says 172.21.15.83:60602) 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f852a59c640 1 -- 172.21.15.83:0/1211163663 learned_addr learned my addr 172.21.15.83:0/1211163663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f852ad9d640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f852c101c50 0x7f852c19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f8523fff640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f852c101290 0x7f852c19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f852a59c640 1 -- 172.21.15.83:0/1211163663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f852c101c50 msgr2=0x7f852c19e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f852a59c640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f852c101c50 0x7f852c19e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f852a59c640 1 -- 172.21.15.83:0/1211163663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f852c101290 msgr2=0x7f852c19bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f852a59c640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f852c101290 0x7f852c19bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f852a59c640 1 -- 172.21.15.83:0/1211163663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c009560 con 0x7f852c1000a0 2024-03-11T19:37:11.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f852a59c640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f852c1000a0 0x7f852c06a3c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f851c000c00 tx=0x7f851c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:11.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.444+0000 7f85237fe640 1 -- 172.21.15.83:0/1211163663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c004df0 con 0x7f852c1000a0 2024-03-11T19:37:11.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.445+0000 7f8530e76640 1 -- 172.21.15.83:0/1211163663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c19f2b0 con 0x7f852c1000a0 2024-03-11T19:37:11.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.445+0000 7f85237fe640 1 -- 172.21.15.83:0/1211163663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f851c005430 con 0x7f852c1000a0 2024-03-11T19:37:11.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.445+0000 7f85237fe640 1 -- 172.21.15.83:0/1211163663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c03e450 con 0x7f852c1000a0 2024-03-11T19:37:11.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.445+0000 7f8530e76640 1 -- 172.21.15.83:0/1211163663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c19f7c0 con 0x7f852c1000a0 2024-03-11T19:37:11.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.445+0000 7f8530e76640 1 -- 172.21.15.83:0/1211163663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852c1073c0 con 0x7f852c1000a0 2024-03-11T19:37:11.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.446+0000 7f85237fe640 1 -- 172.21.15.83:0/1211163663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f851c003680 con 0x7f852c1000a0 2024-03-11T19:37:11.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.447+0000 7f85237fe640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f85180a1a30 0x7f85180a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.447+0000 7f8523fff640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f85180a1a30 0x7f85180a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:11.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.447+0000 7f85237fe640 1 -- 172.21.15.83:0/1211163663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f851c0ef890 con 0x7f852c1000a0 2024-03-11T19:37:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.447+0000 7f8523fff640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f85180a1a30 0x7f85180a3ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8514005fd0 tx=0x7f8514005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:11.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.451+0000 7f85237fe640 1 -- 172.21.15.83:0/1211163663 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f851c0b8bf0 con 0x7f852c1000a0 2024-03-11T19:37:11.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.633+0000 7f85237fe640 1 -- 172.21.15.83:0/1211163663 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f851c0055a0 con 0x7f852c1000a0 2024-03-11T19:37:11.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.678+0000 7f8530e76640 1 -- 172.21.15.83:0/1211163663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f852c10bab0 con 0x7f852c1000a0 2024-03-11T19:37:11.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.679+0000 7f85237fe640 1 -- 172.21.15.83:0/1211163663 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f851c07bfb0 con 0x7f852c1000a0 2024-03-11T19:37:11.684 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:11.684 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:37:09.619596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:37:11.685 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-03-11T19:37:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 -- 172.21.15.83:0/1211163663 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f85180a1a30 msgr2=0x7f85180a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f85180a1a30 0x7f85180a3ef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8514005fd0 tx=0x7f8514005850 comp rx=0 tx=0).stop 2024-03-11T19:37:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 -- 172.21.15.83:0/1211163663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f852c1000a0 msgr2=0x7f852c06a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f852c1000a0 0x7f852c06a3c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f851c000c00 tx=0x7f851c004270 comp rx=0 tx=0).stop 2024-03-11T19:37:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 -- 172.21.15.83:0/1211163663 shutdown_connections 2024-03-11T19:37:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f85180a1a30 0x7f85180a3ef0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f852c101c50 0x7f852c19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f852c101290 0x7f852c19bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 --2- 172.21.15.83:0/1211163663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f852c1000a0 0x7f852c06a3c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 -- 172.21.15.83:0/1211163663 >> 172.21.15.83:0/1211163663 conn(0x7f852c0fb610 msgr2=0x7f852c104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 -- 172.21.15.83:0/1211163663 shutdown_connections 2024-03-11T19:37:11.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.681+0000 7f8530e76640 1 -- 172.21.15.83:0/1211163663 wait complete. 2024-03-11T19:37:11.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:37:11.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.794+0000 7f4b6328f640 1 -- 172.21.15.83:0/1418648216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c102390 msgr2=0x7f4b5c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.794+0000 7f4b6328f640 1 --2- 172.21.15.83:0/1418648216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c102390 0x7f4b5c102810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009900 tx=0x7f4b4c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.794+0000 7f4b6328f640 1 -- 172.21.15.83:0/1418648216 shutdown_connections 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.794+0000 7f4b6328f640 1 --2- 172.21.15.83:0/1418648216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102d50 0x7f4b5c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.794+0000 7f4b6328f640 1 --2- 172.21.15.83:0/1418648216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c102390 0x7f4b5c102810 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.794+0000 7f4b6328f640 1 --2- 172.21.15.83:0/1418648216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b5c1011a0 0x7f4b5c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.794+0000 7f4b6328f640 1 -- 172.21.15.83:0/1418648216 >> 172.21.15.83:0/1418648216 conn(0x7f4b5c0fc730 msgr2=0x7f4b5c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.794+0000 7f4b6328f640 1 -- 172.21.15.83:0/1418648216 shutdown_connections 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.794+0000 7f4b6328f640 1 -- 172.21.15.83:0/1418648216 wait complete. 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6328f640 1 Processor -- start 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6328f640 1 -- start start 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6328f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1011a0 0x7f4b5c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6328f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102390 0x7f4b5c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6328f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b5c102d50 0x7f4b5c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6328f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c19edb0 con 0x7f4b5c1011a0 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6328f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c19ef20 con 0x7f4b5c102390 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6328f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c19f0e0 con 0x7f4b5c102d50 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6102b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1011a0 0x7f4b5c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6102b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1011a0 0x7f4b5c1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60622/0 (socket says 172.21.15.83:60622) 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6102b640 1 -- 172.21.15.83:0/2143208566 learned_addr learned my addr 172.21.15.83:0/2143208566 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6102b640 1 -- 172.21.15.83:0/2143208566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b5c102d50 msgr2=0x7f4b5c19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6102b640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b5c102d50 0x7f4b5c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6102b640 1 -- 172.21.15.83:0/2143208566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102390 msgr2=0x7f4b5c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6102b640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102390 0x7f4b5c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6102b640 1 -- 172.21.15.83:0/2143208566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c009560 con 0x7f4b5c1011a0 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.795+0000 7f4b6102b640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1011a0 0x7f4b5c1972c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800b450 tx=0x7f4b5800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.796+0000 7f4b527fc640 1 -- 172.21.15.83:0/2143208566 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58011890 con 0x7f4b5c1011a0 2024-03-11T19:37:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.796+0000 7f4b527fc640 1 -- 172.21.15.83:0/2143208566 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b580103e0 con 0x7f4b5c1011a0 2024-03-11T19:37:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.796+0000 7f4b6328f640 1 -- 172.21.15.83:0/2143208566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c19f3c0 con 0x7f4b5c1011a0 2024-03-11T19:37:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.796+0000 7f4b527fc640 1 -- 172.21.15.83:0/2143208566 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800f580 con 0x7f4b5c1011a0 2024-03-11T19:37:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.796+0000 7f4b6328f640 1 -- 172.21.15.83:0/2143208566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c19f9a0 con 0x7f4b5c1011a0 2024-03-11T19:37:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.797+0000 7f4b6328f640 1 -- 172.21.15.83:0/2143208566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5c1084c0 con 0x7f4b5c1011a0 2024-03-11T19:37:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.798+0000 7f4b527fc640 1 -- 172.21.15.83:0/2143208566 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f4b580108e0 con 0x7f4b5c1011a0 2024-03-11T19:37:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.798+0000 7f4b527fc640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b480a1b40 0x7f4b480a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.798+0000 7f4b6082a640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b480a1b40 0x7f4b480a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.798+0000 7f4b527fc640 1 -- 172.21.15.83:0/2143208566 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4b580cd0a0 con 0x7f4b5c1011a0 2024-03-11T19:37:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.799+0000 7f4b6082a640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b480a1b40 0x7f4b480a4000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c02c6a0 tx=0x7f4b4c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:11.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.803+0000 7f4b527fc640 1 -- 172.21.15.83:0/2143208566 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b58096410 con 0x7f4b5c1011a0 2024-03-11T19:37:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:11 smithi204 ceph-mon[36739]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:11 smithi204 ceph-mon[36739]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:11 smithi204 ceph-mon[36739]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:11.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.975+0000 7f4b6328f640 1 -- 172.21.15.83:0/2143208566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b5c10ce30 con 0x7f4b5c1011a0 2024-03-11T19:37:11.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.977+0000 7f4b527fc640 1 -- 172.21.15.83:0/2143208566 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22232 (secure 0 0 0) 0x7f4b58099ad0 con 0x7f4b5c1011a0 2024-03-11T19:37:11.982 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:11.982 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":85,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:37:06.317590+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 -- 172.21.15.83:0/2143208566 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b480a1b40 msgr2=0x7f4b480a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b480a1b40 0x7f4b480a4000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c02c6a0 tx=0x7f4b4c000bc0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 -- 172.21.15.83:0/2143208566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1011a0 msgr2=0x7f4b5c1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1011a0 0x7f4b5c1972c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800b450 tx=0x7f4b5800b910 comp rx=0 tx=0).stop 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 -- 172.21.15.83:0/2143208566 shutdown_connections 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b480a1b40 0x7f4b480a4000 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b5c102d50 0x7f4b5c19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b5c102390 0x7f4b5c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 --2- 172.21.15.83:0/2143208566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5c1011a0 0x7f4b5c1972c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 -- 172.21.15.83:0/2143208566 >> 172.21.15.83:0/2143208566 conn(0x7f4b5c0fc730 msgr2=0x7f4b5c105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 -- 172.21.15.83:0/2143208566 shutdown_connections 2024-03-11T19:37:11.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:11.979+0000 7f4b6328f640 1 -- 172.21.15.83:0/2143208566 wait complete. 2024-03-11T19:37:11.993 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:11 smithi083 ceph-mon[31864]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:11.994 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:11 smithi083 ceph-mon[31864]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:11.994 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:11 smithi083 ceph-mon[31864]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:11.995 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-11T19:37:11.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T19:37:12.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.092+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/1797560842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff00ff490 msgr2=0x7f3ff00ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.092+0000 7f3ff7b6c640 1 --2- 172.21.15.83:0/1797560842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff00ff490 0x7f3ff00ff8b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009900 tx=0x7f3fec02bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.092+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/1797560842 shutdown_connections 2024-03-11T19:37:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.092+0000 7f3ff7b6c640 1 --2- 172.21.15.83:0/1797560842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102e20 0x7f3ff01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.092+0000 7f3ff7b6c640 1 --2- 172.21.15.83:0/1797560842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ff0102460 0x7f3ff01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.092+0000 7f3ff7b6c640 1 --2- 172.21.15.83:0/1797560842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff00ff490 0x7f3ff00ff8b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.092+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/1797560842 >> 172.21.15.83:0/1797560842 conn(0x7f3ff00fadc0 msgr2=0x7f3ff00fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.092+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/1797560842 shutdown_connections 2024-03-11T19:37:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.092+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/1797560842 wait complete. 2024-03-11T19:37:12.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.092+0000 7f3ff7b6c640 1 Processor -- start 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff7b6c640 1 -- start start 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff7b6c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff00ff490 0x7f3ff01972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff7b6c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ff0102460 0x7f3ff01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff7b6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102e20 0x7f3ff019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff7b6c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff019eda0 con 0x7f3ff00ff490 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff7b6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff019ef10 con 0x7f3ff0102e20 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff7b6c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff019f0d0 con 0x7f3ff0102460 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff5908640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff00ff490 0x7f3ff01972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff5908640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff00ff490 0x7f3ff01972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60648/0 (socket says 172.21.15.83:60648) 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff5908640 1 -- 172.21.15.83:0/3973930060 learned_addr learned my addr 172.21.15.83:0/3973930060 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff5107640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ff0102460 0x7f3ff01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:12.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff5908640 1 -- 172.21.15.83:0/3973930060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ff0102460 msgr2=0x7f3ff01977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:12.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff5908640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ff0102460 0x7f3ff01977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff5908640 1 -- 172.21.15.83:0/3973930060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102e20 msgr2=0x7f3ff019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:12.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff5908640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102e20 0x7f3ff019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff5908640 1 -- 172.21.15.83:0/3973930060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec009560 con 0x7f3ff00ff490 2024-03-11T19:37:12.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff5908640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff00ff490 0x7f3ff01972b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3fec000c00 tx=0x7f3fec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:12.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3fe6ffd640 1 -- 172.21.15.83:0/3973930060 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec004df0 con 0x7f3ff00ff490 2024-03-11T19:37:12.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3fe6ffd640 1 -- 172.21.15.83:0/3973930060 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fec005430 con 0x7f3ff00ff490 2024-03-11T19:37:12.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3fe6ffd640 1 -- 172.21.15.83:0/3973930060 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec03e450 con 0x7f3ff00ff490 2024-03-11T19:37:12.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/3973930060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff019f350 con 0x7f3ff00ff490 2024-03-11T19:37:12.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.093+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/3973930060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff019f8b0 con 0x7f3ff00ff490 2024-03-11T19:37:12.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.094+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/3973930060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff0108610 con 0x7f3ff00ff490 2024-03-11T19:37:12.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.095+0000 7f3fe6ffd640 1 -- 172.21.15.83:0/3973930060 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f3fec0055a0 con 0x7f3ff00ff490 2024-03-11T19:37:12.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.096+0000 7f3fe6ffd640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fdc0a1b40 0x7f3fdc0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.096+0000 7f3ff5107640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fdc0a1b40 0x7f3fdc0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.096+0000 7f3fe6ffd640 1 -- 172.21.15.83:0/3973930060 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3fec0efca0 con 0x7f3ff00ff490 2024-03-11T19:37:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.096+0000 7f3ff5107640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fdc0a1b40 0x7f3fdc0a4000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0005fd0 tx=0x7f3fe0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:12.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.099+0000 7f3fe6ffd640 1 -- 172.21.15.83:0/3973930060 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3fec0b9010 con 0x7f3ff00ff490 2024-03-11T19:37:12.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:11 smithi137 ceph-mon[36220]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:12.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:11 smithi137 ceph-mon[36220]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:12.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:11 smithi137 ceph-mon[36220]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:37:12.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.269+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/3973930060 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ff002cf10 con 0x7f3fdc0a1b40 2024-03-11T19:37:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.276+0000 7f3fe6ffd640 1 -- 172.21.15.83:0/3973930060 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258686 (secure 0 0 0) 0x7f3ff002cf10 con 0x7f3fdc0a1b40 2024-03-11T19:37:12.284 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:12.286 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-03-11T19:37:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.279+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/3973930060 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fdc0a1b40 msgr2=0x7f3fdc0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.279+0000 7f3ff7b6c640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fdc0a1b40 0x7f3fdc0a4000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0005fd0 tx=0x7f3fe0005850 comp rx=0 tx=0).stop 2024-03-11T19:37:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.279+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/3973930060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff00ff490 msgr2=0x7f3ff01972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.279+0000 7f3ff7b6c640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff00ff490 0x7f3ff01972b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3fec000c00 tx=0x7f3fec004270 comp rx=0 tx=0).stop 2024-03-11T19:37:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.279+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/3973930060 shutdown_connections 2024-03-11T19:37:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.279+0000 7f3ff7b6c640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fdc0a1b40 0x7f3fdc0a4000 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.279+0000 7f3ff7b6c640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102e20 0x7f3ff019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.279+0000 7f3ff7b6c640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ff0102460 0x7f3ff01977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.279+0000 7f3ff7b6c640 1 --2- 172.21.15.83:0/3973930060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff00ff490 0x7f3ff01972b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.279+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/3973930060 >> 172.21.15.83:0/3973930060 conn(0x7f3ff00fadc0 msgr2=0x7f3ff01057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.280+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/3973930060 shutdown_connections 2024-03-11T19:37:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.280+0000 7f3ff7b6c640 1 -- 172.21.15.83:0/3973930060 wait complete. 2024-03-11T19:37:12.295 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-03-11T19:37:10.508378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592686,"num_objects":18,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":214,"num_write_kb":2950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":165,"ondisk_log_size":165,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365876,"kb_used_data":12488,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575948,"statfs":{"total":1151940427776,"available":1151565770752,"internally_reserved":0,"allocated":12787712,"data_stored":6984364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2318,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001233"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707995+0000","last_change":"2024-03-11T19:37:05.340476+0000","last_active":"2024-03-11T19:37:06.707995+0000","last_peered":"2024-03-11T19:37:06.707995+0000","last_clean":"2024-03-11T19:37:06.707995+0000","last_became_active":"2024-03-11T19:37:05.340140+0000","last_became_peered":"2024-03-11T19:37:05.340140+0000","last_unstale":"2024-03-11T19:37:06.707995+0000","last_undegraded":"2024-03-11T19:37:06.707995+0000","last_fullsized":"2024-03-11T19:37:06.707995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:10:35.677374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654563+0000","last_change":"2024-03-11T19:37:05.335500+0000","last_active":"2024-03-11T19:37:06.654563+0000","last_peered":"2024-03-11T19:37:06.654563+0000","last_clean":"2024-03-11T19:37:06.654563+0000","last_became_active":"2024-03-11T19:37:05.335194+0000","last_became_peered":"2024-03-11T19:37:05.335194+0000","last_unstale":"2024-03-11T19:37:06.654563+0000","last_undegraded":"2024-03-11T19:37:06.654563+0000","last_fullsized":"2024-03-11T19:37:06.654563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:03.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654845+0000","last_change":"2024-03-11T19:37:05.334618+0000","last_active":"2024-03-11T19:37:06.654845+0000","last_peered":"2024-03-11T19:37:06.654845+0000","last_clean":"2024-03-11T19:37:06.654845+0000","last_became_active":"2024-03-11T19:37:05.334281+0000","last_became_peered":"2024-03-11T19:37:05.334281+0000","last_unstale":"2024-03-11T19:37:06.654845+0000","last_undegraded":"2024-03-11T19:37:06.654845+0000","last_fullsized":"2024-03-11T19:37:06.654845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:43:22.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655643+0000","last_change":"2024-03-11T19:37:05.332978+0000","last_active":"2024-03-11T19:37:06.655643+0000","last_peered":"2024-03-11T19:37:06.655643+0000","last_clean":"2024-03-11T19:37:06.655643+0000","last_became_active":"2024-03-11T19:37:05.330888+0000","last_became_peered":"2024-03-11T19:37:05.330888+0000","last_unstale":"2024-03-11T19:37:06.655643+0000","last_undegraded":"2024-03-11T19:37:06.655643+0000","last_fullsized":"2024-03-11T19:37:06.655643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:40:36.988348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.329574+0000","last_change":"2024-03-11T19:37:05.329050+0000","last_active":"2024-03-11T19:37:05.329574+0000","last_peered":"2024-03-11T19:37:05.329574+0000","last_clean":"2024-03-11T19:37:05.329574+0000","last_became_active":"2024-03-11T19:37:05.328702+0000","last_became_peered":"2024-03-11T19:37:05.328702+0000","last_unstale":"2024-03-11T19:37:05.329574+0000","last_undegraded":"2024-03-11T19:37:05.329574+0000","last_fullsized":"2024-03-11T19:37:05.329574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:25:56.624661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654655+0000","last_change":"2024-03-11T19:37:05.322748+0000","last_active":"2024-03-11T19:37:06.654655+0000","last_peered":"2024-03-11T19:37:06.654655+0000","last_clean":"2024-03-11T19:37:06.654655+0000","last_became_active":"2024-03-11T19:37:05.322376+0000","last_became_peered":"2024-03-11T19:37:05.322376+0000","last_unstale":"2024-03-11T19:37:06.654655+0000","last_undegraded":"2024-03-11T19:37:06.654655+0000","last_fullsized":"2024-03-11T19:37:06.654655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:40.926854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.324632+0000","last_change":"2024-03-11T19:37:05.323954+0000","last_active":"2024-03-11T19:37:05.324632+0000","last_peered":"2024-03-11T19:37:05.324632+0000","last_clean":"2024-03-11T19:37:05.324632+0000","last_became_active":"2024-03-11T19:37:05.323586+0000","last_became_peered":"2024-03-11T19:37:05.323586+0000","last_unstale":"2024-03-11T19:37:05.324632+0000","last_undegraded":"2024-03-11T19:37:05.324632+0000","last_fullsized":"2024-03-11T19:37:05.324632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:00:29.289434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654048+0000","last_change":"2024-03-11T19:37:05.334138+0000","last_active":"2024-03-11T19:37:06.654048+0000","last_peered":"2024-03-11T19:37:06.654048+0000","last_clean":"2024-03-11T19:37:06.654048+0000","last_became_active":"2024-03-11T19:37:05.333837+0000","last_became_peered":"2024-03-11T19:37:05.333837+0000","last_unstale":"2024-03-11T19:37:06.654048+0000","last_undegraded":"2024-03-11T19:37:06.654048+0000","last_fullsized":"2024-03-11T19:37:06.654048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:49:30.724176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655084+0000","last_change":"2024-03-11T19:37:05.336888+0000","last_active":"2024-03-11T19:37:06.655084+0000","last_peered":"2024-03-11T19:37:06.655084+0000","last_clean":"2024-03-11T19:37:06.655084+0000","last_became_active":"2024-03-11T19:37:05.336256+0000","last_became_peered":"2024-03-11T19:37:05.336256+0000","last_unstale":"2024-03-11T19:37:06.655084+0000","last_undegraded":"2024-03-11T19:37:06.655084+0000","last_fullsized":"2024-03-11T19:37:06.655084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:54:31.526986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655196+0000","last_change":"2024-03-11T19:37:05.321959+0000","last_active":"2024-03-11T19:37:06.655196+0000","last_peered":"2024-03-11T19:37:06.655196+0000","last_clean":"2024-03-11T19:37:06.655196+0000","last_became_active":"2024-03-11T19:37:05.321043+0000","last_became_peered":"2024-03-11T19:37:05.321043+0000","last_unstale":"2024-03-11T19:37:06.655196+0000","last_undegraded":"2024-03-11T19:37:06.655196+0000","last_fullsized":"2024-03-11T19:37:06.655196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:13.121003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.325370+0000","last_change":"2024-03-11T19:37:05.324940+0000","last_active":"2024-03-11T19:37:05.325370+0000","last_peered":"2024-03-11T19:37:05.325370+0000","last_clean":"2024-03-11T19:37:05.325370+0000","last_became_active":"2024-03-11T19:37:05.323427+0000","last_became_peered":"2024-03-11T19:37:05.323427+0000","last_unstale":"2024-03-11T19:37:05.325370+0000","last_undegraded":"2024-03-11T19:37:05.325370+0000","last_fullsized":"2024-03-11T19:37:05.325370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:03:44.568282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655341+0000","last_change":"2024-03-11T19:37:05.332694+0000","last_active":"2024-03-11T19:37:06.655341+0000","last_peered":"2024-03-11T19:37:06.655341+0000","last_clean":"2024-03-11T19:37:06.655341+0000","last_became_active":"2024-03-11T19:37:05.330446+0000","last_became_peered":"2024-03-11T19:37:05.330446+0000","last_unstale":"2024-03-11T19:37:06.655341+0000","last_undegraded":"2024-03-11T19:37:06.655341+0000","last_fullsized":"2024-03-11T19:37:06.655341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:37:40.196933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.323343+0000","last_change":"2024-03-11T19:37:05.325975+0000","last_active":"2024-03-11T19:37:06.323343+0000","last_peered":"2024-03-11T19:37:06.323343+0000","last_clean":"2024-03-11T19:37:06.323343+0000","last_became_active":"2024-03-11T19:37:05.325649+0000","last_became_peered":"2024-03-11T19:37:05.325649+0000","last_unstale":"2024-03-11T19:37:06.323343+0000","last_undegraded":"2024-03-11T19:37:06.323343+0000","last_fullsized":"2024-03-11T19:37:06.323343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:36.373272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654742+0000","last_change":"2024-03-11T19:37:05.335546+0000","last_active":"2024-03-11T19:37:06.654742+0000","last_peered":"2024-03-11T19:37:06.654742+0000","last_clean":"2024-03-11T19:37:06.654742+0000","last_became_active":"2024-03-11T19:37:05.335220+0000","last_became_peered":"2024-03-11T19:37:05.335220+0000","last_unstale":"2024-03-11T19:37:06.654742+0000","last_undegraded":"2024-03-11T19:37:06.654742+0000","last_fullsized":"2024-03-11T19:37:06.654742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:13.116355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654198+0000","last_change":"2024-03-11T19:37:05.338955+0000","last_active":"2024-03-11T19:37:06.654198+0000","last_peered":"2024-03-11T19:37:06.654198+0000","last_clean":"2024-03-11T19:37:06.654198+0000","last_became_active":"2024-03-11T19:37:05.338574+0000","last_became_peered":"2024-03-11T19:37:05.338574+0000","last_unstale":"2024-03-11T19:37:06.654198+0000","last_undegraded":"2024-03-11T19:37:06.654198+0000","last_fullsized":"2024-03-11T19:37:06.654198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:27:27.754077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654332+0000","last_change":"2024-03-11T19:37:05.334578+0000","last_active":"2024-03-11T19:37:06.654332+0000","last_peered":"2024-03-11T19:37:06.654332+0000","last_clean":"2024-03-11T19:37:06.654332+0000","last_became_active":"2024-03-11T19:37:05.334267+0000","last_became_peered":"2024-03-11T19:37:05.334267+0000","last_unstale":"2024-03-11T19:37:06.654332+0000","last_undegraded":"2024-03-11T19:37:06.654332+0000","last_fullsized":"2024-03-11T19:37:06.654332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:31.704187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707867+0000","last_change":"2024-03-11T19:37:05.341178+0000","last_active":"2024-03-11T19:37:06.707867+0000","last_peered":"2024-03-11T19:37:06.707867+0000","last_clean":"2024-03-11T19:37:06.707867+0000","last_became_active":"2024-03-11T19:37:05.340837+0000","last_became_peered":"2024-03-11T19:37:05.340837+0000","last_unstale":"2024-03-11T19:37:06.707867+0000","last_undegraded":"2024-03-11T19:37:06.707867+0000","last_fullsized":"2024-03-11T19:37:06.707867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:39:54.460937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655135+0000","last_change":"2024-03-11T19:37:05.330981+0000","last_active":"2024-03-11T19:37:06.655135+0000","last_peered":"2024-03-11T19:37:06.655135+0000","last_clean":"2024-03-11T19:37:06.655135+0000","last_became_active":"2024-03-11T19:37:05.330638+0000","last_became_peered":"2024-03-11T19:37:05.330638+0000","last_unstale":"2024-03-11T19:37:06.655135+0000","last_undegraded":"2024-03-11T19:37:06.655135+0000","last_fullsized":"2024-03-11T19:37:06.655135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:36:12.565936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655123+0000","last_change":"2024-03-11T19:37:05.329866+0000","last_active":"2024-03-11T19:37:06.655123+0000","last_peered":"2024-03-11T19:37:06.655123+0000","last_clean":"2024-03-11T19:37:06.655123+0000","last_became_active":"2024-03-11T19:37:05.329572+0000","last_became_peered":"2024-03-11T19:37:05.329572+0000","last_unstale":"2024-03-11T19:37:06.655123+0000","last_undegraded":"2024-03-11T19:37:06.655123+0000","last_fullsized":"2024-03-11T19:37:06.655123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:40:04.482929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655063+0000","last_change":"2024-03-11T19:37:04.587149+0000","last_active":"2024-03-11T19:37:06.655063+0000","last_peered":"2024-03-11T19:37:06.655063+0000","last_clean":"2024-03-11T19:37:06.655063+0000","last_became_active":"2024-03-11T19:37:04.586013+0000","last_became_peered":"2024-03-11T19:37:04.586013+0000","last_unstale":"2024-03-11T19:37:06.655063+0000","last_undegraded":"2024-03-11T19:37:06.655063+0000","last_fullsized":"2024-03-11T19:37:06.655063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:15.758154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654974+0000","last_change":"2024-03-11T19:37:05.327979+0000","last_active":"2024-03-11T19:37:06.654974+0000","last_peered":"2024-03-11T19:37:06.654974+0000","last_clean":"2024-03-11T19:37:06.654974+0000","last_became_active":"2024-03-11T19:37:05.327647+0000","last_became_peered":"2024-03-11T19:37:05.327647+0000","last_unstale":"2024-03-11T19:37:06.654974+0000","last_undegraded":"2024-03-11T19:37:06.654974+0000","last_fullsized":"2024-03-11T19:37:06.654974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:09.514766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655141+0000","last_change":"2024-03-11T19:37:04.582937+0000","last_active":"2024-03-11T19:37:06.655141+0000","last_peered":"2024-03-11T19:37:06.655141+0000","last_clean":"2024-03-11T19:37:06.655141+0000","last_became_active":"2024-03-11T19:37:04.582622+0000","last_became_peered":"2024-03-11T19:37:04.582622+0000","last_unstale":"2024-03-11T19:37:06.655141+0000","last_undegraded":"2024-03-11T19:37:06.655141+0000","last_fullsized":"2024-03-11T19:37:06.655141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:02.275698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654213+0000","last_change":"2024-03-11T19:37:05.327191+0000","last_active":"2024-03-11T19:37:06.654213+0000","last_peered":"2024-03-11T19:37:06.654213+0000","last_clean":"2024-03-11T19:37:06.654213+0000","last_became_active":"2024-03-11T19:37:05.326001+0000","last_became_peered":"2024-03-11T19:37:05.326001+0000","last_unstale":"2024-03-11T19:37:06.654213+0000","last_undegraded":"2024-03-11T19:37:06.654213+0000","last_fullsized":"2024-03-11T19:37:06.654213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:27.301321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.324488+0000","last_change":"2024-03-11T19:37:04.587054+0000","last_active":"2024-03-11T19:37:05.324488+0000","last_peered":"2024-03-11T19:37:05.324488+0000","last_clean":"2024-03-11T19:37:05.324488+0000","last_became_active":"2024-03-11T19:37:04.586104+0000","last_became_peered":"2024-03-11T19:37:04.586104+0000","last_unstale":"2024-03-11T19:37:05.324488+0000","last_undegraded":"2024-03-11T19:37:05.324488+0000","last_fullsized":"2024-03-11T19:37:05.324488+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:34.913677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.656102+0000","last_change":"2024-03-11T19:37:05.333288+0000","last_active":"2024-03-11T19:37:06.656102+0000","last_peered":"2024-03-11T19:37:06.656102+0000","last_clean":"2024-03-11T19:37:06.656102+0000","last_became_active":"2024-03-11T19:37:05.332882+0000","last_became_peered":"2024-03-11T19:37:05.332882+0000","last_unstale":"2024-03-11T19:37:06.656102+0000","last_undegraded":"2024-03-11T19:37:06.656102+0000","last_fullsized":"2024-03-11T19:37:06.656102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:45.680608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.324573+0000","last_change":"2024-03-11T19:37:04.583660+0000","last_active":"2024-03-11T19:37:05.324573+0000","last_peered":"2024-03-11T19:37:05.324573+0000","last_clean":"2024-03-11T19:37:05.324573+0000","last_became_active":"2024-03-11T19:37:04.583360+0000","last_became_peered":"2024-03-11T19:37:04.583360+0000","last_unstale":"2024-03-11T19:37:05.324573+0000","last_undegraded":"2024-03-11T19:37:05.324573+0000","last_fullsized":"2024-03-11T19:37:05.324573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:00.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.336390+0000","last_change":"2024-03-11T19:37:05.335649+0000","last_active":"2024-03-11T19:37:05.336390+0000","last_peered":"2024-03-11T19:37:05.336390+0000","last_clean":"2024-03-11T19:37:05.336390+0000","last_became_active":"2024-03-11T19:37:05.335255+0000","last_became_peered":"2024-03-11T19:37:05.335255+0000","last_unstale":"2024-03-11T19:37:05.336390+0000","last_undegraded":"2024-03-11T19:37:05.336390+0000","last_fullsized":"2024-03-11T19:37:05.336390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:10:18.371237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.320761+0000","last_change":"2024-03-11T19:37:04.586002+0000","last_active":"2024-03-11T19:37:05.320761+0000","last_peered":"2024-03-11T19:37:05.320761+0000","last_clean":"2024-03-11T19:37:05.320761+0000","last_became_active":"2024-03-11T19:37:04.585551+0000","last_became_peered":"2024-03-11T19:37:04.585551+0000","last_unstale":"2024-03-11T19:37:05.320761+0000","last_undegraded":"2024-03-11T19:37:05.320761+0000","last_fullsized":"2024-03-11T19:37:05.320761+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:34:12.663801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.325725+0000","last_change":"2024-03-11T19:37:05.325168+0000","last_active":"2024-03-11T19:37:05.325725+0000","last_peered":"2024-03-11T19:37:05.325725+0000","last_clean":"2024-03-11T19:37:05.325725+0000","last_became_active":"2024-03-11T19:37:05.324848+0000","last_became_peered":"2024-03-11T19:37:05.324848+0000","last_unstale":"2024-03-11T19:37:05.325725+0000","last_undegraded":"2024-03-11T19:37:05.325725+0000","last_fullsized":"2024-03-11T19:37:05.325725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:56:28.236067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654399+0000","last_change":"2024-03-11T19:37:04.582727+0000","last_active":"2024-03-11T19:37:06.654399+0000","last_peered":"2024-03-11T19:37:06.654399+0000","last_clean":"2024-03-11T19:37:06.654399+0000","last_became_active":"2024-03-11T19:37:04.582359+0000","last_became_peered":"2024-03-11T19:37:04.582359+0000","last_unstale":"2024-03-11T19:37:06.654399+0000","last_undegraded":"2024-03-11T19:37:06.654399+0000","last_fullsized":"2024-03-11T19:37:06.654399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:05.591537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.336310+0000","last_change":"2024-03-11T19:37:05.335442+0000","last_active":"2024-03-11T19:37:05.336310+0000","last_peered":"2024-03-11T19:37:05.336310+0000","last_clean":"2024-03-11T19:37:05.336310+0000","last_became_active":"2024-03-11T19:37:05.335003+0000","last_became_peered":"2024-03-11T19:37:05.335003+0000","last_unstale":"2024-03-11T19:37:05.336310+0000","last_undegraded":"2024-03-11T19:37:05.336310+0000","last_fullsized":"2024-03-11T19:37:05.336310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:45:25.355535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.322818+0000","last_change":"2024-03-11T19:37:04.589957+0000","last_active":"2024-03-11T19:37:06.322818+0000","last_peered":"2024-03-11T19:37:06.322818+0000","last_clean":"2024-03-11T19:37:06.322818+0000","last_became_active":"2024-03-11T19:37:04.589539+0000","last_became_peered":"2024-03-11T19:37:04.589539+0000","last_unstale":"2024-03-11T19:37:06.322818+0000","last_undegraded":"2024-03-11T19:37:06.322818+0000","last_fullsized":"2024-03-11T19:37:06.322818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:48:52.764057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.653973+0000","last_change":"2024-03-11T19:37:05.331380+0000","last_active":"2024-03-11T19:37:06.653973+0000","last_peered":"2024-03-11T19:37:06.653973+0000","last_clean":"2024-03-11T19:37:06.653973+0000","last_became_active":"2024-03-11T19:37:05.331000+0000","last_became_peered":"2024-03-11T19:37:05.331000+0000","last_unstale":"2024-03-11T19:37:06.653973+0000","last_undegraded":"2024-03-11T19:37:06.653973+0000","last_fullsized":"2024-03-11T19:37:06.653973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:58:47.278195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.319267+0000","last_change":"2024-03-11T19:37:04.584128+0000","last_active":"2024-03-11T19:37:05.319267+0000","last_peered":"2024-03-11T19:37:05.319267+0000","last_clean":"2024-03-11T19:37:05.319267+0000","last_became_active":"2024-03-11T19:37:04.583842+0000","last_became_peered":"2024-03-11T19:37:04.583842+0000","last_unstale":"2024-03-11T19:37:05.319267+0000","last_undegraded":"2024-03-11T19:37:05.319267+0000","last_fullsized":"2024-03-11T19:37:05.319267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:18.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654396+0000","last_change":"2024-03-11T19:37:05.328999+0000","last_active":"2024-03-11T19:37:06.654396+0000","last_peered":"2024-03-11T19:37:06.654396+0000","last_clean":"2024-03-11T19:37:06.654396+0000","last_became_active":"2024-03-11T19:37:05.328620+0000","last_became_peered":"2024-03-11T19:37:05.328620+0000","last_unstale":"2024-03-11T19:37:06.654396+0000","last_undegraded":"2024-03-11T19:37:06.654396+0000","last_fullsized":"2024-03-11T19:37:06.654396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:40.191922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.614634+0000","last_change":"2024-03-11T19:37:04.587956+0000","last_active":"2024-03-11T19:37:07.614634+0000","last_peered":"2024-03-11T19:37:07.614634+0000","last_clean":"2024-03-11T19:37:07.614634+0000","last_became_active":"2024-03-11T19:37:04.587568+0000","last_became_peered":"2024-03-11T19:37:04.587568+0000","last_unstale":"2024-03-11T19:37:07.614634+0000","last_undegraded":"2024-03-11T19:37:07.614634+0000","last_fullsized":"2024-03-11T19:37:07.614634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:45:44.019436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655218+0000","last_change":"2024-03-11T19:37:05.336912+0000","last_active":"2024-03-11T19:37:06.655218+0000","last_peered":"2024-03-11T19:37:06.655218+0000","last_clean":"2024-03-11T19:37:06.655218+0000","last_became_active":"2024-03-11T19:37:05.336397+0000","last_became_peered":"2024-03-11T19:37:05.336397+0000","last_unstale":"2024-03-11T19:37:06.655218+0000","last_undegraded":"2024-03-11T19:37:06.655218+0000","last_fullsized":"2024-03-11T19:37:06.655218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:29.375396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.312499+0000","last_change":"2024-03-11T19:37:04.584382+0000","last_active":"2024-03-11T19:37:05.312499+0000","last_peered":"2024-03-11T19:37:05.312499+0000","last_clean":"2024-03-11T19:37:05.312499+0000","last_became_active":"2024-03-11T19:37:04.584026+0000","last_became_peered":"2024-03-11T19:37:04.584026+0000","last_unstale":"2024-03-11T19:37:05.312499+0000","last_undegraded":"2024-03-11T19:37:05.312499+0000","last_fullsized":"2024-03-11T19:37:05.312499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:28.152903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654957+0000","last_change":"2024-03-11T19:37:05.331385+0000","last_active":"2024-03-11T19:37:06.654957+0000","last_peered":"2024-03-11T19:37:06.654957+0000","last_clean":"2024-03-11T19:37:06.654957+0000","last_became_active":"2024-03-11T19:37:05.331069+0000","last_became_peered":"2024-03-11T19:37:05.331069+0000","last_unstale":"2024-03-11T19:37:06.654957+0000","last_undegraded":"2024-03-11T19:37:06.654957+0000","last_fullsized":"2024-03-11T19:37:06.654957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:18.375605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.320435+0000","last_change":"2024-03-11T19:37:04.586493+0000","last_active":"2024-03-11T19:37:05.320435+0000","last_peered":"2024-03-11T19:37:05.320435+0000","last_clean":"2024-03-11T19:37:05.320435+0000","last_became_active":"2024-03-11T19:37:04.586185+0000","last_became_peered":"2024-03-11T19:37:04.586185+0000","last_unstale":"2024-03-11T19:37:05.320435+0000","last_undegraded":"2024-03-11T19:37:05.320435+0000","last_fullsized":"2024-03-11T19:37:05.320435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:00.899752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.656135+0000","last_change":"2024-03-11T19:37:05.336414+0000","last_active":"2024-03-11T19:37:06.656135+0000","last_peered":"2024-03-11T19:37:06.656135+0000","last_clean":"2024-03-11T19:37:06.656135+0000","last_became_active":"2024-03-11T19:37:05.334536+0000","last_became_peered":"2024-03-11T19:37:05.334536+0000","last_unstale":"2024-03-11T19:37:06.656135+0000","last_undegraded":"2024-03-11T19:37:06.656135+0000","last_fullsized":"2024-03-11T19:37:06.656135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:50:19.109975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.613641+0000","last_change":"2024-03-11T19:37:04.584544+0000","last_active":"2024-03-11T19:37:07.613641+0000","last_peered":"2024-03-11T19:37:07.613641+0000","last_clean":"2024-03-11T19:37:07.613641+0000","last_became_active":"2024-03-11T19:37:04.584198+0000","last_became_peered":"2024-03-11T19:37:04.584198+0000","last_unstale":"2024-03-11T19:37:07.613641+0000","last_undegraded":"2024-03-11T19:37:07.613641+0000","last_fullsized":"2024-03-11T19:37:07.613641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:01:56.744533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.334250+0000","last_change":"2024-03-11T19:37:05.331077+0000","last_active":"2024-03-11T19:37:05.334250+0000","last_peered":"2024-03-11T19:37:05.334250+0000","last_clean":"2024-03-11T19:37:05.334250+0000","last_became_active":"2024-03-11T19:37:05.330708+0000","last_became_peered":"2024-03-11T19:37:05.330708+0000","last_unstale":"2024-03-11T19:37:05.334250+0000","last_undegraded":"2024-03-11T19:37:05.334250+0000","last_fullsized":"2024-03-11T19:37:05.334250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:58:58.433659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.612623+0000","last_change":"2024-03-11T19:37:04.583353+0000","last_active":"2024-03-11T19:37:07.612623+0000","last_peered":"2024-03-11T19:37:07.612623+0000","last_clean":"2024-03-11T19:37:07.612623+0000","last_became_active":"2024-03-11T19:37:04.581974+0000","last_became_peered":"2024-03-11T19:37:04.581974+0000","last_unstale":"2024-03-11T19:37:07.612623+0000","last_undegraded":"2024-03-11T19:37:07.612623+0000","last_fullsized":"2024-03-11T19:37:07.612623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:41.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.322919+0000","last_change":"2024-03-11T19:37:05.329498+0000","last_active":"2024-03-11T19:37:06.322919+0000","last_peered":"2024-03-11T19:37:06.322919+0000","last_clean":"2024-03-11T19:37:06.322919+0000","last_became_active":"2024-03-11T19:37:05.329186+0000","last_became_peered":"2024-03-11T19:37:05.329186+0000","last_unstale":"2024-03-11T19:37:06.322919+0000","last_undegraded":"2024-03-11T19:37:06.322919+0000","last_fullsized":"2024-03-11T19:37:06.322919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:33:45.012049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.321824+0000","last_change":"2024-03-11T19:37:04.582027+0000","last_active":"2024-03-11T19:37:05.321824+0000","last_peered":"2024-03-11T19:37:05.321824+0000","last_clean":"2024-03-11T19:37:05.321824+0000","last_became_active":"2024-03-11T19:37:04.581678+0000","last_became_peered":"2024-03-11T19:37:04.581678+0000","last_unstale":"2024-03-11T19:37:05.321824+0000","last_undegraded":"2024-03-11T19:37:05.321824+0000","last_fullsized":"2024-03-11T19:37:05.321824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:03:40.688053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.325034+0000","last_change":"2024-03-11T19:37:05.324499+0000","last_active":"2024-03-11T19:37:05.325034+0000","last_peered":"2024-03-11T19:37:05.325034+0000","last_clean":"2024-03-11T19:37:05.325034+0000","last_became_active":"2024-03-11T19:37:05.324156+0000","last_became_peered":"2024-03-11T19:37:05.324156+0000","last_unstale":"2024-03-11T19:37:05.325034+0000","last_undegraded":"2024-03-11T19:37:05.325034+0000","last_fullsized":"2024-03-11T19:37:05.325034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:39:58.388499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655390+0000","last_change":"2024-03-11T19:37:04.576415+0000","last_active":"2024-03-11T19:37:06.655390+0000","last_peered":"2024-03-11T19:37:06.655390+0000","last_clean":"2024-03-11T19:37:06.655390+0000","last_became_active":"2024-03-11T19:37:04.575978+0000","last_became_peered":"2024-03-11T19:37:04.575978+0000","last_unstale":"2024-03-11T19:37:06.655390+0000","last_undegraded":"2024-03-11T19:37:06.655390+0000","last_fullsized":"2024-03-11T19:37:06.655390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:20.124813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.331217+0000","last_change":"2024-03-11T19:37:05.326258+0000","last_active":"2024-03-11T19:37:05.331217+0000","last_peered":"2024-03-11T19:37:05.331217+0000","last_clean":"2024-03-11T19:37:05.331217+0000","last_became_active":"2024-03-11T19:37:05.325909+0000","last_became_peered":"2024-03-11T19:37:05.325909+0000","last_unstale":"2024-03-11T19:37:05.331217+0000","last_undegraded":"2024-03-11T19:37:05.331217+0000","last_fullsized":"2024-03-11T19:37:05.331217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:51:32.331547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.319661+0000","last_change":"2024-03-11T19:37:04.586696+0000","last_active":"2024-03-11T19:37:05.319661+0000","last_peered":"2024-03-11T19:37:05.319661+0000","last_clean":"2024-03-11T19:37:05.319661+0000","last_became_active":"2024-03-11T19:37:04.585993+0000","last_became_peered":"2024-03-11T19:37:04.585993+0000","last_unstale":"2024-03-11T19:37:05.319661+0000","last_undegraded":"2024-03-11T19:37:05.319661+0000","last_fullsized":"2024-03-11T19:37:05.319661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:59:07.807487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.337748+0000","last_change":"2024-03-11T19:37:05.336727+0000","last_active":"2024-03-11T19:37:05.337748+0000","last_peered":"2024-03-11T19:37:05.337748+0000","last_clean":"2024-03-11T19:37:05.337748+0000","last_became_active":"2024-03-11T19:37:05.336384+0000","last_became_peered":"2024-03-11T19:37:05.336384+0000","last_unstale":"2024-03-11T19:37:05.337748+0000","last_undegraded":"2024-03-11T19:37:05.337748+0000","last_fullsized":"2024-03-11T19:37:05.337748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:40:27.922710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.324307+0000","last_change":"2024-03-11T19:37:04.586814+0000","last_active":"2024-03-11T19:37:05.324307+0000","last_peered":"2024-03-11T19:37:05.324307+0000","last_clean":"2024-03-11T19:37:05.324307+0000","last_became_active":"2024-03-11T19:37:04.586512+0000","last_became_peered":"2024-03-11T19:37:04.586512+0000","last_unstale":"2024-03-11T19:37:05.324307+0000","last_undegraded":"2024-03-11T19:37:05.324307+0000","last_fullsized":"2024-03-11T19:37:05.324307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:04:23.979783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707821+0000","last_change":"2024-03-11T19:37:05.335353+0000","last_active":"2024-03-11T19:37:06.707821+0000","last_peered":"2024-03-11T19:37:06.707821+0000","last_clean":"2024-03-11T19:37:06.707821+0000","last_became_active":"2024-03-11T19:37:05.335066+0000","last_became_peered":"2024-03-11T19:37:05.335066+0000","last_unstale":"2024-03-11T19:37:06.707821+0000","last_undegraded":"2024-03-11T19:37:06.707821+0000","last_fullsized":"2024-03-11T19:37:06.707821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:11.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654949+0000","last_change":"2024-03-11T19:37:04.586679+0000","last_active":"2024-03-11T19:37:06.654949+0000","last_peered":"2024-03-11T19:37:06.654949+0000","last_clean":"2024-03-11T19:37:06.654949+0000","last_became_active":"2024-03-11T19:37:04.586316+0000","last_became_peered":"2024-03-11T19:37:04.586316+0000","last_unstale":"2024-03-11T19:37:06.654949+0000","last_undegraded":"2024-03-11T19:37:06.654949+0000","last_fullsized":"2024-03-11T19:37:06.654949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:58:21.115526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655325+0000","last_change":"2024-03-11T19:37:05.330206+0000","last_active":"2024-03-11T19:37:06.655325+0000","last_peered":"2024-03-11T19:37:06.655325+0000","last_clean":"2024-03-11T19:37:06.655325+0000","last_became_active":"2024-03-11T19:37:05.329847+0000","last_became_peered":"2024-03-11T19:37:05.329847+0000","last_unstale":"2024-03-11T19:37:06.655325+0000","last_undegraded":"2024-03-11T19:37:06.655325+0000","last_fullsized":"2024-03-11T19:37:06.655325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:14:52.532236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.312403+0000","last_change":"2024-03-11T19:37:04.581031+0000","last_active":"2024-03-11T19:37:05.312403+0000","last_peered":"2024-03-11T19:37:05.312403+0000","last_clean":"2024-03-11T19:37:05.312403+0000","last_became_active":"2024-03-11T19:37:04.580728+0000","last_became_peered":"2024-03-11T19:37:04.580728+0000","last_unstale":"2024-03-11T19:37:05.312403+0000","last_undegraded":"2024-03-11T19:37:05.312403+0000","last_fullsized":"2024-03-11T19:37:05.312403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:43.401898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654768+0000","last_change":"2024-03-11T19:37:05.328686+0000","last_active":"2024-03-11T19:37:06.654768+0000","last_peered":"2024-03-11T19:37:06.654768+0000","last_clean":"2024-03-11T19:37:06.654768+0000","last_became_active":"2024-03-11T19:37:05.328372+0000","last_became_peered":"2024-03-11T19:37:05.328372+0000","last_unstale":"2024-03-11T19:37:06.654768+0000","last_undegraded":"2024-03-11T19:37:06.654768+0000","last_fullsized":"2024-03-11T19:37:06.654768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:38:33.216489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.336447+0000","last_change":"2024-03-11T19:37:05.335057+0000","last_active":"2024-03-11T19:37:05.336447+0000","last_peered":"2024-03-11T19:37:05.336447+0000","last_clean":"2024-03-11T19:37:05.336447+0000","last_became_active":"2024-03-11T19:37:05.334646+0000","last_became_peered":"2024-03-11T19:37:05.334646+0000","last_unstale":"2024-03-11T19:37:05.336447+0000","last_undegraded":"2024-03-11T19:37:05.336447+0000","last_fullsized":"2024-03-11T19:37:05.336447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:07:40.925209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.322723+0000","last_change":"2024-03-11T19:37:04.588095+0000","last_active":"2024-03-11T19:37:06.322723+0000","last_peered":"2024-03-11T19:37:06.322723+0000","last_clean":"2024-03-11T19:37:06.322723+0000","last_became_active":"2024-03-11T19:37:04.587757+0000","last_became_peered":"2024-03-11T19:37:04.587757+0000","last_unstale":"2024-03-11T19:37:06.322723+0000","last_undegraded":"2024-03-11T19:37:06.322723+0000","last_fullsized":"2024-03-11T19:37:06.322723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:57.926858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655062+0000","last_change":"2024-03-11T19:37:05.326806+0000","last_active":"2024-03-11T19:37:06.655062+0000","last_peered":"2024-03-11T19:37:06.655062+0000","last_clean":"2024-03-11T19:37:06.655062+0000","last_became_active":"2024-03-11T19:37:05.326506+0000","last_became_peered":"2024-03-11T19:37:05.326506+0000","last_unstale":"2024-03-11T19:37:06.655062+0000","last_undegraded":"2024-03-11T19:37:06.655062+0000","last_fullsized":"2024-03-11T19:37:06.655062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:20.664857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655054+0000","last_change":"2024-03-11T19:37:04.588459+0000","last_active":"2024-03-11T19:37:06.655054+0000","last_peered":"2024-03-11T19:37:06.655054+0000","last_clean":"2024-03-11T19:37:06.655054+0000","last_became_active":"2024-03-11T19:37:04.588000+0000","last_became_peered":"2024-03-11T19:37:04.588000+0000","last_unstale":"2024-03-11T19:37:06.655054+0000","last_undegraded":"2024-03-11T19:37:06.655054+0000","last_fullsized":"2024-03-11T19:37:06.655054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:20:47.717196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":336,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.312337+0000","last_change":"2024-03-11T19:35:55.814178+0000","last_active":"2024-03-11T19:37:05.312337+0000","last_peered":"2024-03-11T19:37:05.312337+0000","last_clean":"2024-03-11T19:37:05.312337+0000","last_became_active":"2024-03-11T19:35:55.813697+0000","last_became_peered":"2024-03-11T19:35:55.813697+0000","last_unstale":"2024-03-11T19:37:05.312337+0000","last_undegraded":"2024-03-11T19:37:05.312337+0000","last_fullsized":"2024-03-11T19:37:05.312337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_clean_scrub_stamp":"2024-03-11T19:32:34.872292+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:50:27.486733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.322793+0000","last_change":"2024-03-11T19:37:05.319614+0000","last_active":"2024-03-11T19:37:06.322793+0000","last_peered":"2024-03-11T19:37:06.322793+0000","last_clean":"2024-03-11T19:37:06.322793+0000","last_became_active":"2024-03-11T19:37:05.319277+0000","last_became_peered":"2024-03-11T19:37:05.319277+0000","last_unstale":"2024-03-11T19:37:06.322793+0000","last_undegraded":"2024-03-11T19:37:06.322793+0000","last_fullsized":"2024-03-11T19:37:06.322793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:27.329900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.312437+0000","last_change":"2024-03-11T19:37:04.582250+0000","last_active":"2024-03-11T19:37:05.312437+0000","last_peered":"2024-03-11T19:37:05.312437+0000","last_clean":"2024-03-11T19:37:05.312437+0000","last_became_active":"2024-03-11T19:37:04.581938+0000","last_became_peered":"2024-03-11T19:37:04.581938+0000","last_unstale":"2024-03-11T19:37:05.312437+0000","last_undegraded":"2024-03-11T19:37:05.312437+0000","last_fullsized":"2024-03-11T19:37:05.312437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:23.466067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.324486+0000","last_change":"2024-03-11T19:37:04.590699+0000","last_active":"2024-03-11T19:37:05.324486+0000","last_peered":"2024-03-11T19:37:05.324486+0000","last_clean":"2024-03-11T19:37:05.324486+0000","last_became_active":"2024-03-11T19:37:04.590424+0000","last_became_peered":"2024-03-11T19:37:04.590424+0000","last_unstale":"2024-03-11T19:37:05.324486+0000","last_undegraded":"2024-03-11T19:37:05.324486+0000","last_fullsized":"2024-03-11T19:37:05.324486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:49.075335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.323185+0000","last_change":"2024-03-11T19:37:05.323357+0000","last_active":"2024-03-11T19:37:06.323185+0000","last_peered":"2024-03-11T19:37:06.323185+0000","last_clean":"2024-03-11T19:37:06.323185+0000","last_became_active":"2024-03-11T19:37:05.322980+0000","last_became_peered":"2024-03-11T19:37:05.322980+0000","last_unstale":"2024-03-11T19:37:06.323185+0000","last_undegraded":"2024-03-11T19:37:06.323185+0000","last_fullsized":"2024-03-11T19:37:06.323185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:19.106375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.577045+0000","last_change":"2024-03-11T19:37:04.585232+0000","last_active":"2024-03-11T19:37:07.577045+0000","last_peered":"2024-03-11T19:37:07.577045+0000","last_clean":"2024-03-11T19:37:07.577045+0000","last_became_active":"2024-03-11T19:37:04.584904+0000","last_became_peered":"2024-03-11T19:37:04.584904+0000","last_unstale":"2024-03-11T19:37:07.577045+0000","last_undegraded":"2024-03-11T19:37:07.577045+0000","last_fullsized":"2024-03-11T19:37:07.577045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:56:46.519961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654745+0000","last_change":"2024-03-11T19:37:05.332019+0000","last_active":"2024-03-11T19:37:06.654745+0000","last_peered":"2024-03-11T19:37:06.654745+0000","last_clean":"2024-03-11T19:37:06.654745+0000","last_became_active":"2024-03-11T19:37:05.331673+0000","last_became_peered":"2024-03-11T19:37:05.331673+0000","last_unstale":"2024-03-11T19:37:06.654745+0000","last_undegraded":"2024-03-11T19:37:06.654745+0000","last_fullsized":"2024-03-11T19:37:06.654745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:49:45.787082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655700+0000","last_change":"2024-03-11T19:37:04.583404+0000","last_active":"2024-03-11T19:37:06.655700+0000","last_peered":"2024-03-11T19:37:06.655700+0000","last_clean":"2024-03-11T19:37:06.655700+0000","last_became_active":"2024-03-11T19:37:04.583025+0000","last_became_peered":"2024-03-11T19:37:04.583025+0000","last_unstale":"2024-03-11T19:37:06.655700+0000","last_undegraded":"2024-03-11T19:37:06.655700+0000","last_fullsized":"2024-03-11T19:37:06.655700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:02:25.202591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654615+0000","last_change":"2024-03-11T19:37:05.331711+0000","last_active":"2024-03-11T19:37:06.654615+0000","last_peered":"2024-03-11T19:37:06.654615+0000","last_clean":"2024-03-11T19:37:06.654615+0000","last_became_active":"2024-03-11T19:37:05.330851+0000","last_became_peered":"2024-03-11T19:37:05.330851+0000","last_unstale":"2024-03-11T19:37:06.654615+0000","last_undegraded":"2024-03-11T19:37:06.654615+0000","last_fullsized":"2024-03-11T19:37:06.654615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:18:26.692068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.612561+0000","last_change":"2024-03-11T19:37:04.586501+0000","last_active":"2024-03-11T19:37:07.612561+0000","last_peered":"2024-03-11T19:37:07.612561+0000","last_clean":"2024-03-11T19:37:07.612561+0000","last_became_active":"2024-03-11T19:37:04.586200+0000","last_became_peered":"2024-03-11T19:37:04.586200+0000","last_unstale":"2024-03-11T19:37:07.612561+0000","last_undegraded":"2024-03-11T19:37:07.612561+0000","last_fullsized":"2024-03-11T19:37:07.612561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:56.635375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.334543+0000","last_change":"2024-03-11T19:37:05.332402+0000","last_active":"2024-03-11T19:37:05.334543+0000","last_peered":"2024-03-11T19:37:05.334543+0000","last_clean":"2024-03-11T19:37:05.334543+0000","last_became_active":"2024-03-11T19:37:05.332087+0000","last_became_peered":"2024-03-11T19:37:05.332087+0000","last_unstale":"2024-03-11T19:37:05.334543+0000","last_undegraded":"2024-03-11T19:37:05.334543+0000","last_fullsized":"2024-03-11T19:37:05.334543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:15:29.816835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654418+0000","last_change":"2024-03-11T19:37:04.584978+0000","last_active":"2024-03-11T19:37:06.654418+0000","last_peered":"2024-03-11T19:37:06.654418+0000","last_clean":"2024-03-11T19:37:06.654418+0000","last_became_active":"2024-03-11T19:37:04.584556+0000","last_became_peered":"2024-03-11T19:37:04.584556+0000","last_unstale":"2024-03-11T19:37:06.654418+0000","last_undegraded":"2024-03-11T19:37:06.654418+0000","last_fullsized":"2024-03-11T19:37:06.654418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:39:48.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707645+0000","last_change":"2024-03-11T19:37:05.334453+0000","last_active":"2024-03-11T19:37:06.707645+0000","last_peered":"2024-03-11T19:37:06.707645+0000","last_clean":"2024-03-11T19:37:06.707645+0000","last_became_active":"2024-03-11T19:37:05.334009+0000","last_became_peered":"2024-03-11T19:37:05.334009+0000","last_unstale":"2024-03-11T19:37:06.707645+0000","last_undegraded":"2024-03-11T19:37:06.707645+0000","last_fullsized":"2024-03-11T19:37:06.707645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:43.402674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654392+0000","last_change":"2024-03-11T19:37:04.583181+0000","last_active":"2024-03-11T19:37:06.654392+0000","last_peered":"2024-03-11T19:37:06.654392+0000","last_clean":"2024-03-11T19:37:06.654392+0000","last_became_active":"2024-03-11T19:37:04.582881+0000","last_became_peered":"2024-03-11T19:37:04.582881+0000","last_unstale":"2024-03-11T19:37:06.654392+0000","last_undegraded":"2024-03-11T19:37:06.654392+0000","last_fullsized":"2024-03-11T19:37:06.654392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:05:06.634150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654891+0000","last_change":"2024-03-11T19:37:05.333642+0000","last_active":"2024-03-11T19:37:06.654891+0000","last_peered":"2024-03-11T19:37:06.654891+0000","last_clean":"2024-03-11T19:37:06.654891+0000","last_became_active":"2024-03-11T19:37:05.333121+0000","last_became_peered":"2024-03-11T19:37:05.333121+0000","last_unstale":"2024-03-11T19:37:06.654891+0000","last_undegraded":"2024-03-11T19:37:06.654891+0000","last_fullsized":"2024-03-11T19:37:06.654891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:07:21.217101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.324279+0000","last_change":"2024-03-11T19:37:04.586235+0000","last_active":"2024-03-11T19:37:05.324279+0000","last_peered":"2024-03-11T19:37:05.324279+0000","last_clean":"2024-03-11T19:37:05.324279+0000","last_became_active":"2024-03-11T19:37:04.585928+0000","last_became_peered":"2024-03-11T19:37:04.585928+0000","last_unstale":"2024-03-11T19:37:05.324279+0000","last_undegraded":"2024-03-11T19:37:05.324279+0000","last_fullsized":"2024-03-11T19:37:05.324279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:06:22.840169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.331179+0000","last_change":"2024-03-11T19:37:05.326992+0000","last_active":"2024-03-11T19:37:05.331179+0000","last_peered":"2024-03-11T19:37:05.331179+0000","last_clean":"2024-03-11T19:37:05.331179+0000","last_became_active":"2024-03-11T19:37:05.326676+0000","last_became_peered":"2024-03-11T19:37:05.326676+0000","last_unstale":"2024-03-11T19:37:05.331179+0000","last_undegraded":"2024-03-11T19:37:05.331179+0000","last_fullsized":"2024-03-11T19:37:05.331179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:00:32.101812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654908+0000","last_change":"2024-03-11T19:37:04.583631+0000","last_active":"2024-03-11T19:37:06.654908+0000","last_peered":"2024-03-11T19:37:06.654908+0000","last_clean":"2024-03-11T19:37:06.654908+0000","last_became_active":"2024-03-11T19:37:04.583031+0000","last_became_peered":"2024-03-11T19:37:04.583031+0000","last_unstale":"2024-03-11T19:37:06.654908+0000","last_undegraded":"2024-03-11T19:37:06.654908+0000","last_fullsized":"2024-03-11T19:37:06.654908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:20.695496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.332004+0000","last_change":"2024-03-11T19:37:05.331008+0000","last_active":"2024-03-11T19:37:05.332004+0000","last_peered":"2024-03-11T19:37:05.332004+0000","last_clean":"2024-03-11T19:37:05.332004+0000","last_became_active":"2024-03-11T19:37:05.330647+0000","last_became_peered":"2024-03-11T19:37:05.330647+0000","last_unstale":"2024-03-11T19:37:05.332004+0000","last_undegraded":"2024-03-11T19:37:05.332004+0000","last_fullsized":"2024-03-11T19:37:05.332004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:18:24.716254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.320252+0000","last_change":"2024-03-11T19:37:04.582501+0000","last_active":"2024-03-11T19:37:05.320252+0000","last_peered":"2024-03-11T19:37:05.320252+0000","last_clean":"2024-03-11T19:37:05.320252+0000","last_became_active":"2024-03-11T19:37:04.582076+0000","last_became_peered":"2024-03-11T19:37:04.582076+0000","last_unstale":"2024-03-11T19:37:05.320252+0000","last_undegraded":"2024-03-11T19:37:05.320252+0000","last_fullsized":"2024-03-11T19:37:05.320252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:46:05.359211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.331596+0000","last_change":"2024-03-11T19:37:05.329053+0000","last_active":"2024-03-11T19:37:05.331596+0000","last_peered":"2024-03-11T19:37:05.331596+0000","last_clean":"2024-03-11T19:37:05.331596+0000","last_became_active":"2024-03-11T19:37:05.328748+0000","last_became_peered":"2024-03-11T19:37:05.328748+0000","last_unstale":"2024-03-11T19:37:05.331596+0000","last_undegraded":"2024-03-11T19:37:05.331596+0000","last_fullsized":"2024-03-11T19:37:05.331596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:31:07.486850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.323188+0000","last_change":"2024-03-11T19:37:04.583370+0000","last_active":"2024-03-11T19:37:06.323188+0000","last_peered":"2024-03-11T19:37:06.323188+0000","last_clean":"2024-03-11T19:37:06.323188+0000","last_became_active":"2024-03-11T19:37:04.583055+0000","last_became_peered":"2024-03-11T19:37:04.583055+0000","last_unstale":"2024-03-11T19:37:06.323188+0000","last_undegraded":"2024-03-11T19:37:06.323188+0000","last_fullsized":"2024-03-11T19:37:06.323188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:04:02.325054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.331619+0000","last_change":"2024-03-11T19:37:05.328157+0000","last_active":"2024-03-11T19:37:05.331619+0000","last_peered":"2024-03-11T19:37:05.331619+0000","last_clean":"2024-03-11T19:37:05.331619+0000","last_became_active":"2024-03-11T19:37:05.327839+0000","last_became_peered":"2024-03-11T19:37:05.327839+0000","last_unstale":"2024-03-11T19:37:05.331619+0000","last_undegraded":"2024-03-11T19:37:05.331619+0000","last_fullsized":"2024-03-11T19:37:05.331619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:19:44.396402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707661+0000","last_change":"2024-03-11T19:37:05.336160+0000","last_active":"2024-03-11T19:37:06.707661+0000","last_peered":"2024-03-11T19:37:06.707661+0000","last_clean":"2024-03-11T19:37:06.707661+0000","last_became_active":"2024-03-11T19:37:05.335850+0000","last_became_peered":"2024-03-11T19:37:05.335850+0000","last_unstale":"2024-03-11T19:37:06.707661+0000","last_undegraded":"2024-03-11T19:37:06.707661+0000","last_fullsized":"2024-03-11T19:37:06.707661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:09.743661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654699+0000","last_change":"2024-03-11T19:37:05.325332+0000","last_active":"2024-03-11T19:37:06.654699+0000","last_peered":"2024-03-11T19:37:06.654699+0000","last_clean":"2024-03-11T19:37:06.654699+0000","last_became_active":"2024-03-11T19:37:05.324948+0000","last_became_peered":"2024-03-11T19:37:05.324948+0000","last_unstale":"2024-03-11T19:37:06.654699+0000","last_undegraded":"2024-03-11T19:37:06.654699+0000","last_fullsized":"2024-03-11T19:37:06.654699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:31:04.454259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654307+0000","last_change":"2024-03-11T19:37:05.332357+0000","last_active":"2024-03-11T19:37:06.654307+0000","last_peered":"2024-03-11T19:37:06.654307+0000","last_clean":"2024-03-11T19:37:06.654307+0000","last_became_active":"2024-03-11T19:37:05.332003+0000","last_became_peered":"2024-03-11T19:37:05.332003+0000","last_unstale":"2024-03-11T19:37:06.654307+0000","last_undegraded":"2024-03-11T19:37:06.654307+0000","last_fullsized":"2024-03-11T19:37:06.654307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:39.165572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654299+0000","last_change":"2024-03-11T19:37:05.332388+0000","last_active":"2024-03-11T19:37:06.654299+0000","last_peered":"2024-03-11T19:37:06.654299+0000","last_clean":"2024-03-11T19:37:06.654299+0000","last_became_active":"2024-03-11T19:37:05.332032+0000","last_became_peered":"2024-03-11T19:37:05.332032+0000","last_unstale":"2024-03-11T19:37:06.654299+0000","last_undegraded":"2024-03-11T19:37:06.654299+0000","last_fullsized":"2024-03-11T19:37:06.654299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:56:09.416683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.323249+0000","last_change":"2024-03-11T19:37:05.323096+0000","last_active":"2024-03-11T19:37:06.323249+0000","last_peered":"2024-03-11T19:37:06.323249+0000","last_clean":"2024-03-11T19:37:06.323249+0000","last_became_active":"2024-03-11T19:37:05.322525+0000","last_became_peered":"2024-03-11T19:37:05.322525+0000","last_unstale":"2024-03-11T19:37:06.323249+0000","last_undegraded":"2024-03-11T19:37:06.323249+0000","last_fullsized":"2024-03-11T19:37:06.323249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:53:01.870894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707739+0000","last_change":"2024-03-11T19:37:05.331171+0000","last_active":"2024-03-11T19:37:06.707739+0000","last_peered":"2024-03-11T19:37:06.707739+0000","last_clean":"2024-03-11T19:37:06.707739+0000","last_became_active":"2024-03-11T19:37:05.330769+0000","last_became_peered":"2024-03-11T19:37:05.330769+0000","last_unstale":"2024-03-11T19:37:06.707739+0000","last_undegraded":"2024-03-11T19:37:06.707739+0000","last_fullsized":"2024-03-11T19:37:06.707739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:03:06.514039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655145+0000","last_change":"2024-03-11T19:37:05.326877+0000","last_active":"2024-03-11T19:37:06.655145+0000","last_peered":"2024-03-11T19:37:06.655145+0000","last_clean":"2024-03-11T19:37:06.655145+0000","last_became_active":"2024-03-11T19:37:05.326538+0000","last_became_peered":"2024-03-11T19:37:05.326538+0000","last_unstale":"2024-03-11T19:37:06.655145+0000","last_undegraded":"2024-03-11T19:37:06.655145+0000","last_fullsized":"2024-03-11T19:37:06.655145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:11.565697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655182+0000","last_change":"2024-03-11T19:37:05.335010+0000","last_active":"2024-03-11T19:37:06.655182+0000","last_peered":"2024-03-11T19:37:06.655182+0000","last_clean":"2024-03-11T19:37:06.655182+0000","last_became_active":"2024-03-11T19:37:05.334551+0000","last_became_peered":"2024-03-11T19:37:05.334551+0000","last_unstale":"2024-03-11T19:37:06.655182+0000","last_undegraded":"2024-03-11T19:37:06.655182+0000","last_fullsized":"2024-03-11T19:37:06.655182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:59.668178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-11T19:37:05.337178+0000","last_change":"2024-03-11T19:37:05.336273+0000","last_active":"2024-03-11T19:37:05.337178+0000","last_peered":"2024-03-11T19:37:05.337178+0000","last_clean":"2024-03-11T19:37:05.337178+0000","last_became_active":"2024-03-11T19:37:05.335926+0000","last_became_peered":"2024-03-11T19:37:05.335926+0000","last_unstale":"2024-03-11T19:37:05.337178+0000","last_undegraded":"2024-03-11T19:37:05.337178+0000","last_fullsized":"2024-03-11T19:37:05.337178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:40:19.039505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654555+0000","last_change":"2024-03-11T19:37:05.335548+0000","last_active":"2024-03-11T19:37:06.654555+0000","last_peered":"2024-03-11T19:37:06.654555+0000","last_clean":"2024-03-11T19:37:06.654555+0000","last_became_active":"2024-03-11T19:37:05.335222+0000","last_became_peered":"2024-03-11T19:37:05.335222+0000","last_unstale":"2024-03-11T19:37:06.654555+0000","last_undegraded":"2024-03-11T19:37:06.654555+0000","last_fullsized":"2024-03-11T19:37:06.654555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:25.697951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654780+0000","last_change":"2024-03-11T19:37:05.329782+0000","last_active":"2024-03-11T19:37:06.654780+0000","last_peered":"2024-03-11T19:37:06.654780+0000","last_clean":"2024-03-11T19:37:06.654780+0000","last_became_active":"2024-03-11T19:37:05.329434+0000","last_became_peered":"2024-03-11T19:37:05.329434+0000","last_unstale":"2024-03-11T19:37:06.654780+0000","last_undegraded":"2024-03-11T19:37:06.654780+0000","last_fullsized":"2024-03-11T19:37:06.654780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:12:00.516489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.323163+0000","last_change":"2024-03-11T19:37:05.323026+0000","last_active":"2024-03-11T19:37:06.323163+0000","last_peered":"2024-03-11T19:37:06.323163+0000","last_clean":"2024-03-11T19:37:06.323163+0000","last_became_active":"2024-03-11T19:37:05.322562+0000","last_became_peered":"2024-03-11T19:37:05.322562+0000","last_unstale":"2024-03-11T19:37:06.323163+0000","last_undegraded":"2024-03-11T19:37:06.323163+0000","last_fullsized":"2024-03-11T19:37:06.323163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:14:57.606404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707829+0000","last_change":"2024-03-11T19:37:05.333342+0000","last_active":"2024-03-11T19:37:06.707829+0000","last_peered":"2024-03-11T19:37:06.707829+0000","last_clean":"2024-03-11T19:37:06.707829+0000","last_became_active":"2024-03-11T19:37:05.332972+0000","last_became_peered":"2024-03-11T19:37:05.332972+0000","last_unstale":"2024-03-11T19:37:06.707829+0000","last_undegraded":"2024-03-11T19:37:06.707829+0000","last_fullsized":"2024-03-11T19:37:06.707829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:57:27.372761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2318,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":9418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":438135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32676,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712476,"statfs":{"total":95995035648,"available":95961575424,"internally_reserved":0,"allocated":1531904,"data_stored":1030309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":438919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070491,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":420488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332127,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1495040,"data_stored":1019933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495654,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":438135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":439747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":439613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":440403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":438239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920816,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":905216,"data_stored":429587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757290,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1482752,"data_stored":1010856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:37:12.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T19:37:12.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.431+0000 7f58bcb72640 1 -- 172.21.15.83:0/3968168470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b81000a0 msgr2=0x7f58b81004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:12.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.431+0000 7f58bcb72640 1 --2- 172.21.15.83:0/3968168470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b81000a0 0x7f58b81004c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009900 tx=0x7f58ac02bc30 comp rx=0 tx=0).stop 2024-03-11T19:37:12.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.431+0000 7f58bcb72640 1 -- 172.21.15.83:0/3968168470 shutdown_connections 2024-03-11T19:37:12.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.431+0000 7f58bcb72640 1 --2- 172.21.15.83:0/3968168470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58b8101c50 0x7f58b81084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.431+0000 7f58bcb72640 1 --2- 172.21.15.83:0/3968168470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b8101290 0x7f58b8101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.431+0000 7f58bcb72640 1 --2- 172.21.15.83:0/3968168470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b81000a0 0x7f58b81004c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.431+0000 7f58bcb72640 1 -- 172.21.15.83:0/3968168470 >> 172.21.15.83:0/3968168470 conn(0x7f58b80fb610 msgr2=0x7f58b80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.431+0000 7f58bcb72640 1 -- 172.21.15.83:0/3968168470 shutdown_connections 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.431+0000 7f58bcb72640 1 -- 172.21.15.83:0/3968168470 wait complete. 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.432+0000 7f58bcb72640 1 Processor -- start 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.432+0000 7f58bcb72640 1 -- start start 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.432+0000 7f58bcb72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b81000a0 0x7f58b8104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.432+0000 7f58bcb72640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58b8101290 0x7f58b81027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.432+0000 7f58bcb72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b8101c50 0x7f58b8102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.432+0000 7f58bcb72640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8103460 con 0x7f58b81000a0 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.432+0000 7f58bcb72640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81035d0 con 0x7f58b8101c50 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.432+0000 7f58bcb72640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81a12c0 con 0x7f58b8101290 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.432+0000 7f58b659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b81000a0 0x7f58b8104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.432+0000 7f58b659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b81000a0 0x7f58b8104180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60676/0 (socket says 172.21.15.83:60676) 2024-03-11T19:37:12.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.432+0000 7f58b659c640 1 -- 172.21.15.83:0/2740405515 learned_addr learned my addr 172.21.15.83:0/2740405515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58b6d9d640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b8101c50 0x7f58b8102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58b659c640 1 -- 172.21.15.83:0/2740405515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58b8101290 msgr2=0x7f58b81027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58b659c640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58b8101290 0x7f58b81027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58b659c640 1 -- 172.21.15.83:0/2740405515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b8101c50 msgr2=0x7f58b8102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58b659c640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b8101c50 0x7f58b8102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58b659c640 1 -- 172.21.15.83:0/2740405515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac009560 con 0x7f58b81000a0 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58b6d9d640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b8101c50 0x7f58b8102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58b659c640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b81000a0 0x7f58b8104180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58ac036040 tx=0x7f58ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58a77fe640 1 -- 172.21.15.83:0/2740405515 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac004df0 con 0x7f58b81000a0 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58a77fe640 1 -- 172.21.15.83:0/2740405515 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58ac005430 con 0x7f58b81000a0 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58bcb72640 1 -- 172.21.15.83:0/2740405515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b81a1510 con 0x7f58b81000a0 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58a77fe640 1 -- 172.21.15.83:0/2740405515 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac03e450 con 0x7f58b81000a0 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.433+0000 7f58bcb72640 1 -- 172.21.15.83:0/2740405515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b81a1a20 con 0x7f58b81000a0 2024-03-11T19:37:12.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.434+0000 7f58bcb72640 1 -- 172.21.15.83:0/2740405515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b810bab0 con 0x7f58b81000a0 2024-03-11T19:37:12.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.435+0000 7f58a77fe640 1 -- 172.21.15.83:0/2740405515 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f58ac003680 con 0x7f58b81000a0 2024-03-11T19:37:12.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.435+0000 7f58a77fe640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58a00a1ac0 0x7f58a00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.436+0000 7f58b5d9b640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58a00a1ac0 0x7f58a00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.436+0000 7f58a77fe640 1 -- 172.21.15.83:0/2740405515 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f58ac0ef8e0 con 0x7f58b81000a0 2024-03-11T19:37:12.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.436+0000 7f58b5d9b640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58a00a1ac0 0x7f58a00a3f80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f58b8103ef0 tx=0x7f58a8005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:12.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.440+0000 7f58a77fe640 1 -- 172.21.15.83:0/2740405515 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58ac0b8c50 con 0x7f58b81000a0 2024-03-11T19:37:12.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.614+0000 7f58bcb72640 1 -- 172.21.15.83:0/2740405515 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58b80689f0 con 0x7f58a00a1ac0 2024-03-11T19:37:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.621+0000 7f58a77fe640 1 -- 172.21.15.83:0/2740405515 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258742 (secure 0 0 0) 0x7f58b80689f0 con 0x7f58a00a1ac0 2024-03-11T19:37:12.628 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:12.630 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-03-11T19:37:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 -- 172.21.15.83:0/2740405515 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58a00a1ac0 msgr2=0x7f58a00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58a00a1ac0 0x7f58a00a3f80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f58b8103ef0 tx=0x7f58a8005bf0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 -- 172.21.15.83:0/2740405515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b81000a0 msgr2=0x7f58b8104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b81000a0 0x7f58b8104180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58ac036040 tx=0x7f58ac004270 comp rx=0 tx=0).stop 2024-03-11T19:37:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 -- 172.21.15.83:0/2740405515 shutdown_connections 2024-03-11T19:37:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58a00a1ac0 0x7f58a00a3f80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58b8101c50 0x7f58b8102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58b8101290 0x7f58b81027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 --2- 172.21.15.83:0/2740405515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58b81000a0 0x7f58b8104180 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 -- 172.21.15.83:0/2740405515 >> 172.21.15.83:0/2740405515 conn(0x7f58b80fb610 msgr2=0x7f58b80682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 -- 172.21.15.83:0/2740405515 shutdown_connections 2024-03-11T19:37:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:12.624+0000 7f58bcb72640 1 -- 172.21.15.83:0/2740405515 wait complete. 2024-03-11T19:37:12.640 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-03-11T19:37:12.508740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595817,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":149,"num_read_kb":128,"num_write":285,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":254,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365992,"kb_used_data":12640,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575832,"statfs":{"total":1151940427776,"available":1151565651968,"internally_reserved":0,"allocated":12943360,"data_stored":7043197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5449,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":104,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.940127"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707995+0000","last_change":"2024-03-11T19:37:05.340476+0000","last_active":"2024-03-11T19:37:06.707995+0000","last_peered":"2024-03-11T19:37:06.707995+0000","last_clean":"2024-03-11T19:37:06.707995+0000","last_became_active":"2024-03-11T19:37:05.340140+0000","last_became_peered":"2024-03-11T19:37:05.340140+0000","last_unstale":"2024-03-11T19:37:06.707995+0000","last_undegraded":"2024-03-11T19:37:06.707995+0000","last_fullsized":"2024-03-11T19:37:06.707995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:10:35.677374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654563+0000","last_change":"2024-03-11T19:37:05.335500+0000","last_active":"2024-03-11T19:37:06.654563+0000","last_peered":"2024-03-11T19:37:06.654563+0000","last_clean":"2024-03-11T19:37:06.654563+0000","last_became_active":"2024-03-11T19:37:05.335194+0000","last_became_peered":"2024-03-11T19:37:05.335194+0000","last_unstale":"2024-03-11T19:37:06.654563+0000","last_undegraded":"2024-03-11T19:37:06.654563+0000","last_fullsized":"2024-03-11T19:37:06.654563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:03.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654845+0000","last_change":"2024-03-11T19:37:05.334618+0000","last_active":"2024-03-11T19:37:06.654845+0000","last_peered":"2024-03-11T19:37:06.654845+0000","last_clean":"2024-03-11T19:37:06.654845+0000","last_became_active":"2024-03-11T19:37:05.334281+0000","last_became_peered":"2024-03-11T19:37:05.334281+0000","last_unstale":"2024-03-11T19:37:06.654845+0000","last_undegraded":"2024-03-11T19:37:06.654845+0000","last_fullsized":"2024-03-11T19:37:06.654845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:43:22.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655643+0000","last_change":"2024-03-11T19:37:05.332978+0000","last_active":"2024-03-11T19:37:06.655643+0000","last_peered":"2024-03-11T19:37:06.655643+0000","last_clean":"2024-03-11T19:37:06.655643+0000","last_became_active":"2024-03-11T19:37:05.330888+0000","last_became_peered":"2024-03-11T19:37:05.330888+0000","last_unstale":"2024-03-11T19:37:06.655643+0000","last_undegraded":"2024-03-11T19:37:06.655643+0000","last_fullsized":"2024-03-11T19:37:06.655643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:40:36.988348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654390+0000","last_change":"2024-03-11T19:37:05.329050+0000","last_active":"2024-03-11T19:37:06.654390+0000","last_peered":"2024-03-11T19:37:06.654390+0000","last_clean":"2024-03-11T19:37:06.654390+0000","last_became_active":"2024-03-11T19:37:05.328702+0000","last_became_peered":"2024-03-11T19:37:05.328702+0000","last_unstale":"2024-03-11T19:37:06.654390+0000","last_undegraded":"2024-03-11T19:37:06.654390+0000","last_fullsized":"2024-03-11T19:37:06.654390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:19:52.180884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654655+0000","last_change":"2024-03-11T19:37:05.322748+0000","last_active":"2024-03-11T19:37:06.654655+0000","last_peered":"2024-03-11T19:37:06.654655+0000","last_clean":"2024-03-11T19:37:06.654655+0000","last_became_active":"2024-03-11T19:37:05.322376+0000","last_became_peered":"2024-03-11T19:37:05.322376+0000","last_unstale":"2024-03-11T19:37:06.654655+0000","last_undegraded":"2024-03-11T19:37:06.654655+0000","last_fullsized":"2024-03-11T19:37:06.654655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:40.926854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321339+0000","last_change":"2024-03-11T19:37:05.323954+0000","last_active":"2024-03-11T19:37:06.321339+0000","last_peered":"2024-03-11T19:37:06.321339+0000","last_clean":"2024-03-11T19:37:06.321339+0000","last_became_active":"2024-03-11T19:37:05.323586+0000","last_became_peered":"2024-03-11T19:37:05.323586+0000","last_unstale":"2024-03-11T19:37:06.321339+0000","last_undegraded":"2024-03-11T19:37:06.321339+0000","last_fullsized":"2024-03-11T19:37:06.321339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:38.286891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654048+0000","last_change":"2024-03-11T19:37:05.334138+0000","last_active":"2024-03-11T19:37:06.654048+0000","last_peered":"2024-03-11T19:37:06.654048+0000","last_clean":"2024-03-11T19:37:06.654048+0000","last_became_active":"2024-03-11T19:37:05.333837+0000","last_became_peered":"2024-03-11T19:37:05.333837+0000","last_unstale":"2024-03-11T19:37:06.654048+0000","last_undegraded":"2024-03-11T19:37:06.654048+0000","last_fullsized":"2024-03-11T19:37:06.654048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:49:30.724176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655084+0000","last_change":"2024-03-11T19:37:05.336888+0000","last_active":"2024-03-11T19:37:06.655084+0000","last_peered":"2024-03-11T19:37:06.655084+0000","last_clean":"2024-03-11T19:37:06.655084+0000","last_became_active":"2024-03-11T19:37:05.336256+0000","last_became_peered":"2024-03-11T19:37:05.336256+0000","last_unstale":"2024-03-11T19:37:06.655084+0000","last_undegraded":"2024-03-11T19:37:06.655084+0000","last_fullsized":"2024-03-11T19:37:06.655084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:54:31.526986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655196+0000","last_change":"2024-03-11T19:37:05.321959+0000","last_active":"2024-03-11T19:37:06.655196+0000","last_peered":"2024-03-11T19:37:06.655196+0000","last_clean":"2024-03-11T19:37:06.655196+0000","last_became_active":"2024-03-11T19:37:05.321043+0000","last_became_peered":"2024-03-11T19:37:05.321043+0000","last_unstale":"2024-03-11T19:37:06.655196+0000","last_undegraded":"2024-03-11T19:37:06.655196+0000","last_fullsized":"2024-03-11T19:37:06.655196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:13.121003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321760+0000","last_change":"2024-03-11T19:37:05.324940+0000","last_active":"2024-03-11T19:37:06.321760+0000","last_peered":"2024-03-11T19:37:06.321760+0000","last_clean":"2024-03-11T19:37:06.321760+0000","last_became_active":"2024-03-11T19:37:05.323427+0000","last_became_peered":"2024-03-11T19:37:05.323427+0000","last_unstale":"2024-03-11T19:37:06.321760+0000","last_undegraded":"2024-03-11T19:37:06.321760+0000","last_fullsized":"2024-03-11T19:37:06.321760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:23.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655341+0000","last_change":"2024-03-11T19:37:05.332694+0000","last_active":"2024-03-11T19:37:06.655341+0000","last_peered":"2024-03-11T19:37:06.655341+0000","last_clean":"2024-03-11T19:37:06.655341+0000","last_became_active":"2024-03-11T19:37:05.330446+0000","last_became_peered":"2024-03-11T19:37:05.330446+0000","last_unstale":"2024-03-11T19:37:06.655341+0000","last_undegraded":"2024-03-11T19:37:06.655341+0000","last_fullsized":"2024-03-11T19:37:06.655341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:37:40.196933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.323343+0000","last_change":"2024-03-11T19:37:05.325975+0000","last_active":"2024-03-11T19:37:06.323343+0000","last_peered":"2024-03-11T19:37:06.323343+0000","last_clean":"2024-03-11T19:37:06.323343+0000","last_became_active":"2024-03-11T19:37:05.325649+0000","last_became_peered":"2024-03-11T19:37:05.325649+0000","last_unstale":"2024-03-11T19:37:06.323343+0000","last_undegraded":"2024-03-11T19:37:06.323343+0000","last_fullsized":"2024-03-11T19:37:06.323343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:36.373272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654742+0000","last_change":"2024-03-11T19:37:05.335546+0000","last_active":"2024-03-11T19:37:06.654742+0000","last_peered":"2024-03-11T19:37:06.654742+0000","last_clean":"2024-03-11T19:37:06.654742+0000","last_became_active":"2024-03-11T19:37:05.335220+0000","last_became_peered":"2024-03-11T19:37:05.335220+0000","last_unstale":"2024-03-11T19:37:06.654742+0000","last_undegraded":"2024-03-11T19:37:06.654742+0000","last_fullsized":"2024-03-11T19:37:06.654742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:13.116355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654198+0000","last_change":"2024-03-11T19:37:05.338955+0000","last_active":"2024-03-11T19:37:06.654198+0000","last_peered":"2024-03-11T19:37:06.654198+0000","last_clean":"2024-03-11T19:37:06.654198+0000","last_became_active":"2024-03-11T19:37:05.338574+0000","last_became_peered":"2024-03-11T19:37:05.338574+0000","last_unstale":"2024-03-11T19:37:06.654198+0000","last_undegraded":"2024-03-11T19:37:06.654198+0000","last_fullsized":"2024-03-11T19:37:06.654198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:27:27.754077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654332+0000","last_change":"2024-03-11T19:37:05.334578+0000","last_active":"2024-03-11T19:37:06.654332+0000","last_peered":"2024-03-11T19:37:06.654332+0000","last_clean":"2024-03-11T19:37:06.654332+0000","last_became_active":"2024-03-11T19:37:05.334267+0000","last_became_peered":"2024-03-11T19:37:05.334267+0000","last_unstale":"2024-03-11T19:37:06.654332+0000","last_undegraded":"2024-03-11T19:37:06.654332+0000","last_fullsized":"2024-03-11T19:37:06.654332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:31.704187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707867+0000","last_change":"2024-03-11T19:37:05.341178+0000","last_active":"2024-03-11T19:37:06.707867+0000","last_peered":"2024-03-11T19:37:06.707867+0000","last_clean":"2024-03-11T19:37:06.707867+0000","last_became_active":"2024-03-11T19:37:05.340837+0000","last_became_peered":"2024-03-11T19:37:05.340837+0000","last_unstale":"2024-03-11T19:37:06.707867+0000","last_undegraded":"2024-03-11T19:37:06.707867+0000","last_fullsized":"2024-03-11T19:37:06.707867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:39:54.460937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655135+0000","last_change":"2024-03-11T19:37:05.330981+0000","last_active":"2024-03-11T19:37:06.655135+0000","last_peered":"2024-03-11T19:37:06.655135+0000","last_clean":"2024-03-11T19:37:06.655135+0000","last_became_active":"2024-03-11T19:37:05.330638+0000","last_became_peered":"2024-03-11T19:37:05.330638+0000","last_unstale":"2024-03-11T19:37:06.655135+0000","last_undegraded":"2024-03-11T19:37:06.655135+0000","last_fullsized":"2024-03-11T19:37:06.655135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:36:12.565936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655123+0000","last_change":"2024-03-11T19:37:05.329866+0000","last_active":"2024-03-11T19:37:06.655123+0000","last_peered":"2024-03-11T19:37:06.655123+0000","last_clean":"2024-03-11T19:37:06.655123+0000","last_became_active":"2024-03-11T19:37:05.329572+0000","last_became_peered":"2024-03-11T19:37:05.329572+0000","last_unstale":"2024-03-11T19:37:06.655123+0000","last_undegraded":"2024-03-11T19:37:06.655123+0000","last_fullsized":"2024-03-11T19:37:06.655123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:40:04.482929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655063+0000","last_change":"2024-03-11T19:37:04.587149+0000","last_active":"2024-03-11T19:37:06.655063+0000","last_peered":"2024-03-11T19:37:06.655063+0000","last_clean":"2024-03-11T19:37:06.655063+0000","last_became_active":"2024-03-11T19:37:04.586013+0000","last_became_peered":"2024-03-11T19:37:04.586013+0000","last_unstale":"2024-03-11T19:37:06.655063+0000","last_undegraded":"2024-03-11T19:37:06.655063+0000","last_fullsized":"2024-03-11T19:37:06.655063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:15.758154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654974+0000","last_change":"2024-03-11T19:37:05.327979+0000","last_active":"2024-03-11T19:37:06.654974+0000","last_peered":"2024-03-11T19:37:06.654974+0000","last_clean":"2024-03-11T19:37:06.654974+0000","last_became_active":"2024-03-11T19:37:05.327647+0000","last_became_peered":"2024-03-11T19:37:05.327647+0000","last_unstale":"2024-03-11T19:37:06.654974+0000","last_undegraded":"2024-03-11T19:37:06.654974+0000","last_fullsized":"2024-03-11T19:37:06.654974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:09.514766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655141+0000","last_change":"2024-03-11T19:37:04.582937+0000","last_active":"2024-03-11T19:37:06.655141+0000","last_peered":"2024-03-11T19:37:06.655141+0000","last_clean":"2024-03-11T19:37:06.655141+0000","last_became_active":"2024-03-11T19:37:04.582622+0000","last_became_peered":"2024-03-11T19:37:04.582622+0000","last_unstale":"2024-03-11T19:37:06.655141+0000","last_undegraded":"2024-03-11T19:37:06.655141+0000","last_fullsized":"2024-03-11T19:37:06.655141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:02.275698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654213+0000","last_change":"2024-03-11T19:37:05.327191+0000","last_active":"2024-03-11T19:37:06.654213+0000","last_peered":"2024-03-11T19:37:06.654213+0000","last_clean":"2024-03-11T19:37:06.654213+0000","last_became_active":"2024-03-11T19:37:05.326001+0000","last_became_peered":"2024-03-11T19:37:05.326001+0000","last_unstale":"2024-03-11T19:37:06.654213+0000","last_undegraded":"2024-03-11T19:37:06.654213+0000","last_fullsized":"2024-03-11T19:37:06.654213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:27.301321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:08.624438+0000","last_change":"2024-03-11T19:37:04.587054+0000","last_active":"2024-03-11T19:37:08.624438+0000","last_peered":"2024-03-11T19:37:08.624438+0000","last_clean":"2024-03-11T19:37:08.624438+0000","last_became_active":"2024-03-11T19:37:04.586104+0000","last_became_peered":"2024-03-11T19:37:04.586104+0000","last_unstale":"2024-03-11T19:37:08.624438+0000","last_undegraded":"2024-03-11T19:37:08.624438+0000","last_fullsized":"2024-03-11T19:37:08.624438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:34.913677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.656102+0000","last_change":"2024-03-11T19:37:05.333288+0000","last_active":"2024-03-11T19:37:06.656102+0000","last_peered":"2024-03-11T19:37:06.656102+0000","last_clean":"2024-03-11T19:37:06.656102+0000","last_became_active":"2024-03-11T19:37:05.332882+0000","last_became_peered":"2024-03-11T19:37:05.332882+0000","last_unstale":"2024-03-11T19:37:06.656102+0000","last_undegraded":"2024-03-11T19:37:06.656102+0000","last_fullsized":"2024-03-11T19:37:06.656102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:45.680608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654466+0000","last_change":"2024-03-11T19:37:04.583660+0000","last_active":"2024-03-11T19:37:06.654466+0000","last_peered":"2024-03-11T19:37:06.654466+0000","last_clean":"2024-03-11T19:37:06.654466+0000","last_became_active":"2024-03-11T19:37:04.583360+0000","last_became_peered":"2024-03-11T19:37:04.583360+0000","last_unstale":"2024-03-11T19:37:06.654466+0000","last_undegraded":"2024-03-11T19:37:06.654466+0000","last_fullsized":"2024-03-11T19:37:06.654466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:00.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321825+0000","last_change":"2024-03-11T19:37:05.335649+0000","last_active":"2024-03-11T19:37:06.321825+0000","last_peered":"2024-03-11T19:37:06.321825+0000","last_clean":"2024-03-11T19:37:06.321825+0000","last_became_active":"2024-03-11T19:37:05.335255+0000","last_became_peered":"2024-03-11T19:37:05.335255+0000","last_unstale":"2024-03-11T19:37:06.321825+0000","last_undegraded":"2024-03-11T19:37:06.321825+0000","last_fullsized":"2024-03-11T19:37:06.321825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:49:09.772765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:09.629070+0000","last_change":"2024-03-11T19:37:04.586002+0000","last_active":"2024-03-11T19:37:09.629070+0000","last_peered":"2024-03-11T19:37:09.629070+0000","last_clean":"2024-03-11T19:37:09.629070+0000","last_became_active":"2024-03-11T19:37:04.585551+0000","last_became_peered":"2024-03-11T19:37:04.585551+0000","last_unstale":"2024-03-11T19:37:09.629070+0000","last_undegraded":"2024-03-11T19:37:09.629070+0000","last_fullsized":"2024-03-11T19:37:09.629070+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:34:12.663801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654201+0000","last_change":"2024-03-11T19:37:05.325168+0000","last_active":"2024-03-11T19:37:06.654201+0000","last_peered":"2024-03-11T19:37:06.654201+0000","last_clean":"2024-03-11T19:37:06.654201+0000","last_became_active":"2024-03-11T19:37:05.324848+0000","last_became_peered":"2024-03-11T19:37:05.324848+0000","last_unstale":"2024-03-11T19:37:06.654201+0000","last_undegraded":"2024-03-11T19:37:06.654201+0000","last_fullsized":"2024-03-11T19:37:06.654201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:27.423877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654399+0000","last_change":"2024-03-11T19:37:04.582727+0000","last_active":"2024-03-11T19:37:06.654399+0000","last_peered":"2024-03-11T19:37:06.654399+0000","last_clean":"2024-03-11T19:37:06.654399+0000","last_became_active":"2024-03-11T19:37:04.582359+0000","last_became_peered":"2024-03-11T19:37:04.582359+0000","last_unstale":"2024-03-11T19:37:06.654399+0000","last_undegraded":"2024-03-11T19:37:06.654399+0000","last_fullsized":"2024-03-11T19:37:06.654399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:05.591537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321629+0000","last_change":"2024-03-11T19:37:05.335442+0000","last_active":"2024-03-11T19:37:06.321629+0000","last_peered":"2024-03-11T19:37:06.321629+0000","last_clean":"2024-03-11T19:37:06.321629+0000","last_became_active":"2024-03-11T19:37:05.335003+0000","last_became_peered":"2024-03-11T19:37:05.335003+0000","last_unstale":"2024-03-11T19:37:06.321629+0000","last_undegraded":"2024-03-11T19:37:06.321629+0000","last_fullsized":"2024-03-11T19:37:06.321629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:41.590898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:08.624245+0000","last_change":"2024-03-11T19:37:04.589957+0000","last_active":"2024-03-11T19:37:08.624245+0000","last_peered":"2024-03-11T19:37:08.624245+0000","last_clean":"2024-03-11T19:37:08.624245+0000","last_became_active":"2024-03-11T19:37:04.589539+0000","last_became_peered":"2024-03-11T19:37:04.589539+0000","last_unstale":"2024-03-11T19:37:08.624245+0000","last_undegraded":"2024-03-11T19:37:08.624245+0000","last_fullsized":"2024-03-11T19:37:08.624245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:48:52.764057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.653973+0000","last_change":"2024-03-11T19:37:05.331380+0000","last_active":"2024-03-11T19:37:06.653973+0000","last_peered":"2024-03-11T19:37:06.653973+0000","last_clean":"2024-03-11T19:37:06.653973+0000","last_became_active":"2024-03-11T19:37:05.331000+0000","last_became_peered":"2024-03-11T19:37:05.331000+0000","last_unstale":"2024-03-11T19:37:06.653973+0000","last_undegraded":"2024-03-11T19:37:06.653973+0000","last_fullsized":"2024-03-11T19:37:06.653973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:58:47.278195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321337+0000","last_change":"2024-03-11T19:37:04.584128+0000","last_active":"2024-03-11T19:37:06.321337+0000","last_peered":"2024-03-11T19:37:06.321337+0000","last_clean":"2024-03-11T19:37:06.321337+0000","last_became_active":"2024-03-11T19:37:04.583842+0000","last_became_peered":"2024-03-11T19:37:04.583842+0000","last_unstale":"2024-03-11T19:37:06.321337+0000","last_undegraded":"2024-03-11T19:37:06.321337+0000","last_fullsized":"2024-03-11T19:37:06.321337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:18.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654396+0000","last_change":"2024-03-11T19:37:05.328999+0000","last_active":"2024-03-11T19:37:06.654396+0000","last_peered":"2024-03-11T19:37:06.654396+0000","last_clean":"2024-03-11T19:37:06.654396+0000","last_became_active":"2024-03-11T19:37:05.328620+0000","last_became_peered":"2024-03-11T19:37:05.328620+0000","last_unstale":"2024-03-11T19:37:06.654396+0000","last_undegraded":"2024-03-11T19:37:06.654396+0000","last_fullsized":"2024-03-11T19:37:06.654396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:40.191922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.614634+0000","last_change":"2024-03-11T19:37:04.587956+0000","last_active":"2024-03-11T19:37:07.614634+0000","last_peered":"2024-03-11T19:37:07.614634+0000","last_clean":"2024-03-11T19:37:07.614634+0000","last_became_active":"2024-03-11T19:37:04.587568+0000","last_became_peered":"2024-03-11T19:37:04.587568+0000","last_unstale":"2024-03-11T19:37:07.614634+0000","last_undegraded":"2024-03-11T19:37:07.614634+0000","last_fullsized":"2024-03-11T19:37:07.614634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:45:44.019436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655218+0000","last_change":"2024-03-11T19:37:05.336912+0000","last_active":"2024-03-11T19:37:06.655218+0000","last_peered":"2024-03-11T19:37:06.655218+0000","last_clean":"2024-03-11T19:37:06.655218+0000","last_became_active":"2024-03-11T19:37:05.336397+0000","last_became_peered":"2024-03-11T19:37:05.336397+0000","last_unstale":"2024-03-11T19:37:06.655218+0000","last_undegraded":"2024-03-11T19:37:06.655218+0000","last_fullsized":"2024-03-11T19:37:06.655218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:29.375396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.612542+0000","last_change":"2024-03-11T19:37:04.584382+0000","last_active":"2024-03-11T19:37:07.612542+0000","last_peered":"2024-03-11T19:37:07.612542+0000","last_clean":"2024-03-11T19:37:07.612542+0000","last_became_active":"2024-03-11T19:37:04.584026+0000","last_became_peered":"2024-03-11T19:37:04.584026+0000","last_unstale":"2024-03-11T19:37:07.612542+0000","last_undegraded":"2024-03-11T19:37:07.612542+0000","last_fullsized":"2024-03-11T19:37:07.612542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:28.152903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654957+0000","last_change":"2024-03-11T19:37:05.331385+0000","last_active":"2024-03-11T19:37:06.654957+0000","last_peered":"2024-03-11T19:37:06.654957+0000","last_clean":"2024-03-11T19:37:06.654957+0000","last_became_active":"2024-03-11T19:37:05.331069+0000","last_became_peered":"2024-03-11T19:37:05.331069+0000","last_unstale":"2024-03-11T19:37:06.654957+0000","last_undegraded":"2024-03-11T19:37:06.654957+0000","last_fullsized":"2024-03-11T19:37:06.654957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:18.375605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.613578+0000","last_change":"2024-03-11T19:37:04.586493+0000","last_active":"2024-03-11T19:37:07.613578+0000","last_peered":"2024-03-11T19:37:07.613578+0000","last_clean":"2024-03-11T19:37:07.613578+0000","last_became_active":"2024-03-11T19:37:04.586185+0000","last_became_peered":"2024-03-11T19:37:04.586185+0000","last_unstale":"2024-03-11T19:37:07.613578+0000","last_undegraded":"2024-03-11T19:37:07.613578+0000","last_fullsized":"2024-03-11T19:37:07.613578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:00.899752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.656135+0000","last_change":"2024-03-11T19:37:05.336414+0000","last_active":"2024-03-11T19:37:06.656135+0000","last_peered":"2024-03-11T19:37:06.656135+0000","last_clean":"2024-03-11T19:37:06.656135+0000","last_became_active":"2024-03-11T19:37:05.334536+0000","last_became_peered":"2024-03-11T19:37:05.334536+0000","last_unstale":"2024-03-11T19:37:06.656135+0000","last_undegraded":"2024-03-11T19:37:06.656135+0000","last_fullsized":"2024-03-11T19:37:06.656135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:50:19.109975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.613641+0000","last_change":"2024-03-11T19:37:04.584544+0000","last_active":"2024-03-11T19:37:07.613641+0000","last_peered":"2024-03-11T19:37:07.613641+0000","last_clean":"2024-03-11T19:37:07.613641+0000","last_became_active":"2024-03-11T19:37:04.584198+0000","last_became_peered":"2024-03-11T19:37:04.584198+0000","last_unstale":"2024-03-11T19:37:07.613641+0000","last_undegraded":"2024-03-11T19:37:07.613641+0000","last_fullsized":"2024-03-11T19:37:07.613641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:01:56.744533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654359+0000","last_change":"2024-03-11T19:37:05.331077+0000","last_active":"2024-03-11T19:37:06.654359+0000","last_peered":"2024-03-11T19:37:06.654359+0000","last_clean":"2024-03-11T19:37:06.654359+0000","last_became_active":"2024-03-11T19:37:05.330708+0000","last_became_peered":"2024-03-11T19:37:05.330708+0000","last_unstale":"2024-03-11T19:37:06.654359+0000","last_undegraded":"2024-03-11T19:37:06.654359+0000","last_fullsized":"2024-03-11T19:37:06.654359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:54:55.019268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.612623+0000","last_change":"2024-03-11T19:37:04.583353+0000","last_active":"2024-03-11T19:37:07.612623+0000","last_peered":"2024-03-11T19:37:07.612623+0000","last_clean":"2024-03-11T19:37:07.612623+0000","last_became_active":"2024-03-11T19:37:04.581974+0000","last_became_peered":"2024-03-11T19:37:04.581974+0000","last_unstale":"2024-03-11T19:37:07.612623+0000","last_undegraded":"2024-03-11T19:37:07.612623+0000","last_fullsized":"2024-03-11T19:37:07.612623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:41.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.322919+0000","last_change":"2024-03-11T19:37:05.329498+0000","last_active":"2024-03-11T19:37:06.322919+0000","last_peered":"2024-03-11T19:37:06.322919+0000","last_clean":"2024-03-11T19:37:06.322919+0000","last_became_active":"2024-03-11T19:37:05.329186+0000","last_became_peered":"2024-03-11T19:37:05.329186+0000","last_unstale":"2024-03-11T19:37:06.322919+0000","last_undegraded":"2024-03-11T19:37:06.322919+0000","last_fullsized":"2024-03-11T19:37:06.322919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:33:45.012049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.613221+0000","last_change":"2024-03-11T19:37:04.582027+0000","last_active":"2024-03-11T19:37:07.613221+0000","last_peered":"2024-03-11T19:37:07.613221+0000","last_clean":"2024-03-11T19:37:07.613221+0000","last_became_active":"2024-03-11T19:37:04.581678+0000","last_became_peered":"2024-03-11T19:37:04.581678+0000","last_unstale":"2024-03-11T19:37:07.613221+0000","last_undegraded":"2024-03-11T19:37:07.613221+0000","last_fullsized":"2024-03-11T19:37:07.613221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:03:40.688053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321671+0000","last_change":"2024-03-11T19:37:05.324499+0000","last_active":"2024-03-11T19:37:06.321671+0000","last_peered":"2024-03-11T19:37:06.321671+0000","last_clean":"2024-03-11T19:37:06.321671+0000","last_became_active":"2024-03-11T19:37:05.324156+0000","last_became_peered":"2024-03-11T19:37:05.324156+0000","last_unstale":"2024-03-11T19:37:06.321671+0000","last_undegraded":"2024-03-11T19:37:06.321671+0000","last_fullsized":"2024-03-11T19:37:06.321671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:34.903908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:10.631993+0000","last_change":"2024-03-11T19:37:04.576415+0000","last_active":"2024-03-11T19:37:10.631993+0000","last_peered":"2024-03-11T19:37:10.631993+0000","last_clean":"2024-03-11T19:37:10.631993+0000","last_became_active":"2024-03-11T19:37:04.575978+0000","last_became_peered":"2024-03-11T19:37:04.575978+0000","last_unstale":"2024-03-11T19:37:10.631993+0000","last_undegraded":"2024-03-11T19:37:10.631993+0000","last_fullsized":"2024-03-11T19:37:10.631993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:20.124813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321601+0000","last_change":"2024-03-11T19:37:05.326258+0000","last_active":"2024-03-11T19:37:06.321601+0000","last_peered":"2024-03-11T19:37:06.321601+0000","last_clean":"2024-03-11T19:37:06.321601+0000","last_became_active":"2024-03-11T19:37:05.325909+0000","last_became_peered":"2024-03-11T19:37:05.325909+0000","last_unstale":"2024-03-11T19:37:06.321601+0000","last_undegraded":"2024-03-11T19:37:06.321601+0000","last_fullsized":"2024-03-11T19:37:06.321601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:25.875603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:08.626377+0000","last_change":"2024-03-11T19:37:04.586696+0000","last_active":"2024-03-11T19:37:08.626377+0000","last_peered":"2024-03-11T19:37:08.626377+0000","last_clean":"2024-03-11T19:37:08.626377+0000","last_became_active":"2024-03-11T19:37:04.585993+0000","last_became_peered":"2024-03-11T19:37:04.585993+0000","last_unstale":"2024-03-11T19:37:08.626377+0000","last_undegraded":"2024-03-11T19:37:08.626377+0000","last_fullsized":"2024-03-11T19:37:08.626377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:59:07.807487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321286+0000","last_change":"2024-03-11T19:37:05.336727+0000","last_active":"2024-03-11T19:37:06.321286+0000","last_peered":"2024-03-11T19:37:06.321286+0000","last_clean":"2024-03-11T19:37:06.321286+0000","last_became_active":"2024-03-11T19:37:05.336384+0000","last_became_peered":"2024-03-11T19:37:05.336384+0000","last_unstale":"2024-03-11T19:37:06.321286+0000","last_undegraded":"2024-03-11T19:37:06.321286+0000","last_fullsized":"2024-03-11T19:37:06.321286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:02.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:09.620967+0000","last_change":"2024-03-11T19:37:04.586814+0000","last_active":"2024-03-11T19:37:09.620967+0000","last_peered":"2024-03-11T19:37:09.620967+0000","last_clean":"2024-03-11T19:37:09.620967+0000","last_became_active":"2024-03-11T19:37:04.586512+0000","last_became_peered":"2024-03-11T19:37:04.586512+0000","last_unstale":"2024-03-11T19:37:09.620967+0000","last_undegraded":"2024-03-11T19:37:09.620967+0000","last_fullsized":"2024-03-11T19:37:09.620967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:04:23.979783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707821+0000","last_change":"2024-03-11T19:37:05.335353+0000","last_active":"2024-03-11T19:37:06.707821+0000","last_peered":"2024-03-11T19:37:06.707821+0000","last_clean":"2024-03-11T19:37:06.707821+0000","last_became_active":"2024-03-11T19:37:05.335066+0000","last_became_peered":"2024-03-11T19:37:05.335066+0000","last_unstale":"2024-03-11T19:37:06.707821+0000","last_undegraded":"2024-03-11T19:37:06.707821+0000","last_fullsized":"2024-03-11T19:37:06.707821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:11.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654949+0000","last_change":"2024-03-11T19:37:04.586679+0000","last_active":"2024-03-11T19:37:06.654949+0000","last_peered":"2024-03-11T19:37:06.654949+0000","last_clean":"2024-03-11T19:37:06.654949+0000","last_became_active":"2024-03-11T19:37:04.586316+0000","last_became_peered":"2024-03-11T19:37:04.586316+0000","last_unstale":"2024-03-11T19:37:06.654949+0000","last_undegraded":"2024-03-11T19:37:06.654949+0000","last_fullsized":"2024-03-11T19:37:06.654949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:58:21.115526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655325+0000","last_change":"2024-03-11T19:37:05.330206+0000","last_active":"2024-03-11T19:37:06.655325+0000","last_peered":"2024-03-11T19:37:06.655325+0000","last_clean":"2024-03-11T19:37:06.655325+0000","last_became_active":"2024-03-11T19:37:05.329847+0000","last_became_peered":"2024-03-11T19:37:05.329847+0000","last_unstale":"2024-03-11T19:37:06.655325+0000","last_undegraded":"2024-03-11T19:37:06.655325+0000","last_fullsized":"2024-03-11T19:37:06.655325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:14:52.532236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321520+0000","last_change":"2024-03-11T19:37:04.581031+0000","last_active":"2024-03-11T19:37:06.321520+0000","last_peered":"2024-03-11T19:37:06.321520+0000","last_clean":"2024-03-11T19:37:06.321520+0000","last_became_active":"2024-03-11T19:37:04.580728+0000","last_became_peered":"2024-03-11T19:37:04.580728+0000","last_unstale":"2024-03-11T19:37:06.321520+0000","last_undegraded":"2024-03-11T19:37:06.321520+0000","last_fullsized":"2024-03-11T19:37:06.321520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:43.401898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654768+0000","last_change":"2024-03-11T19:37:05.328686+0000","last_active":"2024-03-11T19:37:06.654768+0000","last_peered":"2024-03-11T19:37:06.654768+0000","last_clean":"2024-03-11T19:37:06.654768+0000","last_became_active":"2024-03-11T19:37:05.328372+0000","last_became_peered":"2024-03-11T19:37:05.328372+0000","last_unstale":"2024-03-11T19:37:06.654768+0000","last_undegraded":"2024-03-11T19:37:06.654768+0000","last_fullsized":"2024-03-11T19:37:06.654768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:38:33.216489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321776+0000","last_change":"2024-03-11T19:37:05.335057+0000","last_active":"2024-03-11T19:37:06.321776+0000","last_peered":"2024-03-11T19:37:06.321776+0000","last_clean":"2024-03-11T19:37:06.321776+0000","last_became_active":"2024-03-11T19:37:05.334646+0000","last_became_peered":"2024-03-11T19:37:05.334646+0000","last_unstale":"2024-03-11T19:37:06.321776+0000","last_undegraded":"2024-03-11T19:37:06.321776+0000","last_fullsized":"2024-03-11T19:37:06.321776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:34:15.932429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:10.631139+0000","last_change":"2024-03-11T19:37:04.588095+0000","last_active":"2024-03-11T19:37:10.631139+0000","last_peered":"2024-03-11T19:37:10.631139+0000","last_clean":"2024-03-11T19:37:10.631139+0000","last_became_active":"2024-03-11T19:37:04.587757+0000","last_became_peered":"2024-03-11T19:37:04.587757+0000","last_unstale":"2024-03-11T19:37:10.631139+0000","last_undegraded":"2024-03-11T19:37:10.631139+0000","last_fullsized":"2024-03-11T19:37:10.631139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:57.926858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655062+0000","last_change":"2024-03-11T19:37:05.326806+0000","last_active":"2024-03-11T19:37:06.655062+0000","last_peered":"2024-03-11T19:37:06.655062+0000","last_clean":"2024-03-11T19:37:06.655062+0000","last_became_active":"2024-03-11T19:37:05.326506+0000","last_became_peered":"2024-03-11T19:37:05.326506+0000","last_unstale":"2024-03-11T19:37:06.655062+0000","last_undegraded":"2024-03-11T19:37:06.655062+0000","last_fullsized":"2024-03-11T19:37:06.655062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:20.664857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:09.625312+0000","last_change":"2024-03-11T19:37:04.588459+0000","last_active":"2024-03-11T19:37:09.625312+0000","last_peered":"2024-03-11T19:37:09.625312+0000","last_clean":"2024-03-11T19:37:09.625312+0000","last_became_active":"2024-03-11T19:37:04.588000+0000","last_became_peered":"2024-03-11T19:37:04.588000+0000","last_unstale":"2024-03-11T19:37:09.625312+0000","last_undegraded":"2024-03-11T19:37:09.625312+0000","last_fullsized":"2024-03-11T19:37:09.625312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:20:47.717196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321330+0000","last_change":"2024-03-11T19:35:55.814178+0000","last_active":"2024-03-11T19:37:06.321330+0000","last_peered":"2024-03-11T19:37:06.321330+0000","last_clean":"2024-03-11T19:37:06.321330+0000","last_became_active":"2024-03-11T19:35:55.813697+0000","last_became_peered":"2024-03-11T19:35:55.813697+0000","last_unstale":"2024-03-11T19:37:06.321330+0000","last_undegraded":"2024-03-11T19:37:06.321330+0000","last_fullsized":"2024-03-11T19:37:06.321330+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_clean_scrub_stamp":"2024-03-11T19:32:34.872292+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:50:27.486733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.322793+0000","last_change":"2024-03-11T19:37:05.319614+0000","last_active":"2024-03-11T19:37:06.322793+0000","last_peered":"2024-03-11T19:37:06.322793+0000","last_clean":"2024-03-11T19:37:06.322793+0000","last_became_active":"2024-03-11T19:37:05.319277+0000","last_became_peered":"2024-03-11T19:37:05.319277+0000","last_unstale":"2024-03-11T19:37:06.322793+0000","last_undegraded":"2024-03-11T19:37:06.322793+0000","last_fullsized":"2024-03-11T19:37:06.322793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:27.329900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:08.624881+0000","last_change":"2024-03-11T19:37:04.582250+0000","last_active":"2024-03-11T19:37:08.624881+0000","last_peered":"2024-03-11T19:37:08.624881+0000","last_clean":"2024-03-11T19:37:08.624881+0000","last_became_active":"2024-03-11T19:37:04.581938+0000","last_became_peered":"2024-03-11T19:37:04.581938+0000","last_unstale":"2024-03-11T19:37:08.624881+0000","last_undegraded":"2024-03-11T19:37:08.624881+0000","last_fullsized":"2024-03-11T19:37:08.624881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:23.466067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.611543+0000","last_change":"2024-03-11T19:37:04.590699+0000","last_active":"2024-03-11T19:37:07.611543+0000","last_peered":"2024-03-11T19:37:07.611543+0000","last_clean":"2024-03-11T19:37:07.611543+0000","last_became_active":"2024-03-11T19:37:04.590424+0000","last_became_peered":"2024-03-11T19:37:04.590424+0000","last_unstale":"2024-03-11T19:37:07.611543+0000","last_undegraded":"2024-03-11T19:37:07.611543+0000","last_fullsized":"2024-03-11T19:37:07.611543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:49.075335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.323185+0000","last_change":"2024-03-11T19:37:05.323357+0000","last_active":"2024-03-11T19:37:06.323185+0000","last_peered":"2024-03-11T19:37:06.323185+0000","last_clean":"2024-03-11T19:37:06.323185+0000","last_became_active":"2024-03-11T19:37:05.322980+0000","last_became_peered":"2024-03-11T19:37:05.322980+0000","last_unstale":"2024-03-11T19:37:06.323185+0000","last_undegraded":"2024-03-11T19:37:06.323185+0000","last_fullsized":"2024-03-11T19:37:06.323185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:19.106375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.577045+0000","last_change":"2024-03-11T19:37:04.585232+0000","last_active":"2024-03-11T19:37:07.577045+0000","last_peered":"2024-03-11T19:37:07.577045+0000","last_clean":"2024-03-11T19:37:07.577045+0000","last_became_active":"2024-03-11T19:37:04.584904+0000","last_became_peered":"2024-03-11T19:37:04.584904+0000","last_unstale":"2024-03-11T19:37:07.577045+0000","last_undegraded":"2024-03-11T19:37:07.577045+0000","last_fullsized":"2024-03-11T19:37:07.577045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:56:46.519961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654745+0000","last_change":"2024-03-11T19:37:05.332019+0000","last_active":"2024-03-11T19:37:06.654745+0000","last_peered":"2024-03-11T19:37:06.654745+0000","last_clean":"2024-03-11T19:37:06.654745+0000","last_became_active":"2024-03-11T19:37:05.331673+0000","last_became_peered":"2024-03-11T19:37:05.331673+0000","last_unstale":"2024-03-11T19:37:06.654745+0000","last_undegraded":"2024-03-11T19:37:06.654745+0000","last_fullsized":"2024-03-11T19:37:06.654745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:49:45.787082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.610711+0000","last_change":"2024-03-11T19:37:04.583404+0000","last_active":"2024-03-11T19:37:07.610711+0000","last_peered":"2024-03-11T19:37:07.610711+0000","last_clean":"2024-03-11T19:37:07.610711+0000","last_became_active":"2024-03-11T19:37:04.583025+0000","last_became_peered":"2024-03-11T19:37:04.583025+0000","last_unstale":"2024-03-11T19:37:07.610711+0000","last_undegraded":"2024-03-11T19:37:07.610711+0000","last_fullsized":"2024-03-11T19:37:07.610711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:02:25.202591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654615+0000","last_change":"2024-03-11T19:37:05.331711+0000","last_active":"2024-03-11T19:37:06.654615+0000","last_peered":"2024-03-11T19:37:06.654615+0000","last_clean":"2024-03-11T19:37:06.654615+0000","last_became_active":"2024-03-11T19:37:05.330851+0000","last_became_peered":"2024-03-11T19:37:05.330851+0000","last_unstale":"2024-03-11T19:37:06.654615+0000","last_undegraded":"2024-03-11T19:37:06.654615+0000","last_fullsized":"2024-03-11T19:37:06.654615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:18:26.692068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.612561+0000","last_change":"2024-03-11T19:37:04.586501+0000","last_active":"2024-03-11T19:37:07.612561+0000","last_peered":"2024-03-11T19:37:07.612561+0000","last_clean":"2024-03-11T19:37:07.612561+0000","last_became_active":"2024-03-11T19:37:04.586200+0000","last_became_peered":"2024-03-11T19:37:04.586200+0000","last_unstale":"2024-03-11T19:37:07.612561+0000","last_undegraded":"2024-03-11T19:37:07.612561+0000","last_fullsized":"2024-03-11T19:37:07.612561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:56.635375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654006+0000","last_change":"2024-03-11T19:37:05.332402+0000","last_active":"2024-03-11T19:37:06.654006+0000","last_peered":"2024-03-11T19:37:06.654006+0000","last_clean":"2024-03-11T19:37:06.654006+0000","last_became_active":"2024-03-11T19:37:05.332087+0000","last_became_peered":"2024-03-11T19:37:05.332087+0000","last_unstale":"2024-03-11T19:37:06.654006+0000","last_undegraded":"2024-03-11T19:37:06.654006+0000","last_fullsized":"2024-03-11T19:37:06.654006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:10:23.882296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654418+0000","last_change":"2024-03-11T19:37:04.584978+0000","last_active":"2024-03-11T19:37:06.654418+0000","last_peered":"2024-03-11T19:37:06.654418+0000","last_clean":"2024-03-11T19:37:06.654418+0000","last_became_active":"2024-03-11T19:37:04.584556+0000","last_became_peered":"2024-03-11T19:37:04.584556+0000","last_unstale":"2024-03-11T19:37:06.654418+0000","last_undegraded":"2024-03-11T19:37:06.654418+0000","last_fullsized":"2024-03-11T19:37:06.654418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:39:48.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707645+0000","last_change":"2024-03-11T19:37:05.334453+0000","last_active":"2024-03-11T19:37:06.707645+0000","last_peered":"2024-03-11T19:37:06.707645+0000","last_clean":"2024-03-11T19:37:06.707645+0000","last_became_active":"2024-03-11T19:37:05.334009+0000","last_became_peered":"2024-03-11T19:37:05.334009+0000","last_unstale":"2024-03-11T19:37:06.707645+0000","last_undegraded":"2024-03-11T19:37:06.707645+0000","last_fullsized":"2024-03-11T19:37:06.707645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:43.402674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654392+0000","last_change":"2024-03-11T19:37:04.583181+0000","last_active":"2024-03-11T19:37:06.654392+0000","last_peered":"2024-03-11T19:37:06.654392+0000","last_clean":"2024-03-11T19:37:06.654392+0000","last_became_active":"2024-03-11T19:37:04.582881+0000","last_became_peered":"2024-03-11T19:37:04.582881+0000","last_unstale":"2024-03-11T19:37:06.654392+0000","last_undegraded":"2024-03-11T19:37:06.654392+0000","last_fullsized":"2024-03-11T19:37:06.654392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:05:06.634150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654891+0000","last_change":"2024-03-11T19:37:05.333642+0000","last_active":"2024-03-11T19:37:06.654891+0000","last_peered":"2024-03-11T19:37:06.654891+0000","last_clean":"2024-03-11T19:37:06.654891+0000","last_became_active":"2024-03-11T19:37:05.333121+0000","last_became_peered":"2024-03-11T19:37:05.333121+0000","last_unstale":"2024-03-11T19:37:06.654891+0000","last_undegraded":"2024-03-11T19:37:06.654891+0000","last_fullsized":"2024-03-11T19:37:06.654891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:07:21.217101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:07.612433+0000","last_change":"2024-03-11T19:37:04.586235+0000","last_active":"2024-03-11T19:37:07.612433+0000","last_peered":"2024-03-11T19:37:07.612433+0000","last_clean":"2024-03-11T19:37:07.612433+0000","last_became_active":"2024-03-11T19:37:04.585928+0000","last_became_peered":"2024-03-11T19:37:04.585928+0000","last_unstale":"2024-03-11T19:37:07.612433+0000","last_undegraded":"2024-03-11T19:37:07.612433+0000","last_fullsized":"2024-03-11T19:37:07.612433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:06:22.840169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321770+0000","last_change":"2024-03-11T19:37:05.326992+0000","last_active":"2024-03-11T19:37:06.321770+0000","last_peered":"2024-03-11T19:37:06.321770+0000","last_clean":"2024-03-11T19:37:06.321770+0000","last_became_active":"2024-03-11T19:37:05.326676+0000","last_became_peered":"2024-03-11T19:37:05.326676+0000","last_unstale":"2024-03-11T19:37:06.321770+0000","last_undegraded":"2024-03-11T19:37:06.321770+0000","last_fullsized":"2024-03-11T19:37:06.321770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:31:10.310650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654908+0000","last_change":"2024-03-11T19:37:04.583631+0000","last_active":"2024-03-11T19:37:06.654908+0000","last_peered":"2024-03-11T19:37:06.654908+0000","last_clean":"2024-03-11T19:37:06.654908+0000","last_became_active":"2024-03-11T19:37:04.583031+0000","last_became_peered":"2024-03-11T19:37:04.583031+0000","last_unstale":"2024-03-11T19:37:06.654908+0000","last_undegraded":"2024-03-11T19:37:06.654908+0000","last_fullsized":"2024-03-11T19:37:06.654908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:20.695496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321922+0000","last_change":"2024-03-11T19:37:05.331008+0000","last_active":"2024-03-11T19:37:06.321922+0000","last_peered":"2024-03-11T19:37:06.321922+0000","last_clean":"2024-03-11T19:37:06.321922+0000","last_became_active":"2024-03-11T19:37:05.330647+0000","last_became_peered":"2024-03-11T19:37:05.330647+0000","last_unstale":"2024-03-11T19:37:06.321922+0000","last_undegraded":"2024-03-11T19:37:06.321922+0000","last_fullsized":"2024-03-11T19:37:06.321922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:07:00.576137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321739+0000","last_change":"2024-03-11T19:37:04.582501+0000","last_active":"2024-03-11T19:37:06.321739+0000","last_peered":"2024-03-11T19:37:06.321739+0000","last_clean":"2024-03-11T19:37:06.321739+0000","last_became_active":"2024-03-11T19:37:04.582076+0000","last_became_peered":"2024-03-11T19:37:04.582076+0000","last_unstale":"2024-03-11T19:37:06.321739+0000","last_undegraded":"2024-03-11T19:37:06.321739+0000","last_fullsized":"2024-03-11T19:37:06.321739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:46:05.359211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321696+0000","last_change":"2024-03-11T19:37:05.329053+0000","last_active":"2024-03-11T19:37:06.321696+0000","last_peered":"2024-03-11T19:37:06.321696+0000","last_clean":"2024-03-11T19:37:06.321696+0000","last_became_active":"2024-03-11T19:37:05.328748+0000","last_became_peered":"2024-03-11T19:37:05.328748+0000","last_unstale":"2024-03-11T19:37:06.321696+0000","last_undegraded":"2024-03-11T19:37:06.321696+0000","last_fullsized":"2024-03-11T19:37:06.321696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:48:55.369515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.323188+0000","last_change":"2024-03-11T19:37:04.583370+0000","last_active":"2024-03-11T19:37:06.323188+0000","last_peered":"2024-03-11T19:37:06.323188+0000","last_clean":"2024-03-11T19:37:06.323188+0000","last_became_active":"2024-03-11T19:37:04.583055+0000","last_became_peered":"2024-03-11T19:37:04.583055+0000","last_unstale":"2024-03-11T19:37:06.323188+0000","last_undegraded":"2024-03-11T19:37:06.323188+0000","last_fullsized":"2024-03-11T19:37:06.323188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:04:02.325054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321856+0000","last_change":"2024-03-11T19:37:05.328157+0000","last_active":"2024-03-11T19:37:06.321856+0000","last_peered":"2024-03-11T19:37:06.321856+0000","last_clean":"2024-03-11T19:37:06.321856+0000","last_became_active":"2024-03-11T19:37:05.327839+0000","last_became_peered":"2024-03-11T19:37:05.327839+0000","last_unstale":"2024-03-11T19:37:06.321856+0000","last_undegraded":"2024-03-11T19:37:06.321856+0000","last_fullsized":"2024-03-11T19:37:06.321856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:23:02.297269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707661+0000","last_change":"2024-03-11T19:37:05.336160+0000","last_active":"2024-03-11T19:37:06.707661+0000","last_peered":"2024-03-11T19:37:06.707661+0000","last_clean":"2024-03-11T19:37:06.707661+0000","last_became_active":"2024-03-11T19:37:05.335850+0000","last_became_peered":"2024-03-11T19:37:05.335850+0000","last_unstale":"2024-03-11T19:37:06.707661+0000","last_undegraded":"2024-03-11T19:37:06.707661+0000","last_fullsized":"2024-03-11T19:37:06.707661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:09.743661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654699+0000","last_change":"2024-03-11T19:37:05.325332+0000","last_active":"2024-03-11T19:37:06.654699+0000","last_peered":"2024-03-11T19:37:06.654699+0000","last_clean":"2024-03-11T19:37:06.654699+0000","last_became_active":"2024-03-11T19:37:05.324948+0000","last_became_peered":"2024-03-11T19:37:05.324948+0000","last_unstale":"2024-03-11T19:37:06.654699+0000","last_undegraded":"2024-03-11T19:37:06.654699+0000","last_fullsized":"2024-03-11T19:37:06.654699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:31:04.454259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654307+0000","last_change":"2024-03-11T19:37:05.332357+0000","last_active":"2024-03-11T19:37:06.654307+0000","last_peered":"2024-03-11T19:37:06.654307+0000","last_clean":"2024-03-11T19:37:06.654307+0000","last_became_active":"2024-03-11T19:37:05.332003+0000","last_became_peered":"2024-03-11T19:37:05.332003+0000","last_unstale":"2024-03-11T19:37:06.654307+0000","last_undegraded":"2024-03-11T19:37:06.654307+0000","last_fullsized":"2024-03-11T19:37:06.654307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:39.165572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654299+0000","last_change":"2024-03-11T19:37:05.332388+0000","last_active":"2024-03-11T19:37:06.654299+0000","last_peered":"2024-03-11T19:37:06.654299+0000","last_clean":"2024-03-11T19:37:06.654299+0000","last_became_active":"2024-03-11T19:37:05.332032+0000","last_became_peered":"2024-03-11T19:37:05.332032+0000","last_unstale":"2024-03-11T19:37:06.654299+0000","last_undegraded":"2024-03-11T19:37:06.654299+0000","last_fullsized":"2024-03-11T19:37:06.654299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:56:09.416683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.323249+0000","last_change":"2024-03-11T19:37:05.323096+0000","last_active":"2024-03-11T19:37:06.323249+0000","last_peered":"2024-03-11T19:37:06.323249+0000","last_clean":"2024-03-11T19:37:06.323249+0000","last_became_active":"2024-03-11T19:37:05.322525+0000","last_became_peered":"2024-03-11T19:37:05.322525+0000","last_unstale":"2024-03-11T19:37:06.323249+0000","last_undegraded":"2024-03-11T19:37:06.323249+0000","last_fullsized":"2024-03-11T19:37:06.323249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:53:01.870894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707739+0000","last_change":"2024-03-11T19:37:05.331171+0000","last_active":"2024-03-11T19:37:06.707739+0000","last_peered":"2024-03-11T19:37:06.707739+0000","last_clean":"2024-03-11T19:37:06.707739+0000","last_became_active":"2024-03-11T19:37:05.330769+0000","last_became_peered":"2024-03-11T19:37:05.330769+0000","last_unstale":"2024-03-11T19:37:06.707739+0000","last_undegraded":"2024-03-11T19:37:06.707739+0000","last_fullsized":"2024-03-11T19:37:06.707739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:03:06.514039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655145+0000","last_change":"2024-03-11T19:37:05.326877+0000","last_active":"2024-03-11T19:37:06.655145+0000","last_peered":"2024-03-11T19:37:06.655145+0000","last_clean":"2024-03-11T19:37:06.655145+0000","last_became_active":"2024-03-11T19:37:05.326538+0000","last_became_peered":"2024-03-11T19:37:05.326538+0000","last_unstale":"2024-03-11T19:37:06.655145+0000","last_undegraded":"2024-03-11T19:37:06.655145+0000","last_fullsized":"2024-03-11T19:37:06.655145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:11.565697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.655182+0000","last_change":"2024-03-11T19:37:05.335010+0000","last_active":"2024-03-11T19:37:06.655182+0000","last_peered":"2024-03-11T19:37:06.655182+0000","last_clean":"2024-03-11T19:37:06.655182+0000","last_became_active":"2024-03-11T19:37:05.334551+0000","last_became_peered":"2024-03-11T19:37:05.334551+0000","last_unstale":"2024-03-11T19:37:06.655182+0000","last_undegraded":"2024-03-11T19:37:06.655182+0000","last_fullsized":"2024-03-11T19:37:06.655182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:59.668178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.321407+0000","last_change":"2024-03-11T19:37:05.336273+0000","last_active":"2024-03-11T19:37:06.321407+0000","last_peered":"2024-03-11T19:37:06.321407+0000","last_clean":"2024-03-11T19:37:06.321407+0000","last_became_active":"2024-03-11T19:37:05.335926+0000","last_became_peered":"2024-03-11T19:37:05.335926+0000","last_unstale":"2024-03-11T19:37:06.321407+0000","last_undegraded":"2024-03-11T19:37:06.321407+0000","last_fullsized":"2024-03-11T19:37:06.321407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:31:55.139242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654555+0000","last_change":"2024-03-11T19:37:05.335548+0000","last_active":"2024-03-11T19:37:06.654555+0000","last_peered":"2024-03-11T19:37:06.654555+0000","last_clean":"2024-03-11T19:37:06.654555+0000","last_became_active":"2024-03-11T19:37:05.335222+0000","last_became_peered":"2024-03-11T19:37:05.335222+0000","last_unstale":"2024-03-11T19:37:06.654555+0000","last_undegraded":"2024-03-11T19:37:06.654555+0000","last_fullsized":"2024-03-11T19:37:06.654555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:25.697951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.654780+0000","last_change":"2024-03-11T19:37:05.329782+0000","last_active":"2024-03-11T19:37:06.654780+0000","last_peered":"2024-03-11T19:37:06.654780+0000","last_clean":"2024-03-11T19:37:06.654780+0000","last_became_active":"2024-03-11T19:37:05.329434+0000","last_became_peered":"2024-03-11T19:37:05.329434+0000","last_unstale":"2024-03-11T19:37:06.654780+0000","last_undegraded":"2024-03-11T19:37:06.654780+0000","last_fullsized":"2024-03-11T19:37:06.654780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:12:00.516489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.323163+0000","last_change":"2024-03-11T19:37:05.323026+0000","last_active":"2024-03-11T19:37:06.323163+0000","last_peered":"2024-03-11T19:37:06.323163+0000","last_clean":"2024-03-11T19:37:06.323163+0000","last_became_active":"2024-03-11T19:37:05.322562+0000","last_became_peered":"2024-03-11T19:37:05.322562+0000","last_unstale":"2024-03-11T19:37:06.323163+0000","last_undegraded":"2024-03-11T19:37:06.323163+0000","last_fullsized":"2024-03-11T19:37:06.323163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:14:57.606404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-11T19:37:06.707829+0000","last_change":"2024-03-11T19:37:05.333342+0000","last_active":"2024-03-11T19:37:06.707829+0000","last_peered":"2024-03-11T19:37:06.707829+0000","last_clean":"2024-03-11T19:37:06.707829+0000","last_became_active":"2024-03-11T19:37:05.332972+0000","last_became_peered":"2024-03-11T19:37:05.332972+0000","last_unstale":"2024-03-11T19:37:06.707829+0000","last_undegraded":"2024-03-11T19:37:06.707829+0000","last_fullsized":"2024-03-11T19:37:06.707829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:57:27.372761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5332,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":103,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":15118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":438135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32676,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712476,"statfs":{"total":95995035648,"available":95961575424,"internally_reserved":0,"allocated":1531904,"data_stored":1030309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":438919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070492,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":438873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332128,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1523712,"data_stored":1031077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":438342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":439747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":439613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":440403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":438239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1536000,"data_stored":1030370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:37:12.644 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-11T19:37:12.644 INFO:tasks.cephadm:Setting up client nodes... 2024-03-11T19:37:12.645 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-11T19:37:12.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:12 smithi204 ceph-mon[36739]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-11T19:37:12.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:12 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1211163663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:37:12.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:12 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2143208566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:12.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:12 smithi204 ceph-mon[36739]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:37:12.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:12 smithi204 ceph-mon[36739]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-03-11T19:37:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:13.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:12 smithi083 ceph-mon[31864]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-11T19:37:13.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:12 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1211163663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:37:13.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:12 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2143208566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:13.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:12 smithi083 ceph-mon[31864]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:37:13.039 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:12 smithi083 ceph-mon[31864]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-03-11T19:37:13.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:12 smithi137 ceph-mon[36220]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-11T19:37:13.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:12 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1211163663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:37:13.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:12 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2143208566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:13.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:12 smithi137 ceph-mon[36220]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:37:13.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:12 smithi137 ceph-mon[36220]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-03-11T19:37:13.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:13 smithi083 ceph-mon[31864]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:37:13.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:13 smithi204 ceph-mon[36739]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:37:14.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:13 smithi137 ceph-mon[36220]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:37:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.494+0000 7f8faa50f700 1 -- 172.21.15.83:0/2062351745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa4103860 msgr2=0x7f8fa410a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.494+0000 7f8faa50f700 1 --2- 172.21.15.83:0/2062351745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa4103860 0x7f8fa410a110 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00b0d0 tx=0x7f8f9c00b3e0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.494+0000 7f8faa50f700 1 -- 172.21.15.83:0/2062351745 shutdown_connections 2024-03-11T19:37:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.494+0000 7f8faa50f700 1 --2- 172.21.15.83:0/2062351745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa4103860 0x7f8fa410a110 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.494+0000 7f8faa50f700 1 --2- 172.21.15.83:0/2062351745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa4101cc0 0x7f8fa41020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.494+0000 7f8faa50f700 1 --2- 172.21.15.83:0/2062351745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa4102ec0 0x7f8fa4103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.494+0000 7f8faa50f700 1 -- 172.21.15.83:0/2062351745 >> 172.21.15.83:0/2062351745 conn(0x7f8fa40fd280 msgr2=0x7f8fa40ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.494+0000 7f8faa50f700 1 -- 172.21.15.83:0/2062351745 shutdown_connections 2024-03-11T19:37:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.494+0000 7f8faa50f700 1 -- 172.21.15.83:0/2062351745 wait complete. 2024-03-11T19:37:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.495+0000 7f8faa50f700 1 Processor -- start 2024-03-11T19:37:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.495+0000 7f8faa50f700 1 -- start start 2024-03-11T19:37:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.495+0000 7f8faa50f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa4101cc0 0x7f8fa4197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.495+0000 7f8faa50f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa4102ec0 0x7f8fa41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.495+0000 7f8faa50f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa4103860 0x7f8fa419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.495+0000 7f8faa50f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa419f830 con 0x7f8fa4101cc0 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.495+0000 7f8faa50f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa419f970 con 0x7f8fa4103860 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.495+0000 7f8faa50f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa419fb30 con 0x7f8fa4102ec0 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.495+0000 7f8fa3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa4101cc0 0x7f8fa4197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.495+0000 7f8fa3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa4101cc0 0x7f8fa4197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60688/0 (socket says 172.21.15.83:60688) 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.495+0000 7f8fa3fff700 1 -- 172.21.15.83:0/3110749173 learned_addr learned my addr 172.21.15.83:0/3110749173 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa8aac700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa4103860 0x7f8fa419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa3fff700 1 -- 172.21.15.83:0/3110749173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa4102ec0 msgr2=0x7f8fa41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa37fe700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa4102ec0 0x7f8fa41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa3fff700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa4102ec0 0x7f8fa41981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa3fff700 1 -- 172.21.15.83:0/3110749173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa4103860 msgr2=0x7f8fa419f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa3fff700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa4103860 0x7f8fa419f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa3fff700 1 -- 172.21.15.83:0/3110749173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c009d00 con 0x7f8fa4101cc0 2024-03-11T19:37:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa8aac700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa4103860 0x7f8fa419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:37:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa3fff700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa4101cc0 0x7f8fa4197cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400b500 tx=0x7f8f9400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa17fa700 1 -- 172.21.15.83:0/3110749173 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94011830 con 0x7f8fa4101cc0 2024-03-11T19:37:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa17fa700 1 -- 172.21.15.83:0/3110749173 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f94011e70 con 0x7f8fa4101cc0 2024-03-11T19:37:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8faa50f700 1 -- 172.21.15.83:0/3110749173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa419fe10 con 0x7f8fa4101cc0 2024-03-11T19:37:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8fa17fa700 1 -- 172.21.15.83:0/3110749173 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400f2e0 con 0x7f8fa4101cc0 2024-03-11T19:37:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.496+0000 7f8faa50f700 1 -- 172.21.15.83:0/3110749173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa41a02a0 con 0x7f8fa4101cc0 2024-03-11T19:37:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.497+0000 7f8faa50f700 1 -- 172.21.15.83:0/3110749173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa4108fb0 con 0x7f8fa4101cc0 2024-03-11T19:37:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.498+0000 7f8fa17fa700 1 -- 172.21.15.83:0/3110749173 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f8f940105f0 con 0x7f8fa4101cc0 2024-03-11T19:37:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.499+0000 7f8fa17fa700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8f8c0a1c80 0x7f8f8c0a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.499+0000 7f8fa37fe700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8f8c0a1c80 0x7f8f8c0a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.499+0000 7f8fa17fa700 1 -- 172.21.15.83:0/3110749173 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8f940ccec0 con 0x7f8fa4101cc0 2024-03-11T19:37:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.499+0000 7f8fa37fe700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8f8c0a1c80 0x7f8f8c0a4140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f98005950 tx=0x7f8f980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:14.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.503+0000 7f8fa17fa700 1 -- 172.21.15.83:0/3110749173 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f94096130 con 0x7f8fa4101cc0 2024-03-11T19:37:14.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.781+0000 7f8faa50f700 1 -- 172.21.15.83:0/3110749173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8fa410d6a0 con 0x7f8fa4101cc0 2024-03-11T19:37:14.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.785+0000 7f8fa17fa700 1 -- 172.21.15.83:0/3110749173 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8f94010380 con 0x7f8fa4101cc0 2024-03-11T19:37:14.790 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-03-11T19:37:14.790 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBqXe9l/ou1LhAAwFsR80ScduS40NfZDFkgNg== 2024-03-11T19:37:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.788+0000 7f8faa50f700 1 -- 172.21.15.83:0/3110749173 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8f8c0a1c80 msgr2=0x7f8f8c0a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.788+0000 7f8faa50f700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8f8c0a1c80 0x7f8f8c0a4140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f98005950 tx=0x7f8f980058e0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.788+0000 7f8faa50f700 1 -- 172.21.15.83:0/3110749173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa4101cc0 msgr2=0x7f8fa4197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.788+0000 7f8faa50f700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa4101cc0 0x7f8fa4197cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400b500 tx=0x7f8f9400b8c0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.788+0000 7f8faa50f700 1 -- 172.21.15.83:0/3110749173 shutdown_connections 2024-03-11T19:37:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.788+0000 7f8faa50f700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8f8c0a1c80 0x7f8f8c0a4140 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.788+0000 7f8faa50f700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fa4103860 0x7f8fa419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.788+0000 7f8faa50f700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fa4101cc0 0x7f8fa4197cb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.788+0000 7f8faa50f700 1 --2- 172.21.15.83:0/3110749173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fa4102ec0 0x7f8fa41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.788+0000 7f8faa50f700 1 -- 172.21.15.83:0/3110749173 >> 172.21.15.83:0/3110749173 conn(0x7f8fa40fd280 msgr2=0x7f8fa41060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:14.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.788+0000 7f8faa50f700 1 -- 172.21.15.83:0/3110749173 shutdown_connections 2024-03-11T19:37:14.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:14.789+0000 7f8faa50f700 1 -- 172.21.15.83:0/3110749173 wait complete. 2024-03-11T19:37:15.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:14 smithi083 ceph-mon[31864]: pgmap v288: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 20 op/s 2024-03-11T19:37:15.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:14 smithi137 ceph-mon[36220]: pgmap v288: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 20 op/s 2024-03-11T19:37:15.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:14 smithi204 ceph-mon[36739]: pgmap v288: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 20 op/s 2024-03-11T19:37:15.402 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:37:15.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-11T19:37:15.402 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-11T19:37:15.444 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-11T19:37:15.445 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-11T19:37:15.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph mgr dump --format=json 2024-03-11T19:37:15.724 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:15.960 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:15 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3110749173' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:37:15.960 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:15 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3110749173' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-11T19:37:16.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:15 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3110749173' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:37:16.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:15 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3110749173' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-11T19:37:16.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:15 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3110749173' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-11T19:37:16.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:15 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3110749173' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-11T19:37:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:16 smithi083 ceph-mon[31864]: pgmap v289: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-03-11T19:37:17.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:16 smithi137 ceph-mon[36220]: pgmap v289: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-03-11T19:37:17.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:16 smithi204 ceph-mon[36739]: pgmap v289: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-03-11T19:37:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.398+0000 7f099bb83700 1 -- 172.21.15.83:0/1448167971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0994102680 msgr2=0x7f099410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.398+0000 7f099bb83700 1 --2- 172.21.15.83:0/1448167971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0994102680 0x7f099410eac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f098c00b0d0 tx=0x7f098c00b3e0 comp rx=0 tx=0).stop 2024-03-11T19:37:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.398+0000 7f099bb83700 1 -- 172.21.15.83:0/1448167971 shutdown_connections 2024-03-11T19:37:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.398+0000 7f099bb83700 1 --2- 172.21.15.83:0/1448167971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0994107ca0 0x7f0994108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.398+0000 7f099bb83700 1 --2- 172.21.15.83:0/1448167971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0994102680 0x7f099410eac0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.398+0000 7f099bb83700 1 --2- 172.21.15.83:0/1448167971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0994101cc0 0x7f0994102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.398+0000 7f099bb83700 1 -- 172.21.15.83:0/1448167971 >> 172.21.15.83:0/1448167971 conn(0x7f0994078da0 msgr2=0x7f09940791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.399+0000 7f099bb83700 1 -- 172.21.15.83:0/1448167971 shutdown_connections 2024-03-11T19:37:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.399+0000 7f099bb83700 1 -- 172.21.15.83:0/1448167971 wait complete. 2024-03-11T19:37:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.399+0000 7f099bb83700 1 Processor -- start 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.399+0000 7f099bb83700 1 -- start start 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.399+0000 7f099bb83700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0994101cc0 0x7f0994073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.399+0000 7f099bb83700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0994102680 0x7f09940744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.399+0000 7f099bb83700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0994107ca0 0x7f09940749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.399+0000 7f099bb83700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941a6600 con 0x7f0994102680 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.399+0000 7f099bb83700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941a6770 con 0x7f0994101cc0 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.399+0000 7f099bb83700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941a6930 con 0x7f0994107ca0 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.399+0000 7f099911e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0994102680 0x7f09940744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099911e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0994102680 0x7f09940744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60706/0 (socket says 172.21.15.83:60706) 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099911e700 1 -- 172.21.15.83:0/17875643 learned_addr learned my addr 172.21.15.83:0/17875643 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099991f700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0994101cc0 0x7f0994073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099a120700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0994107ca0 0x7f09940749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099911e700 1 -- 172.21.15.83:0/17875643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0994107ca0 msgr2=0x7f09940749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099911e700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0994107ca0 0x7f09940749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099911e700 1 -- 172.21.15.83:0/17875643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0994101cc0 msgr2=0x7f0994073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099911e700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0994101cc0 0x7f0994073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099911e700 1 -- 172.21.15.83:0/17875643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c009d00 con 0x7f0994102680 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099a120700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0994107ca0 0x7f09940749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099911e700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0994102680 0x7f09940744b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f098400d950 tx=0x7f098400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f098affd700 1 -- 172.21.15.83:0/17875643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984009940 con 0x7f0994102680 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099bb83700 1 -- 172.21.15.83:0/17875643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09941a6c10 con 0x7f0994102680 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f098affd700 1 -- 172.21.15.83:0/17875643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0984010460 con 0x7f0994102680 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f098affd700 1 -- 172.21.15.83:0/17875643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098400f620 con 0x7f0994102680 2024-03-11T19:37:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.400+0000 7f099bb83700 1 -- 172.21.15.83:0/17875643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09941a7110 con 0x7f0994102680 2024-03-11T19:37:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.401+0000 7f099bb83700 1 -- 172.21.15.83:0/17875643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099410a180 con 0x7f0994102680 2024-03-11T19:37:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.402+0000 7f098affd700 1 -- 172.21.15.83:0/17875643 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f09840105d0 con 0x7f0994102680 2024-03-11T19:37:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.403+0000 7f098affd700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09800a1b50 0x7f09800a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.403+0000 7f099991f700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09800a1b50 0x7f09800a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.403+0000 7f098affd700 1 -- 172.21.15.83:0/17875643 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f09840cd2b0 con 0x7f0994102680 2024-03-11T19:37:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.403+0000 7f099991f700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09800a1b50 0x7f09800a4010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0990005950 tx=0x7f099000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.407+0000 7f098affd700 1 -- 172.21.15.83:0/17875643 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f098409a040 con 0x7f0994102680 2024-03-11T19:37:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.657+0000 7f099bb83700 1 -- 172.21.15.83:0/17875643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0994111cb0 con 0x7f0994102680 2024-03-11T19:37:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.661+0000 7f098affd700 1 -- 172.21.15.83:0/17875643 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260660 (secure 0 0 0) 0x7f0984099bd0 con 0x7f0994102680 2024-03-11T19:37:17.670 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:17.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.665+0000 7f099bb83700 1 -- 172.21.15.83:0/17875643 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09800a1b50 msgr2=0x7f09800a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:17.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.665+0000 7f099bb83700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09800a1b50 0x7f09800a4010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0990005950 tx=0x7f099000a300 comp rx=0 tx=0).stop 2024-03-11T19:37:17.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.665+0000 7f099bb83700 1 -- 172.21.15.83:0/17875643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0994102680 msgr2=0x7f09940744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:17.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.665+0000 7f099bb83700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0994102680 0x7f09940744b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f098400d950 tx=0x7f098400dc60 comp rx=0 tx=0).stop 2024-03-11T19:37:17.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.665+0000 7f099bb83700 1 -- 172.21.15.83:0/17875643 shutdown_connections 2024-03-11T19:37:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.665+0000 7f099bb83700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09800a1b50 0x7f09800a4010 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.665+0000 7f099bb83700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0994107ca0 0x7f09940749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.665+0000 7f099bb83700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0994101cc0 0x7f0994073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.665+0000 7f099bb83700 1 --2- 172.21.15.83:0/17875643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0994102680 0x7f09940744b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.665+0000 7f099bb83700 1 -- 172.21.15.83:0/17875643 >> 172.21.15.83:0/17875643 conn(0x7f0994078da0 msgr2=0x7f0994100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.666+0000 7f099bb83700 1 -- 172.21.15.83:0/17875643 shutdown_connections 2024-03-11T19:37:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:17.666+0000 7f099bb83700 1 -- 172.21.15.83:0/17875643 wait complete. 2024-03-11T19:37:18.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:17 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/17875643' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-11T19:37:18.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:17 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/17875643' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-11T19:37:18.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:17 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/17875643' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-11T19:37:18.393 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":3477661212},{"type":"v1","addr":"172.21.15.83:6801","nonce":3477661212}]},"active_addr":"172.21.15.83:6801/3477661212","active_change":"2024-03-11T19:30:08.427972+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24122,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.83:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1955145343}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":42719287}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":487994704}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3164935375}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3417168866}]}]} 2024-03-11T19:37:18.399 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-11T19:37:18.399 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-11T19:37:18.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-11T19:37:18.641 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:19.029 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:18 smithi083 ceph-mon[31864]: pgmap v290: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-03-11T19:37:19.041 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:18 smithi137 ceph-mon[36220]: pgmap v290: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-03-11T19:37:19.057 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:18 smithi204 ceph-mon[36739]: pgmap v290: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.0 KiB/s wr, 19 op/s 2024-03-11T19:37:20.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 -- 172.21.15.83:0/1968375040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b5c106f90 msgr2=0x7f1b5c10eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 --2- 172.21.15.83:0/1968375040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b5c106f90 0x7f1b5c10eb60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400b0a0 tx=0x7f1b5400b3b0 comp rx=0 tx=0).stop 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 -- 172.21.15.83:0/1968375040 shutdown_connections 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 --2- 172.21.15.83:0/1968375040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b5c101eb0 0x7f1b5c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 --2- 172.21.15.83:0/1968375040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b5c106f90 0x7f1b5c10eb60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 --2- 172.21.15.83:0/1968375040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b5c102860 0x7f1b5c1068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 -- 172.21.15.83:0/1968375040 >> 172.21.15.83:0/1968375040 conn(0x7f1b5c0fd720 msgr2=0x7f1b5c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 -- 172.21.15.83:0/1968375040 shutdown_connections 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 -- 172.21.15.83:0/1968375040 wait complete. 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 Processor -- start 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 -- start start 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b5c101eb0 0x7f1b5c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:20.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b5c102860 0x7f1b5c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b5c106f90 0x7f1b5c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c19d1f0 con 0x7f1b5c106f90 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c19d360 con 0x7f1b5c101eb0 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.296+0000 7f1b63f02700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c19d520 con 0x7f1b5c102860 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b6249f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b5c106f90 0x7f1b5c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b6249f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b5c106f90 0x7f1b5c19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33948/0 (socket says 172.21.15.83:33948) 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b6249f700 1 -- 172.21.15.83:0/1168781068 learned_addr learned my addr 172.21.15.83:0/1168781068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b6149d700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b5c102860 0x7f1b5c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b61c9e700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b5c101eb0 0x7f1b5c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b6249f700 1 -- 172.21.15.83:0/1168781068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b5c102860 msgr2=0x7f1b5c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b6249f700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b5c102860 0x7f1b5c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b6249f700 1 -- 172.21.15.83:0/1168781068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b5c101eb0 msgr2=0x7f1b5c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:20.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b6249f700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b5c101eb0 0x7f1b5c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:20.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b6249f700 1 -- 172.21.15.83:0/1168781068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b54009d00 con 0x7f1b5c106f90 2024-03-11T19:37:20.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b6249f700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b5c106f90 0x7f1b5c19cbe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400b070 tx=0x7f1b54004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:20.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b52ffd700 1 -- 172.21.15.83:0/1168781068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5400e040 con 0x7f1b5c106f90 2024-03-11T19:37:20.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b52ffd700 1 -- 172.21.15.83:0/1168781068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b54007850 con 0x7f1b5c106f90 2024-03-11T19:37:20.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b63f02700 1 -- 172.21.15.83:0/1168781068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c19d7a0 con 0x7f1b5c106f90 2024-03-11T19:37:20.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b52ffd700 1 -- 172.21.15.83:0/1168781068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5401b850 con 0x7f1b5c106f90 2024-03-11T19:37:20.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.297+0000 7f1b63f02700 1 -- 172.21.15.83:0/1168781068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c1a5060 con 0x7f1b5c106f90 2024-03-11T19:37:20.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.298+0000 7f1b63f02700 1 -- 172.21.15.83:0/1168781068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c10a2b0 con 0x7f1b5c106f90 2024-03-11T19:37:20.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.299+0000 7f1b52ffd700 1 -- 172.21.15.83:0/1168781068 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f1b5401da80 con 0x7f1b5c106f90 2024-03-11T19:37:20.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.300+0000 7f1b52ffd700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1b480a1bd0 0x7f1b480a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:20.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.300+0000 7f1b61c9e700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1b480a1bd0 0x7f1b480a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:20.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.300+0000 7f1b52ffd700 1 -- 172.21.15.83:0/1168781068 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1b540ceec0 con 0x7f1b5c106f90 2024-03-11T19:37:20.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.300+0000 7f1b61c9e700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1b480a1bd0 0x7f1b480a4090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1b58005950 tx=0x7f1b5800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:20.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.304+0000 7f1b52ffd700 1 -- 172.21.15.83:0/1168781068 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b540981b0 con 0x7f1b5c106f90 2024-03-11T19:37:20.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.499+0000 7f1b63f02700 1 -- 172.21.15.83:0/1168781068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b5c199670 con 0x7f1b5c106f90 2024-03-11T19:37:20.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.500+0000 7f1b52ffd700 1 -- 172.21.15.83:0/1168781068 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22232 (secure 0 0 0) 0x7f1b5409b870 con 0x7f1b5c106f90 2024-03-11T19:37:20.504 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:20.505 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":85,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:37:06.317590+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:37:20.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 -- 172.21.15.83:0/1168781068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1b480a1bd0 msgr2=0x7f1b480a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:20.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1b480a1bd0 0x7f1b480a4090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1b58005950 tx=0x7f1b5800b410 comp rx=0 tx=0).stop 2024-03-11T19:37:20.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 -- 172.21.15.83:0/1168781068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b5c106f90 msgr2=0x7f1b5c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:20.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b5c106f90 0x7f1b5c19cbe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400b070 tx=0x7f1b54004e40 comp rx=0 tx=0).stop 2024-03-11T19:37:20.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 -- 172.21.15.83:0/1168781068 shutdown_connections 2024-03-11T19:37:20.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1b480a1bd0 0x7f1b480a4090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:20.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b5c106f90 0x7f1b5c19cbe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:20.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b5c101eb0 0x7f1b5c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:20.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 --2- 172.21.15.83:0/1168781068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b5c102860 0x7f1b5c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:20.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 -- 172.21.15.83:0/1168781068 >> 172.21.15.83:0/1168781068 conn(0x7f1b5c0fd720 msgr2=0x7f1b5c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:20.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 -- 172.21.15.83:0/1168781068 shutdown_connections 2024-03-11T19:37:20.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:20.503+0000 7f1b63f02700 1 -- 172.21.15.83:0/1168781068 wait complete. 2024-03-11T19:37:20.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:20 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1168781068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:20.799 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:20 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1168781068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:20.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:20 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1168781068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:21.061 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-11T19:37:21.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd dump --format=json 2024-03-11T19:37:21.304 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:21.677 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:21 smithi083 ceph-mon[31864]: pgmap v291: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.0 KiB/s wr, 16 op/s 2024-03-11T19:37:21.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:21 smithi137 ceph-mon[36220]: pgmap v291: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.0 KiB/s wr, 16 op/s 2024-03-11T19:37:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:21 smithi204 ceph-mon[36739]: pgmap v291: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.0 KiB/s wr, 16 op/s 2024-03-11T19:37:23.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.040+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3288095310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b4101530 msgr2=0x7fb4b410d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:23.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.040+0000 7fb4bb8e3700 1 --2- 172.21.15.83:0/3288095310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b4101530 0x7fb4b410d9c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00b0a0 tx=0x7fb4ac00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:37:23.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.040+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3288095310 shutdown_connections 2024-03-11T19:37:23.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.040+0000 7fb4bb8e3700 1 --2- 172.21.15.83:0/3288095310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b4101530 0x7fb4b410d9c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:23.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.040+0000 7fb4bb8e3700 1 --2- 172.21.15.83:0/3288095310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4b4106b80 0x7fb4b4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:23.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.040+0000 7fb4bb8e3700 1 --2- 172.21.15.83:0/3288095310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b4100b70 0x7fb4b4100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.040+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3288095310 >> 172.21.15.83:0/3288095310 conn(0x7fb4b4077aa0 msgr2=0x7fb4b4077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.040+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3288095310 shutdown_connections 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.040+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3288095310 wait complete. 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4bb8e3700 1 Processor -- start 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4bb8e3700 1 -- start start 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4bb8e3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4b4100b70 0x7fb4b4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4bb8e3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b4101530 0x7fb4b4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4bb8e3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b4106b80 0x7fb4b419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4bb8e3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b419d220 con 0x7fb4b4106b80 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4bb8e3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b419d390 con 0x7fb4b4100b70 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4bb8e3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b419d550 con 0x7fb4b4101530 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4b9e80700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b4106b80 0x7fb4b419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4b9e80700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b4106b80 0x7fb4b419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33966/0 (socket says 172.21.15.83:33966) 2024-03-11T19:37:23.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4b9e80700 1 -- 172.21.15.83:0/3407832638 learned_addr learned my addr 172.21.15.83:0/3407832638 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4b967f700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4b4100b70 0x7fb4b4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4b8e7e700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b4101530 0x7fb4b4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4b9e80700 1 -- 172.21.15.83:0/3407832638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b4101530 msgr2=0x7fb4b4198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4b9e80700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b4101530 0x7fb4b4198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4b9e80700 1 -- 172.21.15.83:0/3407832638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4b4100b70 msgr2=0x7fb4b4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4b9e80700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4b4100b70 0x7fb4b4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4b9e80700 1 -- 172.21.15.83:0/3407832638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac009d00 con 0x7fb4b4106b80 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.041+0000 7fb4b9e80700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b4106b80 0x7fb4b419cc10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00b830 tx=0x7fb4ac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.042+0000 7fb4aa7fc700 1 -- 172.21.15.83:0/3407832638 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00e040 con 0x7fb4b4106b80 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.042+0000 7fb4aa7fc700 1 -- 172.21.15.83:0/3407832638 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4ac01bb30 con 0x7fb4b4106b80 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.042+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3407832638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b419d7d0 con 0x7fb4b4106b80 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.042+0000 7fb4aa7fc700 1 -- 172.21.15.83:0/3407832638 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0058b0 con 0x7fb4b4106b80 2024-03-11T19:37:23.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.042+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3407832638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b41a5090 con 0x7fb4b4106b80 2024-03-11T19:37:23.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.043+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3407832638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b4068930 con 0x7fb4b4106b80 2024-03-11T19:37:23.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.044+0000 7fb4aa7fc700 1 -- 172.21.15.83:0/3407832638 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fb4ac01bca0 con 0x7fb4b4106b80 2024-03-11T19:37:23.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.044+0000 7fb4aa7fc700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4a00a1b50 0x7fb4a00a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:23.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.044+0000 7fb4b967f700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4a00a1b50 0x7fb4a00a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:23.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.044+0000 7fb4aa7fc700 1 -- 172.21.15.83:0/3407832638 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb4ac0cec00 con 0x7fb4b4106b80 2024-03-11T19:37:23.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.045+0000 7fb4b967f700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4a00a1b50 0x7fb4a00a4010 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009a10 tx=0x7fb4a4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:23.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.049+0000 7fb4aa7fc700 1 -- 172.21.15.83:0/3407832638 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb4ac0d4020 con 0x7fb4b4106b80 2024-03-11T19:37:23.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.240+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3407832638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4b4110c70 con 0x7fb4b4106b80 2024-03-11T19:37:23.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.242+0000 7fb4aa7fc700 1 -- 172.21.15.83:0/3407832638 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22232 (secure 0 0 0) 0x7fb4ac097ef0 con 0x7fb4b4106b80 2024-03-11T19:37:23.246 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:37:23.246 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":85,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:37:06.317590+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:37:23.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3407832638 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4a00a1b50 msgr2=0x7fb4a00a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:23.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4a00a1b50 0x7fb4a00a4010 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009a10 tx=0x7fb4a4009190 comp rx=0 tx=0).stop 2024-03-11T19:37:23.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3407832638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b4106b80 msgr2=0x7fb4b419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:23.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b4106b80 0x7fb4b419cc10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00b830 tx=0x7fb4ac00cf90 comp rx=0 tx=0).stop 2024-03-11T19:37:23.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3407832638 shutdown_connections 2024-03-11T19:37:23.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4a00a1b50 0x7fb4a00a4010 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:23.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4b4106b80 0x7fb4b419cc10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:23.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4b4100b70 0x7fb4b4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:23.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 --2- 172.21.15.83:0/3407832638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b4101530 0x7fb4b4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:23.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3407832638 >> 172.21.15.83:0/3407832638 conn(0x7fb4b4077aa0 msgr2=0x7fb4b40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:23.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3407832638 shutdown_connections 2024-03-11T19:37:23.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:23.244+0000 7fb4bb8e3700 1 -- 172.21.15.83:0/3407832638 wait complete. 2024-03-11T19:37:23.615 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:23 smithi083 ceph-mon[31864]: pgmap v292: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-03-11T19:37:23.615 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:23 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3407832638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:23.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-11T19:37:23.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-11T19:37:23.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-11T19:37:23.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-11T19:37:23.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-11T19:37:23.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-11T19:37:23.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-11T19:37:23.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-11T19:37:23.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-11T19:37:23.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-11T19:37:23.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-11T19:37:23.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-11T19:37:23.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:23 smithi137 ceph-mon[36220]: pgmap v292: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-03-11T19:37:23.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:23 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3407832638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:23 smithi204 ceph-mon[36739]: pgmap v292: 97 pgs: 97 active+clean; 627 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 17 op/s 2024-03-11T19:37:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:23 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3407832638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:37:24.508 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:24.508 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:24.512 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:25.665 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:25 smithi083 ceph-mon[31864]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-03-11T19:37:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:25.743 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:25.745 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:25.745 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:25.746 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:25.746 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:25.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:25 smithi137 ceph-mon[36220]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-03-11T19:37:25.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:25 smithi204 ceph-mon[36739]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-03-11T19:37:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.701+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3528210712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78107ca0 msgr2=0x7f6d78108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.701+0000 7f6d7dc7b700 1 --2- 172.21.15.83:0/3528210712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78107ca0 0x7f6d78108080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6d680099c0 tx=0x7f6d68009cd0 comp rx=0 tx=0).stop 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.701+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3528210712 shutdown_connections 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.701+0000 7f6d7dc7b700 1 --2- 172.21.15.83:0/3528210712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d78102680 0x7f6d7810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.702+0000 7f6d7dc7b700 1 --2- 172.21.15.83:0/3528210712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78107ca0 0x7f6d78108080 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.702+0000 7f6d7dc7b700 1 --2- 172.21.15.83:0/3528210712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d78101cc0 0x7f6d78102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.702+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3528210712 >> 172.21.15.83:0/3528210712 conn(0x7f6d78078da0 msgr2=0x7f6d780791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.702+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3528210712 shutdown_connections 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.702+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3528210712 wait complete. 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.702+0000 7f6d7dc7b700 1 Processor -- start 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.702+0000 7f6d7dc7b700 1 -- start start 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d7dc7b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d78101cc0 0x7f6d78073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d7dc7b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78102680 0x7f6d780744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d7dc7b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d78107ca0 0x7f6d780749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d7dc7b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d781a6580 con 0x7f6d78102680 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d7dc7b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d781a66f0 con 0x7f6d78107ca0 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d7dc7b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d781a68b0 con 0x7f6d78101cc0 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d76ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78102680 0x7f6d780744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d76ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78102680 0x7f6d780744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33980/0 (socket says 172.21.15.83:33980) 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d76ffd700 1 -- 172.21.15.83:0/3223521226 learned_addr learned my addr 172.21.15.83:0/3223521226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d77fff700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d78107ca0 0x7f6d780749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d777fe700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d78101cc0 0x7f6d78073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d76ffd700 1 -- 172.21.15.83:0/3223521226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d78101cc0 msgr2=0x7f6d78073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d76ffd700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d78101cc0 0x7f6d78073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d76ffd700 1 -- 172.21.15.83:0/3223521226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d78107ca0 msgr2=0x7f6d780749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d76ffd700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d78107ca0 0x7f6d780749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d76ffd700 1 -- 172.21.15.83:0/3223521226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d680095f0 con 0x7f6d78102680 2024-03-11T19:37:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d76ffd700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78102680 0x7f6d780744b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00b530 tx=0x7f6d6c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d74ff9700 1 -- 172.21.15.83:0/3223521226 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c011830 con 0x7f6d78102680 2024-03-11T19:37:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3223521226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d781a6b90 con 0x7f6d78102680 2024-03-11T19:37:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d74ff9700 1 -- 172.21.15.83:0/3223521226 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c011e70 con 0x7f6d78102680 2024-03-11T19:37:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.703+0000 7f6d74ff9700 1 -- 172.21.15.83:0/3223521226 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00f300 con 0x7f6d78102680 2024-03-11T19:37:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.704+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3223521226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d781a7090 con 0x7f6d78102680 2024-03-11T19:37:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.704+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3223521226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d78112000 con 0x7f6d78102680 2024-03-11T19:37:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.706+0000 7f6d74ff9700 1 -- 172.21.15.83:0/3223521226 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f6d6c0106d0 con 0x7f6d78102680 2024-03-11T19:37:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.706+0000 7f6d74ff9700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6d600a1b50 0x7f6d600a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.706+0000 7f6d777fe700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6d600a1b50 0x7f6d600a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.706+0000 7f6d74ff9700 1 -- 172.21.15.83:0/3223521226 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d6c0cd380 con 0x7f6d78102680 2024-03-11T19:37:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.706+0000 7f6d74ff9700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285] conn(0x7f6d600ab330 0x7f6d600ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.706+0000 7f6d74ff9700 1 -- 172.21.15.83:0/3223521226 --> [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d600ade00 con 0x7f6d600ab330 2024-03-11T19:37:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.706+0000 7f6d74ff9700 1 -- 172.21.15.83:0/3223521226 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d6c0cd820 con 0x7f6d78102680 2024-03-11T19:37:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.707+0000 7f6d77fff700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285] conn(0x7f6d600ab330 0x7f6d600ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.707+0000 7f6d777fe700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6d600a1b50 0x7f6d600a4010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009fd0 tx=0x7f6d68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.707+0000 7f6d77fff700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285] conn(0x7f6d600ab330 0x7f6d600ad750 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.707+0000 7f6d74ff9700 1 -- 172.21.15.83:0/3223521226 <== osd.0 v2:172.21.15.83:6802/3703691285 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6d600ade00 con 0x7f6d600ab330 2024-03-11T19:37:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.730+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3223521226 --> [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6d78062380 con 0x7f6d600ab330 2024-03-11T19:37:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.730+0000 7f6d74ff9700 1 -- 172.21.15.83:0/3223521226 <== osd.0 v2:172.21.15.83:6802/3703691285 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6d78062380 con 0x7f6d600ab330 2024-03-11T19:37:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.730+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285] conn(0x7f6d600ab330 msgr2=0x7f6d600ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.730+0000 7f6d7dc7b700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285] conn(0x7f6d600ab330 0x7f6d600ad750 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.730+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6d600a1b50 msgr2=0x7f6d600a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.730+0000 7f6d7dc7b700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6d600a1b50 0x7f6d600a4010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009fd0 tx=0x7f6d68005fd0 comp rx=0 tx=0).stop 2024-03-11T19:37:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.730+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78102680 msgr2=0x7f6d780744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.730+0000 7f6d7dc7b700 1 --2- 172.21.15.83:0/3223521226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d78102680 0x7f6d780744b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00b530 tx=0x7f6d6c00b840 comp rx=0 tx=0).stop 2024-03-11T19:37:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.730+0000 7f6d77fff700 1 -- 172.21.15.83:0/3223521226 reap_dead start 2024-03-11T19:37:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.730+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3223521226 shutdown_connections 2024-03-11T19:37:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.730+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3223521226 >> 172.21.15.83:0/3223521226 conn(0x7f6d78078da0 msgr2=0x7f6d78100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.731+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3223521226 shutdown_connections 2024-03-11T19:37:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:27.731+0000 7f6d7dc7b700 1 -- 172.21.15.83:0/3223521226 wait complete. 2024-03-11T19:37:27.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:27 smithi137 ceph-mon[36220]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-11T19:37:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:27 smithi204 ceph-mon[36739]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-11T19:37:27.914 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:27 smithi083 ceph-mon[31864]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-11T19:37:29.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:29 smithi083 ceph-mon[31864]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-03-11T19:37:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:29 smithi137 ceph-mon[36220]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-03-11T19:37:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:29 smithi204 ceph-mon[36739]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-03-11T19:37:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.576+0000 7f4af4f53700 1 -- 172.21.15.83:0/3893834275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af0106ce0 msgr2=0x7f4af01070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.576+0000 7f4af4f53700 1 --2- 172.21.15.83:0/3893834275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af0106ce0 0x7f4af01070c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009990 tx=0x7f4ae0009ca0 comp rx=0 tx=0).stop 2024-03-11T19:37:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.576+0000 7f4af4f53700 1 -- 172.21.15.83:0/3893834275 shutdown_connections 2024-03-11T19:37:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.576+0000 7f4af4f53700 1 --2- 172.21.15.83:0/3893834275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af0100c20 0x7f4af010eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.576+0000 7f4af4f53700 1 --2- 172.21.15.83:0/3893834275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af0106ce0 0x7f4af01070c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.576+0000 7f4af4f53700 1 --2- 172.21.15.83:0/3893834275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af0100260 0x7f4af01006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.576+0000 7f4af4f53700 1 -- 172.21.15.83:0/3893834275 >> 172.21.15.83:0/3893834275 conn(0x7f4af0077170 msgr2=0x7f4af00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.576+0000 7f4af4f53700 1 -- 172.21.15.83:0/3893834275 shutdown_connections 2024-03-11T19:37:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.576+0000 7f4af4f53700 1 -- 172.21.15.83:0/3893834275 wait complete. 2024-03-11T19:37:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4af4f53700 1 Processor -- start 2024-03-11T19:37:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4af4f53700 1 -- start start 2024-03-11T19:37:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4af4f53700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af0100260 0x7f4af0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4af4f53700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af0100c20 0x7f4af01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4af4f53700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af0106ce0 0x7f4af019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4af4f53700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af019d210 con 0x7f4af0100c20 2024-03-11T19:37:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4af4f53700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af019d380 con 0x7f4af0100260 2024-03-11T19:37:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4af4f53700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af019d540 con 0x7f4af0106ce0 2024-03-11T19:37:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4aedd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af0100c20 0x7f4af01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4aedd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af0100c20 0x7f4af01988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45926/0 (socket says 172.21.15.83:45926) 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4aedd9b700 1 -- 172.21.15.83:0/1658130550 learned_addr learned my addr 172.21.15.83:0/1658130550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4aee59c700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af0100260 0x7f4af0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.577+0000 7f4aeed9d700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af0106ce0 0x7f4af019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.578+0000 7f4aedd9b700 1 -- 172.21.15.83:0/1658130550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af0106ce0 msgr2=0x7f4af019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.578+0000 7f4aedd9b700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af0106ce0 0x7f4af019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.578+0000 7f4aedd9b700 1 -- 172.21.15.83:0/1658130550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af0100260 msgr2=0x7f4af0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.578+0000 7f4aedd9b700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4af0100260 0x7f4af0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.578+0000 7f4aedd9b700 1 -- 172.21.15.83:0/1658130550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae00095f0 con 0x7f4af0100c20 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.578+0000 7f4aedd9b700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af0100c20 0x7f4af01988c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400d950 tx=0x7f4ae400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.579+0000 7f4adf7fe700 1 -- 172.21.15.83:0/1658130550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400f890 con 0x7f4af0100c20 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.579+0000 7f4adf7fe700 1 -- 172.21.15.83:0/1658130550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ae4010460 con 0x7f4af0100c20 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.579+0000 7f4af4f53700 1 -- 172.21.15.83:0/1658130550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af019d820 con 0x7f4af0100c20 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.579+0000 7f4adf7fe700 1 -- 172.21.15.83:0/1658130550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400e610 con 0x7f4af0100c20 2024-03-11T19:37:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.579+0000 7f4af4f53700 1 -- 172.21.15.83:0/1658130550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af01a5080 con 0x7f4af0100c20 2024-03-11T19:37:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.580+0000 7f4af4f53700 1 -- 172.21.15.83:0/1658130550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4af01121e0 con 0x7f4af0100c20 2024-03-11T19:37:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.581+0000 7f4adf7fe700 1 -- 172.21.15.83:0/1658130550 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f4ae4010960 con 0x7f4af0100c20 2024-03-11T19:37:31.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.581+0000 7f4adf7fe700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4ad80a1bd0 0x7f4ad80a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:31.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.582+0000 7f4aee59c700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4ad80a1bd0 0x7f4ad80a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:31.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.582+0000 7f4adf7fe700 1 -- 172.21.15.83:0/1658130550 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4ae40cd670 con 0x7f4af0100c20 2024-03-11T19:37:31.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.582+0000 7f4adf7fe700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819] conn(0x7f4ad80ab3b0 0x7f4ad80ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:31.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.582+0000 7f4adf7fe700 1 -- 172.21.15.83:0/1658130550 --> [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ad80ade80 con 0x7f4ad80ab3b0 2024-03-11T19:37:31.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.582+0000 7f4adf7fe700 1 -- 172.21.15.83:0/1658130550 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ae40cda90 con 0x7f4af0100c20 2024-03-11T19:37:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.582+0000 7f4aee59c700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4ad80a1bd0 0x7f4ad80a4090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000b5c0 tx=0x7f4ae0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.582+0000 7f4aeed9d700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819] conn(0x7f4ad80ab3b0 0x7f4ad80ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.582+0000 7f4aeed9d700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819] conn(0x7f4ad80ab3b0 0x7f4ad80ad7d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.583+0000 7f4adf7fe700 1 -- 172.21.15.83:0/1658130550 <== osd.11 v2:172.21.15.204:6824/2753506819 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4ad80ade80 con 0x7f4ad80ab3b0 2024-03-11T19:37:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.599+0000 7f4af4f53700 1 -- 172.21.15.83:0/1658130550 --> [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4af0062300 con 0x7f4ad80ab3b0 2024-03-11T19:37:31.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.600+0000 7f4adf7fe700 1 -- 172.21.15.83:0/1658130550 <== osd.11 v2:172.21.15.204:6824/2753506819 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4af0062300 con 0x7f4ad80ab3b0 2024-03-11T19:37:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.600+0000 7f4af4f53700 1 -- 172.21.15.83:0/1658130550 >> [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819] conn(0x7f4ad80ab3b0 msgr2=0x7f4ad80ad7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.600+0000 7f4af4f53700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819] conn(0x7f4ad80ab3b0 0x7f4ad80ad7d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.600+0000 7f4af4f53700 1 -- 172.21.15.83:0/1658130550 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4ad80a1bd0 msgr2=0x7f4ad80a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.600+0000 7f4af4f53700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4ad80a1bd0 0x7f4ad80a4090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000b5c0 tx=0x7f4ae0005fd0 comp rx=0 tx=0).stop 2024-03-11T19:37:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.600+0000 7f4af4f53700 1 -- 172.21.15.83:0/1658130550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af0100c20 msgr2=0x7f4af01988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.600+0000 7f4af4f53700 1 --2- 172.21.15.83:0/1658130550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4af0100c20 0x7f4af01988c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400d950 tx=0x7f4ae400dc60 comp rx=0 tx=0).stop 2024-03-11T19:37:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.600+0000 7f4aeed9d700 1 -- 172.21.15.83:0/1658130550 reap_dead start 2024-03-11T19:37:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.601+0000 7f4af4f53700 1 -- 172.21.15.83:0/1658130550 shutdown_connections 2024-03-11T19:37:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.601+0000 7f4af4f53700 1 -- 172.21.15.83:0/1658130550 >> 172.21.15.83:0/1658130550 conn(0x7f4af0077170 msgr2=0x7f4af0104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.601+0000 7f4af4f53700 1 -- 172.21.15.83:0/1658130550 shutdown_connections 2024-03-11T19:37:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:31.601+0000 7f4af4f53700 1 -- 172.21.15.83:0/1658130550 wait complete. 2024-03-11T19:37:31.786 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:31 smithi083 ceph-mon[31864]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-11T19:37:31.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:31 smithi137 ceph-mon[36220]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-11T19:37:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:31 smithi204 ceph-mon[36739]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-11T19:37:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.185+0000 7fb146e61700 1 -- 172.21.15.83:0/1618001596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb14010e940 msgr2=0x7fb140110d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:32.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.185+0000 7fb146e61700 1 --2- 172.21.15.83:0/1618001596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb14010e940 0x7fb140110d30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb13800b0a0 tx=0x7fb13800b3b0 comp rx=0 tx=0).stop 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 -- 172.21.15.83:0/1618001596 shutdown_connections 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 --2- 172.21.15.83:0/1618001596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb14010e940 0x7fb140110d30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 --2- 172.21.15.83:0/1618001596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb140102bd0 0x7fb1401001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 --2- 172.21.15.83:0/1618001596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb140103040 0x7fb140100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 -- 172.21.15.83:0/1618001596 >> 172.21.15.83:0/1618001596 conn(0x7fb140077170 msgr2=0x7fb1400fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 -- 172.21.15.83:0/1618001596 shutdown_connections 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 -- 172.21.15.83:0/1618001596 wait complete. 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 Processor -- start 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 -- start start 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb140102bd0 0x7fb14019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb140103040 0x7fb14019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb14010e940 0x7fb14019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:32.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14019f590 con 0x7fb14010e940 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14019f700 con 0x7fb140102bd0 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.186+0000 7fb146e61700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14019f8c0 con 0x7fb140103040 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb1453fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb14010e940 0x7fb14019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb1453fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb14010e940 0x7fb14019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45938/0 (socket says 172.21.15.83:45938) 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb1453fe700 1 -- 172.21.15.83:0/3750467114 learned_addr learned my addr 172.21.15.83:0/3750467114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb13ffff700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb140103040 0x7fb14019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb1453fe700 1 -- 172.21.15.83:0/3750467114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb140103040 msgr2=0x7fb14019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb1453fe700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb140103040 0x7fb14019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb1453fe700 1 -- 172.21.15.83:0/3750467114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb140102bd0 msgr2=0x7fb14019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb1453fe700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb140102bd0 0x7fb14019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb1453fe700 1 -- 172.21.15.83:0/3750467114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb138009d00 con 0x7fb14010e940 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb13ffff700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb140103040 0x7fb14019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb1453fe700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb14010e940 0x7fb14019ef80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb13800b830 tx=0x7fb1380075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:32.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb13dffb700 1 -- 172.21.15.83:0/3750467114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13800e040 con 0x7fb14010e940 2024-03-11T19:37:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb13dffb700 1 -- 172.21.15.83:0/3750467114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb13801bb30 con 0x7fb14010e940 2024-03-11T19:37:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb146e61700 1 -- 172.21.15.83:0/3750467114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14019fb40 con 0x7fb14010e940 2024-03-11T19:37:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb13dffb700 1 -- 172.21.15.83:0/3750467114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1380058f0 con 0x7fb14010e940 2024-03-11T19:37:32.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.187+0000 7fb146e61700 1 -- 172.21.15.83:0/3750467114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1401a7400 con 0x7fb14010e940 2024-03-11T19:37:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.188+0000 7fb146e61700 1 -- 172.21.15.83:0/3750467114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb140114310 con 0x7fb14010e940 2024-03-11T19:37:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.189+0000 7fb13dffb700 1 -- 172.21.15.83:0/3750467114 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fb138005040 con 0x7fb14010e940 2024-03-11T19:37:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.190+0000 7fb13dffb700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb1300a1bd0 0x7fb1300a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.190+0000 7fb144bfd700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb1300a1bd0 0x7fb1300a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.190+0000 7fb13dffb700 1 -- 172.21.15.83:0/3750467114 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb1380cf700 con 0x7fb14010e940 2024-03-11T19:37:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.190+0000 7fb13dffb700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205] conn(0x7fb1300ab3b0 0x7fb1300ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.190+0000 7fb13dffb700 1 -- 172.21.15.83:0/3750467114 --> [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1300ade80 con 0x7fb1300ab3b0 2024-03-11T19:37:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.190+0000 7fb13dffb700 1 -- 172.21.15.83:0/3750467114 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1380cfba0 con 0x7fb14010e940 2024-03-11T19:37:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.190+0000 7fb144bfd700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb1300a1bd0 0x7fb1300a4090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb1340097b0 tx=0x7fb134006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.190+0000 7fb13ffff700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205] conn(0x7fb1300ab3b0 0x7fb1300ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.191+0000 7fb13ffff700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205] conn(0x7fb1300ab3b0 0x7fb1300ad7d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.192+0000 7fb13dffb700 1 -- 172.21.15.83:0/3750467114 <== osd.1 v2:172.21.15.137:6800/415302205 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb1300ade80 con 0x7fb1300ab3b0 2024-03-11T19:37:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.208+0000 7fb146e61700 1 -- 172.21.15.83:0/3750467114 --> [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb140110760 con 0x7fb1300ab3b0 2024-03-11T19:37:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.208+0000 7fb13dffb700 1 -- 172.21.15.83:0/3750467114 <== osd.1 v2:172.21.15.137:6800/415302205 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb140110760 con 0x7fb1300ab3b0 2024-03-11T19:37:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.209+0000 7fb146e61700 1 -- 172.21.15.83:0/3750467114 >> [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205] conn(0x7fb1300ab3b0 msgr2=0x7fb1300ad7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.209+0000 7fb146e61700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205] conn(0x7fb1300ab3b0 0x7fb1300ad7d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.209+0000 7fb146e61700 1 -- 172.21.15.83:0/3750467114 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb1300a1bd0 msgr2=0x7fb1300a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:32.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.209+0000 7fb146e61700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb1300a1bd0 0x7fb1300a4090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb1340097b0 tx=0x7fb134006d20 comp rx=0 tx=0).stop 2024-03-11T19:37:32.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.209+0000 7fb146e61700 1 -- 172.21.15.83:0/3750467114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb14010e940 msgr2=0x7fb14019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:32.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.209+0000 7fb146e61700 1 --2- 172.21.15.83:0/3750467114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb14010e940 0x7fb14019ef80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb13800b830 tx=0x7fb1380075c0 comp rx=0 tx=0).stop 2024-03-11T19:37:32.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.209+0000 7fb1453fe700 1 -- 172.21.15.83:0/3750467114 reap_dead start 2024-03-11T19:37:32.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.209+0000 7fb146e61700 1 -- 172.21.15.83:0/3750467114 shutdown_connections 2024-03-11T19:37:32.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.209+0000 7fb146e61700 1 -- 172.21.15.83:0/3750467114 >> 172.21.15.83:0/3750467114 conn(0x7fb140077170 msgr2=0x7fb14010fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:32.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.209+0000 7fb146e61700 1 -- 172.21.15.83:0/3750467114 shutdown_connections 2024-03-11T19:37:32.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:32.209+0000 7fb146e61700 1 -- 172.21.15.83:0/3750467114 wait complete. 2024-03-11T19:37:33.715 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:33 smithi083 ceph-mon[31864]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:37:33.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:33 smithi137 ceph-mon[36220]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:37:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:33 smithi204 ceph-mon[36739]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:37:34.077 INFO:teuthology.orchestra.run.smithi083.stdout:343597383698 2024-03-11T19:37:34.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-11T19:37:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:35.716 INFO:teuthology.orchestra.run.smithi083.stdout:34359738440 2024-03-11T19:37:35.717 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-11T19:37:35.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:35 smithi137 ceph-mon[36220]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:37:35.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:35 smithi204 ceph-mon[36739]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:37:36.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:35 smithi083 ceph-mon[31864]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-11T19:37:37.728 INFO:teuthology.orchestra.run.smithi083.stdout:55834574916 2024-03-11T19:37:37.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-11T19:37:37.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 -- 172.21.15.83:0/3822773145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac107ca0 msgr2=0x7ff5ac108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 --2- 172.21.15.83:0/3822773145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac107ca0 0x7ff5ac108080 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff5a00099c0 tx=0x7ff5a0009cd0 comp rx=0 tx=0).stop 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 -- 172.21.15.83:0/3822773145 shutdown_connections 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 --2- 172.21.15.83:0/3822773145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5ac102680 0x7ff5ac10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 --2- 172.21.15.83:0/3822773145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac107ca0 0x7ff5ac108080 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 --2- 172.21.15.83:0/3822773145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5ac101cc0 0x7ff5ac102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 -- 172.21.15.83:0/3822773145 >> 172.21.15.83:0/3822773145 conn(0x7ff5ac078da0 msgr2=0x7ff5ac0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 -- 172.21.15.83:0/3822773145 shutdown_connections 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 -- 172.21.15.83:0/3822773145 wait complete. 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 Processor -- start 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 -- start start 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5ac101cc0 0x7ff5ac073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac102680 0x7ff5ac0744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.393+0000 7ff5b31b9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5ac107ca0 0x7ff5ac0749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5b31b9700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac1a65a0 con 0x7ff5ac102680 2024-03-11T19:37:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5b31b9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac1a6710 con 0x7ff5ac107ca0 2024-03-11T19:37:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5b31b9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac1a68d0 con 0x7ff5ac101cc0 2024-03-11T19:37:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5abfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac102680 0x7ff5ac0744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5abfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac102680 0x7ff5ac0744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45964/0 (socket says 172.21.15.83:45964) 2024-03-11T19:37:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5abfff700 1 -- 172.21.15.83:0/2557232116 learned_addr learned my addr 172.21.15.83:0/2557232116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5b0f55700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5ac101cc0 0x7ff5ac073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5abfff700 1 -- 172.21.15.83:0/2557232116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5ac101cc0 msgr2=0x7ff5ac073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5abfff700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5ac101cc0 0x7ff5ac073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5abfff700 1 -- 172.21.15.83:0/2557232116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5ac107ca0 msgr2=0x7ff5ac0749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5abfff700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5ac107ca0 0x7ff5ac0749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5abfff700 1 -- 172.21.15.83:0/2557232116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a00095f0 con 0x7ff5ac102680 2024-03-11T19:37:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5b0f55700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5ac101cc0 0x7ff5ac073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5abfff700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac102680 0x7ff5ac0744b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c9b0 tx=0x7ff59800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5a9ffb700 1 -- 172.21.15.83:0/2557232116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598007b00 con 0x7ff5ac102680 2024-03-11T19:37:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5b31b9700 1 -- 172.21.15.83:0/2557232116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac1a6bb0 con 0x7ff5ac102680 2024-03-11T19:37:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5a9ffb700 1 -- 172.21.15.83:0/2557232116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff598007c60 con 0x7ff5ac102680 2024-03-11T19:37:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5a9ffb700 1 -- 172.21.15.83:0/2557232116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598018730 con 0x7ff5ac102680 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.394+0000 7ff5b31b9700 1 -- 172.21.15.83:0/2557232116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac1a7100 con 0x7ff5ac102680 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.395+0000 7ff5b31b9700 1 -- 172.21.15.83:0/2557232116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5ac112000 con 0x7ff5ac102680 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.396+0000 7ff5a9ffb700 1 -- 172.21.15.83:0/2557232116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7ff598018890 con 0x7ff5ac102680 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.397+0000 7ff5a9ffb700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff59c0a1b50 0x7ff59c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.397+0000 7ff5b0f55700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff59c0a1b50 0x7ff59c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.397+0000 7ff5a9ffb700 1 -- 172.21.15.83:0/2557232116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff5980cd940 con 0x7ff5ac102680 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.397+0000 7ff5a9ffb700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423] conn(0x7ff59c0ab330 0x7ff59c0ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.397+0000 7ff5a9ffb700 1 -- 172.21.15.83:0/2557232116 --> [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff59c0ade00 con 0x7ff59c0ab330 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.397+0000 7ff5a9ffb700 1 -- 172.21.15.83:0/2557232116 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5980cdde0 con 0x7ff5ac102680 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.397+0000 7ff5b0f55700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff59c0a1b50 0x7ff59c0a4010 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009fd0 tx=0x7ff5a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.397+0000 7ff5b1756700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423] conn(0x7ff59c0ab330 0x7ff59c0ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.398+0000 7ff5b1756700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423] conn(0x7ff59c0ab330 0x7ff59c0ad750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.399+0000 7ff5a9ffb700 1 -- 172.21.15.83:0/2557232116 <== osd.10 v2:172.21.15.137:6824/3042743423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff59c0ade00 con 0x7ff59c0ab330 2024-03-11T19:37:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.414+0000 7ff5b31b9700 1 -- 172.21.15.83:0/2557232116 --> [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff5ac062300 con 0x7ff59c0ab330 2024-03-11T19:37:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.415+0000 7ff5a9ffb700 1 -- 172.21.15.83:0/2557232116 <== osd.10 v2:172.21.15.137:6824/3042743423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff5ac062300 con 0x7ff59c0ab330 2024-03-11T19:37:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.416+0000 7ff5b31b9700 1 -- 172.21.15.83:0/2557232116 >> [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423] conn(0x7ff59c0ab330 msgr2=0x7ff59c0ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.416+0000 7ff5b31b9700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423] conn(0x7ff59c0ab330 0x7ff59c0ad750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.416+0000 7ff5b31b9700 1 -- 172.21.15.83:0/2557232116 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff59c0a1b50 msgr2=0x7ff59c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.416+0000 7ff5b31b9700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff59c0a1b50 0x7ff59c0a4010 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009fd0 tx=0x7ff5a0005fd0 comp rx=0 tx=0).stop 2024-03-11T19:37:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.416+0000 7ff5b31b9700 1 -- 172.21.15.83:0/2557232116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac102680 msgr2=0x7ff5ac0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.416+0000 7ff5b31b9700 1 --2- 172.21.15.83:0/2557232116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac102680 0x7ff5ac0744b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c9b0 tx=0x7ff59800cd70 comp rx=0 tx=0).stop 2024-03-11T19:37:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.416+0000 7ff5b1756700 1 -- 172.21.15.83:0/2557232116 reap_dead start 2024-03-11T19:37:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.416+0000 7ff5b31b9700 1 -- 172.21.15.83:0/2557232116 shutdown_connections 2024-03-11T19:37:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.416+0000 7ff5b31b9700 1 -- 172.21.15.83:0/2557232116 >> 172.21.15.83:0/2557232116 conn(0x7ff5ac078da0 msgr2=0x7ff5ac1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:37.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.416+0000 7ff5b31b9700 1 -- 172.21.15.83:0/2557232116 shutdown_connections 2024-03-11T19:37:37.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:37.416+0000 7ff5b31b9700 1 -- 172.21.15.83:0/2557232116 wait complete. 2024-03-11T19:37:37.858 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:37 smithi083 ceph-mon[31864]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:37:37.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:37 smithi137 ceph-mon[36220]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:37:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:37 smithi204 ceph-mon[36739]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-11T19:37:39.182 INFO:teuthology.orchestra.run.smithi083.stdout:309237645336 2024-03-11T19:37:39.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-11T19:37:39.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.399+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3344014089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b0100ba0 msgr2=0x7f54b0101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:39.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.399+0000 7f54b7e6d700 1 --2- 172.21.15.83:0/3344014089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b0100ba0 0x7f54b0101020 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009990 tx=0x7f54a0009ca0 comp rx=0 tx=0).stop 2024-03-11T19:37:39.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.399+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3344014089 shutdown_connections 2024-03-11T19:37:39.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.399+0000 7f54b7e6d700 1 --2- 172.21.15.83:0/3344014089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b0106bd0 0x7f54b0106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:39.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.399+0000 7f54b7e6d700 1 --2- 172.21.15.83:0/3344014089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b0101560 0x7f54b010d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:39.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.399+0000 7f54b7e6d700 1 --2- 172.21.15.83:0/3344014089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b0100ba0 0x7f54b0101020 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:39.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.399+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3344014089 >> 172.21.15.83:0/3344014089 conn(0x7f54b0076bc0 msgr2=0x7f54b0076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:39.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.399+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3344014089 shutdown_connections 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.399+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3344014089 wait complete. 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b7e6d700 1 Processor -- start 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b7e6d700 1 -- start start 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b7e6d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b0100ba0 0x7f54b01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b7e6d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b0101560 0x7f54b0199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b7e6d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b0106bd0 0x7f54b01a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b7e6d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b0199ed0 con 0x7f54b0106bd0 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b7e6d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b01a46b0 con 0x7f54b0100ba0 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b7e6d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b01a47f0 con 0x7f54b0101560 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b640a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b0106bd0 0x7f54b01a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b640a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b0106bd0 0x7f54b01a4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45976/0 (socket says 172.21.15.83:45976) 2024-03-11T19:37:39.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b5c09700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b0100ba0 0x7f54b01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b640a700 1 -- 172.21.15.83:0/3982161109 learned_addr learned my addr 172.21.15.83:0/3982161109 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b5408700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b0101560 0x7f54b0199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b640a700 1 -- 172.21.15.83:0/3982161109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b0101560 msgr2=0x7f54b0199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b640a700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54b0101560 0x7f54b0199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b640a700 1 -- 172.21.15.83:0/3982161109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b0100ba0 msgr2=0x7f54b01990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b640a700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54b0100ba0 0x7f54b01990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.400+0000 7f54b640a700 1 -- 172.21.15.83:0/3982161109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a00095f0 con 0x7f54b0106bd0 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.401+0000 7f54b640a700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b0106bd0 0x7f54b01a4170 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b9a0 tx=0x7f54a800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.401+0000 7f54a6ffd700 1 -- 172.21.15.83:0/3982161109 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a800d850 con 0x7f54b0106bd0 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.401+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3982161109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b01a4aa0 con 0x7f54b0106bd0 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.401+0000 7f54a6ffd700 1 -- 172.21.15.83:0/3982161109 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54a800de90 con 0x7f54b0106bd0 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.401+0000 7f54a6ffd700 1 -- 172.21.15.83:0/3982161109 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8017420 con 0x7f54b0106bd0 2024-03-11T19:37:39.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.401+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3982161109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b01a4f70 con 0x7f54b0106bd0 2024-03-11T19:37:39.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.402+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3982161109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f54b0110f00 con 0x7f54b0106bd0 2024-03-11T19:37:39.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.403+0000 7f54a6ffd700 1 -- 172.21.15.83:0/3982161109 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f54a800e6b0 con 0x7f54b0106bd0 2024-03-11T19:37:39.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.404+0000 7f54a6ffd700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f549c0a1b50 0x7f549c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:39.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.404+0000 7f54a6ffd700 1 -- 172.21.15.83:0/3982161109 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f54a8006e40 con 0x7f54b0106bd0 2024-03-11T19:37:39.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.404+0000 7f54b5c09700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f549c0a1b50 0x7f549c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:39.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.404+0000 7f54a6ffd700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190] conn(0x7f549c0ab330 0x7f549c0ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:39.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.404+0000 7f54a6ffd700 1 -- 172.21.15.83:0/3982161109 --> [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f549c0ade00 con 0x7f549c0ab330 2024-03-11T19:37:39.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.404+0000 7f54a6ffd700 1 -- 172.21.15.83:0/3982161109 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54a80cd690 con 0x7f54b0106bd0 2024-03-11T19:37:39.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.404+0000 7f54b5408700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190] conn(0x7f549c0ab330 0x7f549c0ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:39.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.404+0000 7f54b5c09700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f549c0a1b50 0x7f549c0a4010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009a10 tx=0x7f54ac009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:39.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.405+0000 7f54b5408700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190] conn(0x7f549c0ab330 0x7f549c0ad750 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:39.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.405+0000 7f54a6ffd700 1 -- 172.21.15.83:0/3982161109 <== osd.9 v2:172.21.15.83:6826/2013526190 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f549c0ade00 con 0x7f549c0ab330 2024-03-11T19:37:39.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.422+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3982161109 --> [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f54b0062330 con 0x7f549c0ab330 2024-03-11T19:37:39.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54a6ffd700 1 -- 172.21.15.83:0/3982161109 <== osd.9 v2:172.21.15.83:6826/2013526190 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f54b0062330 con 0x7f549c0ab330 2024-03-11T19:37:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190] conn(0x7f549c0ab330 msgr2=0x7f549c0ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54b7e6d700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190] conn(0x7f549c0ab330 0x7f549c0ad750 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f549c0a1b50 msgr2=0x7f549c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54b7e6d700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f549c0a1b50 0x7f549c0a4010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009a10 tx=0x7f54ac009190 comp rx=0 tx=0).stop 2024-03-11T19:37:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b0106bd0 msgr2=0x7f54b01a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54b7e6d700 1 --2- 172.21.15.83:0/3982161109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54b0106bd0 0x7f54b01a4170 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b9a0 tx=0x7f54a800bcb0 comp rx=0 tx=0).stop 2024-03-11T19:37:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54b640a700 1 -- 172.21.15.83:0/3982161109 reap_dead start 2024-03-11T19:37:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3982161109 shutdown_connections 2024-03-11T19:37:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3982161109 >> 172.21.15.83:0/3982161109 conn(0x7f54b0076bc0 msgr2=0x7f54b00ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3982161109 shutdown_connections 2024-03-11T19:37:39.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.423+0000 7f54b7e6d700 1 -- 172.21.15.83:0/3982161109 wait complete. 2024-03-11T19:37:39.810 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:39 smithi083 ceph-mon[31864]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-11T19:37:39.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:39 smithi137 ceph-mon[36220]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-11T19:37:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:39 smithi204 ceph-mon[36739]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-11T19:37:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.616+0000 7f7012167700 1 -- 172.21.15.83:0/197819269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c101560 msgr2=0x7f700c10d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.616+0000 7f7012167700 1 --2- 172.21.15.83:0/197819269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c101560 0x7f700c10d9a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f700400b0a0 tx=0x7f700400b3b0 comp rx=0 tx=0).stop 2024-03-11T19:37:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.616+0000 7f7012167700 1 -- 172.21.15.83:0/197819269 shutdown_connections 2024-03-11T19:37:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.616+0000 7f7012167700 1 --2- 172.21.15.83:0/197819269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f700c106bd0 0x7f700c106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.616+0000 7f7012167700 1 --2- 172.21.15.83:0/197819269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c101560 0x7f700c10d9a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.616+0000 7f7012167700 1 --2- 172.21.15.83:0/197819269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c100ba0 0x7f700c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.616+0000 7f7012167700 1 -- 172.21.15.83:0/197819269 >> 172.21.15.83:0/197819269 conn(0x7f700c076bc0 msgr2=0x7f700c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.616+0000 7f7012167700 1 -- 172.21.15.83:0/197819269 shutdown_connections 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.616+0000 7f7012167700 1 -- 172.21.15.83:0/197819269 wait complete. 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.616+0000 7f7012167700 1 Processor -- start 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.616+0000 7f7012167700 1 -- start start 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f7012167700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f700c100ba0 0x7f700c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f7012167700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c101560 0x7f700c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f7012167700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c106bd0 0x7f700c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f7012167700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c19c070 con 0x7f700c101560 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f7012167700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c19c1e0 con 0x7f700c106bd0 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f7012167700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c19c3a0 con 0x7f700c100ba0 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f700affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c101560 0x7f700c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f700affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c101560 0x7f700c197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55476/0 (socket says 172.21.15.83:55476) 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f700affd700 1 -- 172.21.15.83:0/4199402173 learned_addr learned my addr 172.21.15.83:0/4199402173 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f700b7fe700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f700c100ba0 0x7f700c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f700bfff700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c106bd0 0x7f700c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f700affd700 1 -- 172.21.15.83:0/4199402173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f700c100ba0 msgr2=0x7f700c1971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f700affd700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f700c100ba0 0x7f700c1971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f700affd700 1 -- 172.21.15.83:0/4199402173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c106bd0 msgr2=0x7f700c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f700affd700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f700c106bd0 0x7f700c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f700affd700 1 -- 172.21.15.83:0/4199402173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7004009d00 con 0x7f700c101560 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f700affd700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c101560 0x7f700c197720 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f700000c9b0 tx=0x7f700000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f7008ff9700 1 -- 172.21.15.83:0/4199402173 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000007b00 con 0x7f700c101560 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f7012167700 1 -- 172.21.15.83:0/4199402173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c19c680 con 0x7f700c101560 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f7008ff9700 1 -- 172.21.15.83:0/4199402173 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7000007c60 con 0x7f700c101560 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f7008ff9700 1 -- 172.21.15.83:0/4199402173 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000018660 con 0x7f700c101560 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.617+0000 7f7012167700 1 -- 172.21.15.83:0/4199402173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c1a3ee0 con 0x7f700c101560 2024-03-11T19:37:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.618+0000 7f7012167700 1 -- 172.21.15.83:0/4199402173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f700c110e40 con 0x7f700c101560 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.619+0000 7f7008ff9700 1 -- 172.21.15.83:0/4199402173 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f700000f450 con 0x7f700c101560 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.620+0000 7f7008ff9700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ff40a1b50 0x7f6ff40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.620+0000 7f700b7fe700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ff40a1b50 0x7f6ff40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.620+0000 7f7008ff9700 1 -- 172.21.15.83:0/4199402173 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f70000cd840 con 0x7f700c101560 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.620+0000 7f7008ff9700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727] conn(0x7f6ff40ab330 0x7f6ff40ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.620+0000 7f7008ff9700 1 -- 172.21.15.83:0/4199402173 --> [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ff40ade00 con 0x7f6ff40ab330 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.620+0000 7f7008ff9700 1 -- 172.21.15.83:0/4199402173 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70000cdce0 con 0x7f700c101560 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.620+0000 7f700b7fe700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ff40a1b50 0x7f6ff40a4010 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009be0 tx=0x7f6ffc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.620+0000 7f700bfff700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727] conn(0x7f6ff40ab330 0x7f6ff40ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.621+0000 7f700bfff700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727] conn(0x7f6ff40ab330 0x7f6ff40ad750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.622+0000 7f7008ff9700 1 -- 172.21.15.83:0/4199402173 <== osd.2 v2:172.21.15.204:6800/4285299727 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6ff40ade00 con 0x7f6ff40ab330 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.638+0000 7f7012167700 1 -- 172.21.15.83:0/4199402173 --> [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f700c062300 con 0x7f6ff40ab330 2024-03-11T19:37:41.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.638+0000 7f7008ff9700 1 -- 172.21.15.83:0/4199402173 <== osd.2 v2:172.21.15.204:6800/4285299727 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f700c062300 con 0x7f6ff40ab330 2024-03-11T19:37:41.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.638+0000 7f7012167700 1 -- 172.21.15.83:0/4199402173 >> [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727] conn(0x7f6ff40ab330 msgr2=0x7f6ff40ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:41.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.638+0000 7f7012167700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727] conn(0x7f6ff40ab330 0x7f6ff40ad750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:41.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.639+0000 7f7012167700 1 -- 172.21.15.83:0/4199402173 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ff40a1b50 msgr2=0x7f6ff40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:41.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.639+0000 7f7012167700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ff40a1b50 0x7f6ff40a4010 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009be0 tx=0x7f6ffc009260 comp rx=0 tx=0).stop 2024-03-11T19:37:41.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.639+0000 7f7012167700 1 -- 172.21.15.83:0/4199402173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c101560 msgr2=0x7f700c197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:41.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.639+0000 7f7012167700 1 --2- 172.21.15.83:0/4199402173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f700c101560 0x7f700c197720 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f700000c9b0 tx=0x7f700000ccc0 comp rx=0 tx=0).stop 2024-03-11T19:37:41.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.639+0000 7f700bfff700 1 -- 172.21.15.83:0/4199402173 reap_dead start 2024-03-11T19:37:41.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.639+0000 7f7012167700 1 -- 172.21.15.83:0/4199402173 shutdown_connections 2024-03-11T19:37:41.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.639+0000 7f7012167700 1 -- 172.21.15.83:0/4199402173 >> 172.21.15.83:0/4199402173 conn(0x7f700c076bc0 msgr2=0x7f700c0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:41.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.639+0000 7f7012167700 1 -- 172.21.15.83:0/4199402173 shutdown_connections 2024-03-11T19:37:41.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:39.639+0000 7f7012167700 1 -- 172.21.15.83:0/4199402173 wait complete. 2024-03-11T19:37:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:41 smithi137 ceph-mon[36220]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:37:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:41 smithi204 ceph-mon[36739]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:37:42.000 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:41 smithi083 ceph-mon[31864]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:37:42.033 INFO:teuthology.orchestra.run.smithi083.stdout:77309411393 2024-03-11T19:37:42.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-11T19:37:42.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.941+0000 7f3569b39700 1 -- 172.21.15.83:0/3842288977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35641017b0 msgr2=0x7f35641057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:42.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.941+0000 7f3569b39700 1 --2- 172.21.15.83:0/3842288977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35641017b0 0x7f35641057b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3558009990 tx=0x7f3558009ca0 comp rx=0 tx=0).stop 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.941+0000 7f3569b39700 1 -- 172.21.15.83:0/3842288977 shutdown_connections 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.941+0000 7f3569b39700 1 --2- 172.21.15.83:0/3842288977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3564100e00 0x7f35641011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.941+0000 7f3569b39700 1 --2- 172.21.15.83:0/3842288977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3564105e90 0x7f356410da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.941+0000 7f3569b39700 1 --2- 172.21.15.83:0/3842288977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35641017b0 0x7f35641057b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.941+0000 7f3569b39700 1 -- 172.21.15.83:0/3842288977 >> 172.21.15.83:0/3842288977 conn(0x7f3564076b90 msgr2=0x7f3564076fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.941+0000 7f3569b39700 1 -- 172.21.15.83:0/3842288977 shutdown_connections 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.941+0000 7f3569b39700 1 -- 172.21.15.83:0/3842288977 wait complete. 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.942+0000 7f3569b39700 1 Processor -- start 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.942+0000 7f3569b39700 1 -- start start 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3569b39700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3564100e00 0x7f3564197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3569b39700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35641017b0 0x7f3564197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:42.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3569b39700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3564105e90 0x7f356419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3569b39700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356419bfe0 con 0x7f3564105e90 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3569b39700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356419c150 con 0x7f35641017b0 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3569b39700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356419c310 con 0x7f3564100e00 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3563fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3564105e90 0x7f356419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3563fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3564105e90 0x7f356419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55488/0 (socket says 172.21.15.83:55488) 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3563fff700 1 -- 172.21.15.83:0/880657976 learned_addr learned my addr 172.21.15.83:0/880657976 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f35637fe700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3564100e00 0x7f3564197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3562ffd700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35641017b0 0x7f3564197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3563fff700 1 -- 172.21.15.83:0/880657976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3564100e00 msgr2=0x7f3564197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3563fff700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3564100e00 0x7f3564197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3563fff700 1 -- 172.21.15.83:0/880657976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35641017b0 msgr2=0x7f3564197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3563fff700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35641017b0 0x7f3564197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:42.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3563fff700 1 -- 172.21.15.83:0/880657976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35580095f0 con 0x7f3564105e90 2024-03-11T19:37:42.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3562ffd700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35641017b0 0x7f3564197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:42.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3563fff700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3564105e90 0x7f356419b9d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f355c00b970 tx=0x7f355c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:42.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3560ff9700 1 -- 172.21.15.83:0/880657976 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c00c930 con 0x7f3564105e90 2024-03-11T19:37:42.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3569b39700 1 -- 172.21.15.83:0/880657976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356419c5f0 con 0x7f3564105e90 2024-03-11T19:37:42.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3560ff9700 1 -- 172.21.15.83:0/880657976 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f355c014440 con 0x7f3564105e90 2024-03-11T19:37:42.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.943+0000 7f3560ff9700 1 -- 172.21.15.83:0/880657976 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c0075d0 con 0x7f3564105e90 2024-03-11T19:37:42.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.944+0000 7f3569b39700 1 -- 172.21.15.83:0/880657976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35641a3e50 con 0x7f3564105e90 2024-03-11T19:37:42.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.944+0000 7f3569b39700 1 -- 172.21.15.83:0/880657976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3564110f00 con 0x7f3564105e90 2024-03-11T19:37:42.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.945+0000 7f3560ff9700 1 -- 172.21.15.83:0/880657976 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f355c0145b0 con 0x7f3564105e90 2024-03-11T19:37:42.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.946+0000 7f3560ff9700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f354c0a1b50 0x7f354c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:42.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.946+0000 7f35637fe700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f354c0a1b50 0x7f354c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:42.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.946+0000 7f3560ff9700 1 -- 172.21.15.83:0/880657976 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f355c006e40 con 0x7f3564105e90 2024-03-11T19:37:42.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.946+0000 7f3560ff9700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703] conn(0x7f354c0ab330 0x7f354c0ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:42.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.946+0000 7f3560ff9700 1 -- 172.21.15.83:0/880657976 --> [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f354c0ade00 con 0x7f354c0ab330 2024-03-11T19:37:42.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.946+0000 7f3560ff9700 1 -- 172.21.15.83:0/880657976 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f355c0d12c0 con 0x7f3564105e90 2024-03-11T19:37:42.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.946+0000 7f35637fe700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f354c0a1b50 0x7f354c0a4010 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3554005950 tx=0x7f355400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:42.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.947+0000 7f3562ffd700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703] conn(0x7f354c0ab330 0x7f354c0ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:42.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.947+0000 7f3562ffd700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703] conn(0x7f354c0ab330 0x7f354c0ad750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:42.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.948+0000 7f3560ff9700 1 -- 172.21.15.83:0/880657976 <== osd.4 v2:172.21.15.137:6808/4100342703 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f354c0ade00 con 0x7f354c0ab330 2024-03-11T19:37:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.964+0000 7f3569b39700 1 -- 172.21.15.83:0/880657976 --> [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3564062380 con 0x7f354c0ab330 2024-03-11T19:37:42.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.964+0000 7f3560ff9700 1 -- 172.21.15.83:0/880657976 <== osd.4 v2:172.21.15.137:6808/4100342703 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3564062380 con 0x7f354c0ab330 2024-03-11T19:37:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.965+0000 7f3569b39700 1 -- 172.21.15.83:0/880657976 >> [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703] conn(0x7f354c0ab330 msgr2=0x7f354c0ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.965+0000 7f3569b39700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703] conn(0x7f354c0ab330 0x7f354c0ad750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.965+0000 7f3569b39700 1 -- 172.21.15.83:0/880657976 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f354c0a1b50 msgr2=0x7f354c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.965+0000 7f3569b39700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f354c0a1b50 0x7f354c0a4010 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3554005950 tx=0x7f355400b410 comp rx=0 tx=0).stop 2024-03-11T19:37:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.965+0000 7f3569b39700 1 -- 172.21.15.83:0/880657976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3564105e90 msgr2=0x7f356419b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.965+0000 7f3569b39700 1 --2- 172.21.15.83:0/880657976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3564105e90 0x7f356419b9d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f355c00b970 tx=0x7f355c00bc80 comp rx=0 tx=0).stop 2024-03-11T19:37:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.965+0000 7f3563fff700 1 -- 172.21.15.83:0/880657976 reap_dead start 2024-03-11T19:37:42.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.965+0000 7f3569b39700 1 -- 172.21.15.83:0/880657976 shutdown_connections 2024-03-11T19:37:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.965+0000 7f3569b39700 1 -- 172.21.15.83:0/880657976 >> 172.21.15.83:0/880657976 conn(0x7f3564076b90 msgr2=0x7f3564100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.965+0000 7f3569b39700 1 -- 172.21.15.83:0/880657976 shutdown_connections 2024-03-11T19:37:42.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:42.965+0000 7f3569b39700 1 -- 172.21.15.83:0/880657976 wait complete. 2024-03-11T19:37:43.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.535+0000 7f3fe0671700 1 -- 172.21.15.83:0/504444804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd8100800 msgr2=0x7f3fd8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:43.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.535+0000 7f3fe0671700 1 --2- 172.21.15.83:0/504444804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd8100800 0x7f3fd8100be0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009990 tx=0x7f3fd4009ca0 comp rx=0 tx=0).stop 2024-03-11T19:37:43.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.535+0000 7f3fe0671700 1 -- 172.21.15.83:0/504444804 shutdown_connections 2024-03-11T19:37:43.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.535+0000 7f3fe0671700 1 --2- 172.21.15.83:0/504444804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fd8107020 0x7f3fd810eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.535+0000 7f3fe0671700 1 --2- 172.21.15.83:0/504444804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd8100800 0x7f3fd8100be0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.535+0000 7f3fe0671700 1 --2- 172.21.15.83:0/504444804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd81011b0 0x7f3fd81068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.535+0000 7f3fe0671700 1 -- 172.21.15.83:0/504444804 >> 172.21.15.83:0/504444804 conn(0x7f3fd8075ea0 msgr2=0x7f3fd80762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.535+0000 7f3fe0671700 1 -- 172.21.15.83:0/504444804 shutdown_connections 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.535+0000 7f3fe0671700 1 -- 172.21.15.83:0/504444804 wait complete. 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fe0671700 1 Processor -- start 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fe0671700 1 -- start start 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fe0671700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd8100800 0x7f3fd8101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fe0671700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81011b0 0x7f3fd8101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fe0671700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fd8107020 0x7f3fd8102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fe0671700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd81a6640 con 0x7f3fd81011b0 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fe0671700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd81a67b0 con 0x7f3fd8100800 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fe0671700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd81a6970 con 0x7f3fd8107020 2024-03-11T19:37:43.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fddc0c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81011b0 0x7f3fd8101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fddc0c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81011b0 0x7f3fd8101cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55508/0 (socket says 172.21.15.83:55508) 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fddc0c700 1 -- 172.21.15.83:0/166549987 learned_addr learned my addr 172.21.15.83:0/166549987 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.536+0000 7f3fdec0e700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fd8107020 0x7f3fd8102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fde40d700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd8100800 0x7f3fd8101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fddc0c700 1 -- 172.21.15.83:0/166549987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fd8107020 msgr2=0x7f3fd8102200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fddc0c700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fd8107020 0x7f3fd8102200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fddc0c700 1 -- 172.21.15.83:0/166549987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd8100800 msgr2=0x7f3fd8101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fddc0c700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fd8100800 0x7f3fd8101780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fddc0c700 1 -- 172.21.15.83:0/166549987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd40095f0 con 0x7f3fd81011b0 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fddc0c700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81011b0 0x7f3fd8101cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800b4d0 tx=0x7f3fc800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fcf7fe700 1 -- 172.21.15.83:0/166549987 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8011830 con 0x7f3fd81011b0 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fcf7fe700 1 -- 172.21.15.83:0/166549987 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fc8011e70 con 0x7f3fd81011b0 2024-03-11T19:37:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fe0671700 1 -- 172.21.15.83:0/166549987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd81a6c50 con 0x7f3fd81011b0 2024-03-11T19:37:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fcf7fe700 1 -- 172.21.15.83:0/166549987 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc800f2e0 con 0x7f3fd81011b0 2024-03-11T19:37:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.537+0000 7f3fe0671700 1 -- 172.21.15.83:0/166549987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd81a71a0 con 0x7f3fd81011b0 2024-03-11T19:37:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.538+0000 7f3fe0671700 1 -- 172.21.15.83:0/166549987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3fd8112020 con 0x7f3fd81011b0 2024-03-11T19:37:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.539+0000 7f3fcf7fe700 1 -- 172.21.15.83:0/166549987 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f3fc80105a0 con 0x7f3fd81011b0 2024-03-11T19:37:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.539+0000 7f3fcf7fe700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fc40a1bd0 0x7f3fc40a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.540+0000 7f3fde40d700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fc40a1bd0 0x7f3fc40a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.540+0000 7f3fcf7fe700 1 -- 172.21.15.83:0/166549987 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3fc80cd1b0 con 0x7f3fd81011b0 2024-03-11T19:37:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.540+0000 7f3fcf7fe700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907] conn(0x7f3fc40ab3b0 0x7f3fc40ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.540+0000 7f3fcf7fe700 1 -- 172.21.15.83:0/166549987 --> [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3fc40ade80 con 0x7f3fc40ab3b0 2024-03-11T19:37:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.540+0000 7f3fcf7fe700 1 -- 172.21.15.83:0/166549987 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc80cd650 con 0x7f3fd81011b0 2024-03-11T19:37:43.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.540+0000 7f3fdec0e700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907] conn(0x7f3fc40ab3b0 0x7f3fc40ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:43.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.540+0000 7f3fde40d700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fc40a1bd0 0x7f3fc40a4090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4000c00 tx=0x7f3fd4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:43.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.540+0000 7f3fdec0e700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907] conn(0x7f3fc40ab3b0 0x7f3fc40ad7d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:43.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.541+0000 7f3fcf7fe700 1 -- 172.21.15.83:0/166549987 <== osd.6 v2:172.21.15.83:6818/3426343907 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3fc40ade80 con 0x7f3fc40ab3b0 2024-03-11T19:37:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.557+0000 7f3fe0671700 1 -- 172.21.15.83:0/166549987 --> [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3fd8062330 con 0x7f3fc40ab3b0 2024-03-11T19:37:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.557+0000 7f3fcf7fe700 1 -- 172.21.15.83:0/166549987 <== osd.6 v2:172.21.15.83:6818/3426343907 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3fd8062330 con 0x7f3fc40ab3b0 2024-03-11T19:37:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.557+0000 7f3fe0671700 1 -- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907] conn(0x7f3fc40ab3b0 msgr2=0x7f3fc40ad7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.557+0000 7f3fe0671700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907] conn(0x7f3fc40ab3b0 0x7f3fc40ad7d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.557+0000 7f3fe0671700 1 -- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fc40a1bd0 msgr2=0x7f3fc40a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.557+0000 7f3fe0671700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fc40a1bd0 0x7f3fc40a4090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4000c00 tx=0x7f3fd4005fb0 comp rx=0 tx=0).stop 2024-03-11T19:37:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.557+0000 7f3fe0671700 1 -- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81011b0 msgr2=0x7f3fd8101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.557+0000 7f3fe0671700 1 --2- 172.21.15.83:0/166549987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81011b0 0x7f3fd8101cc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800b4d0 tx=0x7f3fc800b7e0 comp rx=0 tx=0).stop 2024-03-11T19:37:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.557+0000 7f3fdec0e700 1 -- 172.21.15.83:0/166549987 reap_dead start 2024-03-11T19:37:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.558+0000 7f3fe0671700 1 -- 172.21.15.83:0/166549987 shutdown_connections 2024-03-11T19:37:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.558+0000 7f3fe0671700 1 -- 172.21.15.83:0/166549987 >> 172.21.15.83:0/166549987 conn(0x7f3fd8075ea0 msgr2=0x7f3fd80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.558+0000 7f3fe0671700 1 -- 172.21.15.83:0/166549987 shutdown_connections 2024-03-11T19:37:43.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:43.558+0000 7f3fe0671700 1 -- 172.21.15.83:0/166549987 wait complete. 2024-03-11T19:37:43.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:43 smithi083 ceph-mon[31864]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:37:43.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:43 smithi137 ceph-mon[36220]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:37:43.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:43 smithi204 ceph-mon[36739]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:37:44.826 INFO:teuthology.orchestra.run.smithi083.stdout:141733920824 2024-03-11T19:37:44.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.358+0000 7fafcf021700 1 -- 172.21.15.83:0/2556911963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc8101530 msgr2=0x7fafc810d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.358+0000 7fafcf021700 1 --2- 172.21.15.83:0/2556911963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc8101530 0x7fafc810d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fafc000b0a0 tx=0x7fafc000b3b0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.358+0000 7fafcf021700 1 -- 172.21.15.83:0/2556911963 shutdown_connections 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.358+0000 7fafcf021700 1 --2- 172.21.15.83:0/2556911963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafc8106b80 0x7fafc8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.358+0000 7fafcf021700 1 --2- 172.21.15.83:0/2556911963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc8101530 0x7fafc810d9c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.358+0000 7fafcf021700 1 --2- 172.21.15.83:0/2556911963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc8100b70 0x7fafc8100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.358+0000 7fafcf021700 1 -- 172.21.15.83:0/2556911963 >> 172.21.15.83:0/2556911963 conn(0x7fafc8077aa0 msgr2=0x7fafc8077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.358+0000 7fafcf021700 1 -- 172.21.15.83:0/2556911963 shutdown_connections 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.358+0000 7fafcf021700 1 -- 172.21.15.83:0/2556911963 wait complete. 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafcf021700 1 Processor -- start 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafcf021700 1 -- start start 2024-03-11T19:37:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafcf021700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc8100b70 0x7fafc8102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafcf021700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafc8101530 0x7fafc8103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafcf021700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc8106b80 0x7fafc81035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafcf021700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc81a6790 con 0x7fafc8100b70 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafcf021700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc81a6900 con 0x7fafc8106b80 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafcf021700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc81a6ac0 con 0x7fafc8101530 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafccdbd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc8100b70 0x7fafc8102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafccdbd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc8100b70 0x7fafc8102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55520/0 (socket says 172.21.15.83:55520) 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafccdbd700 1 -- 172.21.15.83:0/773381695 learned_addr learned my addr 172.21.15.83:0/773381695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafc7fff700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafc8101530 0x7fafc8103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafcd5be700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc8106b80 0x7fafc81035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafccdbd700 1 -- 172.21.15.83:0/773381695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafc8101530 msgr2=0x7fafc8103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafccdbd700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafc8101530 0x7fafc8103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafccdbd700 1 -- 172.21.15.83:0/773381695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc8106b80 msgr2=0x7fafc81035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafccdbd700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafc8106b80 0x7fafc81035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.359+0000 7fafccdbd700 1 -- 172.21.15.83:0/773381695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0009d00 con 0x7fafc8100b70 2024-03-11T19:37:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.360+0000 7fafccdbd700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc8100b70 0x7fafc8102b20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00b530 tx=0x7fafbc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.360+0000 7fafc5ffb700 1 -- 172.21.15.83:0/773381695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc011830 con 0x7fafc8100b70 2024-03-11T19:37:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.360+0000 7fafcf021700 1 -- 172.21.15.83:0/773381695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc81a6da0 con 0x7fafc8100b70 2024-03-11T19:37:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.360+0000 7fafc5ffb700 1 -- 172.21.15.83:0/773381695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafbc011e70 con 0x7fafc8100b70 2024-03-11T19:37:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.360+0000 7fafc5ffb700 1 -- 172.21.15.83:0/773381695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00f2e0 con 0x7fafc8100b70 2024-03-11T19:37:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.360+0000 7fafcf021700 1 -- 172.21.15.83:0/773381695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc81a7260 con 0x7fafc8100b70 2024-03-11T19:37:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.361+0000 7fafcf021700 1 -- 172.21.15.83:0/773381695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fafc81021f0 con 0x7fafc8100b70 2024-03-11T19:37:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.362+0000 7fafc5ffb700 1 -- 172.21.15.83:0/773381695 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fafbc010610 con 0x7fafc8100b70 2024-03-11T19:37:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.362+0000 7fafc5ffb700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fafb80a1b50 0x7fafb80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.362+0000 7fafc7fff700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fafb80a1b50 0x7fafb80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.362+0000 7fafc5ffb700 1 -- 172.21.15.83:0/773381695 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fafbc0cd1c0 con 0x7fafc8100b70 2024-03-11T19:37:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.363+0000 7fafc5ffb700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155] conn(0x7fafb80ab330 0x7fafb80ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.363+0000 7fafc5ffb700 1 -- 172.21.15.83:0/773381695 --> [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafb80ade00 con 0x7fafb80ab330 2024-03-11T19:37:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.363+0000 7fafc5ffb700 1 -- 172.21.15.83:0/773381695 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafbc0cd660 con 0x7fafc8100b70 2024-03-11T19:37:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.363+0000 7fafcd5be700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155] conn(0x7fafb80ab330 0x7fafb80ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.363+0000 7fafc7fff700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fafb80a1b50 0x7fafb80a4010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fafc806aa00 tx=0x7fafb4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.363+0000 7fafcd5be700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155] conn(0x7fafb80ab330 0x7fafb80ad750 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.364+0000 7fafc5ffb700 1 -- 172.21.15.83:0/773381695 <== osd.3 v2:172.21.15.83:6810/780035155 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fafb80ade00 con 0x7fafb80ab330 2024-03-11T19:37:45.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.380+0000 7fafcf021700 1 -- 172.21.15.83:0/773381695 --> [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fafc8110f80 con 0x7fafb80ab330 2024-03-11T19:37:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafc5ffb700 1 -- 172.21.15.83:0/773381695 <== osd.3 v2:172.21.15.83:6810/780035155 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fafc8110f80 con 0x7fafb80ab330 2024-03-11T19:37:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafcf021700 1 -- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155] conn(0x7fafb80ab330 msgr2=0x7fafb80ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafcf021700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155] conn(0x7fafb80ab330 0x7fafb80ad750 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafcf021700 1 -- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fafb80a1b50 msgr2=0x7fafb80a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafcf021700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fafb80a1b50 0x7fafb80a4010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fafc806aa00 tx=0x7fafb4009190 comp rx=0 tx=0).stop 2024-03-11T19:37:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafcf021700 1 -- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc8100b70 msgr2=0x7fafc8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafcf021700 1 --2- 172.21.15.83:0/773381695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafc8100b70 0x7fafc8102b20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00b530 tx=0x7fafbc00b840 comp rx=0 tx=0).stop 2024-03-11T19:37:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafcd5be700 1 -- 172.21.15.83:0/773381695 reap_dead start 2024-03-11T19:37:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafcf021700 1 -- 172.21.15.83:0/773381695 shutdown_connections 2024-03-11T19:37:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafcf021700 1 -- 172.21.15.83:0/773381695 >> 172.21.15.83:0/773381695 conn(0x7fafc8077aa0 msgr2=0x7fafc8104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafcf021700 1 -- 172.21.15.83:0/773381695 shutdown_connections 2024-03-11T19:37:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.381+0000 7fafcf021700 1 -- 172.21.15.83:0/773381695 wait complete. 2024-03-11T19:37:45.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:45 smithi137 ceph-mon[36220]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:37:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:45 smithi204 ceph-mon[36739]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:37:45.923 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:45 smithi083 ceph-mon[31864]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-11T19:37:45.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.919+0000 7fe21547a700 1 -- 172.21.15.83:0/173034053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210100b70 msgr2=0x7fe210100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.919+0000 7fe21547a700 1 --2- 172.21.15.83:0/173034053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210100b70 0x7fe210100ff0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe204009990 tx=0x7fe204009ca0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.919+0000 7fe21547a700 1 -- 172.21.15.83:0/173034053 shutdown_connections 2024-03-11T19:37:45.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.919+0000 7fe21547a700 1 --2- 172.21.15.83:0/173034053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe210101530 0x7fe21010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.919+0000 7fe21547a700 1 --2- 172.21.15.83:0/173034053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe210106b80 0x7fe210106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.919+0000 7fe21547a700 1 --2- 172.21.15.83:0/173034053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210100b70 0x7fe210100ff0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.919+0000 7fe21547a700 1 -- 172.21.15.83:0/173034053 >> 172.21.15.83:0/173034053 conn(0x7fe210077aa0 msgr2=0x7fe210077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:45.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.919+0000 7fe21547a700 1 -- 172.21.15.83:0/173034053 shutdown_connections 2024-03-11T19:37:45.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.919+0000 7fe21547a700 1 -- 172.21.15.83:0/173034053 wait complete. 2024-03-11T19:37:45.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.920+0000 7fe21547a700 1 Processor -- start 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.920+0000 7fe21547a700 1 -- start start 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.920+0000 7fe21547a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe210100b70 0x7fe2101027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.920+0000 7fe21547a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210101530 0x7fe210102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.920+0000 7fe21547a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe210106b80 0x7fe210103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.920+0000 7fe21547a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101a64a0 con 0x7fe210101530 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.920+0000 7fe21547a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101a6610 con 0x7fe210106b80 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.920+0000 7fe21547a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101a67d0 con 0x7fe210100b70 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.920+0000 7fe20e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210101530 0x7fe210102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe20e7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210101530 0x7fe210102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55532/0 (socket says 172.21.15.83:55532) 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe20e7fc700 1 -- 172.21.15.83:0/1170181604 learned_addr learned my addr 172.21.15.83:0/1170181604 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe20effd700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe210100b70 0x7fe2101027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe20f7fe700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe210106b80 0x7fe210103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:45.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe20e7fc700 1 -- 172.21.15.83:0/1170181604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe210100b70 msgr2=0x7fe2101027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe20e7fc700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe210100b70 0x7fe2101027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe20e7fc700 1 -- 172.21.15.83:0/1170181604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe210106b80 msgr2=0x7fe210103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe20e7fc700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe210106b80 0x7fe210103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe20e7fc700 1 -- 172.21.15.83:0/1170181604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2040095f0 con 0x7fe210101530 2024-03-11T19:37:45.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe20effd700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe210100b70 0x7fe2101027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe20e7fc700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210101530 0x7fe210102cf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe2040098e0 tx=0x7fe20400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe1f7fff700 1 -- 172.21.15.83:0/1170181604 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20401d070 con 0x7fe210101530 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe21547a700 1 -- 172.21.15.83:0/1170181604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2101a6a50 con 0x7fe210101530 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe1f7fff700 1 -- 172.21.15.83:0/1170181604 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe204003680 con 0x7fe210101530 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe1f7fff700 1 -- 172.21.15.83:0/1170181604 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204017650 con 0x7fe210101530 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.921+0000 7fe21547a700 1 -- 172.21.15.83:0/1170181604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2101a6e60 con 0x7fe210101530 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.922+0000 7fe21547a700 1 -- 172.21.15.83:0/1170181604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe210110e50 con 0x7fe210101530 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.923+0000 7fe1f7fff700 1 -- 172.21.15.83:0/1170181604 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fe204022470 con 0x7fe210101530 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.924+0000 7fe1f7fff700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe1f80a1b50 0x7fe1f80a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.924+0000 7fe20effd700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe1f80a1b50 0x7fe1f80a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.924+0000 7fe1f7fff700 1 -- 172.21.15.83:0/1170181604 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe2040cec70 con 0x7fe210101530 2024-03-11T19:37:45.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.924+0000 7fe1f7fff700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517] conn(0x7fe1f80ab330 0x7fe1f80ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.924+0000 7fe1f7fff700 1 -- 172.21.15.83:0/1170181604 --> [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1f80ade00 con 0x7fe1f80ab330 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.924+0000 7fe1f7fff700 1 -- 172.21.15.83:0/1170181604 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2040cf110 con 0x7fe210101530 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.924+0000 7fe20effd700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe1f80a1b50 0x7fe1f80a4010 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe200005e10 tx=0x7fe200005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.924+0000 7fe20f7fe700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517] conn(0x7fe1f80ab330 0x7fe1f80ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.925+0000 7fe20f7fe700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517] conn(0x7fe1f80ab330 0x7fe1f80ad750 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.926+0000 7fe1f7fff700 1 -- 172.21.15.83:0/1170181604 <== osd.5 v2:172.21.15.204:6808/2553953517 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe1f80ade00 con 0x7fe1f80ab330 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.941+0000 7fe21547a700 1 -- 172.21.15.83:0/1170181604 --> [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe210069c30 con 0x7fe1f80ab330 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.942+0000 7fe1f7fff700 1 -- 172.21.15.83:0/1170181604 <== osd.5 v2:172.21.15.204:6808/2553953517 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe210069c30 con 0x7fe1f80ab330 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.942+0000 7fe21547a700 1 -- 172.21.15.83:0/1170181604 >> [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517] conn(0x7fe1f80ab330 msgr2=0x7fe1f80ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.942+0000 7fe21547a700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517] conn(0x7fe1f80ab330 0x7fe1f80ad750 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.943+0000 7fe21547a700 1 -- 172.21.15.83:0/1170181604 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe1f80a1b50 msgr2=0x7fe1f80a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.943+0000 7fe21547a700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe1f80a1b50 0x7fe1f80a4010 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe200005e10 tx=0x7fe200005d60 comp rx=0 tx=0).stop 2024-03-11T19:37:45.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.943+0000 7fe21547a700 1 -- 172.21.15.83:0/1170181604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210101530 msgr2=0x7fe210102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:45.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.943+0000 7fe21547a700 1 --2- 172.21.15.83:0/1170181604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210101530 0x7fe210102cf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe2040098e0 tx=0x7fe20400f690 comp rx=0 tx=0).stop 2024-03-11T19:37:45.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.943+0000 7fe20f7fe700 1 -- 172.21.15.83:0/1170181604 reap_dead start 2024-03-11T19:37:45.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.943+0000 7fe21547a700 1 -- 172.21.15.83:0/1170181604 shutdown_connections 2024-03-11T19:37:45.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.943+0000 7fe21547a700 1 -- 172.21.15.83:0/1170181604 >> 172.21.15.83:0/1170181604 conn(0x7fe210077aa0 msgr2=0x7fe2100ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:45.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.943+0000 7fe21547a700 1 -- 172.21.15.83:0/1170181604 shutdown_connections 2024-03-11T19:37:45.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:45.943+0000 7fe21547a700 1 -- 172.21.15.83:0/1170181604 wait complete. 2024-03-11T19:37:46.786 INFO:teuthology.orchestra.run.smithi083.stdout:287762808862 2024-03-11T19:37:46.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-11T19:37:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.005+0000 7eff6a184700 1 -- 172.21.15.83:0/3162471063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64100e50 msgr2=0x7eff641012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.005+0000 7eff6a184700 1 --2- 172.21.15.83:0/3162471063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64100e50 0x7eff641012d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7eff580099c0 tx=0x7eff58009cd0 comp rx=0 tx=0).stop 2024-03-11T19:37:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.005+0000 7eff6a184700 1 -- 172.21.15.83:0/3162471063 shutdown_connections 2024-03-11T19:37:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.005+0000 7eff6a184700 1 --2- 172.21.15.83:0/3162471063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff64101810 0x7eff6410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.005+0000 7eff6a184700 1 --2- 172.21.15.83:0/3162471063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64106ca0 0x7eff64107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.005+0000 7eff6a184700 1 --2- 172.21.15.83:0/3162471063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64100e50 0x7eff641012d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.005+0000 7eff6a184700 1 -- 172.21.15.83:0/3162471063 >> 172.21.15.83:0/3162471063 conn(0x7eff64077a30 msgr2=0x7eff64077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:47.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.006+0000 7eff6a184700 1 -- 172.21.15.83:0/3162471063 shutdown_connections 2024-03-11T19:37:47.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.006+0000 7eff6a184700 1 -- 172.21.15.83:0/3162471063 wait complete. 2024-03-11T19:37:47.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.006+0000 7eff6a184700 1 Processor -- start 2024-03-11T19:37:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.006+0000 7eff6a184700 1 -- start start 2024-03-11T19:37:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff6a184700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff64100e50 0x7eff641971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff6a184700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64101810 0x7eff64197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff6a184700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64106ca0 0x7eff6419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff6a184700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6419c080 con 0x7eff64101810 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff6a184700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6419c1f0 con 0x7eff64106ca0 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff6a184700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6419c3b0 con 0x7eff64100e50 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff62ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64101810 0x7eff64197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff62ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64101810 0x7eff64197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55564/0 (socket says 172.21.15.83:55564) 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff62ffd700 1 -- 172.21.15.83:0/432389327 learned_addr learned my addr 172.21.15.83:0/432389327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff637fe700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff64100e50 0x7eff641971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff63fff700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64106ca0 0x7eff6419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff62ffd700 1 -- 172.21.15.83:0/432389327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff64100e50 msgr2=0x7eff641971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff62ffd700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff64100e50 0x7eff641971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff62ffd700 1 -- 172.21.15.83:0/432389327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64106ca0 msgr2=0x7eff6419ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:47.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff62ffd700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64106ca0 0x7eff6419ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:47.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff62ffd700 1 -- 172.21.15.83:0/432389327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff580095f0 con 0x7eff64101810 2024-03-11T19:37:47.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.007+0000 7eff637fe700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff64100e50 0x7eff641971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:47.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.008+0000 7eff62ffd700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64101810 0x7eff64197730 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7eff58005910 tx=0x7eff58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:47.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.008+0000 7eff60ff9700 1 -- 172.21.15.83:0/432389327 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5801f070 con 0x7eff64101810 2024-03-11T19:37:47.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.008+0000 7eff60ff9700 1 -- 172.21.15.83:0/432389327 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff5801ace0 con 0x7eff64101810 2024-03-11T19:37:47.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.008+0000 7eff6a184700 1 -- 172.21.15.83:0/432389327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6419c630 con 0x7eff64101810 2024-03-11T19:37:47.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.008+0000 7eff6a184700 1 -- 172.21.15.83:0/432389327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6406a4e0 con 0x7eff64101810 2024-03-11T19:37:47.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.008+0000 7eff60ff9700 1 -- 172.21.15.83:0/432389327 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5800f460 con 0x7eff64101810 2024-03-11T19:37:47.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.009+0000 7eff6a184700 1 -- 172.21.15.83:0/432389327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff50001030 con 0x7eff64101810 2024-03-11T19:37:47.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.011+0000 7eff60ff9700 1 -- 172.21.15.83:0/432389327 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7eff58023070 con 0x7eff64101810 2024-03-11T19:37:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.011+0000 7eff60ff9700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff4c0a1c10 0x7eff4c0a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.011+0000 7eff637fe700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff4c0a1c10 0x7eff4c0a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.011+0000 7eff60ff9700 1 -- 172.21.15.83:0/432389327 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7eff580d0b60 con 0x7eff64101810 2024-03-11T19:37:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.011+0000 7eff60ff9700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508] conn(0x7eff4c0ab3f0 0x7eff4c0ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.011+0000 7eff60ff9700 1 -- 172.21.15.83:0/432389327 --> [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff4c0adec0 con 0x7eff4c0ab3f0 2024-03-11T19:37:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.011+0000 7eff60ff9700 1 -- 172.21.15.83:0/432389327 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff580d1000 con 0x7eff64101810 2024-03-11T19:37:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.012+0000 7eff637fe700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff4c0a1c10 0x7eff4c0a40d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff540099d0 tx=0x7eff54008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.012+0000 7eff63fff700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508] conn(0x7eff4c0ab3f0 0x7eff4c0ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.012+0000 7eff63fff700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508] conn(0x7eff4c0ab3f0 0x7eff4c0ad810 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:47.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.013+0000 7eff60ff9700 1 -- 172.21.15.83:0/432389327 <== osd.8 v2:172.21.15.204:6816/3550974508 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7eff4c0adec0 con 0x7eff4c0ab3f0 2024-03-11T19:37:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.035+0000 7eff6a184700 1 -- 172.21.15.83:0/432389327 --> [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7eff50002dc0 con 0x7eff4c0ab3f0 2024-03-11T19:37:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.035+0000 7eff60ff9700 1 -- 172.21.15.83:0/432389327 <== osd.8 v2:172.21.15.204:6816/3550974508 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7eff50002dc0 con 0x7eff4c0ab3f0 2024-03-11T19:37:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.036+0000 7eff6a184700 1 -- 172.21.15.83:0/432389327 >> [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508] conn(0x7eff4c0ab3f0 msgr2=0x7eff4c0ad810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.036+0000 7eff6a184700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508] conn(0x7eff4c0ab3f0 0x7eff4c0ad810 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.036+0000 7eff6a184700 1 -- 172.21.15.83:0/432389327 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff4c0a1c10 msgr2=0x7eff4c0a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.036+0000 7eff6a184700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff4c0a1c10 0x7eff4c0a40d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff540099d0 tx=0x7eff54008040 comp rx=0 tx=0).stop 2024-03-11T19:37:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.036+0000 7eff6a184700 1 -- 172.21.15.83:0/432389327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64101810 msgr2=0x7eff64197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.036+0000 7eff6a184700 1 --2- 172.21.15.83:0/432389327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64101810 0x7eff64197730 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7eff58005910 tx=0x7eff58004730 comp rx=0 tx=0).stop 2024-03-11T19:37:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.036+0000 7eff63fff700 1 -- 172.21.15.83:0/432389327 reap_dead start 2024-03-11T19:37:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.036+0000 7eff6a184700 1 -- 172.21.15.83:0/432389327 shutdown_connections 2024-03-11T19:37:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.036+0000 7eff6a184700 1 -- 172.21.15.83:0/432389327 >> 172.21.15.83:0/432389327 conn(0x7eff64077a30 msgr2=0x7eff640ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.036+0000 7eff6a184700 1 -- 172.21.15.83:0/432389327 shutdown_connections 2024-03-11T19:37:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:47.036+0000 7eff6a184700 1 -- 172.21.15.83:0/432389327 wait complete. 2024-03-11T19:37:47.687 INFO:teuthology.orchestra.run.smithi083.stdout:253403070500 2024-03-11T19:37:47.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-11T19:37:47.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:47 smithi083 ceph-mon[31864]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:37:47.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:47 smithi137 ceph-mon[36220]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:37:47.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:47 smithi204 ceph-mon[36739]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-11T19:37:47.939 INFO:teuthology.orchestra.run.smithi083.stdout:107374182461 2024-03-11T19:37:47.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-11T19:37:48.461 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:48.461 INFO:teuthology.orchestra.run.smithi083.stdout:163208757299 2024-03-11T19:37:48.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-11T19:37:48.917 INFO:teuthology.orchestra.run.smithi083.stdout:197568495662 2024-03-11T19:37:48.917 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-11T19:37:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:49.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:49 smithi083 ceph-mon[31864]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-11T19:37:49.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:49 smithi137 ceph-mon[36220]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-11T19:37:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:49 smithi204 ceph-mon[36739]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-11T19:37:50.100 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:50.107 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:50.151 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:50.157 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:50.975 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:50 smithi083 ceph-mon[31864]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:37:51.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:50 smithi137 ceph-mon[36220]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:37:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:50 smithi204 ceph-mon[36739]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:37:53.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:53 smithi083 ceph-mon[31864]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:37:53.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:53 smithi137 ceph-mon[36220]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:37:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:53 smithi204 ceph-mon[36739]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:37:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.598+0000 7fbbf7746700 1 -- 172.21.15.83:0/1867568483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0101f80 msgr2=0x7fbbf0105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.598+0000 7fbbf7746700 1 --2- 172.21.15.83:0/1867568483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0101f80 0x7fbbf0105810 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009990 tx=0x7fbbe0009ca0 comp rx=0 tx=0).stop 2024-03-11T19:37:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.598+0000 7fbbf7746700 1 -- 172.21.15.83:0/1867568483 shutdown_connections 2024-03-11T19:37:55.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.598+0000 7fbbf7746700 1 --2- 172.21.15.83:0/1867568483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf0105ef0 0x7fbbf010dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.598+0000 7fbbf7746700 1 --2- 172.21.15.83:0/1867568483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbf01015d0 0x7fbbf01019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.598+0000 7fbbf7746700 1 --2- 172.21.15.83:0/1867568483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0101f80 0x7fbbf0105810 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.598+0000 7fbbf7746700 1 -- 172.21.15.83:0/1867568483 >> 172.21.15.83:0/1867568483 conn(0x7fbbf0077170 msgr2=0x7fbbf00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.598+0000 7fbbf7746700 1 -- 172.21.15.83:0/1867568483 shutdown_connections 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.598+0000 7fbbf7746700 1 -- 172.21.15.83:0/1867568483 wait complete. 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf7746700 1 Processor -- start 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf7746700 1 -- start start 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf7746700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbf01015d0 0x7fbbf019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf7746700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf0101f80 0x7fbbf019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf7746700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0105ef0 0x7fbbf019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf7746700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf019f2f0 con 0x7fbbf0105ef0 2024-03-11T19:37:55.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf7746700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf019f460 con 0x7fbbf01015d0 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf7746700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf019f620 con 0x7fbbf0101f80 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf5ce3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0105ef0 0x7fbbf019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf5ce3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0105ef0 0x7fbbf019ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53912/0 (socket says 172.21.15.83:53912) 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf5ce3700 1 -- 172.21.15.83:0/2114035743 learned_addr learned my addr 172.21.15.83:0/2114035743 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf4ce1700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf0101f80 0x7fbbf019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf54e2700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbf01015d0 0x7fbbf019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf5ce3700 1 -- 172.21.15.83:0/2114035743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf0101f80 msgr2=0x7fbbf019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf5ce3700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf0101f80 0x7fbbf019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf5ce3700 1 -- 172.21.15.83:0/2114035743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbf01015d0 msgr2=0x7fbbf019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf5ce3700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbf01015d0 0x7fbbf019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf5ce3700 1 -- 172.21.15.83:0/2114035743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00095f0 con 0x7fbbf0105ef0 2024-03-11T19:37:55.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf54e2700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbbf01015d0 0x7fbbf019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf4ce1700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbf0101f80 0x7fbbf019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbf5ce3700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0105ef0 0x7fbbf019ece0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800ada0 tx=0x7fbbe800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.599+0000 7fbbe67fc700 1 -- 172.21.15.83:0/2114035743 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe800f070 con 0x7fbbf0105ef0 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.600+0000 7fbbe67fc700 1 -- 172.21.15.83:0/2114035743 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbe8007b30 con 0x7fbbf0105ef0 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.600+0000 7fbbf7746700 1 -- 172.21.15.83:0/2114035743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf019f900 con 0x7fbbf0105ef0 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.600+0000 7fbbe67fc700 1 -- 172.21.15.83:0/2114035743 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8005500 con 0x7fbbf0105ef0 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.600+0000 7fbbf7746700 1 -- 172.21.15.83:0/2114035743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf01a7160 con 0x7fbbf0105ef0 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.600+0000 7fbbf7746700 1 -- 172.21.15.83:0/2114035743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbbf0110eb0 con 0x7fbbf0105ef0 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.602+0000 7fbbe67fc700 1 -- 172.21.15.83:0/2114035743 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fbbe8004ad0 con 0x7fbbf0105ef0 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.602+0000 7fbbe67fc700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbbdc0a1b50 0x7fbbdc0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.602+0000 7fbbf54e2700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbbdc0a1b50 0x7fbbdc0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.602+0000 7fbbe67fc700 1 -- 172.21.15.83:0/2114035743 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbbe80cd470 con 0x7fbbf0105ef0 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.602+0000 7fbbe67fc700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017] conn(0x7fbbdc0ab330 0x7fbbdc0ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:55.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.602+0000 7fbbe67fc700 1 -- 172.21.15.83:0/2114035743 --> [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbdc0ade00 con 0x7fbbdc0ab330 2024-03-11T19:37:55.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.602+0000 7fbbe67fc700 1 -- 172.21.15.83:0/2114035743 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbe80cd910 con 0x7fbbf0105ef0 2024-03-11T19:37:55.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.603+0000 7fbbf54e2700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbbdc0a1b50 0x7fbbdc0a4010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009be0 tx=0x7fbbec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:55.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.603+0000 7fbbf4ce1700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017] conn(0x7fbbdc0ab330 0x7fbbdc0ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:55.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.603+0000 7fbbf4ce1700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017] conn(0x7fbbdc0ab330 0x7fbbdc0ad750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:55.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.604+0000 7fbbe67fc700 1 -- 172.21.15.83:0/2114035743 <== osd.7 v2:172.21.15.137:6816/2113619017 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbbdc0ade00 con 0x7fbbdc0ab330 2024-03-11T19:37:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.622+0000 7fbbf7746700 1 -- 172.21.15.83:0/2114035743 --> [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbbf0062380 con 0x7fbbdc0ab330 2024-03-11T19:37:55.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.622+0000 7fbbe67fc700 1 -- 172.21.15.83:0/2114035743 <== osd.7 v2:172.21.15.137:6816/2113619017 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbbf0062380 con 0x7fbbdc0ab330 2024-03-11T19:37:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.622+0000 7fbbf7746700 1 -- 172.21.15.83:0/2114035743 >> [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017] conn(0x7fbbdc0ab330 msgr2=0x7fbbdc0ad750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.622+0000 7fbbf7746700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017] conn(0x7fbbdc0ab330 0x7fbbdc0ad750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.622+0000 7fbbf7746700 1 -- 172.21.15.83:0/2114035743 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbbdc0a1b50 msgr2=0x7fbbdc0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.622+0000 7fbbf7746700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbbdc0a1b50 0x7fbbdc0a4010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009be0 tx=0x7fbbec009260 comp rx=0 tx=0).stop 2024-03-11T19:37:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.622+0000 7fbbf7746700 1 -- 172.21.15.83:0/2114035743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0105ef0 msgr2=0x7fbbf019ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.622+0000 7fbbf7746700 1 --2- 172.21.15.83:0/2114035743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbf0105ef0 0x7fbbf019ece0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800ada0 tx=0x7fbbe800c330 comp rx=0 tx=0).stop 2024-03-11T19:37:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.622+0000 7fbbf5ce3700 1 -- 172.21.15.83:0/2114035743 reap_dead start 2024-03-11T19:37:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.623+0000 7fbbf7746700 1 -- 172.21.15.83:0/2114035743 shutdown_connections 2024-03-11T19:37:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.623+0000 7fbbf7746700 1 -- 172.21.15.83:0/2114035743 >> 172.21.15.83:0/2114035743 conn(0x7fbbf0077170 msgr2=0x7fbbf0100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.623+0000 7fbbf7746700 1 -- 172.21.15.83:0/2114035743 shutdown_connections 2024-03-11T19:37:55.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.623+0000 7fbbf7746700 1 -- 172.21.15.83:0/2114035743 wait complete. 2024-03-11T19:37:55.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:55 smithi137 ceph-mon[36220]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:37:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:55 smithi204 ceph-mon[36739]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:37:55.985 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:55 smithi083 ceph-mon[31864]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:37:56.885 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:56 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3858993115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-11T19:37:56.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:56 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3858993115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-11T19:37:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:56 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3858993115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-11T19:37:57.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.398+0000 7fcf31e33700 1 -- 172.21.15.83:0/3802573576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2c101530 msgr2=0x7fcf2c10d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:57.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.398+0000 7fcf31e33700 1 --2- 172.21.15.83:0/3802573576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2c101530 0x7fcf2c10d9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400b0a0 tx=0x7fcf2400b3b0 comp rx=0 tx=0).stop 2024-03-11T19:37:57.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.398+0000 7fcf31e33700 1 -- 172.21.15.83:0/3802573576 shutdown_connections 2024-03-11T19:37:57.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.398+0000 7fcf31e33700 1 --2- 172.21.15.83:0/3802573576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2c101530 0x7fcf2c10d9c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:57.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.398+0000 7fcf31e33700 1 --2- 172.21.15.83:0/3802573576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf2c106b80 0x7fcf2c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:57.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.398+0000 7fcf31e33700 1 --2- 172.21.15.83:0/3802573576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf2c100b70 0x7fcf2c100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:57.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.398+0000 7fcf31e33700 1 -- 172.21.15.83:0/3802573576 >> 172.21.15.83:0/3802573576 conn(0x7fcf2c077aa0 msgr2=0x7fcf2c077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:57.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.398+0000 7fcf31e33700 1 -- 172.21.15.83:0/3802573576 shutdown_connections 2024-03-11T19:37:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.398+0000 7fcf31e33700 1 -- 172.21.15.83:0/3802573576 wait complete. 2024-03-11T19:37:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf31e33700 1 Processor -- start 2024-03-11T19:37:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf31e33700 1 -- start start 2024-03-11T19:37:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf31e33700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf2c100b70 0x7fcf2c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf31e33700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf2c101530 0x7fcf2c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf31e33700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2c106b80 0x7fcf2c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf31e33700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c19c1a0 con 0x7fcf2c106b80 2024-03-11T19:37:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf31e33700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c19c310 con 0x7fcf2c100b70 2024-03-11T19:37:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf31e33700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c19c4d0 con 0x7fcf2c101530 2024-03-11T19:37:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf2bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2c106b80 0x7fcf2c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf2bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2c106b80 0x7fcf2c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53904/0 (socket says 172.21.15.83:53904) 2024-03-11T19:37:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf2bfff700 1 -- 172.21.15.83:0/3858993115 learned_addr learned my addr 172.21.15.83:0/3858993115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf2b7fe700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf2c100b70 0x7fcf2c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf2bfff700 1 -- 172.21.15.83:0/3858993115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf2c101530 msgr2=0x7fcf2c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.399+0000 7fcf2affd700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf2c101530 0x7fcf2c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.400+0000 7fcf2bfff700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf2c101530 0x7fcf2c197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.400+0000 7fcf2bfff700 1 -- 172.21.15.83:0/3858993115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf2c100b70 msgr2=0x7fcf2c1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:57.222 INFO:teuthology.orchestra.run.smithi083.stdout:163208757300 2024-03-11T19:37:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.400+0000 7fcf2bfff700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf2c100b70 0x7fcf2c1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.400+0000 7fcf2bfff700 1 -- 172.21.15.83:0/3858993115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf24009d00 con 0x7fcf2c106b80 2024-03-11T19:37:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.400+0000 7fcf2b7fe700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf2c100b70 0x7fcf2c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.400+0000 7fcf2bfff700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2c106b80 0x7fcf2c19bb90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400b830 tx=0x7fcf240075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.400+0000 7fcf28ff9700 1 -- 172.21.15.83:0/3858993115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2400e040 con 0x7fcf2c106b80 2024-03-11T19:37:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.400+0000 7fcf28ff9700 1 -- 172.21.15.83:0/3858993115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf2401bb30 con 0x7fcf2c106b80 2024-03-11T19:37:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.400+0000 7fcf31e33700 1 -- 172.21.15.83:0/3858993115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c19c750 con 0x7fcf2c106b80 2024-03-11T19:37:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.400+0000 7fcf28ff9700 1 -- 172.21.15.83:0/3858993115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf24005920 con 0x7fcf2c106b80 2024-03-11T19:37:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.400+0000 7fcf31e33700 1 -- 172.21.15.83:0/3858993115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2c1a4010 con 0x7fcf2c106b80 2024-03-11T19:37:57.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.401+0000 7fcf31e33700 1 -- 172.21.15.83:0/3858993115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2c109020 con 0x7fcf2c106b80 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.402+0000 7fcf28ff9700 1 -- 172.21.15.83:0/3858993115 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fcf2401bca0 con 0x7fcf2c106b80 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.402+0000 7fcf28ff9700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcf140a1b50 0x7fcf140a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.402+0000 7fcf2b7fe700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcf140a1b50 0x7fcf140a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.403+0000 7fcf28ff9700 1 -- 172.21.15.83:0/3858993115 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcf240cf0b0 con 0x7fcf2c106b80 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.403+0000 7fcf2b7fe700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcf140a1b50 0x7fcf140a4010 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0099d0 tx=0x7fcf1c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.407+0000 7fcf28ff9700 1 -- 172.21.15.83:0/3858993115 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf240983a0 con 0x7fcf2c106b80 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.615+0000 7fcf31e33700 1 -- 172.21.15.83:0/3858993115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fcf2c198620 con 0x7fcf2c106b80 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.615+0000 7fcf28ff9700 1 -- 172.21.15.83:0/3858993115 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf240a2070 con 0x7fcf2c106b80 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.617+0000 7fcf31e33700 1 -- 172.21.15.83:0/3858993115 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcf140a1b50 msgr2=0x7fcf140a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.617+0000 7fcf31e33700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcf140a1b50 0x7fcf140a4010 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0099d0 tx=0x7fcf1c008040 comp rx=0 tx=0).stop 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.617+0000 7fcf31e33700 1 -- 172.21.15.83:0/3858993115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2c106b80 msgr2=0x7fcf2c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.617+0000 7fcf31e33700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2c106b80 0x7fcf2c19bb90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400b830 tx=0x7fcf240075c0 comp rx=0 tx=0).stop 2024-03-11T19:37:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.618+0000 7fcf31e33700 1 -- 172.21.15.83:0/3858993115 shutdown_connections 2024-03-11T19:37:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.618+0000 7fcf31e33700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcf140a1b50 0x7fcf140a4010 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.618+0000 7fcf31e33700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2c106b80 0x7fcf2c19bb90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.618+0000 7fcf31e33700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf2c100b70 0x7fcf2c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.618+0000 7fcf31e33700 1 --2- 172.21.15.83:0/3858993115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf2c101530 0x7fcf2c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.618+0000 7fcf31e33700 1 -- 172.21.15.83:0/3858993115 >> 172.21.15.83:0/3858993115 conn(0x7fcf2c077aa0 msgr2=0x7fcf2c0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.618+0000 7fcf31e33700 1 -- 172.21.15.83:0/3858993115 shutdown_connections 2024-03-11T19:37:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:55.618+0000 7fcf31e33700 1 -- 172.21.15.83:0/3858993115 wait complete. 2024-03-11T19:37:57.854 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:57 smithi083 ceph-mon[31864]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-11T19:37:57.855 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757300 for osd.5 2024-03-11T19:37:57.856 DEBUG:teuthology.parallel:result is None 2024-03-11T19:37:57.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:57 smithi137 ceph-mon[36220]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-11T19:37:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:57 smithi204 ceph-mon[36739]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-11T19:37:58.872 INFO:teuthology.orchestra.run.smithi083.stdout:219043332137 2024-03-11T19:37:58.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-11T19:37:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.049+0000 7fc346a67700 1 -- 172.21.15.83:0/1995941950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc340101cc0 msgr2=0x7fc340102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.049+0000 7fc346a67700 1 --2- 172.21.15.83:0/1995941950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc340101cc0 0x7fc340102140 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0099c0 tx=0x7fc32c009cd0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.049+0000 7fc346a67700 1 -- 172.21.15.83:0/1995941950 shutdown_connections 2024-03-11T19:37:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.049+0000 7fc346a67700 1 --2- 172.21.15.83:0/1995941950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc340107ca0 0x7fc340108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.049+0000 7fc346a67700 1 --2- 172.21.15.83:0/1995941950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc340102680 0x7fc34010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.049+0000 7fc346a67700 1 --2- 172.21.15.83:0/1995941950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc340101cc0 0x7fc340102140 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.049+0000 7fc346a67700 1 -- 172.21.15.83:0/1995941950 >> 172.21.15.83:0/1995941950 conn(0x7fc340078da0 msgr2=0x7fc3400791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.049+0000 7fc346a67700 1 -- 172.21.15.83:0/1995941950 shutdown_connections 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.049+0000 7fc346a67700 1 -- 172.21.15.83:0/1995941950 wait complete. 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc346a67700 1 Processor -- start 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc346a67700 1 -- start start 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc346a67700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc340101cc0 0x7fc340073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc346a67700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc340102680 0x7fc3400744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc346a67700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc340107ca0 0x7fc3400749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc346a67700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3401a6400 con 0x7fc340102680 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc346a67700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3401a6570 con 0x7fc340101cc0 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc346a67700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3401a6730 con 0x7fc340107ca0 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc33ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc340102680 0x7fc3400744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc33ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc340102680 0x7fc3400744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53926/0 (socket says 172.21.15.83:53926) 2024-03-11T19:37:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc33ffff700 1 -- 172.21.15.83:0/4177965814 learned_addr learned my addr 172.21.15.83:0/4177965814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc344803700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc340101cc0 0x7fc340073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc345004700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc340107ca0 0x7fc3400749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc33ffff700 1 -- 172.21.15.83:0/4177965814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc340107ca0 msgr2=0x7fc3400749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc33ffff700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc340107ca0 0x7fc3400749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc33ffff700 1 -- 172.21.15.83:0/4177965814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc340101cc0 msgr2=0x7fc340073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc33ffff700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc340101cc0 0x7fc340073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc33ffff700 1 -- 172.21.15.83:0/4177965814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c0095f0 con 0x7fc340102680 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.050+0000 7fc344803700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc340101cc0 0x7fc340073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.051+0000 7fc345004700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc340107ca0 0x7fc3400749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.051+0000 7fc33ffff700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc340102680 0x7fc3400744b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0098e0 tx=0x7fc32c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.051+0000 7fc33dffb700 1 -- 172.21.15.83:0/4177965814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c01d070 con 0x7fc340102680 2024-03-11T19:37:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.051+0000 7fc346a67700 1 -- 172.21.15.83:0/4177965814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3401a69b0 con 0x7fc340102680 2024-03-11T19:37:59.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.051+0000 7fc33dffb700 1 -- 172.21.15.83:0/4177965814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc32c004050 con 0x7fc340102680 2024-03-11T19:37:59.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.051+0000 7fc33dffb700 1 -- 172.21.15.83:0/4177965814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00f460 con 0x7fc340102680 2024-03-11T19:37:59.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.051+0000 7fc346a67700 1 -- 172.21.15.83:0/4177965814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3401a6d10 con 0x7fc340102680 2024-03-11T19:37:59.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.052+0000 7fc346a67700 1 -- 172.21.15.83:0/4177965814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc340068930 con 0x7fc340102680 2024-03-11T19:37:59.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.053+0000 7fc33dffb700 1 -- 172.21.15.83:0/4177965814 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fc32c022600 con 0x7fc340102680 2024-03-11T19:37:59.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.054+0000 7fc33dffb700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc3300a1b50 0x7fc3300a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:37:59.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.054+0000 7fc33dffb700 1 -- 172.21.15.83:0/4177965814 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc32c0ce930 con 0x7fc340102680 2024-03-11T19:37:59.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.054+0000 7fc344803700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc3300a1b50 0x7fc3300a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:37:59.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.054+0000 7fc344803700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc3300a1b50 0x7fc3300a4010 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc334005fd0 tx=0x7fc33400a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:37:59.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.059+0000 7fc33dffb700 1 -- 172.21.15.83:0/4177965814 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc32c05dd20 con 0x7fc340102680 2024-03-11T19:37:59.094 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:37:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.272+0000 7fc346a67700 1 -- 172.21.15.83:0/4177965814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc340111d50 con 0x7fc340102680 2024-03-11T19:37:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.273+0000 7fc33dffb700 1 -- 172.21.15.83:0/4177965814 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc32c027070 con 0x7fc340102680 2024-03-11T19:37:59.278 INFO:teuthology.orchestra.run.smithi083.stdout:141733920827 2024-03-11T19:37:59.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 -- 172.21.15.83:0/4177965814 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc3300a1b50 msgr2=0x7fc3300a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:59.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc3300a1b50 0x7fc3300a4010 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc334005fd0 tx=0x7fc33400a5e0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 -- 172.21.15.83:0/4177965814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc340102680 msgr2=0x7fc3400744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:37:59.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc340102680 0x7fc3400744b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0098e0 tx=0x7fc32c0057f0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 -- 172.21.15.83:0/4177965814 shutdown_connections 2024-03-11T19:37:59.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc3300a1b50 0x7fc3300a4010 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc340107ca0 0x7fc3400749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc340101cc0 0x7fc340073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 --2- 172.21.15.83:0/4177965814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc340102680 0x7fc3400744b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:37:59.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 -- 172.21.15.83:0/4177965814 >> 172.21.15.83:0/4177965814 conn(0x7fc340078da0 msgr2=0x7fc340105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:37:59.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 -- 172.21.15.83:0/4177965814 shutdown_connections 2024-03-11T19:37:59.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:37:59.275+0000 7fc346a67700 1 -- 172.21.15.83:0/4177965814 wait complete. 2024-03-11T19:37:59.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:59 smithi137 ceph-mon[36220]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:37:59.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:37:59 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4177965814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-11T19:37:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:59 smithi204 ceph-mon[36739]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:37:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:37:59 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4177965814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-11T19:38:00.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:59 smithi083 ceph-mon[31864]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-11T19:38:00.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:37:59 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4177965814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-11T19:38:01.868 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:01 smithi083 ceph-mon[31864]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:38:01.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:01 smithi137 ceph-mon[36220]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:38:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:01 smithi204 ceph-mon[36739]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:38:02.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.465+0000 7f0c62bf1700 1 -- 172.21.15.83:0/4170281413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c101f80 msgr2=0x7f0c5c105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:02.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.465+0000 7f0c62bf1700 1 --2- 172.21.15.83:0/4170281413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c101f80 0x7f0c5c105810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009990 tx=0x7f0c48009ca0 comp rx=0 tx=0).stop 2024-03-11T19:38:02.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.465+0000 7f0c62bf1700 1 -- 172.21.15.83:0/4170281413 shutdown_connections 2024-03-11T19:38:02.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.465+0000 7f0c62bf1700 1 --2- 172.21.15.83:0/4170281413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c5c1015d0 0x7f0c5c1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:02.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.465+0000 7f0c62bf1700 1 --2- 172.21.15.83:0/4170281413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5c105ef0 0x7f0c5c10dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:02.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.465+0000 7f0c62bf1700 1 --2- 172.21.15.83:0/4170281413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c101f80 0x7f0c5c105810 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:02.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.465+0000 7f0c62bf1700 1 -- 172.21.15.83:0/4170281413 >> 172.21.15.83:0/4170281413 conn(0x7f0c5c077170 msgr2=0x7f0c5c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:02.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.465+0000 7f0c62bf1700 1 -- 172.21.15.83:0/4170281413 shutdown_connections 2024-03-11T19:38:02.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.465+0000 7f0c62bf1700 1 -- 172.21.15.83:0/4170281413 wait complete. 2024-03-11T19:38:02.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.465+0000 7f0c62bf1700 1 Processor -- start 2024-03-11T19:38:02.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c62bf1700 1 -- start start 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c62bf1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c1015d0 0x7f0c5c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c62bf1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c5c101f80 0x7f0c5c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c62bf1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5c105ef0 0x7f0c5c19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c62bf1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c19c0a0 con 0x7f0c5c1015d0 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c62bf1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c19c210 con 0x7f0c5c105ef0 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c62bf1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c19c3d0 con 0x7f0c5c101f80 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c6098d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c1015d0 0x7f0c5c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c6098d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c1015d0 0x7f0c5c197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47790/0 (socket says 172.21.15.83:47790) 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c6098d700 1 -- 172.21.15.83:0/815493175 learned_addr learned my addr 172.21.15.83:0/815493175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c6118e700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5c105ef0 0x7f0c5c19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c5bfff700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c5c101f80 0x7f0c5c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:02.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c6098d700 1 -- 172.21.15.83:0/815493175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c5c101f80 msgr2=0x7f0c5c197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c6098d700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c5c101f80 0x7f0c5c197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c6098d700 1 -- 172.21.15.83:0/815493175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5c105ef0 msgr2=0x7f0c5c19ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c6098d700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5c105ef0 0x7f0c5c19ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c6098d700 1 -- 172.21.15.83:0/815493175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c480095f0 con 0x7f0c5c1015d0 2024-03-11T19:38:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c6118e700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5c105ef0 0x7f0c5c19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c6098d700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c1015d0 0x7f0c5c197210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000b530 tx=0x7f0c5000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c59ffb700 1 -- 172.21.15.83:0/815493175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50011830 con 0x7f0c5c1015d0 2024-03-11T19:38:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c62bf1700 1 -- 172.21.15.83:0/815493175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c19c6b0 con 0x7f0c5c1015d0 2024-03-11T19:38:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c59ffb700 1 -- 172.21.15.83:0/815493175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c50011e70 con 0x7f0c5c1015d0 2024-03-11T19:38:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.466+0000 7f0c59ffb700 1 -- 172.21.15.83:0/815493175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000f2e0 con 0x7f0c5c1015d0 2024-03-11T19:38:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.467+0000 7f0c62bf1700 1 -- 172.21.15.83:0/815493175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c1a3f10 con 0x7f0c5c1015d0 2024-03-11T19:38:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.468+0000 7f0c62bf1700 1 -- 172.21.15.83:0/815493175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c109210 con 0x7f0c5c1015d0 2024-03-11T19:38:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.469+0000 7f0c59ffb700 1 -- 172.21.15.83:0/815493175 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f0c500105f0 con 0x7f0c5c1015d0 2024-03-11T19:38:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.469+0000 7f0c59ffb700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0c4c0a1b50 0x7f0c4c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.469+0000 7f0c5bfff700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0c4c0a1b50 0x7f0c4c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.469+0000 7f0c59ffb700 1 -- 172.21.15.83:0/815493175 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0c500ccdc0 con 0x7f0c5c1015d0 2024-03-11T19:38:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.470+0000 7f0c5bfff700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0c4c0a1b50 0x7f0c4c0a4010 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c48006010 tx=0x7f0c4801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:02.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.474+0000 7f0c59ffb700 1 -- 172.21.15.83:0/815493175 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c50096130 con 0x7f0c5c1015d0 2024-03-11T19:38:02.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.674+0000 7f0c62bf1700 1 -- 172.21.15.83:0/815493175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0c5c068930 con 0x7f0c5c1015d0 2024-03-11T19:38:02.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.674+0000 7f0c59ffb700 1 -- 172.21.15.83:0/815493175 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0c500997f0 con 0x7f0c5c1015d0 2024-03-11T19:38:02.679 INFO:teuthology.orchestra.run.smithi083.stdout:34359738447 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.676+0000 7f0c62bf1700 1 -- 172.21.15.83:0/815493175 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0c4c0a1b50 msgr2=0x7f0c4c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.677+0000 7f0c62bf1700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0c4c0a1b50 0x7f0c4c0a4010 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c48006010 tx=0x7f0c4801a040 comp rx=0 tx=0).stop 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.677+0000 7f0c62bf1700 1 -- 172.21.15.83:0/815493175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c1015d0 msgr2=0x7f0c5c197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.677+0000 7f0c62bf1700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c1015d0 0x7f0c5c197210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000b530 tx=0x7f0c5000b8f0 comp rx=0 tx=0).stop 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.677+0000 7f0c62bf1700 1 -- 172.21.15.83:0/815493175 shutdown_connections 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.677+0000 7f0c62bf1700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0c4c0a1b50 0x7f0c4c0a4010 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.677+0000 7f0c62bf1700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c5c105ef0 0x7f0c5c19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.677+0000 7f0c62bf1700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c1015d0 0x7f0c5c197210 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.677+0000 7f0c62bf1700 1 --2- 172.21.15.83:0/815493175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c5c101f80 0x7f0c5c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.677+0000 7f0c62bf1700 1 -- 172.21.15.83:0/815493175 >> 172.21.15.83:0/815493175 conn(0x7f0c5c077170 msgr2=0x7f0c5c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.677+0000 7f0c62bf1700 1 -- 172.21.15.83:0/815493175 shutdown_connections 2024-03-11T19:38:02.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:02.677+0000 7f0c62bf1700 1 -- 172.21.15.83:0/815493175 wait complete. 2024-03-11T19:38:03.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.109+0000 7fa33613a700 1 -- 172.21.15.83:0/345013238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330100b70 msgr2=0x7fa330100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:03.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.109+0000 7fa33613a700 1 --2- 172.21.15.83:0/345013238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330100b70 0x7fa330100ff0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa318009990 tx=0x7fa318009ca0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.109+0000 7fa33613a700 1 -- 172.21.15.83:0/345013238 shutdown_connections 2024-03-11T19:38:03.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.109+0000 7fa33613a700 1 --2- 172.21.15.83:0/345013238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330101530 0x7fa33010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.109+0000 7fa33613a700 1 --2- 172.21.15.83:0/345013238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa330106b80 0x7fa330106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.109+0000 7fa33613a700 1 --2- 172.21.15.83:0/345013238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330100b70 0x7fa330100ff0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.109+0000 7fa33613a700 1 -- 172.21.15.83:0/345013238 >> 172.21.15.83:0/345013238 conn(0x7fa330077aa0 msgr2=0x7fa330077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:03.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.109+0000 7fa33613a700 1 -- 172.21.15.83:0/345013238 shutdown_connections 2024-03-11T19:38:03.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.109+0000 7fa33613a700 1 -- 172.21.15.83:0/345013238 wait complete. 2024-03-11T19:38:03.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa33613a700 1 Processor -- start 2024-03-11T19:38:03.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa33613a700 1 -- start start 2024-03-11T19:38:03.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa33613a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa330100b70 0x7fa3301027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:03.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa33613a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330101530 0x7fa330102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:03.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa33613a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330106b80 0x7fa330103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:03.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa33613a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3301a65d0 con 0x7fa330101530 2024-03-11T19:38:03.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa33613a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3301a6740 con 0x7fa330100b70 2024-03-11T19:38:03.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa33613a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3301a6900 con 0x7fa330106b80 2024-03-11T19:38:03.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa32effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330101530 0x7fa330102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:03.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa32effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330101530 0x7fa330102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47804/0 (socket says 172.21.15.83:47804) 2024-03-11T19:38:03.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa32effd700 1 -- 172.21.15.83:0/3422499917 learned_addr learned my addr 172.21.15.83:0/3422499917 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:03.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.110+0000 7fa32f7fe700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa330100b70 0x7fa3301027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:03.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa32effd700 1 -- 172.21.15.83:0/3422499917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330106b80 msgr2=0x7fa330103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:03.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa32ffff700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330106b80 0x7fa330103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:03.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa32effd700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330106b80 0x7fa330103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa32effd700 1 -- 172.21.15.83:0/3422499917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa330100b70 msgr2=0x7fa3301027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:03.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa32effd700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa330100b70 0x7fa3301027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa32effd700 1 -- 172.21.15.83:0/3422499917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3180095f0 con 0x7fa330101530 2024-03-11T19:38:03.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa32f7fe700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa330100b70 0x7fa3301027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:03.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa32effd700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330101530 0x7fa330102cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa3180098e0 tx=0x7fa31800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:03.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa32cff9700 1 -- 172.21.15.83:0/3422499917 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31801d070 con 0x7fa330101530 2024-03-11T19:38:03.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa33613a700 1 -- 172.21.15.83:0/3422499917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3301a6b80 con 0x7fa330101530 2024-03-11T19:38:03.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa32cff9700 1 -- 172.21.15.83:0/3422499917 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa318003680 con 0x7fa330101530 2024-03-11T19:38:03.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa32cff9700 1 -- 172.21.15.83:0/3422499917 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318017650 con 0x7fa330101530 2024-03-11T19:38:03.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.111+0000 7fa33613a700 1 -- 172.21.15.83:0/3422499917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3301a6ff0 con 0x7fa330101530 2024-03-11T19:38:03.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.112+0000 7fa33613a700 1 -- 172.21.15.83:0/3422499917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa330068930 con 0x7fa330101530 2024-03-11T19:38:03.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.113+0000 7fa32cff9700 1 -- 172.21.15.83:0/3422499917 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fa318022470 con 0x7fa330101530 2024-03-11T19:38:03.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.113+0000 7fa32cff9700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa31c0a1b50 0x7fa31c0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:03.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.114+0000 7fa32f7fe700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa31c0a1b50 0x7fa31c0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:03.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.114+0000 7fa32cff9700 1 -- 172.21.15.83:0/3422499917 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa3180ce640 con 0x7fa330101530 2024-03-11T19:38:03.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.114+0000 7fa32f7fe700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa31c0a1b50 0x7fa31c0a4010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa328005950 tx=0x7fa3280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:03.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.118+0000 7fa32cff9700 1 -- 172.21.15.83:0/3422499917 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa318097930 con 0x7fa330101530 2024-03-11T19:38:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.320+0000 7fa33613a700 1 -- 172.21.15.83:0/3422499917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fa330110bf0 con 0x7fa330101530 2024-03-11T19:38:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.320+0000 7fa32cff9700 1 -- 172.21.15.83:0/3422499917 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fa31809aff0 con 0x7fa330101530 2024-03-11T19:38:03.324 INFO:teuthology.orchestra.run.smithi083.stdout:343597383704 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.322+0000 7fa33613a700 1 -- 172.21.15.83:0/3422499917 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa31c0a1b50 msgr2=0x7fa31c0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.322+0000 7fa33613a700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa31c0a1b50 0x7fa31c0a4010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa328005950 tx=0x7fa3280058e0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.322+0000 7fa33613a700 1 -- 172.21.15.83:0/3422499917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330101530 msgr2=0x7fa330102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.322+0000 7fa33613a700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330101530 0x7fa330102cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa3180098e0 tx=0x7fa31800f690 comp rx=0 tx=0).stop 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.323+0000 7fa33613a700 1 -- 172.21.15.83:0/3422499917 shutdown_connections 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.323+0000 7fa33613a700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa31c0a1b50 0x7fa31c0a4010 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.323+0000 7fa33613a700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa330106b80 0x7fa330103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.323+0000 7fa33613a700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa330100b70 0x7fa3301027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.323+0000 7fa33613a700 1 --2- 172.21.15.83:0/3422499917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330101530 0x7fa330102cf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.323+0000 7fa33613a700 1 -- 172.21.15.83:0/3422499917 >> 172.21.15.83:0/3422499917 conn(0x7fa330077aa0 msgr2=0x7fa3300ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.323+0000 7fa33613a700 1 -- 172.21.15.83:0/3422499917 shutdown_connections 2024-03-11T19:38:03.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.323+0000 7fa33613a700 1 -- 172.21.15.83:0/3422499917 wait complete. 2024-03-11T19:38:03.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.349+0000 7f64d16c3700 1 -- 172.21.15.83:0/3293181166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64cc105670 msgr2=0x7f64cc10d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:03.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.349+0000 7f64d16c3700 1 --2- 172.21.15.83:0/3293181166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64cc105670 0x7f64cc10d240 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f64c400b0a0 tx=0x7f64c400b3b0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.349+0000 7f64d16c3700 1 -- 172.21.15.83:0/3293181166 shutdown_connections 2024-03-11T19:38:03.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.349+0000 7f64d16c3700 1 --2- 172.21.15.83:0/3293181166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64cc105670 0x7f64cc10d240 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.349+0000 7f64d16c3700 1 --2- 172.21.15.83:0/3293181166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc100590 0x7f64cc100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.349+0000 7f64d16c3700 1 --2- 172.21.15.83:0/3293181166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64cc100f40 0x7f64cc104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.349+0000 7f64d16c3700 1 -- 172.21.15.83:0/3293181166 >> 172.21.15.83:0/3293181166 conn(0x7f64cc077170 msgr2=0x7f64cc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.349+0000 7f64d16c3700 1 -- 172.21.15.83:0/3293181166 shutdown_connections 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.349+0000 7f64d16c3700 1 -- 172.21.15.83:0/3293181166 wait complete. 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64d16c3700 1 Processor -- start 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64d16c3700 1 -- start start 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64d16c3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64cc100590 0x7f64cc197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64d16c3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc100f40 0x7f64cc1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64d16c3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64cc105670 0x7f64cc19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64d16c3700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc19c160 con 0x7f64cc100590 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64d16c3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc19c2d0 con 0x7f64cc100f40 2024-03-11T19:38:03.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64d16c3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc19c490 con 0x7f64cc105670 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64caffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64cc100590 0x7f64cc197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64caffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64cc100590 0x7f64cc197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47820/0 (socket says 172.21.15.83:47820) 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64caffd700 1 -- 172.21.15.83:0/802245537 learned_addr learned my addr 172.21.15.83:0/802245537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64ca7fc700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc100f40 0x7f64cc1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64cb7fe700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64cc105670 0x7f64cc19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64caffd700 1 -- 172.21.15.83:0/802245537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64cc105670 msgr2=0x7f64cc19bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64caffd700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64cc105670 0x7f64cc19bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64caffd700 1 -- 172.21.15.83:0/802245537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc100f40 msgr2=0x7f64cc1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64caffd700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc100f40 0x7f64cc1977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64caffd700 1 -- 172.21.15.83:0/802245537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c4009d00 con 0x7f64cc100590 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.350+0000 7f64cb7fe700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64cc105670 0x7f64cc19bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.351+0000 7f64caffd700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64cc100590 0x7f64cc197280 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f64c0007910 tx=0x7f64c0007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:03.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.351+0000 7f64b3fff700 1 -- 172.21.15.83:0/802245537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0012070 con 0x7f64cc100590 2024-03-11T19:38:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.351+0000 7f64b3fff700 1 -- 172.21.15.83:0/802245537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64c000d810 con 0x7f64cc100590 2024-03-11T19:38:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.351+0000 7f64d16c3700 1 -- 172.21.15.83:0/802245537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc19c770 con 0x7f64cc100590 2024-03-11T19:38:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.351+0000 7f64b3fff700 1 -- 172.21.15.83:0/802245537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0005280 con 0x7f64cc100590 2024-03-11T19:38:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.351+0000 7f64d16c3700 1 -- 172.21.15.83:0/802245537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc10d950 con 0x7f64cc100590 2024-03-11T19:38:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.352+0000 7f64d16c3700 1 -- 172.21.15.83:0/802245537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc101a70 con 0x7f64cc100590 2024-03-11T19:38:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.353+0000 7f64b3fff700 1 -- 172.21.15.83:0/802245537 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f64c0016070 con 0x7f64cc100590 2024-03-11T19:38:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.354+0000 7f64b3fff700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f64b40a1b50 0x7f64b40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.354+0000 7f64ca7fc700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f64b40a1b50 0x7f64b40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.354+0000 7f64b3fff700 1 -- 172.21.15.83:0/802245537 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f64c00ce330 con 0x7f64cc100590 2024-03-11T19:38:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.354+0000 7f64ca7fc700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f64b40a1b50 0x7f64b40a4010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f64cc107060 tx=0x7f64bc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:03.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.358+0000 7f64b3fff700 1 -- 172.21.15.83:0/802245537 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64c0097590 con 0x7f64cc100590 2024-03-11T19:38:03.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.557+0000 7f64d16c3700 1 -- 172.21.15.83:0/802245537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f64cc110c90 con 0x7f64cc100590 2024-03-11T19:38:03.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.557+0000 7f64b3fff700 1 -- 172.21.15.83:0/802245537 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f64c0095e60 con 0x7f64cc100590 2024-03-11T19:38:03.561 INFO:teuthology.orchestra.run.smithi083.stdout:309237645341 2024-03-11T19:38:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 -- 172.21.15.83:0/802245537 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f64b40a1b50 msgr2=0x7f64b40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f64b40a1b50 0x7f64b40a4010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f64cc107060 tx=0x7f64bc0073d0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 -- 172.21.15.83:0/802245537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64cc100590 msgr2=0x7f64cc197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64cc100590 0x7f64cc197280 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f64c0007910 tx=0x7f64c0007c20 comp rx=0 tx=0).stop 2024-03-11T19:38:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 -- 172.21.15.83:0/802245537 shutdown_connections 2024-03-11T19:38:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f64b40a1b50 0x7f64b40a4010 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64cc105670 0x7f64cc19bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64cc100590 0x7f64cc197280 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 --2- 172.21.15.83:0/802245537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64cc100f40 0x7f64cc1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 -- 172.21.15.83:0/802245537 >> 172.21.15.83:0/802245537 conn(0x7f64cc077170 msgr2=0x7f64cc103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 -- 172.21.15.83:0/802245537 shutdown_connections 2024-03-11T19:38:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:03.559+0000 7f64d16c3700 1 -- 172.21.15.83:0/802245537 wait complete. 2024-03-11T19:38:03.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:03 smithi137 ceph-mon[36220]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:38:03.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:03 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/815493175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-11T19:38:03.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:03 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3422499917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-11T19:38:03.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:03 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/802245537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-11T19:38:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:03 smithi204 ceph-mon[36739]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:38:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:03 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/815493175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-11T19:38:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:03 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3422499917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-11T19:38:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:03 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/802245537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-11T19:38:03.935 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:03 smithi083 ceph-mon[31864]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:38:03.993 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:03 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/815493175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-11T19:38:03.993 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:03 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3422499917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-11T19:38:03.994 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:03 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/802245537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-11T19:38:04.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:04 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:38:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:04 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:38:04.939 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:04 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:38:05.182 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383698 got 343597383704 for osd.11 2024-03-11T19:38:05.182 DEBUG:teuthology.parallel:result is None 2024-03-11T19:38:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.506+0000 7ff2c2a74700 1 -- 172.21.15.83:0/1560517163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2bc100260 msgr2=0x7ff2bc1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.506+0000 7ff2c2a74700 1 --2- 172.21.15.83:0/1560517163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2bc100260 0x7ff2bc1006e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009990 tx=0x7ff2a8009ca0 comp rx=0 tx=0).stop 2024-03-11T19:38:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.506+0000 7ff2c2a74700 1 -- 172.21.15.83:0/1560517163 shutdown_connections 2024-03-11T19:38:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.506+0000 7ff2c2a74700 1 --2- 172.21.15.83:0/1560517163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2bc100c20 0x7ff2bc10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.506+0000 7ff2c2a74700 1 --2- 172.21.15.83:0/1560517163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2bc106ce0 0x7ff2bc1070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.506+0000 7ff2c2a74700 1 --2- 172.21.15.83:0/1560517163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2bc100260 0x7ff2bc1006e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.506+0000 7ff2c2a74700 1 -- 172.21.15.83:0/1560517163 >> 172.21.15.83:0/1560517163 conn(0x7ff2bc077170 msgr2=0x7ff2bc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.507+0000 7ff2c2a74700 1 -- 172.21.15.83:0/1560517163 shutdown_connections 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.507+0000 7ff2c2a74700 1 -- 172.21.15.83:0/1560517163 wait complete. 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c2a74700 1 Processor -- start 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c2a74700 1 -- start start 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c2a74700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2bc100260 0x7ff2bc101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c2a74700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2bc100c20 0x7ff2bc101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c2a74700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2bc106ce0 0x7ff2bc101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c2a74700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc1a6670 con 0x7ff2bc100260 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c2a74700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc1a67e0 con 0x7ff2bc106ce0 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c2a74700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc1a69a0 con 0x7ff2bc100c20 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c0810700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2bc100260 0x7ff2bc101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c0810700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2bc100260 0x7ff2bc101440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47842/0 (socket says 172.21.15.83:47842) 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c0810700 1 -- 172.21.15.83:0/2541420117 learned_addr learned my addr 172.21.15.83:0/2541420117 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2bbfff700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2bc100c20 0x7ff2bc101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:05.736 INFO:teuthology.orchestra.run.smithi083.stdout:287762808867 2024-03-11T19:38:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.508+0000 7ff2c1011700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2bc106ce0 0x7ff2bc101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.509+0000 7ff2c0810700 1 -- 172.21.15.83:0/2541420117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2bc100c20 msgr2=0x7ff2bc101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.509+0000 7ff2c0810700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2bc100c20 0x7ff2bc101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.509+0000 7ff2c0810700 1 -- 172.21.15.83:0/2541420117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2bc106ce0 msgr2=0x7ff2bc101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.509+0000 7ff2c0810700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2bc106ce0 0x7ff2bc101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.509+0000 7ff2c0810700 1 -- 172.21.15.83:0/2541420117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80095f0 con 0x7ff2bc100260 2024-03-11T19:38:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.509+0000 7ff2c0810700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2bc100260 0x7ff2bc101440 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000d980 tx=0x7ff2b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.509+0000 7ff2b9ffb700 1 -- 172.21.15.83:0/2541420117 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0004220 con 0x7ff2bc100260 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.509+0000 7ff2b9ffb700 1 -- 172.21.15.83:0/2541420117 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2b0007640 con 0x7ff2bc100260 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.509+0000 7ff2b9ffb700 1 -- 172.21.15.83:0/2541420117 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b000f690 con 0x7ff2bc100260 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.509+0000 7ff2c2a74700 1 -- 172.21.15.83:0/2541420117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2bc1a6c80 con 0x7ff2bc100260 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.510+0000 7ff2c2a74700 1 -- 172.21.15.83:0/2541420117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc1a7180 con 0x7ff2bc100260 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.511+0000 7ff2c2a74700 1 -- 172.21.15.83:0/2541420117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2bc10a2d0 con 0x7ff2bc100260 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.512+0000 7ff2b9ffb700 1 -- 172.21.15.83:0/2541420117 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7ff2b0004410 con 0x7ff2bc100260 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.512+0000 7ff2b9ffb700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2ac0a1bd0 0x7ff2ac0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.513+0000 7ff2bbfff700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2ac0a1bd0 0x7ff2ac0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.513+0000 7ff2b9ffb700 1 -- 172.21.15.83:0/2541420117 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff2b00cd0d0 con 0x7ff2bc100260 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.513+0000 7ff2bbfff700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2ac0a1bd0 0x7ff2ac0a4090 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc1064e0 tx=0x7ff2a800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.517+0000 7ff2b9ffb700 1 -- 172.21.15.83:0/2541420117 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff2b0096230 con 0x7ff2bc100260 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.723+0000 7ff2c2a74700 1 -- 172.21.15.83:0/2541420117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7ff2bc103560 con 0x7ff2bc100260 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.723+0000 7ff2b9ffb700 1 -- 172.21.15.83:0/2541420117 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff2b0016090 con 0x7ff2bc100260 2024-03-11T19:38:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.725+0000 7ff2c2a74700 1 -- 172.21.15.83:0/2541420117 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2ac0a1bd0 msgr2=0x7ff2ac0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.725+0000 7ff2c2a74700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2ac0a1bd0 0x7ff2ac0a4090 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc1064e0 tx=0x7ff2a800b540 comp rx=0 tx=0).stop 2024-03-11T19:38:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.725+0000 7ff2c2a74700 1 -- 172.21.15.83:0/2541420117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2bc100260 msgr2=0x7ff2bc101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.725+0000 7ff2c2a74700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2bc100260 0x7ff2bc101440 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000d980 tx=0x7ff2b000dc90 comp rx=0 tx=0).stop 2024-03-11T19:38:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.726+0000 7ff2c2a74700 1 -- 172.21.15.83:0/2541420117 shutdown_connections 2024-03-11T19:38:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.726+0000 7ff2c2a74700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2ac0a1bd0 0x7ff2ac0a4090 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.726+0000 7ff2c2a74700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2bc106ce0 0x7ff2bc101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.726+0000 7ff2c2a74700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2bc100260 0x7ff2bc101440 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.726+0000 7ff2c2a74700 1 --2- 172.21.15.83:0/2541420117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2bc100c20 0x7ff2bc101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.726+0000 7ff2c2a74700 1 -- 172.21.15.83:0/2541420117 >> 172.21.15.83:0/2541420117 conn(0x7ff2bc077170 msgr2=0x7ff2bc104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.726+0000 7ff2c2a74700 1 -- 172.21.15.83:0/2541420117 shutdown_connections 2024-03-11T19:38:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:04.726+0000 7ff2c2a74700 1 -- 172.21.15.83:0/2541420117 wait complete. 2024-03-11T19:38:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:05 smithi137 ceph-mon[36220]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:38:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:05 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2541420117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-11T19:38:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:05 smithi204 ceph-mon[36739]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:38:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:05 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2541420117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-11T19:38:06.029 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:05 smithi083 ceph-mon[31864]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:38:06.029 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:05 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2541420117' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-11T19:38:06.363 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:06.367 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808862 got 287762808867 for osd.9 2024-03-11T19:38:06.367 DEBUG:teuthology.parallel:result is None 2024-03-11T19:38:06.775 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920824 got 141733920827 for osd.4 2024-03-11T19:38:06.775 DEBUG:teuthology.parallel:result is None 2024-03-11T19:38:07.155 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645336 got 309237645341 for osd.10 2024-03-11T19:38:07.156 DEBUG:teuthology.parallel:result is None 2024-03-11T19:38:07.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:07 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:07 smithi083 ceph-mon[31864]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:07.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:07 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:07 smithi137 ceph-mon[36220]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:07.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:07 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:07.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:07 smithi204 ceph-mon[36739]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.908+0000 7fef3d4af700 1 -- 172.21.15.83:0/3009399080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef38101530 msgr2=0x7fef3810d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.908+0000 7fef3d4af700 1 --2- 172.21.15.83:0/3009399080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef38101530 0x7fef3810d9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fef3000b0a0 tx=0x7fef3000b3b0 comp rx=0 tx=0).stop 2024-03-11T19:38:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.908+0000 7fef3d4af700 1 -- 172.21.15.83:0/3009399080 shutdown_connections 2024-03-11T19:38:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.908+0000 7fef3d4af700 1 --2- 172.21.15.83:0/3009399080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef38101530 0x7fef3810d9c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.908+0000 7fef3d4af700 1 --2- 172.21.15.83:0/3009399080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef38106b80 0x7fef38106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.908+0000 7fef3d4af700 1 --2- 172.21.15.83:0/3009399080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef38100b70 0x7fef38100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.908+0000 7fef3d4af700 1 -- 172.21.15.83:0/3009399080 >> 172.21.15.83:0/3009399080 conn(0x7fef38077aa0 msgr2=0x7fef38077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.908+0000 7fef3d4af700 1 -- 172.21.15.83:0/3009399080 shutdown_connections 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.908+0000 7fef3d4af700 1 -- 172.21.15.83:0/3009399080 wait complete. 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.909+0000 7fef3d4af700 1 Processor -- start 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.909+0000 7fef3d4af700 1 -- start start 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.909+0000 7fef3d4af700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef38100b70 0x7fef38198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.909+0000 7fef3d4af700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef38101530 0x7fef38199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.909+0000 7fef3d4af700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef38106b80 0x7fef381a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef3d4af700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef38199dc0 con 0x7fef38100b70 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef3d4af700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef381a4580 con 0x7fef38106b80 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef3d4af700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef381a46c0 con 0x7fef38101530 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef36ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef38100b70 0x7fef38198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef36ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef38100b70 0x7fef38198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47868/0 (socket says 172.21.15.83:47868) 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef36ffd700 1 -- 172.21.15.83:0/311767455 learned_addr learned my addr 172.21.15.83:0/311767455 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef367fc700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef38101530 0x7fef38199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef377fe700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef38106b80 0x7fef381a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef36ffd700 1 -- 172.21.15.83:0/311767455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef38101530 msgr2=0x7fef38199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef36ffd700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef38101530 0x7fef38199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef36ffd700 1 -- 172.21.15.83:0/311767455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef38106b80 msgr2=0x7fef381a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef36ffd700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef38106b80 0x7fef381a4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef36ffd700 1 -- 172.21.15.83:0/311767455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30009d00 con 0x7fef38100b70 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef367fc700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef38101530 0x7fef38199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef36ffd700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef38100b70 0x7fef38198fc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fef2800c920 tx=0x7fef2800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef1ffff700 1 -- 172.21.15.83:0/311767455 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28007b00 con 0x7fef38100b70 2024-03-11T19:38:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef3d4af700 1 -- 172.21.15.83:0/311767455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef381a49a0 con 0x7fef38100b70 2024-03-11T19:38:07.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef1ffff700 1 -- 172.21.15.83:0/311767455 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef28004d10 con 0x7fef38100b70 2024-03-11T19:38:07.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef1ffff700 1 -- 172.21.15.83:0/311767455 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef280055c0 con 0x7fef38100b70 2024-03-11T19:38:07.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.910+0000 7fef3d4af700 1 -- 172.21.15.83:0/311767455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef381a4ef0 con 0x7fef38100b70 2024-03-11T19:38:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.911+0000 7fef3d4af700 1 -- 172.21.15.83:0/311767455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef38109020 con 0x7fef38100b70 2024-03-11T19:38:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.913+0000 7fef1ffff700 1 -- 172.21.15.83:0/311767455 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fef28004750 con 0x7fef38100b70 2024-03-11T19:38:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.914+0000 7fef1ffff700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fef200a1b50 0x7fef200a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.914+0000 7fef367fc700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fef200a1b50 0x7fef200a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.914+0000 7fef1ffff700 1 -- 172.21.15.83:0/311767455 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fef280ccfd0 con 0x7fef38100b70 2024-03-11T19:38:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.915+0000 7fef367fc700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fef200a1b50 0x7fef200a4010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef38078030 tx=0x7fef2c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:07.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:07.922+0000 7fef1ffff700 1 -- 172.21.15.83:0/311767455 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef28096230 con 0x7fef38100b70 2024-03-11T19:38:08.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.124+0000 7fef3d4af700 1 -- 172.21.15.83:0/311767455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fef38110b80 con 0x7fef38100b70 2024-03-11T19:38:08.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.124+0000 7fef1ffff700 1 -- 172.21.15.83:0/311767455 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fef2801d090 con 0x7fef38100b70 2024-03-11T19:38:08.128 INFO:teuthology.orchestra.run.smithi083.stdout:55834574923 2024-03-11T19:38:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 -- 172.21.15.83:0/311767455 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fef200a1b50 msgr2=0x7fef200a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fef200a1b50 0x7fef200a4010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef38078030 tx=0x7fef2c005f50 comp rx=0 tx=0).stop 2024-03-11T19:38:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 -- 172.21.15.83:0/311767455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef38100b70 msgr2=0x7fef38198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef38100b70 0x7fef38198fc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fef2800c920 tx=0x7fef2800cce0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 -- 172.21.15.83:0/311767455 shutdown_connections 2024-03-11T19:38:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fef200a1b50 0x7fef200a4010 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef38106b80 0x7fef381a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef38100b70 0x7fef38198fc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 --2- 172.21.15.83:0/311767455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef38101530 0x7fef38199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 -- 172.21.15.83:0/311767455 >> 172.21.15.83:0/311767455 conn(0x7fef38077aa0 msgr2=0x7fef380ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 -- 172.21.15.83:0/311767455 shutdown_connections 2024-03-11T19:38:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.126+0000 7fef3d4af700 1 -- 172.21.15.83:0/311767455 wait complete. 2024-03-11T19:38:08.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:08 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/311767455' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-11T19:38:08.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:08 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/311767455' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-11T19:38:08.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:08 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/311767455' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-11T19:38:08.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.609+0000 7ff99f1d8700 1 -- 172.21.15.83:0/3163163090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9981017a0 msgr2=0x7ff9981057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.609+0000 7ff99f1d8700 1 --2- 172.21.15.83:0/3163163090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9981017a0 0x7ff9981057f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff984009990 tx=0x7ff984009ca0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.610+0000 7ff99f1d8700 1 -- 172.21.15.83:0/3163163090 shutdown_connections 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.610+0000 7ff99f1d8700 1 --2- 172.21.15.83:0/3163163090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff998105ed0 0x7ff99810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.610+0000 7ff99f1d8700 1 --2- 172.21.15.83:0/3163163090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff998100df0 0x7ff9981011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.610+0000 7ff99f1d8700 1 --2- 172.21.15.83:0/3163163090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9981017a0 0x7ff9981057f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.610+0000 7ff99f1d8700 1 -- 172.21.15.83:0/3163163090 >> 172.21.15.83:0/3163163090 conn(0x7ff998077c70 msgr2=0x7ff998078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.610+0000 7ff99f1d8700 1 -- 172.21.15.83:0/3163163090 shutdown_connections 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.610+0000 7ff99f1d8700 1 -- 172.21.15.83:0/3163163090 wait complete. 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.610+0000 7ff99f1d8700 1 Processor -- start 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.610+0000 7ff99f1d8700 1 -- start start 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99f1d8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff998100df0 0x7ff998198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99f1d8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9981017a0 0x7ff998198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:08.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99f1d8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff998105ed0 0x7ff99819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99f1d8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99819d000 con 0x7ff998105ed0 2024-03-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99f1d8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99819d170 con 0x7ff9981017a0 2024-03-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99f1d8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99819d330 con 0x7ff998100df0 2024-03-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99d775700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff998105ed0 0x7ff99819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99d775700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff998105ed0 0x7ff99819c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47886/0 (socket says 172.21.15.83:47886) 2024-03-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99d775700 1 -- 172.21.15.83:0/469941715 learned_addr learned my addr 172.21.15.83:0/469941715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99cf74700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff998100df0 0x7ff998198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff997fff700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9981017a0 0x7ff998198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99d775700 1 -- 172.21.15.83:0/469941715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff998100df0 msgr2=0x7ff998198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:08.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99d775700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff998100df0 0x7ff998198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99d775700 1 -- 172.21.15.83:0/469941715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9981017a0 msgr2=0x7ff998198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99d775700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9981017a0 0x7ff998198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99d775700 1 -- 172.21.15.83:0/469941715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9840095f0 con 0x7ff998105ed0 2024-03-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99cf74700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff998100df0 0x7ff998198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff99d775700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff998105ed0 0x7ff99819c9f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff990007d70 tx=0x7ff99000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.611+0000 7ff995ffb700 1 -- 172.21.15.83:0/469941715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99000ad40 con 0x7ff998105ed0 2024-03-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.612+0000 7ff99f1d8700 1 -- 172.21.15.83:0/469941715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99819d610 con 0x7ff998105ed0 2024-03-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.612+0000 7ff995ffb700 1 -- 172.21.15.83:0/469941715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff990012900 con 0x7ff998105ed0 2024-03-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.612+0000 7ff995ffb700 1 -- 172.21.15.83:0/469941715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990011780 con 0x7ff998105ed0 2024-03-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.612+0000 7ff99f1d8700 1 -- 172.21.15.83:0/469941715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9981a4e70 con 0x7ff998105ed0 2024-03-11T19:38:08.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.613+0000 7ff99f1d8700 1 -- 172.21.15.83:0/469941715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99810a200 con 0x7ff998105ed0 2024-03-11T19:38:08.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.614+0000 7ff995ffb700 1 -- 172.21.15.83:0/469941715 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7ff990012a70 con 0x7ff998105ed0 2024-03-11T19:38:08.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.614+0000 7ff995ffb700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9880a1b50 0x7ff9880a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:08.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.614+0000 7ff99cf74700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9880a1b50 0x7ff9880a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:08.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.614+0000 7ff995ffb700 1 -- 172.21.15.83:0/469941715 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff9900cd4f0 con 0x7ff998105ed0 2024-03-11T19:38:08.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.615+0000 7ff99cf74700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9880a1b50 0x7ff9880a4010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff98c005950 tx=0x7ff98c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:08.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.620+0000 7ff995ffb700 1 -- 172.21.15.83:0/469941715 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff990096860 con 0x7ff998105ed0 2024-03-11T19:38:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.821+0000 7ff99f1d8700 1 -- 172.21.15.83:0/469941715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff9981994b0 con 0x7ff998105ed0 2024-03-11T19:38:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.821+0000 7ff995ffb700 1 -- 172.21.15.83:0/469941715 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff990099f20 con 0x7ff998105ed0 2024-03-11T19:38:08.825 INFO:teuthology.orchestra.run.smithi083.stdout:77309411398 2024-03-11T19:38:08.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 -- 172.21.15.83:0/469941715 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9880a1b50 msgr2=0x7ff9880a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:08.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9880a1b50 0x7ff9880a4010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff98c005950 tx=0x7ff98c005f50 comp rx=0 tx=0).stop 2024-03-11T19:38:08.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 -- 172.21.15.83:0/469941715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff998105ed0 msgr2=0x7ff99819c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:08.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff998105ed0 0x7ff99819c9f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff990007d70 tx=0x7ff99000a340 comp rx=0 tx=0).stop 2024-03-11T19:38:08.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 -- 172.21.15.83:0/469941715 shutdown_connections 2024-03-11T19:38:08.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9880a1b50 0x7ff9880a4010 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff998100df0 0x7ff998198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff998105ed0 0x7ff99819c9f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 --2- 172.21.15.83:0/469941715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9981017a0 0x7ff998198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:08.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 -- 172.21.15.83:0/469941715 >> 172.21.15.83:0/469941715 conn(0x7ff998077c70 msgr2=0x7ff99806a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:08.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 -- 172.21.15.83:0/469941715 shutdown_connections 2024-03-11T19:38:08.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:08.823+0000 7ff99f1d8700 1 -- 172.21.15.83:0/469941715 wait complete. 2024-03-11T19:38:09.176 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574916 got 55834574923 for osd.1 2024-03-11T19:38:09.177 DEBUG:teuthology.parallel:result is None 2024-03-11T19:38:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1, 8, 11]}]: dispatch 2024-03-11T19:38:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-03-11T19:38:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-11T19:38:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-11T19:38:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:09 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-11T19:38:09.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:09 smithi204 ceph-mon[36739]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:38:09.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:09 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/469941715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-11T19:38:09.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1, 8, 11]}]: dispatch 2024-03-11T19:38:09.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-03-11T19:38:09.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-11T19:38:09.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-11T19:38:09.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:09 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-11T19:38:09.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:09 smithi083 ceph-mon[31864]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:38:09.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:09 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/469941715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-11T19:38:09.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1, 8, 11]}]: dispatch 2024-03-11T19:38:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]: dispatch 2024-03-11T19:38:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-11T19:38:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-11T19:38:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:09 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-11T19:38:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:09 smithi137 ceph-mon[36220]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:38:09.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:09 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/469941715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-11T19:38:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.245+0000 7fb59acb5700 1 -- 172.21.15.83:0/106595444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb59406adc0 msgr2=0x7fb5941068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.245+0000 7fb59acb5700 1 --2- 172.21.15.83:0/106595444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb59406adc0 0x7fb5941068a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009990 tx=0x7fb57c009ca0 comp rx=0 tx=0).stop 2024-03-11T19:38:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.245+0000 7fb59acb5700 1 -- 172.21.15.83:0/106595444 shutdown_connections 2024-03-11T19:38:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.245+0000 7fb59acb5700 1 --2- 172.21.15.83:0/106595444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb594106f80 0x7fb59410eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:10.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.245+0000 7fb59acb5700 1 --2- 172.21.15.83:0/106595444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb59406a410 0x7fb59406a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:10.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.245+0000 7fb59acb5700 1 --2- 172.21.15.83:0/106595444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb59406adc0 0x7fb5941068a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:10.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.245+0000 7fb59acb5700 1 -- 172.21.15.83:0/106595444 >> 172.21.15.83:0/106595444 conn(0x7fb594076bc0 msgr2=0x7fb594076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:10.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.245+0000 7fb59acb5700 1 -- 172.21.15.83:0/106595444 shutdown_connections 2024-03-11T19:38:10.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.245+0000 7fb59acb5700 1 -- 172.21.15.83:0/106595444 wait complete. 2024-03-11T19:38:10.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.245+0000 7fb59acb5700 1 Processor -- start 2024-03-11T19:38:10.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.245+0000 7fb59acb5700 1 -- start start 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb59acb5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb59406a410 0x7fb594198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb59acb5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb59406adc0 0x7fb594198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb59acb5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb594106f80 0x7fb59419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb59acb5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59419d030 con 0x7fb59406adc0 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb59acb5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59419d1a0 con 0x7fb59406a410 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb59acb5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59419d360 con 0x7fb594106f80 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb593fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb59406adc0 0x7fb594198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb593fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb59406adc0 0x7fb594198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45372/0 (socket says 172.21.15.83:45372) 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb593fff700 1 -- 172.21.15.83:0/493208513 learned_addr learned my addr 172.21.15.83:0/493208513 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb598a51700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb59406a410 0x7fb594198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb599252700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb594106f80 0x7fb59419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:10.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb593fff700 1 -- 172.21.15.83:0/493208513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb594106f80 msgr2=0x7fb59419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb593fff700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb594106f80 0x7fb59419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb593fff700 1 -- 172.21.15.83:0/493208513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb59406a410 msgr2=0x7fb594198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb593fff700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb59406a410 0x7fb594198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb593fff700 1 -- 172.21.15.83:0/493208513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c0095f0 con 0x7fb59406adc0 2024-03-11T19:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb593fff700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb59406adc0 0x7fb594198690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0098e0 tx=0x7fb57c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb591ffb700 1 -- 172.21.15.83:0/493208513 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c01d070 con 0x7fb59406adc0 2024-03-11T19:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.246+0000 7fb59acb5700 1 -- 172.21.15.83:0/493208513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59419d5e0 con 0x7fb59406adc0 2024-03-11T19:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.247+0000 7fb591ffb700 1 -- 172.21.15.83:0/493208513 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb57c003680 con 0x7fb59406adc0 2024-03-11T19:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.247+0000 7fb591ffb700 1 -- 172.21.15.83:0/493208513 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0173f0 con 0x7fb59406adc0 2024-03-11T19:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.247+0000 7fb59acb5700 1 -- 172.21.15.83:0/493208513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5941a4ea0 con 0x7fb59406adc0 2024-03-11T19:38:10.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.248+0000 7fb59acb5700 1 -- 172.21.15.83:0/493208513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59410a280 con 0x7fb59406adc0 2024-03-11T19:38:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.249+0000 7fb591ffb700 1 -- 172.21.15.83:0/493208513 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fb57c022720 con 0x7fb59406adc0 2024-03-11T19:38:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.249+0000 7fb591ffb700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb5800a1b50 0x7fb5800a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.249+0000 7fb598a51700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb5800a1b50 0x7fb5800a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.249+0000 7fb591ffb700 1 -- 172.21.15.83:0/493208513 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fb57c0ce990 con 0x7fb59406adc0 2024-03-11T19:38:10.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.250+0000 7fb598a51700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb5800a1b50 0x7fb5800a4010 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb584009550 tx=0x7fb5840091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:10.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.253+0000 7fb591ffb700 1 -- 172.21.15.83:0/493208513 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb57c097be0 con 0x7fb59406adc0 2024-03-11T19:38:10.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1, 8, 11]}]': finished 2024-03-11T19:38:10.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-03-11T19:38:10.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-11T19:38:10.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-11T19:38:10.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:10 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-11T19:38:10.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:10 smithi204 ceph-mon[36739]: osdmap e86: 12 total, 12 up, 12 in 2024-03-11T19:38:10.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1, 8, 11]}]': finished 2024-03-11T19:38:10.458 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-03-11T19:38:10.458 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-11T19:38:10.458 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-11T19:38:10.458 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:10 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-11T19:38:10.458 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:10 smithi083 ceph-mon[31864]: osdmap e86: 12 total, 12 up, 12 in 2024-03-11T19:38:10.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.453+0000 7fb59acb5700 1 -- 172.21.15.83:0/493208513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb5941994b0 con 0x7fb59406adc0 2024-03-11T19:38:10.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.454+0000 7fb591ffb700 1 -- 172.21.15.83:0/493208513 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb57c027090 con 0x7fb59406adc0 2024-03-11T19:38:10.459 INFO:teuthology.orchestra.run.smithi083.stdout:197568495667 2024-03-11T19:38:10.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.456+0000 7fb59acb5700 1 -- 172.21.15.83:0/493208513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb5800a1b50 msgr2=0x7fb5800a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.456+0000 7fb59acb5700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb5800a1b50 0x7fb5800a4010 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb584009550 tx=0x7fb5840091b0 comp rx=0 tx=0).stop 2024-03-11T19:38:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.456+0000 7fb59acb5700 1 -- 172.21.15.83:0/493208513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb59406adc0 msgr2=0x7fb594198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.456+0000 7fb59acb5700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb59406adc0 0x7fb594198690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0098e0 tx=0x7fb57c00f690 comp rx=0 tx=0).stop 2024-03-11T19:38:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.457+0000 7fb59acb5700 1 -- 172.21.15.83:0/493208513 shutdown_connections 2024-03-11T19:38:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.457+0000 7fb59acb5700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb5800a1b50 0x7fb5800a4010 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.457+0000 7fb59acb5700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb594106f80 0x7fb59419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.457+0000 7fb59acb5700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb59406a410 0x7fb594198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.457+0000 7fb59acb5700 1 --2- 172.21.15.83:0/493208513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb59406adc0 0x7fb594198690 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.457+0000 7fb59acb5700 1 -- 172.21.15.83:0/493208513 >> 172.21.15.83:0/493208513 conn(0x7fb594076bc0 msgr2=0x7fb5940febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.457+0000 7fb59acb5700 1 -- 172.21.15.83:0/493208513 shutdown_connections 2024-03-11T19:38:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:10.457+0000 7fb59acb5700 1 -- 172.21.15.83:0/493208513 wait complete. 2024-03-11T19:38:10.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1, 8, 11]}]': finished 2024-03-11T19:38:10.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4, 8, 11]}]': finished 2024-03-11T19:38:10.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-11T19:38:10.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-11T19:38:10.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:10 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-11T19:38:10.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:10 smithi137 ceph-mon[36220]: osdmap e86: 12 total, 12 up, 12 in 2024-03-11T19:38:11.213 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:11 smithi083 ceph-mon[31864]: osdmap e87: 12 total, 12 up, 12 in 2024-03-11T19:38:11.213 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:11 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/493208513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-11T19:38:11.213 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:11 smithi083 ceph-mon[31864]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-11T19:38:11.244 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738440 got 34359738447 for osd.0 2024-03-11T19:38:11.245 DEBUG:teuthology.parallel:result is None 2024-03-11T19:38:11.594 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495667 for osd.6 2024-03-11T19:38:11.594 DEBUG:teuthology.parallel:result is None 2024-03-11T19:38:11.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:11 smithi137 ceph-mon[36220]: osdmap e87: 12 total, 12 up, 12 in 2024-03-11T19:38:11.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:11 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/493208513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-11T19:38:11.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:11 smithi137 ceph-mon[36220]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-11T19:38:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:11 smithi204 ceph-mon[36739]: osdmap e87: 12 total, 12 up, 12 in 2024-03-11T19:38:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:11 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/493208513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-11T19:38:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:11 smithi204 ceph-mon[36739]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-11T19:38:11.788 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:12.226 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411398 for osd.2 2024-03-11T19:38:12.226 DEBUG:teuthology.parallel:result is None 2024-03-11T19:38:12.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.765+0000 7f9a8ede6700 1 -- 172.21.15.83:0/1303254176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88101cc0 msgr2=0x7f9a88102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:12.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.765+0000 7f9a8ede6700 1 --2- 172.21.15.83:0/1303254176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88101cc0 0x7f9a88102140 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9a780099c0 tx=0x7f9a78009cd0 comp rx=0 tx=0).stop 2024-03-11T19:38:12.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.765+0000 7f9a8ede6700 1 -- 172.21.15.83:0/1303254176 shutdown_connections 2024-03-11T19:38:12.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.765+0000 7f9a8ede6700 1 --2- 172.21.15.83:0/1303254176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a88102680 0x7f9a8810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:12.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.765+0000 7f9a8ede6700 1 --2- 172.21.15.83:0/1303254176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88107ca0 0x7f9a88108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:12.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.765+0000 7f9a8ede6700 1 --2- 172.21.15.83:0/1303254176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88101cc0 0x7f9a88102140 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:12.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.765+0000 7f9a8ede6700 1 -- 172.21.15.83:0/1303254176 >> 172.21.15.83:0/1303254176 conn(0x7f9a88078da0 msgr2=0x7f9a880791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:12.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.765+0000 7f9a8ede6700 1 -- 172.21.15.83:0/1303254176 shutdown_connections 2024-03-11T19:38:12.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.765+0000 7f9a8ede6700 1 -- 172.21.15.83:0/1303254176 wait complete. 2024-03-11T19:38:12.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.766+0000 7f9a8ede6700 1 Processor -- start 2024-03-11T19:38:12.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.766+0000 7f9a8ede6700 1 -- start start 2024-03-11T19:38:12.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.766+0000 7f9a8ede6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88101cc0 0x7f9a88073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:12.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.766+0000 7f9a8ede6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88102680 0x7f9a880744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:12.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.766+0000 7f9a8ede6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a88107ca0 0x7f9a880749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:12.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.766+0000 7f9a8ede6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a881a6520 con 0x7f9a88102680 2024-03-11T19:38:12.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.766+0000 7f9a8ede6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a881a6690 con 0x7f9a88101cc0 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.766+0000 7f9a8ede6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a881a6850 con 0x7f9a88107ca0 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.766+0000 7f9a87fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88102680 0x7f9a880744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a87fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88102680 0x7f9a880744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45384/0 (socket says 172.21.15.83:45384) 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a87fff700 1 -- 172.21.15.83:0/2200892882 learned_addr learned my addr 172.21.15.83:0/2200892882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a8d383700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a88107ca0 0x7f9a880749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a8cb82700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88101cc0 0x7f9a88073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a87fff700 1 -- 172.21.15.83:0/2200892882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a88107ca0 msgr2=0x7f9a880749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a87fff700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a88107ca0 0x7f9a880749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a87fff700 1 -- 172.21.15.83:0/2200892882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88101cc0 msgr2=0x7f9a88073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a87fff700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88101cc0 0x7f9a88073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a87fff700 1 -- 172.21.15.83:0/2200892882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780095f0 con 0x7f9a88102680 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a8d383700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a88107ca0 0x7f9a880749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:12.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a8cb82700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88101cc0 0x7f9a88073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a87fff700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88102680 0x7f9a880744b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9a780098e0 tx=0x7f9a780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a85ffb700 1 -- 172.21.15.83:0/2200892882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7801d070 con 0x7f9a88102680 2024-03-11T19:38:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a85ffb700 1 -- 172.21.15.83:0/2200892882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a78004050 con 0x7f9a88102680 2024-03-11T19:38:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a8ede6700 1 -- 172.21.15.83:0/2200892882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a881a6ad0 con 0x7f9a88102680 2024-03-11T19:38:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a85ffb700 1 -- 172.21.15.83:0/2200892882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7800f460 con 0x7f9a88102680 2024-03-11T19:38:12.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.767+0000 7f9a8ede6700 1 -- 172.21.15.83:0/2200892882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a881a6ee0 con 0x7f9a88102680 2024-03-11T19:38:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.769+0000 7f9a8ede6700 1 -- 172.21.15.83:0/2200892882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8810a180 con 0x7f9a88102680 2024-03-11T19:38:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.770+0000 7f9a85ffb700 1 -- 172.21.15.83:0/2200892882 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f9a78022690 con 0x7f9a88102680 2024-03-11T19:38:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.770+0000 7f9a85ffb700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a740a1b50 0x7f9a740a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.770+0000 7f9a8cb82700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a740a1b50 0x7f9a740a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.770+0000 7f9a85ffb700 1 -- 172.21.15.83:0/2200892882 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9a780ce7f0 con 0x7f9a88102680 2024-03-11T19:38:12.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.771+0000 7f9a8cb82700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a740a1b50 0x7f9a740a4010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009fa0 tx=0x7f9a80009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:12.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.775+0000 7f9a85ffb700 1 -- 172.21.15.83:0/2200892882 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a78097a40 con 0x7f9a88102680 2024-03-11T19:38:12.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.975+0000 7f9a8ede6700 1 -- 172.21.15.83:0/2200892882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9a88111d30 con 0x7f9a88102680 2024-03-11T19:38:12.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.975+0000 7f9a85ffb700 1 -- 172.21.15.83:0/2200892882 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a7809b100 con 0x7f9a88102680 2024-03-11T19:38:12.980 INFO:teuthology.orchestra.run.smithi083.stdout:107374182466 2024-03-11T19:38:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.977+0000 7f9a8ede6700 1 -- 172.21.15.83:0/2200892882 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a740a1b50 msgr2=0x7f9a740a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.977+0000 7f9a8ede6700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a740a1b50 0x7f9a740a4010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009fa0 tx=0x7f9a80009570 comp rx=0 tx=0).stop 2024-03-11T19:38:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.977+0000 7f9a8ede6700 1 -- 172.21.15.83:0/2200892882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88102680 msgr2=0x7f9a880744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.977+0000 7f9a8ede6700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88102680 0x7f9a880744b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9a780098e0 tx=0x7f9a780057f0 comp rx=0 tx=0).stop 2024-03-11T19:38:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.977+0000 7f9a8ede6700 1 -- 172.21.15.83:0/2200892882 shutdown_connections 2024-03-11T19:38:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.977+0000 7f9a8ede6700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a740a1b50 0x7f9a740a4010 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.977+0000 7f9a8ede6700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a88107ca0 0x7f9a880749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.977+0000 7f9a8ede6700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88101cc0 0x7f9a88073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.977+0000 7f9a8ede6700 1 --2- 172.21.15.83:0/2200892882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88102680 0x7f9a880744b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.977+0000 7f9a8ede6700 1 -- 172.21.15.83:0/2200892882 >> 172.21.15.83:0/2200892882 conn(0x7f9a88078da0 msgr2=0x7f9a881007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.977+0000 7f9a8ede6700 1 -- 172.21.15.83:0/2200892882 shutdown_connections 2024-03-11T19:38:12.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:12.978+0000 7f9a8ede6700 1 -- 172.21.15.83:0/2200892882 wait complete. 2024-03-11T19:38:13.841 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:13 smithi083 ceph-mon[31864]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:13.841 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:13 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2200892882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-11T19:38:13.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:13 smithi137 ceph-mon[36220]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:13.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:13 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2200892882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-11T19:38:13.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:13 smithi204 ceph-mon[36739]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:13.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:13 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2200892882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-11T19:38:14.724 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182466 for osd.3 2024-03-11T19:38:14.724 DEBUG:teuthology.parallel:result is None 2024-03-11T19:38:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.179+0000 7f724abbe700 1 -- 172.21.15.83:0/3225049859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7244106d60 msgr2=0x7f7244107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.179+0000 7f724abbe700 1 --2- 172.21.15.83:0/3225049859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7244106d60 0x7f7244107140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7238009990 tx=0x7f7238009ca0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.179+0000 7f724abbe700 1 -- 172.21.15.83:0/3225049859 shutdown_connections 2024-03-11T19:38:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.179+0000 7f724abbe700 1 --2- 172.21.15.83:0/3225049859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7244101810 0x7f724410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.179+0000 7f724abbe700 1 --2- 172.21.15.83:0/3225049859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7244106d60 0x7f7244107140 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.179+0000 7f724abbe700 1 --2- 172.21.15.83:0/3225049859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7244100e50 0x7f72441012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.179+0000 7f724abbe700 1 -- 172.21.15.83:0/3225049859 >> 172.21.15.83:0/3225049859 conn(0x7f72440779d0 msgr2=0x7f7244077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.179+0000 7f724abbe700 1 -- 172.21.15.83:0/3225049859 shutdown_connections 2024-03-11T19:38:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.179+0000 7f724abbe700 1 -- 172.21.15.83:0/3225049859 wait complete. 2024-03-11T19:38:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724abbe700 1 Processor -- start 2024-03-11T19:38:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724abbe700 1 -- start start 2024-03-11T19:38:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724abbe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7244100e50 0x7f72441972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724abbe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7244101810 0x7f72441977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724abbe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7244106d60 0x7f724419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724abbe700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724419c140 con 0x7f7244106d60 2024-03-11T19:38:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724abbe700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724419c2b0 con 0x7f7244101810 2024-03-11T19:38:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724abbe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724419c470 con 0x7f7244100e50 2024-03-11T19:38:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724915b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7244106d60 0x7f724419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724915b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7244106d60 0x7f724419bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45390/0 (socket says 172.21.15.83:45390) 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724915b700 1 -- 172.21.15.83:0/1107036907 learned_addr learned my addr 172.21.15.83:0/1107036907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f7243fff700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7244101810 0x7f72441977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724895a700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7244100e50 0x7f72441972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.180+0000 7f724915b700 1 -- 172.21.15.83:0/1107036907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7244100e50 msgr2=0x7f72441972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.181+0000 7f724915b700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7244100e50 0x7f72441972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.181+0000 7f724915b700 1 -- 172.21.15.83:0/1107036907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7244101810 msgr2=0x7f72441977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.181+0000 7f724915b700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7244101810 0x7f72441977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.181+0000 7f724915b700 1 -- 172.21.15.83:0/1107036907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72380095f0 con 0x7f7244106d60 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.181+0000 7f724915b700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7244106d60 0x7f724419bb30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f723c00dad0 tx=0x7f723c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.181+0000 7f7241ffb700 1 -- 172.21.15.83:0/1107036907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00e6a0 con 0x7f7244106d60 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.181+0000 7f7241ffb700 1 -- 172.21.15.83:0/1107036907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f723c00ba10 con 0x7f7244106d60 2024-03-11T19:38:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.181+0000 7f7241ffb700 1 -- 172.21.15.83:0/1107036907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c009e90 con 0x7f7244106d60 2024-03-11T19:38:15.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.181+0000 7f724abbe700 1 -- 172.21.15.83:0/1107036907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724419c6f0 con 0x7f7244106d60 2024-03-11T19:38:15.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.181+0000 7f724abbe700 1 -- 172.21.15.83:0/1107036907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724406a580 con 0x7f7244106d60 2024-03-11T19:38:15.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.182+0000 7f724abbe700 1 -- 172.21.15.83:0/1107036907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72441022f0 con 0x7f7244106d60 2024-03-11T19:38:15.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.183+0000 7f7241ffb700 1 -- 172.21.15.83:0/1107036907 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f723c00e800 con 0x7f7244106d60 2024-03-11T19:38:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.183+0000 7f7241ffb700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72340a1b50 0x7f72340a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.184+0000 7f724895a700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72340a1b50 0x7f72340a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.184+0000 7f7241ffb700 1 -- 172.21.15.83:0/1107036907 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f723c006e40 con 0x7f7244106d60 2024-03-11T19:38:15.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.184+0000 7f724895a700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72340a1b50 0x7f72340a4010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7238006010 tx=0x7f723800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.188+0000 7f7241ffb700 1 -- 172.21.15.83:0/1107036907 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f723c050d20 con 0x7f7244106d60 2024-03-11T19:38:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.307+0000 7f40cff03700 1 -- 172.21.15.83:0/760237547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c8106ff0 msgr2=0x7f40c810eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.307+0000 7f40cff03700 1 --2- 172.21.15.83:0/760237547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c8106ff0 0x7f40c810eb70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f40c000b0a0 tx=0x7f40c000b3b0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.307+0000 7f40cff03700 1 -- 172.21.15.83:0/760237547 shutdown_connections 2024-03-11T19:38:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.307+0000 7f40cff03700 1 --2- 172.21.15.83:0/760237547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40c806a410 0x7f40c806a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.307+0000 7f40cff03700 1 --2- 172.21.15.83:0/760237547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c8106ff0 0x7f40c810eb70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.307+0000 7f40cff03700 1 --2- 172.21.15.83:0/760237547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c806adc0 0x7f40c81068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.307+0000 7f40cff03700 1 -- 172.21.15.83:0/760237547 >> 172.21.15.83:0/760237547 conn(0x7f40c8076bc0 msgr2=0x7f40c8076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:15.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.307+0000 7f40cff03700 1 -- 172.21.15.83:0/760237547 shutdown_connections 2024-03-11T19:38:15.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.307+0000 7f40cff03700 1 -- 172.21.15.83:0/760237547 wait complete. 2024-03-11T19:38:15.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cff03700 1 Processor -- start 2024-03-11T19:38:15.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cff03700 1 -- start start 2024-03-11T19:38:15.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cff03700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c806a410 0x7f40c81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:15.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cff03700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40c806adc0 0x7f40c81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:15.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cff03700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c8106ff0 0x7f40c819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:15.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cff03700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c819d130 con 0x7f40c806a410 2024-03-11T19:38:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cff03700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c819d2a0 con 0x7f40c8106ff0 2024-03-11T19:38:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cff03700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c819d460 con 0x7f40c806adc0 2024-03-11T19:38:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cdc9f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c806a410 0x7f40c81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cdc9f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c806a410 0x7f40c81982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45410/0 (socket says 172.21.15.83:45410) 2024-03-11T19:38:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cdc9f700 1 -- 172.21.15.83:0/2648819798 learned_addr learned my addr 172.21.15.83:0/2648819798 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40ce4a0700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c8106ff0 0x7f40c819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cd49e700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40c806adc0 0x7f40c81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cdc9f700 1 -- 172.21.15.83:0/2648819798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40c806adc0 msgr2=0x7f40c81987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cdc9f700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40c806adc0 0x7f40c81987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cdc9f700 1 -- 172.21.15.83:0/2648819798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c8106ff0 msgr2=0x7f40c819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:15.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cdc9f700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c8106ff0 0x7f40c819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.308+0000 7f40cdc9f700 1 -- 172.21.15.83:0/2648819798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0009d00 con 0x7f40c806a410 2024-03-11T19:38:15.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.309+0000 7f40cdc9f700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c806a410 0x7f40c81982a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f40b800e9a0 tx=0x7f40b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:15.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.309+0000 7f40beffd700 1 -- 172.21.15.83:0/2648819798 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8009790 con 0x7f40c806a410 2024-03-11T19:38:15.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.309+0000 7f40cff03700 1 -- 172.21.15.83:0/2648819798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c819d740 con 0x7f40c806a410 2024-03-11T19:38:15.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.309+0000 7f40beffd700 1 -- 172.21.15.83:0/2648819798 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40b800cbd0 con 0x7f40c806a410 2024-03-11T19:38:15.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.309+0000 7f40beffd700 1 -- 172.21.15.83:0/2648819798 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8018680 con 0x7f40c806a410 2024-03-11T19:38:15.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.309+0000 7f40cff03700 1 -- 172.21.15.83:0/2648819798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c81a4fa0 con 0x7f40c806a410 2024-03-11T19:38:15.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.310+0000 7f40cff03700 1 -- 172.21.15.83:0/2648819798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c810a1e0 con 0x7f40c806a410 2024-03-11T19:38:15.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.311+0000 7f40beffd700 1 -- 172.21.15.83:0/2648819798 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f40b800cd40 con 0x7f40c806a410 2024-03-11T19:38:15.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.312+0000 7f40beffd700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40b40a1b50 0x7f40b40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:15.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.312+0000 7f40cd49e700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40b40a1b50 0x7f40b40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:15.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.312+0000 7f40beffd700 1 -- 172.21.15.83:0/2648819798 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f40b8014070 con 0x7f40c806a410 2024-03-11T19:38:15.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.313+0000 7f40cd49e700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40b40a1b50 0x7f40b40a4010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005fd0 tx=0x7f40c400af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:15.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.316+0000 7f40beffd700 1 -- 172.21.15.83:0/2648819798 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f40b8096d30 con 0x7f40c806a410 2024-03-11T19:38:15.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.392+0000 7f724abbe700 1 -- 172.21.15.83:0/1107036907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f724419c8e0 con 0x7f7244106d60 2024-03-11T19:38:15.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.392+0000 7f7241ffb700 1 -- 172.21.15.83:0/1107036907 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f723c097a50 con 0x7f7244106d60 2024-03-11T19:38:15.397 INFO:teuthology.orchestra.run.smithi083.stdout:253403070505 2024-03-11T19:38:15.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.394+0000 7f724abbe700 1 -- 172.21.15.83:0/1107036907 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72340a1b50 msgr2=0x7f72340a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:15.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.394+0000 7f724abbe700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72340a1b50 0x7f72340a4010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7238006010 tx=0x7f723800b540 comp rx=0 tx=0).stop 2024-03-11T19:38:15.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.394+0000 7f724abbe700 1 -- 172.21.15.83:0/1107036907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7244106d60 msgr2=0x7f724419bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:15.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.394+0000 7f724abbe700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7244106d60 0x7f724419bb30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f723c00dad0 tx=0x7f723c00dde0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.395+0000 7f724abbe700 1 -- 172.21.15.83:0/1107036907 shutdown_connections 2024-03-11T19:38:15.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.395+0000 7f724abbe700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f72340a1b50 0x7f72340a4010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.395+0000 7f724abbe700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7244100e50 0x7f72441972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.395+0000 7f724abbe700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7244106d60 0x7f724419bb30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.395+0000 7f724abbe700 1 --2- 172.21.15.83:0/1107036907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7244101810 0x7f72441977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.395+0000 7f724abbe700 1 -- 172.21.15.83:0/1107036907 >> 172.21.15.83:0/1107036907 conn(0x7f72440779d0 msgr2=0x7f72440ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:15.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.396+0000 7f724abbe700 1 -- 172.21.15.83:0/1107036907 shutdown_connections 2024-03-11T19:38:15.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.396+0000 7f724abbe700 1 -- 172.21.15.83:0/1107036907 wait complete. 2024-03-11T19:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.518+0000 7f40cff03700 1 -- 172.21.15.83:0/2648819798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f40c8068930 con 0x7f40c806a410 2024-03-11T19:38:15.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.518+0000 7f40beffd700 1 -- 172.21.15.83:0/2648819798 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f40b809a3f0 con 0x7f40c806a410 2024-03-11T19:38:15.523 INFO:teuthology.orchestra.run.smithi083.stdout:219043332141 2024-03-11T19:38:15.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.520+0000 7f40cff03700 1 -- 172.21.15.83:0/2648819798 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40b40a1b50 msgr2=0x7f40b40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.520+0000 7f40cff03700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40b40a1b50 0x7f40b40a4010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005fd0 tx=0x7f40c400af20 comp rx=0 tx=0).stop 2024-03-11T19:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.520+0000 7f40cff03700 1 -- 172.21.15.83:0/2648819798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c806a410 msgr2=0x7f40c81982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.520+0000 7f40cff03700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c806a410 0x7f40c81982a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f40b800e9a0 tx=0x7f40b800ecb0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.520+0000 7f40cff03700 1 -- 172.21.15.83:0/2648819798 shutdown_connections 2024-03-11T19:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.520+0000 7f40cff03700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40b40a1b50 0x7f40b40a4010 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.520+0000 7f40cff03700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c8106ff0 0x7f40c819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.520+0000 7f40cff03700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40c806a410 0x7f40c81982a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.520+0000 7f40cff03700 1 --2- 172.21.15.83:0/2648819798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40c806adc0 0x7f40c81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.520+0000 7f40cff03700 1 -- 172.21.15.83:0/2648819798 >> 172.21.15.83:0/2648819798 conn(0x7f40c8076bc0 msgr2=0x7f40c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.521+0000 7f40cff03700 1 -- 172.21.15.83:0/2648819798 shutdown_connections 2024-03-11T19:38:15.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:15.521+0000 7f40cff03700 1 -- 172.21.15.83:0/2648819798 wait complete. 2024-03-11T19:38:15.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:15 smithi083 ceph-mon[31864]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-03-11T19:38:15.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:15 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1107036907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-11T19:38:15.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:15 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2648819798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-11T19:38:15.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:15 smithi137 ceph-mon[36220]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-03-11T19:38:15.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:15 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1107036907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-11T19:38:15.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:15 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2648819798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-11T19:38:15.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:15 smithi204 ceph-mon[36739]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 13 B/s, 1 objects/s recovering 2024-03-11T19:38:15.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:15 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1107036907' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-11T19:38:15.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:15 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2648819798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-11T19:38:16.041 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070500 got 253403070505 for osd.8 2024-03-11T19:38:16.041 DEBUG:teuthology.parallel:result is None 2024-03-11T19:38:17.285 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332141 for osd.7 2024-03-11T19:38:17.285 DEBUG:teuthology.parallel:result is None 2024-03-11T19:38:17.285 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-11T19:38:17.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-11T19:38:17.566 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:17.867 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:17 smithi083 ceph-mon[31864]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 13 B/s, 1 objects/s recovering 2024-03-11T19:38:17.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:17 smithi137 ceph-mon[36220]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 13 B/s, 1 objects/s recovering 2024-03-11T19:38:17.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:17 smithi204 ceph-mon[36739]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s; 13 B/s, 1 objects/s recovering 2024-03-11T19:38:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.197+0000 7fdb2c488700 1 -- 172.21.15.83:0/2197457477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb24100b70 msgr2=0x7fdb24100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.197+0000 7fdb2c488700 1 --2- 172.21.15.83:0/2197457477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb24100b70 0x7fdb24100ff0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009990 tx=0x7fdb14009ca0 comp rx=0 tx=0).stop 2024-03-11T19:38:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.197+0000 7fdb2c488700 1 -- 172.21.15.83:0/2197457477 shutdown_connections 2024-03-11T19:38:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.197+0000 7fdb2c488700 1 --2- 172.21.15.83:0/2197457477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb24101530 0x7fdb2410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.197+0000 7fdb2c488700 1 --2- 172.21.15.83:0/2197457477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb24106b80 0x7fdb24106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.197+0000 7fdb2c488700 1 --2- 172.21.15.83:0/2197457477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb24100b70 0x7fdb24100ff0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.197+0000 7fdb2c488700 1 -- 172.21.15.83:0/2197457477 >> 172.21.15.83:0/2197457477 conn(0x7fdb24077aa0 msgr2=0x7fdb24077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.197+0000 7fdb2c488700 1 -- 172.21.15.83:0/2197457477 shutdown_connections 2024-03-11T19:38:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.197+0000 7fdb2c488700 1 -- 172.21.15.83:0/2197457477 wait complete. 2024-03-11T19:38:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2c488700 1 Processor -- start 2024-03-11T19:38:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2c488700 1 -- start start 2024-03-11T19:38:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2c488700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb24100b70 0x7fdb241029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2c488700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb24101530 0x7fdb24102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2c488700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb24106b80 0x7fdb24103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2c488700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb241a64f0 con 0x7fdb24100b70 2024-03-11T19:38:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2c488700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb241a6660 con 0x7fdb24101530 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2c488700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb241a6820 con 0x7fdb24106b80 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2a224700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb24100b70 0x7fdb241029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2a224700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb24100b70 0x7fdb241029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45424/0 (socket says 172.21.15.83:45424) 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2a224700 1 -- 172.21.15.83:0/1399415998 learned_addr learned my addr 172.21.15.83:0/1399415998 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb29a23700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb24101530 0x7fdb24102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2aa25700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb24106b80 0x7fdb24103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2a224700 1 -- 172.21.15.83:0/1399415998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb24106b80 msgr2=0x7fdb24103460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2a224700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb24106b80 0x7fdb24103460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2a224700 1 -- 172.21.15.83:0/1399415998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb24101530 msgr2=0x7fdb24102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2a224700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb24101530 0x7fdb24102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.198+0000 7fdb2a224700 1 -- 172.21.15.83:0/1399415998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb140095f0 con 0x7fdb24100b70 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.199+0000 7fdb2a224700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb24100b70 0x7fdb241029e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c007e20 tx=0x7fdb1c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.199+0000 7fdb1b7fe700 1 -- 172.21.15.83:0/1399415998 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00ac90 con 0x7fdb24100b70 2024-03-11T19:38:19.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.199+0000 7fdb2c488700 1 -- 172.21.15.83:0/1399415998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb241a6b00 con 0x7fdb24100b70 2024-03-11T19:38:19.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.199+0000 7fdb1b7fe700 1 -- 172.21.15.83:0/1399415998 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb1c00adf0 con 0x7fdb24100b70 2024-03-11T19:38:19.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.199+0000 7fdb1b7fe700 1 -- 172.21.15.83:0/1399415998 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0117a0 con 0x7fdb24100b70 2024-03-11T19:38:19.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.199+0000 7fdb2c488700 1 -- 172.21.15.83:0/1399415998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb241a6fd0 con 0x7fdb24100b70 2024-03-11T19:38:19.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.200+0000 7fdb2c488700 1 -- 172.21.15.83:0/1399415998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb24068930 con 0x7fdb24100b70 2024-03-11T19:38:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.201+0000 7fdb1b7fe700 1 -- 172.21.15.83:0/1399415998 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fdb1c012420 con 0x7fdb24100b70 2024-03-11T19:38:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.201+0000 7fdb1b7fe700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb100a1b50 0x7fdb100a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.201+0000 7fdb29a23700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb100a1b50 0x7fdb100a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.201+0000 7fdb1b7fe700 1 -- 172.21.15.83:0/1399415998 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdb1c0ccf90 con 0x7fdb24100b70 2024-03-11T19:38:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.202+0000 7fdb29a23700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb100a1b50 0x7fdb100a4010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400b5c0 tx=0x7fdb14005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.206+0000 7fdb1b7fe700 1 -- 172.21.15.83:0/1399415998 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb1c096260 con 0x7fdb24100b70 2024-03-11T19:38:19.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:19.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:19.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:19 smithi083 ceph-mon[31864]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 11 B/s, 1 objects/s recovering 2024-03-11T19:38:19.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:38:19.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:38:19.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:19.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.397+0000 7fdb2c488700 1 -- 172.21.15.83:0/1399415998 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb24110b90 con 0x7fdb100a1b50 2024-03-11T19:38:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.404+0000 7fdb1b7fe700 1 -- 172.21.15.83:0/1399415998 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258720 (secure 0 0 0) 0x7fdb24110b90 con 0x7fdb100a1b50 2024-03-11T19:38:19.412 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 -- 172.21.15.83:0/1399415998 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb100a1b50 msgr2=0x7fdb100a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb100a1b50 0x7fdb100a4010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400b5c0 tx=0x7fdb14005960 comp rx=0 tx=0).stop 2024-03-11T19:38:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 -- 172.21.15.83:0/1399415998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb24100b70 msgr2=0x7fdb241029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb24100b70 0x7fdb241029e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c007e20 tx=0x7fdb1c00a340 comp rx=0 tx=0).stop 2024-03-11T19:38:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 -- 172.21.15.83:0/1399415998 shutdown_connections 2024-03-11T19:38:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb100a1b50 0x7fdb100a4010 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb24106b80 0x7fdb24103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:19.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb24100b70 0x7fdb241029e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 --2- 172.21.15.83:0/1399415998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb24101530 0x7fdb24102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 -- 172.21.15.83:0/1399415998 >> 172.21.15.83:0/1399415998 conn(0x7fdb24077aa0 msgr2=0x7fdb24104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 -- 172.21.15.83:0/1399415998 shutdown_connections 2024-03-11T19:38:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:19.407+0000 7fdb2c488700 1 -- 172.21.15.83:0/1399415998 wait complete. 2024-03-11T19:38:19.415 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-03-11T19:38:19.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:19.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:19.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:19 smithi137 ceph-mon[36220]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 11 B/s, 1 objects/s recovering 2024-03-11T19:38:19.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:38:19.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:38:19.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:19.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:19.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:19.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:19 smithi204 ceph-mon[36739]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 11 B/s, 1 objects/s recovering 2024-03-11T19:38:19.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:38:19.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:38:19.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:38:20.049 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-03-11T19:38:18.525391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655017,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":560,"num_read_kb":394,"num_write":341,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366544,"kb_used_data":13164,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575280,"statfs":{"total":1151940427776,"available":1151565086720,"internally_reserved":0,"allocated":13479936,"data_stored":7425051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":27,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.383600"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152898+0000","last_change":"2024-03-11T19:37:05.340476+0000","last_active":"2024-03-11T19:38:10.152898+0000","last_peered":"2024-03-11T19:38:10.152898+0000","last_clean":"2024-03-11T19:38:10.152898+0000","last_became_active":"2024-03-11T19:37:05.340140+0000","last_became_peered":"2024-03-11T19:37:05.340140+0000","last_unstale":"2024-03-11T19:38:10.152898+0000","last_undegraded":"2024-03-11T19:38:10.152898+0000","last_fullsized":"2024-03-11T19:38:10.152898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:10:35.677374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152679+0000","last_change":"2024-03-11T19:37:05.335500+0000","last_active":"2024-03-11T19:38:10.152679+0000","last_peered":"2024-03-11T19:38:10.152679+0000","last_clean":"2024-03-11T19:38:10.152679+0000","last_became_active":"2024-03-11T19:37:05.335194+0000","last_became_peered":"2024-03-11T19:37:05.335194+0000","last_unstale":"2024-03-11T19:38:10.152679+0000","last_undegraded":"2024-03-11T19:38:10.152679+0000","last_fullsized":"2024-03-11T19:38:10.152679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:03.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152345+0000","last_change":"2024-03-11T19:37:05.334618+0000","last_active":"2024-03-11T19:38:10.152345+0000","last_peered":"2024-03-11T19:38:10.152345+0000","last_clean":"2024-03-11T19:38:10.152345+0000","last_became_active":"2024-03-11T19:37:05.334281+0000","last_became_peered":"2024-03-11T19:37:05.334281+0000","last_unstale":"2024-03-11T19:38:10.152345+0000","last_undegraded":"2024-03-11T19:38:10.152345+0000","last_fullsized":"2024-03-11T19:38:10.152345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:43:22.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453952+0000","last_change":"2024-03-11T19:37:05.332978+0000","last_active":"2024-03-11T19:38:10.453952+0000","last_peered":"2024-03-11T19:38:10.453952+0000","last_clean":"2024-03-11T19:38:10.453952+0000","last_became_active":"2024-03-11T19:37:05.330888+0000","last_became_peered":"2024-03-11T19:37:05.330888+0000","last_unstale":"2024-03-11T19:38:10.453952+0000","last_undegraded":"2024-03-11T19:38:10.453952+0000","last_fullsized":"2024-03-11T19:38:10.453952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:40:36.988348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151425+0000","last_change":"2024-03-11T19:37:05.329050+0000","last_active":"2024-03-11T19:38:10.151425+0000","last_peered":"2024-03-11T19:38:10.151425+0000","last_clean":"2024-03-11T19:38:10.151425+0000","last_became_active":"2024-03-11T19:37:05.328702+0000","last_became_peered":"2024-03-11T19:37:05.328702+0000","last_unstale":"2024-03-11T19:38:10.151425+0000","last_undegraded":"2024-03-11T19:38:10.151425+0000","last_fullsized":"2024-03-11T19:38:10.151425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:19:52.180884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452959+0000","last_change":"2024-03-11T19:37:05.322748+0000","last_active":"2024-03-11T19:38:10.452959+0000","last_peered":"2024-03-11T19:38:10.452959+0000","last_clean":"2024-03-11T19:38:10.452959+0000","last_became_active":"2024-03-11T19:37:05.322376+0000","last_became_peered":"2024-03-11T19:37:05.322376+0000","last_unstale":"2024-03-11T19:38:10.452959+0000","last_undegraded":"2024-03-11T19:38:10.452959+0000","last_fullsized":"2024-03-11T19:38:10.452959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:40.926854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150130+0000","last_change":"2024-03-11T19:37:05.323954+0000","last_active":"2024-03-11T19:38:10.150130+0000","last_peered":"2024-03-11T19:38:10.150130+0000","last_clean":"2024-03-11T19:38:10.150130+0000","last_became_active":"2024-03-11T19:37:05.323586+0000","last_became_peered":"2024-03-11T19:37:05.323586+0000","last_unstale":"2024-03-11T19:38:10.150130+0000","last_undegraded":"2024-03-11T19:38:10.150130+0000","last_fullsized":"2024-03-11T19:38:10.150130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:38.286891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452465+0000","last_change":"2024-03-11T19:37:05.334138+0000","last_active":"2024-03-11T19:38:10.452465+0000","last_peered":"2024-03-11T19:38:10.452465+0000","last_clean":"2024-03-11T19:38:10.452465+0000","last_became_active":"2024-03-11T19:37:05.333837+0000","last_became_peered":"2024-03-11T19:37:05.333837+0000","last_unstale":"2024-03-11T19:38:10.452465+0000","last_undegraded":"2024-03-11T19:38:10.452465+0000","last_fullsized":"2024-03-11T19:38:10.452465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:49:30.724176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448684+0000","last_change":"2024-03-11T19:37:05.336888+0000","last_active":"2024-03-11T19:38:10.448684+0000","last_peered":"2024-03-11T19:38:10.448684+0000","last_clean":"2024-03-11T19:38:10.448684+0000","last_became_active":"2024-03-11T19:37:05.336256+0000","last_became_peered":"2024-03-11T19:37:05.336256+0000","last_unstale":"2024-03-11T19:38:10.448684+0000","last_undegraded":"2024-03-11T19:38:10.448684+0000","last_fullsized":"2024-03-11T19:38:10.448684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:54:31.526986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453438+0000","last_change":"2024-03-11T19:37:05.321959+0000","last_active":"2024-03-11T19:38:10.453438+0000","last_peered":"2024-03-11T19:38:10.453438+0000","last_clean":"2024-03-11T19:38:10.453438+0000","last_became_active":"2024-03-11T19:37:05.321043+0000","last_became_peered":"2024-03-11T19:37:05.321043+0000","last_unstale":"2024-03-11T19:38:10.453438+0000","last_undegraded":"2024-03-11T19:38:10.453438+0000","last_fullsized":"2024-03-11T19:38:10.453438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:13.121003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150663+0000","last_change":"2024-03-11T19:37:05.324940+0000","last_active":"2024-03-11T19:38:10.150663+0000","last_peered":"2024-03-11T19:38:10.150663+0000","last_clean":"2024-03-11T19:38:10.150663+0000","last_became_active":"2024-03-11T19:37:05.323427+0000","last_became_peered":"2024-03-11T19:37:05.323427+0000","last_unstale":"2024-03-11T19:38:10.150663+0000","last_undegraded":"2024-03-11T19:38:10.150663+0000","last_fullsized":"2024-03-11T19:38:10.150663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:23.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453851+0000","last_change":"2024-03-11T19:37:05.332694+0000","last_active":"2024-03-11T19:38:10.453851+0000","last_peered":"2024-03-11T19:38:10.453851+0000","last_clean":"2024-03-11T19:38:10.453851+0000","last_became_active":"2024-03-11T19:37:05.330446+0000","last_became_peered":"2024-03-11T19:37:05.330446+0000","last_unstale":"2024-03-11T19:38:10.453851+0000","last_undegraded":"2024-03-11T19:38:10.453851+0000","last_fullsized":"2024-03-11T19:38:10.453851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:37:40.196933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453284+0000","last_change":"2024-03-11T19:37:05.325975+0000","last_active":"2024-03-11T19:38:10.453284+0000","last_peered":"2024-03-11T19:38:10.453284+0000","last_clean":"2024-03-11T19:38:10.453284+0000","last_became_active":"2024-03-11T19:37:05.325649+0000","last_became_peered":"2024-03-11T19:37:05.325649+0000","last_unstale":"2024-03-11T19:38:10.453284+0000","last_undegraded":"2024-03-11T19:38:10.453284+0000","last_fullsized":"2024-03-11T19:38:10.453284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:36.373272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152272+0000","last_change":"2024-03-11T19:37:05.335546+0000","last_active":"2024-03-11T19:38:10.152272+0000","last_peered":"2024-03-11T19:38:10.152272+0000","last_clean":"2024-03-11T19:38:10.152272+0000","last_became_active":"2024-03-11T19:37:05.335220+0000","last_became_peered":"2024-03-11T19:37:05.335220+0000","last_unstale":"2024-03-11T19:38:10.152272+0000","last_undegraded":"2024-03-11T19:38:10.152272+0000","last_fullsized":"2024-03-11T19:38:10.152272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:13.116355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152291+0000","last_change":"2024-03-11T19:37:05.338955+0000","last_active":"2024-03-11T19:38:10.152291+0000","last_peered":"2024-03-11T19:38:10.152291+0000","last_clean":"2024-03-11T19:38:10.152291+0000","last_became_active":"2024-03-11T19:37:05.338574+0000","last_became_peered":"2024-03-11T19:37:05.338574+0000","last_unstale":"2024-03-11T19:38:10.152291+0000","last_undegraded":"2024-03-11T19:38:10.152291+0000","last_fullsized":"2024-03-11T19:38:10.152291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:27:27.754077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448190+0000","last_change":"2024-03-11T19:37:05.334578+0000","last_active":"2024-03-11T19:38:10.448190+0000","last_peered":"2024-03-11T19:38:10.448190+0000","last_clean":"2024-03-11T19:38:10.448190+0000","last_became_active":"2024-03-11T19:37:05.334267+0000","last_became_peered":"2024-03-11T19:37:05.334267+0000","last_unstale":"2024-03-11T19:38:10.448190+0000","last_undegraded":"2024-03-11T19:38:10.448190+0000","last_fullsized":"2024-03-11T19:38:10.448190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:31.704187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152760+0000","last_change":"2024-03-11T19:37:05.341178+0000","last_active":"2024-03-11T19:38:10.152760+0000","last_peered":"2024-03-11T19:38:10.152760+0000","last_clean":"2024-03-11T19:38:10.152760+0000","last_became_active":"2024-03-11T19:37:05.340837+0000","last_became_peered":"2024-03-11T19:37:05.340837+0000","last_unstale":"2024-03-11T19:38:10.152760+0000","last_undegraded":"2024-03-11T19:38:10.152760+0000","last_fullsized":"2024-03-11T19:38:10.152760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:39:54.460937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454128+0000","last_change":"2024-03-11T19:37:05.330981+0000","last_active":"2024-03-11T19:38:10.454128+0000","last_peered":"2024-03-11T19:38:10.454128+0000","last_clean":"2024-03-11T19:38:10.454128+0000","last_became_active":"2024-03-11T19:37:05.330638+0000","last_became_peered":"2024-03-11T19:37:05.330638+0000","last_unstale":"2024-03-11T19:38:10.454128+0000","last_undegraded":"2024-03-11T19:38:10.454128+0000","last_fullsized":"2024-03-11T19:38:10.454128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:36:12.565936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155394+0000","last_change":"2024-03-11T19:37:05.329866+0000","last_active":"2024-03-11T19:38:10.155394+0000","last_peered":"2024-03-11T19:38:10.155394+0000","last_clean":"2024-03-11T19:38:10.155394+0000","last_became_active":"2024-03-11T19:37:05.329572+0000","last_became_peered":"2024-03-11T19:37:05.329572+0000","last_unstale":"2024-03-11T19:38:10.155394+0000","last_undegraded":"2024-03-11T19:38:10.155394+0000","last_fullsized":"2024-03-11T19:38:10.155394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:40:04.482929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454096+0000","last_change":"2024-03-11T19:37:04.587149+0000","last_active":"2024-03-11T19:38:10.454096+0000","last_peered":"2024-03-11T19:38:10.454096+0000","last_clean":"2024-03-11T19:38:10.454096+0000","last_became_active":"2024-03-11T19:37:04.586013+0000","last_became_peered":"2024-03-11T19:37:04.586013+0000","last_unstale":"2024-03-11T19:38:10.454096+0000","last_undegraded":"2024-03-11T19:38:10.454096+0000","last_fullsized":"2024-03-11T19:38:10.454096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:15.758154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452775+0000","last_change":"2024-03-11T19:37:05.327979+0000","last_active":"2024-03-11T19:38:10.452775+0000","last_peered":"2024-03-11T19:38:10.452775+0000","last_clean":"2024-03-11T19:38:10.452775+0000","last_became_active":"2024-03-11T19:37:05.327647+0000","last_became_peered":"2024-03-11T19:37:05.327647+0000","last_unstale":"2024-03-11T19:38:10.452775+0000","last_undegraded":"2024-03-11T19:38:10.452775+0000","last_fullsized":"2024-03-11T19:38:10.452775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:09.514766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452950+0000","last_change":"2024-03-11T19:37:04.582937+0000","last_active":"2024-03-11T19:38:10.452950+0000","last_peered":"2024-03-11T19:38:10.452950+0000","last_clean":"2024-03-11T19:38:10.452950+0000","last_became_active":"2024-03-11T19:37:04.582622+0000","last_became_peered":"2024-03-11T19:37:04.582622+0000","last_unstale":"2024-03-11T19:38:10.452950+0000","last_undegraded":"2024-03-11T19:38:10.452950+0000","last_fullsized":"2024-03-11T19:38:10.452950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:02.275698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152466+0000","last_change":"2024-03-11T19:37:05.327191+0000","last_active":"2024-03-11T19:38:10.152466+0000","last_peered":"2024-03-11T19:38:10.152466+0000","last_clean":"2024-03-11T19:38:10.152466+0000","last_became_active":"2024-03-11T19:37:05.326001+0000","last_became_peered":"2024-03-11T19:37:05.326001+0000","last_unstale":"2024-03-11T19:38:10.152466+0000","last_undegraded":"2024-03-11T19:38:10.152466+0000","last_fullsized":"2024-03-11T19:38:10.152466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:27.301321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151969+0000","last_change":"2024-03-11T19:37:04.587054+0000","last_active":"2024-03-11T19:38:10.151969+0000","last_peered":"2024-03-11T19:38:10.151969+0000","last_clean":"2024-03-11T19:38:10.151969+0000","last_became_active":"2024-03-11T19:37:04.586104+0000","last_became_peered":"2024-03-11T19:37:04.586104+0000","last_unstale":"2024-03-11T19:38:10.151969+0000","last_undegraded":"2024-03-11T19:38:10.151969+0000","last_fullsized":"2024-03-11T19:38:10.151969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:34.913677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448312+0000","last_change":"2024-03-11T19:37:05.333288+0000","last_active":"2024-03-11T19:38:10.448312+0000","last_peered":"2024-03-11T19:38:10.448312+0000","last_clean":"2024-03-11T19:38:10.448312+0000","last_became_active":"2024-03-11T19:37:05.332882+0000","last_became_peered":"2024-03-11T19:37:05.332882+0000","last_unstale":"2024-03-11T19:38:10.448312+0000","last_undegraded":"2024-03-11T19:38:10.448312+0000","last_fullsized":"2024-03-11T19:38:10.448312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:45.680608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151628+0000","last_change":"2024-03-11T19:37:04.583660+0000","last_active":"2024-03-11T19:38:10.151628+0000","last_peered":"2024-03-11T19:38:10.151628+0000","last_clean":"2024-03-11T19:38:10.151628+0000","last_became_active":"2024-03-11T19:37:04.583360+0000","last_became_peered":"2024-03-11T19:37:04.583360+0000","last_unstale":"2024-03-11T19:38:10.151628+0000","last_undegraded":"2024-03-11T19:38:10.151628+0000","last_fullsized":"2024-03-11T19:38:10.151628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:00.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453554+0000","last_change":"2024-03-11T19:37:05.335649+0000","last_active":"2024-03-11T19:38:10.453554+0000","last_peered":"2024-03-11T19:38:10.453554+0000","last_clean":"2024-03-11T19:38:10.453554+0000","last_became_active":"2024-03-11T19:37:05.335255+0000","last_became_peered":"2024-03-11T19:37:05.335255+0000","last_unstale":"2024-03-11T19:38:10.453554+0000","last_undegraded":"2024-03-11T19:38:10.453554+0000","last_fullsized":"2024-03-11T19:38:10.453554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:49:09.772765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:13.723195+0000","last_change":"2024-03-11T19:37:04.586002+0000","last_active":"2024-03-11T19:38:13.723195+0000","last_peered":"2024-03-11T19:38:13.723195+0000","last_clean":"2024-03-11T19:38:13.723195+0000","last_became_active":"2024-03-11T19:37:04.585551+0000","last_became_peered":"2024-03-11T19:37:04.585551+0000","last_unstale":"2024-03-11T19:38:13.723195+0000","last_undegraded":"2024-03-11T19:38:13.723195+0000","last_fullsized":"2024-03-11T19:38:13.723195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:34:12.663801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151614+0000","last_change":"2024-03-11T19:37:05.325168+0000","last_active":"2024-03-11T19:38:10.151614+0000","last_peered":"2024-03-11T19:38:10.151614+0000","last_clean":"2024-03-11T19:38:10.151614+0000","last_became_active":"2024-03-11T19:37:05.324848+0000","last_became_peered":"2024-03-11T19:37:05.324848+0000","last_unstale":"2024-03-11T19:38:10.151614+0000","last_undegraded":"2024-03-11T19:38:10.151614+0000","last_fullsized":"2024-03-11T19:38:10.151614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:27.423877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452883+0000","last_change":"2024-03-11T19:37:04.582727+0000","last_active":"2024-03-11T19:38:10.452883+0000","last_peered":"2024-03-11T19:38:10.452883+0000","last_clean":"2024-03-11T19:38:10.452883+0000","last_became_active":"2024-03-11T19:37:04.582359+0000","last_became_peered":"2024-03-11T19:37:04.582359+0000","last_unstale":"2024-03-11T19:38:10.452883+0000","last_undegraded":"2024-03-11T19:38:10.452883+0000","last_fullsized":"2024-03-11T19:38:10.452883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:05.591537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453512+0000","last_change":"2024-03-11T19:37:05.335442+0000","last_active":"2024-03-11T19:38:10.453512+0000","last_peered":"2024-03-11T19:38:10.453512+0000","last_clean":"2024-03-11T19:38:10.453512+0000","last_became_active":"2024-03-11T19:37:05.335003+0000","last_became_peered":"2024-03-11T19:37:05.335003+0000","last_unstale":"2024-03-11T19:38:10.453512+0000","last_undegraded":"2024-03-11T19:38:10.453512+0000","last_fullsized":"2024-03-11T19:38:10.453512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:41.590898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453615+0000","last_change":"2024-03-11T19:37:04.589957+0000","last_active":"2024-03-11T19:38:10.453615+0000","last_peered":"2024-03-11T19:38:10.453615+0000","last_clean":"2024-03-11T19:38:10.453615+0000","last_became_active":"2024-03-11T19:37:04.589539+0000","last_became_peered":"2024-03-11T19:37:04.589539+0000","last_unstale":"2024-03-11T19:38:10.453615+0000","last_undegraded":"2024-03-11T19:38:10.453615+0000","last_fullsized":"2024-03-11T19:38:10.453615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:48:52.764057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452397+0000","last_change":"2024-03-11T19:37:05.331380+0000","last_active":"2024-03-11T19:38:10.452397+0000","last_peered":"2024-03-11T19:38:10.452397+0000","last_clean":"2024-03-11T19:38:10.452397+0000","last_became_active":"2024-03-11T19:37:05.331000+0000","last_became_peered":"2024-03-11T19:37:05.331000+0000","last_unstale":"2024-03-11T19:38:10.452397+0000","last_undegraded":"2024-03-11T19:38:10.452397+0000","last_fullsized":"2024-03-11T19:38:10.452397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:58:47.278195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452916+0000","last_change":"2024-03-11T19:37:04.584128+0000","last_active":"2024-03-11T19:38:10.452916+0000","last_peered":"2024-03-11T19:38:10.452916+0000","last_clean":"2024-03-11T19:38:10.452916+0000","last_became_active":"2024-03-11T19:37:04.583842+0000","last_became_peered":"2024-03-11T19:37:04.583842+0000","last_unstale":"2024-03-11T19:38:10.452916+0000","last_undegraded":"2024-03-11T19:38:10.452916+0000","last_fullsized":"2024-03-11T19:38:10.452916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:18.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454396+0000","last_change":"2024-03-11T19:37:05.328999+0000","last_active":"2024-03-11T19:38:10.454396+0000","last_peered":"2024-03-11T19:38:10.454396+0000","last_clean":"2024-03-11T19:38:10.454396+0000","last_became_active":"2024-03-11T19:37:05.328620+0000","last_became_peered":"2024-03-11T19:37:05.328620+0000","last_unstale":"2024-03-11T19:38:10.454396+0000","last_undegraded":"2024-03-11T19:38:10.454396+0000","last_fullsized":"2024-03-11T19:38:10.454396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:40.191922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":91,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:16.726961+0000","last_change":"2024-03-11T19:37:04.587956+0000","last_active":"2024-03-11T19:38:16.726961+0000","last_peered":"2024-03-11T19:38:16.726961+0000","last_clean":"2024-03-11T19:38:16.726961+0000","last_became_active":"2024-03-11T19:37:04.587568+0000","last_became_peered":"2024-03-11T19:37:04.587568+0000","last_unstale":"2024-03-11T19:38:16.726961+0000","last_undegraded":"2024-03-11T19:38:16.726961+0000","last_fullsized":"2024-03-11T19:38:16.726961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:45:44.019436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448645+0000","last_change":"2024-03-11T19:37:05.336912+0000","last_active":"2024-03-11T19:38:10.448645+0000","last_peered":"2024-03-11T19:38:10.448645+0000","last_clean":"2024-03-11T19:38:10.448645+0000","last_became_active":"2024-03-11T19:37:05.336397+0000","last_became_peered":"2024-03-11T19:37:05.336397+0000","last_unstale":"2024-03-11T19:38:10.448645+0000","last_undegraded":"2024-03-11T19:38:10.448645+0000","last_fullsized":"2024-03-11T19:38:10.448645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:29.375396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150574+0000","last_change":"2024-03-11T19:37:04.584382+0000","last_active":"2024-03-11T19:38:10.150574+0000","last_peered":"2024-03-11T19:38:10.150574+0000","last_clean":"2024-03-11T19:38:10.150574+0000","last_became_active":"2024-03-11T19:37:04.584026+0000","last_became_peered":"2024-03-11T19:37:04.584026+0000","last_unstale":"2024-03-11T19:38:10.150574+0000","last_undegraded":"2024-03-11T19:38:10.150574+0000","last_fullsized":"2024-03-11T19:38:10.150574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:28.152903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152473+0000","last_change":"2024-03-11T19:37:05.331385+0000","last_active":"2024-03-11T19:38:10.152473+0000","last_peered":"2024-03-11T19:38:10.152473+0000","last_clean":"2024-03-11T19:38:10.152473+0000","last_became_active":"2024-03-11T19:37:05.331069+0000","last_became_peered":"2024-03-11T19:37:05.331069+0000","last_unstale":"2024-03-11T19:38:10.152473+0000","last_undegraded":"2024-03-11T19:38:10.152473+0000","last_fullsized":"2024-03-11T19:38:10.152473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:18.375605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453568+0000","last_change":"2024-03-11T19:37:04.586493+0000","last_active":"2024-03-11T19:38:10.453568+0000","last_peered":"2024-03-11T19:38:10.453568+0000","last_clean":"2024-03-11T19:38:10.453568+0000","last_became_active":"2024-03-11T19:37:04.586185+0000","last_became_peered":"2024-03-11T19:37:04.586185+0000","last_unstale":"2024-03-11T19:38:10.453568+0000","last_undegraded":"2024-03-11T19:38:10.453568+0000","last_fullsized":"2024-03-11T19:38:10.453568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:00.899752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448466+0000","last_change":"2024-03-11T19:37:05.336414+0000","last_active":"2024-03-11T19:38:10.448466+0000","last_peered":"2024-03-11T19:38:10.448466+0000","last_clean":"2024-03-11T19:38:10.448466+0000","last_became_active":"2024-03-11T19:37:05.334536+0000","last_became_peered":"2024-03-11T19:37:05.334536+0000","last_unstale":"2024-03-11T19:38:10.448466+0000","last_undegraded":"2024-03-11T19:38:10.448466+0000","last_fullsized":"2024-03-11T19:38:10.448466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:50:19.109975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'29","reported_seq":194,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:17.728831+0000","last_change":"2024-03-11T19:37:04.584544+0000","last_active":"2024-03-11T19:38:17.728831+0000","last_peered":"2024-03-11T19:38:17.728831+0000","last_clean":"2024-03-11T19:38:17.728831+0000","last_became_active":"2024-03-11T19:37:04.584198+0000","last_became_peered":"2024-03-11T19:37:04.584198+0000","last_unstale":"2024-03-11T19:38:17.728831+0000","last_undegraded":"2024-03-11T19:38:17.728831+0000","last_fullsized":"2024-03-11T19:38:17.728831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:01:56.744533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":146,"num_read_kb":131,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151763+0000","last_change":"2024-03-11T19:37:05.331077+0000","last_active":"2024-03-11T19:38:10.151763+0000","last_peered":"2024-03-11T19:38:10.151763+0000","last_clean":"2024-03-11T19:38:10.151763+0000","last_became_active":"2024-03-11T19:37:05.330708+0000","last_became_peered":"2024-03-11T19:37:05.330708+0000","last_unstale":"2024-03-11T19:38:10.151763+0000","last_undegraded":"2024-03-11T19:38:10.151763+0000","last_fullsized":"2024-03-11T19:38:10.151763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:54:55.019268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155548+0000","last_change":"2024-03-11T19:37:04.583353+0000","last_active":"2024-03-11T19:38:10.155548+0000","last_peered":"2024-03-11T19:38:10.155548+0000","last_clean":"2024-03-11T19:38:10.155548+0000","last_became_active":"2024-03-11T19:37:04.581974+0000","last_became_peered":"2024-03-11T19:37:04.581974+0000","last_unstale":"2024-03-11T19:38:10.155548+0000","last_undegraded":"2024-03-11T19:38:10.155548+0000","last_fullsized":"2024-03-11T19:38:10.155548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:41.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453141+0000","last_change":"2024-03-11T19:37:05.329498+0000","last_active":"2024-03-11T19:38:10.453141+0000","last_peered":"2024-03-11T19:38:10.453141+0000","last_clean":"2024-03-11T19:38:10.453141+0000","last_became_active":"2024-03-11T19:37:05.329186+0000","last_became_peered":"2024-03-11T19:37:05.329186+0000","last_unstale":"2024-03-11T19:38:10.453141+0000","last_undegraded":"2024-03-11T19:38:10.453141+0000","last_fullsized":"2024-03-11T19:38:10.453141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:33:45.012049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453150+0000","last_change":"2024-03-11T19:37:04.582027+0000","last_active":"2024-03-11T19:38:10.453150+0000","last_peered":"2024-03-11T19:38:10.453150+0000","last_clean":"2024-03-11T19:38:10.453150+0000","last_became_active":"2024-03-11T19:37:04.581678+0000","last_became_peered":"2024-03-11T19:37:04.581678+0000","last_unstale":"2024-03-11T19:38:10.453150+0000","last_undegraded":"2024-03-11T19:38:10.453150+0000","last_fullsized":"2024-03-11T19:38:10.453150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:03:40.688053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.455076+0000","last_change":"2024-03-11T19:38:10.455076+0000","last_active":"2024-03-11T19:38:10.455076+0000","last_peered":"2024-03-11T19:38:10.455076+0000","last_clean":"2024-03-11T19:38:10.455076+0000","last_became_active":"2024-03-11T19:38:10.454631+0000","last_became_peered":"2024-03-11T19:38:10.454631+0000","last_unstale":"2024-03-11T19:38:10.455076+0000","last_undegraded":"2024-03-11T19:38:10.455076+0000","last_fullsized":"2024-03-11T19:38:10.455076+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:26:36.651449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:15.726255+0000","last_change":"2024-03-11T19:37:04.576415+0000","last_active":"2024-03-11T19:38:15.726255+0000","last_peered":"2024-03-11T19:38:15.726255+0000","last_clean":"2024-03-11T19:38:15.726255+0000","last_became_active":"2024-03-11T19:37:04.575978+0000","last_became_peered":"2024-03-11T19:37:04.575978+0000","last_unstale":"2024-03-11T19:38:15.726255+0000","last_undegraded":"2024-03-11T19:38:15.726255+0000","last_fullsized":"2024-03-11T19:38:15.726255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:20.124813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150379+0000","last_change":"2024-03-11T19:37:05.326258+0000","last_active":"2024-03-11T19:38:10.150379+0000","last_peered":"2024-03-11T19:38:10.150379+0000","last_clean":"2024-03-11T19:38:10.150379+0000","last_became_active":"2024-03-11T19:37:05.325909+0000","last_became_peered":"2024-03-11T19:37:05.325909+0000","last_unstale":"2024-03-11T19:38:10.150379+0000","last_undegraded":"2024-03-11T19:38:10.150379+0000","last_fullsized":"2024-03-11T19:38:10.150379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:25.875603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.482823+0000","last_change":"2024-03-11T19:38:10.482380+0000","last_active":"2024-03-11T19:38:10.482823+0000","last_peered":"2024-03-11T19:38:10.482823+0000","last_clean":"2024-03-11T19:38:10.482823+0000","last_became_active":"2024-03-11T19:38:10.455301+0000","last_became_peered":"2024-03-11T19:38:10.455301+0000","last_unstale":"2024-03-11T19:38:10.482823+0000","last_undegraded":"2024-03-11T19:38:10.482823+0000","last_fullsized":"2024-03-11T19:38:10.482823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:28:41.620522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452916+0000","last_change":"2024-03-11T19:37:05.336727+0000","last_active":"2024-03-11T19:38:10.452916+0000","last_peered":"2024-03-11T19:38:10.452916+0000","last_clean":"2024-03-11T19:38:10.452916+0000","last_became_active":"2024-03-11T19:37:05.336384+0000","last_became_peered":"2024-03-11T19:37:05.336384+0000","last_unstale":"2024-03-11T19:38:10.452916+0000","last_undegraded":"2024-03-11T19:38:10.452916+0000","last_fullsized":"2024-03-11T19:38:10.452916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:02.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151827+0000","last_change":"2024-03-11T19:37:04.586814+0000","last_active":"2024-03-11T19:38:10.151827+0000","last_peered":"2024-03-11T19:38:10.151827+0000","last_clean":"2024-03-11T19:38:10.151827+0000","last_became_active":"2024-03-11T19:37:04.586512+0000","last_became_peered":"2024-03-11T19:37:04.586512+0000","last_unstale":"2024-03-11T19:38:10.151827+0000","last_undegraded":"2024-03-11T19:38:10.151827+0000","last_fullsized":"2024-03-11T19:38:10.151827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:04:23.979783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152417+0000","last_change":"2024-03-11T19:37:05.335353+0000","last_active":"2024-03-11T19:38:10.152417+0000","last_peered":"2024-03-11T19:38:10.152417+0000","last_clean":"2024-03-11T19:38:10.152417+0000","last_became_active":"2024-03-11T19:37:05.335066+0000","last_became_peered":"2024-03-11T19:37:05.335066+0000","last_unstale":"2024-03-11T19:38:10.152417+0000","last_undegraded":"2024-03-11T19:38:10.152417+0000","last_fullsized":"2024-03-11T19:38:10.152417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:11.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454552+0000","last_change":"2024-03-11T19:37:04.586679+0000","last_active":"2024-03-11T19:38:10.454552+0000","last_peered":"2024-03-11T19:38:10.454552+0000","last_clean":"2024-03-11T19:38:10.454552+0000","last_became_active":"2024-03-11T19:37:04.586316+0000","last_became_peered":"2024-03-11T19:37:04.586316+0000","last_unstale":"2024-03-11T19:38:10.454552+0000","last_undegraded":"2024-03-11T19:38:10.454552+0000","last_fullsized":"2024-03-11T19:38:10.454552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:58:21.115526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453359+0000","last_change":"2024-03-11T19:37:05.330206+0000","last_active":"2024-03-11T19:38:10.453359+0000","last_peered":"2024-03-11T19:38:10.453359+0000","last_clean":"2024-03-11T19:38:10.453359+0000","last_became_active":"2024-03-11T19:37:05.329847+0000","last_became_peered":"2024-03-11T19:37:05.329847+0000","last_unstale":"2024-03-11T19:38:10.453359+0000","last_undegraded":"2024-03-11T19:38:10.453359+0000","last_fullsized":"2024-03-11T19:38:10.453359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:14:52.532236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150350+0000","last_change":"2024-03-11T19:37:04.581031+0000","last_active":"2024-03-11T19:38:10.150350+0000","last_peered":"2024-03-11T19:38:10.150350+0000","last_clean":"2024-03-11T19:38:10.150350+0000","last_became_active":"2024-03-11T19:37:04.580728+0000","last_became_peered":"2024-03-11T19:37:04.580728+0000","last_unstale":"2024-03-11T19:38:10.150350+0000","last_undegraded":"2024-03-11T19:38:10.150350+0000","last_fullsized":"2024-03-11T19:38:10.150350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:43.401898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453212+0000","last_change":"2024-03-11T19:37:05.328686+0000","last_active":"2024-03-11T19:38:10.453212+0000","last_peered":"2024-03-11T19:38:10.453212+0000","last_clean":"2024-03-11T19:38:10.453212+0000","last_became_active":"2024-03-11T19:37:05.328372+0000","last_became_peered":"2024-03-11T19:37:05.328372+0000","last_unstale":"2024-03-11T19:38:10.453212+0000","last_undegraded":"2024-03-11T19:38:10.453212+0000","last_fullsized":"2024-03-11T19:38:10.453212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:38:33.216489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453162+0000","last_change":"2024-03-11T19:37:05.335057+0000","last_active":"2024-03-11T19:38:10.453162+0000","last_peered":"2024-03-11T19:38:10.453162+0000","last_clean":"2024-03-11T19:38:10.453162+0000","last_became_active":"2024-03-11T19:37:05.334646+0000","last_became_peered":"2024-03-11T19:37:05.334646+0000","last_unstale":"2024-03-11T19:38:10.453162+0000","last_undegraded":"2024-03-11T19:38:10.453162+0000","last_fullsized":"2024-03-11T19:38:10.453162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:34:15.932429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":91,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:15.725775+0000","last_change":"2024-03-11T19:37:04.588095+0000","last_active":"2024-03-11T19:38:15.725775+0000","last_peered":"2024-03-11T19:38:15.725775+0000","last_clean":"2024-03-11T19:38:15.725775+0000","last_became_active":"2024-03-11T19:37:04.587757+0000","last_became_peered":"2024-03-11T19:37:04.587757+0000","last_unstale":"2024-03-11T19:38:15.725775+0000","last_undegraded":"2024-03-11T19:38:15.725775+0000","last_fullsized":"2024-03-11T19:38:15.725775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:57.926858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452696+0000","last_change":"2024-03-11T19:37:05.326806+0000","last_active":"2024-03-11T19:38:10.452696+0000","last_peered":"2024-03-11T19:38:10.452696+0000","last_clean":"2024-03-11T19:38:10.452696+0000","last_became_active":"2024-03-11T19:37:05.326506+0000","last_became_peered":"2024-03-11T19:37:05.326506+0000","last_unstale":"2024-03-11T19:38:10.452696+0000","last_undegraded":"2024-03-11T19:38:10.452696+0000","last_fullsized":"2024-03-11T19:38:10.452696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:20.664857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453750+0000","last_change":"2024-03-11T19:37:04.588459+0000","last_active":"2024-03-11T19:38:10.453750+0000","last_peered":"2024-03-11T19:38:10.453750+0000","last_clean":"2024-03-11T19:38:10.453750+0000","last_became_active":"2024-03-11T19:37:04.588000+0000","last_became_peered":"2024-03-11T19:37:04.588000+0000","last_unstale":"2024-03-11T19:38:10.453750+0000","last_undegraded":"2024-03-11T19:38:10.453750+0000","last_fullsized":"2024-03-11T19:38:10.453750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:20:47.717196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150226+0000","last_change":"2024-03-11T19:35:55.814178+0000","last_active":"2024-03-11T19:38:10.150226+0000","last_peered":"2024-03-11T19:38:10.150226+0000","last_clean":"2024-03-11T19:38:10.150226+0000","last_became_active":"2024-03-11T19:35:55.813697+0000","last_became_peered":"2024-03-11T19:35:55.813697+0000","last_unstale":"2024-03-11T19:38:10.150226+0000","last_undegraded":"2024-03-11T19:38:10.150226+0000","last_fullsized":"2024-03-11T19:38:10.150226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_clean_scrub_stamp":"2024-03-11T19:32:34.872292+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:50:27.486733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453367+0000","last_change":"2024-03-11T19:37:05.319614+0000","last_active":"2024-03-11T19:38:10.453367+0000","last_peered":"2024-03-11T19:38:10.453367+0000","last_clean":"2024-03-11T19:38:10.453367+0000","last_became_active":"2024-03-11T19:37:05.319277+0000","last_became_peered":"2024-03-11T19:37:05.319277+0000","last_unstale":"2024-03-11T19:38:10.453367+0000","last_undegraded":"2024-03-11T19:38:10.453367+0000","last_fullsized":"2024-03-11T19:38:10.453367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:27.329900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150075+0000","last_change":"2024-03-11T19:37:04.582250+0000","last_active":"2024-03-11T19:38:10.150075+0000","last_peered":"2024-03-11T19:38:10.150075+0000","last_clean":"2024-03-11T19:38:10.150075+0000","last_became_active":"2024-03-11T19:37:04.581938+0000","last_became_peered":"2024-03-11T19:37:04.581938+0000","last_unstale":"2024-03-11T19:38:10.150075+0000","last_undegraded":"2024-03-11T19:38:10.150075+0000","last_fullsized":"2024-03-11T19:38:10.150075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:23.466067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151834+0000","last_change":"2024-03-11T19:37:04.590699+0000","last_active":"2024-03-11T19:38:10.151834+0000","last_peered":"2024-03-11T19:38:10.151834+0000","last_clean":"2024-03-11T19:38:10.151834+0000","last_became_active":"2024-03-11T19:37:04.590424+0000","last_became_peered":"2024-03-11T19:37:04.590424+0000","last_unstale":"2024-03-11T19:38:10.151834+0000","last_undegraded":"2024-03-11T19:38:10.151834+0000","last_fullsized":"2024-03-11T19:38:10.151834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:49.075335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453367+0000","last_change":"2024-03-11T19:37:05.323357+0000","last_active":"2024-03-11T19:38:10.453367+0000","last_peered":"2024-03-11T19:38:10.453367+0000","last_clean":"2024-03-11T19:38:10.453367+0000","last_became_active":"2024-03-11T19:37:05.322980+0000","last_became_peered":"2024-03-11T19:37:05.322980+0000","last_unstale":"2024-03-11T19:38:10.453367+0000","last_undegraded":"2024-03-11T19:38:10.453367+0000","last_fullsized":"2024-03-11T19:38:10.453367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:19.106375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454097+0000","last_change":"2024-03-11T19:37:04.585232+0000","last_active":"2024-03-11T19:38:10.454097+0000","last_peered":"2024-03-11T19:38:10.454097+0000","last_clean":"2024-03-11T19:38:10.454097+0000","last_became_active":"2024-03-11T19:37:04.584904+0000","last_became_peered":"2024-03-11T19:37:04.584904+0000","last_unstale":"2024-03-11T19:38:10.454097+0000","last_undegraded":"2024-03-11T19:38:10.454097+0000","last_fullsized":"2024-03-11T19:38:10.454097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:56:46.519961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453200+0000","last_change":"2024-03-11T19:37:05.332019+0000","last_active":"2024-03-11T19:38:10.453200+0000","last_peered":"2024-03-11T19:38:10.453200+0000","last_clean":"2024-03-11T19:38:10.453200+0000","last_became_active":"2024-03-11T19:37:05.331673+0000","last_became_peered":"2024-03-11T19:37:05.331673+0000","last_unstale":"2024-03-11T19:38:10.453200+0000","last_undegraded":"2024-03-11T19:38:10.453200+0000","last_fullsized":"2024-03-11T19:38:10.453200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:49:45.787082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454347+0000","last_change":"2024-03-11T19:37:04.583404+0000","last_active":"2024-03-11T19:38:10.454347+0000","last_peered":"2024-03-11T19:38:10.454347+0000","last_clean":"2024-03-11T19:38:10.454347+0000","last_became_active":"2024-03-11T19:37:04.583025+0000","last_became_peered":"2024-03-11T19:37:04.583025+0000","last_unstale":"2024-03-11T19:38:10.454347+0000","last_undegraded":"2024-03-11T19:38:10.454347+0000","last_fullsized":"2024-03-11T19:38:10.454347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:02:25.202591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152747+0000","last_change":"2024-03-11T19:37:05.331711+0000","last_active":"2024-03-11T19:38:10.152747+0000","last_peered":"2024-03-11T19:38:10.152747+0000","last_clean":"2024-03-11T19:38:10.152747+0000","last_became_active":"2024-03-11T19:37:05.330851+0000","last_became_peered":"2024-03-11T19:37:05.330851+0000","last_unstale":"2024-03-11T19:38:10.152747+0000","last_undegraded":"2024-03-11T19:38:10.152747+0000","last_fullsized":"2024-03-11T19:38:10.152747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:18:26.692068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448230+0000","last_change":"2024-03-11T19:37:04.586501+0000","last_active":"2024-03-11T19:38:10.448230+0000","last_peered":"2024-03-11T19:38:10.448230+0000","last_clean":"2024-03-11T19:38:10.448230+0000","last_became_active":"2024-03-11T19:37:04.586200+0000","last_became_peered":"2024-03-11T19:37:04.586200+0000","last_unstale":"2024-03-11T19:38:10.448230+0000","last_undegraded":"2024-03-11T19:38:10.448230+0000","last_fullsized":"2024-03-11T19:38:10.448230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:56.635375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151255+0000","last_change":"2024-03-11T19:37:05.332402+0000","last_active":"2024-03-11T19:38:10.151255+0000","last_peered":"2024-03-11T19:38:10.151255+0000","last_clean":"2024-03-11T19:38:10.151255+0000","last_became_active":"2024-03-11T19:37:05.332087+0000","last_became_peered":"2024-03-11T19:37:05.332087+0000","last_unstale":"2024-03-11T19:38:10.151255+0000","last_undegraded":"2024-03-11T19:38:10.151255+0000","last_fullsized":"2024-03-11T19:38:10.151255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:10:23.882296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452565+0000","last_change":"2024-03-11T19:37:04.584978+0000","last_active":"2024-03-11T19:38:10.452565+0000","last_peered":"2024-03-11T19:38:10.452565+0000","last_clean":"2024-03-11T19:38:10.452565+0000","last_became_active":"2024-03-11T19:37:04.584556+0000","last_became_peered":"2024-03-11T19:37:04.584556+0000","last_unstale":"2024-03-11T19:38:10.452565+0000","last_undegraded":"2024-03-11T19:38:10.452565+0000","last_fullsized":"2024-03-11T19:38:10.452565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:39:48.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.153417+0000","last_change":"2024-03-11T19:37:05.334453+0000","last_active":"2024-03-11T19:38:10.153417+0000","last_peered":"2024-03-11T19:38:10.153417+0000","last_clean":"2024-03-11T19:38:10.153417+0000","last_became_active":"2024-03-11T19:37:05.334009+0000","last_became_peered":"2024-03-11T19:37:05.334009+0000","last_unstale":"2024-03-11T19:38:10.153417+0000","last_undegraded":"2024-03-11T19:38:10.153417+0000","last_fullsized":"2024-03-11T19:38:10.153417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:43.402674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155193+0000","last_change":"2024-03-11T19:37:04.583181+0000","last_active":"2024-03-11T19:38:10.155193+0000","last_peered":"2024-03-11T19:38:10.155193+0000","last_clean":"2024-03-11T19:38:10.155193+0000","last_became_active":"2024-03-11T19:37:04.582881+0000","last_became_peered":"2024-03-11T19:37:04.582881+0000","last_unstale":"2024-03-11T19:38:10.155193+0000","last_undegraded":"2024-03-11T19:38:10.155193+0000","last_fullsized":"2024-03-11T19:38:10.155193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:05:06.634150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452686+0000","last_change":"2024-03-11T19:37:05.333642+0000","last_active":"2024-03-11T19:38:10.452686+0000","last_peered":"2024-03-11T19:38:10.452686+0000","last_clean":"2024-03-11T19:38:10.452686+0000","last_became_active":"2024-03-11T19:37:05.333121+0000","last_became_peered":"2024-03-11T19:37:05.333121+0000","last_unstale":"2024-03-11T19:38:10.452686+0000","last_undegraded":"2024-03-11T19:38:10.452686+0000","last_fullsized":"2024-03-11T19:38:10.452686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:07:21.217101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.458830+0000","last_change":"2024-03-11T19:38:10.458358+0000","last_active":"2024-03-11T19:38:10.458830+0000","last_peered":"2024-03-11T19:38:10.458830+0000","last_clean":"2024-03-11T19:38:10.458830+0000","last_became_active":"2024-03-11T19:38:10.449890+0000","last_became_peered":"2024-03-11T19:38:10.449890+0000","last_unstale":"2024-03-11T19:38:10.458830+0000","last_undegraded":"2024-03-11T19:38:10.458830+0000","last_fullsized":"2024-03-11T19:38:10.458830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:51:00.265814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150548+0000","last_change":"2024-03-11T19:37:05.326992+0000","last_active":"2024-03-11T19:38:10.150548+0000","last_peered":"2024-03-11T19:38:10.150548+0000","last_clean":"2024-03-11T19:38:10.150548+0000","last_became_active":"2024-03-11T19:37:05.326676+0000","last_became_peered":"2024-03-11T19:37:05.326676+0000","last_unstale":"2024-03-11T19:38:10.150548+0000","last_undegraded":"2024-03-11T19:38:10.150548+0000","last_fullsized":"2024-03-11T19:38:10.150548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:31:10.310650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155451+0000","last_change":"2024-03-11T19:37:04.583631+0000","last_active":"2024-03-11T19:38:10.155451+0000","last_peered":"2024-03-11T19:38:10.155451+0000","last_clean":"2024-03-11T19:38:10.155451+0000","last_became_active":"2024-03-11T19:37:04.583031+0000","last_became_peered":"2024-03-11T19:37:04.583031+0000","last_unstale":"2024-03-11T19:38:10.155451+0000","last_undegraded":"2024-03-11T19:38:10.155451+0000","last_fullsized":"2024-03-11T19:38:10.155451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:20.695496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150761+0000","last_change":"2024-03-11T19:37:05.331008+0000","last_active":"2024-03-11T19:38:10.150761+0000","last_peered":"2024-03-11T19:38:10.150761+0000","last_clean":"2024-03-11T19:38:10.150761+0000","last_became_active":"2024-03-11T19:37:05.330647+0000","last_became_peered":"2024-03-11T19:37:05.330647+0000","last_unstale":"2024-03-11T19:38:10.150761+0000","last_undegraded":"2024-03-11T19:38:10.150761+0000","last_fullsized":"2024-03-11T19:38:10.150761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:07:00.576137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453400+0000","last_change":"2024-03-11T19:37:04.582501+0000","last_active":"2024-03-11T19:38:10.453400+0000","last_peered":"2024-03-11T19:38:10.453400+0000","last_clean":"2024-03-11T19:38:10.453400+0000","last_became_active":"2024-03-11T19:37:04.582076+0000","last_became_peered":"2024-03-11T19:37:04.582076+0000","last_unstale":"2024-03-11T19:38:10.453400+0000","last_undegraded":"2024-03-11T19:38:10.453400+0000","last_fullsized":"2024-03-11T19:38:10.453400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:46:05.359211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150463+0000","last_change":"2024-03-11T19:37:05.329053+0000","last_active":"2024-03-11T19:38:10.150463+0000","last_peered":"2024-03-11T19:38:10.150463+0000","last_clean":"2024-03-11T19:38:10.150463+0000","last_became_active":"2024-03-11T19:37:05.328748+0000","last_became_peered":"2024-03-11T19:37:05.328748+0000","last_unstale":"2024-03-11T19:38:10.150463+0000","last_undegraded":"2024-03-11T19:38:10.150463+0000","last_fullsized":"2024-03-11T19:38:10.150463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:48:55.369515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453324+0000","last_change":"2024-03-11T19:37:04.583370+0000","last_active":"2024-03-11T19:38:10.453324+0000","last_peered":"2024-03-11T19:38:10.453324+0000","last_clean":"2024-03-11T19:38:10.453324+0000","last_became_active":"2024-03-11T19:37:04.583055+0000","last_became_peered":"2024-03-11T19:37:04.583055+0000","last_unstale":"2024-03-11T19:38:10.453324+0000","last_undegraded":"2024-03-11T19:38:10.453324+0000","last_fullsized":"2024-03-11T19:38:10.453324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:04:02.325054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150768+0000","last_change":"2024-03-11T19:37:05.328157+0000","last_active":"2024-03-11T19:38:10.150768+0000","last_peered":"2024-03-11T19:38:10.150768+0000","last_clean":"2024-03-11T19:38:10.150768+0000","last_became_active":"2024-03-11T19:37:05.327839+0000","last_became_peered":"2024-03-11T19:37:05.327839+0000","last_unstale":"2024-03-11T19:38:10.150768+0000","last_undegraded":"2024-03-11T19:38:10.150768+0000","last_fullsized":"2024-03-11T19:38:10.150768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:23:02.297269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.153533+0000","last_change":"2024-03-11T19:37:05.336160+0000","last_active":"2024-03-11T19:38:10.153533+0000","last_peered":"2024-03-11T19:38:10.153533+0000","last_clean":"2024-03-11T19:38:10.153533+0000","last_became_active":"2024-03-11T19:37:05.335850+0000","last_became_peered":"2024-03-11T19:37:05.335850+0000","last_unstale":"2024-03-11T19:38:10.153533+0000","last_undegraded":"2024-03-11T19:38:10.153533+0000","last_fullsized":"2024-03-11T19:38:10.153533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:09.743661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.455362+0000","last_change":"2024-03-11T19:38:10.455362+0000","last_active":"2024-03-11T19:38:10.455362+0000","last_peered":"2024-03-11T19:38:10.455362+0000","last_clean":"2024-03-11T19:38:10.455362+0000","last_became_active":"2024-03-11T19:38:10.454887+0000","last_became_peered":"2024-03-11T19:38:10.454887+0000","last_unstale":"2024-03-11T19:38:10.455362+0000","last_undegraded":"2024-03-11T19:38:10.455362+0000","last_fullsized":"2024-03-11T19:38:10.455362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:19.289829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155142+0000","last_change":"2024-03-11T19:37:05.332357+0000","last_active":"2024-03-11T19:38:10.155142+0000","last_peered":"2024-03-11T19:38:10.155142+0000","last_clean":"2024-03-11T19:38:10.155142+0000","last_became_active":"2024-03-11T19:37:05.332003+0000","last_became_peered":"2024-03-11T19:37:05.332003+0000","last_unstale":"2024-03-11T19:38:10.155142+0000","last_undegraded":"2024-03-11T19:38:10.155142+0000","last_fullsized":"2024-03-11T19:38:10.155142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:39.165572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152540+0000","last_change":"2024-03-11T19:37:05.332388+0000","last_active":"2024-03-11T19:38:10.152540+0000","last_peered":"2024-03-11T19:38:10.152540+0000","last_clean":"2024-03-11T19:38:10.152540+0000","last_became_active":"2024-03-11T19:37:05.332032+0000","last_became_peered":"2024-03-11T19:37:05.332032+0000","last_unstale":"2024-03-11T19:38:10.152540+0000","last_undegraded":"2024-03-11T19:38:10.152540+0000","last_fullsized":"2024-03-11T19:38:10.152540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:56:09.416683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453438+0000","last_change":"2024-03-11T19:37:05.323096+0000","last_active":"2024-03-11T19:38:10.453438+0000","last_peered":"2024-03-11T19:38:10.453438+0000","last_clean":"2024-03-11T19:38:10.453438+0000","last_became_active":"2024-03-11T19:37:05.322525+0000","last_became_peered":"2024-03-11T19:37:05.322525+0000","last_unstale":"2024-03-11T19:38:10.453438+0000","last_undegraded":"2024-03-11T19:38:10.453438+0000","last_fullsized":"2024-03-11T19:38:10.453438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:53:01.870894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152478+0000","last_change":"2024-03-11T19:37:05.331171+0000","last_active":"2024-03-11T19:38:10.152478+0000","last_peered":"2024-03-11T19:38:10.152478+0000","last_clean":"2024-03-11T19:38:10.152478+0000","last_became_active":"2024-03-11T19:37:05.330769+0000","last_became_peered":"2024-03-11T19:37:05.330769+0000","last_unstale":"2024-03-11T19:38:10.152478+0000","last_undegraded":"2024-03-11T19:38:10.152478+0000","last_fullsized":"2024-03-11T19:38:10.152478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:03:06.514039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155815+0000","last_change":"2024-03-11T19:37:05.326877+0000","last_active":"2024-03-11T19:38:10.155815+0000","last_peered":"2024-03-11T19:38:10.155815+0000","last_clean":"2024-03-11T19:38:10.155815+0000","last_became_active":"2024-03-11T19:37:05.326538+0000","last_became_peered":"2024-03-11T19:37:05.326538+0000","last_unstale":"2024-03-11T19:38:10.155815+0000","last_undegraded":"2024-03-11T19:38:10.155815+0000","last_fullsized":"2024-03-11T19:38:10.155815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:11.565697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453097+0000","last_change":"2024-03-11T19:37:05.335010+0000","last_active":"2024-03-11T19:38:10.453097+0000","last_peered":"2024-03-11T19:38:10.453097+0000","last_clean":"2024-03-11T19:38:10.453097+0000","last_became_active":"2024-03-11T19:37:05.334551+0000","last_became_peered":"2024-03-11T19:37:05.334551+0000","last_unstale":"2024-03-11T19:38:10.453097+0000","last_undegraded":"2024-03-11T19:38:10.453097+0000","last_fullsized":"2024-03-11T19:38:10.453097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:59.668178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452965+0000","last_change":"2024-03-11T19:37:05.336273+0000","last_active":"2024-03-11T19:38:10.452965+0000","last_peered":"2024-03-11T19:38:10.452965+0000","last_clean":"2024-03-11T19:38:10.452965+0000","last_became_active":"2024-03-11T19:37:05.335926+0000","last_became_peered":"2024-03-11T19:37:05.335926+0000","last_unstale":"2024-03-11T19:38:10.452965+0000","last_undegraded":"2024-03-11T19:38:10.452965+0000","last_fullsized":"2024-03-11T19:38:10.452965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:31:55.139242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452598+0000","last_change":"2024-03-11T19:37:05.335548+0000","last_active":"2024-03-11T19:38:10.452598+0000","last_peered":"2024-03-11T19:38:10.452598+0000","last_clean":"2024-03-11T19:38:10.452598+0000","last_became_active":"2024-03-11T19:37:05.335222+0000","last_became_peered":"2024-03-11T19:37:05.335222+0000","last_unstale":"2024-03-11T19:38:10.452598+0000","last_undegraded":"2024-03-11T19:38:10.452598+0000","last_fullsized":"2024-03-11T19:38:10.452598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:25.697951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448454+0000","last_change":"2024-03-11T19:37:05.329782+0000","last_active":"2024-03-11T19:38:10.448454+0000","last_peered":"2024-03-11T19:38:10.448454+0000","last_clean":"2024-03-11T19:38:10.448454+0000","last_became_active":"2024-03-11T19:37:05.329434+0000","last_became_peered":"2024-03-11T19:37:05.329434+0000","last_unstale":"2024-03-11T19:38:10.448454+0000","last_undegraded":"2024-03-11T19:38:10.448454+0000","last_fullsized":"2024-03-11T19:38:10.448454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:12:00.516489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453365+0000","last_change":"2024-03-11T19:37:05.323026+0000","last_active":"2024-03-11T19:38:10.453365+0000","last_peered":"2024-03-11T19:38:10.453365+0000","last_clean":"2024-03-11T19:38:10.453365+0000","last_became_active":"2024-03-11T19:37:05.322562+0000","last_became_peered":"2024-03-11T19:37:05.322562+0000","last_unstale":"2024-03-11T19:38:10.453365+0000","last_undegraded":"2024-03-11T19:38:10.453365+0000","last_fullsized":"2024-03-11T19:38:10.453365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:14:57.606404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.455616+0000","last_change":"2024-03-11T19:38:10.455616+0000","last_active":"2024-03-11T19:38:10.455616+0000","last_peered":"2024-03-11T19:38:10.455616+0000","last_clean":"2024-03-11T19:38:10.455616+0000","last_became_active":"2024-03-11T19:38:10.455092+0000","last_became_peered":"2024-03-11T19:38:10.455092+0000","last_unstale":"2024-03-11T19:38:10.455616+0000","last_undegraded":"2024-03-11T19:38:10.455616+0000","last_fullsized":"2024-03-11T19:38:10.455616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:49:50.407540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64415,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":434,"num_read_kb":285,"num_write":158,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":455694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1638400,"data_stored":1107477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":455818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":455626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1552384,"data_stored":1048059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":455207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":515023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":966656,"data_stored":456478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":516652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":455518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1564672,"data_stored":1047352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:38:20.054 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph pg dump --format=json 2024-03-11T19:38:20.304 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:20.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:20 smithi083 ceph-mon[31864]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:38:20.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:20 smithi137 ceph-mon[36220]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:38:20.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:20 smithi204 ceph-mon[36739]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:38:21.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:21 smithi083 ceph-mon[31864]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-03-11T19:38:21.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:21 smithi137 ceph-mon[36220]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-03-11T19:38:21.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:21 smithi204 ceph-mon[36739]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2024-03-11T19:38:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.911+0000 7f370af9f700 1 -- 172.21.15.83:0/2841097665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3704101cc0 msgr2=0x7f3704102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.911+0000 7f370af9f700 1 --2- 172.21.15.83:0/2841097665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3704101cc0 0x7f3704102140 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f36f00099c0 tx=0x7f36f0009cd0 comp rx=0 tx=0).stop 2024-03-11T19:38:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.911+0000 7f370af9f700 1 -- 172.21.15.83:0/2841097665 shutdown_connections 2024-03-11T19:38:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.911+0000 7f370af9f700 1 --2- 172.21.15.83:0/2841097665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3704107ca0 0x7f3704108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.911+0000 7f370af9f700 1 --2- 172.21.15.83:0/2841097665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3704102680 0x7f370410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.911+0000 7f370af9f700 1 --2- 172.21.15.83:0/2841097665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3704101cc0 0x7f3704102140 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.911+0000 7f370af9f700 1 -- 172.21.15.83:0/2841097665 >> 172.21.15.83:0/2841097665 conn(0x7f3704078da0 msgr2=0x7f37040791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.911+0000 7f370af9f700 1 -- 172.21.15.83:0/2841097665 shutdown_connections 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.911+0000 7f370af9f700 1 -- 172.21.15.83:0/2841097665 wait complete. 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f370af9f700 1 Processor -- start 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f370af9f700 1 -- start start 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f370af9f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3704101cc0 0x7f3704198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f370af9f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3704102680 0x7f3704198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f370af9f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3704107ca0 0x7f370419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f370af9f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370419d120 con 0x7f3704107ca0 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f370af9f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370419d290 con 0x7f3704102680 2024-03-11T19:38:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f370af9f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370419d450 con 0x7f3704101cc0 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f370953c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3704107ca0 0x7f370419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f370953c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3704107ca0 0x7f370419cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59936/0 (socket says 172.21.15.83:59936) 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f370953c700 1 -- 172.21.15.83:0/607878770 learned_addr learned my addr 172.21.15.83:0/607878770 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.912+0000 7f3703fff700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3704102680 0x7f3704198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f3708d3b700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3704101cc0 0x7f3704198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f370953c700 1 -- 172.21.15.83:0/607878770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3704101cc0 msgr2=0x7f3704198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f370953c700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3704101cc0 0x7f3704198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f370953c700 1 -- 172.21.15.83:0/607878770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3704102680 msgr2=0x7f3704198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f370953c700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3704102680 0x7f3704198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f370953c700 1 -- 172.21.15.83:0/607878770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f00095f0 con 0x7f3704107ca0 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f3703fff700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3704102680 0x7f3704198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f370953c700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3704107ca0 0x7f370419cb10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00b970 tx=0x7f36fc00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f3701ffb700 1 -- 172.21.15.83:0/607878770 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc00c9b0 con 0x7f3704107ca0 2024-03-11T19:38:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f370af9f700 1 -- 172.21.15.83:0/607878770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370419d730 con 0x7f3704107ca0 2024-03-11T19:38:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f3701ffb700 1 -- 172.21.15.83:0/607878770 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36fc014440 con 0x7f3704107ca0 2024-03-11T19:38:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f3701ffb700 1 -- 172.21.15.83:0/607878770 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0075d0 con 0x7f3704107ca0 2024-03-11T19:38:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.913+0000 7f370af9f700 1 -- 172.21.15.83:0/607878770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37041a4f90 con 0x7f3704107ca0 2024-03-11T19:38:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.914+0000 7f370af9f700 1 -- 172.21.15.83:0/607878770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370410a180 con 0x7f3704107ca0 2024-03-11T19:38:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.915+0000 7f3701ffb700 1 -- 172.21.15.83:0/607878770 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f36fc014920 con 0x7f3704107ca0 2024-03-11T19:38:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.916+0000 7f3701ffb700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f36f40a1b50 0x7f36f40a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.916+0000 7f3708d3b700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f36f40a1b50 0x7f36f40a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.916+0000 7f3701ffb700 1 -- 172.21.15.83:0/607878770 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f36fc006e40 con 0x7f3704107ca0 2024-03-11T19:38:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.916+0000 7f3708d3b700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f36f40a1b50 0x7f36f40a4010 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f36f8006fd0 tx=0x7f36f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:21.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:21.920+0000 7f3701ffb700 1 -- 172.21.15.83:0/607878770 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f36fc095dc0 con 0x7f3704107ca0 2024-03-11T19:38:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.108+0000 7f370af9f700 1 -- 172.21.15.83:0/607878770 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3704199610 con 0x7f36f40a1b50 2024-03-11T19:38:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.115+0000 7f3701ffb700 1 -- 172.21.15.83:0/607878770 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258721 (secure 0 0 0) 0x7f3704199610 con 0x7f36f40a1b50 2024-03-11T19:38:22.123 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 -- 172.21.15.83:0/607878770 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f36f40a1b50 msgr2=0x7f36f40a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f36f40a1b50 0x7f36f40a4010 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f36f8006fd0 tx=0x7f36f8008040 comp rx=0 tx=0).stop 2024-03-11T19:38:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 -- 172.21.15.83:0/607878770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3704107ca0 msgr2=0x7f370419cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3704107ca0 0x7f370419cb10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00b970 tx=0x7f36fc00bd30 comp rx=0 tx=0).stop 2024-03-11T19:38:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 -- 172.21.15.83:0/607878770 shutdown_connections 2024-03-11T19:38:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f36f40a1b50 0x7f36f40a4010 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3704101cc0 0x7f3704198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3704107ca0 0x7f370419cb10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 --2- 172.21.15.83:0/607878770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3704102680 0x7f3704198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 -- 172.21.15.83:0/607878770 >> 172.21.15.83:0/607878770 conn(0x7f3704078da0 msgr2=0x7f3704100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 -- 172.21.15.83:0/607878770 shutdown_connections 2024-03-11T19:38:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:22.118+0000 7f370af9f700 1 -- 172.21.15.83:0/607878770 wait complete. 2024-03-11T19:38:22.126 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-03-11T19:38:22.764 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-03-11T19:38:20.525858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655017,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":560,"num_read_kb":394,"num_write":341,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366544,"kb_used_data":13164,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575280,"statfs":{"total":1151940427776,"available":1151565086720,"internally_reserved":0,"allocated":13479936,"data_stored":7425051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":27,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.375010"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152898+0000","last_change":"2024-03-11T19:37:05.340476+0000","last_active":"2024-03-11T19:38:10.152898+0000","last_peered":"2024-03-11T19:38:10.152898+0000","last_clean":"2024-03-11T19:38:10.152898+0000","last_became_active":"2024-03-11T19:37:05.340140+0000","last_became_peered":"2024-03-11T19:37:05.340140+0000","last_unstale":"2024-03-11T19:38:10.152898+0000","last_undegraded":"2024-03-11T19:38:10.152898+0000","last_fullsized":"2024-03-11T19:38:10.152898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:10:35.677374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152679+0000","last_change":"2024-03-11T19:37:05.335500+0000","last_active":"2024-03-11T19:38:10.152679+0000","last_peered":"2024-03-11T19:38:10.152679+0000","last_clean":"2024-03-11T19:38:10.152679+0000","last_became_active":"2024-03-11T19:37:05.335194+0000","last_became_peered":"2024-03-11T19:37:05.335194+0000","last_unstale":"2024-03-11T19:38:10.152679+0000","last_undegraded":"2024-03-11T19:38:10.152679+0000","last_fullsized":"2024-03-11T19:38:10.152679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:03.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152345+0000","last_change":"2024-03-11T19:37:05.334618+0000","last_active":"2024-03-11T19:38:10.152345+0000","last_peered":"2024-03-11T19:38:10.152345+0000","last_clean":"2024-03-11T19:38:10.152345+0000","last_became_active":"2024-03-11T19:37:05.334281+0000","last_became_peered":"2024-03-11T19:37:05.334281+0000","last_unstale":"2024-03-11T19:38:10.152345+0000","last_undegraded":"2024-03-11T19:38:10.152345+0000","last_fullsized":"2024-03-11T19:38:10.152345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:43:22.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453952+0000","last_change":"2024-03-11T19:37:05.332978+0000","last_active":"2024-03-11T19:38:10.453952+0000","last_peered":"2024-03-11T19:38:10.453952+0000","last_clean":"2024-03-11T19:38:10.453952+0000","last_became_active":"2024-03-11T19:37:05.330888+0000","last_became_peered":"2024-03-11T19:37:05.330888+0000","last_unstale":"2024-03-11T19:38:10.453952+0000","last_undegraded":"2024-03-11T19:38:10.453952+0000","last_fullsized":"2024-03-11T19:38:10.453952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:40:36.988348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151425+0000","last_change":"2024-03-11T19:37:05.329050+0000","last_active":"2024-03-11T19:38:10.151425+0000","last_peered":"2024-03-11T19:38:10.151425+0000","last_clean":"2024-03-11T19:38:10.151425+0000","last_became_active":"2024-03-11T19:37:05.328702+0000","last_became_peered":"2024-03-11T19:37:05.328702+0000","last_unstale":"2024-03-11T19:38:10.151425+0000","last_undegraded":"2024-03-11T19:38:10.151425+0000","last_fullsized":"2024-03-11T19:38:10.151425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:19:52.180884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452959+0000","last_change":"2024-03-11T19:37:05.322748+0000","last_active":"2024-03-11T19:38:10.452959+0000","last_peered":"2024-03-11T19:38:10.452959+0000","last_clean":"2024-03-11T19:38:10.452959+0000","last_became_active":"2024-03-11T19:37:05.322376+0000","last_became_peered":"2024-03-11T19:37:05.322376+0000","last_unstale":"2024-03-11T19:38:10.452959+0000","last_undegraded":"2024-03-11T19:38:10.452959+0000","last_fullsized":"2024-03-11T19:38:10.452959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:40.926854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150130+0000","last_change":"2024-03-11T19:37:05.323954+0000","last_active":"2024-03-11T19:38:10.150130+0000","last_peered":"2024-03-11T19:38:10.150130+0000","last_clean":"2024-03-11T19:38:10.150130+0000","last_became_active":"2024-03-11T19:37:05.323586+0000","last_became_peered":"2024-03-11T19:37:05.323586+0000","last_unstale":"2024-03-11T19:38:10.150130+0000","last_undegraded":"2024-03-11T19:38:10.150130+0000","last_fullsized":"2024-03-11T19:38:10.150130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:38.286891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452465+0000","last_change":"2024-03-11T19:37:05.334138+0000","last_active":"2024-03-11T19:38:10.452465+0000","last_peered":"2024-03-11T19:38:10.452465+0000","last_clean":"2024-03-11T19:38:10.452465+0000","last_became_active":"2024-03-11T19:37:05.333837+0000","last_became_peered":"2024-03-11T19:37:05.333837+0000","last_unstale":"2024-03-11T19:38:10.452465+0000","last_undegraded":"2024-03-11T19:38:10.452465+0000","last_fullsized":"2024-03-11T19:38:10.452465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:49:30.724176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448684+0000","last_change":"2024-03-11T19:37:05.336888+0000","last_active":"2024-03-11T19:38:10.448684+0000","last_peered":"2024-03-11T19:38:10.448684+0000","last_clean":"2024-03-11T19:38:10.448684+0000","last_became_active":"2024-03-11T19:37:05.336256+0000","last_became_peered":"2024-03-11T19:37:05.336256+0000","last_unstale":"2024-03-11T19:38:10.448684+0000","last_undegraded":"2024-03-11T19:38:10.448684+0000","last_fullsized":"2024-03-11T19:38:10.448684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:54:31.526986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453438+0000","last_change":"2024-03-11T19:37:05.321959+0000","last_active":"2024-03-11T19:38:10.453438+0000","last_peered":"2024-03-11T19:38:10.453438+0000","last_clean":"2024-03-11T19:38:10.453438+0000","last_became_active":"2024-03-11T19:37:05.321043+0000","last_became_peered":"2024-03-11T19:37:05.321043+0000","last_unstale":"2024-03-11T19:38:10.453438+0000","last_undegraded":"2024-03-11T19:38:10.453438+0000","last_fullsized":"2024-03-11T19:38:10.453438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:13.121003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150663+0000","last_change":"2024-03-11T19:37:05.324940+0000","last_active":"2024-03-11T19:38:10.150663+0000","last_peered":"2024-03-11T19:38:10.150663+0000","last_clean":"2024-03-11T19:38:10.150663+0000","last_became_active":"2024-03-11T19:37:05.323427+0000","last_became_peered":"2024-03-11T19:37:05.323427+0000","last_unstale":"2024-03-11T19:38:10.150663+0000","last_undegraded":"2024-03-11T19:38:10.150663+0000","last_fullsized":"2024-03-11T19:38:10.150663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:23.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453851+0000","last_change":"2024-03-11T19:37:05.332694+0000","last_active":"2024-03-11T19:38:10.453851+0000","last_peered":"2024-03-11T19:38:10.453851+0000","last_clean":"2024-03-11T19:38:10.453851+0000","last_became_active":"2024-03-11T19:37:05.330446+0000","last_became_peered":"2024-03-11T19:37:05.330446+0000","last_unstale":"2024-03-11T19:38:10.453851+0000","last_undegraded":"2024-03-11T19:38:10.453851+0000","last_fullsized":"2024-03-11T19:38:10.453851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:37:40.196933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453284+0000","last_change":"2024-03-11T19:37:05.325975+0000","last_active":"2024-03-11T19:38:10.453284+0000","last_peered":"2024-03-11T19:38:10.453284+0000","last_clean":"2024-03-11T19:38:10.453284+0000","last_became_active":"2024-03-11T19:37:05.325649+0000","last_became_peered":"2024-03-11T19:37:05.325649+0000","last_unstale":"2024-03-11T19:38:10.453284+0000","last_undegraded":"2024-03-11T19:38:10.453284+0000","last_fullsized":"2024-03-11T19:38:10.453284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:36.373272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152272+0000","last_change":"2024-03-11T19:37:05.335546+0000","last_active":"2024-03-11T19:38:10.152272+0000","last_peered":"2024-03-11T19:38:10.152272+0000","last_clean":"2024-03-11T19:38:10.152272+0000","last_became_active":"2024-03-11T19:37:05.335220+0000","last_became_peered":"2024-03-11T19:37:05.335220+0000","last_unstale":"2024-03-11T19:38:10.152272+0000","last_undegraded":"2024-03-11T19:38:10.152272+0000","last_fullsized":"2024-03-11T19:38:10.152272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:13.116355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152291+0000","last_change":"2024-03-11T19:37:05.338955+0000","last_active":"2024-03-11T19:38:10.152291+0000","last_peered":"2024-03-11T19:38:10.152291+0000","last_clean":"2024-03-11T19:38:10.152291+0000","last_became_active":"2024-03-11T19:37:05.338574+0000","last_became_peered":"2024-03-11T19:37:05.338574+0000","last_unstale":"2024-03-11T19:38:10.152291+0000","last_undegraded":"2024-03-11T19:38:10.152291+0000","last_fullsized":"2024-03-11T19:38:10.152291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:27:27.754077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448190+0000","last_change":"2024-03-11T19:37:05.334578+0000","last_active":"2024-03-11T19:38:10.448190+0000","last_peered":"2024-03-11T19:38:10.448190+0000","last_clean":"2024-03-11T19:38:10.448190+0000","last_became_active":"2024-03-11T19:37:05.334267+0000","last_became_peered":"2024-03-11T19:37:05.334267+0000","last_unstale":"2024-03-11T19:38:10.448190+0000","last_undegraded":"2024-03-11T19:38:10.448190+0000","last_fullsized":"2024-03-11T19:38:10.448190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:31.704187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152760+0000","last_change":"2024-03-11T19:37:05.341178+0000","last_active":"2024-03-11T19:38:10.152760+0000","last_peered":"2024-03-11T19:38:10.152760+0000","last_clean":"2024-03-11T19:38:10.152760+0000","last_became_active":"2024-03-11T19:37:05.340837+0000","last_became_peered":"2024-03-11T19:37:05.340837+0000","last_unstale":"2024-03-11T19:38:10.152760+0000","last_undegraded":"2024-03-11T19:38:10.152760+0000","last_fullsized":"2024-03-11T19:38:10.152760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:39:54.460937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454128+0000","last_change":"2024-03-11T19:37:05.330981+0000","last_active":"2024-03-11T19:38:10.454128+0000","last_peered":"2024-03-11T19:38:10.454128+0000","last_clean":"2024-03-11T19:38:10.454128+0000","last_became_active":"2024-03-11T19:37:05.330638+0000","last_became_peered":"2024-03-11T19:37:05.330638+0000","last_unstale":"2024-03-11T19:38:10.454128+0000","last_undegraded":"2024-03-11T19:38:10.454128+0000","last_fullsized":"2024-03-11T19:38:10.454128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:36:12.565936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155394+0000","last_change":"2024-03-11T19:37:05.329866+0000","last_active":"2024-03-11T19:38:10.155394+0000","last_peered":"2024-03-11T19:38:10.155394+0000","last_clean":"2024-03-11T19:38:10.155394+0000","last_became_active":"2024-03-11T19:37:05.329572+0000","last_became_peered":"2024-03-11T19:37:05.329572+0000","last_unstale":"2024-03-11T19:38:10.155394+0000","last_undegraded":"2024-03-11T19:38:10.155394+0000","last_fullsized":"2024-03-11T19:38:10.155394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:40:04.482929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454096+0000","last_change":"2024-03-11T19:37:04.587149+0000","last_active":"2024-03-11T19:38:10.454096+0000","last_peered":"2024-03-11T19:38:10.454096+0000","last_clean":"2024-03-11T19:38:10.454096+0000","last_became_active":"2024-03-11T19:37:04.586013+0000","last_became_peered":"2024-03-11T19:37:04.586013+0000","last_unstale":"2024-03-11T19:38:10.454096+0000","last_undegraded":"2024-03-11T19:38:10.454096+0000","last_fullsized":"2024-03-11T19:38:10.454096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:15.758154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452775+0000","last_change":"2024-03-11T19:37:05.327979+0000","last_active":"2024-03-11T19:38:10.452775+0000","last_peered":"2024-03-11T19:38:10.452775+0000","last_clean":"2024-03-11T19:38:10.452775+0000","last_became_active":"2024-03-11T19:37:05.327647+0000","last_became_peered":"2024-03-11T19:37:05.327647+0000","last_unstale":"2024-03-11T19:38:10.452775+0000","last_undegraded":"2024-03-11T19:38:10.452775+0000","last_fullsized":"2024-03-11T19:38:10.452775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:09.514766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452950+0000","last_change":"2024-03-11T19:37:04.582937+0000","last_active":"2024-03-11T19:38:10.452950+0000","last_peered":"2024-03-11T19:38:10.452950+0000","last_clean":"2024-03-11T19:38:10.452950+0000","last_became_active":"2024-03-11T19:37:04.582622+0000","last_became_peered":"2024-03-11T19:37:04.582622+0000","last_unstale":"2024-03-11T19:38:10.452950+0000","last_undegraded":"2024-03-11T19:38:10.452950+0000","last_fullsized":"2024-03-11T19:38:10.452950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:02.275698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152466+0000","last_change":"2024-03-11T19:37:05.327191+0000","last_active":"2024-03-11T19:38:10.152466+0000","last_peered":"2024-03-11T19:38:10.152466+0000","last_clean":"2024-03-11T19:38:10.152466+0000","last_became_active":"2024-03-11T19:37:05.326001+0000","last_became_peered":"2024-03-11T19:37:05.326001+0000","last_unstale":"2024-03-11T19:38:10.152466+0000","last_undegraded":"2024-03-11T19:38:10.152466+0000","last_fullsized":"2024-03-11T19:38:10.152466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:27.301321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151969+0000","last_change":"2024-03-11T19:37:04.587054+0000","last_active":"2024-03-11T19:38:10.151969+0000","last_peered":"2024-03-11T19:38:10.151969+0000","last_clean":"2024-03-11T19:38:10.151969+0000","last_became_active":"2024-03-11T19:37:04.586104+0000","last_became_peered":"2024-03-11T19:37:04.586104+0000","last_unstale":"2024-03-11T19:38:10.151969+0000","last_undegraded":"2024-03-11T19:38:10.151969+0000","last_fullsized":"2024-03-11T19:38:10.151969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:34.913677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448312+0000","last_change":"2024-03-11T19:37:05.333288+0000","last_active":"2024-03-11T19:38:10.448312+0000","last_peered":"2024-03-11T19:38:10.448312+0000","last_clean":"2024-03-11T19:38:10.448312+0000","last_became_active":"2024-03-11T19:37:05.332882+0000","last_became_peered":"2024-03-11T19:37:05.332882+0000","last_unstale":"2024-03-11T19:38:10.448312+0000","last_undegraded":"2024-03-11T19:38:10.448312+0000","last_fullsized":"2024-03-11T19:38:10.448312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:45.680608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151628+0000","last_change":"2024-03-11T19:37:04.583660+0000","last_active":"2024-03-11T19:38:10.151628+0000","last_peered":"2024-03-11T19:38:10.151628+0000","last_clean":"2024-03-11T19:38:10.151628+0000","last_became_active":"2024-03-11T19:37:04.583360+0000","last_became_peered":"2024-03-11T19:37:04.583360+0000","last_unstale":"2024-03-11T19:38:10.151628+0000","last_undegraded":"2024-03-11T19:38:10.151628+0000","last_fullsized":"2024-03-11T19:38:10.151628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:00.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453554+0000","last_change":"2024-03-11T19:37:05.335649+0000","last_active":"2024-03-11T19:38:10.453554+0000","last_peered":"2024-03-11T19:38:10.453554+0000","last_clean":"2024-03-11T19:38:10.453554+0000","last_became_active":"2024-03-11T19:37:05.335255+0000","last_became_peered":"2024-03-11T19:37:05.335255+0000","last_unstale":"2024-03-11T19:38:10.453554+0000","last_undegraded":"2024-03-11T19:38:10.453554+0000","last_fullsized":"2024-03-11T19:38:10.453554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:49:09.772765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:13.723195+0000","last_change":"2024-03-11T19:37:04.586002+0000","last_active":"2024-03-11T19:38:13.723195+0000","last_peered":"2024-03-11T19:38:13.723195+0000","last_clean":"2024-03-11T19:38:13.723195+0000","last_became_active":"2024-03-11T19:37:04.585551+0000","last_became_peered":"2024-03-11T19:37:04.585551+0000","last_unstale":"2024-03-11T19:38:13.723195+0000","last_undegraded":"2024-03-11T19:38:13.723195+0000","last_fullsized":"2024-03-11T19:38:13.723195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:34:12.663801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151614+0000","last_change":"2024-03-11T19:37:05.325168+0000","last_active":"2024-03-11T19:38:10.151614+0000","last_peered":"2024-03-11T19:38:10.151614+0000","last_clean":"2024-03-11T19:38:10.151614+0000","last_became_active":"2024-03-11T19:37:05.324848+0000","last_became_peered":"2024-03-11T19:37:05.324848+0000","last_unstale":"2024-03-11T19:38:10.151614+0000","last_undegraded":"2024-03-11T19:38:10.151614+0000","last_fullsized":"2024-03-11T19:38:10.151614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:27.423877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452883+0000","last_change":"2024-03-11T19:37:04.582727+0000","last_active":"2024-03-11T19:38:10.452883+0000","last_peered":"2024-03-11T19:38:10.452883+0000","last_clean":"2024-03-11T19:38:10.452883+0000","last_became_active":"2024-03-11T19:37:04.582359+0000","last_became_peered":"2024-03-11T19:37:04.582359+0000","last_unstale":"2024-03-11T19:38:10.452883+0000","last_undegraded":"2024-03-11T19:38:10.452883+0000","last_fullsized":"2024-03-11T19:38:10.452883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:05.591537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453512+0000","last_change":"2024-03-11T19:37:05.335442+0000","last_active":"2024-03-11T19:38:10.453512+0000","last_peered":"2024-03-11T19:38:10.453512+0000","last_clean":"2024-03-11T19:38:10.453512+0000","last_became_active":"2024-03-11T19:37:05.335003+0000","last_became_peered":"2024-03-11T19:37:05.335003+0000","last_unstale":"2024-03-11T19:38:10.453512+0000","last_undegraded":"2024-03-11T19:38:10.453512+0000","last_fullsized":"2024-03-11T19:38:10.453512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:41.590898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453615+0000","last_change":"2024-03-11T19:37:04.589957+0000","last_active":"2024-03-11T19:38:10.453615+0000","last_peered":"2024-03-11T19:38:10.453615+0000","last_clean":"2024-03-11T19:38:10.453615+0000","last_became_active":"2024-03-11T19:37:04.589539+0000","last_became_peered":"2024-03-11T19:37:04.589539+0000","last_unstale":"2024-03-11T19:38:10.453615+0000","last_undegraded":"2024-03-11T19:38:10.453615+0000","last_fullsized":"2024-03-11T19:38:10.453615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:48:52.764057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452397+0000","last_change":"2024-03-11T19:37:05.331380+0000","last_active":"2024-03-11T19:38:10.452397+0000","last_peered":"2024-03-11T19:38:10.452397+0000","last_clean":"2024-03-11T19:38:10.452397+0000","last_became_active":"2024-03-11T19:37:05.331000+0000","last_became_peered":"2024-03-11T19:37:05.331000+0000","last_unstale":"2024-03-11T19:38:10.452397+0000","last_undegraded":"2024-03-11T19:38:10.452397+0000","last_fullsized":"2024-03-11T19:38:10.452397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:58:47.278195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452916+0000","last_change":"2024-03-11T19:37:04.584128+0000","last_active":"2024-03-11T19:38:10.452916+0000","last_peered":"2024-03-11T19:38:10.452916+0000","last_clean":"2024-03-11T19:38:10.452916+0000","last_became_active":"2024-03-11T19:37:04.583842+0000","last_became_peered":"2024-03-11T19:37:04.583842+0000","last_unstale":"2024-03-11T19:38:10.452916+0000","last_undegraded":"2024-03-11T19:38:10.452916+0000","last_fullsized":"2024-03-11T19:38:10.452916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:18.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454396+0000","last_change":"2024-03-11T19:37:05.328999+0000","last_active":"2024-03-11T19:38:10.454396+0000","last_peered":"2024-03-11T19:38:10.454396+0000","last_clean":"2024-03-11T19:38:10.454396+0000","last_became_active":"2024-03-11T19:37:05.328620+0000","last_became_peered":"2024-03-11T19:37:05.328620+0000","last_unstale":"2024-03-11T19:38:10.454396+0000","last_undegraded":"2024-03-11T19:38:10.454396+0000","last_fullsized":"2024-03-11T19:38:10.454396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:40.191922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":91,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:16.726961+0000","last_change":"2024-03-11T19:37:04.587956+0000","last_active":"2024-03-11T19:38:16.726961+0000","last_peered":"2024-03-11T19:38:16.726961+0000","last_clean":"2024-03-11T19:38:16.726961+0000","last_became_active":"2024-03-11T19:37:04.587568+0000","last_became_peered":"2024-03-11T19:37:04.587568+0000","last_unstale":"2024-03-11T19:38:16.726961+0000","last_undegraded":"2024-03-11T19:38:16.726961+0000","last_fullsized":"2024-03-11T19:38:16.726961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:45:44.019436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448645+0000","last_change":"2024-03-11T19:37:05.336912+0000","last_active":"2024-03-11T19:38:10.448645+0000","last_peered":"2024-03-11T19:38:10.448645+0000","last_clean":"2024-03-11T19:38:10.448645+0000","last_became_active":"2024-03-11T19:37:05.336397+0000","last_became_peered":"2024-03-11T19:37:05.336397+0000","last_unstale":"2024-03-11T19:38:10.448645+0000","last_undegraded":"2024-03-11T19:38:10.448645+0000","last_fullsized":"2024-03-11T19:38:10.448645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:29.375396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150574+0000","last_change":"2024-03-11T19:37:04.584382+0000","last_active":"2024-03-11T19:38:10.150574+0000","last_peered":"2024-03-11T19:38:10.150574+0000","last_clean":"2024-03-11T19:38:10.150574+0000","last_became_active":"2024-03-11T19:37:04.584026+0000","last_became_peered":"2024-03-11T19:37:04.584026+0000","last_unstale":"2024-03-11T19:38:10.150574+0000","last_undegraded":"2024-03-11T19:38:10.150574+0000","last_fullsized":"2024-03-11T19:38:10.150574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:28.152903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152473+0000","last_change":"2024-03-11T19:37:05.331385+0000","last_active":"2024-03-11T19:38:10.152473+0000","last_peered":"2024-03-11T19:38:10.152473+0000","last_clean":"2024-03-11T19:38:10.152473+0000","last_became_active":"2024-03-11T19:37:05.331069+0000","last_became_peered":"2024-03-11T19:37:05.331069+0000","last_unstale":"2024-03-11T19:38:10.152473+0000","last_undegraded":"2024-03-11T19:38:10.152473+0000","last_fullsized":"2024-03-11T19:38:10.152473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:18.375605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453568+0000","last_change":"2024-03-11T19:37:04.586493+0000","last_active":"2024-03-11T19:38:10.453568+0000","last_peered":"2024-03-11T19:38:10.453568+0000","last_clean":"2024-03-11T19:38:10.453568+0000","last_became_active":"2024-03-11T19:37:04.586185+0000","last_became_peered":"2024-03-11T19:37:04.586185+0000","last_unstale":"2024-03-11T19:38:10.453568+0000","last_undegraded":"2024-03-11T19:38:10.453568+0000","last_fullsized":"2024-03-11T19:38:10.453568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:00.899752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448466+0000","last_change":"2024-03-11T19:37:05.336414+0000","last_active":"2024-03-11T19:38:10.448466+0000","last_peered":"2024-03-11T19:38:10.448466+0000","last_clean":"2024-03-11T19:38:10.448466+0000","last_became_active":"2024-03-11T19:37:05.334536+0000","last_became_peered":"2024-03-11T19:37:05.334536+0000","last_unstale":"2024-03-11T19:38:10.448466+0000","last_undegraded":"2024-03-11T19:38:10.448466+0000","last_fullsized":"2024-03-11T19:38:10.448466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:50:19.109975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'29","reported_seq":194,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:17.728831+0000","last_change":"2024-03-11T19:37:04.584544+0000","last_active":"2024-03-11T19:38:17.728831+0000","last_peered":"2024-03-11T19:38:17.728831+0000","last_clean":"2024-03-11T19:38:17.728831+0000","last_became_active":"2024-03-11T19:37:04.584198+0000","last_became_peered":"2024-03-11T19:37:04.584198+0000","last_unstale":"2024-03-11T19:38:17.728831+0000","last_undegraded":"2024-03-11T19:38:17.728831+0000","last_fullsized":"2024-03-11T19:38:17.728831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:01:56.744533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":146,"num_read_kb":131,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151763+0000","last_change":"2024-03-11T19:37:05.331077+0000","last_active":"2024-03-11T19:38:10.151763+0000","last_peered":"2024-03-11T19:38:10.151763+0000","last_clean":"2024-03-11T19:38:10.151763+0000","last_became_active":"2024-03-11T19:37:05.330708+0000","last_became_peered":"2024-03-11T19:37:05.330708+0000","last_unstale":"2024-03-11T19:38:10.151763+0000","last_undegraded":"2024-03-11T19:38:10.151763+0000","last_fullsized":"2024-03-11T19:38:10.151763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:54:55.019268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155548+0000","last_change":"2024-03-11T19:37:04.583353+0000","last_active":"2024-03-11T19:38:10.155548+0000","last_peered":"2024-03-11T19:38:10.155548+0000","last_clean":"2024-03-11T19:38:10.155548+0000","last_became_active":"2024-03-11T19:37:04.581974+0000","last_became_peered":"2024-03-11T19:37:04.581974+0000","last_unstale":"2024-03-11T19:38:10.155548+0000","last_undegraded":"2024-03-11T19:38:10.155548+0000","last_fullsized":"2024-03-11T19:38:10.155548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:41.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453141+0000","last_change":"2024-03-11T19:37:05.329498+0000","last_active":"2024-03-11T19:38:10.453141+0000","last_peered":"2024-03-11T19:38:10.453141+0000","last_clean":"2024-03-11T19:38:10.453141+0000","last_became_active":"2024-03-11T19:37:05.329186+0000","last_became_peered":"2024-03-11T19:37:05.329186+0000","last_unstale":"2024-03-11T19:38:10.453141+0000","last_undegraded":"2024-03-11T19:38:10.453141+0000","last_fullsized":"2024-03-11T19:38:10.453141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:33:45.012049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453150+0000","last_change":"2024-03-11T19:37:04.582027+0000","last_active":"2024-03-11T19:38:10.453150+0000","last_peered":"2024-03-11T19:38:10.453150+0000","last_clean":"2024-03-11T19:38:10.453150+0000","last_became_active":"2024-03-11T19:37:04.581678+0000","last_became_peered":"2024-03-11T19:37:04.581678+0000","last_unstale":"2024-03-11T19:38:10.453150+0000","last_undegraded":"2024-03-11T19:38:10.453150+0000","last_fullsized":"2024-03-11T19:38:10.453150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:03:40.688053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.455076+0000","last_change":"2024-03-11T19:38:10.455076+0000","last_active":"2024-03-11T19:38:10.455076+0000","last_peered":"2024-03-11T19:38:10.455076+0000","last_clean":"2024-03-11T19:38:10.455076+0000","last_became_active":"2024-03-11T19:38:10.454631+0000","last_became_peered":"2024-03-11T19:38:10.454631+0000","last_unstale":"2024-03-11T19:38:10.455076+0000","last_undegraded":"2024-03-11T19:38:10.455076+0000","last_fullsized":"2024-03-11T19:38:10.455076+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:26:36.651449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:15.726255+0000","last_change":"2024-03-11T19:37:04.576415+0000","last_active":"2024-03-11T19:38:15.726255+0000","last_peered":"2024-03-11T19:38:15.726255+0000","last_clean":"2024-03-11T19:38:15.726255+0000","last_became_active":"2024-03-11T19:37:04.575978+0000","last_became_peered":"2024-03-11T19:37:04.575978+0000","last_unstale":"2024-03-11T19:38:15.726255+0000","last_undegraded":"2024-03-11T19:38:15.726255+0000","last_fullsized":"2024-03-11T19:38:15.726255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:20.124813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150379+0000","last_change":"2024-03-11T19:37:05.326258+0000","last_active":"2024-03-11T19:38:10.150379+0000","last_peered":"2024-03-11T19:38:10.150379+0000","last_clean":"2024-03-11T19:38:10.150379+0000","last_became_active":"2024-03-11T19:37:05.325909+0000","last_became_peered":"2024-03-11T19:37:05.325909+0000","last_unstale":"2024-03-11T19:38:10.150379+0000","last_undegraded":"2024-03-11T19:38:10.150379+0000","last_fullsized":"2024-03-11T19:38:10.150379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:25.875603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.482823+0000","last_change":"2024-03-11T19:38:10.482380+0000","last_active":"2024-03-11T19:38:10.482823+0000","last_peered":"2024-03-11T19:38:10.482823+0000","last_clean":"2024-03-11T19:38:10.482823+0000","last_became_active":"2024-03-11T19:38:10.455301+0000","last_became_peered":"2024-03-11T19:38:10.455301+0000","last_unstale":"2024-03-11T19:38:10.482823+0000","last_undegraded":"2024-03-11T19:38:10.482823+0000","last_fullsized":"2024-03-11T19:38:10.482823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:28:41.620522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452916+0000","last_change":"2024-03-11T19:37:05.336727+0000","last_active":"2024-03-11T19:38:10.452916+0000","last_peered":"2024-03-11T19:38:10.452916+0000","last_clean":"2024-03-11T19:38:10.452916+0000","last_became_active":"2024-03-11T19:37:05.336384+0000","last_became_peered":"2024-03-11T19:37:05.336384+0000","last_unstale":"2024-03-11T19:38:10.452916+0000","last_undegraded":"2024-03-11T19:38:10.452916+0000","last_fullsized":"2024-03-11T19:38:10.452916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:02.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151827+0000","last_change":"2024-03-11T19:37:04.586814+0000","last_active":"2024-03-11T19:38:10.151827+0000","last_peered":"2024-03-11T19:38:10.151827+0000","last_clean":"2024-03-11T19:38:10.151827+0000","last_became_active":"2024-03-11T19:37:04.586512+0000","last_became_peered":"2024-03-11T19:37:04.586512+0000","last_unstale":"2024-03-11T19:38:10.151827+0000","last_undegraded":"2024-03-11T19:38:10.151827+0000","last_fullsized":"2024-03-11T19:38:10.151827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:04:23.979783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152417+0000","last_change":"2024-03-11T19:37:05.335353+0000","last_active":"2024-03-11T19:38:10.152417+0000","last_peered":"2024-03-11T19:38:10.152417+0000","last_clean":"2024-03-11T19:38:10.152417+0000","last_became_active":"2024-03-11T19:37:05.335066+0000","last_became_peered":"2024-03-11T19:37:05.335066+0000","last_unstale":"2024-03-11T19:38:10.152417+0000","last_undegraded":"2024-03-11T19:38:10.152417+0000","last_fullsized":"2024-03-11T19:38:10.152417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:11.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454552+0000","last_change":"2024-03-11T19:37:04.586679+0000","last_active":"2024-03-11T19:38:10.454552+0000","last_peered":"2024-03-11T19:38:10.454552+0000","last_clean":"2024-03-11T19:38:10.454552+0000","last_became_active":"2024-03-11T19:37:04.586316+0000","last_became_peered":"2024-03-11T19:37:04.586316+0000","last_unstale":"2024-03-11T19:38:10.454552+0000","last_undegraded":"2024-03-11T19:38:10.454552+0000","last_fullsized":"2024-03-11T19:38:10.454552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:58:21.115526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453359+0000","last_change":"2024-03-11T19:37:05.330206+0000","last_active":"2024-03-11T19:38:10.453359+0000","last_peered":"2024-03-11T19:38:10.453359+0000","last_clean":"2024-03-11T19:38:10.453359+0000","last_became_active":"2024-03-11T19:37:05.329847+0000","last_became_peered":"2024-03-11T19:37:05.329847+0000","last_unstale":"2024-03-11T19:38:10.453359+0000","last_undegraded":"2024-03-11T19:38:10.453359+0000","last_fullsized":"2024-03-11T19:38:10.453359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:14:52.532236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150350+0000","last_change":"2024-03-11T19:37:04.581031+0000","last_active":"2024-03-11T19:38:10.150350+0000","last_peered":"2024-03-11T19:38:10.150350+0000","last_clean":"2024-03-11T19:38:10.150350+0000","last_became_active":"2024-03-11T19:37:04.580728+0000","last_became_peered":"2024-03-11T19:37:04.580728+0000","last_unstale":"2024-03-11T19:38:10.150350+0000","last_undegraded":"2024-03-11T19:38:10.150350+0000","last_fullsized":"2024-03-11T19:38:10.150350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:43.401898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453212+0000","last_change":"2024-03-11T19:37:05.328686+0000","last_active":"2024-03-11T19:38:10.453212+0000","last_peered":"2024-03-11T19:38:10.453212+0000","last_clean":"2024-03-11T19:38:10.453212+0000","last_became_active":"2024-03-11T19:37:05.328372+0000","last_became_peered":"2024-03-11T19:37:05.328372+0000","last_unstale":"2024-03-11T19:38:10.453212+0000","last_undegraded":"2024-03-11T19:38:10.453212+0000","last_fullsized":"2024-03-11T19:38:10.453212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:38:33.216489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453162+0000","last_change":"2024-03-11T19:37:05.335057+0000","last_active":"2024-03-11T19:38:10.453162+0000","last_peered":"2024-03-11T19:38:10.453162+0000","last_clean":"2024-03-11T19:38:10.453162+0000","last_became_active":"2024-03-11T19:37:05.334646+0000","last_became_peered":"2024-03-11T19:37:05.334646+0000","last_unstale":"2024-03-11T19:38:10.453162+0000","last_undegraded":"2024-03-11T19:38:10.453162+0000","last_fullsized":"2024-03-11T19:38:10.453162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:34:15.932429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":91,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:15.725775+0000","last_change":"2024-03-11T19:37:04.588095+0000","last_active":"2024-03-11T19:38:15.725775+0000","last_peered":"2024-03-11T19:38:15.725775+0000","last_clean":"2024-03-11T19:38:15.725775+0000","last_became_active":"2024-03-11T19:37:04.587757+0000","last_became_peered":"2024-03-11T19:37:04.587757+0000","last_unstale":"2024-03-11T19:38:15.725775+0000","last_undegraded":"2024-03-11T19:38:15.725775+0000","last_fullsized":"2024-03-11T19:38:15.725775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:57.926858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452696+0000","last_change":"2024-03-11T19:37:05.326806+0000","last_active":"2024-03-11T19:38:10.452696+0000","last_peered":"2024-03-11T19:38:10.452696+0000","last_clean":"2024-03-11T19:38:10.452696+0000","last_became_active":"2024-03-11T19:37:05.326506+0000","last_became_peered":"2024-03-11T19:37:05.326506+0000","last_unstale":"2024-03-11T19:38:10.452696+0000","last_undegraded":"2024-03-11T19:38:10.452696+0000","last_fullsized":"2024-03-11T19:38:10.452696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:20.664857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453750+0000","last_change":"2024-03-11T19:37:04.588459+0000","last_active":"2024-03-11T19:38:10.453750+0000","last_peered":"2024-03-11T19:38:10.453750+0000","last_clean":"2024-03-11T19:38:10.453750+0000","last_became_active":"2024-03-11T19:37:04.588000+0000","last_became_peered":"2024-03-11T19:37:04.588000+0000","last_unstale":"2024-03-11T19:38:10.453750+0000","last_undegraded":"2024-03-11T19:38:10.453750+0000","last_fullsized":"2024-03-11T19:38:10.453750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:20:47.717196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150226+0000","last_change":"2024-03-11T19:35:55.814178+0000","last_active":"2024-03-11T19:38:10.150226+0000","last_peered":"2024-03-11T19:38:10.150226+0000","last_clean":"2024-03-11T19:38:10.150226+0000","last_became_active":"2024-03-11T19:35:55.813697+0000","last_became_peered":"2024-03-11T19:35:55.813697+0000","last_unstale":"2024-03-11T19:38:10.150226+0000","last_undegraded":"2024-03-11T19:38:10.150226+0000","last_fullsized":"2024-03-11T19:38:10.150226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_clean_scrub_stamp":"2024-03-11T19:32:34.872292+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:50:27.486733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453367+0000","last_change":"2024-03-11T19:37:05.319614+0000","last_active":"2024-03-11T19:38:10.453367+0000","last_peered":"2024-03-11T19:38:10.453367+0000","last_clean":"2024-03-11T19:38:10.453367+0000","last_became_active":"2024-03-11T19:37:05.319277+0000","last_became_peered":"2024-03-11T19:37:05.319277+0000","last_unstale":"2024-03-11T19:38:10.453367+0000","last_undegraded":"2024-03-11T19:38:10.453367+0000","last_fullsized":"2024-03-11T19:38:10.453367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:27.329900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150075+0000","last_change":"2024-03-11T19:37:04.582250+0000","last_active":"2024-03-11T19:38:10.150075+0000","last_peered":"2024-03-11T19:38:10.150075+0000","last_clean":"2024-03-11T19:38:10.150075+0000","last_became_active":"2024-03-11T19:37:04.581938+0000","last_became_peered":"2024-03-11T19:37:04.581938+0000","last_unstale":"2024-03-11T19:38:10.150075+0000","last_undegraded":"2024-03-11T19:38:10.150075+0000","last_fullsized":"2024-03-11T19:38:10.150075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:23.466067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151834+0000","last_change":"2024-03-11T19:37:04.590699+0000","last_active":"2024-03-11T19:38:10.151834+0000","last_peered":"2024-03-11T19:38:10.151834+0000","last_clean":"2024-03-11T19:38:10.151834+0000","last_became_active":"2024-03-11T19:37:04.590424+0000","last_became_peered":"2024-03-11T19:37:04.590424+0000","last_unstale":"2024-03-11T19:38:10.151834+0000","last_undegraded":"2024-03-11T19:38:10.151834+0000","last_fullsized":"2024-03-11T19:38:10.151834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:49.075335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453367+0000","last_change":"2024-03-11T19:37:05.323357+0000","last_active":"2024-03-11T19:38:10.453367+0000","last_peered":"2024-03-11T19:38:10.453367+0000","last_clean":"2024-03-11T19:38:10.453367+0000","last_became_active":"2024-03-11T19:37:05.322980+0000","last_became_peered":"2024-03-11T19:37:05.322980+0000","last_unstale":"2024-03-11T19:38:10.453367+0000","last_undegraded":"2024-03-11T19:38:10.453367+0000","last_fullsized":"2024-03-11T19:38:10.453367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:19.106375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454097+0000","last_change":"2024-03-11T19:37:04.585232+0000","last_active":"2024-03-11T19:38:10.454097+0000","last_peered":"2024-03-11T19:38:10.454097+0000","last_clean":"2024-03-11T19:38:10.454097+0000","last_became_active":"2024-03-11T19:37:04.584904+0000","last_became_peered":"2024-03-11T19:37:04.584904+0000","last_unstale":"2024-03-11T19:38:10.454097+0000","last_undegraded":"2024-03-11T19:38:10.454097+0000","last_fullsized":"2024-03-11T19:38:10.454097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:56:46.519961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453200+0000","last_change":"2024-03-11T19:37:05.332019+0000","last_active":"2024-03-11T19:38:10.453200+0000","last_peered":"2024-03-11T19:38:10.453200+0000","last_clean":"2024-03-11T19:38:10.453200+0000","last_became_active":"2024-03-11T19:37:05.331673+0000","last_became_peered":"2024-03-11T19:37:05.331673+0000","last_unstale":"2024-03-11T19:38:10.453200+0000","last_undegraded":"2024-03-11T19:38:10.453200+0000","last_fullsized":"2024-03-11T19:38:10.453200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:49:45.787082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454347+0000","last_change":"2024-03-11T19:37:04.583404+0000","last_active":"2024-03-11T19:38:10.454347+0000","last_peered":"2024-03-11T19:38:10.454347+0000","last_clean":"2024-03-11T19:38:10.454347+0000","last_became_active":"2024-03-11T19:37:04.583025+0000","last_became_peered":"2024-03-11T19:37:04.583025+0000","last_unstale":"2024-03-11T19:38:10.454347+0000","last_undegraded":"2024-03-11T19:38:10.454347+0000","last_fullsized":"2024-03-11T19:38:10.454347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:02:25.202591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152747+0000","last_change":"2024-03-11T19:37:05.331711+0000","last_active":"2024-03-11T19:38:10.152747+0000","last_peered":"2024-03-11T19:38:10.152747+0000","last_clean":"2024-03-11T19:38:10.152747+0000","last_became_active":"2024-03-11T19:37:05.330851+0000","last_became_peered":"2024-03-11T19:37:05.330851+0000","last_unstale":"2024-03-11T19:38:10.152747+0000","last_undegraded":"2024-03-11T19:38:10.152747+0000","last_fullsized":"2024-03-11T19:38:10.152747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:18:26.692068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448230+0000","last_change":"2024-03-11T19:37:04.586501+0000","last_active":"2024-03-11T19:38:10.448230+0000","last_peered":"2024-03-11T19:38:10.448230+0000","last_clean":"2024-03-11T19:38:10.448230+0000","last_became_active":"2024-03-11T19:37:04.586200+0000","last_became_peered":"2024-03-11T19:37:04.586200+0000","last_unstale":"2024-03-11T19:38:10.448230+0000","last_undegraded":"2024-03-11T19:38:10.448230+0000","last_fullsized":"2024-03-11T19:38:10.448230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:56.635375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151255+0000","last_change":"2024-03-11T19:37:05.332402+0000","last_active":"2024-03-11T19:38:10.151255+0000","last_peered":"2024-03-11T19:38:10.151255+0000","last_clean":"2024-03-11T19:38:10.151255+0000","last_became_active":"2024-03-11T19:37:05.332087+0000","last_became_peered":"2024-03-11T19:37:05.332087+0000","last_unstale":"2024-03-11T19:38:10.151255+0000","last_undegraded":"2024-03-11T19:38:10.151255+0000","last_fullsized":"2024-03-11T19:38:10.151255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:10:23.882296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452565+0000","last_change":"2024-03-11T19:37:04.584978+0000","last_active":"2024-03-11T19:38:10.452565+0000","last_peered":"2024-03-11T19:38:10.452565+0000","last_clean":"2024-03-11T19:38:10.452565+0000","last_became_active":"2024-03-11T19:37:04.584556+0000","last_became_peered":"2024-03-11T19:37:04.584556+0000","last_unstale":"2024-03-11T19:38:10.452565+0000","last_undegraded":"2024-03-11T19:38:10.452565+0000","last_fullsized":"2024-03-11T19:38:10.452565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:39:48.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.153417+0000","last_change":"2024-03-11T19:37:05.334453+0000","last_active":"2024-03-11T19:38:10.153417+0000","last_peered":"2024-03-11T19:38:10.153417+0000","last_clean":"2024-03-11T19:38:10.153417+0000","last_became_active":"2024-03-11T19:37:05.334009+0000","last_became_peered":"2024-03-11T19:37:05.334009+0000","last_unstale":"2024-03-11T19:38:10.153417+0000","last_undegraded":"2024-03-11T19:38:10.153417+0000","last_fullsized":"2024-03-11T19:38:10.153417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:43.402674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155193+0000","last_change":"2024-03-11T19:37:04.583181+0000","last_active":"2024-03-11T19:38:10.155193+0000","last_peered":"2024-03-11T19:38:10.155193+0000","last_clean":"2024-03-11T19:38:10.155193+0000","last_became_active":"2024-03-11T19:37:04.582881+0000","last_became_peered":"2024-03-11T19:37:04.582881+0000","last_unstale":"2024-03-11T19:38:10.155193+0000","last_undegraded":"2024-03-11T19:38:10.155193+0000","last_fullsized":"2024-03-11T19:38:10.155193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:05:06.634150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452686+0000","last_change":"2024-03-11T19:37:05.333642+0000","last_active":"2024-03-11T19:38:10.452686+0000","last_peered":"2024-03-11T19:38:10.452686+0000","last_clean":"2024-03-11T19:38:10.452686+0000","last_became_active":"2024-03-11T19:37:05.333121+0000","last_became_peered":"2024-03-11T19:37:05.333121+0000","last_unstale":"2024-03-11T19:38:10.452686+0000","last_undegraded":"2024-03-11T19:38:10.452686+0000","last_fullsized":"2024-03-11T19:38:10.452686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:07:21.217101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.458830+0000","last_change":"2024-03-11T19:38:10.458358+0000","last_active":"2024-03-11T19:38:10.458830+0000","last_peered":"2024-03-11T19:38:10.458830+0000","last_clean":"2024-03-11T19:38:10.458830+0000","last_became_active":"2024-03-11T19:38:10.449890+0000","last_became_peered":"2024-03-11T19:38:10.449890+0000","last_unstale":"2024-03-11T19:38:10.458830+0000","last_undegraded":"2024-03-11T19:38:10.458830+0000","last_fullsized":"2024-03-11T19:38:10.458830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:51:00.265814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150548+0000","last_change":"2024-03-11T19:37:05.326992+0000","last_active":"2024-03-11T19:38:10.150548+0000","last_peered":"2024-03-11T19:38:10.150548+0000","last_clean":"2024-03-11T19:38:10.150548+0000","last_became_active":"2024-03-11T19:37:05.326676+0000","last_became_peered":"2024-03-11T19:37:05.326676+0000","last_unstale":"2024-03-11T19:38:10.150548+0000","last_undegraded":"2024-03-11T19:38:10.150548+0000","last_fullsized":"2024-03-11T19:38:10.150548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:31:10.310650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155451+0000","last_change":"2024-03-11T19:37:04.583631+0000","last_active":"2024-03-11T19:38:10.155451+0000","last_peered":"2024-03-11T19:38:10.155451+0000","last_clean":"2024-03-11T19:38:10.155451+0000","last_became_active":"2024-03-11T19:37:04.583031+0000","last_became_peered":"2024-03-11T19:37:04.583031+0000","last_unstale":"2024-03-11T19:38:10.155451+0000","last_undegraded":"2024-03-11T19:38:10.155451+0000","last_fullsized":"2024-03-11T19:38:10.155451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:20.695496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150761+0000","last_change":"2024-03-11T19:37:05.331008+0000","last_active":"2024-03-11T19:38:10.150761+0000","last_peered":"2024-03-11T19:38:10.150761+0000","last_clean":"2024-03-11T19:38:10.150761+0000","last_became_active":"2024-03-11T19:37:05.330647+0000","last_became_peered":"2024-03-11T19:37:05.330647+0000","last_unstale":"2024-03-11T19:38:10.150761+0000","last_undegraded":"2024-03-11T19:38:10.150761+0000","last_fullsized":"2024-03-11T19:38:10.150761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:07:00.576137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453400+0000","last_change":"2024-03-11T19:37:04.582501+0000","last_active":"2024-03-11T19:38:10.453400+0000","last_peered":"2024-03-11T19:38:10.453400+0000","last_clean":"2024-03-11T19:38:10.453400+0000","last_became_active":"2024-03-11T19:37:04.582076+0000","last_became_peered":"2024-03-11T19:37:04.582076+0000","last_unstale":"2024-03-11T19:38:10.453400+0000","last_undegraded":"2024-03-11T19:38:10.453400+0000","last_fullsized":"2024-03-11T19:38:10.453400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:46:05.359211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150463+0000","last_change":"2024-03-11T19:37:05.329053+0000","last_active":"2024-03-11T19:38:10.150463+0000","last_peered":"2024-03-11T19:38:10.150463+0000","last_clean":"2024-03-11T19:38:10.150463+0000","last_became_active":"2024-03-11T19:37:05.328748+0000","last_became_peered":"2024-03-11T19:37:05.328748+0000","last_unstale":"2024-03-11T19:38:10.150463+0000","last_undegraded":"2024-03-11T19:38:10.150463+0000","last_fullsized":"2024-03-11T19:38:10.150463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:48:55.369515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453324+0000","last_change":"2024-03-11T19:37:04.583370+0000","last_active":"2024-03-11T19:38:10.453324+0000","last_peered":"2024-03-11T19:38:10.453324+0000","last_clean":"2024-03-11T19:38:10.453324+0000","last_became_active":"2024-03-11T19:37:04.583055+0000","last_became_peered":"2024-03-11T19:37:04.583055+0000","last_unstale":"2024-03-11T19:38:10.453324+0000","last_undegraded":"2024-03-11T19:38:10.453324+0000","last_fullsized":"2024-03-11T19:38:10.453324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:04:02.325054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150768+0000","last_change":"2024-03-11T19:37:05.328157+0000","last_active":"2024-03-11T19:38:10.150768+0000","last_peered":"2024-03-11T19:38:10.150768+0000","last_clean":"2024-03-11T19:38:10.150768+0000","last_became_active":"2024-03-11T19:37:05.327839+0000","last_became_peered":"2024-03-11T19:37:05.327839+0000","last_unstale":"2024-03-11T19:38:10.150768+0000","last_undegraded":"2024-03-11T19:38:10.150768+0000","last_fullsized":"2024-03-11T19:38:10.150768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:23:02.297269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.153533+0000","last_change":"2024-03-11T19:37:05.336160+0000","last_active":"2024-03-11T19:38:10.153533+0000","last_peered":"2024-03-11T19:38:10.153533+0000","last_clean":"2024-03-11T19:38:10.153533+0000","last_became_active":"2024-03-11T19:37:05.335850+0000","last_became_peered":"2024-03-11T19:37:05.335850+0000","last_unstale":"2024-03-11T19:38:10.153533+0000","last_undegraded":"2024-03-11T19:38:10.153533+0000","last_fullsized":"2024-03-11T19:38:10.153533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:09.743661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.455362+0000","last_change":"2024-03-11T19:38:10.455362+0000","last_active":"2024-03-11T19:38:10.455362+0000","last_peered":"2024-03-11T19:38:10.455362+0000","last_clean":"2024-03-11T19:38:10.455362+0000","last_became_active":"2024-03-11T19:38:10.454887+0000","last_became_peered":"2024-03-11T19:38:10.454887+0000","last_unstale":"2024-03-11T19:38:10.455362+0000","last_undegraded":"2024-03-11T19:38:10.455362+0000","last_fullsized":"2024-03-11T19:38:10.455362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:19.289829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155142+0000","last_change":"2024-03-11T19:37:05.332357+0000","last_active":"2024-03-11T19:38:10.155142+0000","last_peered":"2024-03-11T19:38:10.155142+0000","last_clean":"2024-03-11T19:38:10.155142+0000","last_became_active":"2024-03-11T19:37:05.332003+0000","last_became_peered":"2024-03-11T19:37:05.332003+0000","last_unstale":"2024-03-11T19:38:10.155142+0000","last_undegraded":"2024-03-11T19:38:10.155142+0000","last_fullsized":"2024-03-11T19:38:10.155142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:39.165572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152540+0000","last_change":"2024-03-11T19:37:05.332388+0000","last_active":"2024-03-11T19:38:10.152540+0000","last_peered":"2024-03-11T19:38:10.152540+0000","last_clean":"2024-03-11T19:38:10.152540+0000","last_became_active":"2024-03-11T19:37:05.332032+0000","last_became_peered":"2024-03-11T19:37:05.332032+0000","last_unstale":"2024-03-11T19:38:10.152540+0000","last_undegraded":"2024-03-11T19:38:10.152540+0000","last_fullsized":"2024-03-11T19:38:10.152540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:56:09.416683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453438+0000","last_change":"2024-03-11T19:37:05.323096+0000","last_active":"2024-03-11T19:38:10.453438+0000","last_peered":"2024-03-11T19:38:10.453438+0000","last_clean":"2024-03-11T19:38:10.453438+0000","last_became_active":"2024-03-11T19:37:05.322525+0000","last_became_peered":"2024-03-11T19:37:05.322525+0000","last_unstale":"2024-03-11T19:38:10.453438+0000","last_undegraded":"2024-03-11T19:38:10.453438+0000","last_fullsized":"2024-03-11T19:38:10.453438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:53:01.870894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.152478+0000","last_change":"2024-03-11T19:37:05.331171+0000","last_active":"2024-03-11T19:38:10.152478+0000","last_peered":"2024-03-11T19:38:10.152478+0000","last_clean":"2024-03-11T19:38:10.152478+0000","last_became_active":"2024-03-11T19:37:05.330769+0000","last_became_peered":"2024-03-11T19:37:05.330769+0000","last_unstale":"2024-03-11T19:38:10.152478+0000","last_undegraded":"2024-03-11T19:38:10.152478+0000","last_fullsized":"2024-03-11T19:38:10.152478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:03:06.514039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155815+0000","last_change":"2024-03-11T19:37:05.326877+0000","last_active":"2024-03-11T19:38:10.155815+0000","last_peered":"2024-03-11T19:38:10.155815+0000","last_clean":"2024-03-11T19:38:10.155815+0000","last_became_active":"2024-03-11T19:37:05.326538+0000","last_became_peered":"2024-03-11T19:37:05.326538+0000","last_unstale":"2024-03-11T19:38:10.155815+0000","last_undegraded":"2024-03-11T19:38:10.155815+0000","last_fullsized":"2024-03-11T19:38:10.155815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:11.565697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453097+0000","last_change":"2024-03-11T19:37:05.335010+0000","last_active":"2024-03-11T19:38:10.453097+0000","last_peered":"2024-03-11T19:38:10.453097+0000","last_clean":"2024-03-11T19:38:10.453097+0000","last_became_active":"2024-03-11T19:37:05.334551+0000","last_became_peered":"2024-03-11T19:37:05.334551+0000","last_unstale":"2024-03-11T19:38:10.453097+0000","last_undegraded":"2024-03-11T19:38:10.453097+0000","last_fullsized":"2024-03-11T19:38:10.453097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:59.668178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452965+0000","last_change":"2024-03-11T19:37:05.336273+0000","last_active":"2024-03-11T19:38:10.452965+0000","last_peered":"2024-03-11T19:38:10.452965+0000","last_clean":"2024-03-11T19:38:10.452965+0000","last_became_active":"2024-03-11T19:37:05.335926+0000","last_became_peered":"2024-03-11T19:37:05.335926+0000","last_unstale":"2024-03-11T19:38:10.452965+0000","last_undegraded":"2024-03-11T19:38:10.452965+0000","last_fullsized":"2024-03-11T19:38:10.452965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:31:55.139242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452598+0000","last_change":"2024-03-11T19:37:05.335548+0000","last_active":"2024-03-11T19:38:10.452598+0000","last_peered":"2024-03-11T19:38:10.452598+0000","last_clean":"2024-03-11T19:38:10.452598+0000","last_became_active":"2024-03-11T19:37:05.335222+0000","last_became_peered":"2024-03-11T19:37:05.335222+0000","last_unstale":"2024-03-11T19:38:10.452598+0000","last_undegraded":"2024-03-11T19:38:10.452598+0000","last_fullsized":"2024-03-11T19:38:10.452598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:25.697951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448454+0000","last_change":"2024-03-11T19:37:05.329782+0000","last_active":"2024-03-11T19:38:10.448454+0000","last_peered":"2024-03-11T19:38:10.448454+0000","last_clean":"2024-03-11T19:38:10.448454+0000","last_became_active":"2024-03-11T19:37:05.329434+0000","last_became_peered":"2024-03-11T19:37:05.329434+0000","last_unstale":"2024-03-11T19:38:10.448454+0000","last_undegraded":"2024-03-11T19:38:10.448454+0000","last_fullsized":"2024-03-11T19:38:10.448454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:12:00.516489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453365+0000","last_change":"2024-03-11T19:37:05.323026+0000","last_active":"2024-03-11T19:38:10.453365+0000","last_peered":"2024-03-11T19:38:10.453365+0000","last_clean":"2024-03-11T19:38:10.453365+0000","last_became_active":"2024-03-11T19:37:05.322562+0000","last_became_peered":"2024-03-11T19:37:05.322562+0000","last_unstale":"2024-03-11T19:38:10.453365+0000","last_undegraded":"2024-03-11T19:38:10.453365+0000","last_fullsized":"2024-03-11T19:38:10.453365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:14:57.606404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.455616+0000","last_change":"2024-03-11T19:38:10.455616+0000","last_active":"2024-03-11T19:38:10.455616+0000","last_peered":"2024-03-11T19:38:10.455616+0000","last_clean":"2024-03-11T19:38:10.455616+0000","last_became_active":"2024-03-11T19:38:10.455092+0000","last_became_peered":"2024-03-11T19:38:10.455092+0000","last_unstale":"2024-03-11T19:38:10.455616+0000","last_undegraded":"2024-03-11T19:38:10.455616+0000","last_fullsized":"2024-03-11T19:38:10.455616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:49:50.407540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64415,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":434,"num_read_kb":285,"num_write":158,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":193245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":455694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1638400,"data_stored":1107477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":455818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":455626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1552384,"data_stored":1048059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":455207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":515023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":966656,"data_stored":456478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":516652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":455518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1564672,"data_stored":1047352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:38:22.768 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-11T19:38:22.768 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-11T19:38:22.769 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-11T19:38:22.769 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph health --format=json 2024-03-11T19:38:23.008 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:23.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:23 smithi083 ceph-mon[31864]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:38:23.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:23 smithi083 ceph-mon[31864]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-03-11T19:38:23.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:23 smithi137 ceph-mon[36220]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:38:23.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:23 smithi137 ceph-mon[36220]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-03-11T19:38:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:23 smithi204 ceph-mon[36739]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:38:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:23 smithi204 ceph-mon[36739]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-03-11T19:38:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.611+0000 7f81bcf83700 1 -- 172.21.15.83:0/1434902816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b8102680 msgr2=0x7f81b810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.611+0000 7f81bcf83700 1 --2- 172.21.15.83:0/1434902816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b8102680 0x7f81b810eac0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f81b000b0d0 tx=0x7f81b000b3e0 comp rx=0 tx=0).stop 2024-03-11T19:38:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.612+0000 7f81bcf83700 1 -- 172.21.15.83:0/1434902816 shutdown_connections 2024-03-11T19:38:24.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.612+0000 7f81bcf83700 1 --2- 172.21.15.83:0/1434902816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b8102680 0x7f81b810eac0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:24.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.612+0000 7f81bcf83700 1 --2- 172.21.15.83:0/1434902816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b8107ca0 0x7f81b8108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:24.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.612+0000 7f81bcf83700 1 --2- 172.21.15.83:0/1434902816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81b8101cc0 0x7f81b8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:24.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.612+0000 7f81bcf83700 1 -- 172.21.15.83:0/1434902816 >> 172.21.15.83:0/1434902816 conn(0x7f81b8078da0 msgr2=0x7f81b80791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:24.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.612+0000 7f81bcf83700 1 -- 172.21.15.83:0/1434902816 shutdown_connections 2024-03-11T19:38:24.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.612+0000 7f81bcf83700 1 -- 172.21.15.83:0/1434902816 wait complete. 2024-03-11T19:38:24.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.612+0000 7f81bcf83700 1 Processor -- start 2024-03-11T19:38:24.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.612+0000 7f81bcf83700 1 -- start start 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81bcf83700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b8101cc0 0x7f81b81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81bcf83700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81b8102680 0x7f81b81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81bcf83700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b8107ca0 0x7f81b819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81bcf83700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b819d240 con 0x7f81b8107ca0 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81bcf83700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b819d3b0 con 0x7f81b8101cc0 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81bcf83700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b819d570 con 0x7f81b8102680 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81b6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b8107ca0 0x7f81b819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81b6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b8107ca0 0x7f81b819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59962/0 (socket says 172.21.15.83:59962) 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81b6d9d700 1 -- 172.21.15.83:0/2294819383 learned_addr learned my addr 172.21.15.83:0/2294819383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81b659c700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b8101cc0 0x7f81b81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81b5d9b700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81b8102680 0x7f81b81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81b6d9d700 1 -- 172.21.15.83:0/2294819383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81b8102680 msgr2=0x7f81b81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81b6d9d700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81b8102680 0x7f81b81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81b6d9d700 1 -- 172.21.15.83:0/2294819383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b8101cc0 msgr2=0x7f81b81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81b6d9d700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b8101cc0 0x7f81b81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81b6d9d700 1 -- 172.21.15.83:0/2294819383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0009d00 con 0x7f81b8107ca0 2024-03-11T19:38:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81b6d9d700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b8107ca0 0x7f81b819cc30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81b00079b0 tx=0x7f81b0007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81a77fe700 1 -- 172.21.15.83:0/2294819383 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b000e040 con 0x7f81b8107ca0 2024-03-11T19:38:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81a77fe700 1 -- 172.21.15.83:0/2294819383 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f81b0004e60 con 0x7f81b8107ca0 2024-03-11T19:38:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81bcf83700 1 -- 172.21.15.83:0/2294819383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b819d7f0 con 0x7f81b8107ca0 2024-03-11T19:38:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.613+0000 7f81a77fe700 1 -- 172.21.15.83:0/2294819383 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b0012b60 con 0x7f81b8107ca0 2024-03-11T19:38:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.614+0000 7f81bcf83700 1 -- 172.21.15.83:0/2294819383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b81a50b0 con 0x7f81b8107ca0 2024-03-11T19:38:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.614+0000 7f81bcf83700 1 -- 172.21.15.83:0/2294819383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b810a180 con 0x7f81b8107ca0 2024-03-11T19:38:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.616+0000 7f81a77fe700 1 -- 172.21.15.83:0/2294819383 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f81b0012cc0 con 0x7f81b8107ca0 2024-03-11T19:38:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.616+0000 7f81a77fe700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f81a00a1b50 0x7f81a00a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.616+0000 7f81b659c700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f81a00a1b50 0x7f81a00a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.616+0000 7f81a77fe700 1 -- 172.21.15.83:0/2294819383 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f81b00cf1d0 con 0x7f81b8107ca0 2024-03-11T19:38:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.617+0000 7f81b659c700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f81a00a1b50 0x7f81a00a4010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f81a8006fd0 tx=0x7f81a8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.620+0000 7f81a77fe700 1 -- 172.21.15.83:0/2294819383 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f81b0098420 con 0x7f81b8107ca0 2024-03-11T19:38:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.861+0000 7f81bcf83700 1 -- 172.21.15.83:0/2294819383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f81b81996c0 con 0x7f81b8107ca0 2024-03-11T19:38:24.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.861+0000 7f81a77fe700 1 -- 172.21.15.83:0/2294819383 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f81b009bae0 con 0x7f81b8107ca0 2024-03-11T19:38:24.866 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:24.866 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-11T19:38:24.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.863+0000 7f81bcf83700 1 -- 172.21.15.83:0/2294819383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f81a00a1b50 msgr2=0x7f81a00a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.863+0000 7f81bcf83700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f81a00a1b50 0x7f81a00a4010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f81a8006fd0 tx=0x7f81a8008040 comp rx=0 tx=0).stop 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.863+0000 7f81bcf83700 1 -- 172.21.15.83:0/2294819383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b8107ca0 msgr2=0x7f81b819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.863+0000 7f81bcf83700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b8107ca0 0x7f81b819cc30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81b00079b0 tx=0x7f81b0007a90 comp rx=0 tx=0).stop 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.863+0000 7f81bcf83700 1 -- 172.21.15.83:0/2294819383 shutdown_connections 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.863+0000 7f81bcf83700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f81a00a1b50 0x7f81a00a4010 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.863+0000 7f81bcf83700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b8107ca0 0x7f81b819cc30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.863+0000 7f81bcf83700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81b8101cc0 0x7f81b81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.863+0000 7f81bcf83700 1 --2- 172.21.15.83:0/2294819383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81b8102680 0x7f81b81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.863+0000 7f81bcf83700 1 -- 172.21.15.83:0/2294819383 >> 172.21.15.83:0/2294819383 conn(0x7f81b8078da0 msgr2=0x7f81b8100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.863+0000 7f81bcf83700 1 -- 172.21.15.83:0/2294819383 shutdown_connections 2024-03-11T19:38:24.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:24.864+0000 7f81bcf83700 1 -- 172.21.15.83:0/2294819383 wait complete. 2024-03-11T19:38:25.488 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-11T19:38:25.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:38:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 -- 172.21.15.83:0/1510042554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc128101c50 msgr2=0x7fc1281084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 --2- 172.21.15.83:0/1510042554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc128101c50 0x7fc1281084e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc120009900 tx=0x7fc12002bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 -- 172.21.15.83:0/1510042554 shutdown_connections 2024-03-11T19:38:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 --2- 172.21.15.83:0/1510042554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc128101c50 0x7fc1281084e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 --2- 172.21.15.83:0/1510042554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc128101290 0x7fc128101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 --2- 172.21.15.83:0/1510042554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1281000a0 0x7fc1281004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 -- 172.21.15.83:0/1510042554 >> 172.21.15.83:0/1510042554 conn(0x7fc1280fb610 msgr2=0x7fc1280fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 -- 172.21.15.83:0/1510042554 shutdown_connections 2024-03-11T19:38:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 -- 172.21.15.83:0/1510042554 wait complete. 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 Processor -- start 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 -- start start 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1281000a0 0x7fc1281971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc128101290 0x7fc128197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc128101c50 0x7fc12819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12819ecb0 con 0x7fc128101c50 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12819ee20 con 0x7fc128101290 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.593+0000 7fc1300b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12819efe0 con 0x7fc1281000a0 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc12e652640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc128101c50 0x7fc12819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc12e652640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc128101c50 0x7fc12819e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59976/0 (socket says 172.21.15.83:59976) 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc12e652640 1 -- 172.21.15.83:0/1726280260 learned_addr learned my addr 172.21.15.83:0/1726280260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc12de51640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1281000a0 0x7fc1281971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:25.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc12e652640 1 -- 172.21.15.83:0/1726280260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1281000a0 msgr2=0x7fc1281971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc12e652640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1281000a0 0x7fc1281971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc12e652640 1 -- 172.21.15.83:0/1726280260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc128101290 msgr2=0x7fc128197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc12e652640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc128101290 0x7fc128197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc12e652640 1 -- 172.21.15.83:0/1726280260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc120009560 con 0x7fc128101c50 2024-03-11T19:38:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc12e652640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc128101c50 0x7fc12819e770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc120009230 tx=0x7fc1200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc126ffd640 1 -- 172.21.15.83:0/1726280260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1200044e0 con 0x7fc128101c50 2024-03-11T19:38:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc1300b5640 1 -- 172.21.15.83:0/1726280260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12819f260 con 0x7fc128101c50 2024-03-11T19:38:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc126ffd640 1 -- 172.21.15.83:0/1726280260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1200054e0 con 0x7fc128101c50 2024-03-11T19:38:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc126ffd640 1 -- 172.21.15.83:0/1726280260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12003e450 con 0x7fc128101c50 2024-03-11T19:38:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.594+0000 7fc1300b5640 1 -- 172.21.15.83:0/1726280260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12819f840 con 0x7fc128101c50 2024-03-11T19:38:25.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.595+0000 7fc1300b5640 1 -- 172.21.15.83:0/1726280260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc128107360 con 0x7fc128101c50 2024-03-11T19:38:25.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.596+0000 7fc126ffd640 1 -- 172.21.15.83:0/1726280260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fc120003680 con 0x7fc128101c50 2024-03-11T19:38:25.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.597+0000 7fc126ffd640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1100a1ac0 0x7fc1100a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:25.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.597+0000 7fc12de51640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1100a1ac0 0x7fc1100a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:25.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.597+0000 7fc126ffd640 1 -- 172.21.15.83:0/1726280260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc1200f01f0 con 0x7fc128101c50 2024-03-11T19:38:25.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.597+0000 7fc12de51640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1100a1ac0 0x7fc1100a3f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc118005c40 tx=0x7fc118005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.600+0000 7fc126ffd640 1 -- 172.21.15.83:0/1726280260 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1200b95e0 con 0x7fc128101c50 2024-03-11T19:38:25.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.774+0000 7fc1300b5640 1 -- 172.21.15.83:0/1726280260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc12802cda0 con 0x7fc128101c50 2024-03-11T19:38:25.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.776+0000 7fc126ffd640 1 -- 172.21.15.83:0/1726280260 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7fc120043020 con 0x7fc128101c50 2024-03-11T19:38:25.781 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:25.781 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:38:25.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 -- 172.21.15.83:0/1726280260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1100a1ac0 msgr2=0x7fc1100a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:25.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1100a1ac0 0x7fc1100a3f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc118005c40 tx=0x7fc118005bd0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 -- 172.21.15.83:0/1726280260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc128101c50 msgr2=0x7fc12819e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:25.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc128101c50 0x7fc12819e770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc120009230 tx=0x7fc1200042f0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 -- 172.21.15.83:0/1726280260 shutdown_connections 2024-03-11T19:38:25.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1100a1ac0 0x7fc1100a3f80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc128101c50 0x7fc12819e770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc128101290 0x7fc128197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 --2- 172.21.15.83:0/1726280260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1281000a0 0x7fc1281971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 -- 172.21.15.83:0/1726280260 >> 172.21.15.83:0/1726280260 conn(0x7fc1280fb610 msgr2=0x7fc128104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:25.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 -- 172.21.15.83:0/1726280260 shutdown_connections 2024-03-11T19:38:25.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.778+0000 7fc1300b5640 1 -- 172.21.15.83:0/1726280260 wait complete. 2024-03-11T19:38:25.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:25 smithi083 ceph-mon[31864]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-03-11T19:38:25.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:25 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2294819383' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-11T19:38:25.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:37:03.034982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:37:04.125352+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:38:25.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:38:25.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:25 smithi137 ceph-mon[36220]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-03-11T19:38:25.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:25 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2294819383' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-11T19:38:25.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.894+0000 7f71a0e0f640 1 -- 172.21.15.83:0/719664518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719c100090 msgr2=0x7f719c1004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:25.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.894+0000 7f71a0e0f640 1 --2- 172.21.15.83:0/719664518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719c100090 0x7f719c1004b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f718c009900 tx=0x7f718c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:25.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.894+0000 7f71a0e0f640 1 -- 172.21.15.83:0/719664518 shutdown_connections 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.894+0000 7f71a0e0f640 1 --2- 172.21.15.83:0/719664518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c101c40 0x7f719c1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.894+0000 7f71a0e0f640 1 --2- 172.21.15.83:0/719664518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c101280 0x7f719c101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.894+0000 7f71a0e0f640 1 --2- 172.21.15.83:0/719664518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719c100090 0x7f719c1004b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.894+0000 7f71a0e0f640 1 -- 172.21.15.83:0/719664518 >> 172.21.15.83:0/719664518 conn(0x7f719c0fb620 msgr2=0x7f719c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.894+0000 7f71a0e0f640 1 -- 172.21.15.83:0/719664518 shutdown_connections 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.894+0000 7f71a0e0f640 1 -- 172.21.15.83:0/719664518 wait complete. 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f71a0e0f640 1 Processor -- start 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f71a0e0f640 1 -- start start 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f71a0e0f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c100090 0x7f719c06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f71a0e0f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719c101280 0x7f719c19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f71a0e0f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c101c40 0x7f719c19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f71a0e0f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c19ed20 con 0x7f719c101280 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f71a0e0f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c19ee90 con 0x7f719c101c40 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f71a0e0f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c19f050 con 0x7f719c100090 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f7193fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719c101280 0x7f719c19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f7193fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719c101280 0x7f719c19bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59994/0 (socket says 172.21.15.83:59994) 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f7193fff640 1 -- 172.21.15.83:0/4127070379 learned_addr learned my addr 172.21.15.83:0/4127070379 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f719ad9d640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c101c40 0x7f719c19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f719a59c640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c100090 0x7f719c06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f7193fff640 1 -- 172.21.15.83:0/4127070379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c100090 msgr2=0x7f719c06a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f7193fff640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c100090 0x7f719c06a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f7193fff640 1 -- 172.21.15.83:0/4127070379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c101c40 msgr2=0x7f719c19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f7193fff640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c101c40 0x7f719c19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f7193fff640 1 -- 172.21.15.83:0/4127070379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c009560 con 0x7f719c101280 2024-03-11T19:38:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f719a59c640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c100090 0x7f719c06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f719ad9d640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c101c40 0x7f719c19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.895+0000 7f7193fff640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719c101280 0x7f719c19bf50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f718400d950 tx=0x7f718400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.896+0000 7f71937fe640 1 -- 172.21.15.83:0/4127070379 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184009c30 con 0x7f719c101280 2024-03-11T19:38:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.896+0000 7f71a0e0f640 1 -- 172.21.15.83:0/4127070379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c19f330 con 0x7f719c101280 2024-03-11T19:38:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.896+0000 7f71937fe640 1 -- 172.21.15.83:0/4127070379 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7184009d90 con 0x7f719c101280 2024-03-11T19:38:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.896+0000 7f71937fe640 1 -- 172.21.15.83:0/4127070379 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718400f7e0 con 0x7f719c101280 2024-03-11T19:38:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.896+0000 7f71a0e0f640 1 -- 172.21.15.83:0/4127070379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c19f890 con 0x7f719c101280 2024-03-11T19:38:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.897+0000 7f71a0e0f640 1 -- 172.21.15.83:0/4127070379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c107430 con 0x7f719c101280 2024-03-11T19:38:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.898+0000 7f71937fe640 1 -- 172.21.15.83:0/4127070379 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f7184010ad0 con 0x7f719c101280 2024-03-11T19:38:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.898+0000 7f71937fe640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71880a1ac0 0x7f71880a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.898+0000 7f719a59c640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71880a1ac0 0x7f71880a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.899+0000 7f71937fe640 1 -- 172.21.15.83:0/4127070379 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f71840cd140 con 0x7f719c101280 2024-03-11T19:38:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.899+0000 7f719a59c640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71880a1ac0 0x7f71880a3f80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f718c036040 tx=0x7f718c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:25.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:25 smithi204 ceph-mon[36739]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-03-11T19:38:25.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:25 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2294819383' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-11T19:38:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:25.902+0000 7f71937fe640 1 -- 172.21.15.83:0/4127070379 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7184096300 con 0x7f719c101280 2024-03-11T19:38:26.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.075+0000 7f71a0e0f640 1 -- 172.21.15.83:0/4127070379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f719c10ba60 con 0x7f719c101280 2024-03-11T19:38:26.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.076+0000 7f71937fe640 1 -- 172.21.15.83:0/4127070379 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71840170a0 con 0x7f719c101280 2024-03-11T19:38:26.080 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:38:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.077+0000 7f71a0e0f640 1 -- 172.21.15.83:0/4127070379 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71880a1ac0 msgr2=0x7f71880a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.077+0000 7f71a0e0f640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71880a1ac0 0x7f71880a3f80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f718c036040 tx=0x7f718c000bc0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.077+0000 7f71a0e0f640 1 -- 172.21.15.83:0/4127070379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719c101280 msgr2=0x7f719c19bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.077+0000 7f71a0e0f640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719c101280 0x7f719c19bf50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f718400d950 tx=0x7f718400de10 comp rx=0 tx=0).stop 2024-03-11T19:38:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.077+0000 7f71a0e0f640 1 -- 172.21.15.83:0/4127070379 shutdown_connections 2024-03-11T19:38:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.077+0000 7f71a0e0f640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71880a1ac0 0x7f71880a3f80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.077+0000 7f71a0e0f640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c101c40 0x7f719c19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.077+0000 7f71a0e0f640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f719c101280 0x7f719c19bf50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.077+0000 7f71a0e0f640 1 --2- 172.21.15.83:0/4127070379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c100090 0x7f719c06a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.077+0000 7f71a0e0f640 1 -- 172.21.15.83:0/4127070379 >> 172.21.15.83:0/4127070379 conn(0x7f719c0fb620 msgr2=0x7f719c104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:26.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.078+0000 7f71a0e0f640 1 -- 172.21.15.83:0/4127070379 shutdown_connections 2024-03-11T19:38:26.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.078+0000 7f71a0e0f640 1 -- 172.21.15.83:0/4127070379 wait complete. 2024-03-11T19:38:26.093 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:38:26.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.193+0000 7f1db1039640 1 -- 172.21.15.83:0/807164600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dac100080 msgr2=0x7f1dac1004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.193+0000 7f1db1039640 1 --2- 172.21.15.83:0/807164600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dac100080 0x7f1dac1004a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009900 tx=0x7f1da002bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:26.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.194+0000 7f1db1039640 1 -- 172.21.15.83:0/807164600 shutdown_connections 2024-03-11T19:38:26.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.194+0000 7f1db1039640 1 --2- 172.21.15.83:0/807164600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac101c30 0x7f1dac1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.194+0000 7f1db1039640 1 --2- 172.21.15.83:0/807164600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dac101270 0x7f1dac1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.194+0000 7f1db1039640 1 --2- 172.21.15.83:0/807164600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dac100080 0x7f1dac1004a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.194+0000 7f1db1039640 1 -- 172.21.15.83:0/807164600 >> 172.21.15.83:0/807164600 conn(0x7f1dac0fb610 msgr2=0x7f1dac0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:26.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.194+0000 7f1db1039640 1 -- 172.21.15.83:0/807164600 shutdown_connections 2024-03-11T19:38:26.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.194+0000 7f1db1039640 1 -- 172.21.15.83:0/807164600 wait complete. 2024-03-11T19:38:26.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.194+0000 7f1db1039640 1 Processor -- start 2024-03-11T19:38:26.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.194+0000 7f1db1039640 1 -- start start 2024-03-11T19:38:26.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1db1039640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dac100080 0x7f1dac068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1db1039640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac101270 0x7f1dac19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1db1039640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dac101c30 0x7f1dac19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1db1039640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac19edb0 con 0x7f1dac100080 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1db1039640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac19ef20 con 0x7f1dac101c30 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1db1039640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac19f0e0 con 0x7f1dac101270 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1daad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dac100080 0x7f1dac068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1daad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dac100080 0x7f1dac068e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60008/0 (socket says 172.21.15.83:60008) 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1daad9d640 1 -- 172.21.15.83:0/244369008 learned_addr learned my addr 172.21.15.83:0/244369008 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1dab59e640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dac101c30 0x7f1dac19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1daa59c640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac101270 0x7f1dac19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1daad9d640 1 -- 172.21.15.83:0/244369008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac101270 msgr2=0x7f1dac19bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1daad9d640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac101270 0x7f1dac19bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1daad9d640 1 -- 172.21.15.83:0/244369008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dac101c30 msgr2=0x7f1dac19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1daad9d640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dac101c30 0x7f1dac19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1daad9d640 1 -- 172.21.15.83:0/244369008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da0009560 con 0x7f1dac100080 2024-03-11T19:38:26.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1daa59c640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac101270 0x7f1dac19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:26.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1dab59e640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dac101c30 0x7f1dac19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:26.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1daad9d640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dac100080 0x7f1dac068e70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1da0000c00 tx=0x7f1da0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:26.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1d93fff640 1 -- 172.21.15.83:0/244369008 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0004df0 con 0x7f1dac100080 2024-03-11T19:38:26.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1db1039640 1 -- 172.21.15.83:0/244369008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac19f360 con 0x7f1dac100080 2024-03-11T19:38:26.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1d93fff640 1 -- 172.21.15.83:0/244369008 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1da0005430 con 0x7f1dac100080 2024-03-11T19:38:26.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1d93fff640 1 -- 172.21.15.83:0/244369008 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da003e450 con 0x7f1dac100080 2024-03-11T19:38:26.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.195+0000 7f1db1039640 1 -- 172.21.15.83:0/244369008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac19f8c0 con 0x7f1dac100080 2024-03-11T19:38:26.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.196+0000 7f1db1039640 1 -- 172.21.15.83:0/244369008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac107440 con 0x7f1dac100080 2024-03-11T19:38:26.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.197+0000 7f1d93fff640 1 -- 172.21.15.83:0/244369008 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f1da0003680 con 0x7f1dac100080 2024-03-11T19:38:26.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.198+0000 7f1d93fff640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d940a1ac0 0x7f1d940a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.198+0000 7f1daa59c640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d940a1ac0 0x7f1d940a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.198+0000 7f1d93fff640 1 -- 172.21.15.83:0/244369008 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1da00ef950 con 0x7f1dac100080 2024-03-11T19:38:26.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.198+0000 7f1daa59c640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d940a1ac0 0x7f1d940a3f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1dac068be0 tx=0x7f1d9c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:26.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.202+0000 7f1d93fff640 1 -- 172.21.15.83:0/244369008 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1da00b8c20 con 0x7f1dac100080 2024-03-11T19:38:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.375+0000 7f1db1039640 1 -- 172.21.15.83:0/244369008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1dac10bc90 con 0x7f1dac100080 2024-03-11T19:38:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.376+0000 7f1d93fff640 1 -- 172.21.15.83:0/244369008 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1da00bc2e0 con 0x7f1dac100080 2024-03-11T19:38:26.380 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-03-11T19:38:26.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.377+0000 7f1db1039640 1 -- 172.21.15.83:0/244369008 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d940a1ac0 msgr2=0x7f1d940a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.377+0000 7f1db1039640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d940a1ac0 0x7f1d940a3f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1dac068be0 tx=0x7f1d9c005e50 comp rx=0 tx=0).stop 2024-03-11T19:38:26.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.377+0000 7f1db1039640 1 -- 172.21.15.83:0/244369008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dac100080 msgr2=0x7f1dac068e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.377+0000 7f1db1039640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dac100080 0x7f1dac068e70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1da0000c00 tx=0x7f1da0004270 comp rx=0 tx=0).stop 2024-03-11T19:38:26.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.378+0000 7f1db1039640 1 -- 172.21.15.83:0/244369008 shutdown_connections 2024-03-11T19:38:26.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.378+0000 7f1db1039640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d940a1ac0 0x7f1d940a3f80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.378+0000 7f1db1039640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dac101c30 0x7f1dac19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.378+0000 7f1db1039640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dac101270 0x7f1dac19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.378+0000 7f1db1039640 1 --2- 172.21.15.83:0/244369008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dac100080 0x7f1dac068e70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.378+0000 7f1db1039640 1 -- 172.21.15.83:0/244369008 >> 172.21.15.83:0/244369008 conn(0x7f1dac0fb610 msgr2=0x7f1dac104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.378+0000 7f1db1039640 1 -- 172.21.15.83:0/244369008 shutdown_connections 2024-03-11T19:38:26.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.378+0000 7f1db1039640 1 -- 172.21.15.83:0/244369008 wait complete. 2024-03-11T19:38:26.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:38:26.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.493+0000 7f319f442640 1 -- 172.21.15.83:0/626650000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3198100090 msgr2=0x7f31981004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.493+0000 7f319f442640 1 --2- 172.21.15.83:0/626650000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3198100090 0x7f31981004b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f318c009900 tx=0x7f318c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:26.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.493+0000 7f319f442640 1 -- 172.21.15.83:0/626650000 shutdown_connections 2024-03-11T19:38:26.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.493+0000 7f319f442640 1 --2- 172.21.15.83:0/626650000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3198101c40 0x7f31981084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.493+0000 7f319f442640 1 --2- 172.21.15.83:0/626650000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198101280 0x7f3198101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.493+0000 7f319f442640 1 --2- 172.21.15.83:0/626650000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3198100090 0x7f31981004b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.493+0000 7f319f442640 1 -- 172.21.15.83:0/626650000 >> 172.21.15.83:0/626650000 conn(0x7f31980fb620 msgr2=0x7f31980fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.493+0000 7f319f442640 1 -- 172.21.15.83:0/626650000 shutdown_connections 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.493+0000 7f319f442640 1 -- 172.21.15.83:0/626650000 wait complete. 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319f442640 1 Processor -- start 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319f442640 1 -- start start 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319f442640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3198100090 0x7f319806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319f442640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3198101280 0x7f319819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319f442640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198101c40 0x7f319819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319f442640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319819ecb0 con 0x7f3198100090 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319f442640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319819ee20 con 0x7f3198101c40 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319f442640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319819efe0 con 0x7f3198101280 2024-03-11T19:38:26.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319d1de640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3198100090 0x7f319806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319d1de640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3198100090 0x7f319806a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60018/0 (socket says 172.21.15.83:60018) 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319d1de640 1 -- 172.21.15.83:0/2421786940 learned_addr learned my addr 172.21.15.83:0/2421786940 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319c9dd640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3198101280 0x7f319819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319d9df640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198101c40 0x7f319819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319d1de640 1 -- 172.21.15.83:0/2421786940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3198101280 msgr2=0x7f319819beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319d1de640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3198101280 0x7f319819beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319d1de640 1 -- 172.21.15.83:0/2421786940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198101c40 msgr2=0x7f319819e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319d1de640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198101c40 0x7f319819e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319d1de640 1 -- 172.21.15.83:0/2421786940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c009560 con 0x7f3198100090 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319d9df640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198101c40 0x7f319819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f319d1de640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3198100090 0x7f319806a320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f318c005ab0 tx=0x7f318c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f318a7fc640 1 -- 172.21.15.83:0/2421786940 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c004aa0 con 0x7f3198100090 2024-03-11T19:38:26.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.494+0000 7f318a7fc640 1 -- 172.21.15.83:0/2421786940 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f318c0052d0 con 0x7f3198100090 2024-03-11T19:38:26.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.495+0000 7f319f442640 1 -- 172.21.15.83:0/2421786940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319819f260 con 0x7f3198100090 2024-03-11T19:38:26.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.495+0000 7f318a7fc640 1 -- 172.21.15.83:0/2421786940 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c003e10 con 0x7f3198100090 2024-03-11T19:38:26.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.495+0000 7f319f442640 1 -- 172.21.15.83:0/2421786940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319819f770 con 0x7f3198100090 2024-03-11T19:38:26.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.496+0000 7f319f442640 1 -- 172.21.15.83:0/2421786940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3198107400 con 0x7f3198100090 2024-03-11T19:38:26.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.497+0000 7f318a7fc640 1 -- 172.21.15.83:0/2421786940 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f318c005440 con 0x7f3198100090 2024-03-11T19:38:26.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.497+0000 7f318a7fc640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f31840a1ac0 0x7f31840a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.497+0000 7f319c9dd640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f31840a1ac0 0x7f31840a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.497+0000 7f318a7fc640 1 -- 172.21.15.83:0/2421786940 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f318c0efe40 con 0x7f3198100090 2024-03-11T19:38:26.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.498+0000 7f319c9dd640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f31840a1ac0 0x7f31840a3f80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f319806a090 tx=0x7f319400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:26.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.501+0000 7f318a7fc640 1 -- 172.21.15.83:0/2421786940 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f318c0b9110 con 0x7f3198100090 2024-03-11T19:38:26.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.677+0000 7f319f442640 1 -- 172.21.15.83:0/2421786940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f319810baa0 con 0x7f3198100090 2024-03-11T19:38:26.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.677+0000 7f318a7fc640 1 -- 172.21.15.83:0/2421786940 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f318c0bc7d0 con 0x7f3198100090 2024-03-11T19:38:26.681 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-03-11T19:38:26.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 -- 172.21.15.83:0/2421786940 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f31840a1ac0 msgr2=0x7f31840a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f31840a1ac0 0x7f31840a3f80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f319806a090 tx=0x7f319400a2d0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 -- 172.21.15.83:0/2421786940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3198100090 msgr2=0x7f319806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3198100090 0x7f319806a320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f318c005ab0 tx=0x7f318c004270 comp rx=0 tx=0).stop 2024-03-11T19:38:26.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 -- 172.21.15.83:0/2421786940 shutdown_connections 2024-03-11T19:38:26.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f31840a1ac0 0x7f31840a3f80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3198101c40 0x7f319819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3198101280 0x7f319819beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 --2- 172.21.15.83:0/2421786940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3198100090 0x7f319806a320 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 -- 172.21.15.83:0/2421786940 >> 172.21.15.83:0/2421786940 conn(0x7f31980fb620 msgr2=0x7f31981045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:26.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 -- 172.21.15.83:0/2421786940 shutdown_connections 2024-03-11T19:38:26.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.679+0000 7f319f442640 1 -- 172.21.15.83:0/2421786940 wait complete. 2024-03-11T19:38:26.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:26 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1726280260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:26.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:26 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4127070379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:26.693 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:26 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/244369008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:26.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:38:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.791+0000 7f0ce16c3640 1 -- 172.21.15.83:0/4228474862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cdc109080 msgr2=0x7f0cdc10b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.791+0000 7f0ce16c3640 1 --2- 172.21.15.83:0/4228474862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cdc109080 0x7f0cdc10b510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009ca0 tx=0x7f0cd402c180 comp rx=0 tx=0).stop 2024-03-11T19:38:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.791+0000 7f0ce16c3640 1 -- 172.21.15.83:0/4228474862 shutdown_connections 2024-03-11T19:38:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.791+0000 7f0ce16c3640 1 --2- 172.21.15.83:0/4228474862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cdc109080 0x7f0cdc10b510 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.791+0000 7f0ce16c3640 1 --2- 172.21.15.83:0/4228474862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc073b60 0x7f0cdc073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.791+0000 7f0ce16c3640 1 --2- 172.21.15.83:0/4228474862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cdc075140 0x7f0cdc073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.791+0000 7f0ce16c3640 1 -- 172.21.15.83:0/4228474862 >> 172.21.15.83:0/4228474862 conn(0x7f0cdc0fc720 msgr2=0x7f0cdc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.791+0000 7f0ce16c3640 1 -- 172.21.15.83:0/4228474862 shutdown_connections 2024-03-11T19:38:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.791+0000 7f0ce16c3640 1 -- 172.21.15.83:0/4228474862 wait complete. 2024-03-11T19:38:26.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0ce16c3640 1 Processor -- start 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0ce16c3640 1 -- start start 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0ce16c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc073b60 0x7f0cdc1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0ce16c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cdc075140 0x7f0cdc199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0ce16c3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cdc109080 0x7f0cdc1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0ce16c3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc1a0ec0 con 0x7f0cdc075140 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0ce16c3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc1a1030 con 0x7f0cdc109080 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0ce16c3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc1a11f0 con 0x7f0cdc073b60 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0cd3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cdc075140 0x7f0cdc199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0cd3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cdc075140 0x7f0cdc199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60038/0 (socket says 172.21.15.83:60038) 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0cd3fff640 1 -- 172.21.15.83:0/345806791 learned_addr learned my addr 172.21.15.83:0/345806791 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0cdb7fe640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cdc109080 0x7f0cdc1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0cdaffd640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc073b60 0x7f0cdc1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0cd3fff640 1 -- 172.21.15.83:0/345806791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc073b60 msgr2=0x7f0cdc1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0cd3fff640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc073b60 0x7f0cdc1993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0cd3fff640 1 -- 172.21.15.83:0/345806791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cdc109080 msgr2=0x7f0cdc1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0cd3fff640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cdc109080 0x7f0cdc1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0cd3fff640 1 -- 172.21.15.83:0/345806791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd4009900 con 0x7f0cdc075140 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.792+0000 7f0cdaffd640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc073b60 0x7f0cdc1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.793+0000 7f0cd3fff640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cdc075140 0x7f0cdc199910 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c920 tx=0x7f0cc400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.793+0000 7f0cd8ff9640 1 -- 172.21.15.83:0/345806791 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4007c70 con 0x7f0cdc075140 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.793+0000 7f0cd8ff9640 1 -- 172.21.15.83:0/345806791 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cc4007dd0 con 0x7f0cdc075140 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.793+0000 7f0ce16c3640 1 -- 172.21.15.83:0/345806791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc1a14d0 con 0x7f0cdc075140 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.793+0000 7f0cd8ff9640 1 -- 172.21.15.83:0/345806791 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4018860 con 0x7f0cdc075140 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.793+0000 7f0ce16c3640 1 -- 172.21.15.83:0/345806791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc1a19e0 con 0x7f0cdc075140 2024-03-11T19:38:26.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.794+0000 7f0ce16c3640 1 -- 172.21.15.83:0/345806791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cdc107e40 con 0x7f0cdc075140 2024-03-11T19:38:26.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.795+0000 7f0cd8ff9640 1 -- 172.21.15.83:0/345806791 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f0cc400fac0 con 0x7f0cdc075140 2024-03-11T19:38:26.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.795+0000 7f0cd8ff9640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cc80a1ac0 0x7f0cc80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.795+0000 7f0cdaffd640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cc80a1ac0 0x7f0cc80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.795+0000 7f0cd8ff9640 1 -- 172.21.15.83:0/345806791 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0cc40cd960 con 0x7f0cdc075140 2024-03-11T19:38:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.796+0000 7f0cdaffd640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cc80a1ac0 0x7f0cc80a3f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc006fd0 tx=0x7f0ccc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:26.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:26.799+0000 7f0cd8ff9640 1 -- 172.21.15.83:0/345806791 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0cc4096bb0 con 0x7f0cdc075140 2024-03-11T19:38:26.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:26 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1726280260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:26.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:26 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4127070379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:26.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:26 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/244369008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:26.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:26 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1726280260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:26.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:26 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4127070379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:26.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:26 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/244369008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:27.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.007+0000 7f0ce16c3640 1 -- 172.21.15.83:0/345806791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cdc10ee30 con 0x7f0cdc075140 2024-03-11T19:38:27.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.008+0000 7f0cd8ff9640 1 -- 172.21.15.83:0/345806791 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f0cc409a270 con 0x7f0cdc075140 2024-03-11T19:38:27.012 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:27.012 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:37:09.619596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:38:27.012 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-03-11T19:38:27.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 -- 172.21.15.83:0/345806791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cc80a1ac0 msgr2=0x7f0cc80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cc80a1ac0 0x7f0cc80a3f80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc006fd0 tx=0x7f0ccc008040 comp rx=0 tx=0).stop 2024-03-11T19:38:27.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 -- 172.21.15.83:0/345806791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cdc075140 msgr2=0x7f0cdc199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cdc075140 0x7f0cdc199910 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c920 tx=0x7f0cc400cde0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 -- 172.21.15.83:0/345806791 shutdown_connections 2024-03-11T19:38:27.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cc80a1ac0 0x7f0cc80a3f80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cdc109080 0x7f0cdc1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cdc075140 0x7f0cdc199910 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 --2- 172.21.15.83:0/345806791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cdc073b60 0x7f0cdc1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 -- 172.21.15.83:0/345806791 >> 172.21.15.83:0/345806791 conn(0x7f0cdc0fc720 msgr2=0x7f0cdc105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 -- 172.21.15.83:0/345806791 shutdown_connections 2024-03-11T19:38:27.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.010+0000 7f0ce16c3640 1 -- 172.21.15.83:0/345806791 wait complete. 2024-03-11T19:38:27.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:38:27.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 -- 172.21.15.83:0/1044496530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a8101290 msgr2=0x7f43a8101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 --2- 172.21.15.83:0/1044496530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a8101290 0x7f43a8101710 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4390009900 tx=0x7f439002bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:27.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 -- 172.21.15.83:0/1044496530 shutdown_connections 2024-03-11T19:38:27.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 --2- 172.21.15.83:0/1044496530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43a8101c50 0x7f43a81084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 --2- 172.21.15.83:0/1044496530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a8101290 0x7f43a8101710 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 --2- 172.21.15.83:0/1044496530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43a81000a0 0x7f43a81004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 -- 172.21.15.83:0/1044496530 >> 172.21.15.83:0/1044496530 conn(0x7f43a80fb610 msgr2=0x7f43a80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 -- 172.21.15.83:0/1044496530 shutdown_connections 2024-03-11T19:38:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 -- 172.21.15.83:0/1044496530 wait complete. 2024-03-11T19:38:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 Processor -- start 2024-03-11T19:38:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 -- start start 2024-03-11T19:38:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a81000a0 0x7f43a8196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43a8101290 0x7f43a8196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43a8101c50 0x7f43a819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a819dc00 con 0x7f43a81000a0 2024-03-11T19:38:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a819dd70 con 0x7f43a8101c50 2024-03-11T19:38:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43ad66f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a819df30 con 0x7f43a8101290 2024-03-11T19:38:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43a6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a81000a0 0x7f43a8196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43a6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a81000a0 0x7f43a8196110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60056/0 (socket says 172.21.15.83:60056) 2024-03-11T19:38:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.122+0000 7f43a6ffd640 1 -- 172.21.15.83:0/3229320796 learned_addr learned my addr 172.21.15.83:0/3229320796 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f439ffff640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43a8101290 0x7f43a8196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f43a6ffd640 1 -- 172.21.15.83:0/3229320796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43a8101290 msgr2=0x7f43a8196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f43a6ffd640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43a8101290 0x7f43a8196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f43a6ffd640 1 -- 172.21.15.83:0/3229320796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43a8101c50 msgr2=0x7f43a819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f43a6ffd640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43a8101c50 0x7f43a819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f43a6ffd640 1 -- 172.21.15.83:0/3229320796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4390009560 con 0x7f43a81000a0 2024-03-11T19:38:27.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f439ffff640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43a8101290 0x7f43a8196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:27.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f43a6ffd640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a81000a0 0x7f43a8196110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f439800d770 tx=0x7f439800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:27.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f43a4ff9640 1 -- 172.21.15.83:0/3229320796 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4398009c30 con 0x7f43a81000a0 2024-03-11T19:38:27.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f43a4ff9640 1 -- 172.21.15.83:0/3229320796 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f439800de80 con 0x7f43a81000a0 2024-03-11T19:38:27.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f43ad66f640 1 -- 172.21.15.83:0/3229320796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a819e210 con 0x7f43a81000a0 2024-03-11T19:38:27.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f43a4ff9640 1 -- 172.21.15.83:0/3229320796 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4398010460 con 0x7f43a81000a0 2024-03-11T19:38:27.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.123+0000 7f43ad66f640 1 -- 172.21.15.83:0/3229320796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a819e720 con 0x7f43a81000a0 2024-03-11T19:38:27.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.124+0000 7f43ad66f640 1 -- 172.21.15.83:0/3229320796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a81073c0 con 0x7f43a81000a0 2024-03-11T19:38:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.125+0000 7f43a4ff9640 1 -- 172.21.15.83:0/3229320796 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f4398003f60 con 0x7f43a81000a0 2024-03-11T19:38:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.126+0000 7f43a4ff9640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43940a1ac0 0x7f43940a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.126+0000 7f439ffff640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43940a1ac0 0x7f43940a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.126+0000 7f43a4ff9640 1 -- 172.21.15.83:0/3229320796 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f43980cd2a0 con 0x7f43a81000a0 2024-03-11T19:38:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.126+0000 7f439ffff640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43940a1ac0 0x7f43940a3f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4390036040 tx=0x7f4390000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:27.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.130+0000 7f43a4ff9640 1 -- 172.21.15.83:0/3229320796 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4398096460 con 0x7f43a81000a0 2024-03-11T19:38:27.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.305+0000 7f43ad66f640 1 -- 172.21.15.83:0/3229320796 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43a802cec0 con 0x7f43a81000a0 2024-03-11T19:38:27.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.306+0000 7f43a4ff9640 1 -- 172.21.15.83:0/3229320796 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7f4398099b20 con 0x7f43a81000a0 2024-03-11T19:38:27.311 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:27.311 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:38:27.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 -- 172.21.15.83:0/3229320796 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43940a1ac0 msgr2=0x7f43940a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43940a1ac0 0x7f43940a3f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4390036040 tx=0x7f4390000bc0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 -- 172.21.15.83:0/3229320796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a81000a0 msgr2=0x7f43a8196110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a81000a0 0x7f43a8196110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f439800d770 tx=0x7f439800dc30 comp rx=0 tx=0).stop 2024-03-11T19:38:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 -- 172.21.15.83:0/3229320796 shutdown_connections 2024-03-11T19:38:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43940a1ac0 0x7f43940a3f80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43a8101c50 0x7f43a819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43a8101290 0x7f43a8196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 --2- 172.21.15.83:0/3229320796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43a81000a0 0x7f43a8196110 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 -- 172.21.15.83:0/3229320796 >> 172.21.15.83:0/3229320796 conn(0x7f43a80fb610 msgr2=0x7f43a8104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 -- 172.21.15.83:0/3229320796 shutdown_connections 2024-03-11T19:38:27.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.308+0000 7f43ad66f640 1 -- 172.21.15.83:0/3229320796 wait complete. 2024-03-11T19:38:27.324 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:37:03.034982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:37:04.125352+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:38:27.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:38:27.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 -- 172.21.15.83:0/2976660911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f310c102d40 msgr2=0x7f310c1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 --2- 172.21.15.83:0/2976660911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f310c102d40 0x7f310c1095d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009900 tx=0x7f30fc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:27.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 -- 172.21.15.83:0/2976660911 shutdown_connections 2024-03-11T19:38:27.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 --2- 172.21.15.83:0/2976660911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f310c102d40 0x7f310c1095d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 --2- 172.21.15.83:0/2976660911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f310c102380 0x7f310c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 --2- 172.21.15.83:0/2976660911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f310c101190 0x7f310c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 -- 172.21.15.83:0/2976660911 >> 172.21.15.83:0/2976660911 conn(0x7f310c0fc720 msgr2=0x7f310c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 -- 172.21.15.83:0/2976660911 shutdown_connections 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 -- 172.21.15.83:0/2976660911 wait complete. 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 Processor -- start 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 -- start start 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f310c101190 0x7f310c197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f310c102380 0x7f310c1976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f310c102d40 0x7f310c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c19ec60 con 0x7f310c101190 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c19edd0 con 0x7f310c102d40 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.425+0000 7f3111688640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c19ef90 con 0x7f310c102380 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f310c101190 0x7f310c197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f310c101190 0x7f310c197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60078/0 (socket says 172.21.15.83:60078) 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310affd640 1 -- 172.21.15.83:0/2979496761 learned_addr learned my addr 172.21.15.83:0/2979496761 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:27.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310a7fc640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f310c102380 0x7f310c1976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310affd640 1 -- 172.21.15.83:0/2979496761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f310c102380 msgr2=0x7f310c1976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310affd640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f310c102380 0x7f310c1976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310affd640 1 -- 172.21.15.83:0/2979496761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f310c102d40 msgr2=0x7f310c19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310b7fe640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f310c102d40 0x7f310c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310affd640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f310c102d40 0x7f310c19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310affd640 1 -- 172.21.15.83:0/2979496761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc009560 con 0x7f310c101190 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310a7fc640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f310c102380 0x7f310c1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310b7fe640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f310c102d40 0x7f310c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f310affd640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f310c101190 0x7f310c197170 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f310400ccb0 tx=0x7f310400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f3103fff640 1 -- 172.21.15.83:0/2979496761 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104003b70 con 0x7f310c101190 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f3111688640 1 -- 172.21.15.83:0/2979496761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c19f270 con 0x7f310c101190 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f3111688640 1 -- 172.21.15.83:0/2979496761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c19f7d0 con 0x7f310c101190 2024-03-11T19:38:27.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f3103fff640 1 -- 172.21.15.83:0/2979496761 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f310400ae50 con 0x7f310c101190 2024-03-11T19:38:27.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.426+0000 7f3103fff640 1 -- 172.21.15.83:0/2979496761 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310400f7d0 con 0x7f310c101190 2024-03-11T19:38:27.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.427+0000 7f3111688640 1 -- 172.21.15.83:0/2979496761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c108450 con 0x7f310c101190 2024-03-11T19:38:27.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.429+0000 7f3103fff640 1 -- 172.21.15.83:0/2979496761 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f3104004000 con 0x7f310c101190 2024-03-11T19:38:27.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.429+0000 7f3103fff640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30f80a1ac0 0x7f30f80a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.430+0000 7f310a7fc640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30f80a1ac0 0x7f30f80a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:27.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.430+0000 7f3103fff640 1 -- 172.21.15.83:0/2979496761 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f31040cdb70 con 0x7f310c101190 2024-03-11T19:38:27.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.430+0000 7f310a7fc640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30f80a1ac0 0x7f30f80a3f80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f30f40058c0 tx=0x7f30f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:27.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.433+0000 7f3103fff640 1 -- 172.21.15.83:0/2979496761 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3104096d30 con 0x7f310c101190 2024-03-11T19:38:27.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.607+0000 7f3111688640 1 -- 172.21.15.83:0/2979496761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f310c02cda0 con 0x7f310c101190 2024-03-11T19:38:27.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.608+0000 7f3103fff640 1 -- 172.21.15.83:0/2979496761 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f310401d0a0 con 0x7f310c101190 2024-03-11T19:38:27.612 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:38:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 -- 172.21.15.83:0/2979496761 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30f80a1ac0 msgr2=0x7f30f80a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30f80a1ac0 0x7f30f80a3f80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f30f40058c0 tx=0x7f30f4005850 comp rx=0 tx=0).stop 2024-03-11T19:38:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 -- 172.21.15.83:0/2979496761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f310c101190 msgr2=0x7f310c197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f310c101190 0x7f310c197170 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f310400ccb0 tx=0x7f310400a620 comp rx=0 tx=0).stop 2024-03-11T19:38:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 -- 172.21.15.83:0/2979496761 shutdown_connections 2024-03-11T19:38:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30f80a1ac0 0x7f30f80a3f80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f310c102d40 0x7f310c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f310c102380 0x7f310c1976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 --2- 172.21.15.83:0/2979496761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f310c101190 0x7f310c197170 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 -- 172.21.15.83:0/2979496761 >> 172.21.15.83:0/2979496761 conn(0x7f310c0fc720 msgr2=0x7f310c105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 -- 172.21.15.83:0/2979496761 shutdown_connections 2024-03-11T19:38:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.610+0000 7f3111688640 1 -- 172.21.15.83:0/2979496761 wait complete. 2024-03-11T19:38:27.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:38:27.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.726+0000 7fe2bf255640 1 -- 172.21.15.83:0/2134604923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b81045f0 msgr2=0x7fe2b81069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.726+0000 7fe2bf255640 1 --2- 172.21.15.83:0/2134604923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b81045f0 0x7fe2b81069e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009900 tx=0x7fe2ac02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.726+0000 7fe2bf255640 1 -- 172.21.15.83:0/2134604923 shutdown_connections 2024-03-11T19:38:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.726+0000 7fe2bf255640 1 --2- 172.21.15.83:0/2134604923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b8106f20 0x7fe2b8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.726+0000 7fe2bf255640 1 --2- 172.21.15.83:0/2134604923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b81045f0 0x7fe2b81069e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.726+0000 7fe2bf255640 1 --2- 172.21.15.83:0/2134604923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b8101cc0 0x7fe2b81040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.726+0000 7fe2bf255640 1 -- 172.21.15.83:0/2134604923 >> 172.21.15.83:0/2134604923 conn(0x7fe2b80fb630 msgr2=0x7fe2b80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.726+0000 7fe2bf255640 1 -- 172.21.15.83:0/2134604923 shutdown_connections 2024-03-11T19:38:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.726+0000 7fe2bf255640 1 -- 172.21.15.83:0/2134604923 wait complete. 2024-03-11T19:38:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bf255640 1 Processor -- start 2024-03-11T19:38:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bf255640 1 -- start start 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bf255640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8101cc0 0x7fe2b8101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bf255640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b81045f0 0x7fe2b80ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bf255640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b8106f20 0x7fe2b81001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bf255640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8100910 con 0x7fe2b8101cc0 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bf255640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8100a80 con 0x7fe2b81045f0 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bf255640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b81a1280 con 0x7fe2b8106f20 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bcff1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8101cc0 0x7fe2b8101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bcff1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8101cc0 0x7fe2b8101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60090/0 (socket says 172.21.15.83:60090) 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bcff1640 1 -- 172.21.15.83:0/2295417711 learned_addr learned my addr 172.21.15.83:0/2295417711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2b7fff640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b81045f0 0x7fe2b80ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bcff1640 1 -- 172.21.15.83:0/2295417711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b8106f20 msgr2=0x7fe2b81001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bcff1640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b8106f20 0x7fe2b81001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bcff1640 1 -- 172.21.15.83:0/2295417711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b81045f0 msgr2=0x7fe2b80ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bcff1640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b81045f0 0x7fe2b80ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bcff1640 1 -- 172.21.15.83:0/2295417711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac009560 con 0x7fe2b8101cc0 2024-03-11T19:38:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.727+0000 7fe2bcff1640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8101cc0 0x7fe2b8101630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000dbb0 tx=0x7fe2b000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.728+0000 7fe2b5ffb640 1 -- 172.21.15.83:0/2295417711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0003b70 con 0x7fe2b8101cc0 2024-03-11T19:38:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.728+0000 7fe2b5ffb640 1 -- 172.21.15.83:0/2295417711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2b00099c0 con 0x7fe2b8101cc0 2024-03-11T19:38:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.728+0000 7fe2bf255640 1 -- 172.21.15.83:0/2295417711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b81a1530 con 0x7fe2b8101cc0 2024-03-11T19:38:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.728+0000 7fe2b5ffb640 1 -- 172.21.15.83:0/2295417711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00106a0 con 0x7fe2b8101cc0 2024-03-11T19:38:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.728+0000 7fe2bf255640 1 -- 172.21.15.83:0/2295417711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b81a1a10 con 0x7fe2b8101cc0 2024-03-11T19:38:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.729+0000 7fe2bf255640 1 -- 172.21.15.83:0/2295417711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b80fd120 con 0x7fe2b8101cc0 2024-03-11T19:38:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.730+0000 7fe2b5ffb640 1 -- 172.21.15.83:0/2295417711 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7fe2b0004020 con 0x7fe2b8101cc0 2024-03-11T19:38:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.730+0000 7fe2b5ffb640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2a00a1ac0 0x7fe2a00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.730+0000 7fe2b7fff640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2a00a1ac0 0x7fe2a00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.730+0000 7fe2b5ffb640 1 -- 172.21.15.83:0/2295417711 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe2b00cd600 con 0x7fe2b8101cc0 2024-03-11T19:38:27.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.731+0000 7fe2b7fff640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2a00a1ac0 0x7fe2a00a3f80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006010 tx=0x7fe2ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.735+0000 7fe2b5ffb640 1 -- 172.21.15.83:0/2295417711 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe2b00967c0 con 0x7fe2b8101cc0 2024-03-11T19:38:27.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:27 smithi137 ceph-mon[36220]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:27.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:27 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2421786940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:27.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:27 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/345806791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:27.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:27 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3229320796' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:27 smithi204 ceph-mon[36739]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:27 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2421786940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:27 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/345806791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:27 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3229320796' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:27.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.907+0000 7fe2bf255640 1 -- 172.21.15.83:0/2295417711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2b810cbd0 con 0x7fe2b8101cc0 2024-03-11T19:38:27.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.908+0000 7fe2b5ffb640 1 -- 172.21.15.83:0/2295417711 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2b000be60 con 0x7fe2b8101cc0 2024-03-11T19:38:27.912 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-03-11T19:38:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 -- 172.21.15.83:0/2295417711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2a00a1ac0 msgr2=0x7fe2a00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2a00a1ac0 0x7fe2a00a3f80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006010 tx=0x7fe2ac000bc0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 -- 172.21.15.83:0/2295417711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8101cc0 msgr2=0x7fe2b8101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8101cc0 0x7fe2b8101630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000dbb0 tx=0x7fe2b000b630 comp rx=0 tx=0).stop 2024-03-11T19:38:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 -- 172.21.15.83:0/2295417711 shutdown_connections 2024-03-11T19:38:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2a00a1ac0 0x7fe2a00a3f80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b8106f20 0x7fe2b81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b81045f0 0x7fe2b80ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 --2- 172.21.15.83:0/2295417711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2b8101cc0 0x7fe2b8101630 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:27.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 -- 172.21.15.83:0/2295417711 >> 172.21.15.83:0/2295417711 conn(0x7fe2b80fb630 msgr2=0x7fe2b8102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:27.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 -- 172.21.15.83:0/2295417711 shutdown_connections 2024-03-11T19:38:27.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:27.910+0000 7fe2bf255640 1 -- 172.21.15.83:0/2295417711 wait complete. 2024-03-11T19:38:27.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:27 smithi083 ceph-mon[31864]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:27.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:27 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2421786940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:27.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:27 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/345806791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:27.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:27 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3229320796' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:27.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:38:28.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.025+0000 7f344404d640 1 -- 172.21.15.83:0/2674465101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f343c1013d0 msgr2=0x7f343c107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.025+0000 7f344404d640 1 --2- 172.21.15.83:0/2674465101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f343c1013d0 0x7f343c107c60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3434009ca0 tx=0x7f343402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.025+0000 7f344404d640 1 -- 172.21.15.83:0/2674465101 shutdown_connections 2024-03-11T19:38:28.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.025+0000 7f344404d640 1 --2- 172.21.15.83:0/2674465101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f343c1013d0 0x7f343c107c60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.025+0000 7f344404d640 1 --2- 172.21.15.83:0/2674465101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f343c100a10 0x7f343c100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.025+0000 7f344404d640 1 --2- 172.21.15.83:0/2674465101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343c0ff820 0x7f343c0ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.025+0000 7f344404d640 1 -- 172.21.15.83:0/2674465101 >> 172.21.15.83:0/2674465101 conn(0x7f343c0fadb0 msgr2=0x7f343c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:28.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.025+0000 7f344404d640 1 -- 172.21.15.83:0/2674465101 shutdown_connections 2024-03-11T19:38:28.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.025+0000 7f344404d640 1 -- 172.21.15.83:0/2674465101 wait complete. 2024-03-11T19:38:28.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f344404d640 1 Processor -- start 2024-03-11T19:38:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f344404d640 1 -- start start 2024-03-11T19:38:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f344404d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f343c0ff820 0x7f343c0fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f344404d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f343c100a10 0x7f343c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f344404d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343c1013d0 0x7f343c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f344404d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c19ed30 con 0x7f343c100a10 2024-03-11T19:38:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f344404d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c19eea0 con 0x7f343c1013d0 2024-03-11T19:38:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f344404d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c19f060 con 0x7f343c0ff820 2024-03-11T19:38:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f34415e8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f343c100a10 0x7f343c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f34415e8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f343c100a10 0x7f343c19bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60116/0 (socket says 172.21.15.83:60116) 2024-03-11T19:38:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f34415e8640 1 -- 172.21.15.83:0/110036690 learned_addr learned my addr 172.21.15.83:0/110036690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:28.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f34425ea640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343c1013d0 0x7f343c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f3441de9640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f343c0ff820 0x7f343c0fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f34415e8640 1 -- 172.21.15.83:0/110036690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f343c0ff820 msgr2=0x7f343c0fceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f34415e8640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f343c0ff820 0x7f343c0fceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f34415e8640 1 -- 172.21.15.83:0/110036690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343c1013d0 msgr2=0x7f343c19e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f34415e8640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343c1013d0 0x7f343c19e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f34415e8640 1 -- 172.21.15.83:0/110036690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434009900 con 0x7f343c100a10 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f34415e8640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f343c100a10 0x7f343c19bf60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d770 tx=0x7f342c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.026+0000 7f3432ffd640 1 -- 172.21.15.83:0/110036690 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c009c30 con 0x7f343c100a10 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.027+0000 7f3432ffd640 1 -- 172.21.15.83:0/110036690 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f342c00de80 con 0x7f343c100a10 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.027+0000 7f344404d640 1 -- 172.21.15.83:0/110036690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c19f340 con 0x7f343c100a10 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.027+0000 7f3432ffd640 1 -- 172.21.15.83:0/110036690 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c004e80 con 0x7f343c100a10 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.027+0000 7f344404d640 1 -- 172.21.15.83:0/110036690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c19f850 con 0x7f343c100a10 2024-03-11T19:38:28.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.028+0000 7f344404d640 1 -- 172.21.15.83:0/110036690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c106f70 con 0x7f343c100a10 2024-03-11T19:38:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.028+0000 7f3432ffd640 1 -- 172.21.15.83:0/110036690 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f342c005020 con 0x7f343c100a10 2024-03-11T19:38:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.029+0000 7f3432ffd640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34280a1b40 0x7f34280a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.029+0000 7f3441de9640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34280a1b40 0x7f34280a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.029+0000 7f3432ffd640 1 -- 172.21.15.83:0/110036690 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f342c0cce20 con 0x7f343c100a10 2024-03-11T19:38:28.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.030+0000 7f3441de9640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34280a1b40 0x7f34280a4000 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3438005d00 tx=0x7f3438005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:28.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.034+0000 7f3432ffd640 1 -- 172.21.15.83:0/110036690 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f342c095fe0 con 0x7f343c100a10 2024-03-11T19:38:28.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.206+0000 7f344404d640 1 -- 172.21.15.83:0/110036690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f343c10b3d0 con 0x7f343c100a10 2024-03-11T19:38:28.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.207+0000 7f3432ffd640 1 -- 172.21.15.83:0/110036690 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f342c0170a0 con 0x7f343c100a10 2024-03-11T19:38:28.211 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-03-11T19:38:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 -- 172.21.15.83:0/110036690 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34280a1b40 msgr2=0x7f34280a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34280a1b40 0x7f34280a4000 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3438005d00 tx=0x7f3438005c50 comp rx=0 tx=0).stop 2024-03-11T19:38:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 -- 172.21.15.83:0/110036690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f343c100a10 msgr2=0x7f343c19bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f343c100a10 0x7f343c19bf60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d770 tx=0x7f342c00dc30 comp rx=0 tx=0).stop 2024-03-11T19:38:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 -- 172.21.15.83:0/110036690 shutdown_connections 2024-03-11T19:38:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34280a1b40 0x7f34280a4000 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f343c1013d0 0x7f343c19e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f343c100a10 0x7f343c19bf60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 --2- 172.21.15.83:0/110036690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f343c0ff820 0x7f343c0fceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 -- 172.21.15.83:0/110036690 >> 172.21.15.83:0/110036690 conn(0x7f343c0fadb0 msgr2=0x7f343c104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 -- 172.21.15.83:0/110036690 shutdown_connections 2024-03-11T19:38:28.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.209+0000 7f344404d640 1 -- 172.21.15.83:0/110036690 wait complete. 2024-03-11T19:38:28.224 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:38:28.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.321+0000 7f4627be9640 1 -- 172.21.15.83:0/3202633440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4620104550 msgr2=0x7f4620106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.321+0000 7f4627be9640 1 --2- 172.21.15.83:0/3202633440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4620104550 0x7f4620106940 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4610009900 tx=0x7f461002bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:28.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.321+0000 7f4627be9640 1 -- 172.21.15.83:0/3202633440 shutdown_connections 2024-03-11T19:38:28.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.321+0000 7f4627be9640 1 --2- 172.21.15.83:0/3202633440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4620106e80 0x7f46201092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.321+0000 7f4627be9640 1 --2- 172.21.15.83:0/3202633440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4620104550 0x7f4620106940 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.321+0000 7f4627be9640 1 --2- 172.21.15.83:0/3202633440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4620101c20 0x7f4620104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.321+0000 7f4627be9640 1 -- 172.21.15.83:0/3202633440 >> 172.21.15.83:0/3202633440 conn(0x7f46200fb610 msgr2=0x7f46200fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:28.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.321+0000 7f4627be9640 1 -- 172.21.15.83:0/3202633440 shutdown_connections 2024-03-11T19:38:28.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.321+0000 7f4627be9640 1 -- 172.21.15.83:0/3202633440 wait complete. 2024-03-11T19:38:28.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4627be9640 1 Processor -- start 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4627be9640 1 -- start start 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4627be9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4620101c20 0x7f4620197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4627be9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4620104550 0x7f4620197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4627be9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4620106e80 0x7f462019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4627be9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462019ed30 con 0x7f4620104550 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4627be9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462019eea0 con 0x7f4620101c20 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4627be9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462019f060 con 0x7f4620106e80 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4625184640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4620104550 0x7f4620197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4625184640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4620104550 0x7f4620197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60132/0 (socket says 172.21.15.83:60132) 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4625184640 1 -- 172.21.15.83:0/31246661 learned_addr learned my addr 172.21.15.83:0/31246661 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4626186640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4620106e80 0x7f462019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4625985640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4620101c20 0x7f4620197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4625184640 1 -- 172.21.15.83:0/31246661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4620106e80 msgr2=0x7f462019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4625184640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4620106e80 0x7f462019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4625184640 1 -- 172.21.15.83:0/31246661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4620101c20 msgr2=0x7f4620197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4625184640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4620101c20 0x7f4620197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.322+0000 7f4625184640 1 -- 172.21.15.83:0/31246661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610009560 con 0x7f4620104550 2024-03-11T19:38:28.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.323+0000 7f4625184640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4620104550 0x7f4620197780 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f4610004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:28.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.323+0000 7f4616ffd640 1 -- 172.21.15.83:0/31246661 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610004df0 con 0x7f4620104550 2024-03-11T19:38:28.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.323+0000 7f4616ffd640 1 -- 172.21.15.83:0/31246661 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4610005430 con 0x7f4620104550 2024-03-11T19:38:28.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.323+0000 7f4627be9640 1 -- 172.21.15.83:0/31246661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462019f2e0 con 0x7f4620104550 2024-03-11T19:38:28.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.323+0000 7f4616ffd640 1 -- 172.21.15.83:0/31246661 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461003e450 con 0x7f4620104550 2024-03-11T19:38:28.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.323+0000 7f4627be9640 1 -- 172.21.15.83:0/31246661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462019f840 con 0x7f4620104550 2024-03-11T19:38:28.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.324+0000 7f4627be9640 1 -- 172.21.15.83:0/31246661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46200fd100 con 0x7f4620104550 2024-03-11T19:38:28.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.325+0000 7f4616ffd640 1 -- 172.21.15.83:0/31246661 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f4610003680 con 0x7f4620104550 2024-03-11T19:38:28.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.325+0000 7f4616ffd640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f460c0a1b40 0x7f460c0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.326+0000 7f4625985640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f460c0a1b40 0x7f460c0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.326+0000 7f4616ffd640 1 -- 172.21.15.83:0/31246661 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f46100efe90 con 0x7f4620104550 2024-03-11T19:38:28.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.326+0000 7f4625985640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f460c0a1b40 0x7f460c0a4000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f461c0058c0 tx=0x7f461c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:28.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.329+0000 7f4616ffd640 1 -- 172.21.15.83:0/31246661 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f46100b9160 con 0x7f4620104550 2024-03-11T19:38:28.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.539+0000 7f4627be9640 1 -- 172.21.15.83:0/31246661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f462010ce00 con 0x7f4620104550 2024-03-11T19:38:28.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.540+0000 7f4616ffd640 1 -- 172.21.15.83:0/31246661 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f46100bc820 con 0x7f4620104550 2024-03-11T19:38:28.544 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:28.544 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:37:09.619596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:38:28.544 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-03-11T19:38:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 -- 172.21.15.83:0/31246661 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f460c0a1b40 msgr2=0x7f460c0a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f460c0a1b40 0x7f460c0a4000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f461c0058c0 tx=0x7f461c00a4c0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 -- 172.21.15.83:0/31246661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4620104550 msgr2=0x7f4620197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4620104550 0x7f4620197780 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f4610004270 comp rx=0 tx=0).stop 2024-03-11T19:38:28.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 -- 172.21.15.83:0/31246661 shutdown_connections 2024-03-11T19:38:28.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f460c0a1b40 0x7f460c0a4000 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4620106e80 0x7f462019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4620104550 0x7f4620197780 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 --2- 172.21.15.83:0/31246661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4620101c20 0x7f4620197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 -- 172.21.15.83:0/31246661 >> 172.21.15.83:0/31246661 conn(0x7f46200fb610 msgr2=0x7f4620102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:28.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 -- 172.21.15.83:0/31246661 shutdown_connections 2024-03-11T19:38:28.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.542+0000 7f4627be9640 1 -- 172.21.15.83:0/31246661 wait complete. 2024-03-11T19:38:28.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:38:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.657+0000 7ff640f25640 1 -- 172.21.15.83:0/3749092135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c101c50 msgr2=0x7ff63c1084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.657+0000 7ff640f25640 1 --2- 172.21.15.83:0/3749092135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c101c50 0x7ff63c1084e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff634009ca0 tx=0x7ff63402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.657+0000 7ff640f25640 1 -- 172.21.15.83:0/3749092135 shutdown_connections 2024-03-11T19:38:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.657+0000 7ff640f25640 1 --2- 172.21.15.83:0/3749092135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c101c50 0x7ff63c1084e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.657+0000 7ff640f25640 1 --2- 172.21.15.83:0/3749092135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff63c101290 0x7ff63c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.657+0000 7ff640f25640 1 --2- 172.21.15.83:0/3749092135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff63c1000a0 0x7ff63c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.657+0000 7ff640f25640 1 -- 172.21.15.83:0/3749092135 >> 172.21.15.83:0/3749092135 conn(0x7ff63c0fb610 msgr2=0x7ff63c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.657+0000 7ff640f25640 1 -- 172.21.15.83:0/3749092135 shutdown_connections 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.657+0000 7ff640f25640 1 -- 172.21.15.83:0/3749092135 wait complete. 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff640f25640 1 Processor -- start 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff640f25640 1 -- start start 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff640f25640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff63c1000a0 0x7ff63c06a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff640f25640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff63c101290 0x7ff63c19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff640f25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c101c50 0x7ff63c19e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff640f25640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c19ecc0 con 0x7ff63c101c50 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff640f25640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c19ee30 con 0x7ff63c1000a0 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff640f25640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c19eff0 con 0x7ff63c101290 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff63ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c101c50 0x7ff63c19e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff63ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c101c50 0x7ff63c19e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60148/0 (socket says 172.21.15.83:60148) 2024-03-11T19:38:28.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff63ad9d640 1 -- 172.21.15.83:0/2205606375 learned_addr learned my addr 172.21.15.83:0/2205606375 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff639d9b640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff63c101290 0x7ff63c19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff63a59c640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff63c1000a0 0x7ff63c06a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff63ad9d640 1 -- 172.21.15.83:0/2205606375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff63c101290 msgr2=0x7ff63c19bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff63ad9d640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff63c101290 0x7ff63c19bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff63ad9d640 1 -- 172.21.15.83:0/2205606375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff63c1000a0 msgr2=0x7ff63c06a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff63ad9d640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff63c1000a0 0x7ff63c06a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff63ad9d640 1 -- 172.21.15.83:0/2205606375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff634009900 con 0x7ff63c101c50 2024-03-11T19:38:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.658+0000 7ff63ad9d640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c101c50 0x7ff63c19e530 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff634037040 tx=0x7ff634034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.659+0000 7ff62b7fe640 1 -- 172.21.15.83:0/2205606375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634032eb0 con 0x7ff63c101c50 2024-03-11T19:38:28.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.659+0000 7ff640f25640 1 -- 172.21.15.83:0/2205606375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c19f270 con 0x7ff63c101c50 2024-03-11T19:38:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.659+0000 7ff62b7fe640 1 -- 172.21.15.83:0/2205606375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff634003e50 con 0x7ff63c101c50 2024-03-11T19:38:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.659+0000 7ff62b7fe640 1 -- 172.21.15.83:0/2205606375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634040450 con 0x7ff63c101c50 2024-03-11T19:38:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.659+0000 7ff640f25640 1 -- 172.21.15.83:0/2205606375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c19f770 con 0x7ff63c101c50 2024-03-11T19:38:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.659+0000 7ff640f25640 1 -- 172.21.15.83:0/2205606375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63c1072f0 con 0x7ff63c101c50 2024-03-11T19:38:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.661+0000 7ff62b7fe640 1 -- 172.21.15.83:0/2205606375 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7ff63400a040 con 0x7ff63c101c50 2024-03-11T19:38:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.661+0000 7ff62b7fe640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6240a1ac0 0x7ff6240a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.661+0000 7ff63a59c640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6240a1ac0 0x7ff6240a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.661+0000 7ff62b7fe640 1 -- 172.21.15.83:0/2205606375 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff6340f06c0 con 0x7ff63c101c50 2024-03-11T19:38:28.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.662+0000 7ff63a59c640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6240a1ac0 0x7ff6240a3f80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff62c005fd0 tx=0x7ff62c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:28.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.665+0000 7ff62b7fe640 1 -- 172.21.15.83:0/2205606375 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff6340b9990 con 0x7ff63c101c50 2024-03-11T19:38:28.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:28 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2979496761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:28.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:28 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2295417711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:28.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:28 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/110036690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:28.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:28 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/31246661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:28.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.838+0000 7ff640f25640 1 -- 172.21.15.83:0/2205606375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff63c10bcb0 con 0x7ff63c101c50 2024-03-11T19:38:28.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.839+0000 7ff62b7fe640 1 -- 172.21.15.83:0/2205606375 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7ff6340dbe60 con 0x7ff63c101c50 2024-03-11T19:38:28.844 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:28.844 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:38:28.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 -- 172.21.15.83:0/2205606375 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6240a1ac0 msgr2=0x7ff6240a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6240a1ac0 0x7ff6240a3f80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff62c005fd0 tx=0x7ff62c00b5b0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 -- 172.21.15.83:0/2205606375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c101c50 msgr2=0x7ff63c19e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c101c50 0x7ff63c19e530 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff634037040 tx=0x7ff634034d40 comp rx=0 tx=0).stop 2024-03-11T19:38:28.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 -- 172.21.15.83:0/2205606375 shutdown_connections 2024-03-11T19:38:28.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6240a1ac0 0x7ff6240a3f80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c101c50 0x7ff63c19e530 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff63c101290 0x7ff63c19bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 --2- 172.21.15.83:0/2205606375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff63c1000a0 0x7ff63c06a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 -- 172.21.15.83:0/2205606375 >> 172.21.15.83:0/2205606375 conn(0x7ff63c0fb610 msgr2=0x7ff63c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:28.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 -- 172.21.15.83:0/2205606375 shutdown_connections 2024-03-11T19:38:28.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.841+0000 7ff640f25640 1 -- 172.21.15.83:0/2205606375 wait complete. 2024-03-11T19:38:28.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:38:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:28 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2979496761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:28 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2295417711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:28 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/110036690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:28.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:28 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/31246661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:28 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2979496761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:28 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2295417711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:28.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:28 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/110036690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:28.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:28 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/31246661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.955+0000 7f573e62e640 1 -- 172.21.15.83:0/3623954071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738102d50 msgr2=0x7f57381095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.955+0000 7f573e62e640 1 --2- 172.21.15.83:0/3623954071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738102d50 0x7f57381095e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5730009ca0 tx=0x7f573002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.955+0000 7f573e62e640 1 -- 172.21.15.83:0/3623954071 shutdown_connections 2024-03-11T19:38:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.955+0000 7f573e62e640 1 --2- 172.21.15.83:0/3623954071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738102d50 0x7f57381095e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.955+0000 7f573e62e640 1 --2- 172.21.15.83:0/3623954071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738102390 0x7f5738102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.955+0000 7f573e62e640 1 --2- 172.21.15.83:0/3623954071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57381011a0 0x7f57381015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.955+0000 7f573e62e640 1 -- 172.21.15.83:0/3623954071 >> 172.21.15.83:0/3623954071 conn(0x7f57380fc730 msgr2=0x7f57380feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.955+0000 7f573e62e640 1 -- 172.21.15.83:0/3623954071 shutdown_connections 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.955+0000 7f573e62e640 1 -- 172.21.15.83:0/3623954071 wait complete. 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f573e62e640 1 Processor -- start 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f573e62e640 1 -- start start 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f573e62e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57381011a0 0x7f5738074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f573e62e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738102390 0x7f5738073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f573e62e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738102d50 0x7f5738073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f573e62e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5738074210 con 0x7f5738102390 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f573e62e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5738074380 con 0x7f57381011a0 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f573e62e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57381a11f0 con 0x7f5738102d50 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f57377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738102390 0x7f5738073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f57377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738102390 0x7f5738073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60166/0 (socket says 172.21.15.83:60166) 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f57377fe640 1 -- 172.21.15.83:0/844207491 learned_addr learned my addr 172.21.15.83:0/844207491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f5737fff640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57381011a0 0x7f5738074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f573cbcb640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738102d50 0x7f5738073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f57377fe640 1 -- 172.21.15.83:0/844207491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738102d50 msgr2=0x7f5738073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f57377fe640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738102d50 0x7f5738073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f57377fe640 1 -- 172.21.15.83:0/844207491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57381011a0 msgr2=0x7f5738074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f57377fe640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57381011a0 0x7f5738074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f57377fe640 1 -- 172.21.15.83:0/844207491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5730009900 con 0x7f5738102390 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.956+0000 7f573cbcb640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738102d50 0x7f5738073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.957+0000 7f57377fe640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738102390 0x7f5738073570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d980 tx=0x7f572c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.957+0000 7f57357fa640 1 -- 172.21.15.83:0/844207491 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c009a30 con 0x7f5738102390 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.957+0000 7f573e62e640 1 -- 172.21.15.83:0/844207491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57381a14a0 con 0x7f5738102390 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.957+0000 7f57357fa640 1 -- 172.21.15.83:0/844207491 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f572c009b90 con 0x7f5738102390 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.957+0000 7f57357fa640 1 -- 172.21.15.83:0/844207491 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c004a00 con 0x7f5738102390 2024-03-11T19:38:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.957+0000 7f573e62e640 1 -- 172.21.15.83:0/844207491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57381a1a00 con 0x7f5738102390 2024-03-11T19:38:28.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.958+0000 7f573e62e640 1 -- 172.21.15.83:0/844207491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57381084e0 con 0x7f5738102390 2024-03-11T19:38:28.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.959+0000 7f57357fa640 1 -- 172.21.15.83:0/844207491 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f572c009d00 con 0x7f5738102390 2024-03-11T19:38:28.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.959+0000 7f57357fa640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57200a1ac0 0x7f57200a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:28.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.959+0000 7f5737fff640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57200a1ac0 0x7f57200a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:28.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.959+0000 7f57357fa640 1 -- 172.21.15.83:0/844207491 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f572c0cce80 con 0x7f5738102390 2024-03-11T19:38:28.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.960+0000 7f5737fff640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57200a1ac0 0x7f57200a3f80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f57381021f0 tx=0x7f572800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:28.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:28.964+0000 7f57357fa640 1 -- 172.21.15.83:0/844207491 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f572c096040 con 0x7f5738102390 2024-03-11T19:38:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.171+0000 7f573e62e640 1 -- 172.21.15.83:0/844207491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f573810cbb0 con 0x7f5738102390 2024-03-11T19:38:29.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.172+0000 7f57357fa640 1 -- 172.21.15.83:0/844207491 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f572c017090 con 0x7f5738102390 2024-03-11T19:38:29.176 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:29.176 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:37:09.619596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:38:29.176 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-03-11T19:38:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 -- 172.21.15.83:0/844207491 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57200a1ac0 msgr2=0x7f57200a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57200a1ac0 0x7f57200a3f80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f57381021f0 tx=0x7f572800b390 comp rx=0 tx=0).stop 2024-03-11T19:38:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 -- 172.21.15.83:0/844207491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738102390 msgr2=0x7f5738073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738102390 0x7f5738073570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d980 tx=0x7f572c00de40 comp rx=0 tx=0).stop 2024-03-11T19:38:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 -- 172.21.15.83:0/844207491 shutdown_connections 2024-03-11T19:38:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57200a1ac0 0x7f57200a3f80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5738102d50 0x7f5738073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5738102390 0x7f5738073570 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 --2- 172.21.15.83:0/844207491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57381011a0 0x7f5738074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 -- 172.21.15.83:0/844207491 >> 172.21.15.83:0/844207491 conn(0x7f57380fc730 msgr2=0x7f57381056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 -- 172.21.15.83:0/844207491 shutdown_connections 2024-03-11T19:38:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.174+0000 7f573e62e640 1 -- 172.21.15.83:0/844207491 wait complete. 2024-03-11T19:38:29.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-11T19:37:04.916864+0000', 'modified': '2024-03-11T19:37:09.619596+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24527, 'mds_1': 24502, 'mds_2': 24494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/2604685946', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 2604685946}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 2604685946}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24470': {'gid': 24470, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/1924725912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 1924725912}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 1924725912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/2654992874', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 2654992874}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 2654992874}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.137:6837/3499358906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 3499358906}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 3499358906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.204:6839/862560872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 862560872}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 862560872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.137:6839/709970923', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 709970923}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 709970923}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-11T19:38:29.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-11T19:38:29.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-11T19:38:29.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.288+0000 7f9c7c423640 1 -- 172.21.15.83:0/1816614104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c74104610 msgr2=0x7f9c74106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.288+0000 7f9c7c423640 1 --2- 172.21.15.83:0/1816614104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c74104610 0x7f9c74106a00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009900 tx=0x7f9c6402bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.288+0000 7f9c7c423640 1 -- 172.21.15.83:0/1816614104 shutdown_connections 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.288+0000 7f9c7c423640 1 --2- 172.21.15.83:0/1816614104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c74106f40 0x7f9c74109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.288+0000 7f9c7c423640 1 --2- 172.21.15.83:0/1816614104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c74104610 0x7f9c74106a00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.288+0000 7f9c7c423640 1 --2- 172.21.15.83:0/1816614104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c74101ce0 0x7f9c741040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.288+0000 7f9c7c423640 1 -- 172.21.15.83:0/1816614104 >> 172.21.15.83:0/1816614104 conn(0x7f9c740fb610 msgr2=0x7f9c740fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.288+0000 7f9c7c423640 1 -- 172.21.15.83:0/1816614104 shutdown_connections 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.288+0000 7f9c7c423640 1 -- 172.21.15.83:0/1816614104 wait complete. 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7c423640 1 Processor -- start 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7c423640 1 -- start start 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7c423640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c74101ce0 0x7f9c74197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7c423640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c74104610 0x7f9c74197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7c423640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c74106f40 0x7f9c7419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7c423640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7419edd0 con 0x7f9c74106f40 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7c423640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7419ef40 con 0x7f9c74104610 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7c423640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7419f100 con 0x7f9c74101ce0 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7a9c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c74106f40 0x7f9c7419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7a9c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c74106f40 0x7f9c7419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60198/0 (socket says 172.21.15.83:60198) 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7a9c0640 1 -- 172.21.15.83:0/2244771231 learned_addr learned my addr 172.21.15.83:0/2244771231 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c74101ce0 0x7f9c74197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c799be640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c74104610 0x7f9c74197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7a9c0640 1 -- 172.21.15.83:0/2244771231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c74101ce0 msgr2=0x7f9c74197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7a9c0640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c74101ce0 0x7f9c74197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7a9c0640 1 -- 172.21.15.83:0/2244771231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c74104610 msgr2=0x7f9c74197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7a9c0640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c74104610 0x7f9c74197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7a9c0640 1 -- 172.21.15.83:0/2244771231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c64009560 con 0x7f9c74106f40 2024-03-11T19:38:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c799be640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c74104610 0x7f9c74197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c74101ce0 0x7f9c74197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c7a9c0640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c74106f40 0x7f9c7419e890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00eb10 tx=0x7f9c6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.289+0000 7f9c6b7fe640 1 -- 172.21.15.83:0/2244771231 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c004b20 con 0x7f9c74106f40 2024-03-11T19:38:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.290+0000 7f9c6b7fe640 1 -- 172.21.15.83:0/2244771231 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c6c004c80 con 0x7f9c74106f40 2024-03-11T19:38:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.290+0000 7f9c7c423640 1 -- 172.21.15.83:0/2244771231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7419f3e0 con 0x7f9c74106f40 2024-03-11T19:38:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.290+0000 7f9c6b7fe640 1 -- 172.21.15.83:0/2244771231 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c010650 con 0x7f9c74106f40 2024-03-11T19:38:29.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.290+0000 7f9c7c423640 1 -- 172.21.15.83:0/2244771231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7419f940 con 0x7f9c74106f40 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.292+0000 7f9c6b7fe640 1 -- 172.21.15.83:0/2244771231 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f9c6c005020 con 0x7f9c74106f40 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.292+0000 7f9c7eff7740 1 Processor -- start 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.292+0000 7f9c7eff7740 1 -- start start 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.292+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 0x55742da1b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.292+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da1b970 0x55742da20470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.292+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da20ae0 0x55742da22f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.292+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742d9d6cd0 con 0x55742da1b970 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.292+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742d989f20 con 0x55742da1af80 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.292+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742d987450 con 0x55742da20ae0 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.292+0000 7f9c7a1bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da1b970 0x55742da20470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c6b7fe640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9c600a1b40 0x7f9c600a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c7a1bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da1b970 0x55742da20470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60208/0 (socket says 172.21.15.83:60208) 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c7a1bf640 1 -- 172.21.15.83:0/246220096 learned_addr learned my addr 172.21.15.83:0/246220096 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c7a9c0640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9c600a1b40 0x7f9c600a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c799be640 1 --2- 172.21.15.83:0/246220096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da20ae0 0x55742da22f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c799be640 1 --2- 172.21.15.83:0/246220096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 0x55742da1b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c7a1bf640 1 -- 172.21.15.83:0/246220096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da20ae0 msgr2=0x55742da22f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c6b7fe640 1 -- 172.21.15.83:0/2244771231 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9c6c014070 con 0x7f9c74106f40 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/246220096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da20ae0 0x55742da22f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c7a1bf640 1 -- 172.21.15.83:0/246220096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 msgr2=0x55742da1b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/246220096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 0x55742da1b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c7a1bf640 1 -- 172.21.15.83:0/246220096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55742d937f20 con 0x55742da1b970 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c799be640 1 --2- 172.21.15.83:0/246220096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da20ae0 0x55742da22f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/246220096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da1b970 0x55742da20470 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000bd40 tx=0x7f9c7000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bad3cdd263a552a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c7a9c0640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9c600a1b40 0x7f9c600a4000 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c0049d0 tx=0x7f9c6c069de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c68bfe640 1 -- 172.21.15.83:0/246220096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7001a660 con 0x55742da1b970 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c68bfe640 1 -- 172.21.15.83:0/246220096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c7000be30 con 0x55742da1b970 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.293+0000 7f9c68bfe640 1 -- 172.21.15.83:0/246220096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70011b50 con 0x55742da1b970 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.294+0000 7f9c7eff7740 1 -- 172.21.15.83:0/246220096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da1b970 msgr2=0x55742da20470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.294+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/246220096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da1b970 0x55742da20470 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000bd40 tx=0x7f9c7000f730 comp rx=0 tx=0).stop 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.294+0000 7f9c7eff7740 1 -- 172.21.15.83:0/246220096 shutdown_connections 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.294+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/246220096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da20ae0 0x55742da22f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.294+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/246220096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da1b970 0x55742da20470 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.294+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/246220096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 0x55742da1b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.294+0000 7f9c7eff7740 1 -- 172.21.15.83:0/246220096 >> 172.21.15.83:0/246220096 conn(0x55742da164f0 msgr2=0x55742da168f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.294+0000 7f9c7eff7740 1 -- 172.21.15.83:0/246220096 shutdown_connections 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.294+0000 7f9c7eff7740 1 -- 172.21.15.83:0/246220096 wait complete. 2024-03-11T19:38:29.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.294+0000 7f9c7eff7740 1 -- start start 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.294+0000 7f9c7eff7740 1 Processor -- start 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1af80 0x55742da25160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1b970 0x55742da256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da2c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742d982c30 con 0x55742da20ae0 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742d927d30 con 0x55742da1b970 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742d656310 con 0x55742da1af80 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c7eff7740 10 client.0 _ll_register_callbacks cb 0x55742d92fd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c799be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da2c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c799be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da2c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60220/0 (socket says 172.21.15.83:60220) 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c799be640 1 -- 172.21.15.83:0/458201487 learned_addr learned my addr 172.21.15.83:0/458201487 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c799be640 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1af80 0x55742da25160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c799be640 1 -- 172.21.15.83:0/458201487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1af80 msgr2=0x55742da25160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c799be640 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1af80 0x55742da25160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c799be640 1 -- 172.21.15.83:0/458201487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1b970 msgr2=0x55742da256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1b970 0x55742da256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c799be640 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1b970 0x55742da256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c799be640 1 -- 172.21.15.83:0/458201487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7000b9a0 con 0x55742da20ae0 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1b970 0x55742da256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c799be640 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da2c6e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9c6402c6a0 tx=0x7f9c64009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e074116c46cc51 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c5dffb640 1 -- 172.21.15.83:0/458201487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c640044f0 con 0x55742da20ae0 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c5dffb640 1 -- 172.21.15.83:0/458201487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9c640035f0 con 0x55742da20ae0 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c7eff7740 1 -- 172.21.15.83:0/458201487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55742d6826c0 con 0x55742da20ae0 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.295+0000 7f9c5dffb640 1 -- 172.21.15.83:0/458201487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c640037b0 con 0x55742da20ae0 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.296+0000 7f9c7eff7740 10 client.15012 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:29.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.296+0000 7f9c7eff7740 1 -- 172.21.15.83:0/458201487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55742d9812b0 con 0x55742da20ae0 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.296+0000 7f9c5dffb640 1 -- 172.21.15.83:0/458201487 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c64055020 con 0x55742da20ae0 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.296+0000 7f9c5dffb640 1 -- 172.21.15.83:0/458201487 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9c640053f0 con 0x55742da20ae0 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.296+0000 7f9c7eff7740 10 client.15012 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.296+0000 7f9c7eff7740 10 client.15012 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.296+0000 7f9c7eff7740 20 client.15012 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.296+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55742da31a60 0x55742da30d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.296+0000 7f9c7eff7740 4 client.15012 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.296+0000 7f9c7eff7740 1 -- 172.21.15.83:0/458201487 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55742d6d3270 con 0x55742da31a60 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.296+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55742da31a60 0x55742da30d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.297+0000 7f9c5dffb640 10 client.15012 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.297+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55742da31a60 0x55742da30d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a1bd215488c1726 server_cookie=652198bcece08d9d in_seq=0 out_seq=0 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.297+0000 7f9c5dffb640 1 -- 172.21.15.83:0/458201487 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55742d6d3270 con 0x55742da31a60 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.297+0000 7f9c5dffb640 10 client.15012 handle_command_reply: tid=0 2024-03-11T19:38:29.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.297+0000 7f9c7eff7740 1 client.15012 shutdown 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.297+0000 7f9c7eff7740 1 -- 172.21.15.83:0/458201487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 msgr2=0x55742da2c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.297+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da2c6e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9c6402c6a0 tx=0x7f9c64009260 comp rx=0 tx=0).stop 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.298+0000 7f9c7eff7740 1 -- 172.21.15.83:0/458201487 shutdown_connections 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.298+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55742da31a60 0x55742da30d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.298+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da2c6e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.298+0000 7f9c5dffb640 0 client.15012 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.298+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1b970 0x55742da256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.298+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/458201487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1af80 0x55742da25160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.298+0000 7f9c7eff7740 1 -- 172.21.15.83:0/458201487 >> 172.21.15.83:0/458201487 conn(0x55742da164f0 msgr2=0x55742da157d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.298+0000 7f9c7eff7740 1 -- 172.21.15.83:0/458201487 shutdown_connections 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.298+0000 7f9c7eff7740 1 -- 172.21.15.83:0/458201487 wait complete. 2024-03-11T19:38:29.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.298+0000 7f9c7eff7740 20 client.15012 trim_cache size 0 max 16384 2024-03-11T19:38:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.313+0000 7f9c7eff7740 1 Processor -- start 2024-03-11T19:38:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.313+0000 7f9c7eff7740 1 -- start start 2024-03-11T19:38:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.313+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 0x55742da24430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.313+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1b970 0x55742da24970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.313+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da14f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.313+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742d908490 con 0x55742da20ae0 2024-03-11T19:38:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.313+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742da33a40 con 0x55742da1af80 2024-03-11T19:38:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.313+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742d6d3270 con 0x55742da1b970 2024-03-11T19:38:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.313+0000 7f9c799be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da14f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.313+0000 7f9c799be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da14f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60232/0 (socket says 172.21.15.83:60232) 2024-03-11T19:38:29.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.313+0000 7f9c799be640 1 -- 172.21.15.83:0/1414816156 learned_addr learned my addr 172.21.15.83:0/1414816156 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/1414816156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1b970 0x55742da24970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c799be640 1 -- 172.21.15.83:0/1414816156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1b970 msgr2=0x55742da24970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c799be640 1 --2- 172.21.15.83:0/1414816156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1b970 0x55742da24970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c799be640 1 -- 172.21.15.83:0/1414816156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 msgr2=0x55742da24430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c799be640 1 --2- 172.21.15.83:0/1414816156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 0x55742da24430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c799be640 1 -- 172.21.15.83:0/1414816156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55742d91da20 con 0x55742da20ae0 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c799be640 1 --2- 172.21.15.83:0/1414816156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da14f40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9c64004ae0 tx=0x7f9c64000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e87c122ab4c3d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c5ffff640 1 -- 172.21.15.83:0/1414816156 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c640044f0 con 0x55742da20ae0 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c5ffff640 1 -- 172.21.15.83:0/1414816156 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c64055070 con 0x55742da20ae0 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c5ffff640 1 -- 172.21.15.83:0/1414816156 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c64005210 con 0x55742da20ae0 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c7eff7740 1 -- 172.21.15.83:0/1414816156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 msgr2=0x55742da14f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/1414816156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da14f40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9c64004ae0 tx=0x7f9c64000bc0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c7eff7740 1 -- 172.21.15.83:0/1414816156 shutdown_connections 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/1414816156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da14f40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/1414816156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1b970 0x55742da24970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/1414816156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 0x55742da24430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c7eff7740 1 -- 172.21.15.83:0/1414816156 >> 172.21.15.83:0/1414816156 conn(0x55742da164f0 msgr2=0x55742da1cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c7eff7740 1 -- 172.21.15.83:0/1414816156 shutdown_connections 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.314+0000 7f9c7eff7740 1 -- 172.21.15.83:0/1414816156 wait complete. 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c7eff7740 1 -- start start 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c7eff7740 1 Processor -- start 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 0x55742da84f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1b970 0x55742da85470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c7eff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da8c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742da8c9f0 con 0x55742da20ae0 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742da8cb30 con 0x55742da1af80 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c7eff7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742da8cca0 con 0x55742da1b970 2024-03-11T19:38:29.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c7eff7740 10 client.0 _ll_register_callbacks cb 0x7f9c7000b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c799be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da8c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c799be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da8c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60248/0 (socket says 172.21.15.83:60248) 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c799be640 1 -- 172.21.15.83:0/4192426152 learned_addr learned my addr 172.21.15.83:0/4192426152 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c799be640 1 -- 172.21.15.83:0/4192426152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1b970 msgr2=0x55742da85470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c799be640 1 --2- 172.21.15.83:0/4192426152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1b970 0x55742da85470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c799be640 1 -- 172.21.15.83:0/4192426152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 msgr2=0x55742da84f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c799be640 1 --2- 172.21.15.83:0/4192426152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 0x55742da84f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c799be640 1 -- 172.21.15.83:0/4192426152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55742d9812b0 con 0x55742da20ae0 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c799be640 1 --2- 172.21.15.83:0/4192426152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da8c4b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9c6402cde0 tx=0x7f9c64005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19586e2593ed62fd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.315+0000 7f9c5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c5dffb640 1 -- 172.21.15.83:0/4192426152 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c64005190 con 0x55742da20ae0 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c5dffb640 1 -- 172.21.15.83:0/4192426152 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9c64000e40 con 0x55742da20ae0 2024-03-11T19:38:29.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c7eff7740 1 -- 172.21.15.83:0/4192426152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55742da32a90 con 0x55742da20ae0 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c5dffb640 1 -- 172.21.15.83:0/4192426152 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c640515a0 con 0x55742da20ae0 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c7eff7740 10 client.15018 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c7eff7740 1 -- 172.21.15.83:0/4192426152 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55742d943780 con 0x55742da20ae0 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c5dffb640 1 -- 172.21.15.83:0/4192426152 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55742da32a90 con 0x55742da20ae0 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c5dffb640 1 -- 172.21.15.83:0/4192426152 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9c64003990 con 0x55742da20ae0 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c7eff7740 10 client.15018 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c7eff7740 10 client.15018 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c7eff7740 20 client.15018 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/4192426152 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55742da91b20 0x55742da8df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c7eff7740 4 client.15018 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c7eff7740 1 -- 172.21.15.83:0/4192426152 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f9c7000be30 con 0x55742da91b20 2024-03-11T19:38:29.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.316+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/4192426152 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55742da91b20 0x55742da8df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c5dffb640 10 client.15018 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7a1bf640 1 --2- 172.21.15.83:0/4192426152 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55742da91b20 0x55742da8df90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94303139ff214c50 server_cookie=1969f57c113fc590 in_seq=0 out_seq=0 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c5dffb640 1 -- 172.21.15.83:0/4192426152 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f9c7000be30 con 0x55742da91b20 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c5dffb640 10 client.15018 handle_command_reply: tid=0 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 1 client.15018 shutdown 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 1 -- 172.21.15.83:0/4192426152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 msgr2=0x55742da8c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/4192426152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da8c4b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9c6402cde0 tx=0x7f9c64005fd0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 1 -- 172.21.15.83:0/4192426152 shutdown_connections 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/4192426152 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55742da91b20 0x55742da8df90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/4192426152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55742da20ae0 0x55742da8c4b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c5dffb640 0 client.15018 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/4192426152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55742da1b970 0x55742da85470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 1 --2- 172.21.15.83:0/4192426152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742da1af80 0x55742da84f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 1 -- 172.21.15.83:0/4192426152 >> 172.21.15.83:0/4192426152 conn(0x55742da164f0 msgr2=0x55742da19fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 1 -- 172.21.15.83:0/4192426152 shutdown_connections 2024-03-11T19:38:29.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 1 -- 172.21.15.83:0/4192426152 wait complete. 2024-03-11T19:38:29.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.317+0000 7f9c7eff7740 20 client.15018 trim_cache size 0 max 16384 2024-03-11T19:38:29.328 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:38:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "839f93c8-dfdd-11ee-95c8-87774f69a715", 2024-03-11T19:38:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-03-11T19:38:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24494, 2024-03-11T19:38:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-03-11T19:38:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-03-11T19:38:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-03-11T19:38:29.328 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 80.701685596000004, 2024-03-11T19:38:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 21, 2024-03-11T19:38:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 87, 2024-03-11T19:38:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 85, 2024-03-11T19:38:29.329 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 100.180299969 2024-03-11T19:38:29.329 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 -- 172.21.15.83:0/2244771231 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9c600a1b40 msgr2=0x7f9c600a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9c600a1b40 0x7f9c600a4000 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c0049d0 tx=0x7f9c6c069de0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 -- 172.21.15.83:0/2244771231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c74106f40 msgr2=0x7f9c7419e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c74106f40 0x7f9c7419e890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00eb10 tx=0x7f9c6c00c2d0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 -- 172.21.15.83:0/2244771231 shutdown_connections 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9c600a1b40 0x7f9c600a4000 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c74106f40 0x7f9c7419e890 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c74104610 0x7f9c74197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 --2- 172.21.15.83:0/2244771231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c74101ce0 0x7f9c74197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 -- 172.21.15.83:0/2244771231 >> 172.21.15.83:0/2244771231 conn(0x7f9c740fb610 msgr2=0x7f9c74102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 -- 172.21.15.83:0/2244771231 shutdown_connections 2024-03-11T19:38:29.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.318+0000 7f9c7c423640 1 -- 172.21.15.83:0/2244771231 wait complete. 2024-03-11T19:38:29.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-11T19:38:29.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.431+0000 7f9526a15640 1 -- 172.21.15.83:0/1545942631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9520102d60 msgr2=0x7f95201095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.431+0000 7f9526a15640 1 --2- 172.21.15.83:0/1545942631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9520102d60 0x7f95201095f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9518009ca0 tx=0x7f951802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.431+0000 7f9526a15640 1 -- 172.21.15.83:0/1545942631 shutdown_connections 2024-03-11T19:38:29.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.431+0000 7f9526a15640 1 --2- 172.21.15.83:0/1545942631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9520102d60 0x7f95201095f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.431+0000 7f9526a15640 1 --2- 172.21.15.83:0/1545942631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95201023a0 0x7f9520102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.431+0000 7f9526a15640 1 --2- 172.21.15.83:0/1545942631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95201011b0 0x7f95201015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.431+0000 7f9526a15640 1 -- 172.21.15.83:0/1545942631 >> 172.21.15.83:0/1545942631 conn(0x7f95200fc720 msgr2=0x7f95200feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.431+0000 7f9526a15640 1 -- 172.21.15.83:0/1545942631 shutdown_connections 2024-03-11T19:38:29.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.431+0000 7f9526a15640 1 -- 172.21.15.83:0/1545942631 wait complete. 2024-03-11T19:38:29.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.431+0000 7f9526a15640 1 Processor -- start 2024-03-11T19:38:29.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.431+0000 7f9526a15640 1 -- start start 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9526a15640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95201011b0 0x7f95201971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9526a15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95201023a0 0x7f9520197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9526a15640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9520102d60 0x7f952019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9526a15640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952019ece0 con 0x7f9520102d60 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9526a15640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952019ee50 con 0x7f95201023a0 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9526a15640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952019f010 con 0x7f95201011b0 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9524fb2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9520102d60 0x7f952019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9524fb2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9520102d60 0x7f952019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60268/0 (socket says 172.21.15.83:60268) 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9524fb2640 1 -- 172.21.15.83:0/982930187 learned_addr learned my addr 172.21.15.83:0/982930187 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f951f7fe640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95201023a0 0x7f9520197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f951ffff640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95201011b0 0x7f95201971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9524fb2640 1 -- 172.21.15.83:0/982930187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95201011b0 msgr2=0x7f95201971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9524fb2640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95201011b0 0x7f95201971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9524fb2640 1 -- 172.21.15.83:0/982930187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95201023a0 msgr2=0x7f9520197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9524fb2640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95201023a0 0x7f9520197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9524fb2640 1 -- 172.21.15.83:0/982930187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9518009900 con 0x7f9520102d60 2024-03-11T19:38:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f951f7fe640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95201023a0 0x7f9520197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9524fb2640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9520102d60 0x7f952019e7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9518037040 tx=0x7f9518034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f951d7fa640 1 -- 172.21.15.83:0/982930187 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518032eb0 con 0x7f9520102d60 2024-03-11T19:38:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9526a15640 1 -- 172.21.15.83:0/982930187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952019f290 con 0x7f9520102d60 2024-03-11T19:38:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f951d7fa640 1 -- 172.21.15.83:0/982930187 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9518003e50 con 0x7f9520102d60 2024-03-11T19:38:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f951d7fa640 1 -- 172.21.15.83:0/982930187 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518040450 con 0x7f9520102d60 2024-03-11T19:38:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.432+0000 7f9526a15640 1 -- 172.21.15.83:0/982930187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952019f790 con 0x7f9520102d60 2024-03-11T19:38:29.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.434+0000 7f951d7fa640 1 -- 172.21.15.83:0/982930187 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f951800a040 con 0x7f9520102d60 2024-03-11T19:38:29.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f95295ee740 1 Processor -- start 2024-03-11T19:38:29.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f951d7fa640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f95080a1ac0 0x7f95080a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f95295ee740 1 -- start start 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f951ffff640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f95080a1ac0 0x7f95080a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f951d7fa640 1 -- 172.21.15.83:0/982930187 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f95180f04a0 con 0x7f9520102d60 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f95295ee740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f95295ee740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e59970 0x5572a7e5e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f95295ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e5eae0 0x5572a7e60f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f95295ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7e14cd0 con 0x5572a7e5eae0 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f95295ee740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7dc7f20 con 0x5572a7e59970 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f95295ee740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7dc5450 con 0x5572a7e58f80 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f9524fb2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e5eae0 0x5572a7e60f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f9524fb2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e5eae0 0x5572a7e60f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60270/0 (socket says 172.21.15.83:60270) 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f9524fb2640 1 -- 172.21.15.83:0/2996091420 learned_addr learned my addr 172.21.15.83:0/2996091420 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f951f7fe640 1 --2- 172.21.15.83:0/2996091420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e59970 0x5572a7e5e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f951ffff640 1 --2- 172.21.15.83:0/2996091420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f9524fb2640 1 -- 172.21.15.83:0/2996091420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 msgr2=0x5572a7e593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f951ffff640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f95080a1ac0 0x7f95080a3f80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9510005fd0 tx=0x7f9510009ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f9524fb2640 1 --2- 172.21.15.83:0/2996091420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f9524fb2640 1 -- 172.21.15.83:0/2996091420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e59970 msgr2=0x5572a7e5e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f9524fb2640 1 --2- 172.21.15.83:0/2996091420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e59970 0x5572a7e5e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.435+0000 7f9524fb2640 1 -- 172.21.15.83:0/2996091420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572a7d75f20 con 0x5572a7e5eae0 2024-03-11T19:38:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f9524fb2640 1 --2- 172.21.15.83:0/2996091420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e5eae0 0x5572a7e60f70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f95180bd360 tx=0x7f95180bcdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b1f32852de6e9a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95067fc640 1 -- 172.21.15.83:0/2996091420 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95180bd3d0 con 0x5572a7e5eae0 2024-03-11T19:38:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95067fc640 1 -- 172.21.15.83:0/2996091420 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95180c1520 con 0x5572a7e5eae0 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95067fc640 1 -- 172.21.15.83:0/2996091420 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95180c1170 con 0x5572a7e5eae0 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95295ee740 1 -- 172.21.15.83:0/2996091420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e5eae0 msgr2=0x5572a7e60f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95295ee740 1 --2- 172.21.15.83:0/2996091420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e5eae0 0x5572a7e60f70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f95180bd360 tx=0x7f95180bcdf0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95295ee740 1 -- 172.21.15.83:0/2996091420 shutdown_connections 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95295ee740 1 --2- 172.21.15.83:0/2996091420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e5eae0 0x5572a7e60f70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95295ee740 1 --2- 172.21.15.83:0/2996091420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e59970 0x5572a7e5e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95295ee740 1 --2- 172.21.15.83:0/2996091420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95295ee740 1 -- 172.21.15.83:0/2996091420 >> 172.21.15.83:0/2996091420 conn(0x5572a7e544f0 msgr2=0x5572a7e548f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95295ee740 1 -- 172.21.15.83:0/2996091420 shutdown_connections 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.436+0000 7f95295ee740 1 -- 172.21.15.83:0/2996091420 wait complete. 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f95295ee740 1 -- start start 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f95295ee740 1 Processor -- start 2024-03-11T19:38:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f95295ee740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e62f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f95295ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f95295ee740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e5eae0 0x5572a7e6a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f95295ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7dc0c30 con 0x5572a7e59970 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f95295ee740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7d65d30 con 0x5572a7e5eae0 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f95295ee740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7a94310 con 0x5572a7e58f80 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f95295ee740 10 client.0 _ll_register_callbacks cb 0x5572a7d6dd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f951f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f951f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60282/0 (socket says 172.21.15.83:60282) 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f951f7fe640 1 -- 172.21.15.83:0/3307191030 learned_addr learned my addr 172.21.15.83:0/3307191030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f9524fb2640 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e5eae0 0x5572a7e6a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f951ffff640 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e62f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f951f7fe640 1 -- 172.21.15.83:0/3307191030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 msgr2=0x5572a7e62f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.437+0000 7f951f7fe640 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e62f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f951f7fe640 1 -- 172.21.15.83:0/3307191030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e5eae0 msgr2=0x5572a7e6a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f951f7fe640 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e5eae0 0x5572a7e6a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f951f7fe640 1 -- 172.21.15.83:0/3307191030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952019f790 con 0x5572a7e59970 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f9524fb2640 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e5eae0 0x5572a7e6a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f951f7fe640 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9520198700 tx=0x7f95140117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea6e79e44c77d99 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f94ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f94ff7fe640 1 -- 172.21.15.83:0/3307191030 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514004220 con 0x5572a7e59970 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f94ff7fe640 1 -- 172.21.15.83:0/3307191030 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9514004380 con 0x5572a7e59970 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f95295ee740 1 -- 172.21.15.83:0/3307191030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572a7ac06c0 con 0x5572a7e59970 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f94ff7fe640 1 -- 172.21.15.83:0/3307191030 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9514012660 con 0x5572a7e59970 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f95295ee740 10 client.15030 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f95295ee740 1 -- 172.21.15.83:0/3307191030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5572a7dbf2b0 con 0x5572a7e59970 2024-03-11T19:38:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f94ff7fe640 1 -- 172.21.15.83:0/3307191030 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5572a7ac06c0 con 0x5572a7e59970 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f94ff7fe640 1 -- 172.21.15.83:0/3307191030 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f951400c070 con 0x5572a7e59970 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f95295ee740 10 client.15030 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f95295ee740 10 client.15030 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f95295ee740 20 client.15030 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f95295ee740 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5572a7e6f990 0x5572a7e6ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f95295ee740 4 client.15030 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.438+0000 7f95295ee740 1 -- 172.21.15.83:0/3307191030 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572a7b11270 con 0x5572a7e6f990 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.439+0000 7f951ffff640 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5572a7e6f990 0x5572a7e6ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.439+0000 7f94ff7fe640 10 client.15030 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.439+0000 7f951ffff640 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5572a7e6f990 0x5572a7e6ebe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b2d0587c4da49f8 server_cookie=a6fd407c5d36d052 in_seq=0 out_seq=0 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f94ff7fe640 1 -- 172.21.15.83:0/3307191030 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5572a7b11270 con 0x5572a7e6f990 2024-03-11T19:38:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f94ff7fe640 10 client.15030 handle_command_reply: tid=0 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f95295ee740 1 client.15030 shutdown 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f95295ee740 1 -- 172.21.15.83:0/3307191030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 msgr2=0x5572a7e634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f95295ee740 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9520198700 tx=0x7f95140117b0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f95295ee740 1 -- 172.21.15.83:0/3307191030 shutdown_connections 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f95295ee740 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5572a7e6f990 0x5572a7e6ebe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f95295ee740 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e5eae0 0x5572a7e6a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f95295ee740 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f94ff7fe640 0 client.15030 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f95295ee740 1 --2- 172.21.15.83:0/3307191030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e62f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f95295ee740 1 -- 172.21.15.83:0/3307191030 >> 172.21.15.83:0/3307191030 conn(0x5572a7e544f0 msgr2=0x5572a7e537d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f95295ee740 1 -- 172.21.15.83:0/3307191030 shutdown_connections 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.440+0000 7f95295ee740 1 -- 172.21.15.83:0/3307191030 wait complete. 2024-03-11T19:38:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.441+0000 7f95295ee740 20 client.15030 trim_cache size 0 max 16384 2024-03-11T19:38:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f95295ee740 1 Processor -- start 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f95295ee740 1 -- start start 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f95295ee740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e58f80 0x5572a7e62f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f95295ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f95295ee740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e5eae0 0x5572a7e6a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f95295ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7d46490 con 0x5572a7e59970 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f95295ee740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7e5aa30 con 0x5572a7e58f80 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f95295ee740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7e54fe0 con 0x5572a7e5eae0 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f951f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f951f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60296/0 (socket says 172.21.15.83:60296) 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f951f7fe640 1 -- 172.21.15.83:0/3782260976 learned_addr learned my addr 172.21.15.83:0/3782260976 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f9524fb2640 1 --2- 172.21.15.83:0/3782260976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e5eae0 0x5572a7e6a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f951f7fe640 1 -- 172.21.15.83:0/3782260976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e5eae0 msgr2=0x5572a7e6a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f951ffff640 1 --2- 172.21.15.83:0/3782260976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e58f80 0x5572a7e62f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f951f7fe640 1 --2- 172.21.15.83:0/3782260976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e5eae0 0x5572a7e6a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.456+0000 7f951f7fe640 1 -- 172.21.15.83:0/3782260976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e58f80 msgr2=0x5572a7e62f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f951f7fe640 1 --2- 172.21.15.83:0/3782260976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e58f80 0x5572a7e62f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f951f7fe640 1 -- 172.21.15.83:0/3782260976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572a7d5ba20 con 0x5572a7e59970 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f951f7fe640 1 --2- 172.21.15.83:0/3782260976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9514003ae0 tx=0x7f9514005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2dec37a5285557 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f9505ffb640 1 -- 172.21.15.83:0/3782260976 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140037f0 con 0x5572a7e59970 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f9505ffb640 1 -- 172.21.15.83:0/3782260976 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95140096e0 con 0x5572a7e59970 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f9505ffb640 1 -- 172.21.15.83:0/3782260976 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514012b60 con 0x5572a7e59970 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f95295ee740 1 -- 172.21.15.83:0/3782260976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 msgr2=0x5572a7e634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f95295ee740 1 --2- 172.21.15.83:0/3782260976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9514003ae0 tx=0x7f9514005c10 comp rx=0 tx=0).stop 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f95295ee740 1 -- 172.21.15.83:0/3782260976 shutdown_connections 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f95295ee740 1 --2- 172.21.15.83:0/3782260976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e5eae0 0x5572a7e6a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f95295ee740 1 --2- 172.21.15.83:0/3782260976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e634c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.457+0000 7f95295ee740 1 --2- 172.21.15.83:0/3782260976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e58f80 0x5572a7e62f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 1 -- 172.21.15.83:0/3782260976 >> 172.21.15.83:0/3782260976 conn(0x5572a7e544f0 msgr2=0x5572a7e548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 1 -- 172.21.15.83:0/3782260976 shutdown_connections 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 1 -- 172.21.15.83:0/3782260976 wait complete. 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 1 -- start start 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 1 Processor -- start 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e5eae0 0x5572a7ec5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7e680e0 con 0x5572a7e59970 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7ec5ad0 con 0x5572a7e5eae0 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a7ec5c90 con 0x5572a7e58f80 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f95295ee740 10 client.0 _ll_register_callbacks cb 0x7f9518032bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f951f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f951f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e679e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60298/0 (socket says 172.21.15.83:60298) 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f951f7fe640 1 -- 172.21.15.83:0/2328993901 learned_addr learned my addr 172.21.15.83:0/2328993901 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f9524fb2640 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e5eae0 0x5572a7ec5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.458+0000 7f951ffff640 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f951f7fe640 1 -- 172.21.15.83:0/2328993901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 msgr2=0x5572a7e674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f951f7fe640 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f951f7fe640 1 -- 172.21.15.83:0/2328993901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e5eae0 msgr2=0x5572a7ec5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f951f7fe640 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e5eae0 0x5572a7ec5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f951f7fe640 1 -- 172.21.15.83:0/2328993901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572a7dbf2b0 con 0x5572a7e59970 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f951f7fe640 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e679e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f95140117f0 tx=0x7f9514011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d1c4ff48720550 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f94ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f94ff7fe640 1 -- 172.21.15.83:0/2328993901 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514035040 con 0x5572a7e59970 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f94ff7fe640 1 -- 172.21.15.83:0/2328993901 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9514000e40 con 0x5572a7e59970 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f95295ee740 1 -- 172.21.15.83:0/2328993901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572a7e6f6a0 con 0x5572a7e59970 2024-03-11T19:38:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f94ff7fe640 1 -- 172.21.15.83:0/2328993901 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9514012cf0 con 0x5572a7e59970 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f95295ee740 10 client.15036 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f95295ee740 1 -- 172.21.15.83:0/2328993901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5572a7d81780 con 0x5572a7e59970 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f94ff7fe640 1 -- 172.21.15.83:0/2328993901 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5572a7e6f6a0 con 0x5572a7e59970 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f94ff7fe640 1 -- 172.21.15.83:0/2328993901 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f951400c070 con 0x5572a7e59970 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f95295ee740 10 client.15036 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f95295ee740 10 client.15036 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f95295ee740 20 client.15036 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f95295ee740 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5572a7ecae30 0x5572a7ec72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f95295ee740 4 client.15036 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.459+0000 7f95295ee740 1 -- 172.21.15.83:0/2328993901 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5572a7e709c0 con 0x5572a7ecae30 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f951ffff640 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5572a7ecae30 0x5572a7ec72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f94ff7fe640 10 client.15036 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:38:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f951ffff640 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5572a7ecae30 0x5572a7ec72b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8699144d3dd174c server_cookie=4daf3c7020961ea8 in_seq=0 out_seq=0 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f94ff7fe640 1 -- 172.21.15.83:0/2328993901 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5572a7e709c0 con 0x5572a7ecae30 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f94ff7fe640 10 client.15036 handle_command_reply: tid=0 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f95295ee740 1 client.15036 shutdown 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f95295ee740 1 -- 172.21.15.83:0/2328993901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 msgr2=0x5572a7e679e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f95295ee740 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e679e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f95140117f0 tx=0x7f9514011fa0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f95295ee740 1 -- 172.21.15.83:0/2328993901 shutdown_connections 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f95295ee740 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5572a7ecae30 0x5572a7ec72b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f95295ee740 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572a7e5eae0 0x5572a7ec5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f95295ee740 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5572a7e59970 0x5572a7e679e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f94ff7fe640 0 client.15036 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.460+0000 7f95295ee740 1 --2- 172.21.15.83:0/2328993901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572a7e58f80 0x5572a7e674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f95295ee740 1 -- 172.21.15.83:0/2328993901 >> 172.21.15.83:0/2328993901 conn(0x5572a7e544f0 msgr2=0x5572a7e62f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f95295ee740 1 -- 172.21.15.83:0/2328993901 shutdown_connections 2024-03-11T19:38:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f95295ee740 1 -- 172.21.15.83:0/2328993901 wait complete. 2024-03-11T19:38:29.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f95295ee740 20 client.15036 trim_cache size 0 max 16384 2024-03-11T19:38:29.472 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:38:29.472 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "839f93c8-dfdd-11ee-95c8-87774f69a715", 2024-03-11T19:38:29.472 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-03-11T19:38:29.472 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24502, 2024-03-11T19:38:29.472 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-03-11T19:38:29.472 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-03-11T19:38:29.472 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-03-11T19:38:29.472 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 81.857552928000004, 2024-03-11T19:38:29.472 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 21, 2024-03-11T19:38:29.472 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 87, 2024-03-11T19:38:29.473 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 85, 2024-03-11T19:38:29.473 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 97.468800974999994 2024-03-11T19:38:29.473 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:38:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 -- 172.21.15.83:0/982930187 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f95080a1ac0 msgr2=0x7f95080a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f95080a1ac0 0x7f95080a3f80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9510005fd0 tx=0x7f9510009ec0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 -- 172.21.15.83:0/982930187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9520102d60 msgr2=0x7f952019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9520102d60 0x7f952019e7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9518037040 tx=0x7f9518034d40 comp rx=0 tx=0).stop 2024-03-11T19:38:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 -- 172.21.15.83:0/982930187 shutdown_connections 2024-03-11T19:38:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f95080a1ac0 0x7f95080a3f80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9520102d60 0x7f952019e7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95201023a0 0x7f9520197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 --2- 172.21.15.83:0/982930187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95201011b0 0x7f95201971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 -- 172.21.15.83:0/982930187 >> 172.21.15.83:0/982930187 conn(0x7f95200fc720 msgr2=0x7f95201056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 -- 172.21.15.83:0/982930187 shutdown_connections 2024-03-11T19:38:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.461+0000 7f9526a15640 1 -- 172.21.15.83:0/982930187 wait complete. 2024-03-11T19:38:29.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-11T19:38:29.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.574+0000 7f209c74b640 1 -- 172.21.15.83:0/961476379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2094106e70 msgr2=0x7f20941092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.574+0000 7f209c74b640 1 --2- 172.21.15.83:0/961476379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2094106e70 0x7f20941092c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f208c009ca0 tx=0x7f208c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.574+0000 7f209c74b640 1 -- 172.21.15.83:0/961476379 shutdown_connections 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.574+0000 7f209c74b640 1 --2- 172.21.15.83:0/961476379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2094106e70 0x7f20941092c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.574+0000 7f209c74b640 1 --2- 172.21.15.83:0/961476379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2094104540 0x7f2094106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.574+0000 7f209c74b640 1 --2- 172.21.15.83:0/961476379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2094101c10 0x7f2094104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.574+0000 7f209c74b640 1 -- 172.21.15.83:0/961476379 >> 172.21.15.83:0/961476379 conn(0x7f20940fb620 msgr2=0x7f20940fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.574+0000 7f209c74b640 1 -- 172.21.15.83:0/961476379 shutdown_connections 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.574+0000 7f209c74b640 1 -- 172.21.15.83:0/961476379 wait complete. 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f209c74b640 1 Processor -- start 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f209c74b640 1 -- start start 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f209c74b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2094101c10 0x7f2094197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f209c74b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2094104540 0x7f2094197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f209c74b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2094106e70 0x7f209419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f209c74b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209419edf0 con 0x7f2094104540 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f209c74b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209419ef60 con 0x7f2094101c10 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f209c74b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209419f120 con 0x7f2094106e70 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f2099ce6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2094104540 0x7f2094197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f2099ce6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2094104540 0x7f2094197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50642/0 (socket says 172.21.15.83:50642) 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f2099ce6640 1 -- 172.21.15.83:0/3517081720 learned_addr learned my addr 172.21.15.83:0/3517081720 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f209a4e7640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2094101c10 0x7f2094197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f2099ce6640 1 -- 172.21.15.83:0/3517081720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2094106e70 msgr2=0x7f209419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f2099ce6640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2094106e70 0x7f209419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f2099ce6640 1 -- 172.21.15.83:0/3517081720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2094101c10 msgr2=0x7f2094197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.575+0000 7f2099ce6640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2094101c10 0x7f2094197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.576+0000 7f2099ce6640 1 -- 172.21.15.83:0/3517081720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c009900 con 0x7f2094104540 2024-03-11T19:38:29.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.576+0000 7f209a4e7640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2094101c10 0x7f2094197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:29.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.576+0000 7f2099ce6640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2094104540 0x7f2094197840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f208400b450 tx=0x7f208400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.576+0000 7f208b7fe640 1 -- 172.21.15.83:0/3517081720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084011890 con 0x7f2094104540 2024-03-11T19:38:29.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.576+0000 7f208b7fe640 1 -- 172.21.15.83:0/3517081720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20840103e0 con 0x7f2094104540 2024-03-11T19:38:29.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.576+0000 7f208b7fe640 1 -- 172.21.15.83:0/3517081720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208400f580 con 0x7f2094104540 2024-03-11T19:38:29.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.576+0000 7f209c74b640 1 -- 172.21.15.83:0/3517081720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209419f400 con 0x7f2094104540 2024-03-11T19:38:29.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.576+0000 7f209c74b640 1 -- 172.21.15.83:0/3517081720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209419f990 con 0x7f2094104540 2024-03-11T19:38:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.578+0000 7f208b7fe640 1 -- 172.21.15.83:0/3517081720 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f20840108e0 con 0x7f2094104540 2024-03-11T19:38:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.578+0000 7f209f3f0740 1 Processor -- start 2024-03-11T19:38:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.578+0000 7f209f3f0740 1 -- start start 2024-03-11T19:38:29.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.578+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e11390 0x560348e117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.578+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e16880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.578+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e19380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.578+0000 7f209f3f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348dc8cd0 con 0x560348e16ef0 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.578+0000 7f209f3f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348d7bf20 con 0x560348e11390 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.578+0000 7f209f3f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348d79450 con 0x560348e11d80 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f208b7fe640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20800a1ac0 0x7f20800a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2099ce6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e19380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2099ce6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e19380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50650/0 (socket says 172.21.15.83:50650) 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2099ce6640 1 -- 172.21.15.83:0/3114203327 learned_addr learned my addr 172.21.15.83:0/3114203327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f209ace8640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20800a1ac0 0x7f20800a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f209a4e7640 1 --2- 172.21.15.83:0/3114203327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e16880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f209ace8640 1 --2- 172.21.15.83:0/3114203327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e11390 0x560348e117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f208b7fe640 1 -- 172.21.15.83:0/3517081720 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f20840cd1b0 con 0x7f2094104540 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2099ce6640 1 -- 172.21.15.83:0/3114203327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 msgr2=0x560348e16880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2099ce6640 1 --2- 172.21.15.83:0/3114203327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e16880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2099ce6640 1 -- 172.21.15.83:0/3114203327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e11390 msgr2=0x560348e117b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2099ce6640 1 --2- 172.21.15.83:0/3114203327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e11390 0x560348e117b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2099ce6640 1 -- 172.21.15.83:0/3114203327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560348d29f20 con 0x560348e16ef0 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f209a4e7640 1 --2- 172.21.15.83:0/3114203327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e16880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f209ace8640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20800a1ac0 0x7f20800a3f80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f208c006010 tx=0x7f208c032f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2099ce6640 1 --2- 172.21.15.83:0/3114203327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e19380 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f20840cacf0 tx=0x7f2084010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe08c6f5ec878bd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2088ff9640 1 -- 172.21.15.83:0/3114203327 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208409e330 con 0x560348e16ef0 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2088ff9640 1 -- 172.21.15.83:0/3114203327 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20840a2980 con 0x560348e16ef0 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.579+0000 7f2088ff9640 1 -- 172.21.15.83:0/3114203327 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084099940 con 0x560348e16ef0 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.580+0000 7f209f3f0740 1 -- 172.21.15.83:0/3114203327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 msgr2=0x560348e19380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.580+0000 7f209f3f0740 1 --2- 172.21.15.83:0/3114203327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e19380 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f20840cacf0 tx=0x7f2084010ce0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.580+0000 7f209f3f0740 1 -- 172.21.15.83:0/3114203327 shutdown_connections 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.580+0000 7f209f3f0740 1 --2- 172.21.15.83:0/3114203327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e19380 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.580+0000 7f209f3f0740 1 --2- 172.21.15.83:0/3114203327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e16880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.580+0000 7f209f3f0740 1 --2- 172.21.15.83:0/3114203327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e11390 0x560348e117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.580+0000 7f209f3f0740 1 -- 172.21.15.83:0/3114203327 >> 172.21.15.83:0/3114203327 conn(0x560348e084f0 msgr2=0x560348e088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.580+0000 7f209f3f0740 1 -- 172.21.15.83:0/3114203327 shutdown_connections 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.580+0000 7f209f3f0740 1 -- 172.21.15.83:0/3114203327 wait complete. 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.580+0000 7f209f3f0740 1 -- start start 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.580+0000 7f209f3f0740 1 Processor -- start 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11390 0x560348e1b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e11d80 0x560348e1ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e22a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209f3f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348d74c30 con 0x560348e16ef0 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209f3f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348d19d30 con 0x560348e11d80 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209f3f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348a48310 con 0x560348e11390 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209f3f0740 10 client.0 _ll_register_callbacks cb 0x560348d21d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209ace8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e22a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209ace8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e22a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50662/0 (socket says 172.21.15.83:50662) 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209ace8640 1 -- 172.21.15.83:0/502160803 learned_addr learned my addr 172.21.15.83:0/502160803 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209a4e7640 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11390 0x560348e1b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f2099ce6640 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e11d80 0x560348e1ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209ace8640 1 -- 172.21.15.83:0/502160803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11390 msgr2=0x560348e1b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209ace8640 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11390 0x560348e1b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209ace8640 1 -- 172.21.15.83:0/502160803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e11d80 msgr2=0x560348e1ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209ace8640 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e11d80 0x560348e1ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209ace8640 1 -- 172.21.15.83:0/502160803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209419f990 con 0x560348e16ef0 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209a4e7640 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11390 0x560348e1b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:29.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f2099ce6640 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e11d80 0x560348e1ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209ace8640 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e22a70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f208c036000 tx=0x7f208c0326b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29999ec622a7e534 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f207dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f207dffb640 1 -- 172.21.15.83:0/502160803 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c040900 con 0x560348e16ef0 2024-03-11T19:38:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f207dffb640 1 -- 172.21.15.83:0/502160803 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f208c048040 con 0x560348e16ef0 2024-03-11T19:38:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209f3f0740 1 -- 172.21.15.83:0/502160803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560348a746c0 con 0x560348e16ef0 2024-03-11T19:38:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f207dffb640 1 -- 172.21.15.83:0/502160803 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f208c040a60 con 0x560348e16ef0 2024-03-11T19:38:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209f3f0740 10 client.15048 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:29.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.581+0000 7f209f3f0740 1 -- 172.21.15.83:0/502160803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x560348d732b0 con 0x560348e16ef0 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.582+0000 7f207dffb640 1 -- 172.21.15.83:0/502160803 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560348a746c0 con 0x560348e16ef0 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.582+0000 7f207dffb640 1 -- 172.21.15.83:0/502160803 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f208c042420 con 0x560348e16ef0 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.582+0000 7f209f3f0740 10 client.15048 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.582+0000 7f209f3f0740 10 client.15048 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.582+0000 7f209f3f0740 20 client.15048 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.582+0000 7f209f3f0740 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560348e27f50 0x560348e27190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.582+0000 7f209f3f0740 4 client.15048 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.582+0000 7f209f3f0740 1 -- 172.21.15.83:0/502160803 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560348ac5270 con 0x560348e27f50 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.582+0000 7f209a4e7640 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560348e27f50 0x560348e27190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.582+0000 7f207dffb640 10 client.15048 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.582+0000 7f209a4e7640 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560348e27f50 0x560348e27190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f40c450952e39db7 server_cookie=3d51ec64a5a6c665 in_seq=0 out_seq=0 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f207dffb640 1 -- 172.21.15.83:0/502160803 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560348ac5270 con 0x560348e27f50 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f207dffb640 10 client.15048 handle_command_reply: tid=0 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f209f3f0740 1 client.15048 shutdown 2024-03-11T19:38:29.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f209f3f0740 1 -- 172.21.15.83:0/502160803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 msgr2=0x560348e22a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f209f3f0740 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e22a70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f208c036000 tx=0x7f208c0326b0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f209f3f0740 1 -- 172.21.15.83:0/502160803 shutdown_connections 2024-03-11T19:38:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f209f3f0740 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560348e27f50 0x560348e27190 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f209f3f0740 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e16ef0 0x560348e22a70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f209f3f0740 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e11d80 0x560348e1ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f207dffb640 0 client.15048 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:38:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f209f3f0740 1 --2- 172.21.15.83:0/502160803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11390 0x560348e1b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f209f3f0740 1 -- 172.21.15.83:0/502160803 >> 172.21.15.83:0/502160803 conn(0x560348e084f0 msgr2=0x560348e077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f209f3f0740 1 -- 172.21.15.83:0/502160803 shutdown_connections 2024-03-11T19:38:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.583+0000 7f209f3f0740 1 -- 172.21.15.83:0/502160803 wait complete. 2024-03-11T19:38:29.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.584+0000 7f209f3f0740 20 client.15048 trim_cache size 0 max 16384 2024-03-11T19:38:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209f3f0740 1 Processor -- start 2024-03-11T19:38:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209f3f0740 1 -- start start 2024-03-11T19:38:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e22990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e16ef0 0x560348e088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209f3f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348cfa490 con 0x560348e11390 2024-03-11T19:38:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209f3f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348e236f0 con 0x560348e16ef0 2024-03-11T19:38:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209f3f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348a746c0 con 0x560348e11d80 2024-03-11T19:38:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209a4e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209a4e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50664/0 (socket says 172.21.15.83:50664) 2024-03-11T19:38:29.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209a4e7640 1 -- 172.21.15.83:0/3840728097 learned_addr learned my addr 172.21.15.83:0/3840728097 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f2099ce6640 1 --2- 172.21.15.83:0/3840728097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e22990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.599+0000 7f209ace8640 1 --2- 172.21.15.83:0/3840728097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e16ef0 0x560348e088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.600+0000 7f209a4e7640 1 -- 172.21.15.83:0/3840728097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 msgr2=0x560348e22990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.600+0000 7f209a4e7640 1 --2- 172.21.15.83:0/3840728097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e22990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.600+0000 7f209a4e7640 1 -- 172.21.15.83:0/3840728097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e16ef0 msgr2=0x560348e088d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.600+0000 7f209a4e7640 1 --2- 172.21.15.83:0/3840728097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e16ef0 0x560348e088d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.600+0000 7f209a4e7640 1 -- 172.21.15.83:0/3840728097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560348d0fa20 con 0x560348e11390 2024-03-11T19:38:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.600+0000 7f2099ce6640 1 --2- 172.21.15.83:0/3840728097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e22990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:29.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.600+0000 7f209a4e7640 1 --2- 172.21.15.83:0/3840728097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1a030 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2090006fd0 tx=0x7f209000cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42e7a829e239b5b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.603+0000 7f207ffff640 1 -- 172.21.15.83:0/3840728097 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900159f0 con 0x560348e11390 2024-03-11T19:38:29.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.603+0000 7f207ffff640 1 -- 172.21.15.83:0/3840728097 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f209001d070 con 0x560348e11390 2024-03-11T19:38:29.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.603+0000 7f207ffff640 1 -- 172.21.15.83:0/3840728097 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090020380 con 0x560348e11390 2024-03-11T19:38:29.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 -- 172.21.15.83:0/3840728097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 msgr2=0x560348e1a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 --2- 172.21.15.83:0/3840728097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1a030 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2090006fd0 tx=0x7f209000cfb0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 -- 172.21.15.83:0/3840728097 shutdown_connections 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 --2- 172.21.15.83:0/3840728097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e16ef0 0x560348e088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 --2- 172.21.15.83:0/3840728097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e22990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 --2- 172.21.15.83:0/3840728097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1a030 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 -- 172.21.15.83:0/3840728097 >> 172.21.15.83:0/3840728097 conn(0x560348e084f0 msgr2=0x560348e12820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 -- 172.21.15.83:0/3840728097 shutdown_connections 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 -- 172.21.15.83:0/3840728097 wait complete. 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 -- start start 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 Processor -- start 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e1fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e16ef0 0x560348e7db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348e204b0 con 0x560348e11390 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348e7e090 con 0x560348e16ef0 2024-03-11T19:38:29.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560348e7e1d0 con 0x560348e11d80 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209f3f0740 10 client.0 _ll_register_callbacks cb 0x7f208409a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209a4e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209a4e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50666/0 (socket says 172.21.15.83:50666) 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209a4e7640 1 -- 172.21.15.83:0/1376856165 learned_addr learned my addr 172.21.15.83:0/1376856165 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.604+0000 7f209ace8640 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e16ef0 0x560348e7db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f2099ce6640 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e1fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209a4e7640 1 -- 172.21.15.83:0/1376856165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 msgr2=0x560348e1fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209a4e7640 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e1fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209a4e7640 1 -- 172.21.15.83:0/1376856165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e16ef0 msgr2=0x560348e7db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209a4e7640 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e16ef0 0x560348e7db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209a4e7640 1 -- 172.21.15.83:0/1376856165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209000cc20 con 0x560348e11390 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209a4e7640 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1f930 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2090021dc0 tx=0x7f2090021df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b849a47cf4784d5b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f207dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f207dffb640 1 -- 172.21.15.83:0/1376856165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209001a040 con 0x560348e11390 2024-03-11T19:38:29.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f207dffb640 1 -- 172.21.15.83:0/1376856165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f20900213d0 con 0x560348e11390 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209f3f0740 1 -- 172.21.15.83:0/1376856165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560348e28f80 con 0x560348e11390 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f207dffb640 1 -- 172.21.15.83:0/1376856165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2090015e80 con 0x560348e11390 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209f3f0740 10 client.15054 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209f3f0740 1 -- 172.21.15.83:0/1376856165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x560348d35780 con 0x560348e11390 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f207dffb640 1 -- 172.21.15.83:0/1376856165 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560348e28f80 con 0x560348e11390 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f207dffb640 1 -- 172.21.15.83:0/1376856165 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f20900353d0 con 0x560348e11390 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209f3f0740 10 client.15054 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209f3f0740 10 client.15054 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209f3f0740 20 client.15054 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209f3f0740 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560348e833f0 0x560348e7f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209f3f0740 4 client.15054 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:38:29.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.605+0000 7f209f3f0740 1 -- 172.21.15.83:0/1376856165 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f20840a2980 con 0x560348e833f0 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.606+0000 7f2099ce6640 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560348e833f0 0x560348e7f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.606+0000 7f207dffb640 10 client.15054 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.606+0000 7f2099ce6640 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560348e833f0 0x560348e7f870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac334cdc07421a8 server_cookie=708559ad43717051 in_seq=0 out_seq=0 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f207dffb640 1 -- 172.21.15.83:0/1376856165 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x7f20840a2980 con 0x560348e833f0 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f207dffb640 10 client.15054 handle_command_reply: tid=0 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 1 client.15054 shutdown 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 1 -- 172.21.15.83:0/1376856165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 msgr2=0x560348e1f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1f930 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2090021dc0 tx=0x7f2090021df0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 1 -- 172.21.15.83:0/1376856165 shutdown_connections 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560348e833f0 0x560348e7f870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560348e16ef0 0x560348e7db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f207dffb640 0 client.15054 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560348e11d80 0x560348e1fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 1 --2- 172.21.15.83:0/1376856165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560348e11390 0x560348e1f930 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 1 -- 172.21.15.83:0/1376856165 >> 172.21.15.83:0/1376856165 conn(0x560348e084f0 msgr2=0x560348e1a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 1 -- 172.21.15.83:0/1376856165 shutdown_connections 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 1 -- 172.21.15.83:0/1376856165 wait complete. 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209f3f0740 20 client.15054 trim_cache size 0 max 16384 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "839f93c8-dfdd-11ee-95c8-87774f69a715", 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24527, 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 84.035038556000003, 2024-03-11T19:38:29.616 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 21, 2024-03-11T19:38:29.617 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 87, 2024-03-11T19:38:29.617 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 85, 2024-03-11T19:38:29.617 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 89.166864808 2024-03-11T19:38:29.617 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:38:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 -- 172.21.15.83:0/3517081720 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20800a1ac0 msgr2=0x7f20800a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20800a1ac0 0x7f20800a3f80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f208c006010 tx=0x7f208c032f90 comp rx=0 tx=0).stop 2024-03-11T19:38:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 -- 172.21.15.83:0/3517081720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2094104540 msgr2=0x7f2094197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2094104540 0x7f2094197840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f208400b450 tx=0x7f208400b910 comp rx=0 tx=0).stop 2024-03-11T19:38:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 -- 172.21.15.83:0/3517081720 shutdown_connections 2024-03-11T19:38:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20800a1ac0 0x7f20800a3f80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2094106e70 0x7f209419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2094104540 0x7f2094197840 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 --2- 172.21.15.83:0/3517081720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2094101c10 0x7f2094197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 -- 172.21.15.83:0/3517081720 >> 172.21.15.83:0/3517081720 conn(0x7f20940fb620 msgr2=0x7f20941024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:29.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 -- 172.21.15.83:0/3517081720 shutdown_connections 2024-03-11T19:38:29.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:29.607+0000 7f209c74b640 1 -- 172.21.15.83:0/3517081720 wait complete. 2024-03-11T19:38:29.622 INFO:tasks.cephadm:Setup complete, yielding 2024-03-11T19:38:29.622 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-11T19:38:29.632 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-03-11T19:38:29.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- bash -c 'ceph orch status' 2024-03-11T19:38:29.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:29 smithi083 ceph-mon[31864]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:38:29.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:29 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2205606375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:29.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:29 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/844207491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:29.870 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:29 smithi137 ceph-mon[36220]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:38:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:29 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2205606375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:29 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/844207491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:29 smithi204 ceph-mon[36739]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:38:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:29 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2205606375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:29 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/844207491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:31.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.500+0000 7ff13fbf6700 1 -- 172.21.15.83:0/1122698556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff138100bd0 msgr2=0x7ff138100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.500+0000 7ff13fbf6700 1 --2- 172.21.15.83:0/1122698556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff138100bd0 0x7ff138100ff0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff1280099c0 tx=0x7ff128009cd0 comp rx=0 tx=0).stop 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.500+0000 7ff13fbf6700 1 -- 172.21.15.83:0/1122698556 shutdown_connections 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.500+0000 7ff13fbf6700 1 --2- 172.21.15.83:0/1122698556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff138102770 0x7ff138109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.500+0000 7ff13fbf6700 1 --2- 172.21.15.83:0/1122698556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff138100bd0 0x7ff138100ff0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.500+0000 7ff13fbf6700 1 --2- 172.21.15.83:0/1122698556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff138101dd0 0x7ff138102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.500+0000 7ff13fbf6700 1 -- 172.21.15.83:0/1122698556 >> 172.21.15.83:0/1122698556 conn(0x7ff1380fc170 msgr2=0x7ff1380fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.500+0000 7ff13fbf6700 1 -- 172.21.15.83:0/1122698556 shutdown_connections 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.500+0000 7ff13fbf6700 1 -- 172.21.15.83:0/1122698556 wait complete. 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.501+0000 7ff13fbf6700 1 Processor -- start 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.501+0000 7ff13fbf6700 1 -- start start 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.501+0000 7ff13fbf6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff138100bd0 0x7ff138104c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.501+0000 7ff13fbf6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff138101dd0 0x7ff138103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.501+0000 7ff13fbf6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff138102770 0x7ff1381037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.501+0000 7ff13fbf6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff138103f40 con 0x7ff138100bd0 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.501+0000 7ff13fbf6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1381040b0 con 0x7ff138101dd0 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.501+0000 7ff13fbf6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1381a1d10 con 0x7ff138102770 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.501+0000 7ff13d992700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff138100bd0 0x7ff138104c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.501+0000 7ff13d992700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff138100bd0 0x7ff138104c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50698/0 (socket says 172.21.15.83:50698) 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.501+0000 7ff13d992700 1 -- 172.21.15.83:0/4010345385 learned_addr learned my addr 172.21.15.83:0/4010345385 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff13e193700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff138102770 0x7ff1381037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff13d992700 1 -- 172.21.15.83:0/4010345385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff138102770 msgr2=0x7ff1381037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff13d992700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff138102770 0x7ff1381037c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff13d191700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff138101dd0 0x7ff138103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff13d992700 1 -- 172.21.15.83:0/4010345385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff138101dd0 msgr2=0x7ff138103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff13d992700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff138101dd0 0x7ff138103280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff13d992700 1 -- 172.21.15.83:0/4010345385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1280095f0 con 0x7ff138100bd0 2024-03-11T19:38:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff13d992700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff138100bd0 0x7ff138104c30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff1280038c0 tx=0x7ff128004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff12effd700 1 -- 172.21.15.83:0/4010345385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12801d070 con 0x7ff138100bd0 2024-03-11T19:38:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff12effd700 1 -- 172.21.15.83:0/4010345385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff128004110 con 0x7ff138100bd0 2024-03-11T19:38:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff13fbf6700 1 -- 172.21.15.83:0/4010345385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1381a1f90 con 0x7ff138100bd0 2024-03-11T19:38:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff12effd700 1 -- 172.21.15.83:0/4010345385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128004a20 con 0x7ff138100bd0 2024-03-11T19:38:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.502+0000 7ff13fbf6700 1 -- 172.21.15.83:0/4010345385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1381a2340 con 0x7ff138100bd0 2024-03-11T19:38:31.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.503+0000 7ff13fbf6700 1 -- 172.21.15.83:0/4010345385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13810c590 con 0x7ff138100bd0 2024-03-11T19:38:31.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.504+0000 7ff12effd700 1 -- 172.21.15.83:0/4010345385 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7ff128005150 con 0x7ff138100bd0 2024-03-11T19:38:31.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.505+0000 7ff12effd700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1240a1b50 0x7ff1240a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:31.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.505+0000 7ff13d191700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1240a1b50 0x7ff1240a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:31.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.505+0000 7ff12effd700 1 -- 172.21.15.83:0/4010345385 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff1280ce510 con 0x7ff138100bd0 2024-03-11T19:38:31.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.506+0000 7ff13d191700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1240a1b50 0x7ff1240a4010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff134005950 tx=0x7ff1340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:31.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.509+0000 7ff12effd700 1 -- 172.21.15.83:0/4010345385 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff128097660 con 0x7ff138100bd0 2024-03-11T19:38:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.724+0000 7ff13fbf6700 1 -- 172.21.15.83:0/4010345385 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff13806aa20 con 0x7ff1240a1b50 2024-03-11T19:38:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.725+0000 7ff12effd700 1 -- 172.21.15.83:0/4010345385 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff13806aa20 con 0x7ff1240a1b50 2024-03-11T19:38:31.731 INFO:teuthology.orchestra.run.smithi083.stdout:Backend: cephadm 2024-03-11T19:38:31.731 INFO:teuthology.orchestra.run.smithi083.stdout:Available: Yes 2024-03-11T19:38:31.731 INFO:teuthology.orchestra.run.smithi083.stdout:Paused: No 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 -- 172.21.15.83:0/4010345385 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1240a1b50 msgr2=0x7ff1240a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1240a1b50 0x7ff1240a4010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff134005950 tx=0x7ff1340058e0 comp rx=0 tx=0).stop 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 -- 172.21.15.83:0/4010345385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff138100bd0 msgr2=0x7ff138104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff138100bd0 0x7ff138104c30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff1280038c0 tx=0x7ff128004730 comp rx=0 tx=0).stop 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 -- 172.21.15.83:0/4010345385 shutdown_connections 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1240a1b50 0x7ff1240a4010 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff138102770 0x7ff1381037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff138100bd0 0x7ff138104c30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 --2- 172.21.15.83:0/4010345385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff138101dd0 0x7ff138103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 -- 172.21.15.83:0/4010345385 >> 172.21.15.83:0/4010345385 conn(0x7ff1380fc170 msgr2=0x7ff13806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 -- 172.21.15.83:0/4010345385 shutdown_connections 2024-03-11T19:38:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:31.727+0000 7ff13fbf6700 1 -- 172.21.15.83:0/4010345385 wait complete. 2024-03-11T19:38:31.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:31 smithi137 ceph-mon[36220]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:38:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:31 smithi204 ceph-mon[36739]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:38:32.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:31 smithi083 ceph-mon[31864]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:38:32.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-11T19:38:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:33.649 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:33 smithi083 ceph-mon[31864]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:33.649 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:33 smithi083 ceph-mon[31864]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:38:33.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:33 smithi137 ceph-mon[36220]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:33.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:33 smithi137 ceph-mon[36220]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:38:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:33 smithi204 ceph-mon[36739]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:33 smithi204 ceph-mon[36739]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-11T19:38:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.146+0000 7f1bd8414700 1 -- 172.21.15.83:0/3845551940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd0102ea0 msgr2=0x7f1bd0103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.146+0000 7f1bd8414700 1 --2- 172.21.15.83:0/3845551940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd0102ea0 0x7f1bd0103300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00099c0 tx=0x7f1bc0009cd0 comp rx=0 tx=0).stop 2024-03-11T19:38:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.146+0000 7f1bd8414700 1 -- 172.21.15.83:0/3845551940 shutdown_connections 2024-03-11T19:38:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.146+0000 7f1bd8414700 1 --2- 172.21.15.83:0/3845551940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bd0103840 0x7f1bd010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.146+0000 7f1bd8414700 1 --2- 172.21.15.83:0/3845551940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bd0101ca0 0x7f1bd01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.146+0000 7f1bd8414700 1 --2- 172.21.15.83:0/3845551940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd0102ea0 0x7f1bd0103300 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.146+0000 7f1bd8414700 1 -- 172.21.15.83:0/3845551940 >> 172.21.15.83:0/3845551940 conn(0x7f1bd00fd280 msgr2=0x7f1bd00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.146+0000 7f1bd8414700 1 -- 172.21.15.83:0/3845551940 shutdown_connections 2024-03-11T19:38:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.146+0000 7f1bd8414700 1 -- 172.21.15.83:0/3845551940 wait complete. 2024-03-11T19:38:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.147+0000 7f1bd8414700 1 Processor -- start 2024-03-11T19:38:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.147+0000 7f1bd8414700 1 -- start start 2024-03-11T19:38:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.147+0000 7f1bd8414700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bd0101ca0 0x7f1bd0199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.147+0000 7f1bd8414700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bd0102ea0 0x7f1bd019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.147+0000 7f1bd8414700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd0103840 0x7f1bd01a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.147+0000 7f1bd8414700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd01a1990 con 0x7f1bd0103840 2024-03-11T19:38:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.147+0000 7f1bd8414700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd01a1ad0 con 0x7f1bd0102ea0 2024-03-11T19:38:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.147+0000 7f1bd8414700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd01a1c60 con 0x7f1bd0101ca0 2024-03-11T19:38:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.147+0000 7f1bd69b1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd0103840 0x7f1bd01a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.147+0000 7f1bd69b1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd0103840 0x7f1bd01a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50720/0 (socket says 172.21.15.83:50720) 2024-03-11T19:38:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.147+0000 7f1bd69b1700 1 -- 172.21.15.83:0/2807524032 learned_addr learned my addr 172.21.15.83:0/2807524032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd59af700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bd0102ea0 0x7f1bd019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd61b0700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bd0101ca0 0x7f1bd0199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd69b1700 1 -- 172.21.15.83:0/2807524032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bd0101ca0 msgr2=0x7f1bd0199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd69b1700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bd0101ca0 0x7f1bd0199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd69b1700 1 -- 172.21.15.83:0/2807524032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bd0102ea0 msgr2=0x7f1bd019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd69b1700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bd0102ea0 0x7f1bd019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd69b1700 1 -- 172.21.15.83:0/2807524032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc00095f0 con 0x7f1bd0103840 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd61b0700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bd0101ca0 0x7f1bd0199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd59af700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bd0102ea0 0x7f1bd019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd69b1700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd0103840 0x7f1bd01a13c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8007f10 tx=0x7f1bc800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bc77fe700 1 -- 172.21.15.83:0/2807524032 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8017070 con 0x7f1bd0103840 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bc77fe700 1 -- 172.21.15.83:0/2807524032 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bc800f040 con 0x7f1bd0103840 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd8414700 1 -- 172.21.15.83:0/2807524032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd01a1f40 con 0x7f1bd0103840 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bc77fe700 1 -- 172.21.15.83:0/2807524032 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8005760 con 0x7f1bd0103840 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.148+0000 7f1bd8414700 1 -- 172.21.15.83:0/2807524032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd01a2360 con 0x7f1bd0103840 2024-03-11T19:38:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.149+0000 7f1bd8414700 1 -- 172.21.15.83:0/2807524032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd0068930 con 0x7f1bd0103840 2024-03-11T19:38:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.150+0000 7f1bc77fe700 1 -- 172.21.15.83:0/2807524032 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f1bc8005900 con 0x7f1bd0103840 2024-03-11T19:38:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.150+0000 7f1bc77fe700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bbc0a1b50 0x7f1bbc0a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.151+0000 7f1bd61b0700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bbc0a1b50 0x7f1bbc0a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.151+0000 7f1bc77fe700 1 -- 172.21.15.83:0/2807524032 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1bc80ccd90 con 0x7f1bd0103840 2024-03-11T19:38:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.151+0000 7f1bd61b0700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bbc0a1b50 0x7f1bbc0a4010 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc005fd0 tx=0x7f1bcc005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:34.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.155+0000 7f1bc77fe700 1 -- 172.21.15.83:0/2807524032 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1bc8095f50 con 0x7f1bd0103840 2024-03-11T19:38:34.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.359+0000 7f1bd8414700 1 -- 172.21.15.83:0/2807524032 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bd00755c0 con 0x7f1bbc0a1b50 2024-03-11T19:38:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.373+0000 7f1bc77fe700 1 -- 172.21.15.83:0/2807524032 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f1bd00755c0 con 0x7f1bbc0a1b50 2024-03-11T19:38:34.378 INFO:teuthology.orchestra.run.smithi083.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-11T19:38:34.378 INFO:teuthology.orchestra.run.smithi083.stdout:mds.a smithi083 running (97s) 16s ago 96s 13.8M - 19.0.0-1607-g94a53970 a6477a474d71 5d7b2f08a83b 2024-03-11T19:38:34.378 INFO:teuthology.orchestra.run.smithi083.stdout:mds.b smithi137 running (94s) 28s ago 93s 25.8M - 19.0.0-1607-g94a53970 a6477a474d71 56dd86f430d7 2024-03-11T19:38:34.378 INFO:teuthology.orchestra.run.smithi083.stdout:mds.c smithi204 running (108s) 28s ago 108s 22.3M - 19.0.0-1607-g94a53970 a6477a474d71 2ad65201b1a8 2024-03-11T19:38:34.378 INFO:teuthology.orchestra.run.smithi083.stdout:mds.d smithi083 running (116s) 16s ago 116s 13.9M - 19.0.0-1607-g94a53970 a6477a474d71 d5c2d73f3e13 2024-03-11T19:38:34.378 INFO:teuthology.orchestra.run.smithi083.stdout:mds.e smithi137 running (113s) 28s ago 113s 19.1M - 19.0.0-1607-g94a53970 a6477a474d71 3c3a611e31be 2024-03-11T19:38:34.378 INFO:teuthology.orchestra.run.smithi083.stdout:mds.f smithi204 running (111s) 28s ago 110s 16.8M - 19.0.0-1607-g94a53970 a6477a474d71 055abf3673c8 2024-03-11T19:38:34.378 INFO:teuthology.orchestra.run.smithi083.stdout:mds.g smithi083 running (99s) 16s ago 99s 12.7M - 19.0.0-1607-g94a53970 a6477a474d71 672aa82f8972 2024-03-11T19:38:34.378 INFO:teuthology.orchestra.run.smithi083.stdout:mds.h smithi137 running (105s) 28s ago 105s 17.3M - 19.0.0-1607-g94a53970 a6477a474d71 047e588e571b 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:mds.i smithi204 running (102s) 28s ago 102s 17.7M - 19.0.0-1607-g94a53970 a6477a474d71 9db8ac59f980 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:mds.j smithi083 running (119s) 16s ago 118s 16.2M - 19.0.0-1607-g94a53970 a6477a474d71 f6dc6086f4c5 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:mds.k smithi137 running (2m) 28s ago 2m 16.2M - 19.0.0-1607-g94a53970 a6477a474d71 c2a41a73f12a 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:mds.l smithi204 running (2m) 28s ago 2m 16.1M - 19.0.0-1607-g94a53970 a6477a474d71 582f1dd65852 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.x smithi083 *:9283,8765 running (9m) 16s ago 9m 512M - 19.0.0-1607-g94a53970 a6477a474d71 23a489d69253 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.y smithi137 *:8443,8765 running (7m) 28s ago 7m 451M - 19.0.0-1607-g94a53970 a6477a474d71 24e5dc9168b0 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.z smithi204 *:8443,8765 running (7m) 28s ago 7m 453M - 19.0.0-1607-g94a53970 a6477a474d71 154856ab6423 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:mon.a smithi083 running (9m) 16s ago 9m 50.6M 2048M 19.0.0-1607-g94a53970 a6477a474d71 5c07b05df3a9 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:mon.b smithi137 running (7m) 28s ago 7m 40.5M 2048M 19.0.0-1607-g94a53970 a6477a474d71 055cc9aa4fb3 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:mon.c smithi204 running (7m) 28s ago 7m 37.7M 2048M 19.0.0-1607-g94a53970 a6477a474d71 f7bf0a33d863 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 smithi083 running (6m) 16s ago 6m 57.5M 4272M 19.0.0-1607-g94a53970 a6477a474d71 eb967f3ade90 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 smithi137 running (6m) 28s ago 6m 58.4M 4270M 19.0.0-1607-g94a53970 a6477a474d71 828157dfc404 2024-03-11T19:38:34.379 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 smithi204 running (6m) 28s ago 6m 59.9M 4270M 19.0.0-1607-g94a53970 a6477a474d71 924d8105635e 2024-03-11T19:38:34.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 smithi083 running (5m) 16s ago 5m 46.4M 4272M 19.0.0-1607-g94a53970 a6477a474d71 42abf92700cb 2024-03-11T19:38:34.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 smithi137 running (5m) 28s ago 5m 52.7M 4270M 19.0.0-1607-g94a53970 a6477a474d71 5aa5906080c1 2024-03-11T19:38:34.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 smithi204 running (4m) 28s ago 4m 46.8M 4270M 19.0.0-1607-g94a53970 a6477a474d71 1904bc3f6510 2024-03-11T19:38:34.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 smithi083 running (4m) 16s ago 4m 50.7M 4272M 19.0.0-1607-g94a53970 a6477a474d71 226ebe4cbefe 2024-03-11T19:38:34.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 smithi137 running (4m) 28s ago 4m 45.3M 4270M 19.0.0-1607-g94a53970 a6477a474d71 89d7bdd7833f 2024-03-11T19:38:34.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 smithi204 running (3m) 28s ago 3m 45.6M 4270M 19.0.0-1607-g94a53970 a6477a474d71 340fa14656f5 2024-03-11T19:38:34.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 smithi083 running (3m) 16s ago 3m 43.5M 4272M 19.0.0-1607-g94a53970 a6477a474d71 d67ec13d4b0e 2024-03-11T19:38:34.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 smithi137 running (2m) 28s ago 2m 54.5M 4270M 19.0.0-1607-g94a53970 a6477a474d71 04751bb7c8e4 2024-03-11T19:38:34.380 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 smithi204 running (2m) 28s ago 2m 52.8M 4270M 19.0.0-1607-g94a53970 a6477a474d71 18cc330a9da2 2024-03-11T19:38:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 -- 172.21.15.83:0/2807524032 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bbc0a1b50 msgr2=0x7f1bbc0a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bbc0a1b50 0x7f1bbc0a4010 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc005fd0 tx=0x7f1bcc005e40 comp rx=0 tx=0).stop 2024-03-11T19:38:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 -- 172.21.15.83:0/2807524032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd0103840 msgr2=0x7f1bd01a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd0103840 0x7f1bd01a13c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8007f10 tx=0x7f1bc800d3b0 comp rx=0 tx=0).stop 2024-03-11T19:38:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 -- 172.21.15.83:0/2807524032 shutdown_connections 2024-03-11T19:38:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bbc0a1b50 0x7f1bbc0a4010 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bd0101ca0 0x7f1bd0199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd0103840 0x7f1bd01a13c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 --2- 172.21.15.83:0/2807524032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bd0102ea0 0x7f1bd019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 -- 172.21.15.83:0/2807524032 >> 172.21.15.83:0/2807524032 conn(0x7f1bd00fd280 msgr2=0x7f1bd0073ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 -- 172.21.15.83:0/2807524032 shutdown_connections 2024-03-11T19:38:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:34.375+0000 7f1bd8414700 1 -- 172.21.15.83:0/2807524032 wait complete. 2024-03-11T19:38:34.982 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:34 smithi083 ceph-mon[31864]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:34.982 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:34 smithi083 ceph-mon[31864]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:38:35.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-11T19:38:35.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:34 smithi137 ceph-mon[36220]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:35.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:34 smithi137 ceph-mon[36220]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:38:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:34 smithi204 ceph-mon[36739]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:34 smithi204 ceph-mon[36739]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-11T19:38:35.249 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.828+0000 7f822f56f700 1 -- 172.21.15.83:0/3637672004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82280fffa0 msgr2=0x7f82281003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.828+0000 7f822f56f700 1 --2- 172.21.15.83:0/3637672004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82280fffa0 0x7f82281003c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8224009990 tx=0x7f8224009ca0 comp rx=0 tx=0).stop 2024-03-11T19:38:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.828+0000 7f822f56f700 1 -- 172.21.15.83:0/3637672004 shutdown_connections 2024-03-11T19:38:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.828+0000 7f822f56f700 1 --2- 172.21.15.83:0/3637672004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8228103920 0x7f822810a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.828+0000 7f822f56f700 1 --2- 172.21.15.83:0/3637672004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82280fffa0 0x7f82281003c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.828+0000 7f822f56f700 1 --2- 172.21.15.83:0/3637672004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8228102f80 0x7f82281033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.828+0000 7f822f56f700 1 -- 172.21.15.83:0/3637672004 >> 172.21.15.83:0/3637672004 conn(0x7f82280fb900 msgr2=0x7f82280fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.828+0000 7f822f56f700 1 -- 172.21.15.83:0/3637672004 shutdown_connections 2024-03-11T19:38:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.828+0000 7f822f56f700 1 -- 172.21.15.83:0/3637672004 wait complete. 2024-03-11T19:38:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.829+0000 7f822f56f700 1 Processor -- start 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.829+0000 7f822f56f700 1 -- start start 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.829+0000 7f822f56f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82280fffa0 0x7f8228197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.829+0000 7f822f56f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228102f80 0x7f8228198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.829+0000 7f822f56f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8228103920 0x7f822819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.829+0000 7f822f56f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822819f880 con 0x7f8228102f80 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.829+0000 7f822f56f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822819f9c0 con 0x7f82280fffa0 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.829+0000 7f822f56f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822819fb80 con 0x7f8228103920 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.829+0000 7f822cb0a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228102f80 0x7f8228198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.829+0000 7f822cb0a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228102f80 0x7f8228198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50730/0 (socket says 172.21.15.83:50730) 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.829+0000 7f822cb0a700 1 -- 172.21.15.83:0/618888397 learned_addr learned my addr 172.21.15.83:0/618888397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f822d30b700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82280fffa0 0x7f8228197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f822db0c700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8228103920 0x7f822819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f822cb0a700 1 -- 172.21.15.83:0/618888397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8228103920 msgr2=0x7f822819f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f822cb0a700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8228103920 0x7f822819f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f822cb0a700 1 -- 172.21.15.83:0/618888397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82280fffa0 msgr2=0x7f8228197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f822cb0a700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82280fffa0 0x7f8228197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f822cb0a700 1 -- 172.21.15.83:0/618888397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82240095f0 con 0x7f8228102f80 2024-03-11T19:38:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f822d30b700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82280fffa0 0x7f8228197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f822cb0a700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228102f80 0x7f8228198270 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f821800b530 tx=0x7f821800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f821e7fc700 1 -- 172.21.15.83:0/618888397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218011830 con 0x7f8228102f80 2024-03-11T19:38:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f821e7fc700 1 -- 172.21.15.83:0/618888397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8218011e70 con 0x7f8228102f80 2024-03-11T19:38:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f821e7fc700 1 -- 172.21.15.83:0/618888397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821800f2e0 con 0x7f8228102f80 2024-03-11T19:38:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f822f56f700 1 -- 172.21.15.83:0/618888397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822819fe60 con 0x7f8228102f80 2024-03-11T19:38:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.830+0000 7f822f56f700 1 -- 172.21.15.83:0/618888397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82281a0350 con 0x7f8228102f80 2024-03-11T19:38:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.831+0000 7f822f56f700 1 -- 172.21.15.83:0/618888397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8228068930 con 0x7f8228102f80 2024-03-11T19:38:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.832+0000 7f821e7fc700 1 -- 172.21.15.83:0/618888397 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7f82180105f0 con 0x7f8228102f80 2024-03-11T19:38:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.833+0000 7f821e7fc700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f82140a1bd0 0x7f82140a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.833+0000 7f822d30b700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f82140a1bd0 0x7f82140a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:36.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.833+0000 7f821e7fc700 1 -- 172.21.15.83:0/618888397 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f82180cce50 con 0x7f8228102f80 2024-03-11T19:38:36.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.833+0000 7f822d30b700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f82140a1bd0 0x7f82140a4090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f82240037b0 tx=0x7f822400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:36.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:36.837+0000 7f821e7fc700 1 -- 172.21.15.83:0/618888397 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8218096120 con 0x7f8228102f80 2024-03-11T19:38:37.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.041+0000 7f822f56f700 1 -- 172.21.15.83:0/618888397 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f822802d910 con 0x7f82140a1bd0 2024-03-11T19:38:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.049+0000 7f821e7fc700 1 -- 172.21.15.83:0/618888397 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f822802d910 con 0x7f82140a1bd0 2024-03-11T19:38:37.055 INFO:teuthology.orchestra.run.smithi083.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-11T19:38:37.055 INFO:teuthology.orchestra.run.smithi083.stdout:mds.all 12/12 31s ago 2m smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-11T19:38:37.055 INFO:teuthology.orchestra.run.smithi083.stdout:mgr 3/3 31s ago 7m smithi083=x;smithi137=y;smithi204=z;count:3 2024-03-11T19:38:37.055 INFO:teuthology.orchestra.run.smithi083.stdout:mon 3/3 31s ago 7m smithi083:172.21.15.83=a;smithi137:172.21.15.137=b;smithi204:172.21.15.204=c;count:3 2024-03-11T19:38:37.055 INFO:teuthology.orchestra.run.smithi083.stdout:osd 12 31s ago - 2024-03-11T19:38:37.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.051+0000 7f822f56f700 1 -- 172.21.15.83:0/618888397 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f82140a1bd0 msgr2=0x7f82140a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.051+0000 7f822f56f700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f82140a1bd0 0x7f82140a4090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f82240037b0 tx=0x7f822400b540 comp rx=0 tx=0).stop 2024-03-11T19:38:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.051+0000 7f822f56f700 1 -- 172.21.15.83:0/618888397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228102f80 msgr2=0x7f8228198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.051+0000 7f822f56f700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228102f80 0x7f8228198270 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f821800b530 tx=0x7f821800b8f0 comp rx=0 tx=0).stop 2024-03-11T19:38:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.051+0000 7f822f56f700 1 -- 172.21.15.83:0/618888397 shutdown_connections 2024-03-11T19:38:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.051+0000 7f822f56f700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f82140a1bd0 0x7f82140a4090 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.051+0000 7f822f56f700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8228103920 0x7f822819f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.051+0000 7f822f56f700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82280fffa0 0x7f8228197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.051+0000 7f822f56f700 1 --2- 172.21.15.83:0/618888397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8228102f80 0x7f8228198270 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.051+0000 7f822f56f700 1 -- 172.21.15.83:0/618888397 >> 172.21.15.83:0/618888397 conn(0x7f82280fb900 msgr2=0x7f8228106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.051+0000 7f822f56f700 1 -- 172.21.15.83:0/618888397 shutdown_connections 2024-03-11T19:38:37.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:37.052+0000 7f822f56f700 1 -- 172.21.15.83:0/618888397 wait complete. 2024-03-11T19:38:37.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-11T19:38:37.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:37 smithi083 ceph-mon[31864]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:37.871 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:37.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:37 smithi137 ceph-mon[36220]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:37 smithi204 ceph-mon[36739]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:38.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:38 smithi137 ceph-mon[36220]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:38.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:38 smithi204 ceph-mon[36739]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:38.970 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:38 smithi083 ceph-mon[31864]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:39.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.497+0000 7ffaf794f700 1 -- 172.21.15.83:0/1252520855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaf0101dd0 msgr2=0x7ffaf0102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:39.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.497+0000 7ffaf794f700 1 --2- 172.21.15.83:0/1252520855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaf0101dd0 0x7ffaf0102230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0099c0 tx=0x7ffaec009cd0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.497+0000 7ffaf794f700 1 -- 172.21.15.83:0/1252520855 shutdown_connections 2024-03-11T19:38:39.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.497+0000 7ffaf794f700 1 --2- 172.21.15.83:0/1252520855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf0100bd0 0x7ffaf0100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.497+0000 7ffaf794f700 1 --2- 172.21.15.83:0/1252520855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0102770 0x7ffaf0109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.497+0000 7ffaf794f700 1 --2- 172.21.15.83:0/1252520855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaf0101dd0 0x7ffaf0102230 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.497+0000 7ffaf794f700 1 -- 172.21.15.83:0/1252520855 >> 172.21.15.83:0/1252520855 conn(0x7ffaf00fc190 msgr2=0x7ffaf00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:39.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.497+0000 7ffaf794f700 1 -- 172.21.15.83:0/1252520855 shutdown_connections 2024-03-11T19:38:39.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.497+0000 7ffaf794f700 1 -- 172.21.15.83:0/1252520855 wait complete. 2024-03-11T19:38:39.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf794f700 1 Processor -- start 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf794f700 1 -- start start 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf794f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaf0100bd0 0x7ffaf0197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf794f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf0101dd0 0x7ffaf0198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf794f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0102770 0x7ffaf019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf794f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf019f770 con 0x7ffaf0100bd0 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf794f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf019f8b0 con 0x7ffaf0102770 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf794f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf019fa40 con 0x7ffaf0101dd0 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf56eb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaf0100bd0 0x7ffaf0197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf56eb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaf0100bd0 0x7ffaf0197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50750/0 (socket says 172.21.15.83:50750) 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf56eb700 1 -- 172.21.15.83:0/2000289823 learned_addr learned my addr 172.21.15.83:0/2000289823 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf4eea700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf0101dd0 0x7ffaf0198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf5eec700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0102770 0x7ffaf019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf56eb700 1 -- 172.21.15.83:0/2000289823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf0101dd0 msgr2=0x7ffaf0198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:39.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf56eb700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf0101dd0 0x7ffaf0198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf56eb700 1 -- 172.21.15.83:0/2000289823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0102770 msgr2=0x7ffaf019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf56eb700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0102770 0x7ffaf019f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf56eb700 1 -- 172.21.15.83:0/2000289823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec0095f0 con 0x7ffaf0100bd0 2024-03-11T19:38:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf4eea700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf0101dd0 0x7ffaf0198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf5eec700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0102770 0x7ffaf019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.498+0000 7ffaf56eb700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaf0100bd0 0x7ffaf0197be0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffae000c920 tx=0x7ffae000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.499+0000 7ffae67fc700 1 -- 172.21.15.83:0/2000289823 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0004220 con 0x7ffaf0100bd0 2024-03-11T19:38:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.499+0000 7ffae67fc700 1 -- 172.21.15.83:0/2000289823 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffae0007da0 con 0x7ffaf0100bd0 2024-03-11T19:38:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.499+0000 7ffaf794f700 1 -- 172.21.15.83:0/2000289823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf019fd20 con 0x7ffaf0100bd0 2024-03-11T19:38:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.499+0000 7ffae67fc700 1 -- 172.21.15.83:0/2000289823 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0003da0 con 0x7ffaf0100bd0 2024-03-11T19:38:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.499+0000 7ffaf794f700 1 -- 172.21.15.83:0/2000289823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf01a0170 con 0x7ffaf0100bd0 2024-03-11T19:38:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.500+0000 7ffaf794f700 1 -- 172.21.15.83:0/2000289823 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf010c530 con 0x7ffaf0100bd0 2024-03-11T19:38:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.501+0000 7ffae67fc700 1 -- 172.21.15.83:0/2000289823 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) v1 ==== 136330+0+0 (secure 0 0 0) 0x7ffae00078c0 con 0x7ffaf0100bd0 2024-03-11T19:38:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.501+0000 7ffae67fc700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffadc0a1bd0 0x7ffadc0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.501+0000 7ffaf4eea700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffadc0a1bd0 0x7ffadc0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:39.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.501+0000 7ffae67fc700 1 -- 172.21.15.83:0/2000289823 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ffae00cd0d0 con 0x7ffaf0100bd0 2024-03-11T19:38:39.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.502+0000 7ffaf4eea700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffadc0a1bd0 0x7ffadc0a4090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffaec000c00 tx=0x7ffaec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:39.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.505+0000 7ffae67fc700 1 -- 172.21.15.83:0/2000289823 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffae0096290 con 0x7ffaf0100bd0 2024-03-11T19:38:39.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.600+0000 7ffae67fc700 1 -- 172.21.15.83:0/2000289823 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ffae0096660 con 0x7ffaf0100bd0 2024-03-11T19:38:39.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.711+0000 7ffaf794f700 1 -- 172.21.15.83:0/2000289823 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ffaf00fe310 con 0x7ffadc0a1bd0 2024-03-11T19:38:39.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.713+0000 7ffae67fc700 1 -- 172.21.15.83:0/2000289823 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7ffaf00fe310 con 0x7ffadc0a1bd0 2024-03-11T19:38:39.718 INFO:teuthology.orchestra.run.smithi083.stdout:HOST ADDR LABELS STATUS 2024-03-11T19:38:39.718 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 172.21.15.83 2024-03-11T19:38:39.718 INFO:teuthology.orchestra.run.smithi083.stdout:smithi137 172.21.15.137 2024-03-11T19:38:39.718 INFO:teuthology.orchestra.run.smithi083.stdout:smithi204 172.21.15.204 2024-03-11T19:38:39.718 INFO:teuthology.orchestra.run.smithi083.stdout:3 hosts in cluster 2024-03-11T19:38:39.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 -- 172.21.15.83:0/2000289823 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffadc0a1bd0 msgr2=0x7ffadc0a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:39.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffadc0a1bd0 0x7ffadc0a4090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffaec000c00 tx=0x7ffaec005fd0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 -- 172.21.15.83:0/2000289823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaf0100bd0 msgr2=0x7ffaf0197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:39.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaf0100bd0 0x7ffaf0197be0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffae000c920 tx=0x7ffae000cce0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 -- 172.21.15.83:0/2000289823 shutdown_connections 2024-03-11T19:38:39.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffadc0a1bd0 0x7ffadc0a4090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf0102770 0x7ffaf019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaf0100bd0 0x7ffaf0197be0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 --2- 172.21.15.83:0/2000289823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffaf0101dd0 0x7ffaf0198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:39.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 -- 172.21.15.83:0/2000289823 >> 172.21.15.83:0/2000289823 conn(0x7ffaf00fc190 msgr2=0x7ffaf00fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:39.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 -- 172.21.15.83:0/2000289823 shutdown_connections 2024-03-11T19:38:39.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:39.715+0000 7ffaf794f700 1 -- 172.21.15.83:0/2000289823 wait complete. 2024-03-11T19:38:39.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:39 smithi137 ceph-mon[36220]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:38:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:39 smithi204 ceph-mon[36739]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:38:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:39 smithi083 ceph-mon[31864]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-11T19:38:40.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-11T19:38:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:40.779 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:40 smithi083 ceph-mon[31864]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-11T19:38:40.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:40 smithi137 ceph-mon[36220]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-11T19:38:40.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:40 smithi204 ceph-mon[36739]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-11T19:38:41.869 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:41 smithi083 ceph-mon[31864]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:41.869 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:41 smithi083 ceph-mon[31864]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:38:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:41 smithi137 ceph-mon[36220]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:41 smithi137 ceph-mon[36220]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:38:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:41 smithi204 ceph-mon[36739]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:41 smithi204 ceph-mon[36739]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-11T19:38:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.093+0000 7f3d0d584700 1 -- 172.21.15.83:0/3153305498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08102ea0 msgr2=0x7f3d08103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.093+0000 7f3d0d584700 1 --2- 172.21.15.83:0/3153305498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08102ea0 0x7f3d08103300 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0099c0 tx=0x7f3cfc009cd0 comp rx=0 tx=0).stop 2024-03-11T19:38:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.094+0000 7f3d0d584700 1 -- 172.21.15.83:0/3153305498 shutdown_connections 2024-03-11T19:38:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.094+0000 7f3d0d584700 1 --2- 172.21.15.83:0/3153305498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d08103840 0x7f3d0810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.094+0000 7f3d0d584700 1 --2- 172.21.15.83:0/3153305498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d08101ca0 0x7f3d081020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.094+0000 7f3d0d584700 1 --2- 172.21.15.83:0/3153305498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08102ea0 0x7f3d08103300 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.094+0000 7f3d0d584700 1 -- 172.21.15.83:0/3153305498 >> 172.21.15.83:0/3153305498 conn(0x7f3d080fd280 msgr2=0x7f3d080ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.094+0000 7f3d0d584700 1 -- 172.21.15.83:0/3153305498 shutdown_connections 2024-03-11T19:38:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.094+0000 7f3d0d584700 1 -- 172.21.15.83:0/3153305498 wait complete. 2024-03-11T19:38:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.094+0000 7f3d0d584700 1 Processor -- start 2024-03-11T19:38:42.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.094+0000 7f3d0d584700 1 -- start start 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d0d584700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08101ca0 0x7f3d08075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d0d584700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d08102ea0 0x7f3d08074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d0d584700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d08103840 0x7f3d080745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d0d584700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d08074d20 con 0x7f3d08101ca0 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d0d584700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d08074e90 con 0x7f3d08102ea0 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d0d584700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d081a1cf0 con 0x7f3d08103840 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d06ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08101ca0 0x7f3d08075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d06ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08101ca0 0x7f3d08075a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47952/0 (socket says 172.21.15.83:47952) 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d06ffd700 1 -- 172.21.15.83:0/1087533941 learned_addr learned my addr 172.21.15.83:0/1087533941 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d077fe700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d08103840 0x7f3d080745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d067fc700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d08102ea0 0x7f3d08074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:42.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d06ffd700 1 -- 172.21.15.83:0/1087533941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d08103840 msgr2=0x7f3d080745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d06ffd700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d08103840 0x7f3d080745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d06ffd700 1 -- 172.21.15.83:0/1087533941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d08102ea0 msgr2=0x7f3d08074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d06ffd700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d08102ea0 0x7f3d08074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d06ffd700 1 -- 172.21.15.83:0/1087533941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc0095f0 con 0x7f3d08101ca0 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d067fc700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d08102ea0 0x7f3d08074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d077fe700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d08103840 0x7f3d080745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d06ffd700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08101ca0 0x7f3d08075a10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800b500 tx=0x7f3cf800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3ceffff700 1 -- 172.21.15.83:0/1087533941 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8011830 con 0x7f3d08101ca0 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3ceffff700 1 -- 172.21.15.83:0/1087533941 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3cf8011e70 con 0x7f3d08101ca0 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d0d584700 1 -- 172.21.15.83:0/1087533941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d081a1fd0 con 0x7f3d08101ca0 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3ceffff700 1 -- 172.21.15.83:0/1087533941 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800f2e0 con 0x7f3d08101ca0 2024-03-11T19:38:42.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.095+0000 7f3d0d584700 1 -- 172.21.15.83:0/1087533941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d081a23e0 con 0x7f3d08101ca0 2024-03-11T19:38:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.096+0000 7f3d0d584700 1 -- 172.21.15.83:0/1087533941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d08109060 con 0x7f3d08101ca0 2024-03-11T19:38:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.097+0000 7f3ceffff700 1 -- 172.21.15.83:0/1087533941 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3cf80105f0 con 0x7f3d08101ca0 2024-03-11T19:38:42.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.098+0000 7f3ceffff700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3cf00a1ac0 0x7f3cf00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:42.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.098+0000 7f3d067fc700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3cf00a1ac0 0x7f3cf00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:42.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.098+0000 7f3ceffff700 1 -- 172.21.15.83:0/1087533941 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3cf80cce20 con 0x7f3d08101ca0 2024-03-11T19:38:42.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.098+0000 7f3d067fc700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3cf00a1ac0 0x7f3cf00a3f80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0038c0 tx=0x7f3cfc009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:42.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.102+0000 7f3ceffff700 1 -- 172.21.15.83:0/1087533941 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3cf80960e0 con 0x7f3d08101ca0 2024-03-11T19:38:42.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.305+0000 7f3d0d584700 1 -- 172.21.15.83:0/1087533941 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d0802da20 con 0x7f3cf00a1ac0 2024-03-11T19:38:42.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.307+0000 7f3ceffff700 1 -- 172.21.15.83:0/1087533941 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f3d0802da20 con 0x7f3cf00a1ac0 2024-03-11T19:38:42.311 INFO:teuthology.orchestra.run.smithi083.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-11T19:38:42.311 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040148400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-11T19:38:42.311 INFO:teuthology.orchestra.run.smithi083.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-11T19:38:42.311 INFO:teuthology.orchestra.run.smithi083.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-11T19:38:42.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 -- 172.21.15.83:0/1087533941 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3cf00a1ac0 msgr2=0x7f3cf00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:42.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3cf00a1ac0 0x7f3cf00a3f80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0038c0 tx=0x7f3cfc009f10 comp rx=0 tx=0).stop 2024-03-11T19:38:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 -- 172.21.15.83:0/1087533941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08101ca0 msgr2=0x7f3d08075a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08101ca0 0x7f3d08075a10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800b500 tx=0x7f3cf800b8c0 comp rx=0 tx=0).stop 2024-03-11T19:38:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 -- 172.21.15.83:0/1087533941 shutdown_connections 2024-03-11T19:38:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3cf00a1ac0 0x7f3cf00a3f80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d08103840 0x7f3d080745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d08101ca0 0x7f3d08075a10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 --2- 172.21.15.83:0/1087533941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d08102ea0 0x7f3d08074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 -- 172.21.15.83:0/1087533941 >> 172.21.15.83:0/1087533941 conn(0x7f3d080fd280 msgr2=0x7f3d081061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 -- 172.21.15.83:0/1087533941 shutdown_connections 2024-03-11T19:38:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:42.309+0000 7f3d0d584700 1 -- 172.21.15.83:0/1087533941 wait complete. 2024-03-11T19:38:42.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:42 smithi137 ceph-mon[36220]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:42.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:42 smithi137 ceph-mon[36220]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:38:42.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:42 smithi204 ceph-mon[36739]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:42.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:42 smithi204 ceph-mon[36739]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:38:42.951 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:42 smithi083 ceph-mon[31864]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-11T19:38:42.951 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:42 smithi083 ceph-mon[31864]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:38:42.977 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-11T19:38:42.989 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-03-11T19:38:42.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-11T19:38:43.234 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:44.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.984+0000 7fba32238700 1 -- 172.21.15.83:0/1059917634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba2c101dd0 msgr2=0x7fba2c102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:44.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.984+0000 7fba32238700 1 --2- 172.21.15.83:0/1059917634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba2c101dd0 0x7fba2c102230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fba140099c0 tx=0x7fba14009cd0 comp rx=0 tx=0).stop 2024-03-11T19:38:44.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.984+0000 7fba32238700 1 -- 172.21.15.83:0/1059917634 shutdown_connections 2024-03-11T19:38:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.984+0000 7fba32238700 1 --2- 172.21.15.83:0/1059917634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba2c100bd0 0x7fba2c100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.984+0000 7fba32238700 1 --2- 172.21.15.83:0/1059917634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba2c102770 0x7fba2c109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.984+0000 7fba32238700 1 --2- 172.21.15.83:0/1059917634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba2c101dd0 0x7fba2c102230 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.984+0000 7fba32238700 1 -- 172.21.15.83:0/1059917634 >> 172.21.15.83:0/1059917634 conn(0x7fba2c0fc190 msgr2=0x7fba2c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.984+0000 7fba32238700 1 -- 172.21.15.83:0/1059917634 shutdown_connections 2024-03-11T19:38:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.984+0000 7fba32238700 1 -- 172.21.15.83:0/1059917634 wait complete. 2024-03-11T19:38:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.984+0000 7fba32238700 1 Processor -- start 2024-03-11T19:38:44.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.984+0000 7fba32238700 1 -- start start 2024-03-11T19:38:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba32238700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba2c100bd0 0x7fba2c196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba32238700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba2c101dd0 0x7fba2c1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba32238700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba2c102770 0x7fba2c19e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba32238700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c19e7b0 con 0x7fba2c100bd0 2024-03-11T19:38:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba32238700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c19e8f0 con 0x7fba2c101dd0 2024-03-11T19:38:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba32238700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c19eab0 con 0x7fba2c102770 2024-03-11T19:38:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba2b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba2c100bd0 0x7fba2c196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba2b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba2c100bd0 0x7fba2c196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47980/0 (socket says 172.21.15.83:47980) 2024-03-11T19:38:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba2b7fe700 1 -- 172.21.15.83:0/1632716348 learned_addr learned my addr 172.21.15.83:0/1632716348 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba2bfff700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba2c102770 0x7fba2c19e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba22ffd700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba2c101dd0 0x7fba2c1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba2b7fe700 1 -- 172.21.15.83:0/1632716348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba2c102770 msgr2=0x7fba2c19e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba2b7fe700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba2c102770 0x7fba2c19e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba2b7fe700 1 -- 172.21.15.83:0/1632716348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba2c101dd0 msgr2=0x7fba2c1971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba2b7fe700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba2c101dd0 0x7fba2c1971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba2b7fe700 1 -- 172.21.15.83:0/1632716348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba140095f0 con 0x7fba2c100bd0 2024-03-11T19:38:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba22ffd700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba2c101dd0 0x7fba2c1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba2b7fe700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba2c100bd0 0x7fba2c196c60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00b500 tx=0x7fba1c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba297fa700 1 -- 172.21.15.83:0/1632716348 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c011830 con 0x7fba2c100bd0 2024-03-11T19:38:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba32238700 1 -- 172.21.15.83:0/1632716348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c19ed90 con 0x7fba2c100bd0 2024-03-11T19:38:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba297fa700 1 -- 172.21.15.83:0/1632716348 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba1c011e70 con 0x7fba2c100bd0 2024-03-11T19:38:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba297fa700 1 -- 172.21.15.83:0/1632716348 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00f2e0 con 0x7fba2c100bd0 2024-03-11T19:38:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.985+0000 7fba32238700 1 -- 172.21.15.83:0/1632716348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c19f200 con 0x7fba2c100bd0 2024-03-11T19:38:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.986+0000 7fba32238700 1 -- 172.21.15.83:0/1632716348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c068930 con 0x7fba2c100bd0 2024-03-11T19:38:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.987+0000 7fba297fa700 1 -- 172.21.15.83:0/1632716348 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fba1c0105f0 con 0x7fba2c100bd0 2024-03-11T19:38:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.988+0000 7fba297fa700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba180a1ac0 0x7fba180a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:44.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.988+0000 7fba22ffd700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba180a1ac0 0x7fba180a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:44.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.988+0000 7fba297fa700 1 -- 172.21.15.83:0/1632716348 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fba1c0cce00 con 0x7fba2c100bd0 2024-03-11T19:38:44.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.988+0000 7fba22ffd700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba180a1ac0 0x7fba180a3f80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fba14006010 tx=0x7fba14009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:44.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:44.992+0000 7fba297fa700 1 -- 172.21.15.83:0/1632716348 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba1c0960c0 con 0x7fba2c100bd0 2024-03-11T19:38:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.228+0000 7fba32238700 1 -- 172.21.15.83:0/1632716348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fba2c107f90 con 0x7fba2c100bd0 2024-03-11T19:38:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.229+0000 7fba297fa700 1 -- 172.21.15.83:0/1632716348 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+2945 (secure 0 0 0) 0x7fba1c015ba0 con 0x7fba2c100bd0 2024-03-11T19:38:45.233 INFO:teuthology.orchestra.run.smithi083.stdout:e21 2024-03-11T19:38:45.233 INFO:teuthology.orchestra.run.smithi083.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-11T19:38:45.233 INFO:teuthology.orchestra.run.smithi083.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-11T19:38:45.233 INFO:teuthology.orchestra.run.smithi083.stdout:legacy client fscid: 1 2024-03-11T19:38:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:Filesystem 'cephfs' (1) 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:fs_name cephfs 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 21 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-03-11T19:37:04.916864+0000 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-03-11T19:37:09.619596+0000 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:tableserver 0 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:root 0 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:session_timeout 300 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:session_autoclose 300 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:max_file_size 1099511627776 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:max_xattr_size 65536 2024-03-11T19:38:45.234 INFO:teuthology.orchestra.run.smithi083.stdout:required_client_features {} 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure 0 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure_osd_epoch 0 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:max_mds 3 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:in 0,1,2 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:up {0=24527,1=24502,2=24494} 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:failed 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:damaged 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:stopped 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:data_pools [3] 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:metadata_pool 2 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:inline_data disabled 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:balancer 2024-03-11T19:38:45.235 INFO:teuthology.orchestra.run.smithi083.stdout:bal_rank_mask -1 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout:standby_count_wanted 0 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b{0:24527} state up:active seq 3 addr [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c{0:24478} state up:standby-replay seq 1 addr [v2:172.21.15.204:6836/2654992874,v1:172.21.15.204:6837/2654992874] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.i{1:24502} state up:active seq 5 addr [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.e{1:24470} state up:standby-replay seq 1 addr [v2:172.21.15.137:6834/1924725912,v1:172.21.15.137:6835/1924725912] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.h{2:24494} state up:active seq 6 addr [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.f{2:24466} state up:standby-replay seq 1 addr [v2:172.21.15.204:6834/2604685946,v1:172.21.15.204:6835/2604685946] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout:Standby daemons: 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.j{-1:14508} state up:standby seq 1 addr [v2:172.21.15.83:6834/4213430163,v1:172.21.15.83:6835/4213430163] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d{-1:14520} state up:standby seq 1 addr [v2:172.21.15.83:6836/1285695998,v1:172.21.15.83:6837/1285695998] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.236 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.g{-1:14583} state up:standby seq 1 addr [v2:172.21.15.83:6838/1413247052,v1:172.21.15.83:6839/1413247052] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.237 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a{-1:14595} state up:standby seq 1 addr [v2:172.21.15.83:6840/1703400254,v1:172.21.15.83:6841/1703400254] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.237 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.l{-1:24415} state up:standby seq 1 addr [v2:172.21.15.204:6832/1242693918,v1:172.21.15.204:6833/1242693918] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.237 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.k{-1:24431} state up:standby seq 1 addr [v2:172.21.15.137:6832/1480144108,v1:172.21.15.137:6833/1480144108] compat {c=[1],r=[1],i=[fff]}] 2024-03-11T19:38:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 -- 172.21.15.83:0/1632716348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba180a1ac0 msgr2=0x7fba180a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba180a1ac0 0x7fba180a3f80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fba14006010 tx=0x7fba14009f10 comp rx=0 tx=0).stop 2024-03-11T19:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 -- 172.21.15.83:0/1632716348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba2c100bd0 msgr2=0x7fba2c196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba2c100bd0 0x7fba2c196c60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00b500 tx=0x7fba1c00b810 comp rx=0 tx=0).stop 2024-03-11T19:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 -- 172.21.15.83:0/1632716348 shutdown_connections 2024-03-11T19:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba180a1ac0 0x7fba180a3f80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba2c102770 0x7fba2c19e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba2c100bd0 0x7fba2c196c60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 --2- 172.21.15.83:0/1632716348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba2c101dd0 0x7fba2c1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 -- 172.21.15.83:0/1632716348 >> 172.21.15.83:0/1632716348 conn(0x7fba2c0fc190 msgr2=0x7fba2c1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 -- 172.21.15.83:0/1632716348 shutdown_connections 2024-03-11T19:38:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:45.231+0000 7fba32238700 1 -- 172.21.15.83:0/1632716348 wait complete. 2024-03-11T19:38:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-03-11T19:38:45.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:45 smithi137 ceph-mon[36220]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-11T19:38:45.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:45 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1632716348' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-11T19:38:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:45 smithi204 ceph-mon[36739]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-11T19:38:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:45 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1632716348' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-11T19:38:46.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:45 smithi083 ceph-mon[31864]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-11T19:38:46.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:45 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1632716348' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-11T19:38:46.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-11T19:38:46.314 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:47.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:47 smithi083 ceph-mon[31864]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-11T19:38:47.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:47 smithi137 ceph-mon[36220]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-11T19:38:47.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:47 smithi204 ceph-mon[36739]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-11T19:38:48.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.034+0000 7f37f8838700 1 -- 172.21.15.83:0/4207764343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4107a00 msgr2=0x7f37f4109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:48.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.034+0000 7f37f8838700 1 --2- 172.21.15.83:0/4207764343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4107a00 0x7f37f4109e70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00b0a0 tx=0x7f37ec00b3b0 comp rx=0 tx=0).stop 2024-03-11T19:38:48.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.034+0000 7f37f8838700 1 -- 172.21.15.83:0/4207764343 shutdown_connections 2024-03-11T19:38:48.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.034+0000 7f37f8838700 1 --2- 172.21.15.83:0/4207764343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4107a00 0x7f37f4109e70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:48.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.034+0000 7f37f8838700 1 --2- 172.21.15.83:0/4207764343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37f41027a0 0x7f37f4104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:48.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.034+0000 7f37f8838700 1 --2- 172.21.15.83:0/4207764343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37f41050d0 0x7f37f41074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:48.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.034+0000 7f37f8838700 1 -- 172.21.15.83:0/4207764343 >> 172.21.15.83:0/4207764343 conn(0x7f37f40fc1d0 msgr2=0x7f37f40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:48.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.034+0000 7f37f8838700 1 -- 172.21.15.83:0/4207764343 shutdown_connections 2024-03-11T19:38:48.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.034+0000 7f37f8838700 1 -- 172.21.15.83:0/4207764343 wait complete. 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f8838700 1 Processor -- start 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f8838700 1 -- start start 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f8838700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37f41027a0 0x7f37f4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f8838700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37f41050d0 0x7f37f41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f8838700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4107a00 0x7f37f419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f8838700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f419f940 con 0x7f37f4107a00 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f8838700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f419fa80 con 0x7f37f41027a0 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f8838700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f419fc10 con 0x7f37f41050d0 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4107a00 0x7f37f419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f2d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4107a00 0x7f37f419f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47998/0 (socket says 172.21.15.83:47998) 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f2d9d700 1 -- 172.21.15.83:0/2198168881 learned_addr learned my addr 172.21.15.83:0/2198168881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:48.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37ebfff700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37f41050d0 0x7f37f41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f259c700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37f41027a0 0x7f37f4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f2d9d700 1 -- 172.21.15.83:0/2198168881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37f41050d0 msgr2=0x7f37f41982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f2d9d700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37f41050d0 0x7f37f41982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f2d9d700 1 -- 172.21.15.83:0/2198168881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37f41027a0 msgr2=0x7f37f4197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f2d9d700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37f41027a0 0x7f37f4197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f2d9d700 1 -- 172.21.15.83:0/2198168881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec009d00 con 0x7f37f4107a00 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f259c700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37f41027a0 0x7f37f4197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.035+0000 7f37f2d9d700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4107a00 0x7f37f419f370 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009420 tx=0x7f37ec009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.036+0000 7f37eb7fe700 1 -- 172.21.15.83:0/2198168881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec00e040 con 0x7f37f4107a00 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.036+0000 7f37eb7fe700 1 -- 172.21.15.83:0/2198168881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37ec01d960 con 0x7f37f4107a00 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.036+0000 7f37f8838700 1 -- 172.21.15.83:0/2198168881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f419fe90 con 0x7f37f4107a00 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.036+0000 7f37eb7fe700 1 -- 172.21.15.83:0/2198168881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec01ba80 con 0x7f37f4107a00 2024-03-11T19:38:48.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.036+0000 7f37f8838700 1 -- 172.21.15.83:0/2198168881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f41a0320 con 0x7f37f4107a00 2024-03-11T19:38:48.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.037+0000 7f37f8838700 1 -- 172.21.15.83:0/2198168881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f40fdd50 con 0x7f37f4107a00 2024-03-11T19:38:48.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.038+0000 7f37eb7fe700 1 -- 172.21.15.83:0/2198168881 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f37ec01dad0 con 0x7f37f4107a00 2024-03-11T19:38:48.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.038+0000 7f37eb7fe700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37e00a1ac0 0x7f37e00a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:48.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.038+0000 7f37f259c700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37e00a1ac0 0x7f37e00a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:48.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.039+0000 7f37eb7fe700 1 -- 172.21.15.83:0/2198168881 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f37ec0cef70 con 0x7f37f4107a00 2024-03-11T19:38:48.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.039+0000 7f37f259c700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37e00a1ac0 0x7f37e00a3f80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f37e40099d0 tx=0x7f37e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:48.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.043+0000 7f37eb7fe700 1 -- 172.21.15.83:0/2198168881 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f37ec0982c0 con 0x7f37f4107a00 2024-03-11T19:38:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.238+0000 7f37f8838700 1 -- 172.21.15.83:0/2198168881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f37f410d6e0 con 0x7f37f4107a00 2024-03-11T19:38:48.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.239+0000 7f37eb7fe700 1 -- 172.21.15.83:0/2198168881 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5220 (secure 0 0 0) 0x7f37ec09b980 con 0x7f37f4107a00 2024-03-11T19:38:48.243 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 87 2024-03-11T19:38:48.243 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-03-11T19:29:04.070278+0000 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-03-11T19:38:10.141528+0000 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:crush_version 26 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:full_ratio 0.95 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:backfillfull_ratio 0.9 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:nearfull_ratio 0.85 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:require_min_compat_client luminous 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:min_compat_client luminous 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:require_osd_release squid 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:stretch_mode_enabled false 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-11T19:38:48.244 INFO:teuthology.orchestra.run.smithi083.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:max_osd 12 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6802/3703691285,v1:172.21.15.83:6803/3703691285] [v2:172.21.15.83:6804/3703691285,v1:172.21.15.83:6805/3703691285] exists,up d4626dde-b940-4692-9e2d-2a2f3f59a70f 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6800/415302205,v1:172.21.15.137:6801/415302205] [v2:172.21.15.137:6802/415302205,v1:172.21.15.137:6803/415302205] exists,up bb07056c-aceb-4fd5-8b42-c1e33fede2b3 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/4285299727,v1:172.21.15.204:6801/4285299727] [v2:172.21.15.204:6802/4285299727,v1:172.21.15.204:6803/4285299727] exists,up c0f4f2c4-3f48-4c0f-a438-de6d8f151438 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6810/780035155,v1:172.21.15.83:6811/780035155] [v2:172.21.15.83:6812/780035155,v1:172.21.15.83:6813/780035155] exists,up 12c251e9-dcee-4d62-b731-9a63cd9815fe 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6808/4100342703,v1:172.21.15.137:6809/4100342703] [v2:172.21.15.137:6810/4100342703,v1:172.21.15.137:6811/4100342703] exists,up 438ba247-da85-4ac4-967f-3ded2137b07d 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/2553953517,v1:172.21.15.204:6809/2553953517] [v2:172.21.15.204:6810/2553953517,v1:172.21.15.204:6811/2553953517] exists,up c57d7878-189e-48d6-8d9a-a4b105dea801 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6818/3426343907,v1:172.21.15.83:6819/3426343907] [v2:172.21.15.83:6820/3426343907,v1:172.21.15.83:6821/3426343907] exists,up 829db8a4-7f58-4770-8bd7-8cd258b2b1be 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6816/2113619017,v1:172.21.15.137:6817/2113619017] [v2:172.21.15.137:6818/2113619017,v1:172.21.15.137:6819/2113619017] exists,up 2cf6f477-f5c5-4764-9110-98898779c9f9 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/3550974508,v1:172.21.15.204:6817/3550974508] [v2:172.21.15.204:6818/3550974508,v1:172.21.15.204:6819/3550974508] exists,up 9e808c31-b428-45c6-896d-42eba00a25f5 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 up in weight 1 up_from 67 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6826/2013526190,v1:172.21.15.83:6827/2013526190] [v2:172.21.15.83:6828/2013526190,v1:172.21.15.83:6829/2013526190] exists,up 78fabab9-c022-4e3a-a912-aab73696f4b6 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6824/3042743423,v1:172.21.15.137:6825/3042743423] [v2:172.21.15.137:6826/3042743423,v1:172.21.15.137:6827/3042743423] exists,up c7f76fae-ccd2-4b2b-8885-fb008d99ad9b 2024-03-11T19:38:48.245 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/2753506819,v1:172.21.15.204:6825/2753506819] [v2:172.21.15.204:6826/2753506819,v1:172.21.15.204:6827/2753506819] exists,up 7778999c-58d4-487e-a8ae-eb317e078f58 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.b [2,1,8,11] 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.10 [2,4,8,11] 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.13 [4,9] 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.21 [5,9] 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.2c [4,9] 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1699838714 expires 2024-03-12T19:30:08.427867+0000 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/2543145311 expires 2024-03-12T19:30:08.427867+0000 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1376534596 expires 2024-03-12T19:30:08.427867+0000 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/4058944299 expires 2024-03-12T19:30:08.427867+0000 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/2370098752 expires 2024-03-12T19:29:41.798770+0000 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/2370098752 expires 2024-03-12T19:29:41.798770+0000 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1957272663 expires 2024-03-12T19:29:41.798770+0000 2024-03-11T19:38:48.246 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/407682476 expires 2024-03-12T19:29:41.798770+0000 2024-03-11T19:38:48.247 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3034283770 expires 2024-03-12T19:29:41.798770+0000 2024-03-11T19:38:48.247 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/4058944299 expires 2024-03-12T19:30:08.427867+0000 2024-03-11T19:38:48.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.241+0000 7f37f8838700 1 -- 172.21.15.83:0/2198168881 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37e00a1ac0 msgr2=0x7f37e00a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.241+0000 7f37f8838700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37e00a1ac0 0x7f37e00a3f80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f37e40099d0 tx=0x7f37e4008040 comp rx=0 tx=0).stop 2024-03-11T19:38:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.241+0000 7f37f8838700 1 -- 172.21.15.83:0/2198168881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4107a00 msgr2=0x7f37f419f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.241+0000 7f37f8838700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4107a00 0x7f37f419f370 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009420 tx=0x7f37ec009450 comp rx=0 tx=0).stop 2024-03-11T19:38:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.241+0000 7f37f8838700 1 -- 172.21.15.83:0/2198168881 shutdown_connections 2024-03-11T19:38:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.241+0000 7f37f8838700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37e00a1ac0 0x7f37e00a3f80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.241+0000 7f37f8838700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4107a00 0x7f37f419f370 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.241+0000 7f37f8838700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37f41027a0 0x7f37f4197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.241+0000 7f37f8838700 1 --2- 172.21.15.83:0/2198168881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37f41050d0 0x7f37f41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.242+0000 7f37f8838700 1 -- 172.21.15.83:0/2198168881 >> 172.21.15.83:0/2198168881 conn(0x7f37f40fc1d0 msgr2=0x7f37f4103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.242+0000 7f37f8838700 1 -- 172.21.15.83:0/2198168881 shutdown_connections 2024-03-11T19:38:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:48.242+0000 7f37f8838700 1 -- 172.21.15.83:0/2198168881 wait complete. 2024-03-11T19:38:48.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:48 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2198168881' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-11T19:38:48.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:48 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2198168881' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-11T19:38:48.983 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:48 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2198168881' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-11T19:38:49.010 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-11T19:38:49.022 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:38:49.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.121+0000 7f56ea7bc640 1 -- 172.21.15.83:0/2994734613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56e4102390 msgr2=0x7f56e4102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.121+0000 7f56ea7bc640 1 --2- 172.21.15.83:0/2994734613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56e4102390 0x7f56e4102810 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009900 tx=0x7f56d402bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.121+0000 7f56ea7bc640 1 -- 172.21.15.83:0/2994734613 shutdown_connections 2024-03-11T19:38:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.121+0000 7f56ea7bc640 1 --2- 172.21.15.83:0/2994734613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e4102d50 0x7f56e41095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.121+0000 7f56ea7bc640 1 --2- 172.21.15.83:0/2994734613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56e4102390 0x7f56e4102810 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.121+0000 7f56ea7bc640 1 --2- 172.21.15.83:0/2994734613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56e41011a0 0x7f56e41015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.121+0000 7f56ea7bc640 1 -- 172.21.15.83:0/2994734613 >> 172.21.15.83:0/2994734613 conn(0x7f56e40fc730 msgr2=0x7f56e40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.121+0000 7f56ea7bc640 1 -- 172.21.15.83:0/2994734613 shutdown_connections 2024-03-11T19:38:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.121+0000 7f56ea7bc640 1 -- 172.21.15.83:0/2994734613 wait complete. 2024-03-11T19:38:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56ea7bc640 1 Processor -- start 2024-03-11T19:38:49.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56ea7bc640 1 -- start start 2024-03-11T19:38:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56ea7bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56e41011a0 0x7f56e4074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56ea7bc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56e4102390 0x7f56e4073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56ea7bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e4102d50 0x7f56e4073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56ea7bc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4074210 con 0x7f56e41011a0 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56ea7bc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4074380 con 0x7f56e4102d50 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56ea7bc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e41a1340 con 0x7f56e4102390 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56e41011a0 0x7f56e4074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56e41011a0 0x7f56e4074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48022/0 (socket says 172.21.15.83:48022) 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56e3fff640 1 -- 172.21.15.83:0/197290162 learned_addr learned my addr 172.21.15.83:0/197290162 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56e37fe640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56e4102390 0x7f56e4073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56e8d59640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e4102d50 0x7f56e4073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56e3fff640 1 -- 172.21.15.83:0/197290162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56e4102390 msgr2=0x7f56e4073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.122+0000 7f56e3fff640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56e4102390 0x7f56e4073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.123+0000 7f56e3fff640 1 -- 172.21.15.83:0/197290162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e4102d50 msgr2=0x7f56e4073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.123+0000 7f56e3fff640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e4102d50 0x7f56e4073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.123+0000 7f56e3fff640 1 -- 172.21.15.83:0/197290162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d4009560 con 0x7f56e41011a0 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.123+0000 7f56e8d59640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e4102d50 0x7f56e4073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.123+0000 7f56e3fff640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56e41011a0 0x7f56e4074f30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f56d800c9a0 tx=0x7f56d800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:49.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.123+0000 7f56e17fa640 1 -- 172.21.15.83:0/197290162 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8004220 con 0x7f56e41011a0 2024-03-11T19:38:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.123+0000 7f56e17fa640 1 -- 172.21.15.83:0/197290162 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56d8007e90 con 0x7f56e41011a0 2024-03-11T19:38:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.123+0000 7f56ea7bc640 1 -- 172.21.15.83:0/197290162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e41a15f0 con 0x7f56e41011a0 2024-03-11T19:38:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.123+0000 7f56e17fa640 1 -- 172.21.15.83:0/197290162 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8004a00 con 0x7f56e41011a0 2024-03-11T19:38:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.123+0000 7f56ea7bc640 1 -- 172.21.15.83:0/197290162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e41a1b30 con 0x7f56e41011a0 2024-03-11T19:38:49.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.124+0000 7f56ea7bc640 1 -- 172.21.15.83:0/197290162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e41084c0 con 0x7f56e41011a0 2024-03-11T19:38:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.125+0000 7f56e17fa640 1 -- 172.21.15.83:0/197290162 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f56d8004380 con 0x7f56e41011a0 2024-03-11T19:38:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.125+0000 7f56e17fa640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56cc0a1a30 0x7f56cc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.125+0000 7f56e37fe640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56cc0a1a30 0x7f56cc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.125+0000 7f56e17fa640 1 -- 172.21.15.83:0/197290162 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f56d80cd4c0 con 0x7f56e41011a0 2024-03-11T19:38:49.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.126+0000 7f56e37fe640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56cc0a1a30 0x7f56cc0a3ef0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f56e4074ca0 tx=0x7f56d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:49.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.130+0000 7f56e17fa640 1 -- 172.21.15.83:0/197290162 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56d8096780 con 0x7f56e41011a0 2024-03-11T19:38:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.303+0000 7f56ea7bc640 1 -- 172.21.15.83:0/197290162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56e410cc10 con 0x7f56e41011a0 2024-03-11T19:38:49.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.305+0000 7f56e17fa640 1 -- 172.21.15.83:0/197290162 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7f56d8099e40 con 0x7f56e41011a0 2024-03-11T19:38:49.309 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:49.310 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:38:49.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 -- 172.21.15.83:0/197290162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56cc0a1a30 msgr2=0x7f56cc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56cc0a1a30 0x7f56cc0a3ef0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f56e4074ca0 tx=0x7f56d4000bc0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 -- 172.21.15.83:0/197290162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56e41011a0 msgr2=0x7f56e4074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56e41011a0 0x7f56e4074f30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f56d800c9a0 tx=0x7f56d800ce60 comp rx=0 tx=0).stop 2024-03-11T19:38:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 -- 172.21.15.83:0/197290162 shutdown_connections 2024-03-11T19:38:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56cc0a1a30 0x7f56cc0a3ef0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56e4102d50 0x7f56e4073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56e4102390 0x7f56e4073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 --2- 172.21.15.83:0/197290162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56e41011a0 0x7f56e4074f30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 -- 172.21.15.83:0/197290162 >> 172.21.15.83:0/197290162 conn(0x7f56e40fc730 msgr2=0x7f56e4105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 -- 172.21.15.83:0/197290162 shutdown_connections 2024-03-11T19:38:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.307+0000 7f56ea7bc640 1 -- 172.21.15.83:0/197290162 wait complete. 2024-03-11T19:38:49.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:37:03.034982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:37:04.125352+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:38:49.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:38:49.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.424+0000 7f0fbc240640 1 -- 172.21.15.83:0/3469096921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fb4101c30 msgr2=0x7f0fb41084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.424+0000 7f0fbc240640 1 --2- 172.21.15.83:0/3469096921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fb4101c30 0x7f0fb41084c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009900 tx=0x7f0fac02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.424+0000 7f0fbc240640 1 -- 172.21.15.83:0/3469096921 shutdown_connections 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.424+0000 7f0fbc240640 1 --2- 172.21.15.83:0/3469096921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fb4101c30 0x7f0fb41084c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.424+0000 7f0fbc240640 1 --2- 172.21.15.83:0/3469096921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb4101270 0x7f0fb41016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 --2- 172.21.15.83:0/3469096921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb4100080 0x7f0fb41004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 -- 172.21.15.83:0/3469096921 >> 172.21.15.83:0/3469096921 conn(0x7f0fb40fb610 msgr2=0x7f0fb40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 -- 172.21.15.83:0/3469096921 shutdown_connections 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 -- 172.21.15.83:0/3469096921 wait complete. 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 Processor -- start 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 -- start start 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb4100080 0x7f0fb4196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb4101270 0x7f0fb4196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fb4101c30 0x7f0fb419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb419dc30 con 0x7f0fb4101c30 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb419dda0 con 0x7f0fb4100080 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fbc240640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb419df60 con 0x7f0fb4101270 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fba7dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fb4101c30 0x7f0fb419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fba7dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fb4101c30 0x7f0fb419d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48040/0 (socket says 172.21.15.83:48040) 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.425+0000 7f0fba7dd640 1 -- 172.21.15.83:0/3833609001 learned_addr learned my addr 172.21.15.83:0/3833609001 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0fb97db640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb4101270 0x7f0fb4196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0fb9fdc640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb4100080 0x7f0fb4196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0fba7dd640 1 -- 172.21.15.83:0/3833609001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb4101270 msgr2=0x7f0fb4196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0fba7dd640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb4101270 0x7f0fb4196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0fba7dd640 1 -- 172.21.15.83:0/3833609001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb4100080 msgr2=0x7f0fb4196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0fba7dd640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb4100080 0x7f0fb4196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0fba7dd640 1 -- 172.21.15.83:0/3833609001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac009560 con 0x7f0fb4101c30 2024-03-11T19:38:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0fb9fdc640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb4100080 0x7f0fb4196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0fba7dd640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fb4101c30 0x7f0fb419d6f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009230 tx=0x7f0fac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0faaffd640 1 -- 172.21.15.83:0/3833609001 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0044e0 con 0x7f0fb4101c30 2024-03-11T19:38:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0faaffd640 1 -- 172.21.15.83:0/3833609001 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fac0054e0 con 0x7f0fb4101c30 2024-03-11T19:38:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0fbc240640 1 -- 172.21.15.83:0/3833609001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb419e1e0 con 0x7f0fb4101c30 2024-03-11T19:38:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0faaffd640 1 -- 172.21.15.83:0/3833609001 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac03e450 con 0x7f0fb4101c30 2024-03-11T19:38:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.426+0000 7f0fbc240640 1 -- 172.21.15.83:0/3833609001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb419e770 con 0x7f0fb4101c30 2024-03-11T19:38:49.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.427+0000 7f0fbc240640 1 -- 172.21.15.83:0/3833609001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb41073a0 con 0x7f0fb4101c30 2024-03-11T19:38:49.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.429+0000 7f0faaffd640 1 -- 172.21.15.83:0/3833609001 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0fac003680 con 0x7f0fb4101c30 2024-03-11T19:38:49.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.429+0000 7f0faaffd640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fa00a1a30 0x7f0fa00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.429+0000 7f0fb9fdc640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fa00a1a30 0x7f0fa00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:49.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.429+0000 7f0faaffd640 1 -- 172.21.15.83:0/3833609001 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0fac0f0010 con 0x7f0fb4101c30 2024-03-11T19:38:49.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.430+0000 7f0fb9fdc640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fa00a1a30 0x7f0fa00a3ef0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0fb41010d0 tx=0x7f0fb00082f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.433+0000 7f0faaffd640 1 -- 172.21.15.83:0/3833609001 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0fac0b92d0 con 0x7f0fb4101c30 2024-03-11T19:38:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.621+0000 7f0fbc240640 1 -- 172.21.15.83:0/3833609001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0fb410bad0 con 0x7f0fb4101c30 2024-03-11T19:38:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.621+0000 7f0faaffd640 1 -- 172.21.15.83:0/3833609001 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0fac0bc990 con 0x7f0fb4101c30 2024-03-11T19:38:49.625 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:38:49.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 -- 172.21.15.83:0/3833609001 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fa00a1a30 msgr2=0x7f0fa00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fa00a1a30 0x7f0fa00a3ef0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0fb41010d0 tx=0x7f0fb00082f0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 -- 172.21.15.83:0/3833609001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fb4101c30 msgr2=0x7f0fb419d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fb4101c30 0x7f0fb419d6f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009230 tx=0x7f0fac0042f0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 -- 172.21.15.83:0/3833609001 shutdown_connections 2024-03-11T19:38:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fa00a1a30 0x7f0fa00a3ef0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fb4101c30 0x7f0fb419d6f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fb4101270 0x7f0fb4196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 --2- 172.21.15.83:0/3833609001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb4100080 0x7f0fb4196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 -- 172.21.15.83:0/3833609001 >> 172.21.15.83:0/3833609001 conn(0x7f0fb40fb610 msgr2=0x7f0fb4104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 -- 172.21.15.83:0/3833609001 shutdown_connections 2024-03-11T19:38:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.623+0000 7f0fbc240640 1 -- 172.21.15.83:0/3833609001 wait complete. 2024-03-11T19:38:49.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:38:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.738+0000 7fe98773a640 1 -- 172.21.15.83:0/3125275172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe980102e10 msgr2=0x7fe9801096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.738+0000 7fe98773a640 1 --2- 172.21.15.83:0/3125275172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe980102e10 0x7fe9801096a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe978009ca0 tx=0x7fe97802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.738+0000 7fe98773a640 1 -- 172.21.15.83:0/3125275172 shutdown_connections 2024-03-11T19:38:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.738+0000 7fe98773a640 1 --2- 172.21.15.83:0/3125275172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe980102e10 0x7fe9801096a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.738+0000 7fe98773a640 1 --2- 172.21.15.83:0/3125275172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe980102450 0x7fe9801028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.738+0000 7fe98773a640 1 --2- 172.21.15.83:0/3125275172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9800ff480 0x7fe9800ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.738+0000 7fe98773a640 1 -- 172.21.15.83:0/3125275172 >> 172.21.15.83:0/3125275172 conn(0x7fe9800fadb0 msgr2=0x7fe9800fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.738+0000 7fe98773a640 1 -- 172.21.15.83:0/3125275172 shutdown_connections 2024-03-11T19:38:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.738+0000 7fe98773a640 1 -- 172.21.15.83:0/3125275172 wait complete. 2024-03-11T19:38:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe98773a640 1 Processor -- start 2024-03-11T19:38:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe98773a640 1 -- start start 2024-03-11T19:38:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe98773a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9800ff480 0x7fe9801971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe98773a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe980102450 0x7fe980197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe98773a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe980102e10 0x7fe98019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe98773a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98019ece0 con 0x7fe980102e10 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe98773a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98019ee50 con 0x7fe980102450 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe98773a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98019f010 con 0x7fe9800ff480 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe985cd7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe980102e10 0x7fe98019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe985cd7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe980102e10 0x7fe98019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56214/0 (socket says 172.21.15.83:56214) 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe985cd7640 1 -- 172.21.15.83:0/3669455519 learned_addr learned my addr 172.21.15.83:0/3669455519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe985cd7640 1 -- 172.21.15.83:0/3669455519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9800ff480 msgr2=0x7fe9801971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe985cd7640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9800ff480 0x7fe9801971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe985cd7640 1 -- 172.21.15.83:0/3669455519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe980102450 msgr2=0x7fe980197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe984cd5640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe980102450 0x7fe980197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe985cd7640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe980102450 0x7fe980197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.739+0000 7fe985cd7640 1 -- 172.21.15.83:0/3669455519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe978009900 con 0x7fe980102e10 2024-03-11T19:38:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.740+0000 7fe985cd7640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe980102e10 0x7fe98019e7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe978037040 tx=0x7fe9780346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.740+0000 7fe9767fc640 1 -- 172.21.15.83:0/3669455519 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978032e70 con 0x7fe980102e10 2024-03-11T19:38:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.740+0000 7fe9767fc640 1 -- 172.21.15.83:0/3669455519 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe978003e50 con 0x7fe980102e10 2024-03-11T19:38:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.740+0000 7fe98773a640 1 -- 172.21.15.83:0/3669455519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98019f290 con 0x7fe980102e10 2024-03-11T19:38:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.740+0000 7fe9767fc640 1 -- 172.21.15.83:0/3669455519 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978040450 con 0x7fe980102e10 2024-03-11T19:38:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.740+0000 7fe98773a640 1 -- 172.21.15.83:0/3669455519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98019f740 con 0x7fe980102e10 2024-03-11T19:38:49.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.741+0000 7fe98773a640 1 -- 172.21.15.83:0/3669455519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe980108580 con 0x7fe980102e10 2024-03-11T19:38:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.742+0000 7fe9767fc640 1 -- 172.21.15.83:0/3669455519 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe97800a040 con 0x7fe980102e10 2024-03-11T19:38:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.743+0000 7fe9767fc640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe96c0a1a30 0x7fe96c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.743+0000 7fe9854d6640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe96c0a1a30 0x7fe96c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.743+0000 7fe9767fc640 1 -- 172.21.15.83:0/3669455519 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe9780f07e0 con 0x7fe980102e10 2024-03-11T19:38:49.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.743+0000 7fe9854d6640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe96c0a1a30 0x7fe96c0a3ef0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe97c005fd0 tx=0x7fe97c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:49.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.747+0000 7fe9767fc640 1 -- 172.21.15.83:0/3669455519 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe9780b9bc0 con 0x7fe980102e10 2024-03-11T19:38:49.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:49 smithi137 ceph-mon[36220]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-11T19:38:49.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:49 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/197290162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:49 smithi204 ceph-mon[36739]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-11T19:38:49.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:49 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/197290162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.921+0000 7fe98773a640 1 -- 172.21.15.83:0/3669455519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe98010cbe0 con 0x7fe980102e10 2024-03-11T19:38:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.921+0000 7fe9767fc640 1 -- 172.21.15.83:0/3669455519 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe978039020 con 0x7fe980102e10 2024-03-11T19:38:49.925 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-03-11T19:38:49.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 -- 172.21.15.83:0/3669455519 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe96c0a1a30 msgr2=0x7fe96c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe96c0a1a30 0x7fe96c0a3ef0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe97c005fd0 tx=0x7fe97c00b410 comp rx=0 tx=0).stop 2024-03-11T19:38:49.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 -- 172.21.15.83:0/3669455519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe980102e10 msgr2=0x7fe98019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:49.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe980102e10 0x7fe98019e7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe978037040 tx=0x7fe9780346f0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 -- 172.21.15.83:0/3669455519 shutdown_connections 2024-03-11T19:38:49.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe96c0a1a30 0x7fe96c0a3ef0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe980102e10 0x7fe98019e7a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe980102450 0x7fe980197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 --2- 172.21.15.83:0/3669455519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9800ff480 0x7fe9801971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 -- 172.21.15.83:0/3669455519 >> 172.21.15.83:0/3669455519 conn(0x7fe9800fadb0 msgr2=0x7fe980105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 -- 172.21.15.83:0/3669455519 shutdown_connections 2024-03-11T19:38:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:49.923+0000 7fe98773a640 1 -- 172.21.15.83:0/3669455519 wait complete. 2024-03-11T19:38:49.937 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:49 smithi083 ceph-mon[31864]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-11T19:38:49.937 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:49 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/197290162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:49.938 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:38:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.038+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/1961816592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0c4102de0 msgr2=0x7fc0c4109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.038+0000 7fc0c8f6b640 1 --2- 172.21.15.83:0/1961816592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0c4102de0 0x7fc0c4109670 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009ca0 tx=0x7fc0bc02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.038+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/1961816592 shutdown_connections 2024-03-11T19:38:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.038+0000 7fc0c8f6b640 1 --2- 172.21.15.83:0/1961816592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0c4102de0 0x7fc0c4109670 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.038+0000 7fc0c8f6b640 1 --2- 172.21.15.83:0/1961816592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4102420 0x7fc0c41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.038+0000 7fc0c8f6b640 1 --2- 172.21.15.83:0/1961816592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c40ff450 0x7fc0c40ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.038+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/1961816592 >> 172.21.15.83:0/1961816592 conn(0x7fc0c40fada0 msgr2=0x7fc0c40fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.038+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/1961816592 shutdown_connections 2024-03-11T19:38:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.038+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/1961816592 wait complete. 2024-03-11T19:38:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0c8f6b640 1 Processor -- start 2024-03-11T19:38:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0c8f6b640 1 -- start start 2024-03-11T19:38:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0c8f6b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c40ff450 0x7fc0c4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0c8f6b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0c4102420 0x7fc0c4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0c8f6b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4102de0 0x7fc0c419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0c8f6b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c419ed40 con 0x7fc0c4102420 2024-03-11T19:38:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0c8f6b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c419eeb0 con 0x7fc0c4102de0 2024-03-11T19:38:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0c8f6b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c419f070 con 0x7fc0c40ff450 2024-03-11T19:38:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0b9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0c4102420 0x7fc0c4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0b9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0c4102420 0x7fc0c4197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56224/0 (socket says 172.21.15.83:56224) 2024-03-11T19:38:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0b9d9b640 1 -- 172.21.15.83:0/3761623017 learned_addr learned my addr 172.21.15.83:0/3761623017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0c2d9d640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4102de0 0x7fc0c419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0c259c640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c40ff450 0x7fc0c4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0b9d9b640 1 -- 172.21.15.83:0/3761623017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c40ff450 msgr2=0x7fc0c4197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0b9d9b640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c40ff450 0x7fc0c4197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0b9d9b640 1 -- 172.21.15.83:0/3761623017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4102de0 msgr2=0x7fc0c419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0b9d9b640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4102de0 0x7fc0c419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.039+0000 7fc0b9d9b640 1 -- 172.21.15.83:0/3761623017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc009900 con 0x7fc0c4102420 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.040+0000 7fc0b9d9b640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0c4102420 0x7fc0c4197790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00b4a0 tx=0x7fc0ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.040+0000 7fc0bbfff640 1 -- 172.21.15.83:0/3761623017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac010610 con 0x7fc0c4102420 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.040+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/3761623017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c419f350 con 0x7fc0c4102420 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.040+0000 7fc0bbfff640 1 -- 172.21.15.83:0/3761623017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0ac010c50 con 0x7fc0c4102420 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.040+0000 7fc0bbfff640 1 -- 172.21.15.83:0/3761623017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac017e20 con 0x7fc0c4102420 2024-03-11T19:38:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.040+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/3761623017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c419f860 con 0x7fc0c4102420 2024-03-11T19:38:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.041+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/3761623017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c4108550 con 0x7fc0c4102420 2024-03-11T19:38:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.042+0000 7fc0bbfff640 1 -- 172.21.15.83:0/3761623017 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc0ac017480 con 0x7fc0c4102420 2024-03-11T19:38:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.042+0000 7fc0bbfff640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc0b00a1a30 0x7fc0b00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.042+0000 7fc0c259c640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc0b00a1a30 0x7fc0b00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.042+0000 7fc0bbfff640 1 -- 172.21.15.83:0/3761623017 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc0ac0cc8f0 con 0x7fc0c4102420 2024-03-11T19:38:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.043+0000 7fc0c259c640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc0b00a1a30 0x7fc0b00a3ef0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40058c0 tx=0x7fc0b400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:50.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.047+0000 7fc0bbfff640 1 -- 172.21.15.83:0/3761623017 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc0ac095c60 con 0x7fc0c4102420 2024-03-11T19:38:50.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.219+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/3761623017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0c410cdc0 con 0x7fc0c4102420 2024-03-11T19:38:50.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.220+0000 7fc0bbfff640 1 -- 172.21.15.83:0/3761623017 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0ac099320 con 0x7fc0c4102420 2024-03-11T19:38:50.224 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-03-11T19:38:50.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/3761623017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc0b00a1a30 msgr2=0x7fc0b00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc0b00a1a30 0x7fc0b00a3ef0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40058c0 tx=0x7fc0b400a2d0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/3761623017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0c4102420 msgr2=0x7fc0c4197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0c4102420 0x7fc0c4197790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00b4a0 tx=0x7fc0ac00b960 comp rx=0 tx=0).stop 2024-03-11T19:38:50.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/3761623017 shutdown_connections 2024-03-11T19:38:50.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc0b00a1a30 0x7fc0b00a3ef0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4102de0 0x7fc0c419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc0c4102420 0x7fc0c4197790 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 --2- 172.21.15.83:0/3761623017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0c40ff450 0x7fc0c4197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/3761623017 >> 172.21.15.83:0/3761623017 conn(0x7fc0c40fada0 msgr2=0x7fc0c4105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:50.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/3761623017 shutdown_connections 2024-03-11T19:38:50.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.222+0000 7fc0c8f6b640 1 -- 172.21.15.83:0/3761623017 wait complete. 2024-03-11T19:38:50.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:38:50.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.337+0000 7f00ccc81640 1 -- 172.21.15.83:0/2492997431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00c8101c50 msgr2=0x7f00c81084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.337+0000 7f00ccc81640 1 --2- 172.21.15.83:0/2492997431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00c8101c50 0x7f00c81084e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009ca0 tx=0x7f00c002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.337+0000 7f00ccc81640 1 -- 172.21.15.83:0/2492997431 shutdown_connections 2024-03-11T19:38:50.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.337+0000 7f00ccc81640 1 --2- 172.21.15.83:0/2492997431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00c8101c50 0x7f00c81084e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.337+0000 7f00ccc81640 1 --2- 172.21.15.83:0/2492997431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c8101290 0x7f00c8101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.337+0000 7f00ccc81640 1 --2- 172.21.15.83:0/2492997431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00c81000a0 0x7f00c81004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.337+0000 7f00ccc81640 1 -- 172.21.15.83:0/2492997431 >> 172.21.15.83:0/2492997431 conn(0x7f00c80fb610 msgr2=0x7f00c80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:50.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.337+0000 7f00ccc81640 1 -- 172.21.15.83:0/2492997431 shutdown_connections 2024-03-11T19:38:50.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.337+0000 7f00ccc81640 1 -- 172.21.15.83:0/2492997431 wait complete. 2024-03-11T19:38:50.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00ccc81640 1 Processor -- start 2024-03-11T19:38:50.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00ccc81640 1 -- start start 2024-03-11T19:38:50.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00ccc81640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00c81000a0 0x7f00c8199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00ccc81640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c8101290 0x7f00c81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00ccc81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00c8101c50 0x7f00c81a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00ccc81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c81a0f80 con 0x7f00c8101c50 2024-03-11T19:38:50.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00ccc81640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c81a10f0 con 0x7f00c81000a0 2024-03-11T19:38:50.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00ccc81640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c81a12b0 con 0x7f00c8101290 2024-03-11T19:38:50.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00c6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00c8101c50 0x7f00c81a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00c6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00c8101c50 0x7f00c81a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56248/0 (socket says 172.21.15.83:56248) 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00c6d9d640 1 -- 172.21.15.83:0/3802090972 learned_addr learned my addr 172.21.15.83:0/3802090972 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00c5d9b640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c8101290 0x7f00c81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00c659c640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00c81000a0 0x7f00c8199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00c6d9d640 1 -- 172.21.15.83:0/3802090972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c8101290 msgr2=0x7f00c81999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00c6d9d640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c8101290 0x7f00c81999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00c6d9d640 1 -- 172.21.15.83:0/3802090972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00c81000a0 msgr2=0x7f00c8199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00c6d9d640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00c81000a0 0x7f00c8199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00c6d9d640 1 -- 172.21.15.83:0/3802090972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c0009900 con 0x7f00c8101c50 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00c6d9d640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00c8101c50 0x7f00c81a0a40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f00c0037040 tx=0x7f00c0034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00b77fe640 1 -- 172.21.15.83:0/3802090972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0032eb0 con 0x7f00c8101c50 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.338+0000 7f00b77fe640 1 -- 172.21.15.83:0/3802090972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00c0003e50 con 0x7f00c8101c50 2024-03-11T19:38:50.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.339+0000 7f00ccc81640 1 -- 172.21.15.83:0/3802090972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c81a1530 con 0x7f00c8101c50 2024-03-11T19:38:50.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.339+0000 7f00b77fe640 1 -- 172.21.15.83:0/3802090972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0040450 con 0x7f00c8101c50 2024-03-11T19:38:50.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.339+0000 7f00ccc81640 1 -- 172.21.15.83:0/3802090972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c81a19e0 con 0x7f00c8101c50 2024-03-11T19:38:50.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.340+0000 7f00ccc81640 1 -- 172.21.15.83:0/3802090972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c810bb30 con 0x7f00c8101c50 2024-03-11T19:38:50.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.341+0000 7f00b77fe640 1 -- 172.21.15.83:0/3802090972 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f00c000a040 con 0x7f00c8101c50 2024-03-11T19:38:50.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.341+0000 7f00b77fe640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00b00a1a30 0x7f00b00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.341+0000 7f00c659c640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00b00a1a30 0x7f00b00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.341+0000 7f00b77fe640 1 -- 172.21.15.83:0/3802090972 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f00c00f0330 con 0x7f00c8101c50 2024-03-11T19:38:50.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.342+0000 7f00c659c640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00b00a1a30 0x7f00b00a3ef0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f00c81010f0 tx=0x7f00b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:50.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.346+0000 7f00b77fe640 1 -- 172.21.15.83:0/3802090972 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00c00b95f0 con 0x7f00c8101c50 2024-03-11T19:38:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.564+0000 7f00ccc81640 1 -- 172.21.15.83:0/3802090972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00c81a1e80 con 0x7f00c8101c50 2024-03-11T19:38:50.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.564+0000 7f00b77fe640 1 -- 172.21.15.83:0/3802090972 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f00c00bccb0 con 0x7f00c8101c50 2024-03-11T19:38:50.569 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:50.569 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:37:09.619596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:38:50.569 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-03-11T19:38:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.566+0000 7f00ccc81640 1 -- 172.21.15.83:0/3802090972 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00b00a1a30 msgr2=0x7f00b00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.566+0000 7f00ccc81640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00b00a1a30 0x7f00b00a3ef0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f00c81010f0 tx=0x7f00b8008040 comp rx=0 tx=0).stop 2024-03-11T19:38:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.566+0000 7f00ccc81640 1 -- 172.21.15.83:0/3802090972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00c8101c50 msgr2=0x7f00c81a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.566+0000 7f00ccc81640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00c8101c50 0x7f00c81a0a40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f00c0037040 tx=0x7f00c0034d40 comp rx=0 tx=0).stop 2024-03-11T19:38:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.567+0000 7f00ccc81640 1 -- 172.21.15.83:0/3802090972 shutdown_connections 2024-03-11T19:38:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.567+0000 7f00ccc81640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00b00a1a30 0x7f00b00a3ef0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.567+0000 7f00ccc81640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00c8101c50 0x7f00c81a0a40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.567+0000 7f00ccc81640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00c8101290 0x7f00c81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.567+0000 7f00ccc81640 1 --2- 172.21.15.83:0/3802090972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00c81000a0 0x7f00c8199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.567+0000 7f00ccc81640 1 -- 172.21.15.83:0/3802090972 >> 172.21.15.83:0/3802090972 conn(0x7f00c80fb610 msgr2=0x7f00c80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:50.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.567+0000 7f00ccc81640 1 -- 172.21.15.83:0/3802090972 shutdown_connections 2024-03-11T19:38:50.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.567+0000 7f00ccc81640 1 -- 172.21.15.83:0/3802090972 wait complete. 2024-03-11T19:38:50.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:38:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.694+0000 7f4f90c2e640 1 -- 172.21.15.83:0/1297355534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f8c106ee0 msgr2=0x7f4f8c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.694+0000 7f4f90c2e640 1 --2- 172.21.15.83:0/1297355534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f8c106ee0 0x7f4f8c1092d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009900 tx=0x7f4f7c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.694+0000 7f4f90c2e640 1 -- 172.21.15.83:0/1297355534 shutdown_connections 2024-03-11T19:38:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.694+0000 7f4f90c2e640 1 --2- 172.21.15.83:0/1297355534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f8c106ee0 0x7f4f8c1092d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.694+0000 7f4f90c2e640 1 --2- 172.21.15.83:0/1297355534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f8c1045b0 0x7f4f8c1069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.694+0000 7f4f90c2e640 1 --2- 172.21.15.83:0/1297355534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f8c101c80 0x7f4f8c104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.694+0000 7f4f90c2e640 1 -- 172.21.15.83:0/1297355534 >> 172.21.15.83:0/1297355534 conn(0x7f4f8c0fb610 msgr2=0x7f4f8c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.694+0000 7f4f90c2e640 1 -- 172.21.15.83:0/1297355534 shutdown_connections 2024-03-11T19:38:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.694+0000 7f4f90c2e640 1 -- 172.21.15.83:0/1297355534 wait complete. 2024-03-11T19:38:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f90c2e640 1 Processor -- start 2024-03-11T19:38:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f90c2e640 1 -- start start 2024-03-11T19:38:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f90c2e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f8c101c80 0x7f4f8c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f90c2e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f8c1045b0 0x7f4f8c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f90c2e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f8c106ee0 0x7f4f8c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f90c2e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c19ed40 con 0x7f4f8c106ee0 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f90c2e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c19eeb0 con 0x7f4f8c101c80 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f90c2e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8c19f070 con 0x7f4f8c1045b0 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f8ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f8c106ee0 0x7f4f8c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f8ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f8c106ee0 0x7f4f8c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56266/0 (socket says 172.21.15.83:56266) 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f8ad9d640 1 -- 172.21.15.83:0/3631620068 learned_addr learned my addr 172.21.15.83:0/3631620068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.695+0000 7f4f89d9b640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f8c1045b0 0x7f4f8c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f8a59c640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f8c101c80 0x7f4f8c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f8ad9d640 1 -- 172.21.15.83:0/3631620068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f8c1045b0 msgr2=0x7f4f8c197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f8ad9d640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f8c1045b0 0x7f4f8c197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f8ad9d640 1 -- 172.21.15.83:0/3631620068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f8c101c80 msgr2=0x7f4f8c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f8ad9d640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f8c101c80 0x7f4f8c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f8ad9d640 1 -- 172.21.15.83:0/3631620068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c009560 con 0x7f4f8c106ee0 2024-03-11T19:38:50.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f8a59c640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f8c101c80 0x7f4f8c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f89d9b640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f8c1045b0 0x7f4f8c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f8ad9d640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f8c106ee0 0x7f4f8c19e800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009230 tx=0x7f4f7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f7b7fe640 1 -- 172.21.15.83:0/3631620068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c004df0 con 0x7f4f8c106ee0 2024-03-11T19:38:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f7b7fe640 1 -- 172.21.15.83:0/3631620068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f7c005430 con 0x7f4f8c106ee0 2024-03-11T19:38:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f90c2e640 1 -- 172.21.15.83:0/3631620068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c19f2f0 con 0x7f4f8c106ee0 2024-03-11T19:38:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f7b7fe640 1 -- 172.21.15.83:0/3631620068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c003de0 con 0x7f4f8c106ee0 2024-03-11T19:38:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.696+0000 7f4f90c2e640 1 -- 172.21.15.83:0/3631620068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c19f880 con 0x7f4f8c106ee0 2024-03-11T19:38:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.697+0000 7f4f90c2e640 1 -- 172.21.15.83:0/3631620068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8c0fd100 con 0x7f4f8c106ee0 2024-03-11T19:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.698+0000 7f4f7b7fe640 1 -- 172.21.15.83:0/3631620068 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4f7c0055a0 con 0x7f4f8c106ee0 2024-03-11T19:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.699+0000 7f4f7b7fe640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f740a1a30 0x7f4f740a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.699+0000 7f4f8a59c640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f740a1a30 0x7f4f740a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.699+0000 7f4f7b7fe640 1 -- 172.21.15.83:0/3631620068 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4f7c0f0110 con 0x7f4f8c106ee0 2024-03-11T19:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.699+0000 7f4f8a59c640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f740a1a30 0x7f4f740a3ef0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f84005fd0 tx=0x7f4f84009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.702+0000 7f4f7b7fe640 1 -- 172.21.15.83:0/3631620068 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f7c0b93d0 con 0x7f4f8c106ee0 2024-03-11T19:38:50.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:50 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3833609001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:50.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:50 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3669455519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:50.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:50 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3761623017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:50.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:50 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3802090972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:50.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.885+0000 7f4f90c2e640 1 -- 172.21.15.83:0/3631620068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f8c02ce40 con 0x7f4f8c106ee0 2024-03-11T19:38:50.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:50 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3833609001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:50.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:50 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3669455519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:50.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:50 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3761623017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:50.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:50 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3802090972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:50.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.886+0000 7f4f7b7fe640 1 -- 172.21.15.83:0/3631620068 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7f4f7c0bca90 con 0x7f4f8c106ee0 2024-03-11T19:38:50.890 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:50.891 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:38:50.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.888+0000 7f4f90c2e640 1 -- 172.21.15.83:0/3631620068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f740a1a30 msgr2=0x7f4f740a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.888+0000 7f4f90c2e640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f740a1a30 0x7f4f740a3ef0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f84005fd0 tx=0x7f4f84009520 comp rx=0 tx=0).stop 2024-03-11T19:38:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.888+0000 7f4f90c2e640 1 -- 172.21.15.83:0/3631620068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f8c106ee0 msgr2=0x7f4f8c19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.888+0000 7f4f90c2e640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f8c106ee0 0x7f4f8c19e800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009230 tx=0x7f4f7c004270 comp rx=0 tx=0).stop 2024-03-11T19:38:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.888+0000 7f4f90c2e640 1 -- 172.21.15.83:0/3631620068 shutdown_connections 2024-03-11T19:38:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.888+0000 7f4f90c2e640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f740a1a30 0x7f4f740a3ef0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.888+0000 7f4f90c2e640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f8c106ee0 0x7f4f8c19e800 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.888+0000 7f4f90c2e640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f8c1045b0 0x7f4f8c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.888+0000 7f4f90c2e640 1 --2- 172.21.15.83:0/3631620068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f8c101c80 0x7f4f8c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.888+0000 7f4f90c2e640 1 -- 172.21.15.83:0/3631620068 >> 172.21.15.83:0/3631620068 conn(0x7f4f8c0fb610 msgr2=0x7f4f8c102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.889+0000 7f4f90c2e640 1 -- 172.21.15.83:0/3631620068 shutdown_connections 2024-03-11T19:38:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:50.889+0000 7f4f90c2e640 1 -- 172.21.15.83:0/3631620068 wait complete. 2024-03-11T19:38:50.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:37:03.034982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:37:04.125352+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:38:50.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:38:50.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:50 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3833609001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:50.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:50 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3669455519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:50.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:50 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3761623017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:50.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:50 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3802090972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.008+0000 7fc1d217a640 1 -- 172.21.15.83:0/3420829942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1cc104590 msgr2=0x7fc1cc106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.008+0000 7fc1d217a640 1 --2- 172.21.15.83:0/3420829942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1cc104590 0x7fc1cc106980 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009900 tx=0x7fc1b402bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.008+0000 7fc1d217a640 1 -- 172.21.15.83:0/3420829942 shutdown_connections 2024-03-11T19:38:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.008+0000 7fc1d217a640 1 --2- 172.21.15.83:0/3420829942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1cc106ec0 0x7fc1cc1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.008+0000 7fc1d217a640 1 --2- 172.21.15.83:0/3420829942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1cc104590 0x7fc1cc106980 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.008+0000 7fc1d217a640 1 --2- 172.21.15.83:0/3420829942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1cc101c60 0x7fc1cc104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.008+0000 7fc1d217a640 1 -- 172.21.15.83:0/3420829942 >> 172.21.15.83:0/3420829942 conn(0x7fc1cc0fb610 msgr2=0x7fc1cc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.008+0000 7fc1d217a640 1 -- 172.21.15.83:0/3420829942 shutdown_connections 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.008+0000 7fc1d217a640 1 -- 172.21.15.83:0/3420829942 wait complete. 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1d217a640 1 Processor -- start 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1d217a640 1 -- start start 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1d217a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1cc101c60 0x7fc1cc197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1d217a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1cc104590 0x7fc1cc1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1d217a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1cc106ec0 0x7fc1cc19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1d217a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc19ec70 con 0x7fc1cc101c60 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1d217a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc19ede0 con 0x7fc1cc104590 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1d217a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc19efa0 con 0x7fc1cc106ec0 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1cc101c60 0x7fc1cc197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1cc101c60 0x7fc1cc197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56276/0 (socket says 172.21.15.83:56276) 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1cb7fe640 1 -- 172.21.15.83:0/994208098 learned_addr learned my addr 172.21.15.83:0/994208098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.009+0000 7fc1cbfff640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1cc106ec0 0x7fc1cc19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1caffd640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1cc104590 0x7fc1cc1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1cb7fe640 1 -- 172.21.15.83:0/994208098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1cc106ec0 msgr2=0x7fc1cc19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1cb7fe640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1cc106ec0 0x7fc1cc19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1cb7fe640 1 -- 172.21.15.83:0/994208098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1cc104590 msgr2=0x7fc1cc1976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1cb7fe640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1cc104590 0x7fc1cc1976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1cb7fe640 1 -- 172.21.15.83:0/994208098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4009560 con 0x7fc1cc101c60 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1caffd640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1cc104590 0x7fc1cc1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1cb7fe640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1cc101c60 0x7fc1cc197180 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400cce0 tx=0x7fc1c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1c8ff9640 1 -- 172.21.15.83:0/994208098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c400f650 con 0x7fc1cc101c60 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1c8ff9640 1 -- 172.21.15.83:0/994208098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1c400ce90 con 0x7fc1cc101c60 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1d217a640 1 -- 172.21.15.83:0/994208098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc19f280 con 0x7fc1cc101c60 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1c8ff9640 1 -- 172.21.15.83:0/994208098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4018bf0 con 0x7fc1cc101c60 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.010+0000 7fc1d217a640 1 -- 172.21.15.83:0/994208098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc19f7e0 con 0x7fc1cc101c60 2024-03-11T19:38:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.011+0000 7fc1d217a640 1 -- 172.21.15.83:0/994208098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc0fd100 con 0x7fc1cc101c60 2024-03-11T19:38:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.012+0000 7fc1c8ff9640 1 -- 172.21.15.83:0/994208098 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc1c4018d50 con 0x7fc1cc101c60 2024-03-11T19:38:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.012+0000 7fc1c8ff9640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1b80a1ab0 0x7fc1b80a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.013+0000 7fc1caffd640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1b80a1ab0 0x7fc1b80a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.013+0000 7fc1c8ff9640 1 -- 172.21.15.83:0/994208098 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc1c40cd7e0 con 0x7fc1cc101c60 2024-03-11T19:38:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.013+0000 7fc1caffd640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1b80a1ab0 0x7fc1b80a3f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4036040 tx=0x7fc1b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:51.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.017+0000 7fc1c8ff9640 1 -- 172.21.15.83:0/994208098 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc1c4096aa0 con 0x7fc1cc101c60 2024-03-11T19:38:51.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.197+0000 7fc1d217a640 1 -- 172.21.15.83:0/994208098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1cc10cb90 con 0x7fc1cc101c60 2024-03-11T19:38:51.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.197+0000 7fc1c8ff9640 1 -- 172.21.15.83:0/994208098 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1cc10cb90 con 0x7fc1cc101c60 2024-03-11T19:38:51.201 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:38:51.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 -- 172.21.15.83:0/994208098 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1b80a1ab0 msgr2=0x7fc1b80a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1b80a1ab0 0x7fc1b80a3f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4036040 tx=0x7fc1b4000bc0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 -- 172.21.15.83:0/994208098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1cc101c60 msgr2=0x7fc1cc197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1cc101c60 0x7fc1cc197180 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400cce0 tx=0x7fc1c400a620 comp rx=0 tx=0).stop 2024-03-11T19:38:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 -- 172.21.15.83:0/994208098 shutdown_connections 2024-03-11T19:38:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc1b80a1ab0 0x7fc1b80a3f70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1cc106ec0 0x7fc1cc19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1cc104590 0x7fc1cc1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 --2- 172.21.15.83:0/994208098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1cc101c60 0x7fc1cc197180 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 -- 172.21.15.83:0/994208098 >> 172.21.15.83:0/994208098 conn(0x7fc1cc0fb610 msgr2=0x7fc1cc102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 -- 172.21.15.83:0/994208098 shutdown_connections 2024-03-11T19:38:51.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.199+0000 7fc1d217a640 1 -- 172.21.15.83:0/994208098 wait complete. 2024-03-11T19:38:51.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:38:51.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.315+0000 7f429b455640 1 -- 172.21.15.83:0/124556219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294101c00 msgr2=0x7f4294103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.315+0000 7f429b455640 1 --2- 172.21.15.83:0/124556219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294101c00 0x7f4294103ff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4290009900 tx=0x7f429002bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:51.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.315+0000 7f429b455640 1 -- 172.21.15.83:0/124556219 shutdown_connections 2024-03-11T19:38:51.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.315+0000 7f429b455640 1 --2- 172.21.15.83:0/124556219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106e60 0x7f42941092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.315+0000 7f429b455640 1 --2- 172.21.15.83:0/124556219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4294104530 0x7f4294106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.315+0000 7f429b455640 1 --2- 172.21.15.83:0/124556219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294101c00 0x7f4294103ff0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.315+0000 7f429b455640 1 -- 172.21.15.83:0/124556219 >> 172.21.15.83:0/124556219 conn(0x7f42940fb610 msgr2=0x7f42940fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.315+0000 7f429b455640 1 -- 172.21.15.83:0/124556219 shutdown_connections 2024-03-11T19:38:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.315+0000 7f429b455640 1 -- 172.21.15.83:0/124556219 wait complete. 2024-03-11T19:38:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.316+0000 7f429b455640 1 Processor -- start 2024-03-11T19:38:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.316+0000 7f429b455640 1 -- start start 2024-03-11T19:38:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.316+0000 7f429b455640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4294101c00 0x7f4294197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.316+0000 7f429b455640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294104530 0x7f42941977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.316+0000 7f429b455640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106e60 0x7f429419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.316+0000 7f429b455640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429419ed60 con 0x7f4294104530 2024-03-11T19:38:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.316+0000 7f429b455640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429419eed0 con 0x7f4294106e60 2024-03-11T19:38:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.316+0000 7f429b455640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429419f090 con 0x7f4294101c00 2024-03-11T19:38:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.316+0000 7f42989f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294104530 0x7f42941977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.316+0000 7f42989f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294104530 0x7f42941977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56294/0 (socket says 172.21.15.83:56294) 2024-03-11T19:38:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.316+0000 7f42989f0640 1 -- 172.21.15.83:0/881360813 learned_addr learned my addr 172.21.15.83:0/881360813 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f42999f2640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106e60 0x7f429419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f42991f1640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4294101c00 0x7f4294197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f42989f0640 1 -- 172.21.15.83:0/881360813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4294101c00 msgr2=0x7f4294197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f42989f0640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4294101c00 0x7f4294197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f42989f0640 1 -- 172.21.15.83:0/881360813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106e60 msgr2=0x7f429419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f42989f0640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106e60 0x7f429419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f42989f0640 1 -- 172.21.15.83:0/881360813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4290009560 con 0x7f4294104530 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f42991f1640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4294101c00 0x7f4294197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f42999f2640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106e60 0x7f429419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f42989f0640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294104530 0x7f42941977b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f428400d770 tx=0x7f428400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f428a7fc640 1 -- 172.21.15.83:0/881360813 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4284009c30 con 0x7f4294104530 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f428a7fc640 1 -- 172.21.15.83:0/881360813 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f428400de80 con 0x7f4294104530 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f429b455640 1 -- 172.21.15.83:0/881360813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429419f370 con 0x7f4294104530 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f428a7fc640 1 -- 172.21.15.83:0/881360813 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428400f6b0 con 0x7f4294104530 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.317+0000 7f429b455640 1 -- 172.21.15.83:0/881360813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429419f880 con 0x7f4294104530 2024-03-11T19:38:51.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.318+0000 7f429b455640 1 -- 172.21.15.83:0/881360813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42940fd100 con 0x7f4294104530 2024-03-11T19:38:51.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.319+0000 7f428a7fc640 1 -- 172.21.15.83:0/881360813 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4284010460 con 0x7f4294104530 2024-03-11T19:38:51.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.320+0000 7f428a7fc640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42800a1a30 0x7f42800a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.320+0000 7f42991f1640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42800a1a30 0x7f42800a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.320+0000 7f428a7fc640 1 -- 172.21.15.83:0/881360813 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f42840cd1c0 con 0x7f4294104530 2024-03-11T19:38:51.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.320+0000 7f42991f1640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42800a1a30 0x7f42800a3ef0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4290036040 tx=0x7f4290000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:51.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.324+0000 7f428a7fc640 1 -- 172.21.15.83:0/881360813 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4284096460 con 0x7f4294104530 2024-03-11T19:38:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.513+0000 7f429b455640 1 -- 172.21.15.83:0/881360813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f429402cda0 con 0x7f4294104530 2024-03-11T19:38:51.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.514+0000 7f428a7fc640 1 -- 172.21.15.83:0/881360813 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4284099b20 con 0x7f4294104530 2024-03-11T19:38:51.518 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-03-11T19:38:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 -- 172.21.15.83:0/881360813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42800a1a30 msgr2=0x7f42800a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42800a1a30 0x7f42800a3ef0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4290036040 tx=0x7f4290000bc0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 -- 172.21.15.83:0/881360813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294104530 msgr2=0x7f42941977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294104530 0x7f42941977b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f428400d770 tx=0x7f428400dc30 comp rx=0 tx=0).stop 2024-03-11T19:38:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 -- 172.21.15.83:0/881360813 shutdown_connections 2024-03-11T19:38:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42800a1a30 0x7f42800a3ef0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4294106e60 0x7f429419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294104530 0x7f42941977b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 --2- 172.21.15.83:0/881360813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4294101c00 0x7f4294197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 -- 172.21.15.83:0/881360813 >> 172.21.15.83:0/881360813 conn(0x7f42940fb610 msgr2=0x7f42941026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 -- 172.21.15.83:0/881360813 shutdown_connections 2024-03-11T19:38:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.516+0000 7f429b455640 1 -- 172.21.15.83:0/881360813 wait complete. 2024-03-11T19:38:51.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:38:51.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.632+0000 7f4bab464640 1 -- 172.21.15.83:0/3659855373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ba4100330 msgr2=0x7f4ba41007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.632+0000 7f4bab464640 1 --2- 172.21.15.83:0/3659855373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ba4100330 0x7f4ba41007b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4b94009900 tx=0x7f4b9402bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:51.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 -- 172.21.15.83:0/3659855373 shutdown_connections 2024-03-11T19:38:51.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 --2- 172.21.15.83:0/3659855373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ba4100cf0 0x7f4ba4108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 --2- 172.21.15.83:0/3659855373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ba4100330 0x7f4ba41007b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 --2- 172.21.15.83:0/3659855373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba4101690 0x7f4ba4101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 -- 172.21.15.83:0/3659855373 >> 172.21.15.83:0/3659855373 conn(0x7f4ba40fad80 msgr2=0x7f4ba40fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 -- 172.21.15.83:0/3659855373 shutdown_connections 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 -- 172.21.15.83:0/3659855373 wait complete. 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 Processor -- start 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 -- start start 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ba4100330 0x7f4ba4196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba4100cf0 0x7f4ba4196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ba4101690 0x7f4ba419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba419dc40 con 0x7f4ba4100330 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba419ddb0 con 0x7f4ba4101690 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4bab464640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba419df70 con 0x7f4ba4100cf0 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4ba9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ba4100330 0x7f4ba4196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4ba9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ba4100330 0x7f4ba4196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56316/0 (socket says 172.21.15.83:56316) 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.633+0000 7f4ba9200640 1 -- 172.21.15.83:0/29315857 learned_addr learned my addr 172.21.15.83:0/29315857 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4ba9a01640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ba4101690 0x7f4ba419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4ba89ff640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba4100cf0 0x7f4ba4196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4ba9200640 1 -- 172.21.15.83:0/29315857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba4100cf0 msgr2=0x7f4ba4196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4ba9200640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba4100cf0 0x7f4ba4196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4ba9200640 1 -- 172.21.15.83:0/29315857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ba4101690 msgr2=0x7f4ba419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4ba9200640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ba4101690 0x7f4ba419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4ba9200640 1 -- 172.21.15.83:0/29315857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b94009560 con 0x7f4ba4100330 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4ba9200640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ba4100330 0x7f4ba4196150 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000d950 tx=0x7f4ba000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4b9a7fc640 1 -- 172.21.15.83:0/29315857 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0009c30 con 0x7f4ba4100330 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4b9a7fc640 1 -- 172.21.15.83:0/29315857 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ba0009d90 con 0x7f4ba4100330 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4b9a7fc640 1 -- 172.21.15.83:0/29315857 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000f720 con 0x7f4ba4100330 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4bab464640 1 -- 172.21.15.83:0/29315857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba419e250 con 0x7f4ba4100330 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.634+0000 7f4bab464640 1 -- 172.21.15.83:0/29315857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba419e710 con 0x7f4ba4100330 2024-03-11T19:38:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.635+0000 7f4bab464640 1 -- 172.21.15.83:0/29315857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba4107470 con 0x7f4ba4100330 2024-03-11T19:38:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.636+0000 7f4b9a7fc640 1 -- 172.21.15.83:0/29315857 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4ba0010a60 con 0x7f4ba4100330 2024-03-11T19:38:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.636+0000 7f4b9a7fc640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b900a1a30 0x7f4b900a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.637+0000 7f4ba89ff640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b900a1a30 0x7f4b900a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.637+0000 7f4b9a7fc640 1 -- 172.21.15.83:0/29315857 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4ba00cd1b0 con 0x7f4ba4100330 2024-03-11T19:38:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.637+0000 7f4ba89ff640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b900a1a30 0x7f4b900a3ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b9402c6a0 tx=0x7f4b94005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.641+0000 7f4b9a7fc640 1 -- 172.21.15.83:0/29315857 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ba0096470 con 0x7f4ba4100330 2024-03-11T19:38:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.813+0000 7f4bab464640 1 -- 172.21.15.83:0/29315857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ba402cda0 con 0x7f4ba4100330 2024-03-11T19:38:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.814+0000 7f4b9a7fc640 1 -- 172.21.15.83:0/29315857 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ba0099b30 con 0x7f4ba4100330 2024-03-11T19:38:51.818 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-03-11T19:38:51.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:51 smithi083 ceph-mon[31864]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-11T19:38:51.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:51 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3631620068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:51.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:51 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/994208098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:51.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:51 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/881360813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 -- 172.21.15.83:0/29315857 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b900a1a30 msgr2=0x7f4b900a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b900a1a30 0x7f4b900a3ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b9402c6a0 tx=0x7f4b94005fd0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 -- 172.21.15.83:0/29315857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ba4100330 msgr2=0x7f4ba4196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ba4100330 0x7f4ba4196150 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000d950 tx=0x7f4ba000de10 comp rx=0 tx=0).stop 2024-03-11T19:38:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 -- 172.21.15.83:0/29315857 shutdown_connections 2024-03-11T19:38:51.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4b900a1a30 0x7f4b900a3ef0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ba4101690 0x7f4ba419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ba4100cf0 0x7f4ba4196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 --2- 172.21.15.83:0/29315857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ba4100330 0x7f4ba4196150 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 -- 172.21.15.83:0/29315857 >> 172.21.15.83:0/29315857 conn(0x7f4ba40fad80 msgr2=0x7f4ba4104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:51.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 -- 172.21.15.83:0/29315857 shutdown_connections 2024-03-11T19:38:51.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.816+0000 7f4bab464640 1 -- 172.21.15.83:0/29315857 wait complete. 2024-03-11T19:38:51.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:38:51.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:51 smithi137 ceph-mon[36220]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-11T19:38:51.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:51 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3631620068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:51.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:51 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/994208098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:51.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:51 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/881360813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:51.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:51 smithi204 ceph-mon[36739]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-11T19:38:51.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:51 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3631620068' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:51.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:51 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/994208098' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:51.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:51 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/881360813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:51.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.928+0000 7fe438ac3640 1 -- 172.21.15.83:0/2715687746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe434106f00 msgr2=0x7fe434109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.928+0000 7fe438ac3640 1 --2- 172.21.15.83:0/2715687746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe434106f00 0x7fe434109350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009ca0 tx=0x7fe42c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.928+0000 7fe438ac3640 1 -- 172.21.15.83:0/2715687746 shutdown_connections 2024-03-11T19:38:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.928+0000 7fe438ac3640 1 --2- 172.21.15.83:0/2715687746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe434106f00 0x7fe434109350 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.928+0000 7fe438ac3640 1 --2- 172.21.15.83:0/2715687746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4341045d0 0x7fe4341069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.928+0000 7fe438ac3640 1 --2- 172.21.15.83:0/2715687746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe434101ca0 0x7fe434104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.928+0000 7fe438ac3640 1 -- 172.21.15.83:0/2715687746 >> 172.21.15.83:0/2715687746 conn(0x7fe4340fb6b0 msgr2=0x7fe4340fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.928+0000 7fe438ac3640 1 -- 172.21.15.83:0/2715687746 shutdown_connections 2024-03-11T19:38:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.928+0000 7fe438ac3640 1 -- 172.21.15.83:0/2715687746 wait complete. 2024-03-11T19:38:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.928+0000 7fe438ac3640 1 Processor -- start 2024-03-11T19:38:51.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.928+0000 7fe438ac3640 1 -- start start 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe438ac3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe434101ca0 0x7fe4341971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe438ac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4341045d0 0x7fe434197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe438ac3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe434106f00 0x7fe43419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe438ac3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43419ecd0 con 0x7fe434101ca0 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe438ac3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43419ee40 con 0x7fe4341045d0 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe438ac3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43419f000 con 0x7fe434106f00 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe43259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe434101ca0 0x7fe4341971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe43259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe434101ca0 0x7fe4341971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56328/0 (socket says 172.21.15.83:56328) 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe43259c640 1 -- 172.21.15.83:0/818470538 learned_addr learned my addr 172.21.15.83:0/818470538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe43259c640 1 -- 172.21.15.83:0/818470538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe434106f00 msgr2=0x7fe43419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe432d9d640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe434106f00 0x7fe43419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe43259c640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe434106f00 0x7fe43419e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe43259c640 1 -- 172.21.15.83:0/818470538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4341045d0 msgr2=0x7fe434197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe43259c640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4341045d0 0x7fe434197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe43259c640 1 -- 172.21.15.83:0/818470538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c009900 con 0x7fe434101ca0 2024-03-11T19:38:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe43259c640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe434101ca0 0x7fe4341971e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe42800d770 tx=0x7fe42800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe4237fe640 1 -- 172.21.15.83:0/818470538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe428009c30 con 0x7fe434101ca0 2024-03-11T19:38:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe4237fe640 1 -- 172.21.15.83:0/818470538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe42800de80 con 0x7fe434101ca0 2024-03-11T19:38:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe438ac3640 1 -- 172.21.15.83:0/818470538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43419f2e0 con 0x7fe434101ca0 2024-03-11T19:38:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe4237fe640 1 -- 172.21.15.83:0/818470538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe428004e80 con 0x7fe434101ca0 2024-03-11T19:38:51.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.929+0000 7fe438ac3640 1 -- 172.21.15.83:0/818470538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43419f7f0 con 0x7fe434101ca0 2024-03-11T19:38:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.930+0000 7fe438ac3640 1 -- 172.21.15.83:0/818470538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4340fd1a0 con 0x7fe434101ca0 2024-03-11T19:38:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.931+0000 7fe4237fe640 1 -- 172.21.15.83:0/818470538 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe428005020 con 0x7fe434101ca0 2024-03-11T19:38:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.932+0000 7fe4237fe640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe41c0a1ab0 0x7fe41c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.932+0000 7fe431d9b640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe41c0a1ab0 0x7fe41c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.932+0000 7fe4237fe640 1 -- 172.21.15.83:0/818470538 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe4280cce00 con 0x7fe434101ca0 2024-03-11T19:38:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.932+0000 7fe431d9b640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe41c0a1ab0 0x7fe41c0a3f70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4341986f0 tx=0x7fe424005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:51.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:51.936+0000 7fe4237fe640 1 -- 172.21.15.83:0/818470538 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe4280960c0 con 0x7fe434101ca0 2024-03-11T19:38:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.142+0000 7fe438ac3640 1 -- 172.21.15.83:0/818470538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe43402cdd0 con 0x7fe434101ca0 2024-03-11T19:38:52.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.143+0000 7fe4237fe640 1 -- 172.21.15.83:0/818470538 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11349 (secure 0 0 0) 0x7fe428020090 con 0x7fe434101ca0 2024-03-11T19:38:52.147 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:52.147 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:37:09.619596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:38:52.147 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 21 2024-03-11T19:38:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 -- 172.21.15.83:0/818470538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe41c0a1ab0 msgr2=0x7fe41c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe41c0a1ab0 0x7fe41c0a3f70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4341986f0 tx=0x7fe424005bf0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 -- 172.21.15.83:0/818470538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe434101ca0 msgr2=0x7fe4341971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe434101ca0 0x7fe4341971e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe42800d770 tx=0x7fe42800dc30 comp rx=0 tx=0).stop 2024-03-11T19:38:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 -- 172.21.15.83:0/818470538 shutdown_connections 2024-03-11T19:38:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe41c0a1ab0 0x7fe41c0a3f70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe434106f00 0x7fe43419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4341045d0 0x7fe434197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 --2- 172.21.15.83:0/818470538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe434101ca0 0x7fe4341971e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 -- 172.21.15.83:0/818470538 >> 172.21.15.83:0/818470538 conn(0x7fe4340fb6b0 msgr2=0x7fe434102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 -- 172.21.15.83:0/818470538 shutdown_connections 2024-03-11T19:38:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.145+0000 7fe438ac3640 1 -- 172.21.15.83:0/818470538 wait complete. 2024-03-11T19:38:52.160 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:38:52.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.258+0000 7f45b0a6d640 1 -- 172.21.15.83:0/3599068682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45ac101290 msgr2=0x7f45ac101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.258+0000 7f45b0a6d640 1 --2- 172.21.15.83:0/3599068682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45ac101290 0x7f45ac101710 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4594009900 tx=0x7f459402bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.258+0000 7f45b0a6d640 1 -- 172.21.15.83:0/3599068682 shutdown_connections 2024-03-11T19:38:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.258+0000 7f45b0a6d640 1 --2- 172.21.15.83:0/3599068682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45ac101c50 0x7f45ac1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.258+0000 7f45b0a6d640 1 --2- 172.21.15.83:0/3599068682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45ac101290 0x7f45ac101710 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.258+0000 7f45b0a6d640 1 --2- 172.21.15.83:0/3599068682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ac1000a0 0x7f45ac1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.258+0000 7f45b0a6d640 1 -- 172.21.15.83:0/3599068682 >> 172.21.15.83:0/3599068682 conn(0x7f45ac0fb610 msgr2=0x7f45ac0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.258+0000 7f45b0a6d640 1 -- 172.21.15.83:0/3599068682 shutdown_connections 2024-03-11T19:38:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.258+0000 7f45b0a6d640 1 -- 172.21.15.83:0/3599068682 wait complete. 2024-03-11T19:38:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45b0a6d640 1 Processor -- start 2024-03-11T19:38:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45b0a6d640 1 -- start start 2024-03-11T19:38:52.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45b0a6d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45ac1000a0 0x7f45ac06a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45b0a6d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ac101290 0x7f45ac19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45b0a6d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45ac101c50 0x7f45ac19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45b0a6d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac19ed90 con 0x7f45ac101c50 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45b0a6d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac19ef00 con 0x7f45ac101290 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45b0a6d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac19f0c0 con 0x7f45ac1000a0 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45aad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45ac101c50 0x7f45ac19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45aad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45ac101c50 0x7f45ac19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56350/0 (socket says 172.21.15.83:56350) 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45aad9d640 1 -- 172.21.15.83:0/1568916672 learned_addr learned my addr 172.21.15.83:0/1568916672 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45a3fff640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ac101290 0x7f45ac19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45aad9d640 1 -- 172.21.15.83:0/1568916672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45ac1000a0 msgr2=0x7f45ac06a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45aa59c640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45ac1000a0 0x7f45ac06a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45aad9d640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45ac1000a0 0x7f45ac06a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45aad9d640 1 -- 172.21.15.83:0/1568916672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ac101290 msgr2=0x7f45ac19bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45aad9d640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ac101290 0x7f45ac19bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45aad9d640 1 -- 172.21.15.83:0/1568916672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4594009560 con 0x7f45ac101c50 2024-03-11T19:38:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.259+0000 7f45a3fff640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ac101290 0x7f45ac19bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.260+0000 7f45aad9d640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45ac101c50 0x7f45ac19e5b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f45a400cce0 tx=0x7f45a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.260+0000 7f45a37fe640 1 -- 172.21.15.83:0/1568916672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a400f450 con 0x7f45ac101c50 2024-03-11T19:38:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.260+0000 7f45a37fe640 1 -- 172.21.15.83:0/1568916672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45a400fa90 con 0x7f45ac101c50 2024-03-11T19:38:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.260+0000 7f45b0a6d640 1 -- 172.21.15.83:0/1568916672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac19f340 con 0x7f45ac101c50 2024-03-11T19:38:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.260+0000 7f45a37fe640 1 -- 172.21.15.83:0/1568916672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4018b90 con 0x7f45ac101c50 2024-03-11T19:38:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.260+0000 7f45b0a6d640 1 -- 172.21.15.83:0/1568916672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac19f880 con 0x7f45ac101c50 2024-03-11T19:38:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.261+0000 7f45b0a6d640 1 -- 172.21.15.83:0/1568916672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ac1073c0 con 0x7f45ac101c50 2024-03-11T19:38:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.262+0000 7f45a37fe640 1 -- 172.21.15.83:0/1568916672 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f45a400fc00 con 0x7f45ac101c50 2024-03-11T19:38:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.262+0000 7f45a37fe640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45980a1a30 0x7f45980a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.262+0000 7f45aa59c640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45980a1a30 0x7f45980a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.262+0000 7f45a37fe640 1 -- 172.21.15.83:0/1568916672 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f45a40cdbd0 con 0x7f45ac101c50 2024-03-11T19:38:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.263+0000 7f45aa59c640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45980a1a30 0x7f45980a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45ac1010f0 tx=0x7f459c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.267+0000 7f45a37fe640 1 -- 172.21.15.83:0/1568916672 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f45a4096e90 con 0x7f45ac101c50 2024-03-11T19:38:52.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.441+0000 7f45b0a6d640 1 -- 172.21.15.83:0/1568916672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45ac10bd60 con 0x7f45ac101c50 2024-03-11T19:38:52.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.442+0000 7f45a37fe640 1 -- 172.21.15.83:0/1568916672 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7f45a409a550 con 0x7f45ac101c50 2024-03-11T19:38:52.447 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:52.447 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:38:52.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 -- 172.21.15.83:0/1568916672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45980a1a30 msgr2=0x7f45980a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45980a1a30 0x7f45980a3ef0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f45ac1010f0 tx=0x7f459c008040 comp rx=0 tx=0).stop 2024-03-11T19:38:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 -- 172.21.15.83:0/1568916672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45ac101c50 msgr2=0x7f45ac19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45ac101c50 0x7f45ac19e5b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f45a400cce0 tx=0x7f45a400a620 comp rx=0 tx=0).stop 2024-03-11T19:38:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 -- 172.21.15.83:0/1568916672 shutdown_connections 2024-03-11T19:38:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45980a1a30 0x7f45980a3ef0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45ac101c50 0x7f45ac19e5b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45ac101290 0x7f45ac19bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 --2- 172.21.15.83:0/1568916672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45ac1000a0 0x7f45ac06a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 -- 172.21.15.83:0/1568916672 >> 172.21.15.83:0/1568916672 conn(0x7f45ac0fb610 msgr2=0x7f45ac104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 -- 172.21.15.83:0/1568916672 shutdown_connections 2024-03-11T19:38:52.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.444+0000 7f45b0a6d640 1 -- 172.21.15.83:0/1568916672 wait complete. 2024-03-11T19:38:52.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-11T19:37:04.916864+0000', 'modified': '2024-03-11T19:37:09.619596+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24527, 'mds_1': 24502, 'mds_2': 24494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/2604685946', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 2604685946}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 2604685946}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24470': {'gid': 24470, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/1924725912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 1924725912}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 1924725912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/2654992874', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 2654992874}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 2654992874}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.137:6837/3499358906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 3499358906}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 3499358906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.204:6839/862560872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 862560872}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 862560872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.137:6839/709970923', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 709970923}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 709970923}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-11T19:38:52.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-11T19:38:52.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-11T19:38:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.558+0000 7fd7af039640 1 -- 172.21.15.83:0/262657398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a8100080 msgr2=0x7fd7a81004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 --2- 172.21.15.83:0/262657398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a8100080 0x7fd7a81004a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009900 tx=0x7fd79c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 -- 172.21.15.83:0/262657398 shutdown_connections 2024-03-11T19:38:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 --2- 172.21.15.83:0/262657398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7a8101c30 0x7fd7a81084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 --2- 172.21.15.83:0/262657398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a8101270 0x7fd7a81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 --2- 172.21.15.83:0/262657398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a8100080 0x7fd7a81004a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 -- 172.21.15.83:0/262657398 >> 172.21.15.83:0/262657398 conn(0x7fd7a80fb610 msgr2=0x7fd7a80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 -- 172.21.15.83:0/262657398 shutdown_connections 2024-03-11T19:38:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 -- 172.21.15.83:0/262657398 wait complete. 2024-03-11T19:38:52.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 Processor -- start 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 -- start start 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a8100080 0x7fd7a8197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7a8101270 0x7fd7a81977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a8101c30 0x7fd7a819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a819ed60 con 0x7fd7a8101c30 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a819eed0 con 0x7fd7a8101270 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7af039640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a819f090 con 0x7fd7a8100080 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7ad5d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a8101c30 0x7fd7a819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7ad5d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a8101c30 0x7fd7a819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56370/0 (socket says 172.21.15.83:56370) 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.559+0000 7fd7ad5d6640 1 -- 172.21.15.83:0/615567297 learned_addr learned my addr 172.21.15.83:0/615567297 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a8100080 0x7fd7a8197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7a7fff640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7a8101270 0x7fd7a81977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7ad5d6640 1 -- 172.21.15.83:0/615567297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a8100080 msgr2=0x7fd7a8197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7ad5d6640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a8100080 0x7fd7a8197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7ad5d6640 1 -- 172.21.15.83:0/615567297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7a8101270 msgr2=0x7fd7a81977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7ad5d6640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7a8101270 0x7fd7a81977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7ad5d6640 1 -- 172.21.15.83:0/615567297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c009560 con 0x7fd7a8101c30 2024-03-11T19:38:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a8100080 0x7fd7a8197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7ad5d6640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a8101c30 0x7fd7a819e820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000eaa0 tx=0x7fd7a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7a5ffb640 1 -- 172.21.15.83:0/615567297 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a000cdb0 con 0x7fd7a8101c30 2024-03-11T19:38:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7af039640 1 -- 172.21.15.83:0/615567297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a819f370 con 0x7fd7a8101c30 2024-03-11T19:38:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7a5ffb640 1 -- 172.21.15.83:0/615567297 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7a0005020 con 0x7fd7a8101c30 2024-03-11T19:38:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7a5ffb640 1 -- 172.21.15.83:0/615567297 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a00116b0 con 0x7fd7a8101c30 2024-03-11T19:38:52.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.560+0000 7fd7af039640 1 -- 172.21.15.83:0/615567297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a819f830 con 0x7fd7a8101c30 2024-03-11T19:38:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.562+0000 7fd7a5ffb640 1 -- 172.21.15.83:0/615567297 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd7a0005520 con 0x7fd7a8101c30 2024-03-11T19:38:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7b1c06740 1 Processor -- start 2024-03-11T19:38:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7b1c06740 1 -- start start 2024-03-11T19:38:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895bf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895bf970 0x5628895c4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7a5ffb640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd7980a1a30 0x7fd7980a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895c4ae0 0x5628895c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288957acd0 con 0x5628895bef80 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288952df20 con 0x5628895bf970 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288952b450 con 0x5628895c4ae0 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895bf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd7980a1a30 0x7fd7980a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895bf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56372/0 (socket says 172.21.15.83:56372) 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7a7fff640 1 -- 172.21.15.83:0/276583315 learned_addr learned my addr 172.21.15.83:0/276583315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.563+0000 7fd7a5ffb640 1 -- 172.21.15.83:0/615567297 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd7a00cd700 con 0x7fd7a8101c30 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.564+0000 7fd7ad5d6640 1 --2- 172.21.15.83:0/276583315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895c4ae0 0x5628895c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.564+0000 7fd7a7fff640 1 -- 172.21.15.83:0/276583315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895c4ae0 msgr2=0x5628895c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.564+0000 7fd7a7fff640 1 --2- 172.21.15.83:0/276583315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895c4ae0 0x5628895c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.564+0000 7fd7a7fff640 1 -- 172.21.15.83:0/276583315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895bf970 msgr2=0x5628895c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.564+0000 7fd7a7fff640 1 --2- 172.21.15.83:0/276583315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895bf970 0x5628895c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.564+0000 7fd7a7fff640 1 -- 172.21.15.83:0/276583315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628894dbf20 con 0x5628895bef80 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.564+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd7980a1a30 0x7fd7980a3ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd79c006010 tx=0x7fd79c005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.564+0000 7fd7a7fff640 1 --2- 172.21.15.83:0/276583315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895bf3a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd7940058c0 tx=0x7fd794011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d23e8d0cc897e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.564+0000 7fd792ffd640 1 -- 172.21.15.83:0/276583315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79401a400 con 0x5628895bef80 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.564+0000 7fd792ffd640 1 -- 172.21.15.83:0/276583315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd79401aa40 con 0x5628895bef80 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.564+0000 7fd792ffd640 1 -- 172.21.15.83:0/276583315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794012ae0 con 0x5628895bef80 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 -- 172.21.15.83:0/276583315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 msgr2=0x5628895bf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/276583315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895bf3a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd7940058c0 tx=0x7fd794011750 comp rx=0 tx=0).stop 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 -- 172.21.15.83:0/276583315 shutdown_connections 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/276583315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895c4ae0 0x5628895c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/276583315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895bf970 0x5628895c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/276583315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895bf3a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 -- 172.21.15.83:0/276583315 >> 172.21.15.83:0/276583315 conn(0x5628895ba4f0 msgr2=0x5628895ba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 -- 172.21.15.83:0/276583315 shutdown_connections 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 -- 172.21.15.83:0/276583315 wait complete. 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 -- start start 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 Processor -- start 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bef80 0x5628895c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895bf970 0x5628895c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895c4ae0 0x5628895d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562889526c30 con 0x5628895c4ae0 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628894cbd30 con 0x5628895bf970 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628891fa310 con 0x5628895bef80 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7b1c06740 10 client.0 _ll_register_callbacks cb 0x5628894d3d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.565+0000 7fd7ad5d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895c4ae0 0x5628895d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7ad5d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895c4ae0 0x5628895d06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56374/0 (socket says 172.21.15.83:56374) 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7ad5d6640 1 -- 172.21.15.83:0/2193603394 learned_addr learned my addr 172.21.15.83:0/2193603394 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7a7fff640 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895bf970 0x5628895c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7ad5d6640 1 -- 172.21.15.83:0/2193603394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bef80 msgr2=0x5628895c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7ad5d6640 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bef80 0x5628895c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7ad5d6640 1 -- 172.21.15.83:0/2193603394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895bf970 msgr2=0x5628895c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7ad5d6640 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895bf970 0x5628895c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7ad5d6640 1 -- 172.21.15.83:0/2193603394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79400ba10 con 0x5628895c4ae0 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7ad5d6640 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895c4ae0 0x5628895d06e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0096ae0 tx=0x7fd7a00988f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98c46976aafc067 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd787fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd787fff640 1 -- 172.21.15.83:0/2193603394 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a00b1a20 con 0x5628895c4ae0 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd787fff640 1 -- 172.21.15.83:0/2193603394 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd7a009ad80 con 0x5628895c4ae0 2024-03-11T19:38:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2193603394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628892266c0 con 0x5628895c4ae0 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd787fff640 1 -- 172.21.15.83:0/2193603394 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7a009a680 con 0x5628895c4ae0 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7b1c06740 10 client.15174 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2193603394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5628895252b0 con 0x5628895c4ae0 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd787fff640 1 -- 172.21.15.83:0/2193603394 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5628892266c0 con 0x5628895c4ae0 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd787fff640 1 -- 172.21.15.83:0/2193603394 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd7a0018030 con 0x5628895c4ae0 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.566+0000 7fd7b1c06740 10 client.15174 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.567+0000 7fd7b1c06740 10 client.15174 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.567+0000 7fd7b1c06740 20 client.15174 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.567+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5628895d5ad0 0x5628895d4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.567+0000 7fd7b1c06740 4 client.15174 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.567+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2193603394 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562889277270 con 0x5628895d5ad0 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.567+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5628895d5ad0 0x5628895d4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.567+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5628895d5ad0 0x5628895d4d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=933eb10724b8239a server_cookie=5c7e2dc74de7b7a4 in_seq=0 out_seq=0 2024-03-11T19:38:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.567+0000 7fd787fff640 10 client.15174 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd787fff640 1 -- 172.21.15.83:0/2193603394 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562889277270 con 0x5628895d5ad0 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd787fff640 10 client.15174 handle_command_reply: tid=0 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 1 client.15174 shutdown 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2193603394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895c4ae0 msgr2=0x5628895d06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895c4ae0 0x5628895d06e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0096ae0 tx=0x7fd7a00988f0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2193603394 shutdown_connections 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5628895d5ad0 0x5628895d4d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895c4ae0 0x5628895d06e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895bf970 0x5628895c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd787fff640 0 client.15174 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2193603394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bef80 0x5628895c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2193603394 >> 172.21.15.83:0/2193603394 conn(0x5628895ba4f0 msgr2=0x5628895b97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2193603394 shutdown_connections 2024-03-11T19:38:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2193603394 wait complete. 2024-03-11T19:38:52.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.568+0000 7fd7b1c06740 20 client.15174 trim_cache size 0 max 16384 2024-03-11T19:38:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.583+0000 7fd7b1c06740 1 Processor -- start 2024-03-11T19:38:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7b1c06740 1 -- start start 2024-03-11T19:38:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895d4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bf970 0x5628895d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895c4ae0 0x5628895c03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628894ac490 con 0x5628895bef80 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628895d7ab0 con 0x5628895c4ae0 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562889277270 con 0x5628895bf970 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7acdd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895d4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7acdd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895d4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56378/0 (socket says 172.21.15.83:56378) 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7acdd5640 1 -- 172.21.15.83:0/1627484412 learned_addr learned my addr 172.21.15.83:0/1627484412 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7a7fff640 1 --2- 172.21.15.83:0/1627484412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bf970 0x5628895d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7ad5d6640 1 --2- 172.21.15.83:0/1627484412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895c4ae0 0x5628895c03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7acdd5640 1 -- 172.21.15.83:0/1627484412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bf970 msgr2=0x5628895d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/1627484412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bf970 0x5628895d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7acdd5640 1 -- 172.21.15.83:0/1627484412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895c4ae0 msgr2=0x5628895c03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/1627484412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895c4ae0 0x5628895c03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7acdd5640 1 -- 172.21.15.83:0/1627484412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628894c1a20 con 0x5628895bef80 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/1627484412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895d4d50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd79c02c840 tx=0x7fd79c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5839da891b8ea7b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7927fc640 1 -- 172.21.15.83:0/1627484412 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c004300 con 0x5628895bef80 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7927fc640 1 -- 172.21.15.83:0/1627484412 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd79c004460 con 0x5628895bef80 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.584+0000 7fd7927fc640 1 -- 172.21.15.83:0/1627484412 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0053a0 con 0x5628895bef80 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.585+0000 7fd7b1c06740 1 -- 172.21.15.83:0/1627484412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 msgr2=0x5628895d4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.585+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/1627484412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895d4d50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd79c02c840 tx=0x7fd79c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.585+0000 7fd7b1c06740 1 -- 172.21.15.83:0/1627484412 shutdown_connections 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.585+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/1627484412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895c4ae0 0x5628895c03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.585+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/1627484412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bf970 0x5628895d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.585+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/1627484412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895d4d50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.585+0000 7fd7b1c06740 1 -- 172.21.15.83:0/1627484412 >> 172.21.15.83:0/1627484412 conn(0x5628895ba4f0 msgr2=0x5628895ba8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.585+0000 7fd7b1c06740 1 -- 172.21.15.83:0/1627484412 shutdown_connections 2024-03-11T19:38:52.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.585+0000 7fd7b1c06740 1 -- 172.21.15.83:0/1627484412 wait complete. 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.585+0000 7fd7b1c06740 1 -- start start 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.585+0000 7fd7b1c06740 1 Processor -- start 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bf970 0x5628895cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7b1c06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895c4ae0 0x56288962b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628895cdf50 con 0x5628895bef80 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628895ce0c0 con 0x5628895c4ae0 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7b1c06740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288962bc10 con 0x5628895bf970 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7b1c06740 10 client.0 _ll_register_callbacks cb 0x7fd79400b510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7acdd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7acdd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56394/0 (socket says 172.21.15.83:56394) 2024-03-11T19:38:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7acdd5640 1 -- 172.21.15.83:0/2068074579 learned_addr learned my addr 172.21.15.83:0/2068074579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7a7fff640 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bf970 0x5628895cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7ad5d6640 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895c4ae0 0x56288962b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7acdd5640 1 -- 172.21.15.83:0/2068074579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bf970 msgr2=0x5628895cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bf970 0x5628895cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7acdd5640 1 -- 172.21.15.83:0/2068074579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895c4ae0 msgr2=0x56288962b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895c4ae0 0x56288962b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7acdd5640 1 -- 172.21.15.83:0/2068074579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a80682e0 con 0x5628895bef80 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7ad5d6640 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895c4ae0 0x56288962b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7acdd5640 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895cd4d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd79c036040 tx=0x7fd79c004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e07be6464c24de server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd787fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd787fff640 1 -- 172.21.15.83:0/2068074579 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c005530 con 0x5628895bef80 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd787fff640 1 -- 172.21.15.83:0/2068074579 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd79c046030 con 0x5628895bef80 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2068074579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd79401aa40 con 0x5628895bef80 2024-03-11T19:38:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd787fff640 1 -- 172.21.15.83:0/2068074579 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd79c005690 con 0x5628895bef80 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7b1c06740 10 client.15180 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.586+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2068074579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5628894e7780 con 0x5628895bef80 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd787fff640 1 -- 172.21.15.83:0/2068074579 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd79401aa40 con 0x5628895bef80 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd787fff640 1 -- 172.21.15.83:0/2068074579 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd79c03c040 con 0x5628895bef80 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd7b1c06740 10 client.15180 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd7b1c06740 10 client.15180 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd7b1c06740 20 client.15180 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562889630cf0 0x56288962d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd7b1c06740 4 client.15180 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2068074579 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562889263610 con 0x562889630cf0 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd7a7fff640 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562889630cf0 0x56288962d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd787fff640 10 client.15180 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:38:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd7a7fff640 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562889630cf0 0x56288962d1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fe7544bd9030d69 server_cookie=7ea60d6d14e0b44f in_seq=0 out_seq=0 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd787fff640 1 -- 172.21.15.83:0/2068074579 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x562889263610 con 0x562889630cf0 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.587+0000 7fd787fff640 10 client.15180 handle_command_reply: tid=0 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 1 client.15180 shutdown 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2068074579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 msgr2=0x5628895cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895cd4d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd79c036040 tx=0x7fd79c004190 comp rx=0 tx=0).stop 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2068074579 shutdown_connections 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562889630cf0 0x56288962d1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628895c4ae0 0x56288962b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd787fff640 0 client.15180 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628895bf970 0x5628895cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 1 --2- 172.21.15.83:0/2068074579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628895bef80 0x5628895cd4d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2068074579 >> 172.21.15.83:0/2068074579 conn(0x5628895ba4f0 msgr2=0x5628895c03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2068074579 shutdown_connections 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 1 -- 172.21.15.83:0/2068074579 wait complete. 2024-03-11T19:38:52.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7b1c06740 20 client.15180 trim_cache size 0 max 16384 2024-03-11T19:38:52.600 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:38:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "839f93c8-dfdd-11ee-95c8-87774f69a715", 2024-03-11T19:38:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-03-11T19:38:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24494, 2024-03-11T19:38:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-03-11T19:38:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-03-11T19:38:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-03-11T19:38:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 103.972227135, 2024-03-11T19:38:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 21, 2024-03-11T19:38:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 87, 2024-03-11T19:38:52.600 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 85, 2024-03-11T19:38:52.601 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 123.450843987 2024-03-11T19:38:52.601 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:38:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 -- 172.21.15.83:0/615567297 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd7980a1a30 msgr2=0x7fd7980a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd7980a1a30 0x7fd7980a3ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd79c006010 tx=0x7fd79c005be0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 -- 172.21.15.83:0/615567297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a8101c30 msgr2=0x7fd7a819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a8101c30 0x7fd7a819e820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000eaa0 tx=0x7fd7a000ef60 comp rx=0 tx=0).stop 2024-03-11T19:38:52.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 -- 172.21.15.83:0/615567297 shutdown_connections 2024-03-11T19:38:52.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd7980a1a30 0x7fd7980a3ef0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7a8101c30 0x7fd7a819e820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7a8101270 0x7fd7a81977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 --2- 172.21.15.83:0/615567297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a8100080 0x7fd7a8197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 -- 172.21.15.83:0/615567297 >> 172.21.15.83:0/615567297 conn(0x7fd7a80fb610 msgr2=0x7fd7a8104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 -- 172.21.15.83:0/615567297 shutdown_connections 2024-03-11T19:38:52.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.588+0000 7fd7af039640 1 -- 172.21.15.83:0/615567297 wait complete. 2024-03-11T19:38:52.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-11T19:38:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.702+0000 7f5283708640 1 -- 172.21.15.83:0/2114247956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f527c102380 msgr2=0x7f527c102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.702+0000 7f5283708640 1 --2- 172.21.15.83:0/2114247956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f527c102380 0x7f527c102800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f526c009900 tx=0x7f526c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.702+0000 7f5283708640 1 -- 172.21.15.83:0/2114247956 shutdown_connections 2024-03-11T19:38:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.702+0000 7f5283708640 1 --2- 172.21.15.83:0/2114247956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f527c102d40 0x7f527c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.702+0000 7f5283708640 1 --2- 172.21.15.83:0/2114247956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f527c102380 0x7f527c102800 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.702+0000 7f5283708640 1 --2- 172.21.15.83:0/2114247956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f527c101190 0x7f527c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.702+0000 7f5283708640 1 -- 172.21.15.83:0/2114247956 >> 172.21.15.83:0/2114247956 conn(0x7f527c0fc720 msgr2=0x7f527c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.702+0000 7f5283708640 1 -- 172.21.15.83:0/2114247956 shutdown_connections 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.702+0000 7f5283708640 1 -- 172.21.15.83:0/2114247956 wait complete. 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.702+0000 7f5283708640 1 Processor -- start 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.702+0000 7f5283708640 1 -- start start 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f5283708640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f527c101190 0x7f527c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f5283708640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f527c102380 0x7f527c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f5283708640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f527c102d40 0x7f527c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f5283708640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c19ed90 con 0x7f527c101190 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f5283708640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c19ef00 con 0x7f527c102380 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f5283708640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c19f0c0 con 0x7f527c102d40 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52814a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f527c101190 0x7f527c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52814a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f527c101190 0x7f527c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56416/0 (socket says 172.21.15.83:56416) 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52814a4640 1 -- 172.21.15.83:0/433438636 learned_addr learned my addr 172.21.15.83:0/433438636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f5281ca5640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f527c102d40 0x7f527c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52814a4640 1 -- 172.21.15.83:0/433438636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f527c102d40 msgr2=0x7f527c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52814a4640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f527c102d40 0x7f527c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52814a4640 1 -- 172.21.15.83:0/433438636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f527c102380 msgr2=0x7f527c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f5280ca3640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f527c102380 0x7f527c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52814a4640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f527c102380 0x7f527c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52814a4640 1 -- 172.21.15.83:0/433438636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c009560 con 0x7f527c101190 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f5280ca3640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f527c102380 0x7f527c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52814a4640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f527c101190 0x7f527c1972a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f527800c970 tx=0x7f527800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52727fc640 1 -- 172.21.15.83:0/433438636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5278004220 con 0x7f527c101190 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52727fc640 1 -- 172.21.15.83:0/433438636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5278007e90 con 0x7f527c101190 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f5283708640 1 -- 172.21.15.83:0/433438636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c19f3a0 con 0x7f527c101190 2024-03-11T19:38:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f52727fc640 1 -- 172.21.15.83:0/433438636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5278004a00 con 0x7f527c101190 2024-03-11T19:38:52.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.703+0000 7f5283708640 1 -- 172.21.15.83:0/433438636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c19f930 con 0x7f527c101190 2024-03-11T19:38:52.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f52727fc640 1 -- 172.21.15.83:0/433438636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f52780079b0 con 0x7f527c101190 2024-03-11T19:38:52.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f52863b0740 1 Processor -- start 2024-03-11T19:38:52.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f52863b0740 1 -- start start 2024-03-11T19:38:52.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f52863b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c323a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f52863b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c32970 0x55b146c37470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f52863b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c37ae0 0x55b146c39f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f52863b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b146bedcd0 con 0x55b146c31f80 2024-03-11T19:38:52.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f52863b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b146ba0f20 con 0x55b146c32970 2024-03-11T19:38:52.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f52863b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b146b9e450 con 0x55b146c37ae0 2024-03-11T19:38:52.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f5281ca5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f5281ca5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c323a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56422/0 (socket says 172.21.15.83:56422) 2024-03-11T19:38:52.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f5281ca5640 1 -- 172.21.15.83:0/3843263115 learned_addr learned my addr 172.21.15.83:0/3843263115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.706+0000 7f52727fc640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1ab0 0x7f52680a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f5280ca3640 1 --2- 172.21.15.83:0/3843263115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c37ae0 0x55b146c39f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f52814a4640 1 --2- 172.21.15.83:0/3843263115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c32970 0x55b146c37470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f5281ca5640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1ab0 0x7f52680a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f5281ca5640 1 -- 172.21.15.83:0/3843263115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c37ae0 msgr2=0x55b146c39f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f52727fc640 1 -- 172.21.15.83:0/433438636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f52780cd430 con 0x7f527c101190 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f5281ca5640 1 --2- 172.21.15.83:0/3843263115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c37ae0 0x55b146c39f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f5281ca5640 1 -- 172.21.15.83:0/3843263115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c32970 msgr2=0x55b146c37470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f5281ca5640 1 --2- 172.21.15.83:0/3843263115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c32970 0x55b146c37470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f5281ca5640 1 -- 172.21.15.83:0/3843263115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b146b4ef20 con 0x55b146c31f80 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f52814a4640 1 --2- 172.21.15.83:0/3843263115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c32970 0x55b146c37470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f5281ca5640 1 --2- 172.21.15.83:0/3843263115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c323a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f52740137f0 tx=0x7f5274013cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bc41e71c26fc0b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f52677fe640 1 -- 172.21.15.83:0/3843263115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274022070 con 0x55b146c31f80 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f52677fe640 1 -- 172.21.15.83:0/3843263115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5274016b20 con 0x55b146c31f80 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.707+0000 7f52677fe640 1 -- 172.21.15.83:0/3843263115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527401e900 con 0x55b146c31f80 2024-03-11T19:38:52.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 -- 172.21.15.83:0/3843263115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 msgr2=0x55b146c323a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f5281ca5640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1ab0 0x7f52680a3f70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f527401d000 tx=0x7f527401b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 --2- 172.21.15.83:0/3843263115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c323a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f52740137f0 tx=0x7f5274013cb0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 -- 172.21.15.83:0/3843263115 shutdown_connections 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 --2- 172.21.15.83:0/3843263115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c37ae0 0x55b146c39f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 --2- 172.21.15.83:0/3843263115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c32970 0x55b146c37470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 --2- 172.21.15.83:0/3843263115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c323a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 -- 172.21.15.83:0/3843263115 >> 172.21.15.83:0/3843263115 conn(0x55b146c2d4f0 msgr2=0x55b146c2d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 -- 172.21.15.83:0/3843263115 shutdown_connections 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 -- 172.21.15.83:0/3843263115 wait complete. 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 -- start start 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 Processor -- start 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c31f80 0x55b146c3c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c32970 0x55b146c3c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c37ae0 0x55b146c43680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b146b99c30 con 0x55b146c37ae0 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b146b3ed30 con 0x55b146c31f80 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.708+0000 7f52863b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14686d310 con 0x55b146c32970 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f52863b0740 10 client.0 _ll_register_callbacks cb 0x55b146b46d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5281ca5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c37ae0 0x55b146c43680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5281ca5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c37ae0 0x55b146c43680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56426/0 (socket says 172.21.15.83:56426) 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5281ca5640 1 -- 172.21.15.83:0/2148825369 learned_addr learned my addr 172.21.15.83:0/2148825369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5280ca3640 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c32970 0x55b146c3c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f52814a4640 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c31f80 0x55b146c3c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5281ca5640 1 -- 172.21.15.83:0/2148825369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c32970 msgr2=0x55b146c3c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5281ca5640 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c32970 0x55b146c3c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5281ca5640 1 -- 172.21.15.83:0/2148825369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c31f80 msgr2=0x55b146c3c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5281ca5640 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c31f80 0x55b146c3c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5281ca5640 1 -- 172.21.15.83:0/2148825369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52740133a0 con 0x55b146c37ae0 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5280ca3640 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c32970 0x55b146c3c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f52814a4640 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c31f80 0x55b146c3c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5281ca5640 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c37ae0 0x55b146c43680 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5274010780 tx=0x7f5274013330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a9d5389f36e920 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5264ff9640 1 -- 172.21.15.83:0/2148825369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274022040 con 0x55b146c37ae0 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5264ff9640 1 -- 172.21.15.83:0/2148825369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5274016850 con 0x55b146c37ae0 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f52863b0740 1 -- 172.21.15.83:0/2148825369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1468996c0 con 0x55b146c37ae0 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5264ff9640 1 -- 172.21.15.83:0/2148825369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5274036480 con 0x55b146c37ae0 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f52863b0740 10 client.15192 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:52.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f52863b0740 1 -- 172.21.15.83:0/2148825369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b146b982b0 con 0x55b146c37ae0 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.709+0000 7f5264ff9640 1 -- 172.21.15.83:0/2148825369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b1468996c0 con 0x55b146c37ae0 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.710+0000 7f5264ff9640 1 -- 172.21.15.83:0/2148825369 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5274013050 con 0x55b146c37ae0 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.710+0000 7f52863b0740 10 client.15192 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.710+0000 7f52863b0740 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.710+0000 7f52863b0740 20 client.15192 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.710+0000 7f52863b0740 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b146c48850 0x55b146c47ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.710+0000 7f52863b0740 4 client.15192 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.710+0000 7f52863b0740 1 -- 172.21.15.83:0/2148825369 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1468ea270 con 0x55b146c48850 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.710+0000 7f52814a4640 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b146c48850 0x55b146c47ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.710+0000 7f5264ff9640 10 client.15192 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.710+0000 7f52814a4640 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b146c48850 0x55b146c47ae0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3db0e283d90bc8bc server_cookie=d30cbe3a070a13bd in_seq=0 out_seq=0 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f5264ff9640 1 -- 172.21.15.83:0/2148825369 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b1468ea270 con 0x55b146c48850 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f5264ff9640 10 client.15192 handle_command_reply: tid=0 2024-03-11T19:38:52.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 1 client.15192 shutdown 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 1 -- 172.21.15.83:0/2148825369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c37ae0 msgr2=0x55b146c43680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c37ae0 0x55b146c43680 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5274010780 tx=0x7f5274013330 comp rx=0 tx=0).stop 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 1 -- 172.21.15.83:0/2148825369 shutdown_connections 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b146c48850 0x55b146c47ae0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c37ae0 0x55b146c43680 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f5264ff9640 0 client.15192 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c32970 0x55b146c3c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 1 --2- 172.21.15.83:0/2148825369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c31f80 0x55b146c3c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 1 -- 172.21.15.83:0/2148825369 >> 172.21.15.83:0/2148825369 conn(0x55b146c2d4f0 msgr2=0x55b146c2c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 1 -- 172.21.15.83:0/2148825369 shutdown_connections 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 1 -- 172.21.15.83:0/2148825369 wait complete. 2024-03-11T19:38:52.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.711+0000 7f52863b0740 20 client.15192 trim_cache size 0 max 16384 2024-03-11T19:38:52.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52863b0740 1 Processor -- start 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52863b0740 1 -- start start 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52863b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c39ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52863b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c32970 0x55b146c43640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52863b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c37ae0 0x55b146c2d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52863b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b146b1f490 con 0x55b146c31f80 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52863b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b146c46820 con 0x55b146c37ae0 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52863b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b146c44530 con 0x55b146c32970 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52814a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c39ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52814a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c39ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56442/0 (socket says 172.21.15.83:56442) 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52814a4640 1 -- 172.21.15.83:0/2088315376 learned_addr learned my addr 172.21.15.83:0/2088315376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52814a4640 1 -- 172.21.15.83:0/2088315376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c32970 msgr2=0x55b146c43640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52814a4640 1 --2- 172.21.15.83:0/2088315376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c32970 0x55b146c43640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52814a4640 1 -- 172.21.15.83:0/2088315376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c37ae0 msgr2=0x55b146c2d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52814a4640 1 --2- 172.21.15.83:0/2088315376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c37ae0 0x55b146c2d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.727+0000 7f52814a4640 1 -- 172.21.15.83:0/2088315376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b146b34a20 con 0x55b146c31f80 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f52814a4640 1 --2- 172.21.15.83:0/2088315376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c39ed0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5278096130 tx=0x7f5278013b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb6392e01f0d870 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f5266ffd640 1 -- 172.21.15.83:0/2088315376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52780850b0 con 0x55b146c31f80 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f5266ffd640 1 -- 172.21.15.83:0/2088315376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f527809a450 con 0x55b146c31f80 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f5266ffd640 1 -- 172.21.15.83:0/2088315376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5278098580 con 0x55b146c31f80 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f52863b0740 1 -- 172.21.15.83:0/2088315376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 msgr2=0x55b146c39ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f52863b0740 1 --2- 172.21.15.83:0/2088315376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c39ed0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5278096130 tx=0x7f5278013b80 comp rx=0 tx=0).stop 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f52863b0740 1 -- 172.21.15.83:0/2088315376 shutdown_connections 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f52863b0740 1 --2- 172.21.15.83:0/2088315376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c37ae0 0x55b146c2d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f52863b0740 1 --2- 172.21.15.83:0/2088315376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c32970 0x55b146c43640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f52863b0740 1 --2- 172.21.15.83:0/2088315376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c31f80 0x55b146c39ed0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f52863b0740 1 -- 172.21.15.83:0/2088315376 >> 172.21.15.83:0/2088315376 conn(0x55b146c2d4f0 msgr2=0x55b146c28660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f52863b0740 1 -- 172.21.15.83:0/2088315376 shutdown_connections 2024-03-11T19:38:52.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.728+0000 7f52863b0740 1 -- 172.21.15.83:0/2088315376 wait complete. 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f52863b0740 1 -- start start 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f52863b0740 1 Processor -- start 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f52863b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c31f80 0x55b146c409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f52863b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c32970 0x55b146c40f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f52863b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c37ae0 0x55b146c41f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f52863b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b146c47d10 con 0x55b146c32970 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f52863b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b146c44a00 con 0x55b146c31f80 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f52863b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b146c490e0 con 0x55b146c37ae0 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f52863b0740 10 client.0 _ll_register_callbacks cb 0x7f5274008040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f5280ca3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c32970 0x55b146c40f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f5280ca3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c32970 0x55b146c40f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56444/0 (socket says 172.21.15.83:56444) 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f5280ca3640 1 -- 172.21.15.83:0/2701822440 learned_addr learned my addr 172.21.15.83:0/2701822440 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f52814a4640 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c31f80 0x55b146c409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f5280ca3640 1 -- 172.21.15.83:0/2701822440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c37ae0 msgr2=0x55b146c41f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f5280ca3640 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c37ae0 0x55b146c41f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f5280ca3640 1 -- 172.21.15.83:0/2701822440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c31f80 msgr2=0x55b146c409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f5280ca3640 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c31f80 0x55b146c409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f5280ca3640 1 -- 172.21.15.83:0/2701822440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c19f930 con 0x55b146c32970 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f5280ca3640 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c32970 0x55b146c40f00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f526c009230 tx=0x7f526c004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40017ac34273541 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.729+0000 7f5264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f5264ff9640 1 -- 172.21.15.83:0/2701822440 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c004240 con 0x55b146c32970 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f5264ff9640 1 -- 172.21.15.83:0/2701822440 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f526c0043a0 con 0x55b146c32970 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f52863b0740 1 -- 172.21.15.83:0/2701822440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b146c49220 con 0x55b146c32970 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f5264ff9640 1 -- 172.21.15.83:0/2701822440 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f526c004df0 con 0x55b146c32970 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f52863b0740 10 client.15198 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f52863b0740 1 -- 172.21.15.83:0/2701822440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b146b5a780 con 0x55b146c32970 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f5264ff9640 1 -- 172.21.15.83:0/2701822440 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b146c49220 con 0x55b146c32970 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f5264ff9640 1 -- 172.21.15.83:0/2701822440 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f526c050070 con 0x55b146c32970 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f52863b0740 10 client.15198 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:52.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f52863b0740 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-11T19:38:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f52863b0740 20 client.15198 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f52863b0740 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b146c48850 0x55b146ca8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f52863b0740 4 client.15198 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:38:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f52863b0740 1 -- 172.21.15.83:0/2701822440 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f5274016b20 con 0x55b146c48850 2024-03-11T19:38:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.730+0000 7f52814a4640 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b146c48850 0x55b146ca8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f5264ff9640 10 client.15198 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:38:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52814a4640 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b146c48850 0x55b146ca8cf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d349775eab554608 server_cookie=f420ff46f42eec13 in_seq=0 out_seq=0 2024-03-11T19:38:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f5264ff9640 1 -- 172.21.15.83:0/2701822440 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7f5274016b20 con 0x55b146c48850 2024-03-11T19:38:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f5264ff9640 10 client.15198 handle_command_reply: tid=0 2024-03-11T19:38:52.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 1 client.15198 shutdown 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 1 -- 172.21.15.83:0/2701822440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c32970 msgr2=0x55b146c40f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c32970 0x55b146c40f00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f526c009230 tx=0x7f526c004960 comp rx=0 tx=0).stop 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 1 -- 172.21.15.83:0/2701822440 shutdown_connections 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b146c48850 0x55b146ca8cf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b146c37ae0 0x55b146c41f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b146c32970 0x55b146c40f00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 1 --2- 172.21.15.83:0/2701822440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b146c31f80 0x55b146c409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f5264ff9640 0 client.15198 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 1 -- 172.21.15.83:0/2701822440 >> 172.21.15.83:0/2701822440 conn(0x55b146c2d4f0 msgr2=0x55b146c2d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 1 -- 172.21.15.83:0/2701822440 shutdown_connections 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 1 -- 172.21.15.83:0/2701822440 wait complete. 2024-03-11T19:38:52.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.731+0000 7f52863b0740 20 client.15198 trim_cache size 0 max 16384 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "839f93c8-dfdd-11ee-95c8-87774f69a715", 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24502, 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 105.128307859, 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 21, 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 87, 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 85, 2024-03-11T19:38:52.743 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 120.739555315 2024-03-11T19:38:52.744 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:38:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 -- 172.21.15.83:0/433438636 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1ab0 msgr2=0x7f52680a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1ab0 0x7f52680a3f70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f527401d000 tx=0x7f527401b000 comp rx=0 tx=0).stop 2024-03-11T19:38:52.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 -- 172.21.15.83:0/433438636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f527c101190 msgr2=0x7f527c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f527c101190 0x7f527c1972a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f527800c970 tx=0x7f527800ce30 comp rx=0 tx=0).stop 2024-03-11T19:38:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 -- 172.21.15.83:0/433438636 shutdown_connections 2024-03-11T19:38:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1ab0 0x7f52680a3f70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f527c102d40 0x7f527c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f527c102380 0x7f527c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 --2- 172.21.15.83:0/433438636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f527c101190 0x7f527c1972a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 -- 172.21.15.83:0/433438636 >> 172.21.15.83:0/433438636 conn(0x7f527c0fc720 msgr2=0x7f527c1057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 -- 172.21.15.83:0/433438636 shutdown_connections 2024-03-11T19:38:52.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.732+0000 7f5283708640 1 -- 172.21.15.83:0/433438636 wait complete. 2024-03-11T19:38:52.745 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:52 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/29315857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:52.746 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:52 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/818470538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:52.746 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:52 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1568916672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:52.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-11T19:38:52.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.845+0000 7fdbb853a640 1 -- 172.21.15.83:0/1537910656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb0101190 msgr2=0x7fdbb01015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.845+0000 7fdbb853a640 1 --2- 172.21.15.83:0/1537910656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb0101190 0x7fdbb01015b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009900 tx=0x7fdbac02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:52.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.845+0000 7fdbb853a640 1 -- 172.21.15.83:0/1537910656 shutdown_connections 2024-03-11T19:38:52.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.845+0000 7fdbb853a640 1 --2- 172.21.15.83:0/1537910656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb0102d40 0x7fdbb01095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.845+0000 7fdbb853a640 1 --2- 172.21.15.83:0/1537910656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb0102380 0x7fdbb0102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.845+0000 7fdbb853a640 1 --2- 172.21.15.83:0/1537910656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb0101190 0x7fdbb01015b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.845+0000 7fdbb853a640 1 -- 172.21.15.83:0/1537910656 >> 172.21.15.83:0/1537910656 conn(0x7fdbb00fc720 msgr2=0x7fdbb00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.845+0000 7fdbb853a640 1 -- 172.21.15.83:0/1537910656 shutdown_connections 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.845+0000 7fdbb853a640 1 -- 172.21.15.83:0/1537910656 wait complete. 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb853a640 1 Processor -- start 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb853a640 1 -- start start 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb853a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb0101190 0x7fdbb01972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb853a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb0102380 0x7fdbb0197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb853a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb0102d40 0x7fdbb019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb853a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb019ede0 con 0x7fdbb0101190 2024-03-11T19:38:52.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb853a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb019ef50 con 0x7fdbb0102d40 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb853a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb019f110 con 0x7fdbb0102380 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb62d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb0101190 0x7fdbb01972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb62d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb0101190 0x7fdbb01972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56456/0 (socket says 172.21.15.83:56456) 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb62d6640 1 -- 172.21.15.83:0/1113088952 learned_addr learned my addr 172.21.15.83:0/1113088952 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb6ad7640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb0102d40 0x7fdbb019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb5ad5640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb0102380 0x7fdbb0197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb62d6640 1 -- 172.21.15.83:0/1113088952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb0102380 msgr2=0x7fdbb0197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb0102380 0x7fdbb0197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb62d6640 1 -- 172.21.15.83:0/1113088952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb0102d40 msgr2=0x7fdbb019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb0102d40 0x7fdbb019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb62d6640 1 -- 172.21.15.83:0/1113088952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac009560 con 0x7fdbb0101190 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb6ad7640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb0102d40 0x7fdbb019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:52.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb0101190 0x7fdbb01972f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdbac000c00 tx=0x7fdbac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdba77fe640 1 -- 172.21.15.83:0/1113088952 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac004df0 con 0x7fdbb0101190 2024-03-11T19:38:52.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdba77fe640 1 -- 172.21.15.83:0/1113088952 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdbac005430 con 0x7fdbb0101190 2024-03-11T19:38:52.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb853a640 1 -- 172.21.15.83:0/1113088952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb019f390 con 0x7fdbb0101190 2024-03-11T19:38:52.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdba77fe640 1 -- 172.21.15.83:0/1113088952 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac03e450 con 0x7fdbb0101190 2024-03-11T19:38:52.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.846+0000 7fdbb853a640 1 -- 172.21.15.83:0/1113088952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb019f970 con 0x7fdbb0101190 2024-03-11T19:38:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.848+0000 7fdba77fe640 1 -- 172.21.15.83:0/1113088952 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdbac003680 con 0x7fdbb0101190 2024-03-11T19:38:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbbb1e9740 1 Processor -- start 2024-03-11T19:38:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbbb1e9740 1 -- start start 2024-03-11T19:38:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c45ff80 0x55966c4603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c460970 0x55966c465470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c465ae0 0x55966c467f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c41bcd0 con 0x55966c460970 2024-03-11T19:38:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c3cef20 con 0x55966c45ff80 2024-03-11T19:38:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c3cc450 con 0x55966c465ae0 2024-03-11T19:38:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdba77fe640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb9c0a1ab0 0x7fdb9c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbb62d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c460970 0x55966c465470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbb6ad7640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb9c0a1ab0 0x7fdb9c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbb62d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c460970 0x55966c465470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56458/0 (socket says 172.21.15.83:56458) 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.849+0000 7fdbb62d6640 1 -- 172.21.15.83:0/3780251617 learned_addr learned my addr 172.21.15.83:0/3780251617 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdbb6ad7640 1 --2- 172.21.15.83:0/3780251617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c45ff80 0x55966c4603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdba77fe640 1 -- 172.21.15.83:0/1113088952 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdbac0ef8b0 con 0x7fdbb0101190 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdbb5ad5640 1 --2- 172.21.15.83:0/3780251617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c465ae0 0x55966c467f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdbb62d6640 1 -- 172.21.15.83:0/3780251617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c465ae0 msgr2=0x55966c467f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/3780251617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c465ae0 0x55966c467f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdbb62d6640 1 -- 172.21.15.83:0/3780251617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c45ff80 msgr2=0x55966c4603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/3780251617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c45ff80 0x55966c4603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdbb62d6640 1 -- 172.21.15.83:0/3780251617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55966c37cf20 con 0x55966c460970 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdbb5ad5640 1 --2- 172.21.15.83:0/3780251617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c465ae0 0x55966c467f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:52.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdbb6ad7640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb9c0a1ab0 0x7fdb9c0a3f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdba800afd0 tx=0x7fdba800ad00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/3780251617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c460970 0x55966c465470 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0bc860 tx=0x7fdbac0e6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499512fe8dd10b24 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdba4bfe640 1 -- 172.21.15.83:0/3780251617 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0bc410 con 0x55966c460970 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdba4bfe640 1 -- 172.21.15.83:0/3780251617 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdbac0bc230 con 0x55966c460970 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.850+0000 7fdba4bfe640 1 -- 172.21.15.83:0/3780251617 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0c0a20 con 0x55966c460970 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3780251617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c460970 msgr2=0x55966c465470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/3780251617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c460970 0x55966c465470 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0bc860 tx=0x7fdbac0e6f90 comp rx=0 tx=0).stop 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3780251617 shutdown_connections 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/3780251617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c465ae0 0x55966c467f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/3780251617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c460970 0x55966c465470 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/3780251617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c45ff80 0x55966c4603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3780251617 >> 172.21.15.83:0/3780251617 conn(0x55966c45b4f0 msgr2=0x55966c45b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3780251617 shutdown_connections 2024-03-11T19:38:52.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3780251617 wait complete. 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 -- start start 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 Processor -- start 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c45ff80 0x55966c469ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 0x55966c46a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c465ae0 0x55966c471440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c3c7c30 con 0x55966c45ff80 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c36cd30 con 0x55966c460970 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.851+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c09b310 con 0x55966c465ae0 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbbb1e9740 10 client.0 _ll_register_callbacks cb 0x55966c374d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbb62d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c45ff80 0x55966c469ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbb62d6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c45ff80 0x55966c469ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56462/0 (socket says 172.21.15.83:56462) 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbb62d6640 1 -- 172.21.15.83:0/4239812367 learned_addr learned my addr 172.21.15.83:0/4239812367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbb62d6640 1 -- 172.21.15.83:0/4239812367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c465ae0 msgr2=0x55966c471440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/4239812367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c465ae0 0x55966c471440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbb62d6640 1 -- 172.21.15.83:0/4239812367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 msgr2=0x55966c46a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/4239812367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 0x55966c46a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbb62d6640 1 -- 172.21.15.83:0/4239812367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb019f970 con 0x55966c45ff80 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/4239812367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c45ff80 0x55966c469ec0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0b92a0 tx=0x7fdbac0bcfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a69b60f3ab8ebba server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdb99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdb99ffb640 1 -- 172.21.15.83:0/4239812367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0c50f0 con 0x55966c45ff80 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdb99ffb640 1 -- 172.21.15.83:0/4239812367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdbac0c0a20 con 0x55966c45ff80 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/4239812367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55966c0c76c0 con 0x55966c45ff80 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdb99ffb640 1 -- 172.21.15.83:0/4239812367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbac0babd0 con 0x55966c45ff80 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbbb1e9740 10 client.15210 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/4239812367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55966c3c62b0 con 0x55966c45ff80 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdb99ffb640 1 -- 172.21.15.83:0/4239812367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdbac0ea070 con 0x55966c45ff80 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdb99ffb640 1 -- 172.21.15.83:0/4239812367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdbac0efd20 con 0x55966c45ff80 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbbb1e9740 10 client.15210 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.852+0000 7fdbbb1e9740 10 client.15210 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:38:52.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.853+0000 7fdbbb1e9740 20 client.15210 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.853+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/4239812367 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55966c476470 0x55966c47a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.853+0000 7fdbbb1e9740 4 client.15210 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.853+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/4239812367 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55966c118270 con 0x55966c476470 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.853+0000 7fdbb5ad5640 1 --2- 172.21.15.83:0/4239812367 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55966c476470 0x55966c47a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.853+0000 7fdb99ffb640 10 client.15210 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.853+0000 7fdbb5ad5640 1 --2- 172.21.15.83:0/4239812367 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55966c476470 0x55966c47a8f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b513fe85945024 server_cookie=eb20c005b9be1568 in_seq=0 out_seq=0 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdb99ffb640 1 -- 172.21.15.83:0/4239812367 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55966c118270 con 0x55966c476470 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdb99ffb640 10 client.15210 handle_command_reply: tid=0 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 1 client.15210 shutdown 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/4239812367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c45ff80 msgr2=0x55966c469ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/4239812367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c45ff80 0x55966c469ec0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0b92a0 tx=0x7fdbac0bcfd0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/4239812367 shutdown_connections 2024-03-11T19:38:52.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/4239812367 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55966c476470 0x55966c47a8f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/4239812367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c465ae0 0x55966c471440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/4239812367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 0x55966c46a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdb99ffb640 0 client.15210 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:38:52.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/4239812367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c45ff80 0x55966c469ec0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/4239812367 >> 172.21.15.83:0/4239812367 conn(0x55966c45b4f0 msgr2=0x55966c45a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/4239812367 shutdown_connections 2024-03-11T19:38:52.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/4239812367 wait complete. 2024-03-11T19:38:52.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.854+0000 7fdbbb1e9740 20 client.15210 trim_cache size 0 max 16384 2024-03-11T19:38:52.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbbb1e9740 1 Processor -- start 2024-03-11T19:38:52.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbbb1e9740 1 -- start start 2024-03-11T19:38:52.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c45ff80 0x55966c469ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 0x55966c471400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c46a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c34d490 con 0x55966c465ae0 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c469710 con 0x55966c460970 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c465630 con 0x55966c45ff80 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbb6ad7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c46a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbb6ad7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c46a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56478/0 (socket says 172.21.15.83:56478) 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbb6ad7640 1 -- 172.21.15.83:0/1173079270 learned_addr learned my addr 172.21.15.83:0/1173079270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/1173079270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c45ff80 0x55966c469ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbb6ad7640 1 -- 172.21.15.83:0/1173079270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c45ff80 msgr2=0x55966c469ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbb6ad7640 1 --2- 172.21.15.83:0/1173079270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c45ff80 0x55966c469ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbb6ad7640 1 -- 172.21.15.83:0/1173079270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 msgr2=0x55966c471400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbb6ad7640 1 --2- 172.21.15.83:0/1173079270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 0x55966c471400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbb6ad7640 1 -- 172.21.15.83:0/1173079270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55966c362a20 con 0x55966c465ae0 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdbb6ad7640 1 --2- 172.21.15.83:0/1173079270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c46a010 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdba8014680 tx=0x7fdba8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb960e5e82f1a0c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdb9bfff640 1 -- 172.21.15.83:0/1173079270 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8019070 con 0x55966c465ae0 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdb9bfff640 1 -- 172.21.15.83:0/1173079270 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdba8012eb0 con 0x55966c465ae0 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.870+0000 7fdb9bfff640 1 -- 172.21.15.83:0/1173079270 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba801fbf0 con 0x55966c465ae0 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/1173079270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 msgr2=0x55966c46a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/1173079270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c46a010 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdba8014680 tx=0x7fdba8019000 comp rx=0 tx=0).stop 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/1173079270 shutdown_connections 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/1173079270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c46a010 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/1173079270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 0x55966c471400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/1173079270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c45ff80 0x55966c469ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/1173079270 >> 172.21.15.83:0/1173079270 conn(0x55966c45b4f0 msgr2=0x55966c461e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/1173079270 shutdown_connections 2024-03-11T19:38:52.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/1173079270 wait complete. 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 -- start start 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 Processor -- start 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.871+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c45ff80 0x55966c46e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 0x55966c46e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbbb1e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c4cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c4715f0 con 0x55966c465ae0 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c471760 con 0x55966c460970 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbbb1e9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966c4ccaf0 con 0x55966c45ff80 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbbb1e9740 10 client.0 _ll_register_callbacks cb 0x7fdbac0b8b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb6ad7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c4cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb6ad7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c4cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56480/0 (socket says 172.21.15.83:56480) 2024-03-11T19:38:52.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb6ad7640 1 -- 172.21.15.83:0/3080746852 learned_addr learned my addr 172.21.15.83:0/3080746852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb5ad5640 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 0x55966c46e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c45ff80 0x55966c46e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb6ad7640 1 -- 172.21.15.83:0/3080746852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c45ff80 msgr2=0x55966c46e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb6ad7640 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c45ff80 0x55966c46e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb6ad7640 1 -- 172.21.15.83:0/3080746852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 msgr2=0x55966c46e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb6ad7640 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 0x55966c46e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb6ad7640 1 -- 172.21.15.83:0/3080746852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba800a6a0 con 0x55966c465ae0 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb5ad5640 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 0x55966c46e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbb6ad7640 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c4cc5b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdba8000f80 tx=0x7fdba800a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc598281dfaba1ae server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdb99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdb99ffb640 1 -- 172.21.15.83:0/3080746852 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba801fd40 con 0x55966c465ae0 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdb99ffb640 1 -- 172.21.15.83:0/3080746852 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdba800f7c0 con 0x55966c465ae0 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3080746852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbac0bc230 con 0x55966c465ae0 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdb99ffb640 1 -- 172.21.15.83:0/3080746852 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdba801fea0 con 0x55966c465ae0 2024-03-11T19:38:52.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbbb1e9740 10 client.15216 fetch_fsmap learned FSMap version 21 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.872+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3080746852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55966c388780 con 0x55966c465ae0 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.873+0000 7fdb99ffb640 1 -- 172.21.15.83:0/3080746852 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdbac0bc230 con 0x55966c465ae0 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.873+0000 7fdb99ffb640 1 -- 172.21.15.83:0/3080746852 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdba8030b80 con 0x55966c465ae0 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.873+0000 7fdbbb1e9740 10 client.15216 fetch_fsmap finished waiting for FSMap version 21 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.873+0000 7fdbbb1e9740 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.873+0000 7fdbbb1e9740 20 client.15216 populate_metadata read hostname 'smithi083' 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.873+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55966c4d1cd0 0x55966c4ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.873+0000 7fdbbb1e9740 4 client.15216 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.873+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3080746852 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55966c104610 con 0x55966c4d1cd0 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.873+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55966c4d1cd0 0x55966c4ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.873+0000 7fdbb62d6640 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55966c4d1cd0 0x55966c4ce170 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=235342e8514eb698 server_cookie=a95123ac76b8503b in_seq=0 out_seq=0 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.873+0000 7fdb99ffb640 10 client.15216 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:38:52.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdb99ffb640 1 -- 172.21.15.83:0/3080746852 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55966c104610 con 0x55966c4d1cd0 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdb99ffb640 10 client.15216 handle_command_reply: tid=0 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 1 client.15216 shutdown 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3080746852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 msgr2=0x55966c4cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c4cc5b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdba8000f80 tx=0x7fdba800a9c0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3080746852 shutdown_connections 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55966c4d1cd0 0x55966c4ce170 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55966c465ae0 0x55966c4cc5b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdb99ffb640 0 client.15216 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55966c460970 0x55966c46e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 1 --2- 172.21.15.83:0/3080746852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55966c45ff80 0x55966c46e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3080746852 >> 172.21.15.83:0/3080746852 conn(0x55966c45b4f0 msgr2=0x55966c469ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3080746852 shutdown_connections 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 1 -- 172.21.15.83:0/3080746852 wait complete. 2024-03-11T19:38:52.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbbb1e9740 20 client.15216 trim_cache size 0 max 16384 2024-03-11T19:38:52.886 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:38:52.886 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "839f93c8-dfdd-11ee-95c8-87774f69a715", 2024-03-11T19:38:52.886 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-03-11T19:38:52.886 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24527, 2024-03-11T19:38:52.886 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-03-11T19:38:52.886 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-03-11T19:38:52.886 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-03-11T19:38:52.886 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 107.302094771, 2024-03-11T19:38:52.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 21, 2024-03-11T19:38:52.886 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 87, 2024-03-11T19:38:52.887 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 85, 2024-03-11T19:38:52.887 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 112.433922899 2024-03-11T19:38:52.887 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:38:52.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 -- 172.21.15.83:0/1113088952 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb9c0a1ab0 msgr2=0x7fdb9c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb9c0a1ab0 0x7fdb9c0a3f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdba800afd0 tx=0x7fdba800ad00 comp rx=0 tx=0).stop 2024-03-11T19:38:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 -- 172.21.15.83:0/1113088952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb0101190 msgr2=0x7fdbb01972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb0101190 0x7fdbb01972f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdbac000c00 tx=0x7fdbac004270 comp rx=0 tx=0).stop 2024-03-11T19:38:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 -- 172.21.15.83:0/1113088952 shutdown_connections 2024-03-11T19:38:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb9c0a1ab0 0x7fdb9c0a3f70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbb0102d40 0x7fdbb019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbb0102380 0x7fdbb0197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 --2- 172.21.15.83:0/1113088952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbb0101190 0x7fdbb01972f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 -- 172.21.15.83:0/1113088952 >> 172.21.15.83:0/1113088952 conn(0x7fdbb00fc720 msgr2=0x7fdbb0105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 -- 172.21.15.83:0/1113088952 shutdown_connections 2024-03-11T19:38:52.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:52.874+0000 7fdbb853a640 1 -- 172.21.15.83:0/1113088952 wait complete. 2024-03-11T19:38:52.889 INFO:teuthology.run_tasks:Running task kclient... 2024-03-11T19:38:52.913 INFO:tasks.kclient:Mounting kernel clients... 2024-03-11T19:38:52.913 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-11T19:38:52.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:52.913 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-03-11T19:38:52.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:52 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/29315857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:52.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:52 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/818470538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:52.914 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:52 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1568916672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:52.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:52 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/29315857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:38:52.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:52 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/818470538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:38:52.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:52 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1568916672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:52.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:52.929 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-03-11T19:38:52.998 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-03-11T19:38:52.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:38:53.000 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-03-11T19:38:53.000 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-11T19:38:53.000 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-03-11T19:38:53.000 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-03-11T19:38:53.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph fs ls 2024-03-11T19:38:53.279 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:38:53.729 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:53 smithi083 ceph-mon[31864]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-11T19:38:53.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:53 smithi137 ceph-mon[36220]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-11T19:38:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:53 smithi204 ceph-mon[36739]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-11T19:38:54.717 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:54 smithi083 ceph-mon[31864]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:38:54.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.982+0000 7f61a1c69700 1 -- 172.21.15.83:0/2122450254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f619c107ca0 msgr2=0x7f619c108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:54.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.982+0000 7f61a1c69700 1 --2- 172.21.15.83:0/2122450254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f619c107ca0 0x7f619c108080 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f618c0099c0 tx=0x7f618c009cd0 comp rx=0 tx=0).stop 2024-03-11T19:38:54.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.982+0000 7f61a1c69700 1 -- 172.21.15.83:0/2122450254 shutdown_connections 2024-03-11T19:38:54.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.982+0000 7f61a1c69700 1 --2- 172.21.15.83:0/2122450254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f619c102680 0x7f619c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:54.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.982+0000 7f61a1c69700 1 --2- 172.21.15.83:0/2122450254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f619c107ca0 0x7f619c108080 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:54.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.982+0000 7f61a1c69700 1 --2- 172.21.15.83:0/2122450254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f619c101cc0 0x7f619c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:54.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.982+0000 7f61a1c69700 1 -- 172.21.15.83:0/2122450254 >> 172.21.15.83:0/2122450254 conn(0x7f619c078da0 msgr2=0x7f619c0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.983+0000 7f61a1c69700 1 -- 172.21.15.83:0/2122450254 shutdown_connections 2024-03-11T19:38:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.983+0000 7f61a1c69700 1 -- 172.21.15.83:0/2122450254 wait complete. 2024-03-11T19:38:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.983+0000 7f61a1c69700 1 Processor -- start 2024-03-11T19:38:54.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.983+0000 7f61a1c69700 1 -- start start 2024-03-11T19:38:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f61a1c69700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f619c101cc0 0x7f619c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f61a1c69700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f619c102680 0x7f619c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f61a1c69700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f619c107ca0 0x7f619c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f61a1c69700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19d070 con 0x7f619c102680 2024-03-11T19:38:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f61a1c69700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19d1e0 con 0x7f619c101cc0 2024-03-11T19:38:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f61a1c69700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19d3a0 con 0x7f619c107ca0 2024-03-11T19:38:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f619c102680 0x7f619c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f619c102680 0x7f619c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56504/0 (socket says 172.21.15.83:56504) 2024-03-11T19:38:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619affd700 1 -- 172.21.15.83:0/3709483492 learned_addr learned my addr 172.21.15.83:0/3709483492 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:54.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619bfff700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f619c107ca0 0x7f619c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619b7fe700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f619c101cc0 0x7f619c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619affd700 1 -- 172.21.15.83:0/3709483492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f619c107ca0 msgr2=0x7f619c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619affd700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f619c107ca0 0x7f619c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619affd700 1 -- 172.21.15.83:0/3709483492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f619c101cc0 msgr2=0x7f619c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619affd700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f619c101cc0 0x7f619c198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619affd700 1 -- 172.21.15.83:0/3709483492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c0095f0 con 0x7f619c102680 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619bfff700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f619c107ca0 0x7f619c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f619affd700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f619c102680 0x7f619c1986d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f619000b530 tx=0x7f619000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f6198ff9700 1 -- 172.21.15.83:0/3709483492 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190011830 con 0x7f619c102680 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f6198ff9700 1 -- 172.21.15.83:0/3709483492 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6190011e70 con 0x7f619c102680 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.984+0000 7f61a1c69700 1 -- 172.21.15.83:0/3709483492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c19d680 con 0x7f619c102680 2024-03-11T19:38:54.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.985+0000 7f6198ff9700 1 -- 172.21.15.83:0/3709483492 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000f2e0 con 0x7f619c102680 2024-03-11T19:38:54.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.985+0000 7f61a1c69700 1 -- 172.21.15.83:0/3709483492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c1a4ee0 con 0x7f619c102680 2024-03-11T19:38:54.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.985+0000 7f61a1c69700 1 -- 172.21.15.83:0/3709483492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c10a180 con 0x7f619c102680 2024-03-11T19:38:54.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.987+0000 7f6198ff9700 1 -- 172.21.15.83:0/3709483492 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f61900105f0 con 0x7f619c102680 2024-03-11T19:38:54.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.987+0000 7f6198ff9700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f61840a1ac0 0x7f61840a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:54.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.987+0000 7f619b7fe700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f61840a1ac0 0x7f61840a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:54.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.987+0000 7f6198ff9700 1 -- 172.21.15.83:0/3709483492 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f61900cd2e0 con 0x7f619c102680 2024-03-11T19:38:54.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.988+0000 7f619b7fe700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f61840a1ac0 0x7f61840a3f80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f618c003800 tx=0x7f618c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:54.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:54.991+0000 7f6198ff9700 1 -- 172.21.15.83:0/3709483492 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f61900965a0 con 0x7f619c102680 2024-03-11T19:38:55.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:54 smithi137 ceph-mon[36220]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:38:55.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:54 smithi204 ceph-mon[36739]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:38:55.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.228+0000 7f61a1c69700 1 -- 172.21.15.83:0/3709483492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f619c199520 con 0x7f619c102680 2024-03-11T19:38:55.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.228+0000 7f6198ff9700 1 -- 172.21.15.83:0/3709483492 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6190010380 con 0x7f619c102680 2024-03-11T19:38:55.234 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-11T19:38:55.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.230+0000 7f61a1c69700 1 -- 172.21.15.83:0/3709483492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f61840a1ac0 msgr2=0x7f61840a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.230+0000 7f61a1c69700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f61840a1ac0 0x7f61840a3f80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f618c003800 tx=0x7f618c000bc0 comp rx=0 tx=0).stop 2024-03-11T19:38:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.230+0000 7f61a1c69700 1 -- 172.21.15.83:0/3709483492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f619c102680 msgr2=0x7f619c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.230+0000 7f61a1c69700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f619c102680 0x7f619c1986d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f619000b530 tx=0x7f619000b8f0 comp rx=0 tx=0).stop 2024-03-11T19:38:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.230+0000 7f61a1c69700 1 -- 172.21.15.83:0/3709483492 shutdown_connections 2024-03-11T19:38:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.230+0000 7f61a1c69700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f61840a1ac0 0x7f61840a3f80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.230+0000 7f61a1c69700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f619c107ca0 0x7f619c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.230+0000 7f61a1c69700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f619c101cc0 0x7f619c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.230+0000 7f61a1c69700 1 --2- 172.21.15.83:0/3709483492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f619c102680 0x7f619c1986d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.230+0000 7f61a1c69700 1 -- 172.21.15.83:0/3709483492 >> 172.21.15.83:0/3709483492 conn(0x7f619c078da0 msgr2=0x7f619c100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.231+0000 7f61a1c69700 1 -- 172.21.15.83:0/3709483492 shutdown_connections 2024-03-11T19:38:55.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:55.231+0000 7f61a1c69700 1 -- 172.21.15.83:0/3709483492 wait complete. 2024-03-11T19:38:55.924 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:55 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3709483492' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:38:55.950 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-11T19:38:55.950 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-11T19:38:55.950 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-03-11T19:38:55.951 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-11T19:38:55.951 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-03-11T19:38:55.951 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-11T19:38:55.951 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-11T19:38:55.951 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-11T19:38:55.951 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-03-11T19:38:55.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:55.951 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-03-11T19:38:55.967 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-11T19:38:55.967 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-11T19:38:55.967 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-03-11T19:38:55.967 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:38:55.967 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-03-11T19:38:55.967 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42101sec preferred_lft 42101sec 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-03-11T19:38:55.968 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-03-11T19:38:55.969 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-11T19:38:55.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:55.969 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-03-11T19:38:55.969 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-03-11T19:38:55.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-03-11T19:38:55.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-03-11T19:38:55.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-03-11T19:38:55.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-11T19:38:55.970 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-03-11T19:38:56.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:56.139 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-11T19:38:56.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:55 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3709483492' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:38:56.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:55 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3709483492' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-11T19:38:56.165 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-03-11T19:38:56.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:56.167 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-03-11T19:38:56.247 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-03-11T19:38:56.247 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-11T19:38:56.247 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-11T19:38:56.247 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-11T19:38:56.247 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-11T19:38:56.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:56.247 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-03-11T19:38:56.248 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-03-11T19:38:56.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-11T19:38:56.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-11T19:38:56.248 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-11T19:38:56.248 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-03-11T19:38:56.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:56.527 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-03-11T19:38:56.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:56.542 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-03-11T19:38:56.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:56.598 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-03-11T19:38:56.598 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-03-11T19:38:56.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--tmp-cephfs 2024-03-11T19:38:56.598 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-03-11T19:38:56.598 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-03-11T19:38:56.682 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-03-11T19:38:56.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:56.683 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-03-11T19:38:56.683 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-03-11T19:38:56.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-03-11T19:38:56.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-11T19:38:56.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-03-11T19:38:56.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-03-11T19:38:56.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-03-11T19:38:56.683 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-03-11T19:38:56.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:56.821 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec bash -c ' 2024-03-11T19:38:56.821 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-03-11T19:38:56.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-03-11T19:38:56.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-03-11T19:38:56.821 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-03-11T19:38:56.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T19:38:56.863 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /tmp/cephfs 2024-03-11T19:38:56.918 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/tmp/cephfs' 2024-03-11T19:38:56.918 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T19:38:56.918 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /tmp/cephfs 2024-03-11T19:38:56.973 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-11T19:38:56.973 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-11T19:38:56.973 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/bddea3eb-992d-47ca-8e3a-c7ffc2fc4e6f 2024-03-11T19:38:56.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:56.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/bddea3eb-992d-47ca-8e3a-c7ffc2fc4e6f /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context=system_u:object_r:postgresql_db_t:s0 2024-03-11T19:38:57.037 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:56 smithi083 ceph-mon[31864]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:57.058 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-03-11T19:38:57.058 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-03-11T19:38:57.058 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-03-11T19:38:57.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:56 smithi137 ceph-mon[36220]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:56 smithi204 ceph-mon[36739]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-11T19:38:57.179 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-03-11T19:38:57.179 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-03-11T19:38:57.180 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-03-11T19:38:57.180 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:6789,172.21.15.137:6789,172.21.15.204:6789" 2024-03-11T19:38:57.180 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:6789,172.21.15.137:6789,172.21.15.204:6789:/volumes/qa/sv_1/bddea3eb-992d-47ca-8e3a-c7ffc2fc4e6f 2024-03-11T19:38:57.180 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=839f93c8-dfdd-11ee-95c8-87774f69a715" will pass to kernel 2024-03-11T19:38:57.182 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-11T19:38:57.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:57.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /tmp/cephfs 2024-03-11T19:38:57.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:57.251 DEBUG:teuthology.orchestra.run.smithi083:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-11T19:38:57.307 INFO:teuthology.orchestra.run.smithi083.stdout:client24880 2024-03-11T19:38:57.307 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:38:57.307 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T19:38:57.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:57.364 DEBUG:teuthology.orchestra.run.smithi083:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-11T19:38:57.420 INFO:teuthology.orchestra.run.smithi083.stdout:client24880 2024-03-11T19:38:57.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:57.420 DEBUG:teuthology.orchestra.run.smithi083:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-11T19:38:57.477 INFO:teuthology.orchestra.run.smithi083.stdout:client24880 2024-03-11T19:38:57.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:38:57.478 DEBUG:teuthology.orchestra.run.smithi083:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715.client24880/status') 2024-03-11T19:38:57.544 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.24880 (3)172.21.15.83:0/904160463 2024-03-11T19:38:57.544 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2024-03-11T19:38:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2024-03-11T19:38:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2024-03-11T19:38:57.545 INFO:teuthology.orchestra.run.smithi083.stderr:70 bytes copied, 2.1019e-05 s, 3.3 MB/s 2024-03-11T19:38:57.546 INFO:teuthology.run_tasks:Running task exec... 2024-03-11T19:38:57.557 INFO:teuthology.task.exec:Executing custom commands... 2024-03-11T19:38:57.557 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-03-11T19:38:57.558 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-03-11T19:38:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.685+0000 7fdca196c640 1 -- 172.21.15.83:0/528162649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c101c20 msgr2=0x7fdc9c104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.685+0000 7fdca196c640 1 --2- 172.21.15.83:0/528162649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c101c20 0x7fdc9c104010 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009900 tx=0x7fdc8c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.685+0000 7fdca196c640 1 -- 172.21.15.83:0/528162649 shutdown_connections 2024-03-11T19:38:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.685+0000 7fdca196c640 1 --2- 172.21.15.83:0/528162649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9c106e80 0x7fdc9c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.685+0000 7fdca196c640 1 --2- 172.21.15.83:0/528162649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c104550 0x7fdc9c106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.685+0000 7fdca196c640 1 --2- 172.21.15.83:0/528162649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c101c20 0x7fdc9c104010 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.685+0000 7fdca196c640 1 -- 172.21.15.83:0/528162649 >> 172.21.15.83:0/528162649 conn(0x7fdc9c0fb610 msgr2=0x7fdc9c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.685+0000 7fdca196c640 1 -- 172.21.15.83:0/528162649 shutdown_connections 2024-03-11T19:38:57.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.685+0000 7fdca196c640 1 -- 172.21.15.83:0/528162649 wait complete. 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdca196c640 1 Processor -- start 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdca196c640 1 -- start start 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdca196c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c20 0x7fdc9c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdca196c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c104550 0x7fdc9c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdca196c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9c106e80 0x7fdc9c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdca196c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c19ecd0 con 0x7fdc9c104550 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdca196c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c19ee40 con 0x7fdc9c101c20 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdca196c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c19f000 con 0x7fdc9c106e80 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdc93fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c104550 0x7fdc9c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdc93fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c104550 0x7fdc9c197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56514/0 (socket says 172.21.15.83:56514) 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdc93fff640 1 -- 172.21.15.83:0/2796477438 learned_addr learned my addr 172.21.15.83:0/2796477438 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdc9b7fe640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9c106e80 0x7fdc9c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.686+0000 7fdc9affd640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c20 0x7fdc9c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdc93fff640 1 -- 172.21.15.83:0/2796477438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9c106e80 msgr2=0x7fdc9c19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:57.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdc93fff640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9c106e80 0x7fdc9c19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdc93fff640 1 -- 172.21.15.83:0/2796477438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c20 msgr2=0x7fdc9c1971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdc93fff640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c20 0x7fdc9c1971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdc93fff640 1 -- 172.21.15.83:0/2796477438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c009560 con 0x7fdc9c104550 2024-03-11T19:38:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdc9affd640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c20 0x7fdc9c1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdc93fff640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c104550 0x7fdc9c197720 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400b450 tx=0x7fdc8400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdca096a640 1 -- 172.21.15.83:0/2796477438 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84011890 con 0x7fdc9c104550 2024-03-11T19:38:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdca196c640 1 -- 172.21.15.83:0/2796477438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c19f2e0 con 0x7fdc9c104550 2024-03-11T19:38:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdca096a640 1 -- 172.21.15.83:0/2796477438 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc840103e0 con 0x7fdc9c104550 2024-03-11T19:38:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdca096a640 1 -- 172.21.15.83:0/2796477438 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8400f580 con 0x7fdc9c104550 2024-03-11T19:38:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.687+0000 7fdca196c640 1 -- 172.21.15.83:0/2796477438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c19f870 con 0x7fdc9c104550 2024-03-11T19:38:57.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.688+0000 7fdca196c640 1 -- 172.21.15.83:0/2796477438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9c0fd100 con 0x7fdc9c104550 2024-03-11T19:38:57.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.689+0000 7fdca096a640 1 -- 172.21.15.83:0/2796477438 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdc840108e0 con 0x7fdc9c104550 2024-03-11T19:38:57.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.690+0000 7fdca096a640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdc880a1ad0 0x7fdc880a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:57.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.690+0000 7fdc9affd640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdc880a1ad0 0x7fdc880a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:57.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.690+0000 7fdca096a640 1 -- 172.21.15.83:0/2796477438 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdc840cd100 con 0x7fdc9c104550 2024-03-11T19:38:57.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.690+0000 7fdc9affd640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdc880a1ad0 0x7fdc880a3f90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c036040 tx=0x7fdc8c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:57.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.694+0000 7fdca096a640 1 -- 172.21.15.83:0/2796477438 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdc840963c0 con 0x7fdc9c104550 2024-03-11T19:38:57.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.907+0000 7fdca196c640 1 -- 172.21.15.83:0/2796477438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7fdc9c10cde0 con 0x7fdc9c104550 2024-03-11T19:38:57.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.959+0000 7fdca096a640 1 -- 172.21.15.83:0/2796477438 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v22) v1 ==== 117+0+0 (secure 0 0 0) 0x7fdc84099a80 con 0x7fdc9c104550 2024-03-11T19:38:57.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 -- 172.21.15.83:0/2796477438 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdc880a1ad0 msgr2=0x7fdc880a3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:57.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdc880a1ad0 0x7fdc880a3f90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c036040 tx=0x7fdc8c000bc0 comp rx=0 tx=0).stop 2024-03-11T19:38:57.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 -- 172.21.15.83:0/2796477438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c104550 msgr2=0x7fdc9c197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:57.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c104550 0x7fdc9c197720 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400b450 tx=0x7fdc8400b910 comp rx=0 tx=0).stop 2024-03-11T19:38:57.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 -- 172.21.15.83:0/2796477438 shutdown_connections 2024-03-11T19:38:57.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdc880a1ad0 0x7fdc880a3f90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:57.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc9c106e80 0x7fdc9c19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:57.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc9c104550 0x7fdc9c197720 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:57.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 --2- 172.21.15.83:0/2796477438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c20 0x7fdc9c1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:57.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 -- 172.21.15.83:0/2796477438 >> 172.21.15.83:0/2796477438 conn(0x7fdc9c0fb610 msgr2=0x7fdc9c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:57.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 -- 172.21.15.83:0/2796477438 shutdown_connections 2024-03-11T19:38:57.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:57.961+0000 7fdca196c640 1 -- 172.21.15.83:0/2796477438 wait complete. 2024-03-11T19:38:57.977 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-11T19:38:57.988 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-11T19:38:57.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:38:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.096+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/1784325369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea4101c50 msgr2=0x7f3ea41084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.096+0000 7f3eaa9e2640 1 --2- 172.21.15.83:0/1784325369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea4101c50 0x7f3ea41084e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009ca0 tx=0x7f3e9c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.096+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/1784325369 shutdown_connections 2024-03-11T19:38:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.096+0000 7f3eaa9e2640 1 --2- 172.21.15.83:0/1784325369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea4101c50 0x7f3ea41084e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.096+0000 7f3eaa9e2640 1 --2- 172.21.15.83:0/1784325369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ea4101290 0x7f3ea4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.096+0000 7f3eaa9e2640 1 --2- 172.21.15.83:0/1784325369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ea41000a0 0x7f3ea41004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.096+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/1784325369 >> 172.21.15.83:0/1784325369 conn(0x7f3ea40fb610 msgr2=0x7f3ea40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:58.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.096+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/1784325369 shutdown_connections 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.096+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/1784325369 wait complete. 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.097+0000 7f3eaa9e2640 1 Processor -- start 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.097+0000 7f3eaa9e2640 1 -- start start 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.097+0000 7f3eaa9e2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ea41000a0 0x7f3ea4068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.097+0000 7f3eaa9e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea4101290 0x7f3ea419beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.097+0000 7f3eaa9e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ea4101c50 0x7f3ea419e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.097+0000 7f3eaa9e2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea419ecb0 con 0x7f3ea4101290 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.097+0000 7f3eaa9e2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea419ee20 con 0x7f3ea4101c50 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.097+0000 7f3eaa9e2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea419efe0 con 0x7f3ea41000a0 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.097+0000 7f3ea37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea4101290 0x7f3ea419beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.097+0000 7f3ea37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea4101290 0x7f3ea419beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56536/0 (socket says 172.21.15.83:56536) 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.097+0000 7f3ea37fe640 1 -- 172.21.15.83:0/21338953 learned_addr learned my addr 172.21.15.83:0/21338953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3ea8f7f640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ea4101c50 0x7f3ea419e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3ea3fff640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ea41000a0 0x7f3ea4068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3ea37fe640 1 -- 172.21.15.83:0/21338953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ea41000a0 msgr2=0x7f3ea4068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3ea37fe640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ea41000a0 0x7f3ea4068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3ea37fe640 1 -- 172.21.15.83:0/21338953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ea4101c50 msgr2=0x7f3ea419e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3ea37fe640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ea4101c50 0x7f3ea419e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3ea37fe640 1 -- 172.21.15.83:0/21338953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c009900 con 0x7f3ea4101290 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3ea37fe640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea4101290 0x7f3ea419beb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800c970 tx=0x7f3e9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3ea17fa640 1 -- 172.21.15.83:0/21338953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98007bb0 con 0x7f3ea4101290 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3ea17fa640 1 -- 172.21.15.83:0/21338953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e98007d10 con 0x7f3ea4101290 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/21338953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea419f260 con 0x7f3ea4101290 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3ea17fa640 1 -- 172.21.15.83:0/21338953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800f450 con 0x7f3ea4101290 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.098+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/21338953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea419f720 con 0x7f3ea4101290 2024-03-11T19:38:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.099+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/21338953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea41072f0 con 0x7f3ea4101290 2024-03-11T19:38:58.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.100+0000 7f3ea17fa640 1 -- 172.21.15.83:0/21338953 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3e98004f50 con 0x7f3ea4101290 2024-03-11T19:38:58.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.101+0000 7f3ea17fa640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3e8c0a1a30 0x7f3e8c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.101+0000 7f3ea3fff640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3e8c0a1a30 0x7f3e8c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.101+0000 7f3ea17fa640 1 -- 172.21.15.83:0/21338953 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3e980cd5f0 con 0x7f3ea4101290 2024-03-11T19:38:58.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.101+0000 7f3ea3fff640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3e8c0a1a30 0x7f3e8c0a3ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3e940058c0 tx=0x7f3e94005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:58.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.105+0000 7f3ea17fa640 1 -- 172.21.15.83:0/21338953 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e98096810 con 0x7f3ea4101290 2024-03-11T19:38:58.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.279+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/21338953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ea410bc80 con 0x7f3ea4101290 2024-03-11T19:38:58.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.281+0000 7f3ea17fa640 1 -- 172.21.15.83:0/21338953 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7f3ea410bc80 con 0x7f3ea4101290 2024-03-11T19:38:58.285 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:58.285 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:38:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/21338953 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3e8c0a1a30 msgr2=0x7f3e8c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3e8c0a1a30 0x7f3e8c0a3ef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3e940058c0 tx=0x7f3e94005850 comp rx=0 tx=0).stop 2024-03-11T19:38:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/21338953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea4101290 msgr2=0x7f3ea419beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea4101290 0x7f3ea419beb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800c970 tx=0x7f3e9800ce30 comp rx=0 tx=0).stop 2024-03-11T19:38:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/21338953 shutdown_connections 2024-03-11T19:38:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3e8c0a1a30 0x7f3e8c0a3ef0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ea4101c50 0x7f3ea419e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ea4101290 0x7f3ea419beb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 --2- 172.21.15.83:0/21338953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ea41000a0 0x7f3ea4068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/21338953 >> 172.21.15.83:0/21338953 conn(0x7f3ea40fb610 msgr2=0x7f3ea41044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:58.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/21338953 shutdown_connections 2024-03-11T19:38:58.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.283+0000 7f3eaa9e2640 1 -- 172.21.15.83:0/21338953 wait complete. 2024-03-11T19:38:58.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:57 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2796477438' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-11T19:38:58.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:37:03.034982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:37:04.125352+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:38:58.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:38:58.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:57 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2796477438' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-11T19:38:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.398+0000 7f4c04b4f640 1 -- 172.21.15.83:0/3107502794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c00101c30 msgr2=0x7f4c001084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.398+0000 7f4c04b4f640 1 --2- 172.21.15.83:0/3107502794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c00101c30 0x7f4c001084c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009ca0 tx=0x7f4bf802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 -- 172.21.15.83:0/3107502794 shutdown_connections 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 --2- 172.21.15.83:0/3107502794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c00101c30 0x7f4c001084c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 --2- 172.21.15.83:0/3107502794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c00101270 0x7f4c001016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 --2- 172.21.15.83:0/3107502794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c00100080 0x7f4c001004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 -- 172.21.15.83:0/3107502794 >> 172.21.15.83:0/3107502794 conn(0x7f4c000fb610 msgr2=0x7f4c000fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 -- 172.21.15.83:0/3107502794 shutdown_connections 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 -- 172.21.15.83:0/3107502794 wait complete. 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 Processor -- start 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 -- start start 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c00100080 0x7f4c001971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c00101270 0x7f4c00197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c00101c30 0x7f4c0019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0019ecd0 con 0x7f4c00101270 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0019ee40 con 0x7f4c00100080 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.399+0000 7f4c04b4f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0019f000 con 0x7f4c00101c30 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c00101270 0x7f4c00197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c00101270 0x7f4c00197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56554/0 (socket says 172.21.15.83:56554) 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf7fff640 1 -- 172.21.15.83:0/1273669785 learned_addr learned my addr 172.21.15.83:0/1273669785 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bfe59c640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c00100080 0x7f4c001971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bfed9d640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c00101c30 0x7f4c0019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf7fff640 1 -- 172.21.15.83:0/1273669785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c00101c30 msgr2=0x7f4c0019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf7fff640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c00101c30 0x7f4c0019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf7fff640 1 -- 172.21.15.83:0/1273669785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c00100080 msgr2=0x7f4c001971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf7fff640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c00100080 0x7f4c001971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf7fff640 1 -- 172.21.15.83:0/1273669785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8009900 con 0x7f4c00101270 2024-03-11T19:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf7fff640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c00101270 0x7f4c00197720 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4be800c890 tx=0x7f4be800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf77fe640 1 -- 172.21.15.83:0/1273669785 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8004220 con 0x7f4c00101270 2024-03-11T19:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf77fe640 1 -- 172.21.15.83:0/1273669785 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4be8007e90 con 0x7f4c00101270 2024-03-11T19:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4c04b4f640 1 -- 172.21.15.83:0/1273669785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0019f2e0 con 0x7f4c00101270 2024-03-11T19:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4bf77fe640 1 -- 172.21.15.83:0/1273669785 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8004a00 con 0x7f4c00101270 2024-03-11T19:38:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.400+0000 7f4c04b4f640 1 -- 172.21.15.83:0/1273669785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0019f7f0 con 0x7f4c00101270 2024-03-11T19:38:58.408 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:57 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2796477438' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-03-11T19:38:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.401+0000 7f4c04b4f640 1 -- 172.21.15.83:0/1273669785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c001073a0 con 0x7f4c00101270 2024-03-11T19:38:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.402+0000 7f4bf77fe640 1 -- 172.21.15.83:0/1273669785 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4be80079b0 con 0x7f4c00101270 2024-03-11T19:38:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.403+0000 7f4bf77fe640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4bec0a1a30 0x7f4bec0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.403+0000 7f4bfe59c640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4bec0a1a30 0x7f4bec0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.403+0000 7f4bf77fe640 1 -- 172.21.15.83:0/1273669785 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4be80cd100 con 0x7f4c00101270 2024-03-11T19:38:58.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.403+0000 7f4bfe59c640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4bec0a1a30 0x7f4bec0a3ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00058c0 tx=0x7f4bf000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.407+0000 7f4bf77fe640 1 -- 172.21.15.83:0/1273669785 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4be80963c0 con 0x7f4c00101270 2024-03-11T19:38:58.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.582+0000 7f4c04b4f640 1 -- 172.21.15.83:0/1273669785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c0002ce40 con 0x7f4c00101270 2024-03-11T19:38:58.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.582+0000 7f4bf77fe640 1 -- 172.21.15.83:0/1273669785 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4be8099a80 con 0x7f4c00101270 2024-03-11T19:38:58.587 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:38:58.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 -- 172.21.15.83:0/1273669785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4bec0a1a30 msgr2=0x7f4bec0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4bec0a1a30 0x7f4bec0a3ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00058c0 tx=0x7f4bf000a4c0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 -- 172.21.15.83:0/1273669785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c00101270 msgr2=0x7f4c00197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c00101270 0x7f4c00197720 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4be800c890 tx=0x7f4be800cd50 comp rx=0 tx=0).stop 2024-03-11T19:38:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 -- 172.21.15.83:0/1273669785 shutdown_connections 2024-03-11T19:38:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4bec0a1a30 0x7f4bec0a3ef0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c00101c30 0x7f4c0019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c00101270 0x7f4c00197720 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 --2- 172.21.15.83:0/1273669785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c00100080 0x7f4c001971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 -- 172.21.15.83:0/1273669785 >> 172.21.15.83:0/1273669785 conn(0x7f4c000fb610 msgr2=0x7f4c00104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 -- 172.21.15.83:0/1273669785 shutdown_connections 2024-03-11T19:38:58.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.584+0000 7f4c04b4f640 1 -- 172.21.15.83:0/1273669785 wait complete. 2024-03-11T19:38:58.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:38:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.699+0000 7f23811bc640 1 -- 172.21.15.83:0/2979558808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f237c106e70 msgr2=0x7f237c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.699+0000 7f23811bc640 1 --2- 172.21.15.83:0/2979558808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f237c106e70 0x7f237c1092c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2374009ca0 tx=0x7f237402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.699+0000 7f23811bc640 1 -- 172.21.15.83:0/2979558808 shutdown_connections 2024-03-11T19:38:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.699+0000 7f23811bc640 1 --2- 172.21.15.83:0/2979558808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f237c106e70 0x7f237c1092c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.699+0000 7f23811bc640 1 --2- 172.21.15.83:0/2979558808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f237c104540 0x7f237c106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.699+0000 7f23811bc640 1 --2- 172.21.15.83:0/2979558808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c101c10 0x7f237c104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.699+0000 7f23811bc640 1 -- 172.21.15.83:0/2979558808 >> 172.21.15.83:0/2979558808 conn(0x7f237c0fb620 msgr2=0x7f237c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.699+0000 7f23811bc640 1 -- 172.21.15.83:0/2979558808 shutdown_connections 2024-03-11T19:38:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.699+0000 7f23811bc640 1 -- 172.21.15.83:0/2979558808 wait complete. 2024-03-11T19:38:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.700+0000 7f23811bc640 1 Processor -- start 2024-03-11T19:38:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.700+0000 7f23811bc640 1 -- start start 2024-03-11T19:38:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.700+0000 7f23811bc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f237c101c10 0x7f237c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.700+0000 7f23811bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c104540 0x7f237c1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.700+0000 7f23811bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f237c106e70 0x7f237c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.700+0000 7f23811bc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c19ec80 con 0x7f237c106e70 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.700+0000 7f23811bc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c19edf0 con 0x7f237c104540 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.700+0000 7f23811bc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c19efb0 con 0x7f237c101c10 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.700+0000 7f237b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f237c106e70 0x7f237c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.700+0000 7f237b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f237c106e70 0x7f237c19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56572/0 (socket says 172.21.15.83:56572) 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.700+0000 7f237b59e640 1 -- 172.21.15.83:0/1111502223 learned_addr learned my addr 172.21.15.83:0/1111502223 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f237ad9d640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f237c101c10 0x7f237c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f237a59c640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c104540 0x7f237c1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f237b59e640 1 -- 172.21.15.83:0/1111502223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f237c101c10 msgr2=0x7f237c197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f237b59e640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f237c101c10 0x7f237c197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f237b59e640 1 -- 172.21.15.83:0/1111502223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c104540 msgr2=0x7f237c1976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f237b59e640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c104540 0x7f237c1976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f237b59e640 1 -- 172.21.15.83:0/1111502223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2374009900 con 0x7f237c106e70 2024-03-11T19:38:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f237a59c640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c104540 0x7f237c1976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f237b59e640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f237c106e70 0x7f237c19e740 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2374037040 tx=0x7f2374034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f2363fff640 1 -- 172.21.15.83:0/1111502223 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374032eb0 con 0x7f237c106e70 2024-03-11T19:38:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f23811bc640 1 -- 172.21.15.83:0/1111502223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c19f230 con 0x7f237c106e70 2024-03-11T19:38:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f2363fff640 1 -- 172.21.15.83:0/1111502223 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2374003e50 con 0x7f237c106e70 2024-03-11T19:38:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f2363fff640 1 -- 172.21.15.83:0/1111502223 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374040450 con 0x7f237c106e70 2024-03-11T19:38:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.701+0000 7f23811bc640 1 -- 172.21.15.83:0/1111502223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c19f6e0 con 0x7f237c106e70 2024-03-11T19:38:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.702+0000 7f23811bc640 1 -- 172.21.15.83:0/1111502223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237c0fd110 con 0x7f237c106e70 2024-03-11T19:38:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.703+0000 7f2363fff640 1 -- 172.21.15.83:0/1111502223 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f237400a040 con 0x7f237c106e70 2024-03-11T19:38:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.704+0000 7f2363fff640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f23640a1a30 0x7f23640a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.704+0000 7f2363fff640 1 -- 172.21.15.83:0/1111502223 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f23740f02e0 con 0x7f237c106e70 2024-03-11T19:38:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.704+0000 7f237ad9d640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f23640a1a30 0x7f23640a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:58.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.704+0000 7f237ad9d640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f23640a1a30 0x7f23640a3ef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f236c005d00 tx=0x7f236c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.708+0000 7f2363fff640 1 -- 172.21.15.83:0/1111502223 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f23740b95a0 con 0x7f237c106e70 2024-03-11T19:38:58.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.880+0000 7f23811bc640 1 -- 172.21.15.83:0/1111502223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f237c02ce40 con 0x7f237c106e70 2024-03-11T19:38:58.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.880+0000 7f2363fff640 1 -- 172.21.15.83:0/1111502223 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23740bcc60 con 0x7f237c106e70 2024-03-11T19:38:58.885 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-03-11T19:38:58.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.882+0000 7f23811bc640 1 -- 172.21.15.83:0/1111502223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f23640a1a30 msgr2=0x7f23640a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.882+0000 7f23811bc640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f23640a1a30 0x7f23640a3ef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f236c005d00 tx=0x7f236c00b410 comp rx=0 tx=0).stop 2024-03-11T19:38:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.882+0000 7f23811bc640 1 -- 172.21.15.83:0/1111502223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f237c106e70 msgr2=0x7f237c19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.882+0000 7f23811bc640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f237c106e70 0x7f237c19e740 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2374037040 tx=0x7f2374034d40 comp rx=0 tx=0).stop 2024-03-11T19:38:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.882+0000 7f23811bc640 1 -- 172.21.15.83:0/1111502223 shutdown_connections 2024-03-11T19:38:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.882+0000 7f23811bc640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f23640a1a30 0x7f23640a3ef0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.882+0000 7f23811bc640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f237c106e70 0x7f237c19e740 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.882+0000 7f23811bc640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f237c104540 0x7f237c1976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.882+0000 7f23811bc640 1 --2- 172.21.15.83:0/1111502223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f237c101c10 0x7f237c197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.882+0000 7f23811bc640 1 -- 172.21.15.83:0/1111502223 >> 172.21.15.83:0/1111502223 conn(0x7f237c0fb620 msgr2=0x7f237c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.882+0000 7f23811bc640 1 -- 172.21.15.83:0/1111502223 shutdown_connections 2024-03-11T19:38:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.883+0000 7f23811bc640 1 -- 172.21.15.83:0/1111502223 wait complete. 2024-03-11T19:38:58.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:38:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.999+0000 7f52a368f640 1 -- 172.21.15.83:0/2592911786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529c1003d0 msgr2=0x7f529c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.999+0000 7f52a368f640 1 --2- 172.21.15.83:0/2592911786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529c1003d0 0x7f529c100830 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f528c009900 tx=0x7f528c02bcd0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.999+0000 7f52a368f640 1 -- 172.21.15.83:0/2592911786 shutdown_connections 2024-03-11T19:38:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.999+0000 7f52a368f640 1 --2- 172.21.15.83:0/2592911786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f529c100d70 0x7f529c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.999+0000 7f52a368f640 1 --2- 172.21.15.83:0/2592911786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529c1003d0 0x7f529c100830 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.999+0000 7f52a368f640 1 --2- 172.21.15.83:0/2592911786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f529c0ffa70 0x7f529c0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:58.999+0000 7f52a368f640 1 -- 172.21.15.83:0/2592911786 >> 172.21.15.83:0/2592911786 conn(0x7f529c0fb610 msgr2=0x7f529c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a368f640 1 -- 172.21.15.83:0/2592911786 shutdown_connections 2024-03-11T19:38:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a368f640 1 -- 172.21.15.83:0/2592911786 wait complete. 2024-03-11T19:38:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a368f640 1 Processor -- start 2024-03-11T19:38:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a368f640 1 -- start start 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a368f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f529c0ffa70 0x7f529c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a368f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f529c1003d0 0x7f529c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a368f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529c100d70 0x7f529c1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a368f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a0f00 con 0x7f529c100d70 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a368f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a1070 con 0x7f529c1003d0 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a368f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1a1230 con 0x7f529c0ffa70 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a1c2c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529c100d70 0x7f529c1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a1c2c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529c100d70 0x7f529c1a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56590/0 (socket says 172.21.15.83:56590) 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a1c2c640 1 -- 172.21.15.83:0/1944234065 learned_addr learned my addr 172.21.15.83:0/1944234065 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a0c2a640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f529c1003d0 0x7f529c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.000+0000 7f52a142b640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f529c0ffa70 0x7f529c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52a1c2c640 1 -- 172.21.15.83:0/1944234065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f529c0ffa70 msgr2=0x7f529c199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52a1c2c640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f529c0ffa70 0x7f529c199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52a1c2c640 1 -- 172.21.15.83:0/1944234065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f529c1003d0 msgr2=0x7f529c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52a1c2c640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f529c1003d0 0x7f529c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52a1c2c640 1 -- 172.21.15.83:0/1944234065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c009560 con 0x7f529c100d70 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52a142b640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f529c0ffa70 0x7f529c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52a1c2c640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529c100d70 0x7f529c1a09c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f529400d750 tx=0x7f529400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52927fc640 1 -- 172.21.15.83:0/1944234065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294009bd0 con 0x7f529c100d70 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52927fc640 1 -- 172.21.15.83:0/1944234065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f529400de60 con 0x7f529c100d70 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52a368f640 1 -- 172.21.15.83:0/1944234065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c1a1510 con 0x7f529c100d70 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52927fc640 1 -- 172.21.15.83:0/1944234065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294004e70 con 0x7f529c100d70 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.001+0000 7f52a368f640 1 -- 172.21.15.83:0/1944234065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c1a19f0 con 0x7f529c100d70 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.002+0000 7f52a368f640 1 -- 172.21.15.83:0/1944234065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c10a9a0 con 0x7f529c100d70 2024-03-11T19:38:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.003+0000 7f52927fc640 1 -- 172.21.15.83:0/1944234065 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5294005020 con 0x7f529c100d70 2024-03-11T19:38:59.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.003+0000 7f52927fc640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52880a1a30 0x7f52880a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.003+0000 7f52a142b640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52880a1a30 0x7f52880a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.004+0000 7f52927fc640 1 -- 172.21.15.83:0/1944234065 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f52940cccd0 con 0x7f529c100d70 2024-03-11T19:38:59.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.004+0000 7f52a142b640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52880a1a30 0x7f52880a3ef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5298005fd0 tx=0x7f5298009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.008+0000 7f52927fc640 1 -- 172.21.15.83:0/1944234065 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5294099ac0 con 0x7f529c100d70 2024-03-11T19:38:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.182+0000 7f52a368f640 1 -- 172.21.15.83:0/1944234065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f529c02cdd0 con 0x7f529c100d70 2024-03-11T19:38:59.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.183+0000 7f52927fc640 1 -- 172.21.15.83:0/1944234065 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52940170a0 con 0x7f529c100d70 2024-03-11T19:38:59.187 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-03-11T19:38:59.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.184+0000 7f52a368f640 1 -- 172.21.15.83:0/1944234065 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52880a1a30 msgr2=0x7f52880a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.184+0000 7f52a368f640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52880a1a30 0x7f52880a3ef0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5298005fd0 tx=0x7f5298009520 comp rx=0 tx=0).stop 2024-03-11T19:38:59.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.184+0000 7f52a368f640 1 -- 172.21.15.83:0/1944234065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529c100d70 msgr2=0x7f529c1a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.184+0000 7f52a368f640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529c100d70 0x7f529c1a09c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f529400d750 tx=0x7f529400dc10 comp rx=0 tx=0).stop 2024-03-11T19:38:59.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.185+0000 7f52a368f640 1 -- 172.21.15.83:0/1944234065 shutdown_connections 2024-03-11T19:38:59.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.185+0000 7f52a368f640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52880a1a30 0x7f52880a3ef0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.185+0000 7f52a368f640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f529c100d70 0x7f529c1a09c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.185+0000 7f52a368f640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f529c1003d0 0x7f529c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.185+0000 7f52a368f640 1 --2- 172.21.15.83:0/1944234065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f529c0ffa70 0x7f529c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.185+0000 7f52a368f640 1 -- 172.21.15.83:0/1944234065 >> 172.21.15.83:0/1944234065 conn(0x7f529c0fb610 msgr2=0x7f529c0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.185+0000 7f52a368f640 1 -- 172.21.15.83:0/1944234065 shutdown_connections 2024-03-11T19:38:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.185+0000 7f52a368f640 1 -- 172.21.15.83:0/1944234065 wait complete. 2024-03-11T19:38:59.198 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:58 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2796477438' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-11T19:38:59.198 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:58 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:38:59.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:58 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/21338953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:59.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:58 smithi083 ceph-mon[31864]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:38:59.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:58 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1273669785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:59.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:58 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1111502223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:59.200 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:38:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.297+0000 7fd617840640 1 -- 172.21.15.83:0/1880593457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6101000a0 msgr2=0x7fd6101004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.297+0000 7fd617840640 1 --2- 172.21.15.83:0/1880593457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6101000a0 0x7fd6101004c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd608009ca0 tx=0x7fd60802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.297+0000 7fd617840640 1 -- 172.21.15.83:0/1880593457 shutdown_connections 2024-03-11T19:38:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.297+0000 7fd617840640 1 --2- 172.21.15.83:0/1880593457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd610101c50 0x7fd6101084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.297+0000 7fd617840640 1 --2- 172.21.15.83:0/1880593457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd610101290 0x7fd610101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.297+0000 7fd617840640 1 --2- 172.21.15.83:0/1880593457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6101000a0 0x7fd6101004c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.297+0000 7fd617840640 1 -- 172.21.15.83:0/1880593457 >> 172.21.15.83:0/1880593457 conn(0x7fd6100fb610 msgr2=0x7fd6100fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.297+0000 7fd617840640 1 -- 172.21.15.83:0/1880593457 shutdown_connections 2024-03-11T19:38:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.297+0000 7fd617840640 1 -- 172.21.15.83:0/1880593457 wait complete. 2024-03-11T19:38:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.297+0000 7fd617840640 1 Processor -- start 2024-03-11T19:38:59.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd617840640 1 -- start start 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd617840640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6101000a0 0x7fd610197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd617840640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd610101290 0x7fd6101977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd617840640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd610101c50 0x7fd61019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd617840640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019ed50 con 0x7fd610101c50 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd617840640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019eec0 con 0x7fd6101000a0 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd617840640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019f080 con 0x7fd610101290 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd615ddd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd610101c50 0x7fd61019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd615ddd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd610101c50 0x7fd61019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56606/0 (socket says 172.21.15.83:56606) 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd615ddd640 1 -- 172.21.15.83:0/941248846 learned_addr learned my addr 172.21.15.83:0/941248846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd614ddb640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd610101290 0x7fd6101977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd6155dc640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6101000a0 0x7fd610197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd615ddd640 1 -- 172.21.15.83:0/941248846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd610101290 msgr2=0x7fd6101977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd615ddd640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd610101290 0x7fd6101977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd615ddd640 1 -- 172.21.15.83:0/941248846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6101000a0 msgr2=0x7fd610197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd615ddd640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6101000a0 0x7fd610197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd615ddd640 1 -- 172.21.15.83:0/941248846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd608009900 con 0x7fd610101c50 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd6155dc640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6101000a0 0x7fd610197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd614ddb640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd610101290 0x7fd6101977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd615ddd640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd610101c50 0x7fd61019e810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00d750 tx=0x7fd60c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd6067fc640 1 -- 172.21.15.83:0/941248846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c009bd0 con 0x7fd610101c50 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd6067fc640 1 -- 172.21.15.83:0/941248846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd60c00de60 con 0x7fd610101c50 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd617840640 1 -- 172.21.15.83:0/941248846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61019f360 con 0x7fd610101c50 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd6067fc640 1 -- 172.21.15.83:0/941248846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c004e70 con 0x7fd610101c50 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.298+0000 7fd617840640 1 -- 172.21.15.83:0/941248846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61019f870 con 0x7fd610101c50 2024-03-11T19:38:59.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.299+0000 7fd617840640 1 -- 172.21.15.83:0/941248846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61010bab0 con 0x7fd610101c50 2024-03-11T19:38:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.300+0000 7fd6067fc640 1 -- 172.21.15.83:0/941248846 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd60c005020 con 0x7fd610101c50 2024-03-11T19:38:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.301+0000 7fd6067fc640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd5fc0a1a30 0x7fd5fc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.301+0000 7fd6155dc640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd5fc0a1a30 0x7fd5fc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.301+0000 7fd6067fc640 1 -- 172.21.15.83:0/941248846 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd60c0ccd80 con 0x7fd610101c50 2024-03-11T19:38:59.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.301+0000 7fd6155dc640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd5fc0a1a30 0x7fd5fc0a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd608032540 tx=0x7fd608005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:59.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.305+0000 7fd6067fc640 1 -- 172.21.15.83:0/941248846 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd60c099b70 con 0x7fd610101c50 2024-03-11T19:38:59.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:58 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2796477438' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-11T19:38:59.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:58 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:38:59.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:58 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/21338953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:59.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:58 smithi137 ceph-mon[36220]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:38:59.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:58 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1273669785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:59.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:58 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1111502223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:59.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:58 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2796477438' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-03-11T19:38:59.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:58 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:38:59.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:58 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/21338953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:38:59.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:58 smithi204 ceph-mon[36739]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-11T19:38:59.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:58 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1273669785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:38:59.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:58 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1111502223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:38:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.478+0000 7fd617840640 1 -- 172.21.15.83:0/941248846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd61019fd30 con 0x7fd610101c50 2024-03-11T19:38:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.480+0000 7fd6067fc640 1 -- 172.21.15.83:0/941248846 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7fd60c017090 con 0x7fd610101c50 2024-03-11T19:38:59.484 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:38:59.484 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:38:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 -- 172.21.15.83:0/941248846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd5fc0a1a30 msgr2=0x7fd5fc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd5fc0a1a30 0x7fd5fc0a3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd608032540 tx=0x7fd608005c50 comp rx=0 tx=0).stop 2024-03-11T19:38:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 -- 172.21.15.83:0/941248846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd610101c50 msgr2=0x7fd61019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd610101c50 0x7fd61019e810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00d750 tx=0x7fd60c00dc10 comp rx=0 tx=0).stop 2024-03-11T19:38:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 -- 172.21.15.83:0/941248846 shutdown_connections 2024-03-11T19:38:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd5fc0a1a30 0x7fd5fc0a3ef0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd610101c50 0x7fd61019e810 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd610101290 0x7fd6101977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 --2- 172.21.15.83:0/941248846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6101000a0 0x7fd610197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 -- 172.21.15.83:0/941248846 >> 172.21.15.83:0/941248846 conn(0x7fd6100fb610 msgr2=0x7fd6100697f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 -- 172.21.15.83:0/941248846 shutdown_connections 2024-03-11T19:38:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.482+0000 7fd617840640 1 -- 172.21.15.83:0/941248846 wait complete. 2024-03-11T19:38:59.497 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:37:03.034982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:37:04.125352+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:38:59.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:38:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.598+0000 7f3299a4e640 1 -- 172.21.15.83:0/3310935906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32941012f0 msgr2=0x7f3294101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.598+0000 7f3299a4e640 1 --2- 172.21.15.83:0/3310935906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32941012f0 0x7f3294101770 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f327c009900 tx=0x7f327c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.598+0000 7f3299a4e640 1 -- 172.21.15.83:0/3310935906 shutdown_connections 2024-03-11T19:38:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.598+0000 7f3299a4e640 1 --2- 172.21.15.83:0/3310935906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3294101cb0 0x7f3294108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.598+0000 7f3299a4e640 1 --2- 172.21.15.83:0/3310935906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32941012f0 0x7f3294101770 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.598+0000 7f3299a4e640 1 --2- 172.21.15.83:0/3310935906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3294100100 0x7f3294100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.598+0000 7f3299a4e640 1 -- 172.21.15.83:0/3310935906 >> 172.21.15.83:0/3310935906 conn(0x7f32940fb6b0 msgr2=0x7f32940fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.598+0000 7f3299a4e640 1 -- 172.21.15.83:0/3310935906 shutdown_connections 2024-03-11T19:38:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.598+0000 7f3299a4e640 1 -- 172.21.15.83:0/3310935906 wait complete. 2024-03-11T19:38:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3299a4e640 1 Processor -- start 2024-03-11T19:38:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3299a4e640 1 -- start start 2024-03-11T19:38:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3299a4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3294100100 0x7f329406a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3299a4e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32941012f0 0x7f329419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3299a4e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3294101cb0 0x7f329419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3299a4e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329419ed40 con 0x7f3294100100 2024-03-11T19:38:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3299a4e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329419eeb0 con 0x7f3294101cb0 2024-03-11T19:38:59.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3299a4e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329419f070 con 0x7f32941012f0 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3292ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3294100100 0x7f329406a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3292ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3294100100 0x7f329406a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50416/0 (socket says 172.21.15.83:50416) 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3292ffd640 1 -- 172.21.15.83:0/4193971963 learned_addr learned my addr 172.21.15.83:0/4193971963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f32937fe640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3294101cb0 0x7f329419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f328bfff640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32941012f0 0x7f329419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3292ffd640 1 -- 172.21.15.83:0/4193971963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32941012f0 msgr2=0x7f329419bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3292ffd640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32941012f0 0x7f329419bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3292ffd640 1 -- 172.21.15.83:0/4193971963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3294101cb0 msgr2=0x7f329419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3292ffd640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3294101cb0 0x7f329419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3292ffd640 1 -- 172.21.15.83:0/4193971963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c009560 con 0x7f3294100100 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f32937fe640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3294101cb0 0x7f329419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f328bfff640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32941012f0 0x7f329419bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:38:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.599+0000 7f3292ffd640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3294100100 0x7f329406a280 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f328400e9c0 tx=0x7f328400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.600+0000 7f3298a4c640 1 -- 172.21.15.83:0/4193971963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284009700 con 0x7f3294100100 2024-03-11T19:38:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.600+0000 7f3298a4c640 1 -- 172.21.15.83:0/4193971963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f328400cd80 con 0x7f3294100100 2024-03-11T19:38:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.600+0000 7f3299a4e640 1 -- 172.21.15.83:0/4193971963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329419f350 con 0x7f3294100100 2024-03-11T19:38:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.600+0000 7f3298a4c640 1 -- 172.21.15.83:0/4193971963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284010430 con 0x7f3294100100 2024-03-11T19:38:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.600+0000 7f3299a4e640 1 -- 172.21.15.83:0/4193971963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329419f860 con 0x7f3294100100 2024-03-11T19:38:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.601+0000 7f3299a4e640 1 -- 172.21.15.83:0/4193971963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3294107470 con 0x7f3294100100 2024-03-11T19:38:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.602+0000 7f3298a4c640 1 -- 172.21.15.83:0/4193971963 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3284003680 con 0x7f3294100100 2024-03-11T19:38:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.602+0000 7f3298a4c640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32800a1a30 0x7f32800a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.602+0000 7f328bfff640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32800a1a30 0x7f32800a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.602+0000 7f3298a4c640 1 -- 172.21.15.83:0/4193971963 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3284014070 con 0x7f3294100100 2024-03-11T19:38:59.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.603+0000 7f328bfff640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32800a1a30 0x7f32800a3ef0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f327c036040 tx=0x7f327c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:59.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.606+0000 7f3298a4c640 1 -- 172.21.15.83:0/4193971963 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3284096c80 con 0x7f3294100100 2024-03-11T19:38:59.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.778+0000 7f3299a4e640 1 -- 172.21.15.83:0/4193971963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f329410bc30 con 0x7f3294100100 2024-03-11T19:38:59.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.778+0000 7f3298a4c640 1 -- 172.21.15.83:0/4193971963 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f328409a340 con 0x7f3294100100 2024-03-11T19:38:59.783 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:38:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.780+0000 7f3299a4e640 1 -- 172.21.15.83:0/4193971963 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32800a1a30 msgr2=0x7f32800a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.780+0000 7f3299a4e640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32800a1a30 0x7f32800a3ef0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f327c036040 tx=0x7f327c000bc0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.780+0000 7f3299a4e640 1 -- 172.21.15.83:0/4193971963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3294100100 msgr2=0x7f329406a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.780+0000 7f3299a4e640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3294100100 0x7f329406a280 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f328400e9c0 tx=0x7f328400ee80 comp rx=0 tx=0).stop 2024-03-11T19:38:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.780+0000 7f3299a4e640 1 -- 172.21.15.83:0/4193971963 shutdown_connections 2024-03-11T19:38:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.780+0000 7f3299a4e640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f32800a1a30 0x7f32800a3ef0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.780+0000 7f3299a4e640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3294101cb0 0x7f329419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.780+0000 7f3299a4e640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32941012f0 0x7f329419bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.780+0000 7f3299a4e640 1 --2- 172.21.15.83:0/4193971963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3294100100 0x7f329406a280 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.780+0000 7f3299a4e640 1 -- 172.21.15.83:0/4193971963 >> 172.21.15.83:0/4193971963 conn(0x7f32940fb6b0 msgr2=0x7f3294104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.781+0000 7f3299a4e640 1 -- 172.21.15.83:0/4193971963 shutdown_connections 2024-03-11T19:38:59.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.781+0000 7f3299a4e640 1 -- 172.21.15.83:0/4193971963 wait complete. 2024-03-11T19:38:59.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:38:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.895+0000 7fddbe081640 1 -- 172.21.15.83:0/2187691278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8102780 msgr2=0x7fddb8102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.895+0000 7fddbe081640 1 --2- 172.21.15.83:0/2187691278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8102780 0x7fddb8102ba0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009900 tx=0x7fdda802bc30 comp rx=0 tx=0).stop 2024-03-11T19:38:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.895+0000 7fddbe081640 1 -- 172.21.15.83:0/2187691278 shutdown_connections 2024-03-11T19:38:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.895+0000 7fddbe081640 1 --2- 172.21.15.83:0/2187691278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddb80ffbb0 0x7fddb8109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.895+0000 7fddbe081640 1 --2- 172.21.15.83:0/2187691278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb80ff1f0 0x7fddb80ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.895+0000 7fddbe081640 1 --2- 172.21.15.83:0/2187691278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8102780 0x7fddb8102ba0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.895+0000 7fddbe081640 1 -- 172.21.15.83:0/2187691278 >> 172.21.15.83:0/2187691278 conn(0x7fddb80fadb0 msgr2=0x7fddb80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:38:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.895+0000 7fddbe081640 1 -- 172.21.15.83:0/2187691278 shutdown_connections 2024-03-11T19:38:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.895+0000 7fddbe081640 1 -- 172.21.15.83:0/2187691278 wait complete. 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.896+0000 7fddbe081640 1 Processor -- start 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.896+0000 7fddbe081640 1 -- start start 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.896+0000 7fddbe081640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddb80ff1f0 0x7fddb8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.896+0000 7fddbe081640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb80ffbb0 0x7fddb8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.896+0000 7fddbe081640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8102780 0x7fddb819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.896+0000 7fddbe081640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb819ed20 con 0x7fddb8102780 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.896+0000 7fddbe081640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb819ee90 con 0x7fddb80ffbb0 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.896+0000 7fddbe081640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb819f050 con 0x7fddb80ff1f0 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.896+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8102780 0x7fddb819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.896+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8102780 0x7fddb819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50426/0 (socket says 172.21.15.83:50426) 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.896+0000 7fddb7fff640 1 -- 172.21.15.83:0/999102514 learned_addr learned my addr 172.21.15.83:0/999102514 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddaeffd640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb80ffbb0 0x7fddb8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddb77fe640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddb80ff1f0 0x7fddb8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddb7fff640 1 -- 172.21.15.83:0/999102514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddb80ff1f0 msgr2=0x7fddb8197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddb7fff640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddb80ff1f0 0x7fddb8197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddb7fff640 1 -- 172.21.15.83:0/999102514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb80ffbb0 msgr2=0x7fddb8197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddb7fff640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb80ffbb0 0x7fddb8197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddb7fff640 1 -- 172.21.15.83:0/999102514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda8009560 con 0x7fddb8102780 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddb7fff640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8102780 0x7fddb819e7e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009fd0 tx=0x7fddb000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddb57fa640 1 -- 172.21.15.83:0/999102514 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000bea0 con 0x7fddb8102780 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddbe081640 1 -- 172.21.15.83:0/999102514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb819f330 con 0x7fddb8102780 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddb57fa640 1 -- 172.21.15.83:0/999102514 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fddb0009940 con 0x7fddb8102780 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddb57fa640 1 -- 172.21.15.83:0/999102514 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00106c0 con 0x7fddb8102780 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.897+0000 7fddbe081640 1 -- 172.21.15.83:0/999102514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb819f840 con 0x7fddb8102780 2024-03-11T19:38:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.898+0000 7fddbe081640 1 -- 172.21.15.83:0/999102514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb81086b0 con 0x7fddb8102780 2024-03-11T19:38:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.899+0000 7fddb57fa640 1 -- 172.21.15.83:0/999102514 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fddb0004240 con 0x7fddb8102780 2024-03-11T19:38:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.899+0000 7fddb57fa640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdda40a1a30 0x7fdda40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:38:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.900+0000 7fddb77fe640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdda40a1a30 0x7fdda40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:38:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.900+0000 7fddb57fa640 1 -- 172.21.15.83:0/999102514 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fddb00cd120 con 0x7fddb8102780 2024-03-11T19:38:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.900+0000 7fddb77fe640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdda40a1a30 0x7fdda40a3ef0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdda802c6a0 tx=0x7fdda8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:38:59.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:38:59.904+0000 7fddb57fa640 1 -- 172.21.15.83:0/999102514 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fddb00963e0 con 0x7fddb8102780 2024-03-11T19:39:00.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:59 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1944234065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:39:00.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:59 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/941248846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:39:00.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:38:59 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4193971963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:39:00.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.079+0000 7fddbe081640 1 -- 172.21.15.83:0/999102514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fddb802cdd0 con 0x7fddb8102780 2024-03-11T19:39:00.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.080+0000 7fddb57fa640 1 -- 172.21.15.83:0/999102514 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fddb0099aa0 con 0x7fddb8102780 2024-03-11T19:39:00.084 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-03-11T19:39:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 -- 172.21.15.83:0/999102514 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdda40a1a30 msgr2=0x7fdda40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdda40a1a30 0x7fdda40a3ef0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdda802c6a0 tx=0x7fdda8005fd0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 -- 172.21.15.83:0/999102514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8102780 msgr2=0x7fddb819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8102780 0x7fddb819e7e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009fd0 tx=0x7fddb000b260 comp rx=0 tx=0).stop 2024-03-11T19:39:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 -- 172.21.15.83:0/999102514 shutdown_connections 2024-03-11T19:39:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdda40a1a30 0x7fdda40a3ef0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddb8102780 0x7fddb819e7e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fddb80ffbb0 0x7fddb8197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 --2- 172.21.15.83:0/999102514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddb80ff1f0 0x7fddb8197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 -- 172.21.15.83:0/999102514 >> 172.21.15.83:0/999102514 conn(0x7fddb80fadb0 msgr2=0x7fddb8105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 -- 172.21.15.83:0/999102514 shutdown_connections 2024-03-11T19:39:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.082+0000 7fddbe081640 1 -- 172.21.15.83:0/999102514 wait complete. 2024-03-11T19:39:00.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:39:00.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.198+0000 7f4fa534f640 1 -- 172.21.15.83:0/708560426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa0101270 msgr2=0x7f4fa01016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.198+0000 7f4fa534f640 1 --2- 172.21.15.83:0/708560426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa0101270 0x7f4fa01016f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009900 tx=0x7f4f8802bc30 comp rx=0 tx=0).stop 2024-03-11T19:39:00.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.198+0000 7f4fa534f640 1 -- 172.21.15.83:0/708560426 shutdown_connections 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.198+0000 7f4fa534f640 1 --2- 172.21.15.83:0/708560426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa0101c30 0x7f4fa01084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.198+0000 7f4fa534f640 1 --2- 172.21.15.83:0/708560426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa0101270 0x7f4fa01016f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.198+0000 7f4fa534f640 1 --2- 172.21.15.83:0/708560426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa0100080 0x7f4fa01004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.198+0000 7f4fa534f640 1 -- 172.21.15.83:0/708560426 >> 172.21.15.83:0/708560426 conn(0x7f4fa00fb610 msgr2=0x7f4fa00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.198+0000 7f4fa534f640 1 -- 172.21.15.83:0/708560426 shutdown_connections 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.198+0000 7f4fa534f640 1 -- 172.21.15.83:0/708560426 wait complete. 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.198+0000 7f4fa534f640 1 Processor -- start 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.198+0000 7f4fa534f640 1 -- start start 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4fa534f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa0100080 0x7f4fa0197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4fa534f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa0101270 0x7f4fa0197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4fa534f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa0101c30 0x7f4fa019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4fa534f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa019ed20 con 0x7f4fa0101c30 2024-03-11T19:39:00.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4fa534f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa019ee90 con 0x7f4fa0101270 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4fa534f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa019f050 con 0x7f4fa0100080 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa0101c30 0x7f4fa019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa0101c30 0x7f4fa019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50452/0 (socket says 172.21.15.83:50452) 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9f7fe640 1 -- 172.21.15.83:0/3063013440 learned_addr learned my addr 172.21.15.83:0/3063013440 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f97fff640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa0101270 0x7f4fa0197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9effd640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa0100080 0x7f4fa0197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9f7fe640 1 -- 172.21.15.83:0/3063013440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa0100080 msgr2=0x7f4fa0197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9f7fe640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa0100080 0x7f4fa0197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9f7fe640 1 -- 172.21.15.83:0/3063013440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa0101270 msgr2=0x7f4fa0197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9f7fe640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa0101270 0x7f4fa0197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9f7fe640 1 -- 172.21.15.83:0/3063013440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f88009560 con 0x7f4fa0101c30 2024-03-11T19:39:00.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f97fff640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa0101270 0x7f4fa0197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:39:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9effd640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa0100080 0x7f4fa0197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:39:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9f7fe640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa0101c30 0x7f4fa019e7e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800eaa0 tx=0x7f4f9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9cff9640 1 -- 172.21.15.83:0/3063013440 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800cdb0 con 0x7f4fa0101c30 2024-03-11T19:39:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4fa534f640 1 -- 172.21.15.83:0/3063013440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa019f330 con 0x7f4fa0101c30 2024-03-11T19:39:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9cff9640 1 -- 172.21.15.83:0/3063013440 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f98005020 con 0x7f4fa0101c30 2024-03-11T19:39:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4fa534f640 1 -- 172.21.15.83:0/3063013440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa019f890 con 0x7f4fa0101c30 2024-03-11T19:39:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.199+0000 7f4f9cff9640 1 -- 172.21.15.83:0/3063013440 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98011670 con 0x7f4fa0101c30 2024-03-11T19:39:00.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.200+0000 7f4fa534f640 1 -- 172.21.15.83:0/3063013440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa0107420 con 0x7f4fa0101c30 2024-03-11T19:39:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.201+0000 7f4f9cff9640 1 -- 172.21.15.83:0/3063013440 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4f98005490 con 0x7f4fa0101c30 2024-03-11T19:39:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.202+0000 7f4f9cff9640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f8c0a1a30 0x7f4f8c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.202+0000 7f4f9effd640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f8c0a1a30 0x7f4f8c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.202+0000 7f4f9cff9640 1 -- 172.21.15.83:0/3063013440 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4f980cd8a0 con 0x7f4fa0101c30 2024-03-11T19:39:00.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.202+0000 7f4f9effd640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f8c0a1a30 0x7f4f8c0a3ef0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f90005d00 tx=0x7f4f90005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:00.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.206+0000 7f4f9cff9640 1 -- 172.21.15.83:0/3063013440 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f98096b60 con 0x7f4fa0101c30 2024-03-11T19:39:00.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.379+0000 7f4fa534f640 1 -- 172.21.15.83:0/3063013440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fa002ce40 con 0x7f4fa0101c30 2024-03-11T19:39:00.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.379+0000 7f4f9cff9640 1 -- 172.21.15.83:0/3063013440 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f9801f0a0 con 0x7f4fa0101c30 2024-03-11T19:39:00.384 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-03-11T19:39:00.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.381+0000 7f4fa534f640 1 -- 172.21.15.83:0/3063013440 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f8c0a1a30 msgr2=0x7f4f8c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.381+0000 7f4fa534f640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f8c0a1a30 0x7f4f8c0a3ef0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f90005d00 tx=0x7f4f90005ef0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.381+0000 7f4fa534f640 1 -- 172.21.15.83:0/3063013440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa0101c30 msgr2=0x7f4fa019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.381+0000 7f4fa534f640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa0101c30 0x7f4fa019e7e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800eaa0 tx=0x7f4f9800ef60 comp rx=0 tx=0).stop 2024-03-11T19:39:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.381+0000 7f4fa534f640 1 -- 172.21.15.83:0/3063013440 shutdown_connections 2024-03-11T19:39:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.381+0000 7f4fa534f640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4f8c0a1a30 0x7f4f8c0a3ef0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.381+0000 7f4fa534f640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa0101c30 0x7f4fa019e7e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.381+0000 7f4fa534f640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fa0101270 0x7f4fa0197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.381+0000 7f4fa534f640 1 --2- 172.21.15.83:0/3063013440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa0100080 0x7f4fa0197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.381+0000 7f4fa534f640 1 -- 172.21.15.83:0/3063013440 >> 172.21.15.83:0/3063013440 conn(0x7f4fa00fb610 msgr2=0x7f4fa01045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.382+0000 7f4fa534f640 1 -- 172.21.15.83:0/3063013440 shutdown_connections 2024-03-11T19:39:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.382+0000 7f4fa534f640 1 -- 172.21.15.83:0/3063013440 wait complete. 2024-03-11T19:39:00.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:59 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1944234065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:39:00.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:59 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/941248846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:39:00.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:38:59 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4193971963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:39:00.396 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-11T19:39:00.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:39:00.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:59 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1944234065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:39:00.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:59 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/941248846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:39:00.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:38:59 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4193971963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:39:00.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.495+0000 7f38c3bb5640 1 -- 172.21.15.83:0/248428277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38bc102d50 msgr2=0x7f38bc1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.495+0000 7f38c3bb5640 1 --2- 172.21.15.83:0/248428277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38bc102d50 0x7f38bc1095e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009ca0 tx=0x7f38b402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.495+0000 7f38c3bb5640 1 -- 172.21.15.83:0/248428277 shutdown_connections 2024-03-11T19:39:00.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.495+0000 7f38c3bb5640 1 --2- 172.21.15.83:0/248428277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38bc102d50 0x7f38bc1095e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.495+0000 7f38c3bb5640 1 --2- 172.21.15.83:0/248428277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38bc102390 0x7f38bc102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.495+0000 7f38c3bb5640 1 --2- 172.21.15.83:0/248428277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38bc1011a0 0x7f38bc1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.495+0000 7f38c3bb5640 1 -- 172.21.15.83:0/248428277 >> 172.21.15.83:0/248428277 conn(0x7f38bc0fc730 msgr2=0x7f38bc0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.495+0000 7f38c3bb5640 1 -- 172.21.15.83:0/248428277 shutdown_connections 2024-03-11T19:39:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.495+0000 7f38c3bb5640 1 -- 172.21.15.83:0/248428277 wait complete. 2024-03-11T19:39:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.495+0000 7f38c3bb5640 1 Processor -- start 2024-03-11T19:39:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.495+0000 7f38c3bb5640 1 -- start start 2024-03-11T19:39:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c3bb5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38bc1011a0 0x7f38bc074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c3bb5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38bc102390 0x7f38bc073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c3bb5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38bc102d50 0x7f38bc073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c3bb5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc074210 con 0x7f38bc1011a0 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c3bb5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc074380 con 0x7f38bc102390 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c3bb5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc1a1200 con 0x7f38bc102d50 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c1951640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38bc1011a0 0x7f38bc074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c1951640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38bc1011a0 0x7f38bc074f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50464/0 (socket says 172.21.15.83:50464) 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c1951640 1 -- 172.21.15.83:0/1136741782 learned_addr learned my addr 172.21.15.83:0/1136741782 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c1150640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38bc102390 0x7f38bc073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c2152640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38bc102d50 0x7f38bc073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c1951640 1 -- 172.21.15.83:0/1136741782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38bc102d50 msgr2=0x7f38bc073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c1951640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38bc102d50 0x7f38bc073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c1951640 1 -- 172.21.15.83:0/1136741782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38bc102390 msgr2=0x7f38bc073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c1951640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38bc102390 0x7f38bc073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c1951640 1 -- 172.21.15.83:0/1136741782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b4009900 con 0x7f38bc1011a0 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c1951640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38bc1011a0 0x7f38bc074f30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f38b800d7a0 tx=0x7f38b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:00.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38b2ffd640 1 -- 172.21.15.83:0/1136741782 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8009c30 con 0x7f38bc1011a0 2024-03-11T19:39:00.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38b2ffd640 1 -- 172.21.15.83:0/1136741782 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38b800deb0 con 0x7f38bc1011a0 2024-03-11T19:39:00.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c3bb5640 1 -- 172.21.15.83:0/1136741782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38bc1a14b0 con 0x7f38bc1011a0 2024-03-11T19:39:00.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38b2ffd640 1 -- 172.21.15.83:0/1136741782 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8010460 con 0x7f38bc1011a0 2024-03-11T19:39:00.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.496+0000 7f38c3bb5640 1 -- 172.21.15.83:0/1136741782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc1a19c0 con 0x7f38bc1011a0 2024-03-11T19:39:00.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.497+0000 7f38c3bb5640 1 -- 172.21.15.83:0/1136741782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38bc1084c0 con 0x7f38bc1011a0 2024-03-11T19:39:00.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.498+0000 7f38b2ffd640 1 -- 172.21.15.83:0/1136741782 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f38b8003ef0 con 0x7f38bc1011a0 2024-03-11T19:39:00.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.499+0000 7f38b2ffd640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38a80a1a30 0x7f38a80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.499+0000 7f38c1150640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38a80a1a30 0x7f38a80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:00.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.499+0000 7f38b2ffd640 1 -- 172.21.15.83:0/1136741782 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f38b80cd040 con 0x7f38bc1011a0 2024-03-11T19:39:00.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.499+0000 7f38c1150640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38a80a1a30 0x7f38a80a3ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38bc074ca0 tx=0x7f38ac00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:00.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.503+0000 7f38b2ffd640 1 -- 172.21.15.83:0/1136741782 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f38b80963b0 con 0x7f38bc1011a0 2024-03-11T19:39:00.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.712+0000 7f38c3bb5640 1 -- 172.21.15.83:0/1136741782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38bc10cc90 con 0x7f38bc1011a0 2024-03-11T19:39:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.713+0000 7f38b2ffd640 1 -- 172.21.15.83:0/1136741782 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f38b8020020 con 0x7f38bc1011a0 2024-03-11T19:39:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:00.717 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:38:57.908337+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:39:00.717 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 22 2024-03-11T19:39:00.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 -- 172.21.15.83:0/1136741782 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38a80a1a30 msgr2=0x7f38a80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38a80a1a30 0x7f38a80a3ef0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38bc074ca0 tx=0x7f38ac00b390 comp rx=0 tx=0).stop 2024-03-11T19:39:00.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 -- 172.21.15.83:0/1136741782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38bc1011a0 msgr2=0x7f38bc074f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38bc1011a0 0x7f38bc074f30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f38b800d7a0 tx=0x7f38b800dc60 comp rx=0 tx=0).stop 2024-03-11T19:39:00.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 -- 172.21.15.83:0/1136741782 shutdown_connections 2024-03-11T19:39:00.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f38a80a1a30 0x7f38a80a3ef0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38bc102d50 0x7f38bc073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f38bc102390 0x7f38bc073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 --2- 172.21.15.83:0/1136741782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38bc1011a0 0x7f38bc074f30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 -- 172.21.15.83:0/1136741782 >> 172.21.15.83:0/1136741782 conn(0x7f38bc0fc730 msgr2=0x7f38bc105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:00.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 -- 172.21.15.83:0/1136741782 shutdown_connections 2024-03-11T19:39:00.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.715+0000 7f38c3bb5640 1 -- 172.21.15.83:0/1136741782 wait complete. 2024-03-11T19:39:00.730 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-11T19:39:00.730 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-11T19:39:00.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T19:39:00.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.827+0000 7f4cd312c640 1 -- 172.21.15.83:0/1260453501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ccc1000a0 msgr2=0x7f4ccc1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.827+0000 7f4cd312c640 1 --2- 172.21.15.83:0/1260453501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ccc1000a0 0x7f4ccc1004c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009900 tx=0x7f4cbc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.827+0000 7f4cd312c640 1 -- 172.21.15.83:0/1260453501 shutdown_connections 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.827+0000 7f4cd312c640 1 --2- 172.21.15.83:0/1260453501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ccc101c50 0x7f4ccc1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.827+0000 7f4cd312c640 1 --2- 172.21.15.83:0/1260453501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ccc101290 0x7f4ccc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.827+0000 7f4cd312c640 1 --2- 172.21.15.83:0/1260453501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ccc1000a0 0x7f4ccc1004c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.827+0000 7f4cd312c640 1 -- 172.21.15.83:0/1260453501 >> 172.21.15.83:0/1260453501 conn(0x7f4ccc0fb610 msgr2=0x7f4ccc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.827+0000 7f4cd312c640 1 -- 172.21.15.83:0/1260453501 shutdown_connections 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.827+0000 7f4cd312c640 1 -- 172.21.15.83:0/1260453501 wait complete. 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.828+0000 7f4cd312c640 1 Processor -- start 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.828+0000 7f4cd312c640 1 -- start start 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.828+0000 7f4cd312c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ccc1000a0 0x7f4ccc068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.828+0000 7f4cd312c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ccc101290 0x7f4ccc19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.828+0000 7f4cd312c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ccc101c50 0x7f4ccc19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.828+0000 7f4cd312c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc19ecd0 con 0x7f4ccc1000a0 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.828+0000 7f4cd312c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc19ee40 con 0x7f4ccc101290 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.828+0000 7f4cd312c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc19f000 con 0x7f4ccc101c50 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.828+0000 7f4cd0ec8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ccc1000a0 0x7f4ccc068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.828+0000 7f4cd0ec8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ccc1000a0 0x7f4ccc068e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50494/0 (socket says 172.21.15.83:50494) 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.828+0000 7f4cd0ec8640 1 -- 172.21.15.83:0/2483539348 learned_addr learned my addr 172.21.15.83:0/2483539348 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.829+0000 7f4cd0ec8640 1 -- 172.21.15.83:0/2483539348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ccc101c50 msgr2=0x7f4ccc19e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.829+0000 7f4cd0ec8640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ccc101c50 0x7f4ccc19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.829+0000 7f4cd0ec8640 1 -- 172.21.15.83:0/2483539348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ccc101290 msgr2=0x7f4ccc19bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.829+0000 7f4cd0ec8640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ccc101290 0x7f4ccc19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.829+0000 7f4cd0ec8640 1 -- 172.21.15.83:0/2483539348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cbc009560 con 0x7f4ccc1000a0 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.829+0000 7f4cd0ec8640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ccc1000a0 0x7f4ccc068e70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc000c00 tx=0x7f4cbc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:00.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.829+0000 7f4cc9ffb640 1 -- 172.21.15.83:0/2483539348 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc004df0 con 0x7f4ccc1000a0 2024-03-11T19:39:00.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.829+0000 7f4cc9ffb640 1 -- 172.21.15.83:0/2483539348 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4cbc005430 con 0x7f4ccc1000a0 2024-03-11T19:39:00.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.829+0000 7f4cd312c640 1 -- 172.21.15.83:0/2483539348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ccc19f280 con 0x7f4ccc1000a0 2024-03-11T19:39:00.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.829+0000 7f4cc9ffb640 1 -- 172.21.15.83:0/2483539348 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc03e450 con 0x7f4ccc1000a0 2024-03-11T19:39:00.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.829+0000 7f4cd312c640 1 -- 172.21.15.83:0/2483539348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ccc19f7e0 con 0x7f4ccc1000a0 2024-03-11T19:39:00.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.830+0000 7f4cd312c640 1 -- 172.21.15.83:0/2483539348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ccc107360 con 0x7f4ccc1000a0 2024-03-11T19:39:00.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.831+0000 7f4cc9ffb640 1 -- 172.21.15.83:0/2483539348 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4cbc003680 con 0x7f4ccc1000a0 2024-03-11T19:39:00.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.832+0000 7f4cc9ffb640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cb40a1ab0 0x7f4cb40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:00.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.832+0000 7f4ccbfff640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cb40a1ab0 0x7f4cb40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:00.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.832+0000 7f4cc9ffb640 1 -- 172.21.15.83:0/2483539348 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4cbc0ef8b0 con 0x7f4ccc1000a0 2024-03-11T19:39:00.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.832+0000 7f4ccbfff640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cb40a1ab0 0x7f4cb40a3f70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc068be0 tx=0x7f4cc0005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:00.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:00.836+0000 7f4cc9ffb640 1 -- 172.21.15.83:0/2483539348 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4cbc0b8b70 con 0x7f4ccc1000a0 2024-03-11T19:39:01.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.012+0000 7f4cd312c640 1 -- 172.21.15.83:0/2483539348 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ccc02ceb0 con 0x7f4cb40a1ab0 2024-03-11T19:39:01.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.019+0000 7f4cc9ffb640 1 -- 172.21.15.83:0/2483539348 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258839 (secure 0 0 0) 0x7f4ccc02ceb0 con 0x7f4cb40a1ab0 2024-03-11T19:39:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:01.027 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-03-11T19:39:01.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 -- 172.21.15.83:0/2483539348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cb40a1ab0 msgr2=0x7f4cb40a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cb40a1ab0 0x7f4cb40a3f70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc068be0 tx=0x7f4cc0005c70 comp rx=0 tx=0).stop 2024-03-11T19:39:01.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 -- 172.21.15.83:0/2483539348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ccc1000a0 msgr2=0x7f4ccc068e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ccc1000a0 0x7f4ccc068e70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc000c00 tx=0x7f4cbc004270 comp rx=0 tx=0).stop 2024-03-11T19:39:01.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 -- 172.21.15.83:0/2483539348 shutdown_connections 2024-03-11T19:39:01.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cb40a1ab0 0x7f4cb40a3f70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ccc101c50 0x7f4ccc19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ccc101290 0x7f4ccc19bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 --2- 172.21.15.83:0/2483539348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ccc1000a0 0x7f4ccc068e70 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 -- 172.21.15.83:0/2483539348 >> 172.21.15.83:0/2483539348 conn(0x7f4ccc0fb610 msgr2=0x7f4ccc104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:01.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 -- 172.21.15.83:0/2483539348 shutdown_connections 2024-03-11T19:39:01.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.022+0000 7f4cd312c640 1 -- 172.21.15.83:0/2483539348 wait complete. 2024-03-11T19:39:01.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:00 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/999102514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:39:01.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:00 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3063013440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:39:01.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:00 smithi083 ceph-mon[31864]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-11T19:39:01.036 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:00 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1136741782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:39:01.037 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-03-11T19:39:00.535104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655856,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":804,"num_read_kb":518,"num_write":354,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366476,"kb_used_data":13164,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575348,"statfs":{"total":1151940427776,"available":1151565156352,"internally_reserved":0,"allocated":13479936,"data_stored":7427568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002753"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543207+0000","last_change":"2024-03-11T19:37:05.340476+0000","last_active":"2024-03-11T19:38:38.543207+0000","last_peered":"2024-03-11T19:38:38.543207+0000","last_clean":"2024-03-11T19:38:38.543207+0000","last_became_active":"2024-03-11T19:37:05.340140+0000","last_became_peered":"2024-03-11T19:37:05.340140+0000","last_unstale":"2024-03-11T19:38:38.543207+0000","last_undegraded":"2024-03-11T19:38:38.543207+0000","last_fullsized":"2024-03-11T19:38:38.543207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:10:35.677374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.537288+0000","last_change":"2024-03-11T19:37:05.335500+0000","last_active":"2024-03-11T19:38:38.537288+0000","last_peered":"2024-03-11T19:38:38.537288+0000","last_clean":"2024-03-11T19:38:38.537288+0000","last_became_active":"2024-03-11T19:37:05.335194+0000","last_became_peered":"2024-03-11T19:37:05.335194+0000","last_unstale":"2024-03-11T19:38:38.537288+0000","last_undegraded":"2024-03-11T19:38:38.537288+0000","last_fullsized":"2024-03-11T19:38:38.537288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:03.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.555088+0000","last_change":"2024-03-11T19:37:05.334618+0000","last_active":"2024-03-11T19:38:38.555088+0000","last_peered":"2024-03-11T19:38:38.555088+0000","last_clean":"2024-03-11T19:38:38.555088+0000","last_became_active":"2024-03-11T19:37:05.334281+0000","last_became_peered":"2024-03-11T19:37:05.334281+0000","last_unstale":"2024-03-11T19:38:38.555088+0000","last_undegraded":"2024-03-11T19:38:38.555088+0000","last_fullsized":"2024-03-11T19:38:38.555088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:43:22.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.546651+0000","last_change":"2024-03-11T19:37:05.332978+0000","last_active":"2024-03-11T19:38:38.546651+0000","last_peered":"2024-03-11T19:38:38.546651+0000","last_clean":"2024-03-11T19:38:38.546651+0000","last_became_active":"2024-03-11T19:37:05.330888+0000","last_became_peered":"2024-03-11T19:37:05.330888+0000","last_unstale":"2024-03-11T19:38:38.546651+0000","last_undegraded":"2024-03-11T19:38:38.546651+0000","last_fullsized":"2024-03-11T19:38:38.546651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:40:36.988348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536573+0000","last_change":"2024-03-11T19:37:05.329050+0000","last_active":"2024-03-11T19:38:38.536573+0000","last_peered":"2024-03-11T19:38:38.536573+0000","last_clean":"2024-03-11T19:38:38.536573+0000","last_became_active":"2024-03-11T19:37:05.328702+0000","last_became_peered":"2024-03-11T19:37:05.328702+0000","last_unstale":"2024-03-11T19:38:38.536573+0000","last_undegraded":"2024-03-11T19:38:38.536573+0000","last_fullsized":"2024-03-11T19:38:38.536573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:19:52.180884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549103+0000","last_change":"2024-03-11T19:37:05.322748+0000","last_active":"2024-03-11T19:38:38.549103+0000","last_peered":"2024-03-11T19:38:38.549103+0000","last_clean":"2024-03-11T19:38:38.549103+0000","last_became_active":"2024-03-11T19:37:05.322376+0000","last_became_peered":"2024-03-11T19:37:05.322376+0000","last_unstale":"2024-03-11T19:38:38.549103+0000","last_undegraded":"2024-03-11T19:38:38.549103+0000","last_fullsized":"2024-03-11T19:38:38.549103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:40.926854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543305+0000","last_change":"2024-03-11T19:37:05.323954+0000","last_active":"2024-03-11T19:38:38.543305+0000","last_peered":"2024-03-11T19:38:38.543305+0000","last_clean":"2024-03-11T19:38:38.543305+0000","last_became_active":"2024-03-11T19:37:05.323586+0000","last_became_peered":"2024-03-11T19:37:05.323586+0000","last_unstale":"2024-03-11T19:38:38.543305+0000","last_undegraded":"2024-03-11T19:38:38.543305+0000","last_fullsized":"2024-03-11T19:38:38.543305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:38.286891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536729+0000","last_change":"2024-03-11T19:37:05.334138+0000","last_active":"2024-03-11T19:38:38.536729+0000","last_peered":"2024-03-11T19:38:38.536729+0000","last_clean":"2024-03-11T19:38:38.536729+0000","last_became_active":"2024-03-11T19:37:05.333837+0000","last_became_peered":"2024-03-11T19:37:05.333837+0000","last_unstale":"2024-03-11T19:38:38.536729+0000","last_undegraded":"2024-03-11T19:38:38.536729+0000","last_fullsized":"2024-03-11T19:38:38.536729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:49:30.724176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.552174+0000","last_change":"2024-03-11T19:37:05.336888+0000","last_active":"2024-03-11T19:38:38.552174+0000","last_peered":"2024-03-11T19:38:38.552174+0000","last_clean":"2024-03-11T19:38:38.552174+0000","last_became_active":"2024-03-11T19:37:05.336256+0000","last_became_peered":"2024-03-11T19:37:05.336256+0000","last_unstale":"2024-03-11T19:38:38.552174+0000","last_undegraded":"2024-03-11T19:38:38.552174+0000","last_fullsized":"2024-03-11T19:38:38.552174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:54:31.526986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548957+0000","last_change":"2024-03-11T19:37:05.321959+0000","last_active":"2024-03-11T19:38:38.548957+0000","last_peered":"2024-03-11T19:38:38.548957+0000","last_clean":"2024-03-11T19:38:38.548957+0000","last_became_active":"2024-03-11T19:37:05.321043+0000","last_became_peered":"2024-03-11T19:37:05.321043+0000","last_unstale":"2024-03-11T19:38:38.548957+0000","last_undegraded":"2024-03-11T19:38:38.548957+0000","last_fullsized":"2024-03-11T19:38:38.548957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:13.121003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548202+0000","last_change":"2024-03-11T19:37:05.324940+0000","last_active":"2024-03-11T19:38:38.548202+0000","last_peered":"2024-03-11T19:38:38.548202+0000","last_clean":"2024-03-11T19:38:38.548202+0000","last_became_active":"2024-03-11T19:37:05.323427+0000","last_became_peered":"2024-03-11T19:37:05.323427+0000","last_unstale":"2024-03-11T19:38:38.548202+0000","last_undegraded":"2024-03-11T19:38:38.548202+0000","last_fullsized":"2024-03-11T19:38:38.548202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:23.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548766+0000","last_change":"2024-03-11T19:37:05.332694+0000","last_active":"2024-03-11T19:38:38.548766+0000","last_peered":"2024-03-11T19:38:38.548766+0000","last_clean":"2024-03-11T19:38:38.548766+0000","last_became_active":"2024-03-11T19:37:05.330446+0000","last_became_peered":"2024-03-11T19:37:05.330446+0000","last_unstale":"2024-03-11T19:38:38.548766+0000","last_undegraded":"2024-03-11T19:38:38.548766+0000","last_fullsized":"2024-03-11T19:38:38.548766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:37:40.196933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535856+0000","last_change":"2024-03-11T19:37:05.325975+0000","last_active":"2024-03-11T19:38:38.535856+0000","last_peered":"2024-03-11T19:38:38.535856+0000","last_clean":"2024-03-11T19:38:38.535856+0000","last_became_active":"2024-03-11T19:37:05.325649+0000","last_became_peered":"2024-03-11T19:37:05.325649+0000","last_unstale":"2024-03-11T19:38:38.535856+0000","last_undegraded":"2024-03-11T19:38:38.535856+0000","last_fullsized":"2024-03-11T19:38:38.535856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:36.373272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.547237+0000","last_change":"2024-03-11T19:37:05.335546+0000","last_active":"2024-03-11T19:38:38.547237+0000","last_peered":"2024-03-11T19:38:38.547237+0000","last_clean":"2024-03-11T19:38:38.547237+0000","last_became_active":"2024-03-11T19:37:05.335220+0000","last_became_peered":"2024-03-11T19:37:05.335220+0000","last_unstale":"2024-03-11T19:38:38.547237+0000","last_undegraded":"2024-03-11T19:38:38.547237+0000","last_fullsized":"2024-03-11T19:38:38.547237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:13.116355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.546152+0000","last_change":"2024-03-11T19:37:05.338955+0000","last_active":"2024-03-11T19:38:38.546152+0000","last_peered":"2024-03-11T19:38:38.546152+0000","last_clean":"2024-03-11T19:38:38.546152+0000","last_became_active":"2024-03-11T19:37:05.338574+0000","last_became_peered":"2024-03-11T19:37:05.338574+0000","last_unstale":"2024-03-11T19:38:38.546152+0000","last_undegraded":"2024-03-11T19:38:38.546152+0000","last_fullsized":"2024-03-11T19:38:38.546152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:27:27.754077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553899+0000","last_change":"2024-03-11T19:37:05.334578+0000","last_active":"2024-03-11T19:38:38.553899+0000","last_peered":"2024-03-11T19:38:38.553899+0000","last_clean":"2024-03-11T19:38:38.553899+0000","last_became_active":"2024-03-11T19:37:05.334267+0000","last_became_peered":"2024-03-11T19:37:05.334267+0000","last_unstale":"2024-03-11T19:38:38.553899+0000","last_undegraded":"2024-03-11T19:38:38.553899+0000","last_fullsized":"2024-03-11T19:38:38.553899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:31.704187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543563+0000","last_change":"2024-03-11T19:37:05.341178+0000","last_active":"2024-03-11T19:38:38.543563+0000","last_peered":"2024-03-11T19:38:38.543563+0000","last_clean":"2024-03-11T19:38:38.543563+0000","last_became_active":"2024-03-11T19:37:05.340837+0000","last_became_peered":"2024-03-11T19:37:05.340837+0000","last_unstale":"2024-03-11T19:38:38.543563+0000","last_undegraded":"2024-03-11T19:38:38.543563+0000","last_fullsized":"2024-03-11T19:38:38.543563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:39:54.460937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.538263+0000","last_change":"2024-03-11T19:37:05.330981+0000","last_active":"2024-03-11T19:38:38.538263+0000","last_peered":"2024-03-11T19:38:38.538263+0000","last_clean":"2024-03-11T19:38:38.538263+0000","last_became_active":"2024-03-11T19:37:05.330638+0000","last_became_peered":"2024-03-11T19:37:05.330638+0000","last_unstale":"2024-03-11T19:38:38.538263+0000","last_undegraded":"2024-03-11T19:38:38.538263+0000","last_fullsized":"2024-03-11T19:38:38.538263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:36:12.565936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535474+0000","last_change":"2024-03-11T19:37:05.329866+0000","last_active":"2024-03-11T19:38:38.535474+0000","last_peered":"2024-03-11T19:38:38.535474+0000","last_clean":"2024-03-11T19:38:38.535474+0000","last_became_active":"2024-03-11T19:37:05.329572+0000","last_became_peered":"2024-03-11T19:37:05.329572+0000","last_unstale":"2024-03-11T19:38:38.535474+0000","last_undegraded":"2024-03-11T19:38:38.535474+0000","last_fullsized":"2024-03-11T19:38:38.535474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:40:04.482929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454096+0000","last_change":"2024-03-11T19:37:04.587149+0000","last_active":"2024-03-11T19:38:10.454096+0000","last_peered":"2024-03-11T19:38:10.454096+0000","last_clean":"2024-03-11T19:38:10.454096+0000","last_became_active":"2024-03-11T19:37:04.586013+0000","last_became_peered":"2024-03-11T19:37:04.586013+0000","last_unstale":"2024-03-11T19:38:10.454096+0000","last_undegraded":"2024-03-11T19:38:10.454096+0000","last_fullsized":"2024-03-11T19:38:10.454096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:15.758154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536238+0000","last_change":"2024-03-11T19:37:05.327979+0000","last_active":"2024-03-11T19:38:38.536238+0000","last_peered":"2024-03-11T19:38:38.536238+0000","last_clean":"2024-03-11T19:38:38.536238+0000","last_became_active":"2024-03-11T19:37:05.327647+0000","last_became_peered":"2024-03-11T19:37:05.327647+0000","last_unstale":"2024-03-11T19:38:38.536238+0000","last_undegraded":"2024-03-11T19:38:38.536238+0000","last_fullsized":"2024-03-11T19:38:38.536238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:09.514766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452950+0000","last_change":"2024-03-11T19:37:04.582937+0000","last_active":"2024-03-11T19:38:10.452950+0000","last_peered":"2024-03-11T19:38:10.452950+0000","last_clean":"2024-03-11T19:38:10.452950+0000","last_became_active":"2024-03-11T19:37:04.582622+0000","last_became_peered":"2024-03-11T19:37:04.582622+0000","last_unstale":"2024-03-11T19:38:10.452950+0000","last_undegraded":"2024-03-11T19:38:10.452950+0000","last_fullsized":"2024-03-11T19:38:10.452950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:02.275698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.555090+0000","last_change":"2024-03-11T19:37:05.327191+0000","last_active":"2024-03-11T19:38:38.555090+0000","last_peered":"2024-03-11T19:38:38.555090+0000","last_clean":"2024-03-11T19:38:38.555090+0000","last_became_active":"2024-03-11T19:37:05.326001+0000","last_became_peered":"2024-03-11T19:37:05.326001+0000","last_unstale":"2024-03-11T19:38:38.555090+0000","last_undegraded":"2024-03-11T19:38:38.555090+0000","last_fullsized":"2024-03-11T19:38:38.555090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:27.301321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151969+0000","last_change":"2024-03-11T19:37:04.587054+0000","last_active":"2024-03-11T19:38:10.151969+0000","last_peered":"2024-03-11T19:38:10.151969+0000","last_clean":"2024-03-11T19:38:10.151969+0000","last_became_active":"2024-03-11T19:37:04.586104+0000","last_became_peered":"2024-03-11T19:37:04.586104+0000","last_unstale":"2024-03-11T19:38:10.151969+0000","last_undegraded":"2024-03-11T19:38:10.151969+0000","last_fullsized":"2024-03-11T19:38:10.151969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:34.913677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549578+0000","last_change":"2024-03-11T19:37:05.333288+0000","last_active":"2024-03-11T19:38:38.549578+0000","last_peered":"2024-03-11T19:38:38.549578+0000","last_clean":"2024-03-11T19:38:38.549578+0000","last_became_active":"2024-03-11T19:37:05.332882+0000","last_became_peered":"2024-03-11T19:37:05.332882+0000","last_unstale":"2024-03-11T19:38:38.549578+0000","last_undegraded":"2024-03-11T19:38:38.549578+0000","last_fullsized":"2024-03-11T19:38:38.549578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:45.680608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151628+0000","last_change":"2024-03-11T19:37:04.583660+0000","last_active":"2024-03-11T19:38:10.151628+0000","last_peered":"2024-03-11T19:38:10.151628+0000","last_clean":"2024-03-11T19:38:10.151628+0000","last_became_active":"2024-03-11T19:37:04.583360+0000","last_became_peered":"2024-03-11T19:37:04.583360+0000","last_unstale":"2024-03-11T19:38:10.151628+0000","last_undegraded":"2024-03-11T19:38:10.151628+0000","last_fullsized":"2024-03-11T19:38:10.151628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:00.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.554758+0000","last_change":"2024-03-11T19:37:05.335649+0000","last_active":"2024-03-11T19:38:38.554758+0000","last_peered":"2024-03-11T19:38:38.554758+0000","last_clean":"2024-03-11T19:38:38.554758+0000","last_became_active":"2024-03-11T19:37:05.335255+0000","last_became_peered":"2024-03-11T19:37:05.335255+0000","last_unstale":"2024-03-11T19:38:38.554758+0000","last_undegraded":"2024-03-11T19:38:38.554758+0000","last_fullsized":"2024-03-11T19:38:38.554758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:49:09.772765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":136,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:53.782803+0000","last_change":"2024-03-11T19:37:04.586002+0000","last_active":"2024-03-11T19:38:53.782803+0000","last_peered":"2024-03-11T19:38:53.782803+0000","last_clean":"2024-03-11T19:38:53.782803+0000","last_became_active":"2024-03-11T19:37:04.585551+0000","last_became_peered":"2024-03-11T19:37:04.585551+0000","last_unstale":"2024-03-11T19:38:53.782803+0000","last_undegraded":"2024-03-11T19:38:53.782803+0000","last_fullsized":"2024-03-11T19:38:53.782803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:34:12.663801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.542150+0000","last_change":"2024-03-11T19:37:05.325168+0000","last_active":"2024-03-11T19:38:38.542150+0000","last_peered":"2024-03-11T19:38:38.542150+0000","last_clean":"2024-03-11T19:38:38.542150+0000","last_became_active":"2024-03-11T19:37:05.324848+0000","last_became_peered":"2024-03-11T19:37:05.324848+0000","last_unstale":"2024-03-11T19:38:38.542150+0000","last_undegraded":"2024-03-11T19:38:38.542150+0000","last_fullsized":"2024-03-11T19:38:38.542150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:27.423877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452883+0000","last_change":"2024-03-11T19:37:04.582727+0000","last_active":"2024-03-11T19:38:10.452883+0000","last_peered":"2024-03-11T19:38:10.452883+0000","last_clean":"2024-03-11T19:38:10.452883+0000","last_became_active":"2024-03-11T19:37:04.582359+0000","last_became_peered":"2024-03-11T19:37:04.582359+0000","last_unstale":"2024-03-11T19:38:10.452883+0000","last_undegraded":"2024-03-11T19:38:10.452883+0000","last_fullsized":"2024-03-11T19:38:10.452883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:05.591537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.544557+0000","last_change":"2024-03-11T19:37:05.335442+0000","last_active":"2024-03-11T19:38:38.544557+0000","last_peered":"2024-03-11T19:38:38.544557+0000","last_clean":"2024-03-11T19:38:38.544557+0000","last_became_active":"2024-03-11T19:37:05.335003+0000","last_became_peered":"2024-03-11T19:37:05.335003+0000","last_unstale":"2024-03-11T19:38:38.544557+0000","last_undegraded":"2024-03-11T19:38:38.544557+0000","last_fullsized":"2024-03-11T19:38:38.544557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:41.590898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453615+0000","last_change":"2024-03-11T19:37:04.589957+0000","last_active":"2024-03-11T19:38:10.453615+0000","last_peered":"2024-03-11T19:38:10.453615+0000","last_clean":"2024-03-11T19:38:10.453615+0000","last_became_active":"2024-03-11T19:37:04.589539+0000","last_became_peered":"2024-03-11T19:37:04.589539+0000","last_unstale":"2024-03-11T19:38:10.453615+0000","last_undegraded":"2024-03-11T19:38:10.453615+0000","last_fullsized":"2024-03-11T19:38:10.453615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:48:52.764057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535727+0000","last_change":"2024-03-11T19:37:05.331380+0000","last_active":"2024-03-11T19:38:38.535727+0000","last_peered":"2024-03-11T19:38:38.535727+0000","last_clean":"2024-03-11T19:38:38.535727+0000","last_became_active":"2024-03-11T19:37:05.331000+0000","last_became_peered":"2024-03-11T19:37:05.331000+0000","last_unstale":"2024-03-11T19:38:38.535727+0000","last_undegraded":"2024-03-11T19:38:38.535727+0000","last_fullsized":"2024-03-11T19:38:38.535727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:58:47.278195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452916+0000","last_change":"2024-03-11T19:37:04.584128+0000","last_active":"2024-03-11T19:38:10.452916+0000","last_peered":"2024-03-11T19:38:10.452916+0000","last_clean":"2024-03-11T19:38:10.452916+0000","last_became_active":"2024-03-11T19:37:04.583842+0000","last_became_peered":"2024-03-11T19:37:04.583842+0000","last_unstale":"2024-03-11T19:38:10.452916+0000","last_undegraded":"2024-03-11T19:38:10.452916+0000","last_fullsized":"2024-03-11T19:38:10.452916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:18.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.534652+0000","last_change":"2024-03-11T19:37:05.328999+0000","last_active":"2024-03-11T19:38:38.534652+0000","last_peered":"2024-03-11T19:38:38.534652+0000","last_clean":"2024-03-11T19:38:38.534652+0000","last_became_active":"2024-03-11T19:37:05.328620+0000","last_became_peered":"2024-03-11T19:37:05.328620+0000","last_unstale":"2024-03-11T19:38:38.534652+0000","last_undegraded":"2024-03-11T19:38:38.534652+0000","last_fullsized":"2024-03-11T19:38:38.534652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:40.191922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":132,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:56.786615+0000","last_change":"2024-03-11T19:37:04.587956+0000","last_active":"2024-03-11T19:38:56.786615+0000","last_peered":"2024-03-11T19:38:56.786615+0000","last_clean":"2024-03-11T19:38:56.786615+0000","last_became_active":"2024-03-11T19:37:04.587568+0000","last_became_peered":"2024-03-11T19:37:04.587568+0000","last_unstale":"2024-03-11T19:38:56.786615+0000","last_undegraded":"2024-03-11T19:38:56.786615+0000","last_fullsized":"2024-03-11T19:38:56.786615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:45:44.019436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549054+0000","last_change":"2024-03-11T19:37:05.336912+0000","last_active":"2024-03-11T19:38:38.549054+0000","last_peered":"2024-03-11T19:38:38.549054+0000","last_clean":"2024-03-11T19:38:38.549054+0000","last_became_active":"2024-03-11T19:37:05.336397+0000","last_became_peered":"2024-03-11T19:37:05.336397+0000","last_unstale":"2024-03-11T19:38:38.549054+0000","last_undegraded":"2024-03-11T19:38:38.549054+0000","last_fullsized":"2024-03-11T19:38:38.549054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:29.375396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150574+0000","last_change":"2024-03-11T19:37:04.584382+0000","last_active":"2024-03-11T19:38:10.150574+0000","last_peered":"2024-03-11T19:38:10.150574+0000","last_clean":"2024-03-11T19:38:10.150574+0000","last_became_active":"2024-03-11T19:37:04.584026+0000","last_became_peered":"2024-03-11T19:37:04.584026+0000","last_unstale":"2024-03-11T19:38:10.150574+0000","last_undegraded":"2024-03-11T19:38:10.150574+0000","last_fullsized":"2024-03-11T19:38:10.150574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:28.152903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.546098+0000","last_change":"2024-03-11T19:37:05.331385+0000","last_active":"2024-03-11T19:38:38.546098+0000","last_peered":"2024-03-11T19:38:38.546098+0000","last_clean":"2024-03-11T19:38:38.546098+0000","last_became_active":"2024-03-11T19:37:05.331069+0000","last_became_peered":"2024-03-11T19:37:05.331069+0000","last_unstale":"2024-03-11T19:38:38.546098+0000","last_undegraded":"2024-03-11T19:38:38.546098+0000","last_fullsized":"2024-03-11T19:38:38.546098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:18.375605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453568+0000","last_change":"2024-03-11T19:37:04.586493+0000","last_active":"2024-03-11T19:38:10.453568+0000","last_peered":"2024-03-11T19:38:10.453568+0000","last_clean":"2024-03-11T19:38:10.453568+0000","last_became_active":"2024-03-11T19:37:04.586185+0000","last_became_peered":"2024-03-11T19:37:04.586185+0000","last_unstale":"2024-03-11T19:38:10.453568+0000","last_undegraded":"2024-03-11T19:38:10.453568+0000","last_fullsized":"2024-03-11T19:38:10.453568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:00.899752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.555300+0000","last_change":"2024-03-11T19:37:05.336414+0000","last_active":"2024-03-11T19:38:38.555300+0000","last_peered":"2024-03-11T19:38:38.555300+0000","last_clean":"2024-03-11T19:38:38.555300+0000","last_became_active":"2024-03-11T19:37:05.334536+0000","last_became_peered":"2024-03-11T19:37:05.334536+0000","last_unstale":"2024-03-11T19:38:38.555300+0000","last_undegraded":"2024-03-11T19:38:38.555300+0000","last_fullsized":"2024-03-11T19:38:38.555300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:50:19.109975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'34","reported_seq":281,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:57.788731+0000","last_change":"2024-03-11T19:37:04.584544+0000","last_active":"2024-03-11T19:38:57.788731+0000","last_peered":"2024-03-11T19:38:57.788731+0000","last_clean":"2024-03-11T19:38:57.788731+0000","last_became_active":"2024-03-11T19:37:04.584198+0000","last_became_peered":"2024-03-11T19:37:04.584198+0000","last_unstale":"2024-03-11T19:38:57.788731+0000","last_undegraded":"2024-03-11T19:38:57.788731+0000","last_fullsized":"2024-03-11T19:38:57.788731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:01:56.744533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60580,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":228,"num_read_kb":173,"num_write":35,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.533749+0000","last_change":"2024-03-11T19:37:05.331077+0000","last_active":"2024-03-11T19:38:38.533749+0000","last_peered":"2024-03-11T19:38:38.533749+0000","last_clean":"2024-03-11T19:38:38.533749+0000","last_became_active":"2024-03-11T19:37:05.330708+0000","last_became_peered":"2024-03-11T19:37:05.330708+0000","last_unstale":"2024-03-11T19:38:38.533749+0000","last_undegraded":"2024-03-11T19:38:38.533749+0000","last_fullsized":"2024-03-11T19:38:38.533749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:54:55.019268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155548+0000","last_change":"2024-03-11T19:37:04.583353+0000","last_active":"2024-03-11T19:38:10.155548+0000","last_peered":"2024-03-11T19:38:10.155548+0000","last_clean":"2024-03-11T19:38:10.155548+0000","last_became_active":"2024-03-11T19:37:04.581974+0000","last_became_peered":"2024-03-11T19:37:04.581974+0000","last_unstale":"2024-03-11T19:38:10.155548+0000","last_undegraded":"2024-03-11T19:38:10.155548+0000","last_fullsized":"2024-03-11T19:38:10.155548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:41.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553558+0000","last_change":"2024-03-11T19:37:05.329498+0000","last_active":"2024-03-11T19:38:38.553558+0000","last_peered":"2024-03-11T19:38:38.553558+0000","last_clean":"2024-03-11T19:38:38.553558+0000","last_became_active":"2024-03-11T19:37:05.329186+0000","last_became_peered":"2024-03-11T19:37:05.329186+0000","last_unstale":"2024-03-11T19:38:38.553558+0000","last_undegraded":"2024-03-11T19:38:38.553558+0000","last_fullsized":"2024-03-11T19:38:38.553558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:33:45.012049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453150+0000","last_change":"2024-03-11T19:37:04.582027+0000","last_active":"2024-03-11T19:38:10.453150+0000","last_peered":"2024-03-11T19:38:10.453150+0000","last_clean":"2024-03-11T19:38:10.453150+0000","last_became_active":"2024-03-11T19:37:04.581678+0000","last_became_peered":"2024-03-11T19:37:04.581678+0000","last_unstale":"2024-03-11T19:38:10.453150+0000","last_undegraded":"2024-03-11T19:38:10.453150+0000","last_fullsized":"2024-03-11T19:38:10.453150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:03:40.688053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553484+0000","last_change":"2024-03-11T19:38:10.455076+0000","last_active":"2024-03-11T19:38:38.553484+0000","last_peered":"2024-03-11T19:38:38.553484+0000","last_clean":"2024-03-11T19:38:38.553484+0000","last_became_active":"2024-03-11T19:38:10.454631+0000","last_became_peered":"2024-03-11T19:38:10.454631+0000","last_unstale":"2024-03-11T19:38:38.553484+0000","last_undegraded":"2024-03-11T19:38:38.553484+0000","last_fullsized":"2024-03-11T19:38:38.553484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:26:36.651449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":136,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:55.785449+0000","last_change":"2024-03-11T19:37:04.576415+0000","last_active":"2024-03-11T19:38:55.785449+0000","last_peered":"2024-03-11T19:38:55.785449+0000","last_clean":"2024-03-11T19:38:55.785449+0000","last_became_active":"2024-03-11T19:37:04.575978+0000","last_became_peered":"2024-03-11T19:37:04.575978+0000","last_unstale":"2024-03-11T19:38:55.785449+0000","last_undegraded":"2024-03-11T19:38:55.785449+0000","last_fullsized":"2024-03-11T19:38:55.785449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:20.124813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.550544+0000","last_change":"2024-03-11T19:37:05.326258+0000","last_active":"2024-03-11T19:38:38.550544+0000","last_peered":"2024-03-11T19:38:38.550544+0000","last_clean":"2024-03-11T19:38:38.550544+0000","last_became_active":"2024-03-11T19:37:05.325909+0000","last_became_peered":"2024-03-11T19:37:05.325909+0000","last_unstale":"2024-03-11T19:38:38.550544+0000","last_undegraded":"2024-03-11T19:38:38.550544+0000","last_fullsized":"2024-03-11T19:38:38.550544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:25.875603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.482823+0000","last_change":"2024-03-11T19:38:10.482380+0000","last_active":"2024-03-11T19:38:10.482823+0000","last_peered":"2024-03-11T19:38:10.482823+0000","last_clean":"2024-03-11T19:38:10.482823+0000","last_became_active":"2024-03-11T19:38:10.455301+0000","last_became_peered":"2024-03-11T19:38:10.455301+0000","last_unstale":"2024-03-11T19:38:10.482823+0000","last_undegraded":"2024-03-11T19:38:10.482823+0000","last_fullsized":"2024-03-11T19:38:10.482823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:28:41.620522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553853+0000","last_change":"2024-03-11T19:37:05.336727+0000","last_active":"2024-03-11T19:38:38.553853+0000","last_peered":"2024-03-11T19:38:38.553853+0000","last_clean":"2024-03-11T19:38:38.553853+0000","last_became_active":"2024-03-11T19:37:05.336384+0000","last_became_peered":"2024-03-11T19:37:05.336384+0000","last_unstale":"2024-03-11T19:38:38.553853+0000","last_undegraded":"2024-03-11T19:38:38.553853+0000","last_fullsized":"2024-03-11T19:38:38.553853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:02.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151827+0000","last_change":"2024-03-11T19:37:04.586814+0000","last_active":"2024-03-11T19:38:10.151827+0000","last_peered":"2024-03-11T19:38:10.151827+0000","last_clean":"2024-03-11T19:38:10.151827+0000","last_became_active":"2024-03-11T19:37:04.586512+0000","last_became_peered":"2024-03-11T19:37:04.586512+0000","last_unstale":"2024-03-11T19:38:10.151827+0000","last_undegraded":"2024-03-11T19:38:10.151827+0000","last_fullsized":"2024-03-11T19:38:10.151827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:04:23.979783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548507+0000","last_change":"2024-03-11T19:37:05.335353+0000","last_active":"2024-03-11T19:38:38.548507+0000","last_peered":"2024-03-11T19:38:38.548507+0000","last_clean":"2024-03-11T19:38:38.548507+0000","last_became_active":"2024-03-11T19:37:05.335066+0000","last_became_peered":"2024-03-11T19:37:05.335066+0000","last_unstale":"2024-03-11T19:38:38.548507+0000","last_undegraded":"2024-03-11T19:38:38.548507+0000","last_fullsized":"2024-03-11T19:38:38.548507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:11.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454552+0000","last_change":"2024-03-11T19:37:04.586679+0000","last_active":"2024-03-11T19:38:10.454552+0000","last_peered":"2024-03-11T19:38:10.454552+0000","last_clean":"2024-03-11T19:38:10.454552+0000","last_became_active":"2024-03-11T19:37:04.586316+0000","last_became_peered":"2024-03-11T19:37:04.586316+0000","last_unstale":"2024-03-11T19:38:10.454552+0000","last_undegraded":"2024-03-11T19:38:10.454552+0000","last_fullsized":"2024-03-11T19:38:10.454552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:58:21.115526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535653+0000","last_change":"2024-03-11T19:37:05.330206+0000","last_active":"2024-03-11T19:38:38.535653+0000","last_peered":"2024-03-11T19:38:38.535653+0000","last_clean":"2024-03-11T19:38:38.535653+0000","last_became_active":"2024-03-11T19:37:05.329847+0000","last_became_peered":"2024-03-11T19:37:05.329847+0000","last_unstale":"2024-03-11T19:38:38.535653+0000","last_undegraded":"2024-03-11T19:38:38.535653+0000","last_fullsized":"2024-03-11T19:38:38.535653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:14:52.532236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150350+0000","last_change":"2024-03-11T19:37:04.581031+0000","last_active":"2024-03-11T19:38:10.150350+0000","last_peered":"2024-03-11T19:38:10.150350+0000","last_clean":"2024-03-11T19:38:10.150350+0000","last_became_active":"2024-03-11T19:37:04.580728+0000","last_became_peered":"2024-03-11T19:37:04.580728+0000","last_unstale":"2024-03-11T19:38:10.150350+0000","last_undegraded":"2024-03-11T19:38:10.150350+0000","last_fullsized":"2024-03-11T19:38:10.150350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:43.401898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540064+0000","last_change":"2024-03-11T19:37:05.328686+0000","last_active":"2024-03-11T19:38:38.540064+0000","last_peered":"2024-03-11T19:38:38.540064+0000","last_clean":"2024-03-11T19:38:38.540064+0000","last_became_active":"2024-03-11T19:37:05.328372+0000","last_became_peered":"2024-03-11T19:37:05.328372+0000","last_unstale":"2024-03-11T19:38:38.540064+0000","last_undegraded":"2024-03-11T19:38:38.540064+0000","last_fullsized":"2024-03-11T19:38:38.540064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:38:33.216489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.544161+0000","last_change":"2024-03-11T19:37:05.335057+0000","last_active":"2024-03-11T19:38:38.544161+0000","last_peered":"2024-03-11T19:38:38.544161+0000","last_clean":"2024-03-11T19:38:38.544161+0000","last_became_active":"2024-03-11T19:37:05.334646+0000","last_became_peered":"2024-03-11T19:37:05.334646+0000","last_unstale":"2024-03-11T19:38:38.544161+0000","last_undegraded":"2024-03-11T19:38:38.544161+0000","last_fullsized":"2024-03-11T19:38:38.544161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:34:15.932429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":132,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:55.784973+0000","last_change":"2024-03-11T19:37:04.588095+0000","last_active":"2024-03-11T19:38:55.784973+0000","last_peered":"2024-03-11T19:38:55.784973+0000","last_clean":"2024-03-11T19:38:55.784973+0000","last_became_active":"2024-03-11T19:37:04.587757+0000","last_became_peered":"2024-03-11T19:37:04.587757+0000","last_unstale":"2024-03-11T19:38:55.784973+0000","last_undegraded":"2024-03-11T19:38:55.784973+0000","last_fullsized":"2024-03-11T19:38:55.784973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:57.926858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540458+0000","last_change":"2024-03-11T19:37:05.326806+0000","last_active":"2024-03-11T19:38:38.540458+0000","last_peered":"2024-03-11T19:38:38.540458+0000","last_clean":"2024-03-11T19:38:38.540458+0000","last_became_active":"2024-03-11T19:37:05.326506+0000","last_became_peered":"2024-03-11T19:37:05.326506+0000","last_unstale":"2024-03-11T19:38:38.540458+0000","last_undegraded":"2024-03-11T19:38:38.540458+0000","last_fullsized":"2024-03-11T19:38:38.540458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:20.664857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453750+0000","last_change":"2024-03-11T19:37:04.588459+0000","last_active":"2024-03-11T19:38:10.453750+0000","last_peered":"2024-03-11T19:38:10.453750+0000","last_clean":"2024-03-11T19:38:10.453750+0000","last_became_active":"2024-03-11T19:37:04.588000+0000","last_became_peered":"2024-03-11T19:37:04.588000+0000","last_unstale":"2024-03-11T19:38:10.453750+0000","last_undegraded":"2024-03-11T19:38:10.453750+0000","last_fullsized":"2024-03-11T19:38:10.453750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:20:47.717196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150281+0000","last_change":"2024-03-11T19:35:55.814178+0000","last_active":"2024-03-11T19:38:10.150281+0000","last_peered":"2024-03-11T19:38:10.150281+0000","last_clean":"2024-03-11T19:38:10.150281+0000","last_became_active":"2024-03-11T19:35:55.813697+0000","last_became_peered":"2024-03-11T19:35:55.813697+0000","last_unstale":"2024-03-11T19:38:10.150281+0000","last_undegraded":"2024-03-11T19:38:10.150281+0000","last_fullsized":"2024-03-11T19:38:10.150281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_clean_scrub_stamp":"2024-03-11T19:32:34.872292+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:50:27.486733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543638+0000","last_change":"2024-03-11T19:37:05.319614+0000","last_active":"2024-03-11T19:38:38.543638+0000","last_peered":"2024-03-11T19:38:38.543638+0000","last_clean":"2024-03-11T19:38:38.543638+0000","last_became_active":"2024-03-11T19:37:05.319277+0000","last_became_peered":"2024-03-11T19:37:05.319277+0000","last_unstale":"2024-03-11T19:38:38.543638+0000","last_undegraded":"2024-03-11T19:38:38.543638+0000","last_fullsized":"2024-03-11T19:38:38.543638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:27.329900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150075+0000","last_change":"2024-03-11T19:37:04.582250+0000","last_active":"2024-03-11T19:38:10.150075+0000","last_peered":"2024-03-11T19:38:10.150075+0000","last_clean":"2024-03-11T19:38:10.150075+0000","last_became_active":"2024-03-11T19:37:04.581938+0000","last_became_peered":"2024-03-11T19:37:04.581938+0000","last_unstale":"2024-03-11T19:38:10.150075+0000","last_undegraded":"2024-03-11T19:38:10.150075+0000","last_fullsized":"2024-03-11T19:38:10.150075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:23.466067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151834+0000","last_change":"2024-03-11T19:37:04.590699+0000","last_active":"2024-03-11T19:38:10.151834+0000","last_peered":"2024-03-11T19:38:10.151834+0000","last_clean":"2024-03-11T19:38:10.151834+0000","last_became_active":"2024-03-11T19:37:04.590424+0000","last_became_peered":"2024-03-11T19:37:04.590424+0000","last_unstale":"2024-03-11T19:38:10.151834+0000","last_undegraded":"2024-03-11T19:38:10.151834+0000","last_fullsized":"2024-03-11T19:38:10.151834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:49.075335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.538473+0000","last_change":"2024-03-11T19:37:05.323357+0000","last_active":"2024-03-11T19:38:38.538473+0000","last_peered":"2024-03-11T19:38:38.538473+0000","last_clean":"2024-03-11T19:38:38.538473+0000","last_became_active":"2024-03-11T19:37:05.322980+0000","last_became_peered":"2024-03-11T19:37:05.322980+0000","last_unstale":"2024-03-11T19:38:38.538473+0000","last_undegraded":"2024-03-11T19:38:38.538473+0000","last_fullsized":"2024-03-11T19:38:38.538473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:19.106375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.589118+0000","last_change":"2024-03-11T19:37:04.585232+0000","last_active":"2024-03-11T19:38:38.589118+0000","last_peered":"2024-03-11T19:38:38.589118+0000","last_clean":"2024-03-11T19:38:38.589118+0000","last_became_active":"2024-03-11T19:37:04.584904+0000","last_became_peered":"2024-03-11T19:37:04.584904+0000","last_unstale":"2024-03-11T19:38:38.589118+0000","last_undegraded":"2024-03-11T19:38:38.589118+0000","last_fullsized":"2024-03-11T19:38:38.589118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:56:46.519961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553409+0000","last_change":"2024-03-11T19:37:05.332019+0000","last_active":"2024-03-11T19:38:38.553409+0000","last_peered":"2024-03-11T19:38:38.553409+0000","last_clean":"2024-03-11T19:38:38.553409+0000","last_became_active":"2024-03-11T19:37:05.331673+0000","last_became_peered":"2024-03-11T19:37:05.331673+0000","last_unstale":"2024-03-11T19:38:38.553409+0000","last_undegraded":"2024-03-11T19:38:38.553409+0000","last_fullsized":"2024-03-11T19:38:38.553409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:49:45.787082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454347+0000","last_change":"2024-03-11T19:37:04.583404+0000","last_active":"2024-03-11T19:38:10.454347+0000","last_peered":"2024-03-11T19:38:10.454347+0000","last_clean":"2024-03-11T19:38:10.454347+0000","last_became_active":"2024-03-11T19:37:04.583025+0000","last_became_peered":"2024-03-11T19:37:04.583025+0000","last_unstale":"2024-03-11T19:38:10.454347+0000","last_undegraded":"2024-03-11T19:38:10.454347+0000","last_fullsized":"2024-03-11T19:38:10.454347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:02:25.202591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549246+0000","last_change":"2024-03-11T19:37:05.331711+0000","last_active":"2024-03-11T19:38:38.549246+0000","last_peered":"2024-03-11T19:38:38.549246+0000","last_clean":"2024-03-11T19:38:38.549246+0000","last_became_active":"2024-03-11T19:37:05.330851+0000","last_became_peered":"2024-03-11T19:37:05.330851+0000","last_unstale":"2024-03-11T19:38:38.549246+0000","last_undegraded":"2024-03-11T19:38:38.549246+0000","last_fullsized":"2024-03-11T19:38:38.549246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:18:26.692068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448230+0000","last_change":"2024-03-11T19:37:04.586501+0000","last_active":"2024-03-11T19:38:10.448230+0000","last_peered":"2024-03-11T19:38:10.448230+0000","last_clean":"2024-03-11T19:38:10.448230+0000","last_became_active":"2024-03-11T19:37:04.586200+0000","last_became_peered":"2024-03-11T19:37:04.586200+0000","last_unstale":"2024-03-11T19:38:10.448230+0000","last_undegraded":"2024-03-11T19:38:10.448230+0000","last_fullsized":"2024-03-11T19:38:10.448230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:56.635375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.510995+0000","last_change":"2024-03-11T19:37:05.332402+0000","last_active":"2024-03-11T19:38:38.510995+0000","last_peered":"2024-03-11T19:38:38.510995+0000","last_clean":"2024-03-11T19:38:38.510995+0000","last_became_active":"2024-03-11T19:37:05.332087+0000","last_became_peered":"2024-03-11T19:37:05.332087+0000","last_unstale":"2024-03-11T19:38:38.510995+0000","last_undegraded":"2024-03-11T19:38:38.510995+0000","last_fullsized":"2024-03-11T19:38:38.510995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:10:23.882296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452565+0000","last_change":"2024-03-11T19:37:04.584978+0000","last_active":"2024-03-11T19:38:10.452565+0000","last_peered":"2024-03-11T19:38:10.452565+0000","last_clean":"2024-03-11T19:38:10.452565+0000","last_became_active":"2024-03-11T19:37:04.584556+0000","last_became_peered":"2024-03-11T19:37:04.584556+0000","last_unstale":"2024-03-11T19:38:10.452565+0000","last_undegraded":"2024-03-11T19:38:10.452565+0000","last_fullsized":"2024-03-11T19:38:10.452565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:39:48.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.539294+0000","last_change":"2024-03-11T19:37:05.334453+0000","last_active":"2024-03-11T19:38:38.539294+0000","last_peered":"2024-03-11T19:38:38.539294+0000","last_clean":"2024-03-11T19:38:38.539294+0000","last_became_active":"2024-03-11T19:37:05.334009+0000","last_became_peered":"2024-03-11T19:37:05.334009+0000","last_unstale":"2024-03-11T19:38:38.539294+0000","last_undegraded":"2024-03-11T19:38:38.539294+0000","last_fullsized":"2024-03-11T19:38:38.539294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:43.402674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155193+0000","last_change":"2024-03-11T19:37:04.583181+0000","last_active":"2024-03-11T19:38:10.155193+0000","last_peered":"2024-03-11T19:38:10.155193+0000","last_clean":"2024-03-11T19:38:10.155193+0000","last_became_active":"2024-03-11T19:37:04.582881+0000","last_became_peered":"2024-03-11T19:37:04.582881+0000","last_unstale":"2024-03-11T19:38:10.155193+0000","last_undegraded":"2024-03-11T19:38:10.155193+0000","last_fullsized":"2024-03-11T19:38:10.155193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:05:06.634150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.530930+0000","last_change":"2024-03-11T19:37:05.333642+0000","last_active":"2024-03-11T19:38:38.530930+0000","last_peered":"2024-03-11T19:38:38.530930+0000","last_clean":"2024-03-11T19:38:38.530930+0000","last_became_active":"2024-03-11T19:37:05.333121+0000","last_became_peered":"2024-03-11T19:37:05.333121+0000","last_unstale":"2024-03-11T19:38:38.530930+0000","last_undegraded":"2024-03-11T19:38:38.530930+0000","last_fullsized":"2024-03-11T19:38:38.530930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:07:21.217101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.458830+0000","last_change":"2024-03-11T19:38:10.458358+0000","last_active":"2024-03-11T19:38:10.458830+0000","last_peered":"2024-03-11T19:38:10.458830+0000","last_clean":"2024-03-11T19:38:10.458830+0000","last_became_active":"2024-03-11T19:38:10.449890+0000","last_became_peered":"2024-03-11T19:38:10.449890+0000","last_unstale":"2024-03-11T19:38:10.458830+0000","last_undegraded":"2024-03-11T19:38:10.458830+0000","last_fullsized":"2024-03-11T19:38:10.458830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:51:00.265814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.554245+0000","last_change":"2024-03-11T19:37:05.326992+0000","last_active":"2024-03-11T19:38:38.554245+0000","last_peered":"2024-03-11T19:38:38.554245+0000","last_clean":"2024-03-11T19:38:38.554245+0000","last_became_active":"2024-03-11T19:37:05.326676+0000","last_became_peered":"2024-03-11T19:37:05.326676+0000","last_unstale":"2024-03-11T19:38:38.554245+0000","last_undegraded":"2024-03-11T19:38:38.554245+0000","last_fullsized":"2024-03-11T19:38:38.554245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:31:10.310650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155451+0000","last_change":"2024-03-11T19:37:04.583631+0000","last_active":"2024-03-11T19:38:10.155451+0000","last_peered":"2024-03-11T19:38:10.155451+0000","last_clean":"2024-03-11T19:38:10.155451+0000","last_became_active":"2024-03-11T19:37:04.583031+0000","last_became_peered":"2024-03-11T19:37:04.583031+0000","last_unstale":"2024-03-11T19:38:10.155451+0000","last_undegraded":"2024-03-11T19:38:10.155451+0000","last_fullsized":"2024-03-11T19:38:10.155451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:20.695496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553916+0000","last_change":"2024-03-11T19:37:05.331008+0000","last_active":"2024-03-11T19:38:38.553916+0000","last_peered":"2024-03-11T19:38:38.553916+0000","last_clean":"2024-03-11T19:38:38.553916+0000","last_became_active":"2024-03-11T19:37:05.330647+0000","last_became_peered":"2024-03-11T19:37:05.330647+0000","last_unstale":"2024-03-11T19:38:38.553916+0000","last_undegraded":"2024-03-11T19:38:38.553916+0000","last_fullsized":"2024-03-11T19:38:38.553916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:07:00.576137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453400+0000","last_change":"2024-03-11T19:37:04.582501+0000","last_active":"2024-03-11T19:38:10.453400+0000","last_peered":"2024-03-11T19:38:10.453400+0000","last_clean":"2024-03-11T19:38:10.453400+0000","last_became_active":"2024-03-11T19:37:04.582076+0000","last_became_peered":"2024-03-11T19:37:04.582076+0000","last_unstale":"2024-03-11T19:38:10.453400+0000","last_undegraded":"2024-03-11T19:38:10.453400+0000","last_fullsized":"2024-03-11T19:38:10.453400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:46:05.359211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549826+0000","last_change":"2024-03-11T19:37:05.329053+0000","last_active":"2024-03-11T19:38:38.549826+0000","last_peered":"2024-03-11T19:38:38.549826+0000","last_clean":"2024-03-11T19:38:38.549826+0000","last_became_active":"2024-03-11T19:37:05.328748+0000","last_became_peered":"2024-03-11T19:37:05.328748+0000","last_unstale":"2024-03-11T19:38:38.549826+0000","last_undegraded":"2024-03-11T19:38:38.549826+0000","last_fullsized":"2024-03-11T19:38:38.549826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:48:55.369515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453324+0000","last_change":"2024-03-11T19:37:04.583370+0000","last_active":"2024-03-11T19:38:10.453324+0000","last_peered":"2024-03-11T19:38:10.453324+0000","last_clean":"2024-03-11T19:38:10.453324+0000","last_became_active":"2024-03-11T19:37:04.583055+0000","last_became_peered":"2024-03-11T19:37:04.583055+0000","last_unstale":"2024-03-11T19:38:10.453324+0000","last_undegraded":"2024-03-11T19:38:10.453324+0000","last_fullsized":"2024-03-11T19:38:10.453324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:04:02.325054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.550503+0000","last_change":"2024-03-11T19:37:05.328157+0000","last_active":"2024-03-11T19:38:38.550503+0000","last_peered":"2024-03-11T19:38:38.550503+0000","last_clean":"2024-03-11T19:38:38.550503+0000","last_became_active":"2024-03-11T19:37:05.327839+0000","last_became_peered":"2024-03-11T19:37:05.327839+0000","last_unstale":"2024-03-11T19:38:38.550503+0000","last_undegraded":"2024-03-11T19:38:38.550503+0000","last_fullsized":"2024-03-11T19:38:38.550503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:23:02.297269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543426+0000","last_change":"2024-03-11T19:37:05.336160+0000","last_active":"2024-03-11T19:38:38.543426+0000","last_peered":"2024-03-11T19:38:38.543426+0000","last_clean":"2024-03-11T19:38:38.543426+0000","last_became_active":"2024-03-11T19:37:05.335850+0000","last_became_peered":"2024-03-11T19:37:05.335850+0000","last_unstale":"2024-03-11T19:38:38.543426+0000","last_undegraded":"2024-03-11T19:38:38.543426+0000","last_fullsized":"2024-03-11T19:38:38.543426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:09.743661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536039+0000","last_change":"2024-03-11T19:38:10.455362+0000","last_active":"2024-03-11T19:38:38.536039+0000","last_peered":"2024-03-11T19:38:38.536039+0000","last_clean":"2024-03-11T19:38:38.536039+0000","last_became_active":"2024-03-11T19:38:10.454887+0000","last_became_peered":"2024-03-11T19:38:10.454887+0000","last_unstale":"2024-03-11T19:38:38.536039+0000","last_undegraded":"2024-03-11T19:38:38.536039+0000","last_fullsized":"2024-03-11T19:38:38.536039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:19.289829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.542420+0000","last_change":"2024-03-11T19:37:05.332357+0000","last_active":"2024-03-11T19:38:38.542420+0000","last_peered":"2024-03-11T19:38:38.542420+0000","last_clean":"2024-03-11T19:38:38.542420+0000","last_became_active":"2024-03-11T19:37:05.332003+0000","last_became_peered":"2024-03-11T19:37:05.332003+0000","last_unstale":"2024-03-11T19:38:38.542420+0000","last_undegraded":"2024-03-11T19:38:38.542420+0000","last_fullsized":"2024-03-11T19:38:38.542420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:39.165572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549322+0000","last_change":"2024-03-11T19:37:05.332388+0000","last_active":"2024-03-11T19:38:38.549322+0000","last_peered":"2024-03-11T19:38:38.549322+0000","last_clean":"2024-03-11T19:38:38.549322+0000","last_became_active":"2024-03-11T19:37:05.332032+0000","last_became_peered":"2024-03-11T19:37:05.332032+0000","last_unstale":"2024-03-11T19:38:38.549322+0000","last_undegraded":"2024-03-11T19:38:38.549322+0000","last_fullsized":"2024-03-11T19:38:38.549322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:56:09.416683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.545668+0000","last_change":"2024-03-11T19:37:05.323096+0000","last_active":"2024-03-11T19:38:38.545668+0000","last_peered":"2024-03-11T19:38:38.545668+0000","last_clean":"2024-03-11T19:38:38.545668+0000","last_became_active":"2024-03-11T19:37:05.322525+0000","last_became_peered":"2024-03-11T19:37:05.322525+0000","last_unstale":"2024-03-11T19:38:38.545668+0000","last_undegraded":"2024-03-11T19:38:38.545668+0000","last_fullsized":"2024-03-11T19:38:38.545668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:53:01.870894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540652+0000","last_change":"2024-03-11T19:37:05.331171+0000","last_active":"2024-03-11T19:38:38.540652+0000","last_peered":"2024-03-11T19:38:38.540652+0000","last_clean":"2024-03-11T19:38:38.540652+0000","last_became_active":"2024-03-11T19:37:05.330769+0000","last_became_peered":"2024-03-11T19:37:05.330769+0000","last_unstale":"2024-03-11T19:38:38.540652+0000","last_undegraded":"2024-03-11T19:38:38.540652+0000","last_fullsized":"2024-03-11T19:38:38.540652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:03:06.514039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553501+0000","last_change":"2024-03-11T19:37:05.326877+0000","last_active":"2024-03-11T19:38:38.553501+0000","last_peered":"2024-03-11T19:38:38.553501+0000","last_clean":"2024-03-11T19:38:38.553501+0000","last_became_active":"2024-03-11T19:37:05.326538+0000","last_became_peered":"2024-03-11T19:37:05.326538+0000","last_unstale":"2024-03-11T19:38:38.553501+0000","last_undegraded":"2024-03-11T19:38:38.553501+0000","last_fullsized":"2024-03-11T19:38:38.553501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:11.565697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.530558+0000","last_change":"2024-03-11T19:37:05.335010+0000","last_active":"2024-03-11T19:38:38.530558+0000","last_peered":"2024-03-11T19:38:38.530558+0000","last_clean":"2024-03-11T19:38:38.530558+0000","last_became_active":"2024-03-11T19:37:05.334551+0000","last_became_peered":"2024-03-11T19:37:05.334551+0000","last_unstale":"2024-03-11T19:38:38.530558+0000","last_undegraded":"2024-03-11T19:38:38.530558+0000","last_fullsized":"2024-03-11T19:38:38.530558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:59.668178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540099+0000","last_change":"2024-03-11T19:37:05.336273+0000","last_active":"2024-03-11T19:38:38.540099+0000","last_peered":"2024-03-11T19:38:38.540099+0000","last_clean":"2024-03-11T19:38:38.540099+0000","last_became_active":"2024-03-11T19:37:05.335926+0000","last_became_peered":"2024-03-11T19:37:05.335926+0000","last_unstale":"2024-03-11T19:38:38.540099+0000","last_undegraded":"2024-03-11T19:38:38.540099+0000","last_fullsized":"2024-03-11T19:38:38.540099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:31:55.139242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.533346+0000","last_change":"2024-03-11T19:37:05.335548+0000","last_active":"2024-03-11T19:38:38.533346+0000","last_peered":"2024-03-11T19:38:38.533346+0000","last_clean":"2024-03-11T19:38:38.533346+0000","last_became_active":"2024-03-11T19:37:05.335222+0000","last_became_peered":"2024-03-11T19:37:05.335222+0000","last_unstale":"2024-03-11T19:38:38.533346+0000","last_undegraded":"2024-03-11T19:38:38.533346+0000","last_fullsized":"2024-03-11T19:38:38.533346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:25.697951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.554788+0000","last_change":"2024-03-11T19:37:05.329782+0000","last_active":"2024-03-11T19:38:38.554788+0000","last_peered":"2024-03-11T19:38:38.554788+0000","last_clean":"2024-03-11T19:38:38.554788+0000","last_became_active":"2024-03-11T19:37:05.329434+0000","last_became_peered":"2024-03-11T19:37:05.329434+0000","last_unstale":"2024-03-11T19:38:38.554788+0000","last_undegraded":"2024-03-11T19:38:38.554788+0000","last_fullsized":"2024-03-11T19:38:38.554788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:12:00.516489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553896+0000","last_change":"2024-03-11T19:37:05.323026+0000","last_active":"2024-03-11T19:38:38.553896+0000","last_peered":"2024-03-11T19:38:38.553896+0000","last_clean":"2024-03-11T19:38:38.553896+0000","last_became_active":"2024-03-11T19:37:05.322562+0000","last_became_peered":"2024-03-11T19:37:05.322562+0000","last_unstale":"2024-03-11T19:38:38.553896+0000","last_undegraded":"2024-03-11T19:38:38.553896+0000","last_fullsized":"2024-03-11T19:38:38.553896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:14:57.606404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.533719+0000","last_change":"2024-03-11T19:38:10.455616+0000","last_active":"2024-03-11T19:38:38.533719+0000","last_peered":"2024-03-11T19:38:38.533719+0000","last_clean":"2024-03-11T19:38:38.533719+0000","last_became_active":"2024-03-11T19:38:10.455092+0000","last_became_peered":"2024-03-11T19:38:10.455092+0000","last_unstale":"2024-03-11T19:38:38.533719+0000","last_undegraded":"2024-03-11T19:38:38.533719+0000","last_fullsized":"2024-03-11T19:38:38.533719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:49:50.407540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65254,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":678,"num_read_kb":409,"num_write":171,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383716,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":455694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32776,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712376,"statfs":{"total":95995035648,"available":95961473024,"internally_reserved":0,"allocated":1638400,"data_stored":1108175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":455818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":455626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1048341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":455207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":515580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":517350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":455518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1047493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:39:01.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T19:39:01.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.174+0000 7f705a8fb640 1 -- 172.21.15.83:0/3058566808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054102460 msgr2=0x7f70541028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.174+0000 7f705a8fb640 1 --2- 172.21.15.83:0/3058566808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054102460 0x7f70541028e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7048009900 tx=0x7f704802bc30 comp rx=0 tx=0).stop 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.174+0000 7f705a8fb640 1 -- 172.21.15.83:0/3058566808 shutdown_connections 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.174+0000 7f705a8fb640 1 --2- 172.21.15.83:0/3058566808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7054102e20 0x7f70541096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.174+0000 7f705a8fb640 1 --2- 172.21.15.83:0/3058566808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054102460 0x7f70541028e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.174+0000 7f705a8fb640 1 --2- 172.21.15.83:0/3058566808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70540ff490 0x7f70540ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.174+0000 7f705a8fb640 1 -- 172.21.15.83:0/3058566808 >> 172.21.15.83:0/3058566808 conn(0x7f70540fadc0 msgr2=0x7f70540fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.174+0000 7f705a8fb640 1 -- 172.21.15.83:0/3058566808 shutdown_connections 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.174+0000 7f705a8fb640 1 -- 172.21.15.83:0/3058566808 wait complete. 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f705a8fb640 1 Processor -- start 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f705a8fb640 1 -- start start 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f705a8fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70540ff490 0x7f70541972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f705a8fb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054102460 0x7f7054197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f705a8fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054102e20 0x7f705419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f705a8fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705419edd0 con 0x7f7054102e20 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f705a8fb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705419ef40 con 0x7f7054102460 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f705a8fb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705419f100 con 0x7f70540ff490 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f7058e98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054102e20 0x7f705419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f7058e98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054102e20 0x7f705419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50516/0 (socket says 172.21.15.83:50516) 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f7058e98640 1 -- 172.21.15.83:0/315805443 learned_addr learned my addr 172.21.15.83:0/315805443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f7053fff640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70540ff490 0x7f70541972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f70537fe640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054102460 0x7f7054197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f7058e98640 1 -- 172.21.15.83:0/315805443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70540ff490 msgr2=0x7f70541972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f7058e98640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70540ff490 0x7f70541972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f7058e98640 1 -- 172.21.15.83:0/315805443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054102460 msgr2=0x7f7054197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f7058e98640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054102460 0x7f7054197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f7058e98640 1 -- 172.21.15.83:0/315805443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7048009560 con 0x7f7054102e20 2024-03-11T19:39:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.175+0000 7f7058e98640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054102e20 0x7f705419e890 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f704c00eaa0 tx=0x7f704c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.176+0000 7f70517fa640 1 -- 172.21.15.83:0/315805443 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c00cdb0 con 0x7f7054102e20 2024-03-11T19:39:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.176+0000 7f705a8fb640 1 -- 172.21.15.83:0/315805443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705419f3e0 con 0x7f7054102e20 2024-03-11T19:39:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.176+0000 7f70517fa640 1 -- 172.21.15.83:0/315805443 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f704c005020 con 0x7f7054102e20 2024-03-11T19:39:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.176+0000 7f70517fa640 1 -- 172.21.15.83:0/315805443 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c0116d0 con 0x7f7054102e20 2024-03-11T19:39:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.176+0000 7f705a8fb640 1 -- 172.21.15.83:0/315805443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705419f9c0 con 0x7f7054102e20 2024-03-11T19:39:01.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.177+0000 7f705a8fb640 1 -- 172.21.15.83:0/315805443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7054108590 con 0x7f7054102e20 2024-03-11T19:39:01.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.178+0000 7f70517fa640 1 -- 172.21.15.83:0/315805443 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f704c005490 con 0x7f7054102e20 2024-03-11T19:39:01.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.178+0000 7f70517fa640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f703c0a1a30 0x7f703c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.178+0000 7f7053fff640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f703c0a1a30 0x7f703c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.178+0000 7f70517fa640 1 -- 172.21.15.83:0/315805443 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f704c0cd950 con 0x7f7054102e20 2024-03-11T19:39:01.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.179+0000 7f7053fff640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f703c0a1a30 0x7f703c0a3ef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f70540695d0 tx=0x7f7044008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:01.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.181+0000 7f70517fa640 1 -- 172.21.15.83:0/315805443 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f704c096c10 con 0x7f7054102e20 2024-03-11T19:39:01.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.351+0000 7f705a8fb640 1 -- 172.21.15.83:0/315805443 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f705402ce10 con 0x7f703c0a1a30 2024-03-11T19:39:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.358+0000 7f70517fa640 1 -- 172.21.15.83:0/315805443 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258839 (secure 0 0 0) 0x7f705402ce10 con 0x7f703c0a1a30 2024-03-11T19:39:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:01.367 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-03-11T19:39:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.361+0000 7f705a8fb640 1 -- 172.21.15.83:0/315805443 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f703c0a1a30 msgr2=0x7f703c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.361+0000 7f705a8fb640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f703c0a1a30 0x7f703c0a3ef0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f70540695d0 tx=0x7f7044008040 comp rx=0 tx=0).stop 2024-03-11T19:39:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.361+0000 7f705a8fb640 1 -- 172.21.15.83:0/315805443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054102e20 msgr2=0x7f705419e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.361+0000 7f705a8fb640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054102e20 0x7f705419e890 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f704c00eaa0 tx=0x7f704c00ef60 comp rx=0 tx=0).stop 2024-03-11T19:39:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.361+0000 7f705a8fb640 1 -- 172.21.15.83:0/315805443 shutdown_connections 2024-03-11T19:39:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.361+0000 7f705a8fb640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f703c0a1a30 0x7f703c0a3ef0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.361+0000 7f705a8fb640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054102e20 0x7f705419e890 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.361+0000 7f705a8fb640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054102460 0x7f7054197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.361+0000 7f705a8fb640 1 --2- 172.21.15.83:0/315805443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70540ff490 0x7f70541972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.361+0000 7f705a8fb640 1 -- 172.21.15.83:0/315805443 >> 172.21.15.83:0/315805443 conn(0x7f70540fadc0 msgr2=0x7f7054105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.361+0000 7f705a8fb640 1 -- 172.21.15.83:0/315805443 shutdown_connections 2024-03-11T19:39:01.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.362+0000 7f705a8fb640 1 -- 172.21.15.83:0/315805443 wait complete. 2024-03-11T19:39:01.376 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-03-11T19:39:00.535104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655856,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":804,"num_read_kb":518,"num_write":354,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366476,"kb_used_data":13164,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575348,"statfs":{"total":1151940427776,"available":1151565156352,"internally_reserved":0,"allocated":13479936,"data_stored":7427568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002753"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543207+0000","last_change":"2024-03-11T19:37:05.340476+0000","last_active":"2024-03-11T19:38:38.543207+0000","last_peered":"2024-03-11T19:38:38.543207+0000","last_clean":"2024-03-11T19:38:38.543207+0000","last_became_active":"2024-03-11T19:37:05.340140+0000","last_became_peered":"2024-03-11T19:37:05.340140+0000","last_unstale":"2024-03-11T19:38:38.543207+0000","last_undegraded":"2024-03-11T19:38:38.543207+0000","last_fullsized":"2024-03-11T19:38:38.543207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:10:35.677374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.537288+0000","last_change":"2024-03-11T19:37:05.335500+0000","last_active":"2024-03-11T19:38:38.537288+0000","last_peered":"2024-03-11T19:38:38.537288+0000","last_clean":"2024-03-11T19:38:38.537288+0000","last_became_active":"2024-03-11T19:37:05.335194+0000","last_became_peered":"2024-03-11T19:37:05.335194+0000","last_unstale":"2024-03-11T19:38:38.537288+0000","last_undegraded":"2024-03-11T19:38:38.537288+0000","last_fullsized":"2024-03-11T19:38:38.537288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:03.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.555088+0000","last_change":"2024-03-11T19:37:05.334618+0000","last_active":"2024-03-11T19:38:38.555088+0000","last_peered":"2024-03-11T19:38:38.555088+0000","last_clean":"2024-03-11T19:38:38.555088+0000","last_became_active":"2024-03-11T19:37:05.334281+0000","last_became_peered":"2024-03-11T19:37:05.334281+0000","last_unstale":"2024-03-11T19:38:38.555088+0000","last_undegraded":"2024-03-11T19:38:38.555088+0000","last_fullsized":"2024-03-11T19:38:38.555088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:43:22.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.546651+0000","last_change":"2024-03-11T19:37:05.332978+0000","last_active":"2024-03-11T19:38:38.546651+0000","last_peered":"2024-03-11T19:38:38.546651+0000","last_clean":"2024-03-11T19:38:38.546651+0000","last_became_active":"2024-03-11T19:37:05.330888+0000","last_became_peered":"2024-03-11T19:37:05.330888+0000","last_unstale":"2024-03-11T19:38:38.546651+0000","last_undegraded":"2024-03-11T19:38:38.546651+0000","last_fullsized":"2024-03-11T19:38:38.546651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:40:36.988348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536573+0000","last_change":"2024-03-11T19:37:05.329050+0000","last_active":"2024-03-11T19:38:38.536573+0000","last_peered":"2024-03-11T19:38:38.536573+0000","last_clean":"2024-03-11T19:38:38.536573+0000","last_became_active":"2024-03-11T19:37:05.328702+0000","last_became_peered":"2024-03-11T19:37:05.328702+0000","last_unstale":"2024-03-11T19:38:38.536573+0000","last_undegraded":"2024-03-11T19:38:38.536573+0000","last_fullsized":"2024-03-11T19:38:38.536573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:19:52.180884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549103+0000","last_change":"2024-03-11T19:37:05.322748+0000","last_active":"2024-03-11T19:38:38.549103+0000","last_peered":"2024-03-11T19:38:38.549103+0000","last_clean":"2024-03-11T19:38:38.549103+0000","last_became_active":"2024-03-11T19:37:05.322376+0000","last_became_peered":"2024-03-11T19:37:05.322376+0000","last_unstale":"2024-03-11T19:38:38.549103+0000","last_undegraded":"2024-03-11T19:38:38.549103+0000","last_fullsized":"2024-03-11T19:38:38.549103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:40.926854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543305+0000","last_change":"2024-03-11T19:37:05.323954+0000","last_active":"2024-03-11T19:38:38.543305+0000","last_peered":"2024-03-11T19:38:38.543305+0000","last_clean":"2024-03-11T19:38:38.543305+0000","last_became_active":"2024-03-11T19:37:05.323586+0000","last_became_peered":"2024-03-11T19:37:05.323586+0000","last_unstale":"2024-03-11T19:38:38.543305+0000","last_undegraded":"2024-03-11T19:38:38.543305+0000","last_fullsized":"2024-03-11T19:38:38.543305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:38.286891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536729+0000","last_change":"2024-03-11T19:37:05.334138+0000","last_active":"2024-03-11T19:38:38.536729+0000","last_peered":"2024-03-11T19:38:38.536729+0000","last_clean":"2024-03-11T19:38:38.536729+0000","last_became_active":"2024-03-11T19:37:05.333837+0000","last_became_peered":"2024-03-11T19:37:05.333837+0000","last_unstale":"2024-03-11T19:38:38.536729+0000","last_undegraded":"2024-03-11T19:38:38.536729+0000","last_fullsized":"2024-03-11T19:38:38.536729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:49:30.724176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.552174+0000","last_change":"2024-03-11T19:37:05.336888+0000","last_active":"2024-03-11T19:38:38.552174+0000","last_peered":"2024-03-11T19:38:38.552174+0000","last_clean":"2024-03-11T19:38:38.552174+0000","last_became_active":"2024-03-11T19:37:05.336256+0000","last_became_peered":"2024-03-11T19:37:05.336256+0000","last_unstale":"2024-03-11T19:38:38.552174+0000","last_undegraded":"2024-03-11T19:38:38.552174+0000","last_fullsized":"2024-03-11T19:38:38.552174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:54:31.526986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548957+0000","last_change":"2024-03-11T19:37:05.321959+0000","last_active":"2024-03-11T19:38:38.548957+0000","last_peered":"2024-03-11T19:38:38.548957+0000","last_clean":"2024-03-11T19:38:38.548957+0000","last_became_active":"2024-03-11T19:37:05.321043+0000","last_became_peered":"2024-03-11T19:37:05.321043+0000","last_unstale":"2024-03-11T19:38:38.548957+0000","last_undegraded":"2024-03-11T19:38:38.548957+0000","last_fullsized":"2024-03-11T19:38:38.548957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:13.121003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548202+0000","last_change":"2024-03-11T19:37:05.324940+0000","last_active":"2024-03-11T19:38:38.548202+0000","last_peered":"2024-03-11T19:38:38.548202+0000","last_clean":"2024-03-11T19:38:38.548202+0000","last_became_active":"2024-03-11T19:37:05.323427+0000","last_became_peered":"2024-03-11T19:37:05.323427+0000","last_unstale":"2024-03-11T19:38:38.548202+0000","last_undegraded":"2024-03-11T19:38:38.548202+0000","last_fullsized":"2024-03-11T19:38:38.548202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:23.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548766+0000","last_change":"2024-03-11T19:37:05.332694+0000","last_active":"2024-03-11T19:38:38.548766+0000","last_peered":"2024-03-11T19:38:38.548766+0000","last_clean":"2024-03-11T19:38:38.548766+0000","last_became_active":"2024-03-11T19:37:05.330446+0000","last_became_peered":"2024-03-11T19:37:05.330446+0000","last_unstale":"2024-03-11T19:38:38.548766+0000","last_undegraded":"2024-03-11T19:38:38.548766+0000","last_fullsized":"2024-03-11T19:38:38.548766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:37:40.196933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535856+0000","last_change":"2024-03-11T19:37:05.325975+0000","last_active":"2024-03-11T19:38:38.535856+0000","last_peered":"2024-03-11T19:38:38.535856+0000","last_clean":"2024-03-11T19:38:38.535856+0000","last_became_active":"2024-03-11T19:37:05.325649+0000","last_became_peered":"2024-03-11T19:37:05.325649+0000","last_unstale":"2024-03-11T19:38:38.535856+0000","last_undegraded":"2024-03-11T19:38:38.535856+0000","last_fullsized":"2024-03-11T19:38:38.535856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:36.373272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.547237+0000","last_change":"2024-03-11T19:37:05.335546+0000","last_active":"2024-03-11T19:38:38.547237+0000","last_peered":"2024-03-11T19:38:38.547237+0000","last_clean":"2024-03-11T19:38:38.547237+0000","last_became_active":"2024-03-11T19:37:05.335220+0000","last_became_peered":"2024-03-11T19:37:05.335220+0000","last_unstale":"2024-03-11T19:38:38.547237+0000","last_undegraded":"2024-03-11T19:38:38.547237+0000","last_fullsized":"2024-03-11T19:38:38.547237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:13.116355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.546152+0000","last_change":"2024-03-11T19:37:05.338955+0000","last_active":"2024-03-11T19:38:38.546152+0000","last_peered":"2024-03-11T19:38:38.546152+0000","last_clean":"2024-03-11T19:38:38.546152+0000","last_became_active":"2024-03-11T19:37:05.338574+0000","last_became_peered":"2024-03-11T19:37:05.338574+0000","last_unstale":"2024-03-11T19:38:38.546152+0000","last_undegraded":"2024-03-11T19:38:38.546152+0000","last_fullsized":"2024-03-11T19:38:38.546152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:27:27.754077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553899+0000","last_change":"2024-03-11T19:37:05.334578+0000","last_active":"2024-03-11T19:38:38.553899+0000","last_peered":"2024-03-11T19:38:38.553899+0000","last_clean":"2024-03-11T19:38:38.553899+0000","last_became_active":"2024-03-11T19:37:05.334267+0000","last_became_peered":"2024-03-11T19:37:05.334267+0000","last_unstale":"2024-03-11T19:38:38.553899+0000","last_undegraded":"2024-03-11T19:38:38.553899+0000","last_fullsized":"2024-03-11T19:38:38.553899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:31.704187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543563+0000","last_change":"2024-03-11T19:37:05.341178+0000","last_active":"2024-03-11T19:38:38.543563+0000","last_peered":"2024-03-11T19:38:38.543563+0000","last_clean":"2024-03-11T19:38:38.543563+0000","last_became_active":"2024-03-11T19:37:05.340837+0000","last_became_peered":"2024-03-11T19:37:05.340837+0000","last_unstale":"2024-03-11T19:38:38.543563+0000","last_undegraded":"2024-03-11T19:38:38.543563+0000","last_fullsized":"2024-03-11T19:38:38.543563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:39:54.460937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.538263+0000","last_change":"2024-03-11T19:37:05.330981+0000","last_active":"2024-03-11T19:38:38.538263+0000","last_peered":"2024-03-11T19:38:38.538263+0000","last_clean":"2024-03-11T19:38:38.538263+0000","last_became_active":"2024-03-11T19:37:05.330638+0000","last_became_peered":"2024-03-11T19:37:05.330638+0000","last_unstale":"2024-03-11T19:38:38.538263+0000","last_undegraded":"2024-03-11T19:38:38.538263+0000","last_fullsized":"2024-03-11T19:38:38.538263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:36:12.565936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535474+0000","last_change":"2024-03-11T19:37:05.329866+0000","last_active":"2024-03-11T19:38:38.535474+0000","last_peered":"2024-03-11T19:38:38.535474+0000","last_clean":"2024-03-11T19:38:38.535474+0000","last_became_active":"2024-03-11T19:37:05.329572+0000","last_became_peered":"2024-03-11T19:37:05.329572+0000","last_unstale":"2024-03-11T19:38:38.535474+0000","last_undegraded":"2024-03-11T19:38:38.535474+0000","last_fullsized":"2024-03-11T19:38:38.535474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:40:04.482929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454096+0000","last_change":"2024-03-11T19:37:04.587149+0000","last_active":"2024-03-11T19:38:10.454096+0000","last_peered":"2024-03-11T19:38:10.454096+0000","last_clean":"2024-03-11T19:38:10.454096+0000","last_became_active":"2024-03-11T19:37:04.586013+0000","last_became_peered":"2024-03-11T19:37:04.586013+0000","last_unstale":"2024-03-11T19:38:10.454096+0000","last_undegraded":"2024-03-11T19:38:10.454096+0000","last_fullsized":"2024-03-11T19:38:10.454096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:15.758154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536238+0000","last_change":"2024-03-11T19:37:05.327979+0000","last_active":"2024-03-11T19:38:38.536238+0000","last_peered":"2024-03-11T19:38:38.536238+0000","last_clean":"2024-03-11T19:38:38.536238+0000","last_became_active":"2024-03-11T19:37:05.327647+0000","last_became_peered":"2024-03-11T19:37:05.327647+0000","last_unstale":"2024-03-11T19:38:38.536238+0000","last_undegraded":"2024-03-11T19:38:38.536238+0000","last_fullsized":"2024-03-11T19:38:38.536238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:09.514766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452950+0000","last_change":"2024-03-11T19:37:04.582937+0000","last_active":"2024-03-11T19:38:10.452950+0000","last_peered":"2024-03-11T19:38:10.452950+0000","last_clean":"2024-03-11T19:38:10.452950+0000","last_became_active":"2024-03-11T19:37:04.582622+0000","last_became_peered":"2024-03-11T19:37:04.582622+0000","last_unstale":"2024-03-11T19:38:10.452950+0000","last_undegraded":"2024-03-11T19:38:10.452950+0000","last_fullsized":"2024-03-11T19:38:10.452950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:02.275698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.555090+0000","last_change":"2024-03-11T19:37:05.327191+0000","last_active":"2024-03-11T19:38:38.555090+0000","last_peered":"2024-03-11T19:38:38.555090+0000","last_clean":"2024-03-11T19:38:38.555090+0000","last_became_active":"2024-03-11T19:37:05.326001+0000","last_became_peered":"2024-03-11T19:37:05.326001+0000","last_unstale":"2024-03-11T19:38:38.555090+0000","last_undegraded":"2024-03-11T19:38:38.555090+0000","last_fullsized":"2024-03-11T19:38:38.555090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:27.301321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151969+0000","last_change":"2024-03-11T19:37:04.587054+0000","last_active":"2024-03-11T19:38:10.151969+0000","last_peered":"2024-03-11T19:38:10.151969+0000","last_clean":"2024-03-11T19:38:10.151969+0000","last_became_active":"2024-03-11T19:37:04.586104+0000","last_became_peered":"2024-03-11T19:37:04.586104+0000","last_unstale":"2024-03-11T19:38:10.151969+0000","last_undegraded":"2024-03-11T19:38:10.151969+0000","last_fullsized":"2024-03-11T19:38:10.151969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:34.913677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549578+0000","last_change":"2024-03-11T19:37:05.333288+0000","last_active":"2024-03-11T19:38:38.549578+0000","last_peered":"2024-03-11T19:38:38.549578+0000","last_clean":"2024-03-11T19:38:38.549578+0000","last_became_active":"2024-03-11T19:37:05.332882+0000","last_became_peered":"2024-03-11T19:37:05.332882+0000","last_unstale":"2024-03-11T19:38:38.549578+0000","last_undegraded":"2024-03-11T19:38:38.549578+0000","last_fullsized":"2024-03-11T19:38:38.549578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:45.680608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151628+0000","last_change":"2024-03-11T19:37:04.583660+0000","last_active":"2024-03-11T19:38:10.151628+0000","last_peered":"2024-03-11T19:38:10.151628+0000","last_clean":"2024-03-11T19:38:10.151628+0000","last_became_active":"2024-03-11T19:37:04.583360+0000","last_became_peered":"2024-03-11T19:37:04.583360+0000","last_unstale":"2024-03-11T19:38:10.151628+0000","last_undegraded":"2024-03-11T19:38:10.151628+0000","last_fullsized":"2024-03-11T19:38:10.151628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:00.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.554758+0000","last_change":"2024-03-11T19:37:05.335649+0000","last_active":"2024-03-11T19:38:38.554758+0000","last_peered":"2024-03-11T19:38:38.554758+0000","last_clean":"2024-03-11T19:38:38.554758+0000","last_became_active":"2024-03-11T19:37:05.335255+0000","last_became_peered":"2024-03-11T19:37:05.335255+0000","last_unstale":"2024-03-11T19:38:38.554758+0000","last_undegraded":"2024-03-11T19:38:38.554758+0000","last_fullsized":"2024-03-11T19:38:38.554758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:49:09.772765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":136,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:53.782803+0000","last_change":"2024-03-11T19:37:04.586002+0000","last_active":"2024-03-11T19:38:53.782803+0000","last_peered":"2024-03-11T19:38:53.782803+0000","last_clean":"2024-03-11T19:38:53.782803+0000","last_became_active":"2024-03-11T19:37:04.585551+0000","last_became_peered":"2024-03-11T19:37:04.585551+0000","last_unstale":"2024-03-11T19:38:53.782803+0000","last_undegraded":"2024-03-11T19:38:53.782803+0000","last_fullsized":"2024-03-11T19:38:53.782803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:34:12.663801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.542150+0000","last_change":"2024-03-11T19:37:05.325168+0000","last_active":"2024-03-11T19:38:38.542150+0000","last_peered":"2024-03-11T19:38:38.542150+0000","last_clean":"2024-03-11T19:38:38.542150+0000","last_became_active":"2024-03-11T19:37:05.324848+0000","last_became_peered":"2024-03-11T19:37:05.324848+0000","last_unstale":"2024-03-11T19:38:38.542150+0000","last_undegraded":"2024-03-11T19:38:38.542150+0000","last_fullsized":"2024-03-11T19:38:38.542150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:27.423877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452883+0000","last_change":"2024-03-11T19:37:04.582727+0000","last_active":"2024-03-11T19:38:10.452883+0000","last_peered":"2024-03-11T19:38:10.452883+0000","last_clean":"2024-03-11T19:38:10.452883+0000","last_became_active":"2024-03-11T19:37:04.582359+0000","last_became_peered":"2024-03-11T19:37:04.582359+0000","last_unstale":"2024-03-11T19:38:10.452883+0000","last_undegraded":"2024-03-11T19:38:10.452883+0000","last_fullsized":"2024-03-11T19:38:10.452883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:05.591537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.544557+0000","last_change":"2024-03-11T19:37:05.335442+0000","last_active":"2024-03-11T19:38:38.544557+0000","last_peered":"2024-03-11T19:38:38.544557+0000","last_clean":"2024-03-11T19:38:38.544557+0000","last_became_active":"2024-03-11T19:37:05.335003+0000","last_became_peered":"2024-03-11T19:37:05.335003+0000","last_unstale":"2024-03-11T19:38:38.544557+0000","last_undegraded":"2024-03-11T19:38:38.544557+0000","last_fullsized":"2024-03-11T19:38:38.544557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:41.590898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453615+0000","last_change":"2024-03-11T19:37:04.589957+0000","last_active":"2024-03-11T19:38:10.453615+0000","last_peered":"2024-03-11T19:38:10.453615+0000","last_clean":"2024-03-11T19:38:10.453615+0000","last_became_active":"2024-03-11T19:37:04.589539+0000","last_became_peered":"2024-03-11T19:37:04.589539+0000","last_unstale":"2024-03-11T19:38:10.453615+0000","last_undegraded":"2024-03-11T19:38:10.453615+0000","last_fullsized":"2024-03-11T19:38:10.453615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:48:52.764057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535727+0000","last_change":"2024-03-11T19:37:05.331380+0000","last_active":"2024-03-11T19:38:38.535727+0000","last_peered":"2024-03-11T19:38:38.535727+0000","last_clean":"2024-03-11T19:38:38.535727+0000","last_became_active":"2024-03-11T19:37:05.331000+0000","last_became_peered":"2024-03-11T19:37:05.331000+0000","last_unstale":"2024-03-11T19:38:38.535727+0000","last_undegraded":"2024-03-11T19:38:38.535727+0000","last_fullsized":"2024-03-11T19:38:38.535727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:58:47.278195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452916+0000","last_change":"2024-03-11T19:37:04.584128+0000","last_active":"2024-03-11T19:38:10.452916+0000","last_peered":"2024-03-11T19:38:10.452916+0000","last_clean":"2024-03-11T19:38:10.452916+0000","last_became_active":"2024-03-11T19:37:04.583842+0000","last_became_peered":"2024-03-11T19:37:04.583842+0000","last_unstale":"2024-03-11T19:38:10.452916+0000","last_undegraded":"2024-03-11T19:38:10.452916+0000","last_fullsized":"2024-03-11T19:38:10.452916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:18.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.534652+0000","last_change":"2024-03-11T19:37:05.328999+0000","last_active":"2024-03-11T19:38:38.534652+0000","last_peered":"2024-03-11T19:38:38.534652+0000","last_clean":"2024-03-11T19:38:38.534652+0000","last_became_active":"2024-03-11T19:37:05.328620+0000","last_became_peered":"2024-03-11T19:37:05.328620+0000","last_unstale":"2024-03-11T19:38:38.534652+0000","last_undegraded":"2024-03-11T19:38:38.534652+0000","last_fullsized":"2024-03-11T19:38:38.534652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:40.191922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":132,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:56.786615+0000","last_change":"2024-03-11T19:37:04.587956+0000","last_active":"2024-03-11T19:38:56.786615+0000","last_peered":"2024-03-11T19:38:56.786615+0000","last_clean":"2024-03-11T19:38:56.786615+0000","last_became_active":"2024-03-11T19:37:04.587568+0000","last_became_peered":"2024-03-11T19:37:04.587568+0000","last_unstale":"2024-03-11T19:38:56.786615+0000","last_undegraded":"2024-03-11T19:38:56.786615+0000","last_fullsized":"2024-03-11T19:38:56.786615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:45:44.019436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549054+0000","last_change":"2024-03-11T19:37:05.336912+0000","last_active":"2024-03-11T19:38:38.549054+0000","last_peered":"2024-03-11T19:38:38.549054+0000","last_clean":"2024-03-11T19:38:38.549054+0000","last_became_active":"2024-03-11T19:37:05.336397+0000","last_became_peered":"2024-03-11T19:37:05.336397+0000","last_unstale":"2024-03-11T19:38:38.549054+0000","last_undegraded":"2024-03-11T19:38:38.549054+0000","last_fullsized":"2024-03-11T19:38:38.549054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:29.375396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150574+0000","last_change":"2024-03-11T19:37:04.584382+0000","last_active":"2024-03-11T19:38:10.150574+0000","last_peered":"2024-03-11T19:38:10.150574+0000","last_clean":"2024-03-11T19:38:10.150574+0000","last_became_active":"2024-03-11T19:37:04.584026+0000","last_became_peered":"2024-03-11T19:37:04.584026+0000","last_unstale":"2024-03-11T19:38:10.150574+0000","last_undegraded":"2024-03-11T19:38:10.150574+0000","last_fullsized":"2024-03-11T19:38:10.150574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:28.152903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.546098+0000","last_change":"2024-03-11T19:37:05.331385+0000","last_active":"2024-03-11T19:38:38.546098+0000","last_peered":"2024-03-11T19:38:38.546098+0000","last_clean":"2024-03-11T19:38:38.546098+0000","last_became_active":"2024-03-11T19:37:05.331069+0000","last_became_peered":"2024-03-11T19:37:05.331069+0000","last_unstale":"2024-03-11T19:38:38.546098+0000","last_undegraded":"2024-03-11T19:38:38.546098+0000","last_fullsized":"2024-03-11T19:38:38.546098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:18.375605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453568+0000","last_change":"2024-03-11T19:37:04.586493+0000","last_active":"2024-03-11T19:38:10.453568+0000","last_peered":"2024-03-11T19:38:10.453568+0000","last_clean":"2024-03-11T19:38:10.453568+0000","last_became_active":"2024-03-11T19:37:04.586185+0000","last_became_peered":"2024-03-11T19:37:04.586185+0000","last_unstale":"2024-03-11T19:38:10.453568+0000","last_undegraded":"2024-03-11T19:38:10.453568+0000","last_fullsized":"2024-03-11T19:38:10.453568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:00.899752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.555300+0000","last_change":"2024-03-11T19:37:05.336414+0000","last_active":"2024-03-11T19:38:38.555300+0000","last_peered":"2024-03-11T19:38:38.555300+0000","last_clean":"2024-03-11T19:38:38.555300+0000","last_became_active":"2024-03-11T19:37:05.334536+0000","last_became_peered":"2024-03-11T19:37:05.334536+0000","last_unstale":"2024-03-11T19:38:38.555300+0000","last_undegraded":"2024-03-11T19:38:38.555300+0000","last_fullsized":"2024-03-11T19:38:38.555300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:50:19.109975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'34","reported_seq":281,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:57.788731+0000","last_change":"2024-03-11T19:37:04.584544+0000","last_active":"2024-03-11T19:38:57.788731+0000","last_peered":"2024-03-11T19:38:57.788731+0000","last_clean":"2024-03-11T19:38:57.788731+0000","last_became_active":"2024-03-11T19:37:04.584198+0000","last_became_peered":"2024-03-11T19:37:04.584198+0000","last_unstale":"2024-03-11T19:38:57.788731+0000","last_undegraded":"2024-03-11T19:38:57.788731+0000","last_fullsized":"2024-03-11T19:38:57.788731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:01:56.744533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60580,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":228,"num_read_kb":173,"num_write":35,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.533749+0000","last_change":"2024-03-11T19:37:05.331077+0000","last_active":"2024-03-11T19:38:38.533749+0000","last_peered":"2024-03-11T19:38:38.533749+0000","last_clean":"2024-03-11T19:38:38.533749+0000","last_became_active":"2024-03-11T19:37:05.330708+0000","last_became_peered":"2024-03-11T19:37:05.330708+0000","last_unstale":"2024-03-11T19:38:38.533749+0000","last_undegraded":"2024-03-11T19:38:38.533749+0000","last_fullsized":"2024-03-11T19:38:38.533749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:54:55.019268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155548+0000","last_change":"2024-03-11T19:37:04.583353+0000","last_active":"2024-03-11T19:38:10.155548+0000","last_peered":"2024-03-11T19:38:10.155548+0000","last_clean":"2024-03-11T19:38:10.155548+0000","last_became_active":"2024-03-11T19:37:04.581974+0000","last_became_peered":"2024-03-11T19:37:04.581974+0000","last_unstale":"2024-03-11T19:38:10.155548+0000","last_undegraded":"2024-03-11T19:38:10.155548+0000","last_fullsized":"2024-03-11T19:38:10.155548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:41.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553558+0000","last_change":"2024-03-11T19:37:05.329498+0000","last_active":"2024-03-11T19:38:38.553558+0000","last_peered":"2024-03-11T19:38:38.553558+0000","last_clean":"2024-03-11T19:38:38.553558+0000","last_became_active":"2024-03-11T19:37:05.329186+0000","last_became_peered":"2024-03-11T19:37:05.329186+0000","last_unstale":"2024-03-11T19:38:38.553558+0000","last_undegraded":"2024-03-11T19:38:38.553558+0000","last_fullsized":"2024-03-11T19:38:38.553558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:33:45.012049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453150+0000","last_change":"2024-03-11T19:37:04.582027+0000","last_active":"2024-03-11T19:38:10.453150+0000","last_peered":"2024-03-11T19:38:10.453150+0000","last_clean":"2024-03-11T19:38:10.453150+0000","last_became_active":"2024-03-11T19:37:04.581678+0000","last_became_peered":"2024-03-11T19:37:04.581678+0000","last_unstale":"2024-03-11T19:38:10.453150+0000","last_undegraded":"2024-03-11T19:38:10.453150+0000","last_fullsized":"2024-03-11T19:38:10.453150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:03:40.688053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553484+0000","last_change":"2024-03-11T19:38:10.455076+0000","last_active":"2024-03-11T19:38:38.553484+0000","last_peered":"2024-03-11T19:38:38.553484+0000","last_clean":"2024-03-11T19:38:38.553484+0000","last_became_active":"2024-03-11T19:38:10.454631+0000","last_became_peered":"2024-03-11T19:38:10.454631+0000","last_unstale":"2024-03-11T19:38:38.553484+0000","last_undegraded":"2024-03-11T19:38:38.553484+0000","last_fullsized":"2024-03-11T19:38:38.553484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:26:36.651449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":136,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:55.785449+0000","last_change":"2024-03-11T19:37:04.576415+0000","last_active":"2024-03-11T19:38:55.785449+0000","last_peered":"2024-03-11T19:38:55.785449+0000","last_clean":"2024-03-11T19:38:55.785449+0000","last_became_active":"2024-03-11T19:37:04.575978+0000","last_became_peered":"2024-03-11T19:37:04.575978+0000","last_unstale":"2024-03-11T19:38:55.785449+0000","last_undegraded":"2024-03-11T19:38:55.785449+0000","last_fullsized":"2024-03-11T19:38:55.785449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:20.124813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.550544+0000","last_change":"2024-03-11T19:37:05.326258+0000","last_active":"2024-03-11T19:38:38.550544+0000","last_peered":"2024-03-11T19:38:38.550544+0000","last_clean":"2024-03-11T19:38:38.550544+0000","last_became_active":"2024-03-11T19:37:05.325909+0000","last_became_peered":"2024-03-11T19:37:05.325909+0000","last_unstale":"2024-03-11T19:38:38.550544+0000","last_undegraded":"2024-03-11T19:38:38.550544+0000","last_fullsized":"2024-03-11T19:38:38.550544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:25.875603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.482823+0000","last_change":"2024-03-11T19:38:10.482380+0000","last_active":"2024-03-11T19:38:10.482823+0000","last_peered":"2024-03-11T19:38:10.482823+0000","last_clean":"2024-03-11T19:38:10.482823+0000","last_became_active":"2024-03-11T19:38:10.455301+0000","last_became_peered":"2024-03-11T19:38:10.455301+0000","last_unstale":"2024-03-11T19:38:10.482823+0000","last_undegraded":"2024-03-11T19:38:10.482823+0000","last_fullsized":"2024-03-11T19:38:10.482823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:28:41.620522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553853+0000","last_change":"2024-03-11T19:37:05.336727+0000","last_active":"2024-03-11T19:38:38.553853+0000","last_peered":"2024-03-11T19:38:38.553853+0000","last_clean":"2024-03-11T19:38:38.553853+0000","last_became_active":"2024-03-11T19:37:05.336384+0000","last_became_peered":"2024-03-11T19:37:05.336384+0000","last_unstale":"2024-03-11T19:38:38.553853+0000","last_undegraded":"2024-03-11T19:38:38.553853+0000","last_fullsized":"2024-03-11T19:38:38.553853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:02.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151827+0000","last_change":"2024-03-11T19:37:04.586814+0000","last_active":"2024-03-11T19:38:10.151827+0000","last_peered":"2024-03-11T19:38:10.151827+0000","last_clean":"2024-03-11T19:38:10.151827+0000","last_became_active":"2024-03-11T19:37:04.586512+0000","last_became_peered":"2024-03-11T19:37:04.586512+0000","last_unstale":"2024-03-11T19:38:10.151827+0000","last_undegraded":"2024-03-11T19:38:10.151827+0000","last_fullsized":"2024-03-11T19:38:10.151827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:04:23.979783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548507+0000","last_change":"2024-03-11T19:37:05.335353+0000","last_active":"2024-03-11T19:38:38.548507+0000","last_peered":"2024-03-11T19:38:38.548507+0000","last_clean":"2024-03-11T19:38:38.548507+0000","last_became_active":"2024-03-11T19:37:05.335066+0000","last_became_peered":"2024-03-11T19:37:05.335066+0000","last_unstale":"2024-03-11T19:38:38.548507+0000","last_undegraded":"2024-03-11T19:38:38.548507+0000","last_fullsized":"2024-03-11T19:38:38.548507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:11.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454552+0000","last_change":"2024-03-11T19:37:04.586679+0000","last_active":"2024-03-11T19:38:10.454552+0000","last_peered":"2024-03-11T19:38:10.454552+0000","last_clean":"2024-03-11T19:38:10.454552+0000","last_became_active":"2024-03-11T19:37:04.586316+0000","last_became_peered":"2024-03-11T19:37:04.586316+0000","last_unstale":"2024-03-11T19:38:10.454552+0000","last_undegraded":"2024-03-11T19:38:10.454552+0000","last_fullsized":"2024-03-11T19:38:10.454552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:58:21.115526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535653+0000","last_change":"2024-03-11T19:37:05.330206+0000","last_active":"2024-03-11T19:38:38.535653+0000","last_peered":"2024-03-11T19:38:38.535653+0000","last_clean":"2024-03-11T19:38:38.535653+0000","last_became_active":"2024-03-11T19:37:05.329847+0000","last_became_peered":"2024-03-11T19:37:05.329847+0000","last_unstale":"2024-03-11T19:38:38.535653+0000","last_undegraded":"2024-03-11T19:38:38.535653+0000","last_fullsized":"2024-03-11T19:38:38.535653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:14:52.532236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150350+0000","last_change":"2024-03-11T19:37:04.581031+0000","last_active":"2024-03-11T19:38:10.150350+0000","last_peered":"2024-03-11T19:38:10.150350+0000","last_clean":"2024-03-11T19:38:10.150350+0000","last_became_active":"2024-03-11T19:37:04.580728+0000","last_became_peered":"2024-03-11T19:37:04.580728+0000","last_unstale":"2024-03-11T19:38:10.150350+0000","last_undegraded":"2024-03-11T19:38:10.150350+0000","last_fullsized":"2024-03-11T19:38:10.150350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:43.401898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540064+0000","last_change":"2024-03-11T19:37:05.328686+0000","last_active":"2024-03-11T19:38:38.540064+0000","last_peered":"2024-03-11T19:38:38.540064+0000","last_clean":"2024-03-11T19:38:38.540064+0000","last_became_active":"2024-03-11T19:37:05.328372+0000","last_became_peered":"2024-03-11T19:37:05.328372+0000","last_unstale":"2024-03-11T19:38:38.540064+0000","last_undegraded":"2024-03-11T19:38:38.540064+0000","last_fullsized":"2024-03-11T19:38:38.540064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:38:33.216489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.544161+0000","last_change":"2024-03-11T19:37:05.335057+0000","last_active":"2024-03-11T19:38:38.544161+0000","last_peered":"2024-03-11T19:38:38.544161+0000","last_clean":"2024-03-11T19:38:38.544161+0000","last_became_active":"2024-03-11T19:37:05.334646+0000","last_became_peered":"2024-03-11T19:37:05.334646+0000","last_unstale":"2024-03-11T19:38:38.544161+0000","last_undegraded":"2024-03-11T19:38:38.544161+0000","last_fullsized":"2024-03-11T19:38:38.544161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:34:15.932429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":132,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:55.784973+0000","last_change":"2024-03-11T19:37:04.588095+0000","last_active":"2024-03-11T19:38:55.784973+0000","last_peered":"2024-03-11T19:38:55.784973+0000","last_clean":"2024-03-11T19:38:55.784973+0000","last_became_active":"2024-03-11T19:37:04.587757+0000","last_became_peered":"2024-03-11T19:37:04.587757+0000","last_unstale":"2024-03-11T19:38:55.784973+0000","last_undegraded":"2024-03-11T19:38:55.784973+0000","last_fullsized":"2024-03-11T19:38:55.784973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:57.926858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540458+0000","last_change":"2024-03-11T19:37:05.326806+0000","last_active":"2024-03-11T19:38:38.540458+0000","last_peered":"2024-03-11T19:38:38.540458+0000","last_clean":"2024-03-11T19:38:38.540458+0000","last_became_active":"2024-03-11T19:37:05.326506+0000","last_became_peered":"2024-03-11T19:37:05.326506+0000","last_unstale":"2024-03-11T19:38:38.540458+0000","last_undegraded":"2024-03-11T19:38:38.540458+0000","last_fullsized":"2024-03-11T19:38:38.540458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:20.664857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453750+0000","last_change":"2024-03-11T19:37:04.588459+0000","last_active":"2024-03-11T19:38:10.453750+0000","last_peered":"2024-03-11T19:38:10.453750+0000","last_clean":"2024-03-11T19:38:10.453750+0000","last_became_active":"2024-03-11T19:37:04.588000+0000","last_became_peered":"2024-03-11T19:37:04.588000+0000","last_unstale":"2024-03-11T19:38:10.453750+0000","last_undegraded":"2024-03-11T19:38:10.453750+0000","last_fullsized":"2024-03-11T19:38:10.453750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:20:47.717196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150281+0000","last_change":"2024-03-11T19:35:55.814178+0000","last_active":"2024-03-11T19:38:10.150281+0000","last_peered":"2024-03-11T19:38:10.150281+0000","last_clean":"2024-03-11T19:38:10.150281+0000","last_became_active":"2024-03-11T19:35:55.813697+0000","last_became_peered":"2024-03-11T19:35:55.813697+0000","last_unstale":"2024-03-11T19:38:10.150281+0000","last_undegraded":"2024-03-11T19:38:10.150281+0000","last_fullsized":"2024-03-11T19:38:10.150281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_clean_scrub_stamp":"2024-03-11T19:32:34.872292+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:50:27.486733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543638+0000","last_change":"2024-03-11T19:37:05.319614+0000","last_active":"2024-03-11T19:38:38.543638+0000","last_peered":"2024-03-11T19:38:38.543638+0000","last_clean":"2024-03-11T19:38:38.543638+0000","last_became_active":"2024-03-11T19:37:05.319277+0000","last_became_peered":"2024-03-11T19:37:05.319277+0000","last_unstale":"2024-03-11T19:38:38.543638+0000","last_undegraded":"2024-03-11T19:38:38.543638+0000","last_fullsized":"2024-03-11T19:38:38.543638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:27.329900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150075+0000","last_change":"2024-03-11T19:37:04.582250+0000","last_active":"2024-03-11T19:38:10.150075+0000","last_peered":"2024-03-11T19:38:10.150075+0000","last_clean":"2024-03-11T19:38:10.150075+0000","last_became_active":"2024-03-11T19:37:04.581938+0000","last_became_peered":"2024-03-11T19:37:04.581938+0000","last_unstale":"2024-03-11T19:38:10.150075+0000","last_undegraded":"2024-03-11T19:38:10.150075+0000","last_fullsized":"2024-03-11T19:38:10.150075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:23.466067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151834+0000","last_change":"2024-03-11T19:37:04.590699+0000","last_active":"2024-03-11T19:38:10.151834+0000","last_peered":"2024-03-11T19:38:10.151834+0000","last_clean":"2024-03-11T19:38:10.151834+0000","last_became_active":"2024-03-11T19:37:04.590424+0000","last_became_peered":"2024-03-11T19:37:04.590424+0000","last_unstale":"2024-03-11T19:38:10.151834+0000","last_undegraded":"2024-03-11T19:38:10.151834+0000","last_fullsized":"2024-03-11T19:38:10.151834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:49.075335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.538473+0000","last_change":"2024-03-11T19:37:05.323357+0000","last_active":"2024-03-11T19:38:38.538473+0000","last_peered":"2024-03-11T19:38:38.538473+0000","last_clean":"2024-03-11T19:38:38.538473+0000","last_became_active":"2024-03-11T19:37:05.322980+0000","last_became_peered":"2024-03-11T19:37:05.322980+0000","last_unstale":"2024-03-11T19:38:38.538473+0000","last_undegraded":"2024-03-11T19:38:38.538473+0000","last_fullsized":"2024-03-11T19:38:38.538473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:19.106375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.589118+0000","last_change":"2024-03-11T19:37:04.585232+0000","last_active":"2024-03-11T19:38:38.589118+0000","last_peered":"2024-03-11T19:38:38.589118+0000","last_clean":"2024-03-11T19:38:38.589118+0000","last_became_active":"2024-03-11T19:37:04.584904+0000","last_became_peered":"2024-03-11T19:37:04.584904+0000","last_unstale":"2024-03-11T19:38:38.589118+0000","last_undegraded":"2024-03-11T19:38:38.589118+0000","last_fullsized":"2024-03-11T19:38:38.589118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:56:46.519961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553409+0000","last_change":"2024-03-11T19:37:05.332019+0000","last_active":"2024-03-11T19:38:38.553409+0000","last_peered":"2024-03-11T19:38:38.553409+0000","last_clean":"2024-03-11T19:38:38.553409+0000","last_became_active":"2024-03-11T19:37:05.331673+0000","last_became_peered":"2024-03-11T19:37:05.331673+0000","last_unstale":"2024-03-11T19:38:38.553409+0000","last_undegraded":"2024-03-11T19:38:38.553409+0000","last_fullsized":"2024-03-11T19:38:38.553409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:49:45.787082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454347+0000","last_change":"2024-03-11T19:37:04.583404+0000","last_active":"2024-03-11T19:38:10.454347+0000","last_peered":"2024-03-11T19:38:10.454347+0000","last_clean":"2024-03-11T19:38:10.454347+0000","last_became_active":"2024-03-11T19:37:04.583025+0000","last_became_peered":"2024-03-11T19:37:04.583025+0000","last_unstale":"2024-03-11T19:38:10.454347+0000","last_undegraded":"2024-03-11T19:38:10.454347+0000","last_fullsized":"2024-03-11T19:38:10.454347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:02:25.202591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549246+0000","last_change":"2024-03-11T19:37:05.331711+0000","last_active":"2024-03-11T19:38:38.549246+0000","last_peered":"2024-03-11T19:38:38.549246+0000","last_clean":"2024-03-11T19:38:38.549246+0000","last_became_active":"2024-03-11T19:37:05.330851+0000","last_became_peered":"2024-03-11T19:37:05.330851+0000","last_unstale":"2024-03-11T19:38:38.549246+0000","last_undegraded":"2024-03-11T19:38:38.549246+0000","last_fullsized":"2024-03-11T19:38:38.549246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:18:26.692068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448230+0000","last_change":"2024-03-11T19:37:04.586501+0000","last_active":"2024-03-11T19:38:10.448230+0000","last_peered":"2024-03-11T19:38:10.448230+0000","last_clean":"2024-03-11T19:38:10.448230+0000","last_became_active":"2024-03-11T19:37:04.586200+0000","last_became_peered":"2024-03-11T19:37:04.586200+0000","last_unstale":"2024-03-11T19:38:10.448230+0000","last_undegraded":"2024-03-11T19:38:10.448230+0000","last_fullsized":"2024-03-11T19:38:10.448230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:56.635375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.510995+0000","last_change":"2024-03-11T19:37:05.332402+0000","last_active":"2024-03-11T19:38:38.510995+0000","last_peered":"2024-03-11T19:38:38.510995+0000","last_clean":"2024-03-11T19:38:38.510995+0000","last_became_active":"2024-03-11T19:37:05.332087+0000","last_became_peered":"2024-03-11T19:37:05.332087+0000","last_unstale":"2024-03-11T19:38:38.510995+0000","last_undegraded":"2024-03-11T19:38:38.510995+0000","last_fullsized":"2024-03-11T19:38:38.510995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:10:23.882296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452565+0000","last_change":"2024-03-11T19:37:04.584978+0000","last_active":"2024-03-11T19:38:10.452565+0000","last_peered":"2024-03-11T19:38:10.452565+0000","last_clean":"2024-03-11T19:38:10.452565+0000","last_became_active":"2024-03-11T19:37:04.584556+0000","last_became_peered":"2024-03-11T19:37:04.584556+0000","last_unstale":"2024-03-11T19:38:10.452565+0000","last_undegraded":"2024-03-11T19:38:10.452565+0000","last_fullsized":"2024-03-11T19:38:10.452565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:39:48.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.539294+0000","last_change":"2024-03-11T19:37:05.334453+0000","last_active":"2024-03-11T19:38:38.539294+0000","last_peered":"2024-03-11T19:38:38.539294+0000","last_clean":"2024-03-11T19:38:38.539294+0000","last_became_active":"2024-03-11T19:37:05.334009+0000","last_became_peered":"2024-03-11T19:37:05.334009+0000","last_unstale":"2024-03-11T19:38:38.539294+0000","last_undegraded":"2024-03-11T19:38:38.539294+0000","last_fullsized":"2024-03-11T19:38:38.539294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:43.402674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155193+0000","last_change":"2024-03-11T19:37:04.583181+0000","last_active":"2024-03-11T19:38:10.155193+0000","last_peered":"2024-03-11T19:38:10.155193+0000","last_clean":"2024-03-11T19:38:10.155193+0000","last_became_active":"2024-03-11T19:37:04.582881+0000","last_became_peered":"2024-03-11T19:37:04.582881+0000","last_unstale":"2024-03-11T19:38:10.155193+0000","last_undegraded":"2024-03-11T19:38:10.155193+0000","last_fullsized":"2024-03-11T19:38:10.155193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:05:06.634150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.530930+0000","last_change":"2024-03-11T19:37:05.333642+0000","last_active":"2024-03-11T19:38:38.530930+0000","last_peered":"2024-03-11T19:38:38.530930+0000","last_clean":"2024-03-11T19:38:38.530930+0000","last_became_active":"2024-03-11T19:37:05.333121+0000","last_became_peered":"2024-03-11T19:37:05.333121+0000","last_unstale":"2024-03-11T19:38:38.530930+0000","last_undegraded":"2024-03-11T19:38:38.530930+0000","last_fullsized":"2024-03-11T19:38:38.530930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:07:21.217101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.458830+0000","last_change":"2024-03-11T19:38:10.458358+0000","last_active":"2024-03-11T19:38:10.458830+0000","last_peered":"2024-03-11T19:38:10.458830+0000","last_clean":"2024-03-11T19:38:10.458830+0000","last_became_active":"2024-03-11T19:38:10.449890+0000","last_became_peered":"2024-03-11T19:38:10.449890+0000","last_unstale":"2024-03-11T19:38:10.458830+0000","last_undegraded":"2024-03-11T19:38:10.458830+0000","last_fullsized":"2024-03-11T19:38:10.458830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:51:00.265814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.554245+0000","last_change":"2024-03-11T19:37:05.326992+0000","last_active":"2024-03-11T19:38:38.554245+0000","last_peered":"2024-03-11T19:38:38.554245+0000","last_clean":"2024-03-11T19:38:38.554245+0000","last_became_active":"2024-03-11T19:37:05.326676+0000","last_became_peered":"2024-03-11T19:37:05.326676+0000","last_unstale":"2024-03-11T19:38:38.554245+0000","last_undegraded":"2024-03-11T19:38:38.554245+0000","last_fullsized":"2024-03-11T19:38:38.554245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:31:10.310650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155451+0000","last_change":"2024-03-11T19:37:04.583631+0000","last_active":"2024-03-11T19:38:10.155451+0000","last_peered":"2024-03-11T19:38:10.155451+0000","last_clean":"2024-03-11T19:38:10.155451+0000","last_became_active":"2024-03-11T19:37:04.583031+0000","last_became_peered":"2024-03-11T19:37:04.583031+0000","last_unstale":"2024-03-11T19:38:10.155451+0000","last_undegraded":"2024-03-11T19:38:10.155451+0000","last_fullsized":"2024-03-11T19:38:10.155451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:20.695496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553916+0000","last_change":"2024-03-11T19:37:05.331008+0000","last_active":"2024-03-11T19:38:38.553916+0000","last_peered":"2024-03-11T19:38:38.553916+0000","last_clean":"2024-03-11T19:38:38.553916+0000","last_became_active":"2024-03-11T19:37:05.330647+0000","last_became_peered":"2024-03-11T19:37:05.330647+0000","last_unstale":"2024-03-11T19:38:38.553916+0000","last_undegraded":"2024-03-11T19:38:38.553916+0000","last_fullsized":"2024-03-11T19:38:38.553916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:07:00.576137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453400+0000","last_change":"2024-03-11T19:37:04.582501+0000","last_active":"2024-03-11T19:38:10.453400+0000","last_peered":"2024-03-11T19:38:10.453400+0000","last_clean":"2024-03-11T19:38:10.453400+0000","last_became_active":"2024-03-11T19:37:04.582076+0000","last_became_peered":"2024-03-11T19:37:04.582076+0000","last_unstale":"2024-03-11T19:38:10.453400+0000","last_undegraded":"2024-03-11T19:38:10.453400+0000","last_fullsized":"2024-03-11T19:38:10.453400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:46:05.359211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549826+0000","last_change":"2024-03-11T19:37:05.329053+0000","last_active":"2024-03-11T19:38:38.549826+0000","last_peered":"2024-03-11T19:38:38.549826+0000","last_clean":"2024-03-11T19:38:38.549826+0000","last_became_active":"2024-03-11T19:37:05.328748+0000","last_became_peered":"2024-03-11T19:37:05.328748+0000","last_unstale":"2024-03-11T19:38:38.549826+0000","last_undegraded":"2024-03-11T19:38:38.549826+0000","last_fullsized":"2024-03-11T19:38:38.549826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:48:55.369515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453324+0000","last_change":"2024-03-11T19:37:04.583370+0000","last_active":"2024-03-11T19:38:10.453324+0000","last_peered":"2024-03-11T19:38:10.453324+0000","last_clean":"2024-03-11T19:38:10.453324+0000","last_became_active":"2024-03-11T19:37:04.583055+0000","last_became_peered":"2024-03-11T19:37:04.583055+0000","last_unstale":"2024-03-11T19:38:10.453324+0000","last_undegraded":"2024-03-11T19:38:10.453324+0000","last_fullsized":"2024-03-11T19:38:10.453324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:04:02.325054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.550503+0000","last_change":"2024-03-11T19:37:05.328157+0000","last_active":"2024-03-11T19:38:38.550503+0000","last_peered":"2024-03-11T19:38:38.550503+0000","last_clean":"2024-03-11T19:38:38.550503+0000","last_became_active":"2024-03-11T19:37:05.327839+0000","last_became_peered":"2024-03-11T19:37:05.327839+0000","last_unstale":"2024-03-11T19:38:38.550503+0000","last_undegraded":"2024-03-11T19:38:38.550503+0000","last_fullsized":"2024-03-11T19:38:38.550503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:23:02.297269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543426+0000","last_change":"2024-03-11T19:37:05.336160+0000","last_active":"2024-03-11T19:38:38.543426+0000","last_peered":"2024-03-11T19:38:38.543426+0000","last_clean":"2024-03-11T19:38:38.543426+0000","last_became_active":"2024-03-11T19:37:05.335850+0000","last_became_peered":"2024-03-11T19:37:05.335850+0000","last_unstale":"2024-03-11T19:38:38.543426+0000","last_undegraded":"2024-03-11T19:38:38.543426+0000","last_fullsized":"2024-03-11T19:38:38.543426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:09.743661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536039+0000","last_change":"2024-03-11T19:38:10.455362+0000","last_active":"2024-03-11T19:38:38.536039+0000","last_peered":"2024-03-11T19:38:38.536039+0000","last_clean":"2024-03-11T19:38:38.536039+0000","last_became_active":"2024-03-11T19:38:10.454887+0000","last_became_peered":"2024-03-11T19:38:10.454887+0000","last_unstale":"2024-03-11T19:38:38.536039+0000","last_undegraded":"2024-03-11T19:38:38.536039+0000","last_fullsized":"2024-03-11T19:38:38.536039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:19.289829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.542420+0000","last_change":"2024-03-11T19:37:05.332357+0000","last_active":"2024-03-11T19:38:38.542420+0000","last_peered":"2024-03-11T19:38:38.542420+0000","last_clean":"2024-03-11T19:38:38.542420+0000","last_became_active":"2024-03-11T19:37:05.332003+0000","last_became_peered":"2024-03-11T19:37:05.332003+0000","last_unstale":"2024-03-11T19:38:38.542420+0000","last_undegraded":"2024-03-11T19:38:38.542420+0000","last_fullsized":"2024-03-11T19:38:38.542420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:39.165572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549322+0000","last_change":"2024-03-11T19:37:05.332388+0000","last_active":"2024-03-11T19:38:38.549322+0000","last_peered":"2024-03-11T19:38:38.549322+0000","last_clean":"2024-03-11T19:38:38.549322+0000","last_became_active":"2024-03-11T19:37:05.332032+0000","last_became_peered":"2024-03-11T19:37:05.332032+0000","last_unstale":"2024-03-11T19:38:38.549322+0000","last_undegraded":"2024-03-11T19:38:38.549322+0000","last_fullsized":"2024-03-11T19:38:38.549322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:56:09.416683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.545668+0000","last_change":"2024-03-11T19:37:05.323096+0000","last_active":"2024-03-11T19:38:38.545668+0000","last_peered":"2024-03-11T19:38:38.545668+0000","last_clean":"2024-03-11T19:38:38.545668+0000","last_became_active":"2024-03-11T19:37:05.322525+0000","last_became_peered":"2024-03-11T19:37:05.322525+0000","last_unstale":"2024-03-11T19:38:38.545668+0000","last_undegraded":"2024-03-11T19:38:38.545668+0000","last_fullsized":"2024-03-11T19:38:38.545668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:53:01.870894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540652+0000","last_change":"2024-03-11T19:37:05.331171+0000","last_active":"2024-03-11T19:38:38.540652+0000","last_peered":"2024-03-11T19:38:38.540652+0000","last_clean":"2024-03-11T19:38:38.540652+0000","last_became_active":"2024-03-11T19:37:05.330769+0000","last_became_peered":"2024-03-11T19:37:05.330769+0000","last_unstale":"2024-03-11T19:38:38.540652+0000","last_undegraded":"2024-03-11T19:38:38.540652+0000","last_fullsized":"2024-03-11T19:38:38.540652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:03:06.514039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553501+0000","last_change":"2024-03-11T19:37:05.326877+0000","last_active":"2024-03-11T19:38:38.553501+0000","last_peered":"2024-03-11T19:38:38.553501+0000","last_clean":"2024-03-11T19:38:38.553501+0000","last_became_active":"2024-03-11T19:37:05.326538+0000","last_became_peered":"2024-03-11T19:37:05.326538+0000","last_unstale":"2024-03-11T19:38:38.553501+0000","last_undegraded":"2024-03-11T19:38:38.553501+0000","last_fullsized":"2024-03-11T19:38:38.553501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:11.565697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.530558+0000","last_change":"2024-03-11T19:37:05.335010+0000","last_active":"2024-03-11T19:38:38.530558+0000","last_peered":"2024-03-11T19:38:38.530558+0000","last_clean":"2024-03-11T19:38:38.530558+0000","last_became_active":"2024-03-11T19:37:05.334551+0000","last_became_peered":"2024-03-11T19:37:05.334551+0000","last_unstale":"2024-03-11T19:38:38.530558+0000","last_undegraded":"2024-03-11T19:38:38.530558+0000","last_fullsized":"2024-03-11T19:38:38.530558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:59.668178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540099+0000","last_change":"2024-03-11T19:37:05.336273+0000","last_active":"2024-03-11T19:38:38.540099+0000","last_peered":"2024-03-11T19:38:38.540099+0000","last_clean":"2024-03-11T19:38:38.540099+0000","last_became_active":"2024-03-11T19:37:05.335926+0000","last_became_peered":"2024-03-11T19:37:05.335926+0000","last_unstale":"2024-03-11T19:38:38.540099+0000","last_undegraded":"2024-03-11T19:38:38.540099+0000","last_fullsized":"2024-03-11T19:38:38.540099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:31:55.139242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.533346+0000","last_change":"2024-03-11T19:37:05.335548+0000","last_active":"2024-03-11T19:38:38.533346+0000","last_peered":"2024-03-11T19:38:38.533346+0000","last_clean":"2024-03-11T19:38:38.533346+0000","last_became_active":"2024-03-11T19:37:05.335222+0000","last_became_peered":"2024-03-11T19:37:05.335222+0000","last_unstale":"2024-03-11T19:38:38.533346+0000","last_undegraded":"2024-03-11T19:38:38.533346+0000","last_fullsized":"2024-03-11T19:38:38.533346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:25.697951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.554788+0000","last_change":"2024-03-11T19:37:05.329782+0000","last_active":"2024-03-11T19:38:38.554788+0000","last_peered":"2024-03-11T19:38:38.554788+0000","last_clean":"2024-03-11T19:38:38.554788+0000","last_became_active":"2024-03-11T19:37:05.329434+0000","last_became_peered":"2024-03-11T19:37:05.329434+0000","last_unstale":"2024-03-11T19:38:38.554788+0000","last_undegraded":"2024-03-11T19:38:38.554788+0000","last_fullsized":"2024-03-11T19:38:38.554788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:12:00.516489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553896+0000","last_change":"2024-03-11T19:37:05.323026+0000","last_active":"2024-03-11T19:38:38.553896+0000","last_peered":"2024-03-11T19:38:38.553896+0000","last_clean":"2024-03-11T19:38:38.553896+0000","last_became_active":"2024-03-11T19:37:05.322562+0000","last_became_peered":"2024-03-11T19:37:05.322562+0000","last_unstale":"2024-03-11T19:38:38.553896+0000","last_undegraded":"2024-03-11T19:38:38.553896+0000","last_fullsized":"2024-03-11T19:38:38.553896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:14:57.606404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.533719+0000","last_change":"2024-03-11T19:38:10.455616+0000","last_active":"2024-03-11T19:38:38.533719+0000","last_peered":"2024-03-11T19:38:38.533719+0000","last_clean":"2024-03-11T19:38:38.533719+0000","last_became_active":"2024-03-11T19:38:10.455092+0000","last_became_peered":"2024-03-11T19:38:10.455092+0000","last_unstale":"2024-03-11T19:38:38.533719+0000","last_undegraded":"2024-03-11T19:38:38.533719+0000","last_fullsized":"2024-03-11T19:38:38.533719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:49:50.407540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65254,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":678,"num_read_kb":409,"num_write":171,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383716,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":455694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32776,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712376,"statfs":{"total":95995035648,"available":95961473024,"internally_reserved":0,"allocated":1638400,"data_stored":1108175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":455818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":455626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1048341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":455207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":515580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":517350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":455518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1047493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:39:01.380 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-11T19:39:01.381 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T19:39:01.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:00 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/999102514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:39:01.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:00 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3063013440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:39:01.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:00 smithi137 ceph-mon[36220]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-11T19:39:01.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:00 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1136741782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:39:01.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:00 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/999102514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:39:01.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:00 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3063013440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:39:01.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:00 smithi204 ceph-mon[36739]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-11T19:39:01.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:00 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1136741782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:39:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.515+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3641442879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cc80ffba0 msgr2=0x7f0cc8109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.515+0000 7f0ccf9ac640 1 --2- 172.21.15.83:0/3641442879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cc80ffba0 0x7f0cc8109750 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009ca0 tx=0x7f0cc002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.515+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3641442879 shutdown_connections 2024-03-11T19:39:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.515+0000 7f0ccf9ac640 1 --2- 172.21.15.83:0/3641442879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cc80ffba0 0x7f0cc8109750 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.515+0000 7f0ccf9ac640 1 --2- 172.21.15.83:0/3641442879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc80ff1e0 0x7f0cc80ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.515+0000 7f0ccf9ac640 1 --2- 172.21.15.83:0/3641442879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cc8102770 0x7f0cc8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.515+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3641442879 >> 172.21.15.83:0/3641442879 conn(0x7f0cc80fad80 msgr2=0x7f0cc80fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:01.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.515+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3641442879 shutdown_connections 2024-03-11T19:39:01.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.515+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3641442879 wait complete. 2024-03-11T19:39:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccf9ac640 1 Processor -- start 2024-03-11T19:39:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccf9ac640 1 -- start start 2024-03-11T19:39:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccf9ac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cc80ff1e0 0x7f0cc8101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccf9ac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc80ffba0 0x7f0cc81004b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccf9ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cc8102770 0x7f0cc81009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccf9ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc8101150 con 0x7f0cc8102770 2024-03-11T19:39:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccf9ac640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc81012c0 con 0x7f0cc80ff1e0 2024-03-11T19:39:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccf9ac640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc81a11c0 con 0x7f0cc80ffba0 2024-03-11T19:39:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccdf49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cc8102770 0x7f0cc81009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccdf49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cc8102770 0x7f0cc81009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50536/0 (socket says 172.21.15.83:50536) 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccdf49640 1 -- 172.21.15.83:0/3381832402 learned_addr learned my addr 172.21.15.83:0/3381832402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccd748640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cc80ff1e0 0x7f0cc8101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0cccf47640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc80ffba0 0x7f0cc81004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccdf49640 1 -- 172.21.15.83:0/3381832402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc80ffba0 msgr2=0x7f0cc81004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccdf49640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc80ffba0 0x7f0cc81004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccdf49640 1 -- 172.21.15.83:0/3381832402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cc80ff1e0 msgr2=0x7f0cc8101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccdf49640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cc80ff1e0 0x7f0cc8101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccdf49640 1 -- 172.21.15.83:0/3381832402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc0009900 con 0x7f0cc8102770 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.516+0000 7f0ccdf49640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cc8102770 0x7f0cc81009f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00346f0 tx=0x7f0cc0034720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.517+0000 7f0cbe7fc640 1 -- 172.21.15.83:0/3381832402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0003970 con 0x7f0cc8102770 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.517+0000 7f0cbe7fc640 1 -- 172.21.15.83:0/3381832402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cc0004040 con 0x7f0cc8102770 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.517+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3381832402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc81a1410 con 0x7f0cc8102770 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.517+0000 7f0cbe7fc640 1 -- 172.21.15.83:0/3381832402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00406b0 con 0x7f0cc8102770 2024-03-11T19:39:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.517+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3381832402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc81a18c0 con 0x7f0cc8102770 2024-03-11T19:39:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.518+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3381832402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc8108560 con 0x7f0cc8102770 2024-03-11T19:39:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.519+0000 7f0cbe7fc640 1 -- 172.21.15.83:0/3381832402 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0cc000a040 con 0x7f0cc8102770 2024-03-11T19:39:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.519+0000 7f0cbe7fc640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cb40a1a30 0x7f0cb40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.520+0000 7f0cbe7fc640 1 -- 172.21.15.83:0/3381832402 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0cc00f0540 con 0x7f0cc8102770 2024-03-11T19:39:01.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.520+0000 7f0ccd748640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cb40a1a30 0x7f0cb40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.520+0000 7f0ccd748640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cb40a1a30 0x7f0cb40a3ef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8068000 tx=0x7f0cc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.524+0000 7f0cbe7fc640 1 -- 172.21.15.83:0/3381832402 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0cc00b9800 con 0x7f0cc8102770 2024-03-11T19:39:01.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.698+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3381832402 --> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0cc802ceb0 con 0x7f0cb40a1a30 2024-03-11T19:39:01.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.705+0000 7f0cbe7fc640 1 -- 172.21.15.83:0/3381832402 <== mgr.14152 v2:172.21.15.83:6800/3477661212 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258839 (secure 0 0 0) 0x7f0cc802ceb0 con 0x7f0cb40a1a30 2024-03-11T19:39:01.712 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:01.713 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3381832402 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cb40a1a30 msgr2=0x7f0cb40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cb40a1a30 0x7f0cb40a3ef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8068000 tx=0x7f0cc4005f90 comp rx=0 tx=0).stop 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3381832402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cc8102770 msgr2=0x7f0cc81009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cc8102770 0x7f0cc81009f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00346f0 tx=0x7f0cc0034720 comp rx=0 tx=0).stop 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3381832402 shutdown_connections 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cb40a1a30 0x7f0cb40a3ef0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cc8102770 0x7f0cc81009f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cc80ffba0 0x7f0cc81004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 --2- 172.21.15.83:0/3381832402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cc80ff1e0 0x7f0cc8101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3381832402 >> 172.21.15.83:0/3381832402 conn(0x7f0cc80fad80 msgr2=0x7f0cc8105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3381832402 shutdown_connections 2024-03-11T19:39:01.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.708+0000 7f0ccf9ac640 1 -- 172.21.15.83:0/3381832402 wait complete. 2024-03-11T19:39:01.724 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-03-11T19:39:00.535104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655856,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":804,"num_read_kb":518,"num_write":354,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366476,"kb_used_data":13164,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575348,"statfs":{"total":1151940427776,"available":1151565156352,"internally_reserved":0,"allocated":13479936,"data_stored":7427568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002753"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543207+0000","last_change":"2024-03-11T19:37:05.340476+0000","last_active":"2024-03-11T19:38:38.543207+0000","last_peered":"2024-03-11T19:38:38.543207+0000","last_clean":"2024-03-11T19:38:38.543207+0000","last_became_active":"2024-03-11T19:37:05.340140+0000","last_became_peered":"2024-03-11T19:37:05.340140+0000","last_unstale":"2024-03-11T19:38:38.543207+0000","last_undegraded":"2024-03-11T19:38:38.543207+0000","last_fullsized":"2024-03-11T19:38:38.543207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:10:35.677374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.537288+0000","last_change":"2024-03-11T19:37:05.335500+0000","last_active":"2024-03-11T19:38:38.537288+0000","last_peered":"2024-03-11T19:38:38.537288+0000","last_clean":"2024-03-11T19:38:38.537288+0000","last_became_active":"2024-03-11T19:37:05.335194+0000","last_became_peered":"2024-03-11T19:37:05.335194+0000","last_unstale":"2024-03-11T19:38:38.537288+0000","last_undegraded":"2024-03-11T19:38:38.537288+0000","last_fullsized":"2024-03-11T19:38:38.537288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:03.332994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.555088+0000","last_change":"2024-03-11T19:37:05.334618+0000","last_active":"2024-03-11T19:38:38.555088+0000","last_peered":"2024-03-11T19:38:38.555088+0000","last_clean":"2024-03-11T19:38:38.555088+0000","last_became_active":"2024-03-11T19:37:05.334281+0000","last_became_peered":"2024-03-11T19:37:05.334281+0000","last_unstale":"2024-03-11T19:38:38.555088+0000","last_undegraded":"2024-03-11T19:38:38.555088+0000","last_fullsized":"2024-03-11T19:38:38.555088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:43:22.998462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.546651+0000","last_change":"2024-03-11T19:37:05.332978+0000","last_active":"2024-03-11T19:38:38.546651+0000","last_peered":"2024-03-11T19:38:38.546651+0000","last_clean":"2024-03-11T19:38:38.546651+0000","last_became_active":"2024-03-11T19:37:05.330888+0000","last_became_peered":"2024-03-11T19:37:05.330888+0000","last_unstale":"2024-03-11T19:38:38.546651+0000","last_undegraded":"2024-03-11T19:38:38.546651+0000","last_fullsized":"2024-03-11T19:38:38.546651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:40:36.988348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536573+0000","last_change":"2024-03-11T19:37:05.329050+0000","last_active":"2024-03-11T19:38:38.536573+0000","last_peered":"2024-03-11T19:38:38.536573+0000","last_clean":"2024-03-11T19:38:38.536573+0000","last_became_active":"2024-03-11T19:37:05.328702+0000","last_became_peered":"2024-03-11T19:37:05.328702+0000","last_unstale":"2024-03-11T19:38:38.536573+0000","last_undegraded":"2024-03-11T19:38:38.536573+0000","last_fullsized":"2024-03-11T19:38:38.536573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:19:52.180884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549103+0000","last_change":"2024-03-11T19:37:05.322748+0000","last_active":"2024-03-11T19:38:38.549103+0000","last_peered":"2024-03-11T19:38:38.549103+0000","last_clean":"2024-03-11T19:38:38.549103+0000","last_became_active":"2024-03-11T19:37:05.322376+0000","last_became_peered":"2024-03-11T19:37:05.322376+0000","last_unstale":"2024-03-11T19:38:38.549103+0000","last_undegraded":"2024-03-11T19:38:38.549103+0000","last_fullsized":"2024-03-11T19:38:38.549103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:29:40.926854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543305+0000","last_change":"2024-03-11T19:37:05.323954+0000","last_active":"2024-03-11T19:38:38.543305+0000","last_peered":"2024-03-11T19:38:38.543305+0000","last_clean":"2024-03-11T19:38:38.543305+0000","last_became_active":"2024-03-11T19:37:05.323586+0000","last_became_peered":"2024-03-11T19:37:05.323586+0000","last_unstale":"2024-03-11T19:38:38.543305+0000","last_undegraded":"2024-03-11T19:38:38.543305+0000","last_fullsized":"2024-03-11T19:38:38.543305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:59:38.286891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536729+0000","last_change":"2024-03-11T19:37:05.334138+0000","last_active":"2024-03-11T19:38:38.536729+0000","last_peered":"2024-03-11T19:38:38.536729+0000","last_clean":"2024-03-11T19:38:38.536729+0000","last_became_active":"2024-03-11T19:37:05.333837+0000","last_became_peered":"2024-03-11T19:37:05.333837+0000","last_unstale":"2024-03-11T19:38:38.536729+0000","last_undegraded":"2024-03-11T19:38:38.536729+0000","last_fullsized":"2024-03-11T19:38:38.536729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:49:30.724176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.552174+0000","last_change":"2024-03-11T19:37:05.336888+0000","last_active":"2024-03-11T19:38:38.552174+0000","last_peered":"2024-03-11T19:38:38.552174+0000","last_clean":"2024-03-11T19:38:38.552174+0000","last_became_active":"2024-03-11T19:37:05.336256+0000","last_became_peered":"2024-03-11T19:37:05.336256+0000","last_unstale":"2024-03-11T19:38:38.552174+0000","last_undegraded":"2024-03-11T19:38:38.552174+0000","last_fullsized":"2024-03-11T19:38:38.552174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:54:31.526986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548957+0000","last_change":"2024-03-11T19:37:05.321959+0000","last_active":"2024-03-11T19:38:38.548957+0000","last_peered":"2024-03-11T19:38:38.548957+0000","last_clean":"2024-03-11T19:38:38.548957+0000","last_became_active":"2024-03-11T19:37:05.321043+0000","last_became_peered":"2024-03-11T19:37:05.321043+0000","last_unstale":"2024-03-11T19:38:38.548957+0000","last_undegraded":"2024-03-11T19:38:38.548957+0000","last_fullsized":"2024-03-11T19:38:38.548957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:13.121003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548202+0000","last_change":"2024-03-11T19:37:05.324940+0000","last_active":"2024-03-11T19:38:38.548202+0000","last_peered":"2024-03-11T19:38:38.548202+0000","last_clean":"2024-03-11T19:38:38.548202+0000","last_became_active":"2024-03-11T19:37:05.323427+0000","last_became_peered":"2024-03-11T19:37:05.323427+0000","last_unstale":"2024-03-11T19:38:38.548202+0000","last_undegraded":"2024-03-11T19:38:38.548202+0000","last_fullsized":"2024-03-11T19:38:38.548202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:46:23.633709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548766+0000","last_change":"2024-03-11T19:37:05.332694+0000","last_active":"2024-03-11T19:38:38.548766+0000","last_peered":"2024-03-11T19:38:38.548766+0000","last_clean":"2024-03-11T19:38:38.548766+0000","last_became_active":"2024-03-11T19:37:05.330446+0000","last_became_peered":"2024-03-11T19:37:05.330446+0000","last_unstale":"2024-03-11T19:38:38.548766+0000","last_undegraded":"2024-03-11T19:38:38.548766+0000","last_fullsized":"2024-03-11T19:38:38.548766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:37:40.196933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535856+0000","last_change":"2024-03-11T19:37:05.325975+0000","last_active":"2024-03-11T19:38:38.535856+0000","last_peered":"2024-03-11T19:38:38.535856+0000","last_clean":"2024-03-11T19:38:38.535856+0000","last_became_active":"2024-03-11T19:37:05.325649+0000","last_became_peered":"2024-03-11T19:37:05.325649+0000","last_unstale":"2024-03-11T19:38:38.535856+0000","last_undegraded":"2024-03-11T19:38:38.535856+0000","last_fullsized":"2024-03-11T19:38:38.535856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:36.373272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.547237+0000","last_change":"2024-03-11T19:37:05.335546+0000","last_active":"2024-03-11T19:38:38.547237+0000","last_peered":"2024-03-11T19:38:38.547237+0000","last_clean":"2024-03-11T19:38:38.547237+0000","last_became_active":"2024-03-11T19:37:05.335220+0000","last_became_peered":"2024-03-11T19:37:05.335220+0000","last_unstale":"2024-03-11T19:38:38.547237+0000","last_undegraded":"2024-03-11T19:38:38.547237+0000","last_fullsized":"2024-03-11T19:38:38.547237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:05:13.116355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.546152+0000","last_change":"2024-03-11T19:37:05.338955+0000","last_active":"2024-03-11T19:38:38.546152+0000","last_peered":"2024-03-11T19:38:38.546152+0000","last_clean":"2024-03-11T19:38:38.546152+0000","last_became_active":"2024-03-11T19:37:05.338574+0000","last_became_peered":"2024-03-11T19:37:05.338574+0000","last_unstale":"2024-03-11T19:38:38.546152+0000","last_undegraded":"2024-03-11T19:38:38.546152+0000","last_fullsized":"2024-03-11T19:38:38.546152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:27:27.754077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553899+0000","last_change":"2024-03-11T19:37:05.334578+0000","last_active":"2024-03-11T19:38:38.553899+0000","last_peered":"2024-03-11T19:38:38.553899+0000","last_clean":"2024-03-11T19:38:38.553899+0000","last_became_active":"2024-03-11T19:37:05.334267+0000","last_became_peered":"2024-03-11T19:37:05.334267+0000","last_unstale":"2024-03-11T19:38:38.553899+0000","last_undegraded":"2024-03-11T19:38:38.553899+0000","last_fullsized":"2024-03-11T19:38:38.553899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:31.704187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543563+0000","last_change":"2024-03-11T19:37:05.341178+0000","last_active":"2024-03-11T19:38:38.543563+0000","last_peered":"2024-03-11T19:38:38.543563+0000","last_clean":"2024-03-11T19:38:38.543563+0000","last_became_active":"2024-03-11T19:37:05.340837+0000","last_became_peered":"2024-03-11T19:37:05.340837+0000","last_unstale":"2024-03-11T19:38:38.543563+0000","last_undegraded":"2024-03-11T19:38:38.543563+0000","last_fullsized":"2024-03-11T19:38:38.543563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:39:54.460937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.538263+0000","last_change":"2024-03-11T19:37:05.330981+0000","last_active":"2024-03-11T19:38:38.538263+0000","last_peered":"2024-03-11T19:38:38.538263+0000","last_clean":"2024-03-11T19:38:38.538263+0000","last_became_active":"2024-03-11T19:37:05.330638+0000","last_became_peered":"2024-03-11T19:37:05.330638+0000","last_unstale":"2024-03-11T19:38:38.538263+0000","last_undegraded":"2024-03-11T19:38:38.538263+0000","last_fullsized":"2024-03-11T19:38:38.538263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:36:12.565936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535474+0000","last_change":"2024-03-11T19:37:05.329866+0000","last_active":"2024-03-11T19:38:38.535474+0000","last_peered":"2024-03-11T19:38:38.535474+0000","last_clean":"2024-03-11T19:38:38.535474+0000","last_became_active":"2024-03-11T19:37:05.329572+0000","last_became_peered":"2024-03-11T19:37:05.329572+0000","last_unstale":"2024-03-11T19:38:38.535474+0000","last_undegraded":"2024-03-11T19:38:38.535474+0000","last_fullsized":"2024-03-11T19:38:38.535474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:40:04.482929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454096+0000","last_change":"2024-03-11T19:37:04.587149+0000","last_active":"2024-03-11T19:38:10.454096+0000","last_peered":"2024-03-11T19:38:10.454096+0000","last_clean":"2024-03-11T19:38:10.454096+0000","last_became_active":"2024-03-11T19:37:04.586013+0000","last_became_peered":"2024-03-11T19:37:04.586013+0000","last_unstale":"2024-03-11T19:38:10.454096+0000","last_undegraded":"2024-03-11T19:38:10.454096+0000","last_fullsized":"2024-03-11T19:38:10.454096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:15.758154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536238+0000","last_change":"2024-03-11T19:37:05.327979+0000","last_active":"2024-03-11T19:38:38.536238+0000","last_peered":"2024-03-11T19:38:38.536238+0000","last_clean":"2024-03-11T19:38:38.536238+0000","last_became_active":"2024-03-11T19:37:05.327647+0000","last_became_peered":"2024-03-11T19:37:05.327647+0000","last_unstale":"2024-03-11T19:38:38.536238+0000","last_undegraded":"2024-03-11T19:38:38.536238+0000","last_fullsized":"2024-03-11T19:38:38.536238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:09.514766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452950+0000","last_change":"2024-03-11T19:37:04.582937+0000","last_active":"2024-03-11T19:38:10.452950+0000","last_peered":"2024-03-11T19:38:10.452950+0000","last_clean":"2024-03-11T19:38:10.452950+0000","last_became_active":"2024-03-11T19:37:04.582622+0000","last_became_peered":"2024-03-11T19:37:04.582622+0000","last_unstale":"2024-03-11T19:38:10.452950+0000","last_undegraded":"2024-03-11T19:38:10.452950+0000","last_fullsized":"2024-03-11T19:38:10.452950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:02.275698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.555090+0000","last_change":"2024-03-11T19:37:05.327191+0000","last_active":"2024-03-11T19:38:38.555090+0000","last_peered":"2024-03-11T19:38:38.555090+0000","last_clean":"2024-03-11T19:38:38.555090+0000","last_became_active":"2024-03-11T19:37:05.326001+0000","last_became_peered":"2024-03-11T19:37:05.326001+0000","last_unstale":"2024-03-11T19:38:38.555090+0000","last_undegraded":"2024-03-11T19:38:38.555090+0000","last_fullsized":"2024-03-11T19:38:38.555090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:45:27.301321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151969+0000","last_change":"2024-03-11T19:37:04.587054+0000","last_active":"2024-03-11T19:38:10.151969+0000","last_peered":"2024-03-11T19:38:10.151969+0000","last_clean":"2024-03-11T19:38:10.151969+0000","last_became_active":"2024-03-11T19:37:04.586104+0000","last_became_peered":"2024-03-11T19:37:04.586104+0000","last_unstale":"2024-03-11T19:38:10.151969+0000","last_undegraded":"2024-03-11T19:38:10.151969+0000","last_fullsized":"2024-03-11T19:38:10.151969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:34.913677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549578+0000","last_change":"2024-03-11T19:37:05.333288+0000","last_active":"2024-03-11T19:38:38.549578+0000","last_peered":"2024-03-11T19:38:38.549578+0000","last_clean":"2024-03-11T19:38:38.549578+0000","last_became_active":"2024-03-11T19:37:05.332882+0000","last_became_peered":"2024-03-11T19:37:05.332882+0000","last_unstale":"2024-03-11T19:38:38.549578+0000","last_undegraded":"2024-03-11T19:38:38.549578+0000","last_fullsized":"2024-03-11T19:38:38.549578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:45.680608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151628+0000","last_change":"2024-03-11T19:37:04.583660+0000","last_active":"2024-03-11T19:38:10.151628+0000","last_peered":"2024-03-11T19:38:10.151628+0000","last_clean":"2024-03-11T19:38:10.151628+0000","last_became_active":"2024-03-11T19:37:04.583360+0000","last_became_peered":"2024-03-11T19:37:04.583360+0000","last_unstale":"2024-03-11T19:38:10.151628+0000","last_undegraded":"2024-03-11T19:38:10.151628+0000","last_fullsized":"2024-03-11T19:38:10.151628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:00.716741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.554758+0000","last_change":"2024-03-11T19:37:05.335649+0000","last_active":"2024-03-11T19:38:38.554758+0000","last_peered":"2024-03-11T19:38:38.554758+0000","last_clean":"2024-03-11T19:38:38.554758+0000","last_became_active":"2024-03-11T19:37:05.335255+0000","last_became_peered":"2024-03-11T19:37:05.335255+0000","last_unstale":"2024-03-11T19:38:38.554758+0000","last_undegraded":"2024-03-11T19:38:38.554758+0000","last_fullsized":"2024-03-11T19:38:38.554758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:49:09.772765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":136,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:53.782803+0000","last_change":"2024-03-11T19:37:04.586002+0000","last_active":"2024-03-11T19:38:53.782803+0000","last_peered":"2024-03-11T19:38:53.782803+0000","last_clean":"2024-03-11T19:38:53.782803+0000","last_became_active":"2024-03-11T19:37:04.585551+0000","last_became_peered":"2024-03-11T19:37:04.585551+0000","last_unstale":"2024-03-11T19:38:53.782803+0000","last_undegraded":"2024-03-11T19:38:53.782803+0000","last_fullsized":"2024-03-11T19:38:53.782803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:34:12.663801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.542150+0000","last_change":"2024-03-11T19:37:05.325168+0000","last_active":"2024-03-11T19:38:38.542150+0000","last_peered":"2024-03-11T19:38:38.542150+0000","last_clean":"2024-03-11T19:38:38.542150+0000","last_became_active":"2024-03-11T19:37:05.324848+0000","last_became_peered":"2024-03-11T19:37:05.324848+0000","last_unstale":"2024-03-11T19:38:38.542150+0000","last_undegraded":"2024-03-11T19:38:38.542150+0000","last_fullsized":"2024-03-11T19:38:38.542150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:58:27.423877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452883+0000","last_change":"2024-03-11T19:37:04.582727+0000","last_active":"2024-03-11T19:38:10.452883+0000","last_peered":"2024-03-11T19:38:10.452883+0000","last_clean":"2024-03-11T19:38:10.452883+0000","last_became_active":"2024-03-11T19:37:04.582359+0000","last_became_peered":"2024-03-11T19:37:04.582359+0000","last_unstale":"2024-03-11T19:38:10.452883+0000","last_undegraded":"2024-03-11T19:38:10.452883+0000","last_fullsized":"2024-03-11T19:38:10.452883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:05.591537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.544557+0000","last_change":"2024-03-11T19:37:05.335442+0000","last_active":"2024-03-11T19:38:38.544557+0000","last_peered":"2024-03-11T19:38:38.544557+0000","last_clean":"2024-03-11T19:38:38.544557+0000","last_became_active":"2024-03-11T19:37:05.335003+0000","last_became_peered":"2024-03-11T19:37:05.335003+0000","last_unstale":"2024-03-11T19:38:38.544557+0000","last_undegraded":"2024-03-11T19:38:38.544557+0000","last_fullsized":"2024-03-11T19:38:38.544557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:07:41.590898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453615+0000","last_change":"2024-03-11T19:37:04.589957+0000","last_active":"2024-03-11T19:38:10.453615+0000","last_peered":"2024-03-11T19:38:10.453615+0000","last_clean":"2024-03-11T19:38:10.453615+0000","last_became_active":"2024-03-11T19:37:04.589539+0000","last_became_peered":"2024-03-11T19:37:04.589539+0000","last_unstale":"2024-03-11T19:38:10.453615+0000","last_undegraded":"2024-03-11T19:38:10.453615+0000","last_fullsized":"2024-03-11T19:38:10.453615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:48:52.764057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535727+0000","last_change":"2024-03-11T19:37:05.331380+0000","last_active":"2024-03-11T19:38:38.535727+0000","last_peered":"2024-03-11T19:38:38.535727+0000","last_clean":"2024-03-11T19:38:38.535727+0000","last_became_active":"2024-03-11T19:37:05.331000+0000","last_became_peered":"2024-03-11T19:37:05.331000+0000","last_unstale":"2024-03-11T19:38:38.535727+0000","last_undegraded":"2024-03-11T19:38:38.535727+0000","last_fullsized":"2024-03-11T19:38:38.535727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:58:47.278195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452916+0000","last_change":"2024-03-11T19:37:04.584128+0000","last_active":"2024-03-11T19:38:10.452916+0000","last_peered":"2024-03-11T19:38:10.452916+0000","last_clean":"2024-03-11T19:38:10.452916+0000","last_became_active":"2024-03-11T19:37:04.583842+0000","last_became_peered":"2024-03-11T19:37:04.583842+0000","last_unstale":"2024-03-11T19:38:10.452916+0000","last_undegraded":"2024-03-11T19:38:10.452916+0000","last_fullsized":"2024-03-11T19:38:10.452916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:57:18.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.534652+0000","last_change":"2024-03-11T19:37:05.328999+0000","last_active":"2024-03-11T19:38:38.534652+0000","last_peered":"2024-03-11T19:38:38.534652+0000","last_clean":"2024-03-11T19:38:38.534652+0000","last_became_active":"2024-03-11T19:37:05.328620+0000","last_became_peered":"2024-03-11T19:37:05.328620+0000","last_unstale":"2024-03-11T19:38:38.534652+0000","last_undegraded":"2024-03-11T19:38:38.534652+0000","last_fullsized":"2024-03-11T19:38:38.534652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:40.191922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":132,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:56.786615+0000","last_change":"2024-03-11T19:37:04.587956+0000","last_active":"2024-03-11T19:38:56.786615+0000","last_peered":"2024-03-11T19:38:56.786615+0000","last_clean":"2024-03-11T19:38:56.786615+0000","last_became_active":"2024-03-11T19:37:04.587568+0000","last_became_peered":"2024-03-11T19:37:04.587568+0000","last_unstale":"2024-03-11T19:38:56.786615+0000","last_undegraded":"2024-03-11T19:38:56.786615+0000","last_fullsized":"2024-03-11T19:38:56.786615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:45:44.019436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549054+0000","last_change":"2024-03-11T19:37:05.336912+0000","last_active":"2024-03-11T19:38:38.549054+0000","last_peered":"2024-03-11T19:38:38.549054+0000","last_clean":"2024-03-11T19:38:38.549054+0000","last_became_active":"2024-03-11T19:37:05.336397+0000","last_became_peered":"2024-03-11T19:37:05.336397+0000","last_unstale":"2024-03-11T19:38:38.549054+0000","last_undegraded":"2024-03-11T19:38:38.549054+0000","last_fullsized":"2024-03-11T19:38:38.549054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:06:29.375396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150574+0000","last_change":"2024-03-11T19:37:04.584382+0000","last_active":"2024-03-11T19:38:10.150574+0000","last_peered":"2024-03-11T19:38:10.150574+0000","last_clean":"2024-03-11T19:38:10.150574+0000","last_became_active":"2024-03-11T19:37:04.584026+0000","last_became_peered":"2024-03-11T19:37:04.584026+0000","last_unstale":"2024-03-11T19:38:10.150574+0000","last_undegraded":"2024-03-11T19:38:10.150574+0000","last_fullsized":"2024-03-11T19:38:10.150574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:04:28.152903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.546098+0000","last_change":"2024-03-11T19:37:05.331385+0000","last_active":"2024-03-11T19:38:38.546098+0000","last_peered":"2024-03-11T19:38:38.546098+0000","last_clean":"2024-03-11T19:38:38.546098+0000","last_became_active":"2024-03-11T19:37:05.331069+0000","last_became_peered":"2024-03-11T19:37:05.331069+0000","last_unstale":"2024-03-11T19:38:38.546098+0000","last_undegraded":"2024-03-11T19:38:38.546098+0000","last_fullsized":"2024-03-11T19:38:38.546098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:22:18.375605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453568+0000","last_change":"2024-03-11T19:37:04.586493+0000","last_active":"2024-03-11T19:38:10.453568+0000","last_peered":"2024-03-11T19:38:10.453568+0000","last_clean":"2024-03-11T19:38:10.453568+0000","last_became_active":"2024-03-11T19:37:04.586185+0000","last_became_peered":"2024-03-11T19:37:04.586185+0000","last_unstale":"2024-03-11T19:38:10.453568+0000","last_undegraded":"2024-03-11T19:38:10.453568+0000","last_fullsized":"2024-03-11T19:38:10.453568+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:37:00.899752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.555300+0000","last_change":"2024-03-11T19:37:05.336414+0000","last_active":"2024-03-11T19:38:38.555300+0000","last_peered":"2024-03-11T19:38:38.555300+0000","last_clean":"2024-03-11T19:38:38.555300+0000","last_became_active":"2024-03-11T19:37:05.334536+0000","last_became_peered":"2024-03-11T19:37:05.334536+0000","last_unstale":"2024-03-11T19:38:38.555300+0000","last_undegraded":"2024-03-11T19:38:38.555300+0000","last_fullsized":"2024-03-11T19:38:38.555300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:50:19.109975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'34","reported_seq":281,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:57.788731+0000","last_change":"2024-03-11T19:37:04.584544+0000","last_active":"2024-03-11T19:38:57.788731+0000","last_peered":"2024-03-11T19:38:57.788731+0000","last_clean":"2024-03-11T19:38:57.788731+0000","last_became_active":"2024-03-11T19:37:04.584198+0000","last_became_peered":"2024-03-11T19:37:04.584198+0000","last_unstale":"2024-03-11T19:38:57.788731+0000","last_undegraded":"2024-03-11T19:38:57.788731+0000","last_fullsized":"2024-03-11T19:38:57.788731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:01:56.744533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60580,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":228,"num_read_kb":173,"num_write":35,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.533749+0000","last_change":"2024-03-11T19:37:05.331077+0000","last_active":"2024-03-11T19:38:38.533749+0000","last_peered":"2024-03-11T19:38:38.533749+0000","last_clean":"2024-03-11T19:38:38.533749+0000","last_became_active":"2024-03-11T19:37:05.330708+0000","last_became_peered":"2024-03-11T19:37:05.330708+0000","last_unstale":"2024-03-11T19:38:38.533749+0000","last_undegraded":"2024-03-11T19:38:38.533749+0000","last_fullsized":"2024-03-11T19:38:38.533749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:54:55.019268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155548+0000","last_change":"2024-03-11T19:37:04.583353+0000","last_active":"2024-03-11T19:38:10.155548+0000","last_peered":"2024-03-11T19:38:10.155548+0000","last_clean":"2024-03-11T19:38:10.155548+0000","last_became_active":"2024-03-11T19:37:04.581974+0000","last_became_peered":"2024-03-11T19:37:04.581974+0000","last_unstale":"2024-03-11T19:38:10.155548+0000","last_undegraded":"2024-03-11T19:38:10.155548+0000","last_fullsized":"2024-03-11T19:38:10.155548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:37:41.161354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553558+0000","last_change":"2024-03-11T19:37:05.329498+0000","last_active":"2024-03-11T19:38:38.553558+0000","last_peered":"2024-03-11T19:38:38.553558+0000","last_clean":"2024-03-11T19:38:38.553558+0000","last_became_active":"2024-03-11T19:37:05.329186+0000","last_became_peered":"2024-03-11T19:37:05.329186+0000","last_unstale":"2024-03-11T19:38:38.553558+0000","last_undegraded":"2024-03-11T19:38:38.553558+0000","last_fullsized":"2024-03-11T19:38:38.553558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:33:45.012049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453150+0000","last_change":"2024-03-11T19:37:04.582027+0000","last_active":"2024-03-11T19:38:10.453150+0000","last_peered":"2024-03-11T19:38:10.453150+0000","last_clean":"2024-03-11T19:38:10.453150+0000","last_became_active":"2024-03-11T19:37:04.581678+0000","last_became_peered":"2024-03-11T19:37:04.581678+0000","last_unstale":"2024-03-11T19:38:10.453150+0000","last_undegraded":"2024-03-11T19:38:10.453150+0000","last_fullsized":"2024-03-11T19:38:10.453150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:03:40.688053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553484+0000","last_change":"2024-03-11T19:38:10.455076+0000","last_active":"2024-03-11T19:38:38.553484+0000","last_peered":"2024-03-11T19:38:38.553484+0000","last_clean":"2024-03-11T19:38:38.553484+0000","last_became_active":"2024-03-11T19:38:10.454631+0000","last_became_peered":"2024-03-11T19:38:10.454631+0000","last_unstale":"2024-03-11T19:38:38.553484+0000","last_undegraded":"2024-03-11T19:38:38.553484+0000","last_fullsized":"2024-03-11T19:38:38.553484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:26:36.651449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":136,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:55.785449+0000","last_change":"2024-03-11T19:37:04.576415+0000","last_active":"2024-03-11T19:38:55.785449+0000","last_peered":"2024-03-11T19:38:55.785449+0000","last_clean":"2024-03-11T19:38:55.785449+0000","last_became_active":"2024-03-11T19:37:04.575978+0000","last_became_peered":"2024-03-11T19:37:04.575978+0000","last_unstale":"2024-03-11T19:38:55.785449+0000","last_undegraded":"2024-03-11T19:38:55.785449+0000","last_fullsized":"2024-03-11T19:38:55.785449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:25:20.124813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.550544+0000","last_change":"2024-03-11T19:37:05.326258+0000","last_active":"2024-03-11T19:38:38.550544+0000","last_peered":"2024-03-11T19:38:38.550544+0000","last_clean":"2024-03-11T19:38:38.550544+0000","last_became_active":"2024-03-11T19:37:05.325909+0000","last_became_peered":"2024-03-11T19:37:05.325909+0000","last_unstale":"2024-03-11T19:38:38.550544+0000","last_undegraded":"2024-03-11T19:38:38.550544+0000","last_fullsized":"2024-03-11T19:38:38.550544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:25.875603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.482823+0000","last_change":"2024-03-11T19:38:10.482380+0000","last_active":"2024-03-11T19:38:10.482823+0000","last_peered":"2024-03-11T19:38:10.482823+0000","last_clean":"2024-03-11T19:38:10.482823+0000","last_became_active":"2024-03-11T19:38:10.455301+0000","last_became_peered":"2024-03-11T19:38:10.455301+0000","last_unstale":"2024-03-11T19:38:10.482823+0000","last_undegraded":"2024-03-11T19:38:10.482823+0000","last_fullsized":"2024-03-11T19:38:10.482823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:28:41.620522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,4],"acting":[11,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553853+0000","last_change":"2024-03-11T19:37:05.336727+0000","last_active":"2024-03-11T19:38:38.553853+0000","last_peered":"2024-03-11T19:38:38.553853+0000","last_clean":"2024-03-11T19:38:38.553853+0000","last_became_active":"2024-03-11T19:37:05.336384+0000","last_became_peered":"2024-03-11T19:37:05.336384+0000","last_unstale":"2024-03-11T19:38:38.553853+0000","last_undegraded":"2024-03-11T19:38:38.553853+0000","last_fullsized":"2024-03-11T19:38:38.553853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:07:02.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151827+0000","last_change":"2024-03-11T19:37:04.586814+0000","last_active":"2024-03-11T19:38:10.151827+0000","last_peered":"2024-03-11T19:38:10.151827+0000","last_clean":"2024-03-11T19:38:10.151827+0000","last_became_active":"2024-03-11T19:37:04.586512+0000","last_became_peered":"2024-03-11T19:37:04.586512+0000","last_unstale":"2024-03-11T19:38:10.151827+0000","last_undegraded":"2024-03-11T19:38:10.151827+0000","last_fullsized":"2024-03-11T19:38:10.151827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:04:23.979783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.548507+0000","last_change":"2024-03-11T19:37:05.335353+0000","last_active":"2024-03-11T19:38:38.548507+0000","last_peered":"2024-03-11T19:38:38.548507+0000","last_clean":"2024-03-11T19:38:38.548507+0000","last_became_active":"2024-03-11T19:37:05.335066+0000","last_became_peered":"2024-03-11T19:37:05.335066+0000","last_unstale":"2024-03-11T19:38:38.548507+0000","last_undegraded":"2024-03-11T19:38:38.548507+0000","last_fullsized":"2024-03-11T19:38:38.548507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:44:11.915614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454552+0000","last_change":"2024-03-11T19:37:04.586679+0000","last_active":"2024-03-11T19:38:10.454552+0000","last_peered":"2024-03-11T19:38:10.454552+0000","last_clean":"2024-03-11T19:38:10.454552+0000","last_became_active":"2024-03-11T19:37:04.586316+0000","last_became_peered":"2024-03-11T19:37:04.586316+0000","last_unstale":"2024-03-11T19:38:10.454552+0000","last_undegraded":"2024-03-11T19:38:10.454552+0000","last_fullsized":"2024-03-11T19:38:10.454552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:58:21.115526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.535653+0000","last_change":"2024-03-11T19:37:05.330206+0000","last_active":"2024-03-11T19:38:38.535653+0000","last_peered":"2024-03-11T19:38:38.535653+0000","last_clean":"2024-03-11T19:38:38.535653+0000","last_became_active":"2024-03-11T19:37:05.329847+0000","last_became_peered":"2024-03-11T19:37:05.329847+0000","last_unstale":"2024-03-11T19:38:38.535653+0000","last_undegraded":"2024-03-11T19:38:38.535653+0000","last_fullsized":"2024-03-11T19:38:38.535653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:14:52.532236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150350+0000","last_change":"2024-03-11T19:37:04.581031+0000","last_active":"2024-03-11T19:38:10.150350+0000","last_peered":"2024-03-11T19:38:10.150350+0000","last_clean":"2024-03-11T19:38:10.150350+0000","last_became_active":"2024-03-11T19:37:04.580728+0000","last_became_peered":"2024-03-11T19:37:04.580728+0000","last_unstale":"2024-03-11T19:38:10.150350+0000","last_undegraded":"2024-03-11T19:38:10.150350+0000","last_fullsized":"2024-03-11T19:38:10.150350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:08:43.401898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540064+0000","last_change":"2024-03-11T19:37:05.328686+0000","last_active":"2024-03-11T19:38:38.540064+0000","last_peered":"2024-03-11T19:38:38.540064+0000","last_clean":"2024-03-11T19:38:38.540064+0000","last_became_active":"2024-03-11T19:37:05.328372+0000","last_became_peered":"2024-03-11T19:37:05.328372+0000","last_unstale":"2024-03-11T19:38:38.540064+0000","last_undegraded":"2024-03-11T19:38:38.540064+0000","last_fullsized":"2024-03-11T19:38:38.540064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:38:33.216489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.544161+0000","last_change":"2024-03-11T19:37:05.335057+0000","last_active":"2024-03-11T19:38:38.544161+0000","last_peered":"2024-03-11T19:38:38.544161+0000","last_clean":"2024-03-11T19:38:38.544161+0000","last_became_active":"2024-03-11T19:37:05.334646+0000","last_became_peered":"2024-03-11T19:37:05.334646+0000","last_unstale":"2024-03-11T19:38:38.544161+0000","last_undegraded":"2024-03-11T19:38:38.544161+0000","last_fullsized":"2024-03-11T19:38:38.544161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:34:15.932429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":132,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:55.784973+0000","last_change":"2024-03-11T19:37:04.588095+0000","last_active":"2024-03-11T19:38:55.784973+0000","last_peered":"2024-03-11T19:38:55.784973+0000","last_clean":"2024-03-11T19:38:55.784973+0000","last_became_active":"2024-03-11T19:37:04.587757+0000","last_became_peered":"2024-03-11T19:37:04.587757+0000","last_unstale":"2024-03-11T19:38:55.784973+0000","last_undegraded":"2024-03-11T19:38:55.784973+0000","last_fullsized":"2024-03-11T19:38:55.784973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:57.926858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540458+0000","last_change":"2024-03-11T19:37:05.326806+0000","last_active":"2024-03-11T19:38:38.540458+0000","last_peered":"2024-03-11T19:38:38.540458+0000","last_clean":"2024-03-11T19:38:38.540458+0000","last_became_active":"2024-03-11T19:37:05.326506+0000","last_became_peered":"2024-03-11T19:37:05.326506+0000","last_unstale":"2024-03-11T19:38:38.540458+0000","last_undegraded":"2024-03-11T19:38:38.540458+0000","last_fullsized":"2024-03-11T19:38:38.540458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:20.664857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453750+0000","last_change":"2024-03-11T19:37:04.588459+0000","last_active":"2024-03-11T19:38:10.453750+0000","last_peered":"2024-03-11T19:38:10.453750+0000","last_clean":"2024-03-11T19:38:10.453750+0000","last_became_active":"2024-03-11T19:37:04.588000+0000","last_became_peered":"2024-03-11T19:37:04.588000+0000","last_unstale":"2024-03-11T19:38:10.453750+0000","last_undegraded":"2024-03-11T19:38:10.453750+0000","last_fullsized":"2024-03-11T19:38:10.453750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:20:47.717196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150281+0000","last_change":"2024-03-11T19:35:55.814178+0000","last_active":"2024-03-11T19:38:10.150281+0000","last_peered":"2024-03-11T19:38:10.150281+0000","last_clean":"2024-03-11T19:38:10.150281+0000","last_became_active":"2024-03-11T19:35:55.813697+0000","last_became_peered":"2024-03-11T19:35:55.813697+0000","last_unstale":"2024-03-11T19:38:10.150281+0000","last_undegraded":"2024-03-11T19:38:10.150281+0000","last_fullsized":"2024-03-11T19:38:10.150281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:32:34.872292+0000","last_clean_scrub_stamp":"2024-03-11T19:32:34.872292+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:50:27.486733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543638+0000","last_change":"2024-03-11T19:37:05.319614+0000","last_active":"2024-03-11T19:38:38.543638+0000","last_peered":"2024-03-11T19:38:38.543638+0000","last_clean":"2024-03-11T19:38:38.543638+0000","last_became_active":"2024-03-11T19:37:05.319277+0000","last_became_peered":"2024-03-11T19:37:05.319277+0000","last_unstale":"2024-03-11T19:38:38.543638+0000","last_undegraded":"2024-03-11T19:38:38.543638+0000","last_fullsized":"2024-03-11T19:38:38.543638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:03:27.329900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.150075+0000","last_change":"2024-03-11T19:37:04.582250+0000","last_active":"2024-03-11T19:38:10.150075+0000","last_peered":"2024-03-11T19:38:10.150075+0000","last_clean":"2024-03-11T19:38:10.150075+0000","last_became_active":"2024-03-11T19:37:04.581938+0000","last_became_peered":"2024-03-11T19:37:04.581938+0000","last_unstale":"2024-03-11T19:38:10.150075+0000","last_undegraded":"2024-03-11T19:38:10.150075+0000","last_fullsized":"2024-03-11T19:38:10.150075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:53:23.466067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.151834+0000","last_change":"2024-03-11T19:37:04.590699+0000","last_active":"2024-03-11T19:38:10.151834+0000","last_peered":"2024-03-11T19:38:10.151834+0000","last_clean":"2024-03-11T19:38:10.151834+0000","last_became_active":"2024-03-11T19:37:04.590424+0000","last_became_peered":"2024-03-11T19:37:04.590424+0000","last_unstale":"2024-03-11T19:38:10.151834+0000","last_undegraded":"2024-03-11T19:38:10.151834+0000","last_fullsized":"2024-03-11T19:38:10.151834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:18:49.075335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.538473+0000","last_change":"2024-03-11T19:37:05.323357+0000","last_active":"2024-03-11T19:38:38.538473+0000","last_peered":"2024-03-11T19:38:38.538473+0000","last_clean":"2024-03-11T19:38:38.538473+0000","last_became_active":"2024-03-11T19:37:05.322980+0000","last_became_peered":"2024-03-11T19:37:05.322980+0000","last_unstale":"2024-03-11T19:38:38.538473+0000","last_undegraded":"2024-03-11T19:38:38.538473+0000","last_fullsized":"2024-03-11T19:38:38.538473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:29:19.106375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.589118+0000","last_change":"2024-03-11T19:37:04.585232+0000","last_active":"2024-03-11T19:38:38.589118+0000","last_peered":"2024-03-11T19:38:38.589118+0000","last_clean":"2024-03-11T19:38:38.589118+0000","last_became_active":"2024-03-11T19:37:04.584904+0000","last_became_peered":"2024-03-11T19:37:04.584904+0000","last_unstale":"2024-03-11T19:38:38.589118+0000","last_undegraded":"2024-03-11T19:38:38.589118+0000","last_fullsized":"2024-03-11T19:38:38.589118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:56:46.519961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553409+0000","last_change":"2024-03-11T19:37:05.332019+0000","last_active":"2024-03-11T19:38:38.553409+0000","last_peered":"2024-03-11T19:38:38.553409+0000","last_clean":"2024-03-11T19:38:38.553409+0000","last_became_active":"2024-03-11T19:37:05.331673+0000","last_became_peered":"2024-03-11T19:37:05.331673+0000","last_unstale":"2024-03-11T19:38:38.553409+0000","last_undegraded":"2024-03-11T19:38:38.553409+0000","last_fullsized":"2024-03-11T19:38:38.553409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:49:45.787082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.454347+0000","last_change":"2024-03-11T19:37:04.583404+0000","last_active":"2024-03-11T19:38:10.454347+0000","last_peered":"2024-03-11T19:38:10.454347+0000","last_clean":"2024-03-11T19:38:10.454347+0000","last_became_active":"2024-03-11T19:37:04.583025+0000","last_became_peered":"2024-03-11T19:37:04.583025+0000","last_unstale":"2024-03-11T19:38:10.454347+0000","last_undegraded":"2024-03-11T19:38:10.454347+0000","last_fullsized":"2024-03-11T19:38:10.454347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:02:25.202591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549246+0000","last_change":"2024-03-11T19:37:05.331711+0000","last_active":"2024-03-11T19:38:38.549246+0000","last_peered":"2024-03-11T19:38:38.549246+0000","last_clean":"2024-03-11T19:38:38.549246+0000","last_became_active":"2024-03-11T19:37:05.330851+0000","last_became_peered":"2024-03-11T19:37:05.330851+0000","last_unstale":"2024-03-11T19:38:38.549246+0000","last_undegraded":"2024-03-11T19:38:38.549246+0000","last_fullsized":"2024-03-11T19:38:38.549246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:18:26.692068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.448230+0000","last_change":"2024-03-11T19:37:04.586501+0000","last_active":"2024-03-11T19:38:10.448230+0000","last_peered":"2024-03-11T19:38:10.448230+0000","last_clean":"2024-03-11T19:38:10.448230+0000","last_became_active":"2024-03-11T19:37:04.586200+0000","last_became_peered":"2024-03-11T19:37:04.586200+0000","last_unstale":"2024-03-11T19:38:10.448230+0000","last_undegraded":"2024-03-11T19:38:10.448230+0000","last_fullsized":"2024-03-11T19:38:10.448230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:12:56.635375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.510995+0000","last_change":"2024-03-11T19:37:05.332402+0000","last_active":"2024-03-11T19:38:38.510995+0000","last_peered":"2024-03-11T19:38:38.510995+0000","last_clean":"2024-03-11T19:38:38.510995+0000","last_became_active":"2024-03-11T19:37:05.332087+0000","last_became_peered":"2024-03-11T19:37:05.332087+0000","last_unstale":"2024-03-11T19:38:38.510995+0000","last_undegraded":"2024-03-11T19:38:38.510995+0000","last_fullsized":"2024-03-11T19:38:38.510995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:10:23.882296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.452565+0000","last_change":"2024-03-11T19:37:04.584978+0000","last_active":"2024-03-11T19:38:10.452565+0000","last_peered":"2024-03-11T19:38:10.452565+0000","last_clean":"2024-03-11T19:38:10.452565+0000","last_became_active":"2024-03-11T19:37:04.584556+0000","last_became_peered":"2024-03-11T19:37:04.584556+0000","last_unstale":"2024-03-11T19:38:10.452565+0000","last_undegraded":"2024-03-11T19:38:10.452565+0000","last_fullsized":"2024-03-11T19:38:10.452565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:39:48.367112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.539294+0000","last_change":"2024-03-11T19:37:05.334453+0000","last_active":"2024-03-11T19:38:38.539294+0000","last_peered":"2024-03-11T19:38:38.539294+0000","last_clean":"2024-03-11T19:38:38.539294+0000","last_became_active":"2024-03-11T19:37:05.334009+0000","last_became_peered":"2024-03-11T19:37:05.334009+0000","last_unstale":"2024-03-11T19:38:38.539294+0000","last_undegraded":"2024-03-11T19:38:38.539294+0000","last_fullsized":"2024-03-11T19:38:38.539294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:43.402674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155193+0000","last_change":"2024-03-11T19:37:04.583181+0000","last_active":"2024-03-11T19:38:10.155193+0000","last_peered":"2024-03-11T19:38:10.155193+0000","last_clean":"2024-03-11T19:38:10.155193+0000","last_became_active":"2024-03-11T19:37:04.582881+0000","last_became_peered":"2024-03-11T19:37:04.582881+0000","last_unstale":"2024-03-11T19:38:10.155193+0000","last_undegraded":"2024-03-11T19:38:10.155193+0000","last_fullsized":"2024-03-11T19:38:10.155193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:05:06.634150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.530930+0000","last_change":"2024-03-11T19:37:05.333642+0000","last_active":"2024-03-11T19:38:38.530930+0000","last_peered":"2024-03-11T19:38:38.530930+0000","last_clean":"2024-03-11T19:38:38.530930+0000","last_became_active":"2024-03-11T19:37:05.333121+0000","last_became_peered":"2024-03-11T19:37:05.333121+0000","last_unstale":"2024-03-11T19:38:38.530930+0000","last_undegraded":"2024-03-11T19:38:38.530930+0000","last_fullsized":"2024-03-11T19:38:38.530930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:07:21.217101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.458830+0000","last_change":"2024-03-11T19:38:10.458358+0000","last_active":"2024-03-11T19:38:10.458830+0000","last_peered":"2024-03-11T19:38:10.458830+0000","last_clean":"2024-03-11T19:38:10.458830+0000","last_became_active":"2024-03-11T19:38:10.449890+0000","last_became_peered":"2024-03-11T19:38:10.449890+0000","last_unstale":"2024-03-11T19:38:10.458830+0000","last_undegraded":"2024-03-11T19:38:10.458830+0000","last_fullsized":"2024-03-11T19:38:10.458830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:51:00.265814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.554245+0000","last_change":"2024-03-11T19:37:05.326992+0000","last_active":"2024-03-11T19:38:38.554245+0000","last_peered":"2024-03-11T19:38:38.554245+0000","last_clean":"2024-03-11T19:38:38.554245+0000","last_became_active":"2024-03-11T19:37:05.326676+0000","last_became_peered":"2024-03-11T19:37:05.326676+0000","last_unstale":"2024-03-11T19:38:38.554245+0000","last_undegraded":"2024-03-11T19:38:38.554245+0000","last_fullsized":"2024-03-11T19:38:38.554245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:31:10.310650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.155451+0000","last_change":"2024-03-11T19:37:04.583631+0000","last_active":"2024-03-11T19:38:10.155451+0000","last_peered":"2024-03-11T19:38:10.155451+0000","last_clean":"2024-03-11T19:38:10.155451+0000","last_became_active":"2024-03-11T19:37:04.583031+0000","last_became_peered":"2024-03-11T19:37:04.583031+0000","last_unstale":"2024-03-11T19:38:10.155451+0000","last_undegraded":"2024-03-11T19:38:10.155451+0000","last_fullsized":"2024-03-11T19:38:10.155451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:52:20.695496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553916+0000","last_change":"2024-03-11T19:37:05.331008+0000","last_active":"2024-03-11T19:38:38.553916+0000","last_peered":"2024-03-11T19:38:38.553916+0000","last_clean":"2024-03-11T19:38:38.553916+0000","last_became_active":"2024-03-11T19:37:05.330647+0000","last_became_peered":"2024-03-11T19:37:05.330647+0000","last_unstale":"2024-03-11T19:38:38.553916+0000","last_undegraded":"2024-03-11T19:38:38.553916+0000","last_fullsized":"2024-03-11T19:38:38.553916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:07:00.576137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453400+0000","last_change":"2024-03-11T19:37:04.582501+0000","last_active":"2024-03-11T19:38:10.453400+0000","last_peered":"2024-03-11T19:38:10.453400+0000","last_clean":"2024-03-11T19:38:10.453400+0000","last_became_active":"2024-03-11T19:37:04.582076+0000","last_became_peered":"2024-03-11T19:37:04.582076+0000","last_unstale":"2024-03-11T19:38:10.453400+0000","last_undegraded":"2024-03-11T19:38:10.453400+0000","last_fullsized":"2024-03-11T19:38:10.453400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:46:05.359211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549826+0000","last_change":"2024-03-11T19:37:05.329053+0000","last_active":"2024-03-11T19:38:38.549826+0000","last_peered":"2024-03-11T19:38:38.549826+0000","last_clean":"2024-03-11T19:38:38.549826+0000","last_became_active":"2024-03-11T19:37:05.328748+0000","last_became_peered":"2024-03-11T19:37:05.328748+0000","last_unstale":"2024-03-11T19:38:38.549826+0000","last_undegraded":"2024-03-11T19:38:38.549826+0000","last_fullsized":"2024-03-11T19:38:38.549826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:48:55.369515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:10.453324+0000","last_change":"2024-03-11T19:37:04.583370+0000","last_active":"2024-03-11T19:38:10.453324+0000","last_peered":"2024-03-11T19:38:10.453324+0000","last_clean":"2024-03-11T19:38:10.453324+0000","last_became_active":"2024-03-11T19:37:04.583055+0000","last_became_peered":"2024-03-11T19:37:04.583055+0000","last_unstale":"2024-03-11T19:38:10.453324+0000","last_undegraded":"2024-03-11T19:38:10.453324+0000","last_fullsized":"2024-03-11T19:38:10.453324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:03.553377+0000","last_clean_scrub_stamp":"2024-03-11T19:37:03.553377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:04:02.325054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.550503+0000","last_change":"2024-03-11T19:37:05.328157+0000","last_active":"2024-03-11T19:38:38.550503+0000","last_peered":"2024-03-11T19:38:38.550503+0000","last_clean":"2024-03-11T19:38:38.550503+0000","last_became_active":"2024-03-11T19:37:05.327839+0000","last_became_peered":"2024-03-11T19:37:05.327839+0000","last_unstale":"2024-03-11T19:38:38.550503+0000","last_undegraded":"2024-03-11T19:38:38.550503+0000","last_fullsized":"2024-03-11T19:38:38.550503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:23:02.297269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.543426+0000","last_change":"2024-03-11T19:37:05.336160+0000","last_active":"2024-03-11T19:38:38.543426+0000","last_peered":"2024-03-11T19:38:38.543426+0000","last_clean":"2024-03-11T19:38:38.543426+0000","last_became_active":"2024-03-11T19:37:05.335850+0000","last_became_peered":"2024-03-11T19:37:05.335850+0000","last_unstale":"2024-03-11T19:38:38.543426+0000","last_undegraded":"2024-03-11T19:38:38.543426+0000","last_fullsized":"2024-03-11T19:38:38.543426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:15:09.743661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.536039+0000","last_change":"2024-03-11T19:38:10.455362+0000","last_active":"2024-03-11T19:38:38.536039+0000","last_peered":"2024-03-11T19:38:38.536039+0000","last_clean":"2024-03-11T19:38:38.536039+0000","last_became_active":"2024-03-11T19:38:10.454887+0000","last_became_peered":"2024-03-11T19:38:10.454887+0000","last_unstale":"2024-03-11T19:38:38.536039+0000","last_undegraded":"2024-03-11T19:38:38.536039+0000","last_fullsized":"2024-03-11T19:38:38.536039+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:48:19.289829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.542420+0000","last_change":"2024-03-11T19:37:05.332357+0000","last_active":"2024-03-11T19:38:38.542420+0000","last_peered":"2024-03-11T19:38:38.542420+0000","last_clean":"2024-03-11T19:38:38.542420+0000","last_became_active":"2024-03-11T19:37:05.332003+0000","last_became_peered":"2024-03-11T19:37:05.332003+0000","last_unstale":"2024-03-11T19:38:38.542420+0000","last_undegraded":"2024-03-11T19:38:38.542420+0000","last_fullsized":"2024-03-11T19:38:38.542420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:35:39.165572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.549322+0000","last_change":"2024-03-11T19:37:05.332388+0000","last_active":"2024-03-11T19:38:38.549322+0000","last_peered":"2024-03-11T19:38:38.549322+0000","last_clean":"2024-03-11T19:38:38.549322+0000","last_became_active":"2024-03-11T19:37:05.332032+0000","last_became_peered":"2024-03-11T19:37:05.332032+0000","last_unstale":"2024-03-11T19:38:38.549322+0000","last_undegraded":"2024-03-11T19:38:38.549322+0000","last_fullsized":"2024-03-11T19:38:38.549322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:56:09.416683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.545668+0000","last_change":"2024-03-11T19:37:05.323096+0000","last_active":"2024-03-11T19:38:38.545668+0000","last_peered":"2024-03-11T19:38:38.545668+0000","last_clean":"2024-03-11T19:38:38.545668+0000","last_became_active":"2024-03-11T19:37:05.322525+0000","last_became_peered":"2024-03-11T19:37:05.322525+0000","last_unstale":"2024-03-11T19:38:38.545668+0000","last_undegraded":"2024-03-11T19:38:38.545668+0000","last_fullsized":"2024-03-11T19:38:38.545668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:53:01.870894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540652+0000","last_change":"2024-03-11T19:37:05.331171+0000","last_active":"2024-03-11T19:38:38.540652+0000","last_peered":"2024-03-11T19:38:38.540652+0000","last_clean":"2024-03-11T19:38:38.540652+0000","last_became_active":"2024-03-11T19:37:05.330769+0000","last_became_peered":"2024-03-11T19:37:05.330769+0000","last_unstale":"2024-03-11T19:38:38.540652+0000","last_undegraded":"2024-03-11T19:38:38.540652+0000","last_fullsized":"2024-03-11T19:38:38.540652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:03:06.514039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553501+0000","last_change":"2024-03-11T19:37:05.326877+0000","last_active":"2024-03-11T19:38:38.553501+0000","last_peered":"2024-03-11T19:38:38.553501+0000","last_clean":"2024-03-11T19:38:38.553501+0000","last_became_active":"2024-03-11T19:37:05.326538+0000","last_became_peered":"2024-03-11T19:37:05.326538+0000","last_unstale":"2024-03-11T19:38:38.553501+0000","last_undegraded":"2024-03-11T19:38:38.553501+0000","last_fullsized":"2024-03-11T19:38:38.553501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:11.565697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.530558+0000","last_change":"2024-03-11T19:37:05.335010+0000","last_active":"2024-03-11T19:38:38.530558+0000","last_peered":"2024-03-11T19:38:38.530558+0000","last_clean":"2024-03-11T19:38:38.530558+0000","last_became_active":"2024-03-11T19:37:05.334551+0000","last_became_peered":"2024-03-11T19:37:05.334551+0000","last_unstale":"2024-03-11T19:38:38.530558+0000","last_undegraded":"2024-03-11T19:38:38.530558+0000","last_fullsized":"2024-03-11T19:38:38.530558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:54:59.668178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.540099+0000","last_change":"2024-03-11T19:37:05.336273+0000","last_active":"2024-03-11T19:38:38.540099+0000","last_peered":"2024-03-11T19:38:38.540099+0000","last_clean":"2024-03-11T19:38:38.540099+0000","last_became_active":"2024-03-11T19:37:05.335926+0000","last_became_peered":"2024-03-11T19:37:05.335926+0000","last_unstale":"2024-03-11T19:38:38.540099+0000","last_undegraded":"2024-03-11T19:38:38.540099+0000","last_fullsized":"2024-03-11T19:38:38.540099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:31:55.139242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.533346+0000","last_change":"2024-03-11T19:37:05.335548+0000","last_active":"2024-03-11T19:38:38.533346+0000","last_peered":"2024-03-11T19:38:38.533346+0000","last_clean":"2024-03-11T19:38:38.533346+0000","last_became_active":"2024-03-11T19:37:05.335222+0000","last_became_peered":"2024-03-11T19:37:05.335222+0000","last_unstale":"2024-03-11T19:38:38.533346+0000","last_undegraded":"2024-03-11T19:38:38.533346+0000","last_fullsized":"2024-03-11T19:38:38.533346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:07:25.697951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.554788+0000","last_change":"2024-03-11T19:37:05.329782+0000","last_active":"2024-03-11T19:38:38.554788+0000","last_peered":"2024-03-11T19:38:38.554788+0000","last_clean":"2024-03-11T19:38:38.554788+0000","last_became_active":"2024-03-11T19:37:05.329434+0000","last_became_peered":"2024-03-11T19:37:05.329434+0000","last_unstale":"2024-03-11T19:38:38.554788+0000","last_undegraded":"2024-03-11T19:38:38.554788+0000","last_fullsized":"2024-03-11T19:38:38.554788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:12:00.516489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.553896+0000","last_change":"2024-03-11T19:37:05.323026+0000","last_active":"2024-03-11T19:38:38.553896+0000","last_peered":"2024-03-11T19:38:38.553896+0000","last_clean":"2024-03-11T19:38:38.553896+0000","last_became_active":"2024-03-11T19:37:05.322562+0000","last_became_peered":"2024-03-11T19:37:05.322562+0000","last_unstale":"2024-03-11T19:38:38.553896+0000","last_undegraded":"2024-03-11T19:38:38.553896+0000","last_fullsized":"2024-03-11T19:38:38.553896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:14:57.606404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-11T19:38:38.533719+0000","last_change":"2024-03-11T19:38:10.455616+0000","last_active":"2024-03-11T19:38:38.533719+0000","last_peered":"2024-03-11T19:38:38.533719+0000","last_clean":"2024-03-11T19:38:38.533719+0000","last_became_active":"2024-03-11T19:38:10.455092+0000","last_became_peered":"2024-03-11T19:38:10.455092+0000","last_unstale":"2024-03-11T19:38:38.533719+0000","last_undegraded":"2024-03-11T19:38:38.533719+0000","last_fullsized":"2024-03-11T19:38:38.533719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T19:37:04.562222+0000","last_clean_scrub_stamp":"2024-03-11T19:37:04.562222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:49:50.407540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65254,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":678,"num_read_kb":409,"num_write":171,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383716,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":455694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32776,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712376,"statfs":{"total":95995035648,"available":95961473024,"internally_reserved":0,"allocated":1638400,"data_stored":1108175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":455818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":455626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1048341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":455207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":515580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":456478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":517350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":455518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1047493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T19:39:01.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:39:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.862+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/736548503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0102390 msgr2=0x7f0ba0102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.862+0000 7f0ba4f4b640 1 --2- 172.21.15.83:0/736548503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0102390 0x7f0ba0102810 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009900 tx=0x7f0b9402bc30 comp rx=0 tx=0).stop 2024-03-11T19:39:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.862+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/736548503 shutdown_connections 2024-03-11T19:39:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.862+0000 7f0ba4f4b640 1 --2- 172.21.15.83:0/736548503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba0102d50 0x7f0ba01095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.862+0000 7f0ba4f4b640 1 --2- 172.21.15.83:0/736548503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0102390 0x7f0ba0102810 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.862+0000 7f0ba4f4b640 1 --2- 172.21.15.83:0/736548503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba01011a0 0x7f0ba01015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.862+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/736548503 >> 172.21.15.83:0/736548503 conn(0x7f0ba00fc730 msgr2=0x7f0ba00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.862+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/736548503 shutdown_connections 2024-03-11T19:39:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.862+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/736548503 wait complete. 2024-03-11T19:39:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0ba4f4b640 1 Processor -- start 2024-03-11T19:39:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0ba4f4b640 1 -- start start 2024-03-11T19:39:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0ba4f4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba01011a0 0x7f0ba0197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0ba4f4b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0102390 0x7f0ba0197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0ba4f4b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba0102d50 0x7f0ba019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0ba4f4b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba019ed00 con 0x7f0ba0102390 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0ba4f4b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba019ee70 con 0x7f0ba01011a0 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0ba4f4b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba019f030 con 0x7f0ba0102d50 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0102390 0x7f0ba0197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0102390 0x7f0ba0197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50560/0 (socket says 172.21.15.83:50560) 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9dd9b640 1 -- 172.21.15.83:0/2357602817 learned_addr learned my addr 172.21.15.83:0/2357602817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9ed9d640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba0102d50 0x7f0ba019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9e59c640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba01011a0 0x7f0ba0197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9dd9b640 1 -- 172.21.15.83:0/2357602817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba0102d50 msgr2=0x7f0ba019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9dd9b640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba0102d50 0x7f0ba019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9dd9b640 1 -- 172.21.15.83:0/2357602817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba01011a0 msgr2=0x7f0ba0197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9dd9b640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba01011a0 0x7f0ba0197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9dd9b640 1 -- 172.21.15.83:0/2357602817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b94009560 con 0x7f0ba0102390 2024-03-11T19:39:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9e59c640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba01011a0 0x7f0ba0197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:39:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.863+0000 7f0b9dd9b640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0102390 0x7f0ba0197750 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0b94006010 tx=0x7f0b940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.864+0000 7f0b8f7fe640 1 -- 172.21.15.83:0/2357602817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b940044e0 con 0x7f0ba0102390 2024-03-11T19:39:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.864+0000 7f0b8f7fe640 1 -- 172.21.15.83:0/2357602817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b940054e0 con 0x7f0ba0102390 2024-03-11T19:39:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.864+0000 7f0b8f7fe640 1 -- 172.21.15.83:0/2357602817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9403e450 con 0x7f0ba0102390 2024-03-11T19:39:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.864+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/2357602817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba019f2b0 con 0x7f0ba0102390 2024-03-11T19:39:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.864+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/2357602817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba019f7c0 con 0x7f0ba0102390 2024-03-11T19:39:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.865+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/2357602817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba01084c0 con 0x7f0ba0102390 2024-03-11T19:39:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.866+0000 7f0b8f7fe640 1 -- 172.21.15.83:0/2357602817 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0b940035f0 con 0x7f0ba0102390 2024-03-11T19:39:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.866+0000 7f0b8f7fe640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b880a1a30 0x7f0b880a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.866+0000 7f0b9e59c640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b880a1a30 0x7f0b880a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.867+0000 7f0b8f7fe640 1 -- 172.21.15.83:0/2357602817 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0b940ef750 con 0x7f0ba0102390 2024-03-11T19:39:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.867+0000 7f0b9e59c640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b880a1a30 0x7f0b880a3ef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b900058c0 tx=0x7f0b90005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:01.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:01.871+0000 7f0b8f7fe640 1 -- 172.21.15.83:0/2357602817 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0b940b8a10 con 0x7f0ba0102390 2024-03-11T19:39:02.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.046+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/2357602817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ba010ce00 con 0x7f0ba0102390 2024-03-11T19:39:02.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.048+0000 7f0b8f7fe640 1 -- 172.21.15.83:0/2357602817 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7f0b940bc0d0 con 0x7f0ba0102390 2024-03-11T19:39:02.052 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:02.052 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:39:02.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/2357602817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b880a1a30 msgr2=0x7f0b880a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b880a1a30 0x7f0b880a3ef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b900058c0 tx=0x7f0b90005850 comp rx=0 tx=0).stop 2024-03-11T19:39:02.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/2357602817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0102390 msgr2=0x7f0ba0197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0102390 0x7f0ba0197750 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0b94006010 tx=0x7f0b940042f0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/2357602817 shutdown_connections 2024-03-11T19:39:02.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b880a1a30 0x7f0b880a3ef0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ba0102d50 0x7f0ba019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba0102390 0x7f0ba0197750 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 --2- 172.21.15.83:0/2357602817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ba01011a0 0x7f0ba0197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/2357602817 >> 172.21.15.83:0/2357602817 conn(0x7f0ba00fc730 msgr2=0x7f0ba0105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:02.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/2357602817 shutdown_connections 2024-03-11T19:39:02.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.050+0000 7f0ba4f4b640 1 -- 172.21.15.83:0/2357602817 wait complete. 2024-03-11T19:39:02.066 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:37:03.034982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:37:04.125352+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:39:02.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:39:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.167+0000 7f8c64fe9640 1 -- 172.21.15.83:0/1409571862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c60100080 msgr2=0x7f8c601004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.167+0000 7f8c64fe9640 1 --2- 172.21.15.83:0/1409571862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c60100080 0x7f8c601004a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009900 tx=0x7f8c5402bc30 comp rx=0 tx=0).stop 2024-03-11T19:39:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.167+0000 7f8c64fe9640 1 -- 172.21.15.83:0/1409571862 shutdown_connections 2024-03-11T19:39:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.167+0000 7f8c64fe9640 1 --2- 172.21.15.83:0/1409571862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c60101c30 0x7f8c601084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.167+0000 7f8c64fe9640 1 --2- 172.21.15.83:0/1409571862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c60101270 0x7f8c601016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.167+0000 7f8c64fe9640 1 --2- 172.21.15.83:0/1409571862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c60100080 0x7f8c601004a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.167+0000 7f8c64fe9640 1 -- 172.21.15.83:0/1409571862 >> 172.21.15.83:0/1409571862 conn(0x7f8c600fb610 msgr2=0x7f8c600fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.167+0000 7f8c64fe9640 1 -- 172.21.15.83:0/1409571862 shutdown_connections 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.167+0000 7f8c64fe9640 1 -- 172.21.15.83:0/1409571862 wait complete. 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.167+0000 7f8c64fe9640 1 Processor -- start 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.167+0000 7f8c64fe9640 1 -- start start 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c64fe9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c60100080 0x7f8c60197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c64fe9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c60101270 0x7f8c60197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c64fe9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c60101c30 0x7f8c6019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c64fe9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6019ef20 con 0x7f8c60101270 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c64fe9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6019f090 con 0x7f8c60101c30 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c64fe9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6019f250 con 0x7f8c60100080 2024-03-11T19:39:02.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c60101270 0x7f8c60197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c60101270 0x7f8c60197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50576/0 (socket says 172.21.15.83:50576) 2024-03-11T19:39:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5dd9b640 1 -- 172.21.15.83:0/259187806 learned_addr learned my addr 172.21.15.83:0/259187806 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5e59c640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c60100080 0x7f8c60197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5ed9d640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c60101c30 0x7f8c6019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5dd9b640 1 -- 172.21.15.83:0/259187806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c60100080 msgr2=0x7f8c60197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5dd9b640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c60100080 0x7f8c60197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5dd9b640 1 -- 172.21.15.83:0/259187806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c60101c30 msgr2=0x7f8c6019e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5dd9b640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c60101c30 0x7f8c6019e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5dd9b640 1 -- 172.21.15.83:0/259187806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c54009560 con 0x7f8c60101270 2024-03-11T19:39:02.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5ed9d640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c60101c30 0x7f8c6019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:39:02.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c5dd9b640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c60101270 0x7f8c60197970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000b4a0 tx=0x7f8c5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:02.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c4f7fe640 1 -- 172.21.15.83:0/259187806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c50010610 con 0x7f8c60101270 2024-03-11T19:39:02.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c64fe9640 1 -- 172.21.15.83:0/259187806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6019f530 con 0x7f8c60101270 2024-03-11T19:39:02.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c4f7fe640 1 -- 172.21.15.83:0/259187806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c50010c50 con 0x7f8c60101270 2024-03-11T19:39:02.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c4f7fe640 1 -- 172.21.15.83:0/259187806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c50017e20 con 0x7f8c60101270 2024-03-11T19:39:02.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.168+0000 7f8c64fe9640 1 -- 172.21.15.83:0/259187806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6019fa70 con 0x7f8c60101270 2024-03-11T19:39:02.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.170+0000 7f8c64fe9640 1 -- 172.21.15.83:0/259187806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6010bb50 con 0x7f8c60101270 2024-03-11T19:39:02.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.170+0000 7f8c4f7fe640 1 -- 172.21.15.83:0/259187806 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8c50017480 con 0x7f8c60101270 2024-03-11T19:39:02.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.171+0000 7f8c4f7fe640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8c480a1a30 0x7f8c480a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.171+0000 7f8c5e59c640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8c480a1a30 0x7f8c480a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.171+0000 7f8c4f7fe640 1 -- 172.21.15.83:0/259187806 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8c500cca80 con 0x7f8c60101270 2024-03-11T19:39:02.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.172+0000 7f8c5e59c640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8c480a1a30 0x7f8c480a3ef0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8c54006010 tx=0x7f8c54000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:02.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.176+0000 7f8c4f7fe640 1 -- 172.21.15.83:0/259187806 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c50095d40 con 0x7f8c60101270 2024-03-11T19:39:02.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:01 smithi083 ceph-mon[31864]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:39:02.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:01 smithi083 ceph-mon[31864]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:39:02.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.349+0000 7f8c64fe9640 1 -- 172.21.15.83:0/259187806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c6019fe80 con 0x7f8c60101270 2024-03-11T19:39:02.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.349+0000 7f8c4f7fe640 1 -- 172.21.15.83:0/259187806 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c50099400 con 0x7f8c60101270 2024-03-11T19:39:02.353 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:39:02.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 -- 172.21.15.83:0/259187806 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8c480a1a30 msgr2=0x7f8c480a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8c480a1a30 0x7f8c480a3ef0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8c54006010 tx=0x7f8c54000bc0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 -- 172.21.15.83:0/259187806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c60101270 msgr2=0x7f8c60197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c60101270 0x7f8c60197970 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000b4a0 tx=0x7f8c5000b960 comp rx=0 tx=0).stop 2024-03-11T19:39:02.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 -- 172.21.15.83:0/259187806 shutdown_connections 2024-03-11T19:39:02.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8c480a1a30 0x7f8c480a3ef0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c60101c30 0x7f8c6019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c60101270 0x7f8c60197970 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 --2- 172.21.15.83:0/259187806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c60100080 0x7f8c60197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 -- 172.21.15.83:0/259187806 >> 172.21.15.83:0/259187806 conn(0x7f8c600fb610 msgr2=0x7f8c60068390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:02.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 -- 172.21.15.83:0/259187806 shutdown_connections 2024-03-11T19:39:02.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.351+0000 7f8c64fe9640 1 -- 172.21.15.83:0/259187806 wait complete. 2024-03-11T19:39:02.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:39:02.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:01 smithi137 ceph-mon[36220]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:39:02.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:01 smithi137 ceph-mon[36220]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:39:02.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:01 smithi204 ceph-mon[36739]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:39:02.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:01 smithi204 ceph-mon[36739]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:39:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.467+0000 7f58eacb5640 1 -- 172.21.15.83:0/3741799456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e4101c30 msgr2=0x7f58e41084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.467+0000 7f58eacb5640 1 --2- 172.21.15.83:0/3741799456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e4101c30 0x7f58e41084c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009ca0 tx=0x7f58dc02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.467+0000 7f58eacb5640 1 -- 172.21.15.83:0/3741799456 shutdown_connections 2024-03-11T19:39:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.467+0000 7f58eacb5640 1 --2- 172.21.15.83:0/3741799456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e4101c30 0x7f58e41084c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.467+0000 7f58eacb5640 1 --2- 172.21.15.83:0/3741799456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58e4101270 0x7f58e41016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.467+0000 7f58eacb5640 1 --2- 172.21.15.83:0/3741799456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4100080 0x7f58e41004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.467+0000 7f58eacb5640 1 -- 172.21.15.83:0/3741799456 >> 172.21.15.83:0/3741799456 conn(0x7f58e40fb5f0 msgr2=0x7f58e40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.467+0000 7f58eacb5640 1 -- 172.21.15.83:0/3741799456 shutdown_connections 2024-03-11T19:39:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.467+0000 7f58eacb5640 1 -- 172.21.15.83:0/3741799456 wait complete. 2024-03-11T19:39:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.467+0000 7f58eacb5640 1 Processor -- start 2024-03-11T19:39:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.467+0000 7f58eacb5640 1 -- start start 2024-03-11T19:39:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58eacb5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58e4100080 0x7f58e406a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58eacb5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e4101270 0x7f58e419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58eacb5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4101c30 0x7f58e419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58eacb5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e419eca0 con 0x7f58e4101270 2024-03-11T19:39:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58eacb5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e419ee10 con 0x7f58e4101c30 2024-03-11T19:39:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58eacb5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e419efd0 con 0x7f58e4100080 2024-03-11T19:39:02.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e4101270 0x7f58e419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e4101270 0x7f58e419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50600/0 (socket says 172.21.15.83:50600) 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e3fff640 1 -- 172.21.15.83:0/3954810583 learned_addr learned my addr 172.21.15.83:0/3954810583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e8a51640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58e4100080 0x7f58e406a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e3fff640 1 -- 172.21.15.83:0/3954810583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58e4100080 msgr2=0x7f58e406a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e3fff640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58e4100080 0x7f58e406a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e3fff640 1 -- 172.21.15.83:0/3954810583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4101c30 msgr2=0x7f58e419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e9252640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4101c30 0x7f58e419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e3fff640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4101c30 0x7f58e419e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e3fff640 1 -- 172.21.15.83:0/3954810583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc009900 con 0x7f58e4101270 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e9252640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4101c30 0x7f58e419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e3fff640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e4101270 0x7f58e419bed0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f58d000c920 tx=0x7f58d000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e1ffb640 1 -- 172.21.15.83:0/3954810583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0007c70 con 0x7f58e4101270 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e1ffb640 1 -- 172.21.15.83:0/3954810583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58d0007dd0 con 0x7f58e4101270 2024-03-11T19:39:02.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58eacb5640 1 -- 172.21.15.83:0/3954810583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e419f2b0 con 0x7f58e4101270 2024-03-11T19:39:02.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58e1ffb640 1 -- 172.21.15.83:0/3954810583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0018860 con 0x7f58e4101270 2024-03-11T19:39:02.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.468+0000 7f58eacb5640 1 -- 172.21.15.83:0/3954810583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e419f7c0 con 0x7f58e4101270 2024-03-11T19:39:02.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.469+0000 7f58eacb5640 1 -- 172.21.15.83:0/3954810583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e41073a0 con 0x7f58e4101270 2024-03-11T19:39:02.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.470+0000 7f58e1ffb640 1 -- 172.21.15.83:0/3954810583 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f58d000fac0 con 0x7f58e4101270 2024-03-11T19:39:02.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.471+0000 7f58e1ffb640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58d40a1a30 0x7f58d40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.471+0000 7f58e1ffb640 1 -- 172.21.15.83:0/3954810583 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f58d00cd510 con 0x7f58e4101270 2024-03-11T19:39:02.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.471+0000 7f58e8a51640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58d40a1a30 0x7f58d40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.471+0000 7f58e8a51640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58d40a1a30 0x7f58d40a3ef0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f58e41010d0 tx=0x7f58d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:02.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.475+0000 7f58e1ffb640 1 -- 172.21.15.83:0/3954810583 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58d00967d0 con 0x7f58e4101270 2024-03-11T19:39:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.653+0000 7f58eacb5640 1 -- 172.21.15.83:0/3954810583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58e410ba90 con 0x7f58e4101270 2024-03-11T19:39:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.653+0000 7f58e1ffb640 1 -- 172.21.15.83:0/3954810583 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58d0099e90 con 0x7f58e4101270 2024-03-11T19:39:02.658 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 -- 172.21.15.83:0/3954810583 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58d40a1a30 msgr2=0x7f58d40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58d40a1a30 0x7f58d40a3ef0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f58e41010d0 tx=0x7f58d8005f90 comp rx=0 tx=0).stop 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 -- 172.21.15.83:0/3954810583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e4101270 msgr2=0x7f58e419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e4101270 0x7f58e419bed0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f58d000c920 tx=0x7f58d000cde0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 -- 172.21.15.83:0/3954810583 shutdown_connections 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f58d40a1a30 0x7f58d40a3ef0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4101c30 0x7f58e419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e4101270 0x7f58e419bed0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 --2- 172.21.15.83:0/3954810583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58e4100080 0x7f58e406a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 -- 172.21.15.83:0/3954810583 >> 172.21.15.83:0/3954810583 conn(0x7f58e40fb5f0 msgr2=0x7f58e4104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 -- 172.21.15.83:0/3954810583 shutdown_connections 2024-03-11T19:39:02.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.656+0000 7f58eacb5640 1 -- 172.21.15.83:0/3954810583 wait complete. 2024-03-11T19:39:02.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:39:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.771+0000 7f527322e640 1 -- 172.21.15.83:0/2703909818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c0ff1f0 msgr2=0x7f526c0ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.771+0000 7f527322e640 1 --2- 172.21.15.83:0/2703909818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c0ff1f0 0x7f526c0ff670 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5258009900 tx=0x7f525802bc30 comp rx=0 tx=0).stop 2024-03-11T19:39:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 -- 172.21.15.83:0/2703909818 shutdown_connections 2024-03-11T19:39:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 --2- 172.21.15.83:0/2703909818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f526c0ffbb0 0x7f526c109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 --2- 172.21.15.83:0/2703909818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c0ff1f0 0x7f526c0ff670 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 --2- 172.21.15.83:0/2703909818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f526c102780 0x7f526c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 -- 172.21.15.83:0/2703909818 >> 172.21.15.83:0/2703909818 conn(0x7f526c0fad70 msgr2=0x7f526c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 -- 172.21.15.83:0/2703909818 shutdown_connections 2024-03-11T19:39:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 -- 172.21.15.83:0/2703909818 wait complete. 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 Processor -- start 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 -- start start 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f526c0ff1f0 0x7f526c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c0ffbb0 0x7f526c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f526c102780 0x7f526c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c19ecc0 con 0x7f526c0ffbb0 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c19ee30 con 0x7f526c102780 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f527322e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526c19eff0 con 0x7f526c0ff1f0 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f526bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c0ffbb0 0x7f526c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f526bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c0ffbb0 0x7f526c197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50614/0 (socket says 172.21.15.83:50614) 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f526bfff640 1 -- 172.21.15.83:0/1894346486 learned_addr learned my addr 172.21.15.83:0/1894346486 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f52717cb640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f526c102780 0x7f526c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.772+0000 7f5270fca640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f526c0ff1f0 0x7f526c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.773+0000 7f526bfff640 1 -- 172.21.15.83:0/1894346486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f526c0ff1f0 msgr2=0x7f526c1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.773+0000 7f526bfff640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f526c0ff1f0 0x7f526c1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.773+0000 7f526bfff640 1 -- 172.21.15.83:0/1894346486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f526c102780 msgr2=0x7f526c19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.773+0000 7f526bfff640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f526c102780 0x7f526c19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.773+0000 7f526bfff640 1 -- 172.21.15.83:0/1894346486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5258009560 con 0x7f526c0ffbb0 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.773+0000 7f526bfff640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c0ffbb0 0x7f526c197710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5258005ab0 tx=0x7f5258004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.773+0000 7f5269ffb640 1 -- 172.21.15.83:0/1894346486 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5258004df0 con 0x7f526c0ffbb0 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.773+0000 7f5269ffb640 1 -- 172.21.15.83:0/1894346486 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5258005430 con 0x7f526c0ffbb0 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.773+0000 7f527322e640 1 -- 172.21.15.83:0/1894346486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526c19f270 con 0x7f526c0ffbb0 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.773+0000 7f5269ffb640 1 -- 172.21.15.83:0/1894346486 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525803e450 con 0x7f526c0ffbb0 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.773+0000 7f527322e640 1 -- 172.21.15.83:0/1894346486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526c19f800 con 0x7f526c0ffbb0 2024-03-11T19:39:02.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.774+0000 7f527322e640 1 -- 172.21.15.83:0/1894346486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f526c108640 con 0x7f526c0ffbb0 2024-03-11T19:39:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.775+0000 7f5269ffb640 1 -- 172.21.15.83:0/1894346486 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f52580055a0 con 0x7f526c0ffbb0 2024-03-11T19:39:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.775+0000 7f5269ffb640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f525c0a1a30 0x7f525c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.776+0000 7f5270fca640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f525c0a1a30 0x7f525c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.776+0000 7f5269ffb640 1 -- 172.21.15.83:0/1894346486 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f52580efae0 con 0x7f526c0ffbb0 2024-03-11T19:39:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.776+0000 7f5270fca640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f525c0a1a30 0x7f525c0a3ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52600058c0 tx=0x7f5260005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.780+0000 7f5269ffb640 1 -- 172.21.15.83:0/1894346486 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f52580b8da0 con 0x7f526c0ffbb0 2024-03-11T19:39:02.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.953+0000 7f527322e640 1 -- 172.21.15.83:0/1894346486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f526c02cda0 con 0x7f526c0ffbb0 2024-03-11T19:39:02.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.953+0000 7f5269ffb640 1 -- 172.21.15.83:0/1894346486 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52580bc460 con 0x7f526c0ffbb0 2024-03-11T19:39:02.957 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-03-11T19:39:02.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 -- 172.21.15.83:0/1894346486 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f525c0a1a30 msgr2=0x7f525c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f525c0a1a30 0x7f525c0a3ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52600058c0 tx=0x7f5260005850 comp rx=0 tx=0).stop 2024-03-11T19:39:02.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 -- 172.21.15.83:0/1894346486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c0ffbb0 msgr2=0x7f526c197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:02.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c0ffbb0 0x7f526c197710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5258005ab0 tx=0x7f5258004270 comp rx=0 tx=0).stop 2024-03-11T19:39:02.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 -- 172.21.15.83:0/1894346486 shutdown_connections 2024-03-11T19:39:02.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f525c0a1a30 0x7f525c0a3ef0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f526c102780 0x7f526c19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c0ffbb0 0x7f526c197710 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 --2- 172.21.15.83:0/1894346486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f526c0ff1f0 0x7f526c1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:02.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 -- 172.21.15.83:0/1894346486 >> 172.21.15.83:0/1894346486 conn(0x7f526c0fad70 msgr2=0x7f526c105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:02.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 -- 172.21.15.83:0/1894346486 shutdown_connections 2024-03-11T19:39:02.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:02.955+0000 7f527322e640 1 -- 172.21.15.83:0/1894346486 wait complete. 2024-03-11T19:39:02.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:39:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.068+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2037198800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6080ff830 msgr2=0x7fa6080ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.068+0000 7fa60cc4e640 1 --2- 172.21.15.83:0/2037198800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6080ff830 0x7fa6080ffc50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009900 tx=0x7fa5f802bc30 comp rx=0 tx=0).stop 2024-03-11T19:39:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.068+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2037198800 shutdown_connections 2024-03-11T19:39:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.068+0000 7fa60cc4e640 1 --2- 172.21.15.83:0/2037198800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6081013e0 0x7fa608107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.068+0000 7fa60cc4e640 1 --2- 172.21.15.83:0/2037198800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa608100a20 0x7fa608100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.068+0000 7fa60cc4e640 1 --2- 172.21.15.83:0/2037198800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6080ff830 0x7fa6080ffc50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.068+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2037198800 >> 172.21.15.83:0/2037198800 conn(0x7fa6080fada0 msgr2=0x7fa6080fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.069+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2037198800 shutdown_connections 2024-03-11T19:39:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.069+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2037198800 wait complete. 2024-03-11T19:39:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.069+0000 7fa60cc4e640 1 Processor -- start 2024-03-11T19:39:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.069+0000 7fa60cc4e640 1 -- start start 2024-03-11T19:39:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.069+0000 7fa60cc4e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6080ff830 0x7fa608195020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.069+0000 7fa60cc4e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa608100a20 0x7fa608195560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.069+0000 7fa60cc4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6081013e0 0x7fa60819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.069+0000 7fa60cc4e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60819cb10 con 0x7fa6081013e0 2024-03-11T19:39:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.069+0000 7fa60cc4e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60819cc80 con 0x7fa6080ff830 2024-03-11T19:39:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.069+0000 7fa60cc4e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60819ce40 con 0x7fa608100a20 2024-03-11T19:39:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.069+0000 7fa606d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6081013e0 0x7fa60819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa606d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6081013e0 0x7fa60819c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50634/0 (socket says 172.21.15.83:50634) 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa606d9d640 1 -- 172.21.15.83:0/2444103565 learned_addr learned my addr 172.21.15.83:0/2444103565 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa605d9b640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa608100a20 0x7fa608195560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa60659c640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6080ff830 0x7fa608195020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa606d9d640 1 -- 172.21.15.83:0/2444103565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa608100a20 msgr2=0x7fa608195560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa606d9d640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa608100a20 0x7fa608195560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa606d9d640 1 -- 172.21.15.83:0/2444103565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6080ff830 msgr2=0x7fa608195020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa606d9d640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6080ff830 0x7fa608195020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa606d9d640 1 -- 172.21.15.83:0/2444103565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8009560 con 0x7fa6081013e0 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa606d9d640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6081013e0 0x7fa60819c5d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa60000eaa0 tx=0x7fa60000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa5f77fe640 1 -- 172.21.15.83:0/2444103565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000cdb0 con 0x7fa6081013e0 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa5f77fe640 1 -- 172.21.15.83:0/2444103565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa600005020 con 0x7fa6081013e0 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2444103565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60819d120 con 0x7fa6081013e0 2024-03-11T19:39:03.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa5f77fe640 1 -- 172.21.15.83:0/2444103565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6000116b0 con 0x7fa6081013e0 2024-03-11T19:39:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.070+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2444103565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60819d630 con 0x7fa6081013e0 2024-03-11T19:39:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.071+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2444103565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa608106b50 con 0x7fa6081013e0 2024-03-11T19:39:03.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.072+0000 7fa5f77fe640 1 -- 172.21.15.83:0/2444103565 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa600005520 con 0x7fa6081013e0 2024-03-11T19:39:03.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.073+0000 7fa5f77fe640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5f00a1a30 0x7fa5f00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.073+0000 7fa60659c640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5f00a1a30 0x7fa5f00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.073+0000 7fa5f77fe640 1 -- 172.21.15.83:0/2444103565 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa6000cd4f0 con 0x7fa6081013e0 2024-03-11T19:39:03.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.073+0000 7fa60659c640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5f00a1a30 0x7fa5f00a3ef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005ab0 tx=0x7fa5f8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.077+0000 7fa5f77fe640 1 -- 172.21.15.83:0/2444103565 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa600096860 con 0x7fa6081013e0 2024-03-11T19:39:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.286+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2444103565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa60810a9c0 con 0x7fa6081013e0 2024-03-11T19:39:03.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.287+0000 7fa5f77fe640 1 -- 172.21.15.83:0/2444103565 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11349 (secure 0 0 0) 0x7fa600099f20 con 0x7fa6081013e0 2024-03-11T19:39:03.291 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:03.291 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:38:57.908337+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:39:03.291 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 22 2024-03-11T19:39:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2444103565 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5f00a1a30 msgr2=0x7fa5f00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5f00a1a30 0x7fa5f00a3ef0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005ab0 tx=0x7fa5f8005fd0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2444103565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6081013e0 msgr2=0x7fa60819c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6081013e0 0x7fa60819c5d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa60000eaa0 tx=0x7fa60000ef60 comp rx=0 tx=0).stop 2024-03-11T19:39:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2444103565 shutdown_connections 2024-03-11T19:39:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5f00a1a30 0x7fa5f00a3ef0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6081013e0 0x7fa60819c5d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa608100a20 0x7fa608195560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 --2- 172.21.15.83:0/2444103565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6080ff830 0x7fa608195020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2444103565 >> 172.21.15.83:0/2444103565 conn(0x7fa6080fada0 msgr2=0x7fa608103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2444103565 shutdown_connections 2024-03-11T19:39:03.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.289+0000 7fa60cc4e640 1 -- 172.21.15.83:0/2444103565 wait complete. 2024-03-11T19:39:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:02 smithi083 ceph-mon[31864]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:39:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:02 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2357602817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:39:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:02 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/259187806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:39:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:02 smithi083 ceph-mon[31864]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:39:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:02 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3954810583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:39:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:02 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1894346486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:39:03.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:39:03.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:02 smithi137 ceph-mon[36220]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:39:03.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:02 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2357602817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:39:03.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:02 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/259187806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:39:03.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:02 smithi137 ceph-mon[36220]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:39:03.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:02 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3954810583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:39:03.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:02 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1894346486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:39:03.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.401+0000 7f4312a1c640 1 -- 172.21.15.83:0/957869505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c104610 msgr2=0x7f430c106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.401+0000 7f4312a1c640 1 --2- 172.21.15.83:0/957869505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c104610 0x7f430c106a00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009900 tx=0x7f42fc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:39:03.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.401+0000 7f4312a1c640 1 -- 172.21.15.83:0/957869505 shutdown_connections 2024-03-11T19:39:03.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.401+0000 7f4312a1c640 1 --2- 172.21.15.83:0/957869505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c106f40 0x7f430c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.401+0000 7f4312a1c640 1 --2- 172.21.15.83:0/957869505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c104610 0x7f430c106a00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.401+0000 7f4312a1c640 1 --2- 172.21.15.83:0/957869505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f430c101ce0 0x7f430c1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.402+0000 7f4312a1c640 1 -- 172.21.15.83:0/957869505 >> 172.21.15.83:0/957869505 conn(0x7f430c0fb630 msgr2=0x7f430c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.402+0000 7f4312a1c640 1 -- 172.21.15.83:0/957869505 shutdown_connections 2024-03-11T19:39:03.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.402+0000 7f4312a1c640 1 -- 172.21.15.83:0/957869505 wait complete. 2024-03-11T19:39:03.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.402+0000 7f4312a1c640 1 Processor -- start 2024-03-11T19:39:03.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.402+0000 7f4312a1c640 1 -- start start 2024-03-11T19:39:03.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.402+0000 7f4312a1c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c101ce0 0x7f430c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.402+0000 7f4312a1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f430c104610 0x7f430c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.402+0000 7f4312a1c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c106f40 0x7f430c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.402+0000 7f4312a1c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c19ed60 con 0x7f430c101ce0 2024-03-11T19:39:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.402+0000 7f4312a1c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c19eed0 con 0x7f430c104610 2024-03-11T19:39:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.402+0000 7f4312a1c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c19f090 con 0x7f430c106f40 2024-03-11T19:39:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f430bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c101ce0 0x7f430c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f430bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c101ce0 0x7f430c197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50654/0 (socket says 172.21.15.83:50654) 2024-03-11T19:39:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f430bfff640 1 -- 172.21.15.83:0/886245159 learned_addr learned my addr 172.21.15.83:0/886245159 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f4310fb9640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c106f40 0x7f430c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f430b7fe640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f430c104610 0x7f430c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f430bfff640 1 -- 172.21.15.83:0/886245159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c106f40 msgr2=0x7f430c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f430bfff640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c106f40 0x7f430c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f430bfff640 1 -- 172.21.15.83:0/886245159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f430c104610 msgr2=0x7f430c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f430bfff640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f430c104610 0x7f430c1977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f430bfff640 1 -- 172.21.15.83:0/886245159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc009560 con 0x7f430c101ce0 2024-03-11T19:39:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f430bfff640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c101ce0 0x7f430c197270 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f42f400d950 tx=0x7f42f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f43097fa640 1 -- 172.21.15.83:0/886245159 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4009a30 con 0x7f430c101ce0 2024-03-11T19:39:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f43097fa640 1 -- 172.21.15.83:0/886245159 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42f4009b90 con 0x7f430c101ce0 2024-03-11T19:39:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f43097fa640 1 -- 172.21.15.83:0/886245159 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4004a00 con 0x7f430c101ce0 2024-03-11T19:39:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f4312a1c640 1 -- 172.21.15.83:0/886245159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430c19f370 con 0x7f430c101ce0 2024-03-11T19:39:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.403+0000 7f4312a1c640 1 -- 172.21.15.83:0/886245159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430c19f880 con 0x7f430c101ce0 2024-03-11T19:39:03.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.404+0000 7f4312a1c640 1 -- 172.21.15.83:0/886245159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430c0fd120 con 0x7f430c101ce0 2024-03-11T19:39:03.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:02 smithi204 ceph-mon[36739]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-11T19:39:03.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:02 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2357602817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:39:03.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:02 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/259187806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:39:03.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:02 smithi204 ceph-mon[36739]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-11T19:39:03.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:02 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3954810583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:39:03.412 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:02 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1894346486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:39:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.405+0000 7f43097fa640 1 -- 172.21.15.83:0/886245159 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f42f4009d00 con 0x7f430c101ce0 2024-03-11T19:39:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.406+0000 7f43097fa640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42f80a1a30 0x7f42f80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.406+0000 7f430b7fe640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42f80a1a30 0x7f42f80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.406+0000 7f43097fa640 1 -- 172.21.15.83:0/886245159 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f42f40cce00 con 0x7f430c101ce0 2024-03-11T19:39:03.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.406+0000 7f430b7fe640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42f80a1a30 0x7f42f80a3ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f42fc036040 tx=0x7f42fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.410+0000 7f43097fa640 1 -- 172.21.15.83:0/886245159 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42f40960c0 con 0x7f430c101ce0 2024-03-11T19:39:03.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.588+0000 7f4312a1c640 1 -- 172.21.15.83:0/886245159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f430c02ce40 con 0x7f430c101ce0 2024-03-11T19:39:03.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.590+0000 7f43097fa640 1 -- 172.21.15.83:0/886245159 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7f42f4099780 con 0x7f430c101ce0 2024-03-11T19:39:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:03.596 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:39:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 -- 172.21.15.83:0/886245159 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42f80a1a30 msgr2=0x7f42f80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42f80a1a30 0x7f42f80a3ef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f42fc036040 tx=0x7f42fc005fd0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 -- 172.21.15.83:0/886245159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c101ce0 msgr2=0x7f430c197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c101ce0 0x7f430c197270 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f42f400d950 tx=0x7f42f400de10 comp rx=0 tx=0).stop 2024-03-11T19:39:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 -- 172.21.15.83:0/886245159 shutdown_connections 2024-03-11T19:39:03.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f42f80a1a30 0x7f42f80a3ef0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f430c106f40 0x7f430c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f430c104610 0x7f430c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 --2- 172.21.15.83:0/886245159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f430c101ce0 0x7f430c197270 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 -- 172.21.15.83:0/886245159 >> 172.21.15.83:0/886245159 conn(0x7f430c0fb630 msgr2=0x7f430c1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 -- 172.21.15.83:0/886245159 shutdown_connections 2024-03-11T19:39:03.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.592+0000 7f4312a1c640 1 -- 172.21.15.83:0/886245159 wait complete. 2024-03-11T19:39:03.608 DEBUG:tasks.fwd_scrub:Yielding 2024-03-11T19:39:03.608 INFO:teuthology.run_tasks:Running task exec... 2024-03-11T19:39:03.617 INFO:teuthology.task.exec:Executing custom commands... 2024-03-11T19:39:03.617 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi083.front.sepia.ceph.com 2024-03-11T19:39:03.617 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-03-11T19:39:03.618 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-11T19:39:03.618 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:39:03.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:39:03.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:39:03.656 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:03.670 INFO:teuthology.orchestra.run.smithi083.stdout:Filesystem Size Used Avail Use% Mounted on 2024-03-11T19:39:03.670 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6789,172.21.15.137:6789,172.21.15.204:6789:/volumes/qa/sv_1/bddea3eb-992d-47ca-8e3a-c7ffc2fc4e6f 340G 0 340G 0% /tmp/cephfs 2024-03-11T19:39:03.673 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-03-11T19:39:03.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.721+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2773358624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4101270 msgr2=0x7f59b41016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.721+0000 7f59b9b5c640 1 --2- 172.21.15.83:0/2773358624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4101270 0x7f59b41016f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009900 tx=0x7f59a802bc30 comp rx=0 tx=0).stop 2024-03-11T19:39:03.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.721+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2773358624 shutdown_connections 2024-03-11T19:39:03.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.721+0000 7f59b9b5c640 1 --2- 172.21.15.83:0/2773358624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59b4101c30 0x7f59b41084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.721+0000 7f59b9b5c640 1 --2- 172.21.15.83:0/2773358624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4101270 0x7f59b41016f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.721+0000 7f59b9b5c640 1 --2- 172.21.15.83:0/2773358624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59b4100080 0x7f59b41004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.721+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2773358624 >> 172.21.15.83:0/2773358624 conn(0x7f59b40fb610 msgr2=0x7f59b40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.722+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2773358624 shutdown_connections 2024-03-11T19:39:03.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.722+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2773358624 wait complete. 2024-03-11T19:39:03.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.722+0000 7f59b9b5c640 1 Processor -- start 2024-03-11T19:39:03.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.722+0000 7f59b9b5c640 1 -- start start 2024-03-11T19:39:03.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.722+0000 7f59b9b5c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59b4100080 0x7f59b4197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.722+0000 7f59b9b5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59b4101270 0x7f59b4197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.722+0000 7f59b9b5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4101c30 0x7f59b419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.722+0000 7f59b9b5c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b419ee10 con 0x7f59b4101c30 2024-03-11T19:39:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.722+0000 7f59b9b5c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b419ef80 con 0x7f59b4101270 2024-03-11T19:39:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.722+0000 7f59b9b5c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b419f140 con 0x7f59b4100080 2024-03-11T19:39:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4101c30 0x7f59b419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4101c30 0x7f59b419e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50672/0 (socket says 172.21.15.83:50672) 2024-03-11T19:39:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b3fff640 1 -- 172.21.15.83:0/2825689444 learned_addr learned my addr 172.21.15.83:0/2825689444 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59b4101270 0x7f59b4197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b37fe640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59b4100080 0x7f59b4197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b3fff640 1 -- 172.21.15.83:0/2825689444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59b4100080 msgr2=0x7f59b4197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b3fff640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59b4100080 0x7f59b4197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b3fff640 1 -- 172.21.15.83:0/2825689444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59b4101270 msgr2=0x7f59b4197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b3fff640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59b4101270 0x7f59b4197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b3fff640 1 -- 172.21.15.83:0/2825689444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a8009560 con 0x7f59b4101c30 2024-03-11T19:39:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b37fe640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59b4100080 0x7f59b4197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:39:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b3fff640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4101c30 0x7f59b419e8d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00eaa0 tx=0x7f59ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b0ff9640 1 -- 172.21.15.83:0/2825689444 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00cdb0 con 0x7f59b4101c30 2024-03-11T19:39:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b0ff9640 1 -- 172.21.15.83:0/2825689444 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59ac005020 con 0x7f59b4101c30 2024-03-11T19:39:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2825689444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b419f420 con 0x7f59b4101c30 2024-03-11T19:39:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b0ff9640 1 -- 172.21.15.83:0/2825689444 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0116b0 con 0x7f59b4101c30 2024-03-11T19:39:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.723+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2825689444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b419f930 con 0x7f59b4101c30 2024-03-11T19:39:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.725+0000 7f59b0ff9640 1 -- 172.21.15.83:0/2825689444 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f59ac005520 con 0x7f59b4101c30 2024-03-11T19:39:03.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59bc805740 1 Processor -- start 2024-03-11T19:39:03.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59bc805740 1 -- start start 2024-03-11T19:39:03.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59bc805740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 0x555bf88e28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59bc805740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e3690 0x555bf88e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59bc805740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf88ea8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59bc805740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf8851f90 con 0x555bf88e24c0 2024-03-11T19:39:03.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59bc805740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf884f4c0 con 0x555bf88e3690 2024-03-11T19:39:03.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59bc805740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf884aca0 con 0x555bf88e4050 2024-03-11T19:39:03.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b0ff9640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f599c0a1a30 0x7f599c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 0x555bf88e28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e3690 0x555bf88e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 0x555bf88e28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50678/0 (socket says 172.21.15.83:50678) 2024-03-11T19:39:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b2ffd640 1 -- 172.21.15.83:0/3062969437 learned_addr learned my addr 172.21.15.83:0/3062969437 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b3fff640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f599c0a1a30 0x7f599c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/3062969437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf88ea8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b0ff9640 1 -- 172.21.15.83:0/2825689444 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f59ac0cd610 con 0x7f59b4101c30 2024-03-11T19:39:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b37fe640 1 -- 172.21.15.83:0/3062969437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 msgr2=0x555bf88ea8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b37fe640 1 --2- 172.21.15.83:0/3062969437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf88ea8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b37fe640 1 -- 172.21.15.83:0/3062969437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 msgr2=0x555bf88e28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b37fe640 1 --2- 172.21.15.83:0/3062969437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 0x555bf88e28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.726+0000 7f59b37fe640 1 -- 172.21.15.83:0/3062969437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bf880d0b0 con 0x555bf88e3690 2024-03-11T19:39:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.727+0000 7f59b3fff640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f599c0a1a30 0x7f599c0a3ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f59ac097000 tx=0x7f59ac0659c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.727+0000 7f59b37fe640 1 --2- 172.21.15.83:0/3062969437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e3690 0x555bf88e3b10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f59a400a910 tx=0x7f59a400add0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d472b9349df2d115 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.727+0000 7f5999ffb640 1 -- 172.21.15.83:0/3062969437 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400cc00 con 0x555bf88e3690 2024-03-11T19:39:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.727+0000 7f5999ffb640 1 -- 172.21.15.83:0/3062969437 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59a400cd60 con 0x555bf88e3690 2024-03-11T19:39:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.727+0000 7f5999ffb640 1 -- 172.21.15.83:0/3062969437 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40186e0 con 0x555bf88e3690 2024-03-11T19:39:03.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.728+0000 7f59bc805740 1 -- 172.21.15.83:0/3062969437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e3690 msgr2=0x555bf88e3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.728+0000 7f59bc805740 1 --2- 172.21.15.83:0/3062969437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e3690 0x555bf88e3b10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f59a400a910 tx=0x7f59a400add0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.728+0000 7f59bc805740 1 -- 172.21.15.83:0/3062969437 shutdown_connections 2024-03-11T19:39:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.728+0000 7f59bc805740 1 --2- 172.21.15.83:0/3062969437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf88ea8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.728+0000 7f59bc805740 1 --2- 172.21.15.83:0/3062969437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e3690 0x555bf88e3b10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.728+0000 7f59bc805740 1 --2- 172.21.15.83:0/3062969437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 0x555bf88e28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.728+0000 7f59bc805740 1 -- 172.21.15.83:0/3062969437 >> 172.21.15.83:0/3062969437 conn(0x555bf88dd6a0 msgr2=0x555bf88ddaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.728+0000 7f59bc805740 1 -- 172.21.15.83:0/3062969437 shutdown_connections 2024-03-11T19:39:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.728+0000 7f59bc805740 1 -- 172.21.15.83:0/3062969437 wait complete. 2024-03-11T19:39:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.728+0000 7f59bc805740 1 -- start start 2024-03-11T19:39:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.728+0000 7f59bc805740 1 Processor -- start 2024-03-11T19:39:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59bc805740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 0x555bf88ec710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59bc805740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e3690 0x555bf88ecc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59bc805740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e4050 0x555bf88f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59bc805740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf87eff40 con 0x555bf88e24c0 2024-03-11T19:39:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59bc805740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf851e4b0 con 0x555bf88e4050 2024-03-11T19:39:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59bc805740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf87d0670 con 0x555bf88e3690 2024-03-11T19:39:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59bc805740 10 client.0 _ll_register_callbacks cb 0x555bf883df70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59b2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 0x555bf88ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59b2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 0x555bf88ec710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50690/0 (socket says 172.21.15.83:50690) 2024-03-11T19:39:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59b2ffd640 1 -- 172.21.15.83:0/13456108 learned_addr learned my addr 172.21.15.83:0/13456108 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59b37fe640 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e3690 0x555bf88ecc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e4050 0x555bf88f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59b2ffd640 1 -- 172.21.15.83:0/13456108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e3690 msgr2=0x555bf88ecc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e3690 0x555bf88ecc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59b2ffd640 1 -- 172.21.15.83:0/13456108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e4050 msgr2=0x555bf88f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e4050 0x555bf88f3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59b2ffd640 1 -- 172.21.15.83:0/13456108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a400a570 con 0x555bf88e24c0 2024-03-11T19:39:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 0x555bf88ec710 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f59a8036040 tx=0x7f59a8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e8b12df1f39df server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f5992ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:39:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f5992ffd640 1 -- 172.21.15.83:0/13456108 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8004480 con 0x555bf88e24c0 2024-03-11T19:39:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f5992ffd640 1 -- 172.21.15.83:0/13456108 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f59a8004df0 con 0x555bf88e24c0 2024-03-11T19:39:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59bc805740 1 -- 172.21.15.83:0/13456108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bf8824f40 con 0x555bf88e24c0 2024-03-11T19:39:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f5992ffd640 1 -- 172.21.15.83:0/13456108 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59a8004fd0 con 0x555bf88e24c0 2024-03-11T19:39:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59bc805740 10 client.15348 fetch_fsmap learned FSMap version 22 2024-03-11T19:39:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.729+0000 7f59bc805740 1 -- 172.21.15.83:0/13456108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555bf8849320 con 0x555bf88e24c0 2024-03-11T19:39:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.730+0000 7f5992ffd640 1 -- 172.21.15.83:0/13456108 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555bf8824f40 con 0x555bf88e24c0 2024-03-11T19:39:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.730+0000 7f5992ffd640 1 -- 172.21.15.83:0/13456108 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59a803c070 con 0x555bf88e24c0 2024-03-11T19:39:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.730+0000 7f59bc805740 10 client.15348 fetch_fsmap finished waiting for FSMap version 22 2024-03-11T19:39:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.730+0000 7f59bc805740 10 client.15348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.730+0000 7f59bc805740 20 client.15348 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.730+0000 7f59bc805740 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555bf88f87e0 0x555bf88fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.730+0000 7f59bc805740 4 client.15348 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:39:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.730+0000 7f59bc805740 1 -- 172.21.15.83:0/13456108 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bf8841310 con 0x555bf88f87e0 2024-03-11T19:39:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.730+0000 7f59b37fe640 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555bf88f87e0 0x555bf88fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f5992ffd640 10 client.15348 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f59b37fe640 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555bf88f87e0 0x555bf88fabf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3924ed5c6bfbcb server_cookie=3f0c27f6f1f7b65e in_seq=0 out_seq=0 2024-03-11T19:39:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f5992ffd640 1 -- 172.21.15.83:0/13456108 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555bf8841310 con 0x555bf88f87e0 2024-03-11T19:39:03.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f5992ffd640 10 client.15348 handle_command_reply: tid=0 2024-03-11T19:39:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f59bc805740 1 client.15348 shutdown 2024-03-11T19:39:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f59bc805740 1 -- 172.21.15.83:0/13456108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 msgr2=0x555bf88ec710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f59bc805740 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 0x555bf88ec710 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f59a8036040 tx=0x7f59a8005fd0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f59bc805740 1 -- 172.21.15.83:0/13456108 shutdown_connections 2024-03-11T19:39:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f59bc805740 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555bf88f87e0 0x555bf88fabf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f59bc805740 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e4050 0x555bf88f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f59bc805740 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e3690 0x555bf88ecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f59bc805740 1 --2- 172.21.15.83:0/13456108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e24c0 0x555bf88ec710 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f5992ffd640 0 client.15348 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.731+0000 7f59bc805740 1 -- 172.21.15.83:0/13456108 >> 172.21.15.83:0/13456108 conn(0x555bf88dd6a0 msgr2=0x555bf88dc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.732+0000 7f59bc805740 1 -- 172.21.15.83:0/13456108 shutdown_connections 2024-03-11T19:39:03.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.732+0000 7f59bc805740 1 -- 172.21.15.83:0/13456108 wait complete. 2024-03-11T19:39:03.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.732+0000 7f59bc805740 20 client.15348 trim_cache size 0 max 16384 2024-03-11T19:39:03.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59bc805740 1 Processor -- start 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59bc805740 1 -- start start 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59bc805740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e24c0 0x555bf88ec320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59bc805740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59bc805740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf88ec860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59bc805740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf88d2350 con 0x555bf88e3690 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59bc805740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf88e1f00 con 0x555bf88e24c0 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59bc805740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf8841310 con 0x555bf88e4050 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50692/0 (socket says 172.21.15.83:50692) 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b37fe640 1 -- 172.21.15.83:0/2716089958 learned_addr learned my addr 172.21.15.83:0/2716089958 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/2716089958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e24c0 0x555bf88ec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/2716089958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf88ec860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b37fe640 1 -- 172.21.15.83:0/2716089958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 msgr2=0x555bf88ec860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b37fe640 1 --2- 172.21.15.83:0/2716089958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf88ec860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b37fe640 1 -- 172.21.15.83:0/2716089958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e24c0 msgr2=0x555bf88ec320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b37fe640 1 --2- 172.21.15.83:0/2716089958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e24c0 0x555bf88ec320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b37fe640 1 -- 172.21.15.83:0/2716089958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bf87e5c00 con 0x555bf88e3690 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/2716089958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf88ec860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59b37fe640 1 --2- 172.21.15.83:0/2716089958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f3be0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x555bf88de610 tx=0x7f59a401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b454def78f5076 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.747+0000 7f59997fa640 1 -- 172.21.15.83:0/2716089958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400a200 con 0x555bf88e3690 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.748+0000 7f59997fa640 1 -- 172.21.15.83:0/2716089958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59a4011e30 con 0x555bf88e3690 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.748+0000 7f59997fa640 1 -- 172.21.15.83:0/2716089958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4028470 con 0x555bf88e3690 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.748+0000 7f59bc805740 1 -- 172.21.15.83:0/2716089958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 msgr2=0x555bf88f3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.748+0000 7f59bc805740 1 --2- 172.21.15.83:0/2716089958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f3be0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x555bf88de610 tx=0x7f59a401c040 comp rx=0 tx=0).stop 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.748+0000 7f59bc805740 1 -- 172.21.15.83:0/2716089958 shutdown_connections 2024-03-11T19:39:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.748+0000 7f59bc805740 1 --2- 172.21.15.83:0/2716089958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf88ec860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.748+0000 7f59bc805740 1 --2- 172.21.15.83:0/2716089958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f3be0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.748+0000 7f59bc805740 1 --2- 172.21.15.83:0/2716089958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e24c0 0x555bf88ec320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.748+0000 7f59bc805740 1 -- 172.21.15.83:0/2716089958 >> 172.21.15.83:0/2716089958 conn(0x555bf88dd6a0 msgr2=0x555bf88e72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.748+0000 7f59bc805740 1 -- 172.21.15.83:0/2716089958 shutdown_connections 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.748+0000 7f59bc805740 1 -- 172.21.15.83:0/2716089958 wait complete. 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59bc805740 1 -- start start 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59bc805740 1 Processor -- start 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59bc805740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e24c0 0x555bf88f0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59bc805740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59bc805740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf894ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59bc805740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf88f3d40 con 0x555bf88e3690 2024-03-11T19:39:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59bc805740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf88f3eb0 con 0x555bf88e24c0 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59bc805740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf88f4070 con 0x555bf88e4050 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59bc805740 10 client.0 _ll_register_callbacks cb 0x7f59a400a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59b37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59b37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50696/0 (socket says 172.21.15.83:50696) 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59b37fe640 1 -- 172.21.15.83:0/2725652913 learned_addr learned my addr 172.21.15.83:0/2725652913 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e24c0 0x555bf88f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59b37fe640 1 -- 172.21.15.83:0/2725652913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 msgr2=0x555bf894ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59b37fe640 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf894ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59b37fe640 1 -- 172.21.15.83:0/2725652913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e24c0 msgr2=0x555bf88f0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59b37fe640 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e24c0 0x555bf88f0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.749+0000 7f59b37fe640 1 -- 172.21.15.83:0/2725652913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bf880d0b0 con 0x555bf88e3690 2024-03-11T19:39:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f59b37fe640 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f0dc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f59a400ca80 tx=0x7f59a400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e29427e751adc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f5992ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f5992ffd640 1 -- 172.21.15.83:0/2725652913 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4011800 con 0x555bf88e3690 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f5992ffd640 1 -- 172.21.15.83:0/2725652913 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f59a401f040 con 0x555bf88e3690 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f59bc805740 1 -- 172.21.15.83:0/2725652913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bf8870900 con 0x555bf88e3690 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f5992ffd640 1 -- 172.21.15.83:0/2725652913 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59a400ced0 con 0x555bf88e3690 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f59bc805740 10 client.15354 fetch_fsmap learned FSMap version 22 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f59bc805740 1 -- 172.21.15.83:0/2725652913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555bf880b910 con 0x555bf88e3690 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f5992ffd640 1 -- 172.21.15.83:0/2725652913 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555bf8870900 con 0x555bf88e3690 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f5992ffd640 1 -- 172.21.15.83:0/2725652913 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59a40348c0 con 0x555bf88e3690 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f59bc805740 10 client.15354 fetch_fsmap finished waiting for FSMap version 22 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f59bc805740 10 client.15354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f59bc805740 20 client.15354 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f59bc805740 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555bf89508c0 0x555bf8952ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f59bc805740 4 client.15354 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f59bc805740 1 -- 172.21.15.83:0/2725652913 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555bf854a860 con 0x555bf89508c0 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.750+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555bf89508c0 0x555bf8952ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.751+0000 7f59b2ffd640 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555bf89508c0 0x555bf8952ce0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c10d5b67a678a0 server_cookie=5e1417b3f9f15d21 in_seq=0 out_seq=0 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.751+0000 7f5992ffd640 10 client.15354 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.752+0000 7f5992ffd640 1 -- 172.21.15.83:0/2725652913 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555bf854a860 con 0x555bf89508c0 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.752+0000 7f5992ffd640 10 client.15354 handle_command_reply: tid=0 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.752+0000 7f59bc805740 1 client.15354 shutdown 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59bc805740 1 -- 172.21.15.83:0/2725652913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 msgr2=0x555bf88f0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59bc805740 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f0dc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f59a400ca80 tx=0x7f59a400cab0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59bc805740 1 -- 172.21.15.83:0/2725652913 shutdown_connections 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59bc805740 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555bf89508c0 0x555bf8952ce0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59bc805740 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555bf88e4050 0x555bf894ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59bc805740 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bf88e3690 0x555bf88f0dc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59bc805740 1 --2- 172.21.15.83:0/2725652913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555bf88e24c0 0x555bf88f0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59bc805740 1 -- 172.21.15.83:0/2725652913 >> 172.21.15.83:0/2725652913 conn(0x555bf88dd6a0 msgr2=0x555bf88ec320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f5992ffd640 0 client.15354 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:03.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59bc805740 1 -- 172.21.15.83:0/2725652913 shutdown_connections 2024-03-11T19:39:03.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59bc805740 1 -- 172.21.15.83:0/2725652913 wait complete. 2024-03-11T19:39:03.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59bc805740 20 client.15354 trim_cache size 0 max 16384 2024-03-11T19:39:03.774 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:39:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:39:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "0878a811-bdc1-47f4-a37a-dcdbc9e0cea5", 2024-03-11T19:39:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:39:03.775 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:39:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2825689444 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f599c0a1a30 msgr2=0x7f599c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59b9b5c640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f599c0a1a30 0x7f599c0a3ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f59ac097000 tx=0x7f59ac0659c0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2825689444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4101c30 msgr2=0x7f59b419e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59b9b5c640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4101c30 0x7f59b419e8d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00eaa0 tx=0x7f59ac00ef60 comp rx=0 tx=0).stop 2024-03-11T19:39:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2825689444 shutdown_connections 2024-03-11T19:39:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59b9b5c640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f599c0a1a30 0x7f599c0a3ef0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59b9b5c640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59b4101c30 0x7f59b419e8d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59b9b5c640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59b4101270 0x7f59b4197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59b9b5c640 1 --2- 172.21.15.83:0/2825689444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59b4100080 0x7f59b4197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.753+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2825689444 >> 172.21.15.83:0/2825689444 conn(0x7f59b40fb610 msgr2=0x7f59b40fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.754+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2825689444 shutdown_connections 2024-03-11T19:39:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.754+0000 7f59b9b5c640 1 -- 172.21.15.83:0/2825689444 wait complete. 2024-03-11T19:39:03.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:39:03.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:39:03.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-03-11T19:39:03.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-03-11T19:39:03.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.919+0000 7f0c0018c640 1 -- 172.21.15.83:0/1926810584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf8100080 msgr2=0x7f0bf81004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.919+0000 7f0c0018c640 1 --2- 172.21.15.83:0/1926810584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf8100080 0x7f0bf81004a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009900 tx=0x7f0bf402bc30 comp rx=0 tx=0).stop 2024-03-11T19:39:03.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.919+0000 7f0c0018c640 1 -- 172.21.15.83:0/1926810584 shutdown_connections 2024-03-11T19:39:03.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.919+0000 7f0c0018c640 1 --2- 172.21.15.83:0/1926810584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf8101c30 0x7f0bf81084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.919+0000 7f0c0018c640 1 --2- 172.21.15.83:0/1926810584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8101270 0x7f0bf81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.919+0000 7f0c0018c640 1 --2- 172.21.15.83:0/1926810584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf8100080 0x7f0bf81004a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.919+0000 7f0c0018c640 1 -- 172.21.15.83:0/1926810584 >> 172.21.15.83:0/1926810584 conn(0x7f0bf80fb610 msgr2=0x7f0bf80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.919+0000 7f0c0018c640 1 -- 172.21.15.83:0/1926810584 shutdown_connections 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.919+0000 7f0c0018c640 1 -- 172.21.15.83:0/1926810584 wait complete. 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0c0018c640 1 Processor -- start 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0c0018c640 1 -- start start 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0c0018c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8100080 0x7f0bf806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0c0018c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf8101270 0x7f0bf819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0c0018c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf8101c30 0x7f0bf819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0c0018c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf819ed80 con 0x7f0bf8101c30 2024-03-11T19:39:03.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0c0018c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf819eef0 con 0x7f0bf8100080 2024-03-11T19:39:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0c0018c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf819f0b0 con 0x7f0bf8101270 2024-03-11T19:39:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0bfe729640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf8101c30 0x7f0bf819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0bfe729640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf8101c30 0x7f0bf819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50718/0 (socket says 172.21.15.83:50718) 2024-03-11T19:39:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0bfe729640 1 -- 172.21.15.83:0/3861262160 learned_addr learned my addr 172.21.15.83:0/3861262160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8100080 0x7f0bf806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0bfd727640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf8101270 0x7f0bf819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0bfe729640 1 -- 172.21.15.83:0/3861262160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf8101270 msgr2=0x7f0bf819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0bfe729640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf8101270 0x7f0bf819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.920+0000 7f0bfe729640 1 -- 172.21.15.83:0/3861262160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8100080 msgr2=0x7f0bf806a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.921+0000 7f0bfe729640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8100080 0x7f0bf806a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.921+0000 7f0bfe729640 1 -- 172.21.15.83:0/3861262160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf4009560 con 0x7f0bf8101c30 2024-03-11T19:39:03.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.921+0000 7f0bfe729640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf8101c30 0x7f0bf819e5a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000cad0 tx=0x7f0bf000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.921+0000 7f0beeffd640 1 -- 172.21.15.83:0/3861262160 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000ae20 con 0x7f0bf8101c30 2024-03-11T19:39:03.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.921+0000 7f0beeffd640 1 -- 172.21.15.83:0/3861262160 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bf0005020 con 0x7f0bf8101c30 2024-03-11T19:39:03.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.921+0000 7f0beeffd640 1 -- 172.21.15.83:0/3861262160 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000f6b0 con 0x7f0bf8101c30 2024-03-11T19:39:03.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.921+0000 7f0c0018c640 1 -- 172.21.15.83:0/3861262160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf819f330 con 0x7f0bf8101c30 2024-03-11T19:39:03.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.921+0000 7f0c0018c640 1 -- 172.21.15.83:0/3861262160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf819f840 con 0x7f0bf8101c30 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.923+0000 7f0beeffd640 1 -- 172.21.15.83:0/3861262160 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0bf0005520 con 0x7f0bf8101c30 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.923+0000 7f0c02d63740 1 Processor -- start 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0c02d63740 1 -- start start 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d841a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841acc0 0x5599d841f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841fe30 0x5599d84222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0c02d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d838a280 con 0x5599d841a2d0 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0c02d63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d83877b0 con 0x5599d841fe30 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0c02d63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d8382f90 con 0x5599d841acc0 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0beeffd640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0be40a1a30 0x7f0be40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfd727640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d841a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfd727640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d841a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50732/0 (socket says 172.21.15.83:50732) 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfd727640 1 -- 172.21.15.83:0/582877491 learned_addr learned my addr 172.21.15.83:0/582877491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfe729640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0be40a1a30 0x7f0be40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfd727640 1 --2- 172.21.15.83:0/582877491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841fe30 0x5599d84222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/582877491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841acc0 0x5599d841f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0beeffd640 1 -- 172.21.15.83:0/3861262160 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0bf00cd630 con 0x7f0bf8101c30 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfd727640 1 -- 172.21.15.83:0/582877491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841acc0 msgr2=0x5599d841f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfd727640 1 --2- 172.21.15.83:0/582877491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841acc0 0x5599d841f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfd727640 1 -- 172.21.15.83:0/582877491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841fe30 msgr2=0x5599d84222c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfd727640 1 --2- 172.21.15.83:0/582877491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841fe30 0x5599d84222c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfd727640 1 -- 172.21.15.83:0/582877491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599d8341570 con 0x5599d841a2d0 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.924+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/582877491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841acc0 0x5599d841f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.925+0000 7f0bfd727640 1 --2- 172.21.15.83:0/582877491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d841a6f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0be8013580 tx=0x7f0be800a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cfbc6ac978a1d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.925+0000 7f0bfe729640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0be40a1a30 0x7f0be40a3ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0096b60 tx=0x7f0bf00cb040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.925+0000 7f0be3fff640 1 -- 172.21.15.83:0/582877491 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8005020 con 0x5599d841a2d0 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.925+0000 7f0be3fff640 1 -- 172.21.15.83:0/582877491 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0be8005660 con 0x5599d841a2d0 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.925+0000 7f0be3fff640 1 -- 172.21.15.83:0/582877491 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be800f8a0 con 0x5599d841a2d0 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 -- 172.21.15.83:0/582877491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 msgr2=0x5599d841a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 --2- 172.21.15.83:0/582877491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d841a6f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0be8013580 tx=0x7f0be800a670 comp rx=0 tx=0).stop 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 -- 172.21.15.83:0/582877491 shutdown_connections 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 --2- 172.21.15.83:0/582877491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841fe30 0x5599d84222c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 --2- 172.21.15.83:0/582877491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841acc0 0x5599d841f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 --2- 172.21.15.83:0/582877491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d841a6f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 -- 172.21.15.83:0/582877491 >> 172.21.15.83:0/582877491 conn(0x5599d84157f0 msgr2=0x5599d8415bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 -- 172.21.15.83:0/582877491 shutdown_connections 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 -- 172.21.15.83:0/582877491 wait complete. 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 -- start start 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 Processor -- start 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841a2d0 0x5599d84242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.926+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841fe30 0x5599d842b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0c02d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d8056430 con 0x5599d841acc0 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0c02d63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d8012e00 con 0x5599d841a2d0 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0c02d63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d8307fd0 con 0x5599d841fe30 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0c02d63740 10 client.0 _ll_register_callbacks cb 0x5599d835dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfdf28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfdf28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50738/0 (socket says 172.21.15.83:50738) 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfdf28640 1 -- 172.21.15.83:0/162412123 learned_addr learned my addr 172.21.15.83:0/162412123 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfd727640 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841fe30 0x5599d842b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfd727640 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841a2d0 0x5599d84242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfdf28640 1 -- 172.21.15.83:0/162412123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841fe30 msgr2=0x5599d842b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841fe30 0x5599d842b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfdf28640 1 -- 172.21.15.83:0/162412123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841a2d0 msgr2=0x5599d84242d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841a2d0 0x5599d84242d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfdf28640 1 -- 172.21.15.83:0/162412123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be800ad90 con 0x5599d841acc0 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfd727640 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841a2d0 0x5599d84242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0bf81010d0 tx=0x7f0bf40049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef3cce8f4584e01 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0be17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0be17fa640 1 -- 172.21.15.83:0/162412123 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4004250 con 0x5599d841acc0 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0be17fa640 1 -- 172.21.15.83:0/162412123 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0bf40043b0 con 0x5599d841acc0 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0c02d63740 1 -- 172.21.15.83:0/162412123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599d80827e0 con 0x5599d841acc0 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.927+0000 7f0be17fa640 1 -- 172.21.15.83:0/162412123 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf4004df0 con 0x5599d841acc0 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.928+0000 7f0c02d63740 10 client.15366 fetch_fsmap learned FSMap version 22 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.928+0000 7f0c02d63740 1 -- 172.21.15.83:0/162412123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5599d8381610 con 0x5599d841acc0 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.928+0000 7f0be17fa640 1 -- 172.21.15.83:0/162412123 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5599d80827e0 con 0x5599d841acc0 2024-03-11T19:39:03.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.928+0000 7f0be17fa640 1 -- 172.21.15.83:0/162412123 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0bf4052070 con 0x5599d841acc0 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.928+0000 7f0c02d63740 10 client.15366 fetch_fsmap finished waiting for FSMap version 22 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.928+0000 7f0c02d63740 10 client.15366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.928+0000 7f0c02d63740 20 client.15366 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.928+0000 7f0c02d63740 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d84304b0 0x5599d84328d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.928+0000 7f0c02d63740 4 client.15366 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.928+0000 7f0c02d63740 1 -- 172.21.15.83:0/162412123 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599d80bf720 con 0x5599d84304b0 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.928+0000 7f0bfd727640 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d84304b0 0x5599d84328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0bfd727640 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d84304b0 0x5599d84328d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc1ad6e75bd76ae server_cookie=b92672cdb1df64a8 in_seq=0 out_seq=0 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0be17fa640 10 client.15366 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0be17fa640 1 -- 172.21.15.83:0/162412123 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f0be802ba90 con 0x5599d84304b0 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0be17fa640 10 client.15366 handle_command_reply: tid=0 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0c02d63740 1 client.15366 shutdown 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0c02d63740 1 -- 172.21.15.83:0/162412123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 msgr2=0x5599d8424810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0c02d63740 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0bf81010d0 tx=0x7f0bf40049c0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0c02d63740 1 -- 172.21.15.83:0/162412123 shutdown_connections 2024-03-11T19:39:03.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0c02d63740 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d84304b0 0x5599d84328d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0c02d63740 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841fe30 0x5599d842b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0c02d63740 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0c02d63740 1 --2- 172.21.15.83:0/162412123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841a2d0 0x5599d84242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0be17fa640 0 client.15366 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:03.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.929+0000 7f0c02d63740 1 -- 172.21.15.83:0/162412123 >> 172.21.15.83:0/162412123 conn(0x5599d84157f0 msgr2=0x5599d8414a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.930+0000 7f0c02d63740 1 -- 172.21.15.83:0/162412123 shutdown_connections 2024-03-11T19:39:03.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.930+0000 7f0c02d63740 1 -- 172.21.15.83:0/162412123 wait complete. 2024-03-11T19:39:03.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.930+0000 7f0c02d63740 20 client.15366 trim_cache size 0 max 16384 2024-03-11T19:39:03.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0c02d63740 1 Processor -- start 2024-03-11T19:39:03.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0c02d63740 1 -- start start 2024-03-11T19:39:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841a2d0 0x5599d84242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841fe30 0x5599d842b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0c02d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d84110d0 con 0x5599d841acc0 2024-03-11T19:39:03.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0c02d63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d84167e0 con 0x5599d841fe30 2024-03-11T19:39:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0c02d63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d841f270 con 0x5599d841a2d0 2024-03-11T19:39:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0bfdf28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0bfdf28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50746/0 (socket says 172.21.15.83:50746) 2024-03-11T19:39:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0bfdf28640 1 -- 172.21.15.83:0/3779338937 learned_addr learned my addr 172.21.15.83:0/3779338937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0bfdf28640 1 -- 172.21.15.83:0/3779338937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841a2d0 msgr2=0x5599d84242d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/3779338937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841a2d0 0x5599d84242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0bfdf28640 1 -- 172.21.15.83:0/3779338937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841fe30 msgr2=0x5599d842b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/3779338937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841fe30 0x5599d842b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0bfdf28640 1 -- 172.21.15.83:0/3779338937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599d831dac0 con 0x5599d841acc0 2024-03-11T19:39:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/3779338937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0bf402bc70 tx=0x7f0bf4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6652f56e07baa5f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0be37fe640 1 -- 172.21.15.83:0/3779338937 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4005d50 con 0x5599d841acc0 2024-03-11T19:39:03.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0be37fe640 1 -- 172.21.15.83:0/3779338937 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bf404f070 con 0x5599d841acc0 2024-03-11T19:39:03.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.946+0000 7f0be37fe640 1 -- 172.21.15.83:0/3779338937 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40349a0 con 0x5599d841acc0 2024-03-11T19:39:03.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.947+0000 7f0c02d63740 1 -- 172.21.15.83:0/3779338937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 msgr2=0x5599d8424810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.947+0000 7f0c02d63740 1 --2- 172.21.15.83:0/3779338937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0bf402bc70 tx=0x7f0bf4009830 comp rx=0 tx=0).stop 2024-03-11T19:39:03.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.947+0000 7f0c02d63740 1 -- 172.21.15.83:0/3779338937 shutdown_connections 2024-03-11T19:39:03.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.947+0000 7f0c02d63740 1 --2- 172.21.15.83:0/3779338937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841fe30 0x5599d842b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.947+0000 7f0c02d63740 1 --2- 172.21.15.83:0/3779338937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841acc0 0x5599d8424810 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.947+0000 7f0c02d63740 1 --2- 172.21.15.83:0/3779338937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841a2d0 0x5599d84242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.947+0000 7f0c02d63740 1 -- 172.21.15.83:0/3779338937 >> 172.21.15.83:0/3779338937 conn(0x5599d84157f0 msgr2=0x5599d8437270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.947+0000 7f0c02d63740 1 -- 172.21.15.83:0/3779338937 shutdown_connections 2024-03-11T19:39:03.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.947+0000 7f0c02d63740 1 -- 172.21.15.83:0/3779338937 wait complete. 2024-03-11T19:39:03.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.947+0000 7f0c02d63740 1 -- start start 2024-03-11T19:39:03.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0c02d63740 1 Processor -- start 2024-03-11T19:39:03.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d842a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841acc0 0x5599d842a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0c02d63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841fe30 0x5599d842aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0c02d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d842bbd0 con 0x5599d841a2d0 2024-03-11T19:39:03.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0c02d63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d84869e0 con 0x5599d841acc0 2024-03-11T19:39:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0c02d63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d8486ba0 con 0x5599d841fe30 2024-03-11T19:39:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0c02d63740 10 client.0 _ll_register_callbacks cb 0x7f0be800a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0bfd727640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d842a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0bfd727640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d842a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50762/0 (socket says 172.21.15.83:50762) 2024-03-11T19:39:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0bfd727640 1 -- 172.21.15.83:0/1952714220 learned_addr learned my addr 172.21.15.83:0/1952714220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:03.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841acc0 0x5599d842a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0bfd727640 1 -- 172.21.15.83:0/1952714220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841fe30 msgr2=0x5599d842aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0bfd727640 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841fe30 0x5599d842aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0bfd727640 1 -- 172.21.15.83:0/1952714220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841acc0 msgr2=0x5599d842a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0bfd727640 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841acc0 0x5599d842a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0bfd727640 1 -- 172.21.15.83:0/1952714220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599d8381610 con 0x5599d841a2d0 2024-03-11T19:39:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841acc0 0x5599d842a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:39:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0bfd727640 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d842a060 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0be8013580 tx=0x7f0be800a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3530d43ddcb9cdc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.948+0000 7f0be17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:39:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0be17fa640 1 -- 172.21.15.83:0/1952714220 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8025070 con 0x5599d841a2d0 2024-03-11T19:39:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0be17fa640 1 -- 172.21.15.83:0/1952714220 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0be8013260 con 0x5599d841a2d0 2024-03-11T19:39:03.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0c02d63740 1 -- 172.21.15.83:0/1952714220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0be8005660 con 0x5599d841a2d0 2024-03-11T19:39:03.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0be17fa640 1 -- 172.21.15.83:0/1952714220 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0be8023030 con 0x5599d841a2d0 2024-03-11T19:39:03.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0c02d63740 10 client.15372 fetch_fsmap learned FSMap version 22 2024-03-11T19:39:03.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0c02d63740 1 -- 172.21.15.83:0/1952714220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5599d83436a0 con 0x5599d841a2d0 2024-03-11T19:39:03.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0be17fa640 1 -- 172.21.15.83:0/1952714220 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0be8005660 con 0x5599d841a2d0 2024-03-11T19:39:03.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0be17fa640 1 -- 172.21.15.83:0/1952714220 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0be801e860 con 0x5599d841a2d0 2024-03-11T19:39:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0c02d63740 10 client.15372 fetch_fsmap finished waiting for FSMap version 22 2024-03-11T19:39:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0c02d63740 10 client.15372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0c02d63740 20 client.15372 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0c02d63740 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d84886f0 0x5599d848ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0c02d63740 4 client.15372 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:39:03.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0c02d63740 1 -- 172.21.15.83:0/1952714220 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599d810e890 con 0x5599d84886f0 2024-03-11T19:39:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.949+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d84886f0 0x5599d848ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0be17fa640 10 client.15372 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0bfdf28640 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d84886f0 0x5599d848ab10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f021a926a370415 server_cookie=2670c3cc50caa7b2 in_seq=0 out_seq=0 2024-03-11T19:39:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0be17fa640 1 -- 172.21.15.83:0/1952714220 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5599d810e890 con 0x5599d84886f0 2024-03-11T19:39:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0be17fa640 10 client.15372 handle_command_reply: tid=0 2024-03-11T19:39:03.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 1 client.15372 shutdown 2024-03-11T19:39:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 1 -- 172.21.15.83:0/1952714220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 msgr2=0x5599d842a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d842a060 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0be8013580 tx=0x7f0be800a5b0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 1 -- 172.21.15.83:0/1952714220 shutdown_connections 2024-03-11T19:39:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d84886f0 0x5599d848ab10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d841fe30 0x5599d842aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d841acc0 0x5599d842a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0be17fa640 0 client.15372 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:03.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 1 --2- 172.21.15.83:0/1952714220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d841a2d0 0x5599d842a060 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 1 -- 172.21.15.83:0/1952714220 >> 172.21.15.83:0/1952714220 conn(0x5599d84157f0 msgr2=0x5599d8424230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 1 -- 172.21.15.83:0/1952714220 shutdown_connections 2024-03-11T19:39:03.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 1 -- 172.21.15.83:0/1952714220 wait complete. 2024-03-11T19:39:03.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.950+0000 7f0c02d63740 20 client.15372 trim_cache size 0 max 16384 2024-03-11T19:39:03.976 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:39:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:39:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:39:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "0878a811-bdc1-47f4-a37a-dcdbc9e0cea5": { 2024-03-11T19:39:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:39:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "0878a811-bdc1-47f4-a37a-dcdbc9e0cea5", 2024-03-11T19:39:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:39:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:39:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:39:03.977 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:39:03.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 -- 172.21.15.83:0/3861262160 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0be40a1a30 msgr2=0x7f0be40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0be40a1a30 0x7f0be40a3ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0096b60 tx=0x7f0bf00cb040 comp rx=0 tx=0).stop 2024-03-11T19:39:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 -- 172.21.15.83:0/3861262160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf8101c30 msgr2=0x7f0bf819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf8101c30 0x7f0bf819e5a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000cad0 tx=0x7f0bf000cf90 comp rx=0 tx=0).stop 2024-03-11T19:39:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 -- 172.21.15.83:0/3861262160 shutdown_connections 2024-03-11T19:39:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0be40a1a30 0x7f0be40a3ef0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bf8101c30 0x7f0bf819e5a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bf8101270 0x7f0bf819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 --2- 172.21.15.83:0/3861262160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf8100080 0x7f0bf806a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 -- 172.21.15.83:0/3861262160 >> 172.21.15.83:0/3861262160 conn(0x7f0bf80fb610 msgr2=0x7f0bf81045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 -- 172.21.15.83:0/3861262160 shutdown_connections 2024-03-11T19:39:03.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:03.951+0000 7f0c0018c640 1 -- 172.21.15.83:0/3861262160 wait complete. 2024-03-11T19:39:03.981 INFO:tasks.cephfs.filesystem:scrub status for tag:0878a811-bdc1-47f4-a37a-dcdbc9e0cea5 - {'path': '/', 'tag': '0878a811-bdc1-47f4-a37a-dcdbc9e0cea5', 'options': 'recursive,force'} 2024-03-11T19:39:04.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:03 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2444103565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:39:04.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:03 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/886245159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:39:04.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:03 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2444103565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:39:04.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:03 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/886245159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:39:04.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:03 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2444103565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:39:04.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:03 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/886245159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:39:05.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:04 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:39:05.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:04 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:05.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:04 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:39:05.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:04 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:05.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:04 smithi083 ceph-mon[31864]: pgmap v345: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-11T19:39:05.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:04 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:39:05.448 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:04 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:05.449 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:04 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:39:05.449 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:04 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:05.449 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:04 smithi137 ceph-mon[36220]: pgmap v345: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-11T19:39:05.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:04 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:39:05.449 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:04 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:05.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:04 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:39:05.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:04 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:05.450 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:04 smithi204 ceph-mon[36739]: pgmap v345: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-11T19:39:05.573 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-03-11T19:39:05.614 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-03-11T19:39:05.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-03-11T19:39:05.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-11T19:39:05.784 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/: 2024-03-11T19:39:05.784 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:05.784 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 . 2024-03-11T19:39:05.784 INFO:teuthology.orchestra.run.smithi083.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 .. 2024-03-11T19:39:05.784 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 data 2024-03-11T19:39:05.784 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:05.784 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data: 2024-03-11T19:39:05.784 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:05.784 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:05.784 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 .. 2024-03-11T19:39:05.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-03-11T19:39:05.837 INFO:teuthology.orchestra.run.smithi083.stdout:[Service] 2024-03-11T19:39:05.837 INFO:teuthology.orchestra.run.smithi083.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-03-11T19:39:05.837 INFO:teuthology.orchestra.run.smithi083.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-03-11T19:39:05.838 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-03-11T19:39:05.917 INFO:teuthology.orchestra.run.smithi083.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-03-11T19:39:06.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:05 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:39:06.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:05 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:39:06.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:05 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:39:07.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:07 smithi083 ceph-mon[31864]: pgmap v346: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-11T19:39:07.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:07 smithi137 ceph-mon[36220]: pgmap v346: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-11T19:39:07.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:07 smithi204 ceph-mon[36739]: pgmap v346: 97 pgs: 97 active+clean; 646 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-11T19:39:09.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:09 smithi137 ceph-mon[36220]: pgmap v347: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 81 KiB/s wr, 10 op/s 2024-03-11T19:39:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:09 smithi204 ceph-mon[36739]: pgmap v347: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 81 KiB/s wr, 10 op/s 2024-03-11T19:39:10.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:09 smithi083 ceph-mon[31864]: pgmap v347: 97 pgs: 97 active+clean; 1.5 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 81 KiB/s wr, 10 op/s 2024-03-11T19:39:11.093 INFO:teuthology.orchestra.run.smithi083.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-03-11T19:39:11.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-11T19:39:11.142 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/: 2024-03-11T19:39:11.142 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.142 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 . 2024-03-11T19:39:11.142 INFO:teuthology.orchestra.run.smithi083.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 .. 2024-03-11T19:39:11.142 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 data 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data: 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:total 35 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 . 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 .. 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 base 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Mar 11 19:39 global 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 log 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_commit_ts 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_dynshmem 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Mar 11 19:39 pg_hba.conf 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Mar 11 19:39 pg_ident.conf 2024-03-11T19:39:11.143 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 pg_logical 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 11 19:39 pg_multixact 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_notify 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_replslot 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_serial 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_snapshots 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_stat 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_stat_tmp 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 pg_subtrans 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_tblspc 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_twophase 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 PG_VERSION 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 11 19:39 pg_wal 2024-03-11T19:39:11.144 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 pg_xact 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Mar 11 19:39 postgresql.auto.conf 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Mar 11 19:39 postgresql.conf 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base: 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 . 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 11 19:39 1 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 11 19:39 13493 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 11 19:39 13494 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/1: 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:total 7777 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 11 19:39 . 2024-03-11T19:39:11.145 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 .. 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 112 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 113 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 1247 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1247_fsm 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1247_vm 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 11 19:39 1249 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1249_fsm 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1249_vm 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 11 19:39 1255 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1255_fsm 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1255_vm 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 11 19:39 1259 2024-03-11T19:39:11.146 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1259_fsm 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1259_vm 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 11 19:39 13344 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13344_fsm 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13344_vm 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13346 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13348 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13349_fsm 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349_vm 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13351 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13353 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13354_fsm 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354_vm 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13356 2024-03-11T19:39:11.147 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13358 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13359_fsm 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359_vm 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13361 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13363 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1417 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1418 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 174 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 175 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2187 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2224 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2328 2024-03-11T19:39:11.148 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2336 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2337 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2579 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2600 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2600_fsm 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2600_vm 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2601_fsm 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601_vm 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2602 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2602_fsm 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2602_vm 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2603 2024-03-11T19:39:11.149 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2603_fsm 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2603_vm 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2604 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2605 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2605_fsm 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2605_vm 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2606_fsm 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606_vm 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2607 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2607_fsm 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2607_vm 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 11 19:39 2608 2024-03-11T19:39:11.150 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2608_fsm 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2608_vm 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 11 19:39 2609 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2609_fsm 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2609_vm 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2610 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2610_fsm 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2610_vm 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2611 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2612_fsm 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612_vm 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2613 2024-03-11T19:39:11.151 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2615_fsm 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615_vm 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2616 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2616_fsm 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2616_vm 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 11 19:39 2617 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2617_fsm 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2617_vm 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 11 19:39 2618 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2618_fsm 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2618_vm 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 11 19:39 2619 2024-03-11T19:39:11.152 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2619_fsm 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2619_vm 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2620 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2650 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2651 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2652 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2653 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2654 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2655 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2656 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2657 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 11 19:39 2658 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 2659 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2660 2024-03-11T19:39:11.153 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2661 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2662 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2663 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2664 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2665 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2666 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2667 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2668 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2669 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2670 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 11 19:39 2673 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 11 19:39 2674 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 11 19:39 2675 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2678 2024-03-11T19:39:11.154 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2679 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2680 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2681 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2682 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2683 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2684 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2685 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2686 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2687 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2688 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2689 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 11 19:39 2690 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 11 19:39 2691 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2692 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2693 2024-03-11T19:39:11.155 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2696 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2699 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2701 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2702 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2703 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2704 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2753 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2753_fsm 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2753_vm 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2754 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2755 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2756 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2757 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2830 2024-03-11T19:39:11.156 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2831 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2832 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2833 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2834 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2835 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2836 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2837 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 11 19:39 2838 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2838_fsm 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2838_vm 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2839 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840_fsm 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2840_vm 2024-03-11T19:39:11.157 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2841 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2995 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2996 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3079_fsm 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079_vm 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3080 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3081 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3085 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3118 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3119 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 11 19:39 3164 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3256 2024-03-11T19:39:11.158 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3257 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3258 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3350 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3351 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3379 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3380 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3381 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3394 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3394_fsm 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3394_vm 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3395 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3429 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3430 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3431 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3433 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3439 2024-03-11T19:39:11.159 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3440 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3455 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 11 19:39 3456 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3456_fsm 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3456_vm 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3466 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3467 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3468 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3501 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3502 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3503 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3534 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3541_fsm 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541_vm 2024-03-11T19:39:11.160 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3542 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3574 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3575 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3576 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3596 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3597 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3598 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3599 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3600_fsm 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600_vm 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3601_fsm 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601_vm 2024-03-11T19:39:11.161 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3602_fsm 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602_vm 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603_fsm 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3603_vm 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3604 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3605 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3606 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3607 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3608 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 3609 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3712 2024-03-11T19:39:11.162 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3764_fsm 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764_vm 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3766 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3767 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3997 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4143 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4144 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4145 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4146 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4147 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4148 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4149 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4150 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4151 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4152 2024-03-11T19:39:11.163 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4153 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4154 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4155 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4156 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4157 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4158 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4159 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4160 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4161 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4162 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4163 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4164 2024-03-11T19:39:11.164 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4165 2024-03-11T19:39:11.167 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4166 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4167 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4168 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4169 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4170 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4171 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4172 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4173 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4174 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 5002 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 548 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 549 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6102 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6104 2024-03-11T19:39:11.168 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6106 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6110 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6111 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6112 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6113 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6117 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 826 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 827 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 828 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 11 19:39 pg_filenode.map 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 PG_VERSION 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-03-11T19:39:11.169 INFO:teuthology.orchestra.run.smithi083.stdout:total 7777 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 11 19:39 . 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 .. 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 112 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 113 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 1247 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1247_fsm 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1247_vm 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 11 19:39 1249 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1249_fsm 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1249_vm 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 11 19:39 1255 2024-03-11T19:39:11.170 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1255_fsm 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1255_vm 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 11 19:39 1259 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1259_fsm 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1259_vm 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 11 19:39 13344 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13344_fsm 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13344_vm 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13346 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13348 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13349_fsm 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349_vm 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13351 2024-03-11T19:39:11.171 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13353 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13354_fsm 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354_vm 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13356 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13358 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13359_fsm 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359_vm 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13361 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13363 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1417 2024-03-11T19:39:11.172 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1418 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 174 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 175 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2187 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2224 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2328 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2336 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2337 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2579 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2600 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2600_fsm 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2600_vm 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2601_fsm 2024-03-11T19:39:11.173 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601_vm 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2602 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2602_fsm 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2602_vm 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2603 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2603_fsm 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2603_vm 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2604 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2605 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2605_fsm 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2605_vm 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2606_fsm 2024-03-11T19:39:11.174 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606_vm 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2607 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2607_fsm 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2607_vm 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 11 19:39 2608 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2608_fsm 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2608_vm 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 11 19:39 2609 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2609_fsm 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2609_vm 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2610 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2610_fsm 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2610_vm 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2611 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612 2024-03-11T19:39:11.175 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2612_fsm 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612_vm 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2613 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2615_fsm 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615_vm 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2616 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2616_fsm 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2616_vm 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 11 19:39 2617 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2617_fsm 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2617_vm 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 11 19:39 2618 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2618_fsm 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2618_vm 2024-03-11T19:39:11.176 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 11 19:39 2619 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2619_fsm 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2619_vm 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2620 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2650 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2651 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2652 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2653 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2654 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2655 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2656 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2657 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 11 19:39 2658 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 2659 2024-03-11T19:39:11.177 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2660 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2661 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2662 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2663 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2664 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2665 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2666 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2667 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2668 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2669 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2670 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 11 19:39 2673 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 11 19:39 2674 2024-03-11T19:39:11.178 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 11 19:39 2675 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2678 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2679 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2680 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2681 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2682 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2683 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2684 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2685 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2686 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2687 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2688 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2689 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 11 19:39 2690 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 11 19:39 2691 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2692 2024-03-11T19:39:11.179 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2693 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2696 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2699 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2701 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2702 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2703 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2704 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2753 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2753_fsm 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2753_vm 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2754 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2755 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2756 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2757 2024-03-11T19:39:11.180 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2830 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2831 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2832 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2833 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2834 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2835 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2836 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2837 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 11 19:39 2838 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2838_fsm 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2838_vm 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2839 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840_fsm 2024-03-11T19:39:11.181 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2840_vm 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2841 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2995 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2996 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3079_fsm 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079_vm 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3080 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3081 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3085 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3118 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3119 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 11 19:39 3164 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3256 2024-03-11T19:39:11.182 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3257 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3258 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3350 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3351 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3379 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3380 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3381 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3394 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3394_fsm 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3394_vm 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3395 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3429 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3430 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3431 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3433 2024-03-11T19:39:11.183 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3439 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3440 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3455 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 11 19:39 3456 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3456_fsm 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3456_vm 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3466 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3467 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3468 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3501 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3502 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3503 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3534 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3541_fsm 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541_vm 2024-03-11T19:39:11.184 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3542 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3574 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3575 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3576 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3596 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3597 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3598 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3599 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3600_fsm 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600_vm 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3601_fsm 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601_vm 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602 2024-03-11T19:39:11.185 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3602_fsm 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602_vm 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603_fsm 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3603_vm 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3604 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3605 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3606 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3607 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3608 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 3609 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3712 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764 2024-03-11T19:39:11.186 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3764_fsm 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764_vm 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3766 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3767 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3997 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4143 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4144 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4145 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4146 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4147 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4148 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4149 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4150 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4151 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4152 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4153 2024-03-11T19:39:11.187 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4154 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4155 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4156 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4157 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4158 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4159 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4160 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4161 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4162 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4163 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4164 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4165 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4166 2024-03-11T19:39:11.188 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4167 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4168 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4169 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4170 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4171 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4172 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4173 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4174 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 5002 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 548 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 549 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6102 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6104 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6106 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6110 2024-03-11T19:39:11.189 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6111 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6112 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6113 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6117 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 826 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 827 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 828 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 11 19:39 pg_filenode.map 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 PG_VERSION 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:total 7777 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 11 19:39 . 2024-03-11T19:39:11.190 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 .. 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 112 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 113 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 1247 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1247_fsm 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1247_vm 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 11 19:39 1249 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1249_fsm 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1249_vm 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 11 19:39 1255 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1255_fsm 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1255_vm 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 11 19:39 1259 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1259_fsm 2024-03-11T19:39:11.191 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1259_vm 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 11 19:39 13344 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13344_fsm 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13344_vm 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13346 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13348 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13349_fsm 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349_vm 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13351 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13353 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13354_fsm 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354_vm 2024-03-11T19:39:11.192 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13356 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13358 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13359_fsm 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359_vm 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13361 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13363 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1417 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1418 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 174 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 175 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2187 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2224 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2328 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2336 2024-03-11T19:39:11.193 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2337 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2579 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2600 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2600_fsm 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2600_vm 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2601_fsm 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601_vm 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2602 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2602_fsm 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2602_vm 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2603 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2603_fsm 2024-03-11T19:39:11.194 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2603_vm 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2604 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2605 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2605_fsm 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2605_vm 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2606_fsm 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606_vm 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2607 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2607_fsm 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2607_vm 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 11 19:39 2608 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2608_fsm 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2608_vm 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 11 19:39 2609 2024-03-11T19:39:11.195 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2609_fsm 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2609_vm 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2610 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2610_fsm 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2610_vm 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2611 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2612_fsm 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612_vm 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2613 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2615_fsm 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615_vm 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2616 2024-03-11T19:39:11.196 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2616_fsm 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2616_vm 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 11 19:39 2617 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2617_fsm 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2617_vm 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 11 19:39 2618 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2618_fsm 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2618_vm 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 11 19:39 2619 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2619_fsm 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2619_vm 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2620 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2650 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2651 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2652 2024-03-11T19:39:11.197 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2653 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2654 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2655 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2656 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2657 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 11 19:39 2658 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 2659 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2660 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2661 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2662 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2663 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2664 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2665 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2666 2024-03-11T19:39:11.198 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2667 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2668 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2669 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2670 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 11 19:39 2673 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 11 19:39 2674 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 11 19:39 2675 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2678 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2679 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2680 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2681 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2682 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2683 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2684 2024-03-11T19:39:11.199 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2685 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2686 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2687 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2688 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2689 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 11 19:39 2690 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 11 19:39 2691 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2692 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2693 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2696 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2699 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2701 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2702 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2703 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2704 2024-03-11T19:39:11.200 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2753 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2753_fsm 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2753_vm 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2754 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2755 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2756 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2757 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2830 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2831 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2832 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2833 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2834 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2835 2024-03-11T19:39:11.201 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2836 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2837 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 11 19:39 2838 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2838_fsm 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2838_vm 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2839 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840_fsm 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2840_vm 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2841 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2995 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2996 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3079_fsm 2024-03-11T19:39:11.202 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079_vm 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3080 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3081 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3085 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3118 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3119 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 11 19:39 3164 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3256 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3257 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3258 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3350 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3351 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3379 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3380 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3381 2024-03-11T19:39:11.203 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3394 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3394_fsm 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3394_vm 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3395 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3429 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3430 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3431 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3433 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3439 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3440 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3455 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 11 19:39 3456 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3456_fsm 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3456_vm 2024-03-11T19:39:11.204 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3466 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3467 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3468 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3501 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3502 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3503 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3534 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3541_fsm 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541_vm 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3542 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3574 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3575 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3576 2024-03-11T19:39:11.205 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3596 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3597 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3598 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3599 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3600_fsm 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600_vm 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3601_fsm 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601_vm 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3602_fsm 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602_vm 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603 2024-03-11T19:39:11.206 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603_fsm 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3603_vm 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3604 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3605 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3606 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3607 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3608 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 3609 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3712 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3764_fsm 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764_vm 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3766 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3767 2024-03-11T19:39:11.207 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3997 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4143 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4144 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4145 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4146 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4147 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4148 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4149 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4150 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4151 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4152 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4153 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4154 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4155 2024-03-11T19:39:11.208 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4156 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4157 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4158 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4159 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4160 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4161 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4162 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4163 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4164 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4165 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4166 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4167 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4168 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4169 2024-03-11T19:39:11.209 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4170 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4171 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4172 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4173 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4174 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 5002 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 548 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 549 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6102 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6104 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6106 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6110 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6111 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6112 2024-03-11T19:39:11.210 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6113 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6117 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 826 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 827 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 828 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 11 19:39 pg_filenode.map 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 PG_VERSION 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/global: 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:total 537 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Mar 11 19:39 . 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1213 2024-03-11T19:39:11.211 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1213_fsm 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1213_vm 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1214 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1214_fsm 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1214_vm 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 1232 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 1233 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1260 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1260_fsm 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1260_vm 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1261 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1261_fsm 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1261_vm 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1262 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1262_fsm 2024-03-11T19:39:11.212 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1262_vm 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2396 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2396_fsm 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2396_vm 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2397 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2671 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2672 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2676 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2677 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2694 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2695 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2697 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2698 2024-03-11T19:39:11.213 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2846 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2847 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2964 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2965 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2966 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2967 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3592 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3593 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4060 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4061 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4175 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4176 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4177 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4178 2024-03-11T19:39:11.214 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4181 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4182 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4183 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4184 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4185 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4186 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6000 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6001 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6002 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6100 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6114 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6115 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 pg_control 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 11 19:39 pg_filenode.map 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/log: 2024-03-11T19:39:11.215 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:total 1 2024-03-11T19:39:11.216 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 . 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 mappings 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Mar 11 19:39 replorigin_checkpoint 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 snapshots 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 .. 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 .. 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.217 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 11 19:39 . 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 members 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 offsets 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 . 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 11 19:39 .. 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 0000 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-03-11T19:39:11.218 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 . 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 11 19:39 .. 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 0000 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-03-11T19:39:11.219 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-03-11T19:39:11.220 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 . 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 0000 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.221 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout:total 16384 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 11 19:39 . 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:39 000000010000000000000001 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 archive_status 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.222 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-03-11T19:39:11.223 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:39:11.223 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:39:11.223 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 11 19:39 .. 2024-03-11T19:39:11.223 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:11.223 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-03-11T19:39:11.223 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-03-11T19:39:11.223 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 . 2024-03-11T19:39:11.223 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 11 19:39 .. 2024-03-11T19:39:11.223 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 0000 2024-03-11T19:39:11.223 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-03-11T19:39:11.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-03-11T19:39:11.394 INFO:teuthology.orchestra.run.smithi083.stderr:dropping old tables... 2024-03-11T19:39:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-03-11T19:39:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-03-11T19:39:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-03-11T19:39:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-03-11T19:39:11.395 INFO:teuthology.orchestra.run.smithi083.stderr:creating tables... 2024-03-11T19:39:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:generating data (client-side)... 2024-03-11T19:39:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.79 s) 2024-03-11T19:39:11.634 INFO:teuthology.orchestra.run.smithi083.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.21 s, remaining 3.18 s) 2024-03-11T19:39:11.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:11 smithi137 ceph-mon[36220]: pgmap v348: 97 pgs: 97 active+clean; 3.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-03-11T19:39:11.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:11 smithi204 ceph-mon[36739]: pgmap v348: 97 pgs: 97 active+clean; 3.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-03-11T19:39:11.722 INFO:teuthology.orchestra.run.smithi083.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.30 s, remaining 2.91 s) 2024-03-11T19:39:11.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:11 smithi083 ceph-mon[31864]: pgmap v348: 97 pgs: 97 active+clean; 3.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-03-11T19:39:11.855 INFO:teuthology.orchestra.run.smithi083.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.43 s, remaining 3.03 s) 2024-03-11T19:39:11.995 INFO:teuthology.orchestra.run.smithi083.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.57 s, remaining 3.09 s) 2024-03-11T19:39:12.076 INFO:teuthology.orchestra.run.smithi083.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.65 s, remaining 2.84 s) 2024-03-11T19:39:12.226 INFO:teuthology.orchestra.run.smithi083.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.80 s, remaining 2.87 s) 2024-03-11T19:39:12.415 INFO:teuthology.orchestra.run.smithi083.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.99 s, remaining 2.98 s) 2024-03-11T19:39:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.09 s, remaining 2.80 s) 2024-03-11T19:39:12.650 INFO:teuthology.orchestra.run.smithi083.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.23 s, remaining 2.70 s) 2024-03-11T19:39:12.826 INFO:teuthology.orchestra.run.smithi083.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.40 s, remaining 2.68 s) 2024-03-11T19:39:12.927 INFO:teuthology.orchestra.run.smithi083.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.51 s, remaining 2.51 s) 2024-03-11T19:39:13.073 INFO:teuthology.orchestra.run.smithi083.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.65 s, remaining 2.41 s) 2024-03-11T19:39:13.251 INFO:teuthology.orchestra.run.smithi083.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.83 s, remaining 2.35 s) 2024-03-11T19:39:13.345 INFO:teuthology.orchestra.run.smithi083.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.92 s, remaining 2.18 s) 2024-03-11T19:39:13.483 INFO:teuthology.orchestra.run.smithi083.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.06 s, remaining 2.06 s) 2024-03-11T19:39:13.580 INFO:teuthology.orchestra.run.smithi083.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.16 s, remaining 1.90 s) 2024-03-11T19:39:13.746 INFO:teuthology.orchestra.run.smithi083.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.32 s, remaining 1.81 s) 2024-03-11T19:39:13.882 INFO:teuthology.orchestra.run.smithi083.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.46 s, remaining 1.68 s) 2024-03-11T19:39:13.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:13 smithi137 ceph-mon[36220]: pgmap v349: 97 pgs: 97 active+clean; 24 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.9 MiB/s wr, 39 op/s 2024-03-11T19:39:13.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:13 smithi204 ceph-mon[36739]: pgmap v349: 97 pgs: 97 active+clean; 24 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.9 MiB/s wr, 39 op/s 2024-03-11T19:39:13.978 INFO:teuthology.orchestra.run.smithi083.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.56 s, remaining 1.53 s) 2024-03-11T19:39:14.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:13 smithi083 ceph-mon[31864]: pgmap v349: 97 pgs: 97 active+clean; 24 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.9 MiB/s wr, 39 op/s 2024-03-11T19:39:14.116 INFO:teuthology.orchestra.run.smithi083.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.69 s, remaining 1.41 s) 2024-03-11T19:39:14.294 INFO:teuthology.orchestra.run.smithi083.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.87 s, remaining 1.31 s) 2024-03-11T19:39:14.384 INFO:teuthology.orchestra.run.smithi083.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.96 s, remaining 1.16 s) 2024-03-11T19:39:14.521 INFO:teuthology.orchestra.run.smithi083.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.10 s, remaining 1.03 s) 2024-03-11T19:39:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.28 s, remaining 0.92 s) 2024-03-11T19:39:14.808 INFO:teuthology.orchestra.run.smithi083.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.39 s, remaining 0.78 s) 2024-03-11T19:39:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.54 s, remaining 0.66 s) 2024-03-11T19:39:15.068 INFO:teuthology.orchestra.run.smithi083.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.65 s, remaining 0.52 s) 2024-03-11T19:39:15.218 INFO:teuthology.orchestra.run.smithi083.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.80 s, remaining 0.39 s) 2024-03-11T19:39:15.352 INFO:teuthology.orchestra.run.smithi083.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.93 s, remaining 0.26 s) 2024-03-11T19:39:15.461 INFO:teuthology.orchestra.run.smithi083.stderr:3100000 of 3200000 tuples (96%) done (elapsed 4.04 s, remaining 0.13 s) 2024-03-11T19:39:15.593 INFO:teuthology.orchestra.run.smithi083.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.17 s, remaining 0.00 s) 2024-03-11T19:39:15.662 INFO:teuthology.orchestra.run.smithi083.stderr:vacuuming... 2024-03-11T19:39:15.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:15 smithi137 ceph-mon[36220]: pgmap v350: 97 pgs: 97 active+clean; 105 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 15 MiB/s wr, 113 op/s 2024-03-11T19:39:15.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:15 smithi204 ceph-mon[36739]: pgmap v350: 97 pgs: 97 active+clean; 105 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 15 MiB/s wr, 113 op/s 2024-03-11T19:39:16.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:15 smithi083 ceph-mon[31864]: pgmap v350: 97 pgs: 97 active+clean; 105 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 15 MiB/s wr, 113 op/s 2024-03-11T19:39:16.249 INFO:teuthology.orchestra.run.smithi083.stderr:creating primary keys... 2024-03-11T19:39:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:16 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:39:16.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:16 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:39:16.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:16 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:39:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:done in 6.04 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.26 s, vacuum 0.59 s, primary keys 1.19 s). 2024-03-11T19:39:17.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-03-11T19:39:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:starting vacuum...end. 2024-03-11T19:39:17.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:17 smithi083 ceph-mon[31864]: pgmap v351: 97 pgs: 97 active+clean; 224 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 35 MiB/s wr, 133 op/s 2024-03-11T19:39:17.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:17 smithi137 ceph-mon[36220]: pgmap v351: 97 pgs: 97 active+clean; 224 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 35 MiB/s wr, 133 op/s 2024-03-11T19:39:17.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:17 smithi204 ceph-mon[36739]: pgmap v351: 97 pgs: 97 active+clean; 224 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 35 MiB/s wr, 133 op/s 2024-03-11T19:39:18.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:18 smithi137 ceph-mon[36220]: pgmap v352: 97 pgs: 97 active+clean; 409 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 62 MiB/s wr, 147 op/s 2024-03-11T19:39:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:18 smithi204 ceph-mon[36739]: pgmap v352: 97 pgs: 97 active+clean; 409 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 62 MiB/s wr, 147 op/s 2024-03-11T19:39:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:18 smithi083 ceph-mon[31864]: pgmap v352: 97 pgs: 97 active+clean; 409 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 62 MiB/s wr, 147 op/s 2024-03-11T19:39:19.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:19 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:39:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:19 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:39:19.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:19 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:39:20.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:20 smithi137 ceph-mon[36220]: pgmap v353: 97 pgs: 97 active+clean; 471 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 67 MiB/s wr, 147 op/s 2024-03-11T19:39:20.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:20 smithi204 ceph-mon[36739]: pgmap v353: 97 pgs: 97 active+clean; 471 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 67 MiB/s wr, 147 op/s 2024-03-11T19:39:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:20 smithi083 ceph-mon[31864]: pgmap v353: 97 pgs: 97 active+clean; 471 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 67 MiB/s wr, 147 op/s 2024-03-11T19:39:21.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:21 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:39:21.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:21.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:21.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:21.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:21.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:21.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:21 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:21 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:39:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:21 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:21 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:39:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:21 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 5.0 s, 6047.3 tps, lat 16.197 ms stddev 16.877 2024-03-11T19:39:22.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:22 smithi204 ceph-mon[36739]: pgmap v354: 97 pgs: 97 active+clean; 682 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 78 MiB/s wr, 348 op/s 2024-03-11T19:39:22.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:22 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:39:22.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:22 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:39:22.910 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:22 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:22 smithi083 ceph-mon[31864]: pgmap v354: 97 pgs: 97 active+clean; 682 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 78 MiB/s wr, 348 op/s 2024-03-11T19:39:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:22 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:39:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:22 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:39:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:22 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:23.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:22 smithi137 ceph-mon[36220]: pgmap v354: 97 pgs: 97 active+clean; 682 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 78 MiB/s wr, 348 op/s 2024-03-11T19:39:23.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:22 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:39:23.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:22 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:39:23.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:22 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:39:25.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:25 smithi137 ceph-mon[36220]: pgmap v355: 97 pgs: 97 active+clean; 888 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 78 MiB/s wr, 482 op/s 2024-03-11T19:39:25.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:25 smithi204 ceph-mon[36739]: pgmap v355: 97 pgs: 97 active+clean; 888 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 78 MiB/s wr, 482 op/s 2024-03-11T19:39:26.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:25 smithi083 ceph-mon[31864]: pgmap v355: 97 pgs: 97 active+clean; 888 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 78 MiB/s wr, 482 op/s 2024-03-11T19:39:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 10.0 s, 6973.1 tps, lat 14.426 ms stddev 15.050 2024-03-11T19:39:27.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:27 smithi137 ceph-mon[36220]: pgmap v356: 97 pgs: 97 active+clean; 918 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 68 MiB/s wr, 528 op/s 2024-03-11T19:39:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:27 smithi204 ceph-mon[36739]: pgmap v356: 97 pgs: 97 active+clean; 918 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 68 MiB/s wr, 528 op/s 2024-03-11T19:39:28.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:27 smithi083 ceph-mon[31864]: pgmap v356: 97 pgs: 97 active+clean; 918 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 68 MiB/s wr, 528 op/s 2024-03-11T19:39:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:29 smithi137 ceph-mon[36220]: pgmap v357: 97 pgs: 97 active+clean; 934 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 50 MiB/s wr, 668 op/s 2024-03-11T19:39:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:29 smithi204 ceph-mon[36739]: pgmap v357: 97 pgs: 97 active+clean; 934 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 50 MiB/s wr, 668 op/s 2024-03-11T19:39:30.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:29 smithi083 ceph-mon[31864]: pgmap v357: 97 pgs: 97 active+clean; 934 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 50 MiB/s wr, 668 op/s 2024-03-11T19:39:31.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:31 smithi137 ceph-mon[36220]: pgmap v358: 97 pgs: 97 active+clean; 938 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 25 MiB/s wr, 792 op/s 2024-03-11T19:39:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:31 smithi204 ceph-mon[36739]: pgmap v358: 97 pgs: 97 active+clean; 938 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 25 MiB/s wr, 792 op/s 2024-03-11T19:39:32.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:31 smithi083 ceph-mon[31864]: pgmap v358: 97 pgs: 97 active+clean; 938 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 25 MiB/s wr, 792 op/s 2024-03-11T19:39:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 15.0 s, 7260.4 tps, lat 13.775 ms stddev 13.946 2024-03-11T19:39:33.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:33 smithi137 ceph-mon[36220]: pgmap v359: 97 pgs: 97 active+clean; 950 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 22 MiB/s wr, 927 op/s 2024-03-11T19:39:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:33 smithi204 ceph-mon[36739]: pgmap v359: 97 pgs: 97 active+clean; 950 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 22 MiB/s wr, 927 op/s 2024-03-11T19:39:33.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:39:33.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:39:34.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:33 smithi083 ceph-mon[31864]: pgmap v359: 97 pgs: 97 active+clean; 950 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 22 MiB/s wr, 927 op/s 2024-03-11T19:39:34.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.149+0000 7f633dad4640 1 -- 172.21.15.83:0/2781169414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6338102370 msgr2=0x7f63381027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.149+0000 7f633dad4640 1 --2- 172.21.15.83:0/2781169414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6338102370 0x7f63381027f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f632c009900 tx=0x7f632c02bc40 comp rx=0 tx=0).stop 2024-03-11T19:39:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.149+0000 7f633dad4640 1 -- 172.21.15.83:0/2781169414 shutdown_connections 2024-03-11T19:39:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.149+0000 7f633dad4640 1 --2- 172.21.15.83:0/2781169414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338102d30 0x7f63381095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.149+0000 7f633dad4640 1 --2- 172.21.15.83:0/2781169414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6338102370 0x7f63381027f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.149+0000 7f633dad4640 1 --2- 172.21.15.83:0/2781169414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338101180 0x7f63381015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.149+0000 7f633dad4640 1 -- 172.21.15.83:0/2781169414 >> 172.21.15.83:0/2781169414 conn(0x7f63380fc6f0 msgr2=0x7f63380feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.149+0000 7f633dad4640 1 -- 172.21.15.83:0/2781169414 shutdown_connections 2024-03-11T19:39:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.149+0000 7f633dad4640 1 -- 172.21.15.83:0/2781169414 wait complete. 2024-03-11T19:39:34.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.150+0000 7f633dad4640 1 Processor -- start 2024-03-11T19:39:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.150+0000 7f633dad4640 1 -- start start 2024-03-11T19:39:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.150+0000 7f633dad4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6338101180 0x7f6338199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.150+0000 7f633dad4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338102370 0x7f6338199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.150+0000 7f633dad4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338102d30 0x7f63381a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.150+0000 7f633dad4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63381a0f30 con 0x7f6338101180 2024-03-11T19:39:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.150+0000 7f633dad4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63381a10a0 con 0x7f6338102370 2024-03-11T19:39:34.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.150+0000 7f633dad4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63381a1260 con 0x7f6338102d30 2024-03-11T19:39:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.152+0000 7f63377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6338101180 0x7f6338199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.152+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338102370 0x7f6338199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.152+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338102370 0x7f6338199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:53184/0 (socket says 172.21.15.83:53184) 2024-03-11T19:39:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.152+0000 7f6336ffd640 1 -- 172.21.15.83:0/2265072558 learned_addr learned my addr 172.21.15.83:0/2265072558 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:34.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.152+0000 7f6337fff640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338102d30 0x7f63381a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.152+0000 7f63377fe640 1 -- 172.21.15.83:0/2265072558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338102d30 msgr2=0x7f63381a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.152+0000 7f63377fe640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338102d30 0x7f63381a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.152+0000 7f63377fe640 1 -- 172.21.15.83:0/2265072558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338102370 msgr2=0x7f6338199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.152+0000 7f63377fe640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338102370 0x7f6338199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.152+0000 7f63377fe640 1 -- 172.21.15.83:0/2265072558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c009560 con 0x7f6338101180 2024-03-11T19:39:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.153+0000 7f6337fff640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338102d30 0x7f63381a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:39:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.153+0000 7f6336ffd640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338102370 0x7f6338199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:39:34.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.153+0000 7f63377fe640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6338101180 0x7f6338199440 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f63381021d0 tx=0x7f632800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.153+0000 7f6334ff9640 1 -- 172.21.15.83:0/2265072558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328009c30 con 0x7f6338101180 2024-03-11T19:39:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.153+0000 7f633dad4640 1 -- 172.21.15.83:0/2265072558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63381a1540 con 0x7f6338101180 2024-03-11T19:39:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.153+0000 7f6334ff9640 1 -- 172.21.15.83:0/2265072558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6328009d90 con 0x7f6338101180 2024-03-11T19:39:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.154+0000 7f6334ff9640 1 -- 172.21.15.83:0/2265072558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632800f740 con 0x7f6338101180 2024-03-11T19:39:34.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.154+0000 7f633dad4640 1 -- 172.21.15.83:0/2265072558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63381a1a50 con 0x7f6338101180 2024-03-11T19:39:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.157+0000 7f6340789740 1 Processor -- start 2024-03-11T19:39:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.157+0000 7f6340789740 1 -- start start 2024-03-11T19:39:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.157+0000 7f6340789740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e38282f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.157+0000 7f6340789740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e382834d0 0x555e38287fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.157+0000 7f6340789740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e38288640 0x555e3828aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.157+0000 7f6340789740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e381ea280 con 0x555e382834d0 2024-03-11T19:39:34.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.157+0000 7f6340789740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e381e77b0 con 0x555e38288640 2024-03-11T19:39:34.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.157+0000 7f6340789740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e381e2f90 con 0x555e38282ae0 2024-03-11T19:39:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.158+0000 7f6337fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e38282f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.158+0000 7f6337fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e38282f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53082/0 (socket says 172.21.15.83:53082) 2024-03-11T19:39:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.158+0000 7f6337fff640 1 -- 172.21.15.83:0/3910865211 learned_addr learned my addr 172.21.15.83:0/3910865211 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.158+0000 7f6337fff640 1 -- 172.21.15.83:0/3910865211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e38288640 msgr2=0x555e3828aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.158+0000 7f6337fff640 1 --2- 172.21.15.83:0/3910865211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e38288640 0x555e3828aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.158+0000 7f6337fff640 1 -- 172.21.15.83:0/3910865211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e382834d0 msgr2=0x555e38287fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:39:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.158+0000 7f6337fff640 1 --2- 172.21.15.83:0/3910865211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e382834d0 0x555e38287fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.158+0000 7f6337fff640 1 -- 172.21.15.83:0/3910865211 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e381a1570 con 0x555e38282ae0 2024-03-11T19:39:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.158+0000 7f6334ff9640 1 -- 172.21.15.83:0/2265072558 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f63280109d0 con 0x7f6338101180 2024-03-11T19:39:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.158+0000 7f6337fff640 1 --2- 172.21.15.83:0/3910865211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e38282f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f633000c450 tx=0x7f633000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee8143cf173e1a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:34.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.159+0000 7f631dffb640 1 -- 172.21.15.83:0/3910865211 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330011070 con 0x555e38282ae0 2024-03-11T19:39:34.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.159+0000 7f631dffb640 1 -- 172.21.15.83:0/3910865211 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6330009b20 con 0x555e38282ae0 2024-03-11T19:39:34.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.159+0000 7f631dffb640 1 -- 172.21.15.83:0/3910865211 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633000a960 con 0x555e38282ae0 2024-03-11T19:39:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.159+0000 7f6334ff9640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f63200a1c00 0x7f63200a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.160+0000 7f6334ff9640 1 -- 172.21.15.83:0/2265072558 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f63280cd410 con 0x7f6338101180 2024-03-11T19:39:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.160+0000 7f6340789740 1 -- 172.21.15.83:0/3910865211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 msgr2=0x555e38282f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.160+0000 7f6340789740 1 --2- 172.21.15.83:0/3910865211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e38282f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f633000c450 tx=0x7f633000c910 comp rx=0 tx=0).stop 2024-03-11T19:39:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.160+0000 7f6340789740 1 -- 172.21.15.83:0/3910865211 shutdown_connections 2024-03-11T19:39:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.160+0000 7f63377fe640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f63200a1c00 0x7f63200a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.160+0000 7f6340789740 1 --2- 172.21.15.83:0/3910865211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e38288640 0x555e3828aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.160+0000 7f6340789740 1 --2- 172.21.15.83:0/3910865211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e382834d0 0x555e38287fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.160+0000 7f6340789740 1 --2- 172.21.15.83:0/3910865211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e38282f00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.160+0000 7f6340789740 1 -- 172.21.15.83:0/3910865211 >> 172.21.15.83:0/3910865211 conn(0x555e382757f0 msgr2=0x555e38275bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.160+0000 7f6340789740 1 -- 172.21.15.83:0/3910865211 shutdown_connections 2024-03-11T19:39:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.160+0000 7f6340789740 1 -- 172.21.15.83:0/3910865211 wait complete. 2024-03-11T19:39:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.161+0000 7f63377fe640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f63200a1c00 0x7f63200a40c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6328096d60 tx=0x7f6328000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.161+0000 7f6340789740 1 -- start start 2024-03-11T19:39:34.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.161+0000 7f6340789740 1 Processor -- start 2024-03-11T19:39:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.162+0000 7f6340789740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e3827b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.162+0000 7f6340789740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 0x555e3827bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.162+0000 7f6340789740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e38280cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.162+0000 7f6340789740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e37eb6430 con 0x555e38288640 2024-03-11T19:39:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.162+0000 7f6340789740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e37e72e00 con 0x555e382834d0 2024-03-11T19:39:34.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.162+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 0x555e3827bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.162+0000 7f6337fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e3827b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.162+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 0x555e3827bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:53206/0 (socket says 172.21.15.83:53206) 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.162+0000 7f6336ffd640 1 -- 172.21.15.83:0/1041361579 learned_addr learned my addr 172.21.15.83:0/1041361579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.162+0000 7f6340789740 1 -- 172.21.15.83:0/1041361579 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e38167fd0 con 0x555e38282ae0 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.162+0000 7f6340789740 10 client.0 _ll_register_callbacks cb 0x555e381bdfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.163+0000 7f6337fff640 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e38280cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.163+0000 7f6336ffd640 1 -- 172.21.15.83:0/1041361579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 msgr2=0x555e3827b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.163+0000 7f6336ffd640 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e3827b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.163+0000 7f6336ffd640 1 -- 172.21.15.83:0/1041361579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 msgr2=0x555e38280cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.163+0000 7f6336ffd640 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e38280cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.163+0000 7f6336ffd640 1 -- 172.21.15.83:0/1041361579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6330009ca0 con 0x555e382834d0 2024-03-11T19:39:34.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.163+0000 7f6337fff640 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e38280cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:39:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.163+0000 7f6336ffd640 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 0x555e3827bc80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f632c02c730 tx=0x7f632c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc7dabf40b550330 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.164+0000 7f6316ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:39:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.164+0000 7f6316ffd640 1 -- 172.21.15.83:0/1041361579 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c02c900 con 0x555e382834d0 2024-03-11T19:39:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.164+0000 7f6340789740 1 -- 172.21.15.83:0/1041361579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e37f1f720 con 0x555e382834d0 2024-03-11T19:39:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.164+0000 7f6316ffd640 1 -- 172.21.15.83:0/1041361579 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f632c004df0 con 0x555e382834d0 2024-03-11T19:39:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.165+0000 7f6316ffd640 1 -- 172.21.15.83:0/1041361579 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f632c005000 con 0x555e382834d0 2024-03-11T19:39:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.165+0000 7f6340789740 10 client.24902 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.165+0000 7f6340789740 1 -- 172.21.15.83:0/1041361579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555e381e1610 con 0x555e382834d0 2024-03-11T19:39:34.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.165+0000 7f6316ffd640 1 -- 172.21.15.83:0/1041361579 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555e37f1f720 con 0x555e382834d0 2024-03-11T19:39:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.165+0000 7f6316ffd640 1 -- 172.21.15.83:0/1041361579 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f632c03c050 con 0x555e382834d0 2024-03-11T19:39:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.165+0000 7f6340789740 10 client.24902 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.165+0000 7f6340789740 10 client.24902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.165+0000 7f6340789740 20 client.24902 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.165+0000 7f6340789740 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555e382a1340 0x555e382a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.165+0000 7f6340789740 4 client.24902 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:39:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.165+0000 7f6340789740 1 -- 172.21.15.83:0/1041361579 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e37f6e890 con 0x555e382a1340 2024-03-11T19:39:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.166+0000 7f6337fff640 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555e382a1340 0x555e382a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.166+0000 7f6337fff640 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555e382a1340 0x555e382a3760 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e22a606f11f645 server_cookie=23f2cf6524514a5e in_seq=0 out_seq=0 2024-03-11T19:39:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.166+0000 7f6316ffd640 10 client.24902 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6316ffd640 1 -- 172.21.15.83:0/1041361579 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e37f6e890 con 0x555e382a1340 2024-03-11T19:39:34.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6316ffd640 10 client.24902 handle_command_reply: tid=0 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 1 client.24902 shutdown 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 1 -- 172.21.15.83:0/1041361579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 msgr2=0x555e3827bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 0x555e3827bc80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f632c02c730 tx=0x7f632c004270 comp rx=0 tx=0).stop 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 1 -- 172.21.15.83:0/1041361579 shutdown_connections 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555e382a1340 0x555e382a3760 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e38280cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6316ffd640 0 client.24902 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 0x555e3827bc80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 1 --2- 172.21.15.83:0/1041361579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e3827b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 1 -- 172.21.15.83:0/1041361579 >> 172.21.15.83:0/1041361579 conn(0x555e382757f0 msgr2=0x555e38274a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 1 -- 172.21.15.83:0/1041361579 shutdown_connections 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 1 -- 172.21.15.83:0/1041361579 wait complete. 2024-03-11T19:39:34.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.167+0000 7f6340789740 20 client.24902 trim_cache size 0 max 16384 2024-03-11T19:39:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.193+0000 7f6340789740 1 Processor -- start 2024-03-11T19:39:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.193+0000 7f6340789740 1 -- start start 2024-03-11T19:39:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.193+0000 7f6340789740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e38282ae0 0x555e382a80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.193+0000 7f6340789740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e382834d0 0x555e382b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.193+0000 7f6340789740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e382b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.193+0000 7f6340789740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e38283bc0 con 0x555e38288640 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.193+0000 7f6340789740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3827a850 con 0x555e38282ae0 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.193+0000 7f6340789740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e38271370 con 0x555e382834d0 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e382834d0 0x555e382b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e382834d0 0x555e382b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53110/0 (socket says 172.21.15.83:53110) 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6336ffd640 1 -- 172.21.15.83:0/2480359903 learned_addr learned my addr 172.21.15.83:0/2480359903 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6337fff640 1 --2- 172.21.15.83:0/2480359903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e38282ae0 0x555e382a80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6337fff640 1 --2- 172.21.15.83:0/2480359903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e382b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6336ffd640 1 -- 172.21.15.83:0/2480359903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e38282ae0 msgr2=0x555e382a80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6336ffd640 1 --2- 172.21.15.83:0/2480359903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e38282ae0 0x555e382a80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6336ffd640 1 -- 172.21.15.83:0/2480359903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 msgr2=0x555e382b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6336ffd640 1 --2- 172.21.15.83:0/2480359903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e382b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6336ffd640 1 -- 172.21.15.83:0/2480359903 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e3817dac0 con 0x555e382834d0 2024-03-11T19:39:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6337fff640 1 --2- 172.21.15.83:0/2480359903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e38282ae0 0x555e382a80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:39:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.194+0000 7f6336ffd640 1 --2- 172.21.15.83:0/2480359903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e382834d0 0x555e382b1600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f632c009230 tx=0x7f632c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b2b44037e50d7a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.195+0000 7f631d7fa640 1 -- 172.21.15.83:0/2480359903 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c0037f0 con 0x555e382834d0 2024-03-11T19:39:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.195+0000 7f631d7fa640 1 -- 172.21.15.83:0/2480359903 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f632c004df0 con 0x555e382834d0 2024-03-11T19:39:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.196+0000 7f631d7fa640 1 -- 172.21.15.83:0/2480359903 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c03f6a0 con 0x555e382834d0 2024-03-11T19:39:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.196+0000 7f6340789740 1 -- 172.21.15.83:0/2480359903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e382834d0 msgr2=0x555e382b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.196+0000 7f6340789740 1 --2- 172.21.15.83:0/2480359903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e382834d0 0x555e382b1600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f632c009230 tx=0x7f632c004610 comp rx=0 tx=0).stop 2024-03-11T19:39:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.196+0000 7f6340789740 1 -- 172.21.15.83:0/2480359903 shutdown_connections 2024-03-11T19:39:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.196+0000 7f6340789740 1 --2- 172.21.15.83:0/2480359903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e382b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.196+0000 7f6340789740 1 --2- 172.21.15.83:0/2480359903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e382834d0 0x555e382b1600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.196+0000 7f6340789740 1 --2- 172.21.15.83:0/2480359903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e38282ae0 0x555e382a80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.196+0000 7f6340789740 1 -- 172.21.15.83:0/2480359903 >> 172.21.15.83:0/2480359903 conn(0x555e382757f0 msgr2=0x555e38270840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.197+0000 7f6340789740 1 -- 172.21.15.83:0/2480359903 shutdown_connections 2024-03-11T19:39:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.197+0000 7f6340789740 1 -- 172.21.15.83:0/2480359903 wait complete. 2024-03-11T19:39:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.198+0000 7f6340789740 1 -- start start 2024-03-11T19:39:34.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.198+0000 7f6340789740 1 Processor -- start 2024-03-11T19:39:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.198+0000 7f6340789740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e382f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.198+0000 7f6340789740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 0x555e382f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.198+0000 7f6340789740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e382f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.198+0000 7f6340789740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e382b1f10 con 0x555e38288640 2024-03-11T19:39:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.198+0000 7f6340789740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e382f7280 con 0x555e382834d0 2024-03-11T19:39:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.198+0000 7f6340789740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e382f7440 con 0x555e38282ae0 2024-03-11T19:39:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.198+0000 7f6340789740 10 client.0 _ll_register_callbacks cb 0x7f6330009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6337fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e382f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6337fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e382f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53118/0 (socket says 172.21.15.83:53118) 2024-03-11T19:39:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6337fff640 1 -- 172.21.15.83:0/760848780 learned_addr learned my addr 172.21.15.83:0/760848780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6337fff640 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e382f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6336ffd640 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 0x555e382f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6337fff640 1 -- 172.21.15.83:0/760848780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 msgr2=0x555e382f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6337fff640 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 0x555e382f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6337fff640 1 -- 172.21.15.83:0/760848780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 msgr2=0x555e382f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6337fff640 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e382f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6337fff640 1 -- 172.21.15.83:0/760848780 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e381e1610 con 0x555e38282ae0 2024-03-11T19:39:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6336ffd640 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 0x555e382f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:39:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6337fff640 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e382f22a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f633000c420 tx=0x7f633000a4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd7c56bde692dc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.199+0000 7f6316ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:39:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.200+0000 7f6316ffd640 1 -- 172.21.15.83:0/760848780 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330011040 con 0x555e38282ae0 2024-03-11T19:39:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.200+0000 7f6316ffd640 1 -- 172.21.15.83:0/760848780 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f633002d030 con 0x555e38282ae0 2024-03-11T19:39:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.200+0000 7f6340789740 1 -- 172.21.15.83:0/760848780 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e37ee27e0 con 0x555e38282ae0 2024-03-11T19:39:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.200+0000 7f6316ffd640 1 -- 172.21.15.83:0/760848780 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f633000ae90 con 0x555e38282ae0 2024-03-11T19:39:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.200+0000 7f6340789740 10 client.24991 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.200+0000 7f6340789740 1 -- 172.21.15.83:0/760848780 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555e381a36a0 con 0x555e38282ae0 2024-03-11T19:39:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.200+0000 7f6316ffd640 1 -- 172.21.15.83:0/760848780 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6330036030 con 0x555e38282ae0 2024-03-11T19:39:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.201+0000 7f6316ffd640 1 -- 172.21.15.83:0/760848780 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6330008040 con 0x555e38282ae0 2024-03-11T19:39:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.201+0000 7f6340789740 10 client.24991 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.201+0000 7f6340789740 10 client.24991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.201+0000 7f6340789740 20 client.24991 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.201+0000 7f6340789740 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555e382f8df0 0x555e382fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.201+0000 7f6340789740 4 client.24991 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:39:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.201+0000 7f6340789740 1 -- 172.21.15.83:0/760848780 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e37ed14e0 con 0x555e382f8df0 2024-03-11T19:39:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.201+0000 7f6336ffd640 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555e382f8df0 0x555e382fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.202+0000 7f6316ffd640 10 client.24991 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.202+0000 7f6336ffd640 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555e382f8df0 0x555e382fb210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97d12c03ebff2c6d server_cookie=b93e06acc95b4036 in_seq=0 out_seq=0 2024-03-11T19:39:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.203+0000 7f6316ffd640 1 -- 172.21.15.83:0/760848780 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555e37ed14e0 con 0x555e382f8df0 2024-03-11T19:39:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.203+0000 7f6316ffd640 10 client.24991 handle_command_reply: tid=0 2024-03-11T19:39:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.203+0000 7f6340789740 1 client.24991 shutdown 2024-03-11T19:39:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.203+0000 7f6340789740 1 -- 172.21.15.83:0/760848780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 msgr2=0x555e382f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.203+0000 7f6340789740 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e382f22a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f633000c420 tx=0x7f633000a4a0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.203+0000 7f6340789740 1 -- 172.21.15.83:0/760848780 shutdown_connections 2024-03-11T19:39:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.203+0000 7f6340789740 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555e382f8df0 0x555e382fb210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.203+0000 7f6340789740 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e38288640 0x555e382f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.203+0000 7f6340789740 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e382834d0 0x555e382f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.204+0000 7f6340789740 1 --2- 172.21.15.83:0/760848780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e38282ae0 0x555e382f22a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.204+0000 7f6340789740 1 -- 172.21.15.83:0/760848780 >> 172.21.15.83:0/760848780 conn(0x555e382757f0 msgr2=0x555e38284610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.204+0000 7f6316ffd640 0 client.24991 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.205+0000 7f6340789740 1 -- 172.21.15.83:0/760848780 shutdown_connections 2024-03-11T19:39:34.210 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:39:34.210 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:39:34.210 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:39:34.210 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:39:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.206+0000 7f6340789740 1 -- 172.21.15.83:0/760848780 wait complete. 2024-03-11T19:39:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.206+0000 7f6340789740 20 client.24991 trim_cache size 0 max 16384 2024-03-11T19:39:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.207+0000 7f633dad4640 1 -- 172.21.15.83:0/2265072558 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f63200a1c00 msgr2=0x7f63200a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.207+0000 7f633dad4640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f63200a1c00 0x7f63200a40c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6328096d60 tx=0x7f6328000bc0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.207+0000 7f633dad4640 1 -- 172.21.15.83:0/2265072558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6338101180 msgr2=0x7f6338199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.207+0000 7f633dad4640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6338101180 0x7f6338199440 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f63381021d0 tx=0x7f632800dde0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.207+0000 7f633dad4640 1 -- 172.21.15.83:0/2265072558 shutdown_connections 2024-03-11T19:39:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.207+0000 7f633dad4640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f63200a1c00 0x7f63200a40c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.207+0000 7f633dad4640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6338102d30 0x7f63381a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.207+0000 7f633dad4640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338102370 0x7f6338199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.207+0000 7f633dad4640 1 --2- 172.21.15.83:0/2265072558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6338101180 0x7f6338199440 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.207+0000 7f633dad4640 1 -- 172.21.15.83:0/2265072558 >> 172.21.15.83:0/2265072558 conn(0x7f63380fc6f0 msgr2=0x7f6338105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.208+0000 7f633dad4640 1 -- 172.21.15.83:0/2265072558 shutdown_connections 2024-03-11T19:39:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.208+0000 7f633dad4640 1 -- 172.21.15.83:0/2265072558 wait complete. 2024-03-11T19:39:34.229 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:39:34.230 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:39:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.432+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/2663812222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dcc100a20 msgr2=0x7f1dcc100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.432+0000 7f1dd3b8a640 1 --2- 172.21.15.83:0/2663812222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dcc100a20 0x7f1dcc100e80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00a970 tx=0x7f1dbc02ced0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.433+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/2663812222 shutdown_connections 2024-03-11T19:39:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.433+0000 7f1dd3b8a640 1 --2- 172.21.15.83:0/2663812222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dcc1013c0 0x7f1dcc107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.433+0000 7f1dd3b8a640 1 --2- 172.21.15.83:0/2663812222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dcc100a20 0x7f1dcc100e80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.433+0000 7f1dd3b8a640 1 --2- 172.21.15.83:0/2663812222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dcc0ff970 0x7f1dcc0ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.433+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/2663812222 >> 172.21.15.83:0/2663812222 conn(0x7f1dcc0fa980 msgr2=0x7f1dcc0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.433+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/2663812222 shutdown_connections 2024-03-11T19:39:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.433+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/2663812222 wait complete. 2024-03-11T19:39:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.434+0000 7f1dd3b8a640 1 Processor -- start 2024-03-11T19:39:34.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.435+0000 7f1dd3b8a640 1 -- start start 2024-03-11T19:39:34.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.435+0000 7f1dd3b8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dcc0ff970 0x7f1dcc1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.435+0000 7f1dd3b8a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dcc100a20 0x7f1dcc199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.435+0000 7f1dd3b8a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dcc1013c0 0x7f1dcc1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.435+0000 7f1dd3b8a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc1a0fe0 con 0x7f1dcc0ff970 2024-03-11T19:39:34.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.435+0000 7f1dd3b8a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc1a1150 con 0x7f1dcc100a20 2024-03-11T19:39:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.435+0000 7f1dd3b8a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc1a1310 con 0x7f1dcc1013c0 2024-03-11T19:39:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.436+0000 7f1dd2127640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dcc1013c0 0x7f1dcc1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.436+0000 7f1dd2127640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dcc1013c0 0x7f1dcc1a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53132/0 (socket says 172.21.15.83:53132) 2024-03-11T19:39:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.436+0000 7f1dd2127640 1 -- 172.21.15.83:0/4160314805 learned_addr learned my addr 172.21.15.83:0/4160314805 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.436+0000 7f1dd1926640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dcc0ff970 0x7f1dcc1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.436+0000 7f1dd2127640 1 -- 172.21.15.83:0/4160314805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dcc100a20 msgr2=0x7f1dcc199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.436+0000 7f1dd2127640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dcc100a20 0x7f1dcc199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.436+0000 7f1dd2127640 1 -- 172.21.15.83:0/4160314805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dcc0ff970 msgr2=0x7f1dcc1994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.436+0000 7f1dd2127640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dcc0ff970 0x7f1dcc1994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.436+0000 7f1dd2127640 1 -- 172.21.15.83:0/4160314805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc4009900 con 0x7f1dcc1013c0 2024-03-11T19:39:34.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.436+0000 7f1dd1926640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dcc0ff970 0x7f1dcc1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:39:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.437+0000 7f1dd2127640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dcc1013c0 0x7f1dcc1a0aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400bcc0 tx=0x7f1dc400a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.437+0000 7f1dc2ffd640 1 -- 172.21.15.83:0/4160314805 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4019070 con 0x7f1dcc1013c0 2024-03-11T19:39:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.437+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/4160314805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dbc00a5a0 con 0x7f1dcc1013c0 2024-03-11T19:39:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.437+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/4160314805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc1906a0 con 0x7f1dcc1013c0 2024-03-11T19:39:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.437+0000 7f1dc2ffd640 1 -- 172.21.15.83:0/4160314805 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dc4003e50 con 0x7f1dcc1013c0 2024-03-11T19:39:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.437+0000 7f1dc2ffd640 1 -- 172.21.15.83:0/4160314805 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4019070 con 0x7f1dcc1013c0 2024-03-11T19:39:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.439+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/4160314805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db0005290 con 0x7f1dcc1013c0 2024-03-11T19:39:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.441+0000 7f1dc2ffd640 1 -- 172.21.15.83:0/4160314805 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1dc400a040 con 0x7f1dcc1013c0 2024-03-11T19:39:34.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.442+0000 7f1dc2ffd640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1db80a1a90 0x7f1db80a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:34.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.442+0000 7f1dc2ffd640 1 -- 172.21.15.83:0/4160314805 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1dc40d52a0 con 0x7f1dcc1013c0 2024-03-11T19:39:34.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.443+0000 7f1dd1926640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1db80a1a90 0x7f1db80a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.445+0000 7f1dd1926640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1db80a1a90 0x7f1db80a3f50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8006fd0 tx=0x7f1dc8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:34.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.447+0000 7f1dc2ffd640 1 -- 172.21.15.83:0/4160314805 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1dc4057d20 con 0x7f1dcc1013c0 2024-03-11T19:39:34.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.858+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/4160314805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1db0005810 con 0x7f1dcc1013c0 2024-03-11T19:39:34.864 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:39:34.864 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:39:20.446302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:39:34.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.859+0000 7f1dc2ffd640 1 -- 172.21.15.83:0/4160314805 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11352 (secure 0 0 0) 0x7f1dc409e560 con 0x7f1dcc1013c0 2024-03-11T19:39:34.864 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-03-11T19:39:34.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.863+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/4160314805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1db80a1a90 msgr2=0x7f1db80a3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.863+0000 7f1dd3b8a640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1db80a1a90 0x7f1db80a3f50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8006fd0 tx=0x7f1dc8008040 comp rx=0 tx=0).stop 2024-03-11T19:39:34.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.863+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/4160314805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dcc1013c0 msgr2=0x7f1dcc1a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:34.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.863+0000 7f1dd3b8a640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dcc1013c0 0x7f1dcc1a0aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400bcc0 tx=0x7f1dc400a6c0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.864+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/4160314805 shutdown_connections 2024-03-11T19:39:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.864+0000 7f1dd3b8a640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1db80a1a90 0x7f1db80a3f50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.864+0000 7f1dd3b8a640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dcc1013c0 0x7f1dcc1a0aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.864+0000 7f1dd3b8a640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dcc100a20 0x7f1dcc199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.864+0000 7f1dd3b8a640 1 --2- 172.21.15.83:0/4160314805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dcc0ff970 0x7f1dcc1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.864+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/4160314805 >> 172.21.15.83:0/4160314805 conn(0x7f1dcc0fa980 msgr2=0x7f1dcc103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.865+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/4160314805 shutdown_connections 2024-03-11T19:39:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:34.866+0000 7f1dd3b8a640 1 -- 172.21.15.83:0/4160314805 wait complete. 2024-03-11T19:39:34.886 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:39:34.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:39:35.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.054+0000 7f0aa627b640 1 -- 172.21.15.83:0/4249033528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a980990f0 msgr2=0x7f0a98099570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.054+0000 7f0aa627b640 1 --2- 172.21.15.83:0/4249033528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a980990f0 0x7f0a98099570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009890 tx=0x7f0a9c03cb80 comp rx=0 tx=0).stop 2024-03-11T19:39:35.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.054+0000 7f0aa627b640 1 -- 172.21.15.83:0/4249033528 shutdown_connections 2024-03-11T19:39:35.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.054+0000 7f0aa627b640 1 --2- 172.21.15.83:0/4249033528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a98099ab0 0x7f0a980a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.054+0000 7f0aa627b640 1 --2- 172.21.15.83:0/4249033528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a980990f0 0x7f0a98099570 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009890 tx=0x7f0a9c03cb80 comp rx=0 tx=0).stop 2024-03-11T19:39:35.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.054+0000 7f0aa627b640 1 --2- 172.21.15.83:0/4249033528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98097f20 0x7f0a98098360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.054+0000 7f0aa627b640 1 -- 172.21.15.83:0/4249033528 >> 172.21.15.83:0/4249033528 conn(0x7f0a980934e0 msgr2=0x7f0a98095920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.056+0000 7f0aa627b640 1 -- 172.21.15.83:0/4249033528 shutdown_connections 2024-03-11T19:39:35.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.056+0000 7f0aa627b640 1 -- 172.21.15.83:0/4249033528 wait complete. 2024-03-11T19:39:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.057+0000 7f0aa627b640 1 Processor -- start 2024-03-11T19:39:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.057+0000 7f0aa627b640 1 -- start start 2024-03-11T19:39:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.057+0000 7f0aa627b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a98097f20 0x7f0a98129de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.057+0000 7f0aa627b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98099ab0 0x7f0a9812a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.057+0000 7f0aa627b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a9812f380 0x7f0a98131770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.057+0000 7f0aa627b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98131cb0 con 0x7f0a98099ab0 2024-03-11T19:39:35.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.057+0000 7f0aa627b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98131e20 con 0x7f0a9812f380 2024-03-11T19:39:35.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.057+0000 7f0aa627b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98131fe0 con 0x7f0a98097f20 2024-03-11T19:39:35.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.058+0000 7f0aa5279640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a98097f20 0x7f0a98129de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.058+0000 7f0aa5279640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a98097f20 0x7f0a98129de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53148/0 (socket says 172.21.15.83:53148) 2024-03-11T19:39:35.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.058+0000 7f0aa5279640 1 -- 172.21.15.83:0/1401665768 learned_addr learned my addr 172.21.15.83:0/1401665768 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.058+0000 7f0aa4a78640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98099ab0 0x7f0a9812a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.058+0000 7f0aa5279640 1 -- 172.21.15.83:0/1401665768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a9812f380 msgr2=0x7f0a98131770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-11T19:39:35.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.058+0000 7f0aa5279640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a9812f380 0x7f0a98131770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.058+0000 7f0aa5279640 1 -- 172.21.15.83:0/1401665768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98099ab0 msgr2=0x7f0a9812a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.058+0000 7f0aa5279640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98099ab0 0x7f0a9812a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.058+0000 7f0aa5279640 1 -- 172.21.15.83:0/1401665768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c009490 con 0x7f0a98097f20 2024-03-11T19:39:35.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.059+0000 7f0aa4a78640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98099ab0 0x7f0a9812a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:39:35.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.059+0000 7f0aa5279640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a98097f20 0x7f0a98129de0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000e9f0 tx=0x7f0a9000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.059+0000 7f0a967fc640 1 -- 172.21.15.83:0/1401665768 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9000ccc0 con 0x7f0a98097f20 2024-03-11T19:39:35.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.059+0000 7f0a967fc640 1 -- 172.21.15.83:0/1401665768 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a9000ce20 con 0x7f0a98097f20 2024-03-11T19:39:35.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.059+0000 7f0aa627b640 1 -- 172.21.15.83:0/1401665768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a981322c0 con 0x7f0a98097f20 2024-03-11T19:39:35.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.059+0000 7f0aa627b640 1 -- 172.21.15.83:0/1401665768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a981327f0 con 0x7f0a98097f20 2024-03-11T19:39:35.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.060+0000 7f0a967fc640 1 -- 172.21.15.83:0/1401665768 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90010430 con 0x7f0a98097f20 2024-03-11T19:39:35.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.063+0000 7f0aaa182740 1 Processor -- start 2024-03-11T19:39:35.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.063+0000 7f0a967fc640 1 -- 172.21.15.83:0/1401665768 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0a90004020 con 0x7f0a98097f20 2024-03-11T19:39:35.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.063+0000 7f0aaa182740 1 -- start start 2024-03-11T19:39:35.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6e70 0x555c0ffab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aaa182740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0ff62960 con 0x555c0ffabfe0 2024-03-11T19:39:35.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aaa182740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0ff16310 con 0x555c0ffa6e70 2024-03-11T19:39:35.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aaa182740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0ff13840 con 0x555c0ffa6480 2024-03-11T19:39:35.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa5a7a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa5a7a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53158/0 (socket says 172.21.15.83:53158) 2024-03-11T19:39:35.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa5a7a640 1 -- 172.21.15.83:0/3866136942 learned_addr learned my addr 172.21.15.83:0/3866136942 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa5279640 1 --2- 172.21.15.83:0/3866136942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6e70 0x555c0ffab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa5279640 1 -- 172.21.15.83:0/3866136942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 msgr2=0x555c0ffa68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa5279640 1 --2- 172.21.15.83:0/3866136942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa5279640 1 -- 172.21.15.83:0/3866136942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 msgr2=0x555c0ffae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa4a78640 1 --2- 172.21.15.83:0/3866136942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa5279640 1 --2- 172.21.15.83:0/3866136942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa5279640 1 -- 172.21.15.83:0/3866136942 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c0fed1060 con 0x555c0ffa6e70 2024-03-11T19:39:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa5a7a640 1 --2- 172.21.15.83:0/3866136942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:39:35.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.064+0000 7f0aa4a78640 1 --2- 172.21.15.83:0/3866136942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:39:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.066+0000 7f0aa5279640 1 --2- 172.21.15.83:0/3866136942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6e70 0x555c0ffab970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a900cb2b0 tx=0x7f0a900d38b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d78ae6c465cbb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.066+0000 7f0a967fc640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a8c0a1ab0 0x7f0a8c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.066+0000 7f0aa5a7a640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a8c0a1ab0 0x7f0a8c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.066+0000 7f0a967fc640 1 -- 172.21.15.83:0/1401665768 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0a90014070 con 0x7f0a98097f20 2024-03-11T19:39:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.066+0000 7f0a8b7fe640 1 -- 172.21.15.83:0/3866136942 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90096970 con 0x555c0ffa6e70 2024-03-11T19:39:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.066+0000 7f0a8b7fe640 1 -- 172.21.15.83:0/3866136942 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a90095010 con 0x555c0ffa6e70 2024-03-11T19:39:35.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.066+0000 7f0a8b7fe640 1 -- 172.21.15.83:0/3866136942 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9009e820 con 0x555c0ffa6e70 2024-03-11T19:39:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.068+0000 7f0aaa182740 1 -- 172.21.15.83:0/3866136942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6e70 msgr2=0x555c0ffab970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.068+0000 7f0aaa182740 1 --2- 172.21.15.83:0/3866136942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6e70 0x555c0ffab970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a900cb2b0 tx=0x7f0a900d38b0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.068+0000 7f0aa5a7a640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a8c0a1ab0 0x7f0a8c0a3f70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0066a40 tx=0x7f0aa0078040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.068+0000 7f0aaa182740 1 -- 172.21.15.83:0/3866136942 shutdown_connections 2024-03-11T19:39:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.068+0000 7f0aaa182740 1 --2- 172.21.15.83:0/3866136942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.068+0000 7f0aaa182740 1 --2- 172.21.15.83:0/3866136942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6e70 0x555c0ffab970 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.068+0000 7f0aaa182740 1 --2- 172.21.15.83:0/3866136942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.068+0000 7f0aaa182740 1 -- 172.21.15.83:0/3866136942 >> 172.21.15.83:0/3866136942 conn(0x555c0ffa19f0 msgr2=0x555c0ffa1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.068+0000 7f0aaa182740 1 -- 172.21.15.83:0/3866136942 shutdown_connections 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.068+0000 7f0aaa182740 1 -- 172.21.15.83:0/3866136942 wait complete. 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.069+0000 7f0aaa182740 1 -- start start 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.069+0000 7f0aaa182740 1 Processor -- start 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.069+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6480 0x555c0ffc16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.069+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6e70 0x555c0ffc1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.069+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffc8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.069+0000 7f0aaa182740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0ff0f020 con 0x555c0ffabfe0 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.069+0000 7f0aaa182740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0fbe2430 con 0x555c0ffa6480 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.069+0000 7f0aaa182740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0fb9ee00 con 0x555c0ffa6e70 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.069+0000 7f0aaa182740 10 client.0 _ll_register_callbacks cb 0x555c0feb6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.070+0000 7f0aa4a78640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6e70 0x555c0ffc1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.070+0000 7f0aa4a78640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6e70 0x555c0ffc1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53160/0 (socket says 172.21.15.83:53160) 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.070+0000 7f0aa4a78640 1 -- 172.21.15.83:0/3099279596 learned_addr learned my addr 172.21.15.83:0/3099279596 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.070+0000 7f0aa5a7a640 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffc8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.070+0000 7f0aa5279640 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6480 0x555c0ffc16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.070+0000 7f0aa5a7a640 1 -- 172.21.15.83:0/3099279596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6e70 msgr2=0x555c0ffc1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.070+0000 7f0aa5a7a640 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6e70 0x555c0ffc1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.070+0000 7f0aa5a7a640 1 -- 172.21.15.83:0/3099279596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6480 msgr2=0x555c0ffc16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.070+0000 7f0aa5a7a640 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6480 0x555c0ffc16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.070+0000 7f0aa5a7a640 1 -- 172.21.15.83:0/3099279596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a981327f0 con 0x555c0ffabfe0 2024-03-11T19:39:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.071+0000 7f0aa5a7a640 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffc8c20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0064670 tx=0x7f0aa0067020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59adb0e4cdbbb3c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.072+0000 7f0a88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:39:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.072+0000 7f0a88ff9640 1 -- 172.21.15.83:0/3099279596 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0068250 con 0x555c0ffabfe0 2024-03-11T19:39:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.072+0000 7f0aaa182740 1 -- 172.21.15.83:0/3099279596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c0fc0e7e0 con 0x555c0ffabfe0 2024-03-11T19:39:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.072+0000 7f0a88ff9640 1 -- 172.21.15.83:0/3099279596 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0aa007de10 con 0x555c0ffabfe0 2024-03-11T19:39:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.073+0000 7f0a88ff9640 1 -- 172.21.15.83:0/3099279596 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa008f030 con 0x555c0ffabfe0 2024-03-11T19:39:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.073+0000 7f0aaa182740 10 client.15402 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:35.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.073+0000 7f0aaa182740 1 -- 172.21.15.83:0/3099279596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555c0ff0d6a0 con 0x555c0ffabfe0 2024-03-11T19:39:35.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.073+0000 7f0a88ff9640 1 -- 172.21.15.83:0/3099279596 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555c0fc0e7e0 con 0x555c0ffabfe0 2024-03-11T19:39:35.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.074+0000 7f0a88ff9640 1 -- 172.21.15.83:0/3099279596 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0aa0087020 con 0x555c0ffabfe0 2024-03-11T19:39:35.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.074+0000 7f0aaa182740 10 client.15402 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:35.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.074+0000 7f0aaa182740 10 client.15402 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:39:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.074+0000 7f0aaa182740 20 client.15402 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.074+0000 7f0aaa182740 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555c0ffb44c0 0x555c0ffb68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.074+0000 7f0aaa182740 4 client.15402 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:39:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.074+0000 7f0aaa182740 1 -- 172.21.15.83:0/3099279596 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c0fc4b720 con 0x555c0ffb44c0 2024-03-11T19:39:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.074+0000 7f0aa5279640 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555c0ffb44c0 0x555c0ffb68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.075+0000 7f0a88ff9640 10 client.15402 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:39:35.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.075+0000 7f0aa5279640 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555c0ffb44c0 0x555c0ffb68e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec59928aa7e5001b server_cookie=e6f0ec1e0a7b59b2 in_seq=0 out_seq=0 2024-03-11T19:39:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.076+0000 7f0a88ff9640 1 -- 172.21.15.83:0/3099279596 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c0fc4b720 con 0x555c0ffb44c0 2024-03-11T19:39:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.076+0000 7f0a88ff9640 10 client.15402 handle_command_reply: tid=0 2024-03-11T19:39:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.076+0000 7f0aaa182740 1 client.15402 shutdown 2024-03-11T19:39:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.077+0000 7f0aaa182740 1 -- 172.21.15.83:0/3099279596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 msgr2=0x555c0ffc8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.077+0000 7f0aaa182740 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffc8c20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0064670 tx=0x7f0aa0067020 comp rx=0 tx=0).stop 2024-03-11T19:39:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.078+0000 7f0aaa182740 1 -- 172.21.15.83:0/3099279596 shutdown_connections 2024-03-11T19:39:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.078+0000 7f0aaa182740 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555c0ffb44c0 0x555c0ffb68e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.078+0000 7f0aaa182740 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffc8c20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.078+0000 7f0aaa182740 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6e70 0x555c0ffc1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.078+0000 7f0aaa182740 1 --2- 172.21.15.83:0/3099279596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6480 0x555c0ffc16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.078+0000 7f0aaa182740 1 -- 172.21.15.83:0/3099279596 >> 172.21.15.83:0/3099279596 conn(0x555c0ffa19f0 msgr2=0x555c0ffa0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.078+0000 7f0a88ff9640 0 client.15402 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:39:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.078+0000 7f0aaa182740 1 -- 172.21.15.83:0/3099279596 shutdown_connections 2024-03-11T19:39:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.078+0000 7f0aaa182740 1 -- 172.21.15.83:0/3099279596 wait complete. 2024-03-11T19:39:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.078+0000 7f0aaa182740 20 client.15402 trim_cache size 0 max 16384 2024-03-11T19:39:35.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.101+0000 7f0aaa182740 1 Processor -- start 2024-03-11T19:39:35.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.101+0000 7f0aaa182740 1 -- start start 2024-03-11T19:39:35.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.102+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.102+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffa6e70 0x555c0ffc8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.102+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffabfe0 0x555c0ffa88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.102+0000 7f0aaa182740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0fe94130 con 0x555c0ffa6e70 2024-03-11T19:39:35.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.102+0000 7f0aaa182740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0ffaf440 con 0x555c0ffabfe0 2024-03-11T19:39:35.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.102+0000 7f0aaa182740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0ffa7d70 con 0x555c0ffa6480 2024-03-11T19:39:35.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.102+0000 7f0aa5279640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.102+0000 7f0aa5279640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53174/0 (socket says 172.21.15.83:53174) 2024-03-11T19:39:35.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.102+0000 7f0aa5279640 1 -- 172.21.15.83:0/851292317 learned_addr learned my addr 172.21.15.83:0/851292317 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.102+0000 7f0aa5a7a640 1 --2- 172.21.15.83:0/851292317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffabfe0 0x555c0ffa88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.103+0000 7f0aa5279640 1 -- 172.21.15.83:0/851292317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffabfe0 msgr2=0x555c0ffa88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.103+0000 7f0aa5279640 1 --2- 172.21.15.83:0/851292317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffabfe0 0x555c0ffa88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.103+0000 7f0aa5279640 1 -- 172.21.15.83:0/851292317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffa6e70 msgr2=0x555c0ffc8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.103+0000 7f0aa5279640 1 --2- 172.21.15.83:0/851292317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffa6e70 0x555c0ffc8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.103+0000 7f0aa5279640 1 -- 172.21.15.83:0/851292317 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c0fea9c20 con 0x555c0ffa6480 2024-03-11T19:39:35.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.103+0000 7f0aa5279640 1 --2- 172.21.15.83:0/851292317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa8390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000e9c0 tx=0x7f0a900d3060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe960ba05f67c55 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.103+0000 7f0a8affd640 1 -- 172.21.15.83:0/851292317 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90083ea0 con 0x555c0ffa6480 2024-03-11T19:39:35.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.104+0000 7f0a8affd640 1 -- 172.21.15.83:0/851292317 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a9009a4a0 con 0x555c0ffa6480 2024-03-11T19:39:35.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.104+0000 7f0a8affd640 1 -- 172.21.15.83:0/851292317 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a900a2e70 con 0x555c0ffa6480 2024-03-11T19:39:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.105+0000 7f0aaa182740 1 -- 172.21.15.83:0/851292317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 msgr2=0x555c0ffa8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.105+0000 7f0aaa182740 1 --2- 172.21.15.83:0/851292317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa8390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000e9c0 tx=0x7f0a900d3060 comp rx=0 tx=0).stop 2024-03-11T19:39:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.105+0000 7f0aaa182740 1 -- 172.21.15.83:0/851292317 shutdown_connections 2024-03-11T19:39:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.105+0000 7f0aaa182740 1 --2- 172.21.15.83:0/851292317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffabfe0 0x555c0ffa88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.105+0000 7f0aaa182740 1 --2- 172.21.15.83:0/851292317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffa6e70 0x555c0ffc8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.105+0000 7f0aaa182740 1 --2- 172.21.15.83:0/851292317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffa8390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.105+0000 7f0aaa182740 1 -- 172.21.15.83:0/851292317 >> 172.21.15.83:0/851292317 conn(0x555c0ffa19f0 msgr2=0x555c0ffbb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.105+0000 7f0aaa182740 1 -- 172.21.15.83:0/851292317 shutdown_connections 2024-03-11T19:39:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.105+0000 7f0aaa182740 1 -- 172.21.15.83:0/851292317 wait complete. 2024-03-11T19:39:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.106+0000 7f0aaa182740 1 -- start start 2024-03-11T19:39:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.106+0000 7f0aaa182740 1 Processor -- start 2024-03-11T19:39:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.106+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffc67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.106+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6e70 0x555c0ffc6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.106+0000 7f0aaa182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffc7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.106+0000 7f0aaa182740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0ffc8230 con 0x555c0ffabfe0 2024-03-11T19:39:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.106+0000 7f0aaa182740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0ffc83a0 con 0x555c0ffa6e70 2024-03-11T19:39:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.106+0000 7f0aaa182740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0ffc8560 con 0x555c0ffa6480 2024-03-11T19:39:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.106+0000 7f0aaa182740 10 client.0 _ll_register_callbacks cb 0x7f0a900d0ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.107+0000 7f0aa5a7a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffc7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.107+0000 7f0aa5a7a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffc7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39602/0 (socket says 172.21.15.83:39602) 2024-03-11T19:39:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.107+0000 7f0aa5a7a640 1 -- 172.21.15.83:0/2076157262 learned_addr learned my addr 172.21.15.83:0/2076157262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.107+0000 7f0aa4a78640 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6e70 0x555c0ffc6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.107+0000 7f0aa5279640 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffc67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.107+0000 7f0aa5a7a640 1 -- 172.21.15.83:0/2076157262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 msgr2=0x555c0ffc67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.107+0000 7f0aa5a7a640 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffc67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.107+0000 7f0aa5a7a640 1 -- 172.21.15.83:0/2076157262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6e70 msgr2=0x555c0ffc6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.107+0000 7f0aa5a7a640 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6e70 0x555c0ffc6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.107+0000 7f0aa5a7a640 1 -- 172.21.15.83:0/2076157262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c0fed1060 con 0x555c0ffabfe0 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.108+0000 7f0aa5a7a640 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffc7cf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0089450 tx=0x7f0aa0068720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1fc92479214662 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.108+0000 7f0a88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.108+0000 7f0a88ff9640 1 -- 172.21.15.83:0/2076157262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa008f070 con 0x555c0ffabfe0 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.108+0000 7f0aaa182740 1 -- 172.21.15.83:0/2076157262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a90095010 con 0x555c0ffabfe0 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.108+0000 7f0a88ff9640 1 -- 172.21.15.83:0/2076157262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0aa0090070 con 0x555c0ffabfe0 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.108+0000 7f0a88ff9640 1 -- 172.21.15.83:0/2076157262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa007de10 con 0x555c0ffabfe0 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.108+0000 7f0aaa182740 10 client.15405 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.108+0000 7f0aaa182740 1 -- 172.21.15.83:0/2076157262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555c0fecf8c0 con 0x555c0ffabfe0 2024-03-11T19:39:35.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.109+0000 7f0a88ff9640 1 -- 172.21.15.83:0/2076157262 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a90095010 con 0x555c0ffabfe0 2024-03-11T19:39:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.109+0000 7f0a88ff9640 1 -- 172.21.15.83:0/2076157262 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0aa0087040 con 0x555c0ffabfe0 2024-03-11T19:39:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.109+0000 7f0aaa182740 10 client.15405 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.109+0000 7f0aaa182740 10 client.15405 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:39:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.109+0000 7f0aaa182740 20 client.15405 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.109+0000 7f0aaa182740 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555c10036170 0x555c10038590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.109+0000 7f0aaa182740 4 client.15405 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:39:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.109+0000 7f0aaa182740 1 -- 172.21.15.83:0/2076157262 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c0fc9a890 con 0x555c10036170 2024-03-11T19:39:35.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.109+0000 7f0aa5279640 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555c10036170 0x555c10038590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.111+0000 7f0aa5279640 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555c10036170 0x555c10038590 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c970379aa6edea8 server_cookie=5afa7b49797fa3aa in_seq=0 out_seq=0 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.111+0000 7f0a88ff9640 10 client.15405 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.111+0000 7f0a88ff9640 1 -- 172.21.15.83:0/2076157262 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c0fc9a890 con 0x555c10036170 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.111+0000 7f0a88ff9640 10 client.15405 handle_command_reply: tid=0 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 1 client.15405 shutdown 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 1 -- 172.21.15.83:0/2076157262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 msgr2=0x555c0ffc7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffc7cf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0089450 tx=0x7f0aa0068720 comp rx=0 tx=0).stop 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 1 -- 172.21.15.83:0/2076157262 shutdown_connections 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555c10036170 0x555c10038590 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c0ffabfe0 0x555c0ffc7cf0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c0ffa6e70 0x555c0ffc6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 1 --2- 172.21.15.83:0/2076157262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c0ffa6480 0x555c0ffc67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 1 -- 172.21.15.83:0/2076157262 >> 172.21.15.83:0/2076157262 conn(0x555c0ffa19f0 msgr2=0x555c0ffa8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0a88ff9640 0 client.15405 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:39:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 1 -- 172.21.15.83:0/2076157262 shutdown_connections 2024-03-11T19:39:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 1 -- 172.21.15.83:0/2076157262 wait complete. 2024-03-11T19:39:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.112+0000 7f0aaa182740 20 client.15405 trim_cache size 0 max 16384 2024-03-11T19:39:35.118 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:39:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.114+0000 7f0aa627b640 1 -- 172.21.15.83:0/1401665768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a8c0a1ab0 msgr2=0x7f0a8c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.114+0000 7f0aa627b640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a8c0a1ab0 0x7f0a8c0a3f70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0066a40 tx=0x7f0aa0078040 comp rx=0 tx=0).stop 2024-03-11T19:39:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.114+0000 7f0aa627b640 1 -- 172.21.15.83:0/1401665768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a98097f20 msgr2=0x7f0a98129de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.114+0000 7f0aa627b640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a98097f20 0x7f0a98129de0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000e9f0 tx=0x7f0a9000eeb0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.114+0000 7f0aa627b640 1 -- 172.21.15.83:0/1401665768 shutdown_connections 2024-03-11T19:39:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.114+0000 7f0aa627b640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a8c0a1ab0 0x7f0a8c0a3f70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.114+0000 7f0aa627b640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a9812f380 0x7f0a98131770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.114+0000 7f0aa627b640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98099ab0 0x7f0a9812a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.114+0000 7f0aa627b640 1 --2- 172.21.15.83:0/1401665768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a98097f20 0x7f0a98129de0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.114+0000 7f0aa627b640 1 -- 172.21.15.83:0/1401665768 >> 172.21.15.83:0/1401665768 conn(0x7f0a980934e0 msgr2=0x7f0a98093e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.114+0000 7f0aa627b640 1 -- 172.21.15.83:0/1401665768 shutdown_connections 2024-03-11T19:39:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.115+0000 7f0aa627b640 1 -- 172.21.15.83:0/1401665768 wait complete. 2024-03-11T19:39:35.136 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:39:35.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:39:35.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.307+0000 7f0a249a1640 1 -- 172.21.15.83:0/590528449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a201000c0 msgr2=0x7f0a201004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.307+0000 7f0a249a1640 1 --2- 172.21.15.83:0/590528449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a201000c0 0x7f0a201004e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009860 tx=0x7f0a1002bb70 comp rx=0 tx=0).stop 2024-03-11T19:39:35.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.308+0000 7f0a249a1640 1 -- 172.21.15.83:0/590528449 shutdown_connections 2024-03-11T19:39:35.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.308+0000 7f0a249a1640 1 --2- 172.21.15.83:0/590528449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a20101c70 0x7f0a20108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.308+0000 7f0a249a1640 1 --2- 172.21.15.83:0/590528449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a201012b0 0x7f0a20101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.308+0000 7f0a249a1640 1 --2- 172.21.15.83:0/590528449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a201000c0 0x7f0a201004e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.308+0000 7f0a249a1640 1 -- 172.21.15.83:0/590528449 >> 172.21.15.83:0/590528449 conn(0x7f0a200fb630 msgr2=0x7f0a200fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.308+0000 7f0a249a1640 1 -- 172.21.15.83:0/590528449 shutdown_connections 2024-03-11T19:39:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.308+0000 7f0a249a1640 1 -- 172.21.15.83:0/590528449 wait complete. 2024-03-11T19:39:35.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.309+0000 7f0a249a1640 1 Processor -- start 2024-03-11T19:39:35.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.309+0000 7f0a249a1640 1 -- start start 2024-03-11T19:39:35.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a249a1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a201000c0 0x7f0a20199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a249a1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a201012b0 0x7f0a20199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a249a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a20101c70 0x7f0a201a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a249a1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a201a1010 con 0x7f0a201000c0 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a249a1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a201a1180 con 0x7f0a20101c70 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a201000c0 0x7f0a20199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a201000c0 0x7f0a20199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39616/0 (socket says 172.21.15.83:39616) 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a1e59c640 1 -- 172.21.15.83:0/3565509419 learned_addr learned my addr 172.21.15.83:0/3565509419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a201012b0 0x7f0a20199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a1ed9d640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a20101c70 0x7f0a201a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a249a1640 1 -- 172.21.15.83:0/3565509419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a201a1340 con 0x7f0a201012b0 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a1ed9d640 1 -- 172.21.15.83:0/3565509419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a201012b0 msgr2=0x7f0a20199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a1ed9d640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a201012b0 0x7f0a20199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a1ed9d640 1 -- 172.21.15.83:0/3565509419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a201000c0 msgr2=0x7f0a20199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a1ed9d640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a201000c0 0x7f0a20199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.310+0000 7f0a1ed9d640 1 -- 172.21.15.83:0/3565509419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a10009490 con 0x7f0a20101c70 2024-03-11T19:39:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.311+0000 7f0a1ed9d640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a20101c70 0x7f0a201a0ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009fd0 tx=0x7f0a1800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.311+0000 7f0a0f7fe640 1 -- 172.21.15.83:0/3565509419 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1800cd60 con 0x7f0a20101c70 2024-03-11T19:39:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.311+0000 7f0a0f7fe640 1 -- 172.21.15.83:0/3565509419 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a18005020 con 0x7f0a20101c70 2024-03-11T19:39:35.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.311+0000 7f0a249a1640 1 -- 172.21.15.83:0/3565509419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a201913a0 con 0x7f0a20101c70 2024-03-11T19:39:35.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.311+0000 7f0a249a1640 1 -- 172.21.15.83:0/3565509419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a201918b0 con 0x7f0a20101c70 2024-03-11T19:39:35.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.311+0000 7f0a0f7fe640 1 -- 172.21.15.83:0/3565509419 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a180186d0 con 0x7f0a20101c70 2024-03-11T19:39:35.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.314+0000 7f0a0f7fe640 1 -- 172.21.15.83:0/3565509419 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0a18018830 con 0x7f0a20101c70 2024-03-11T19:39:35.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.314+0000 7f0a27578740 1 Processor -- start 2024-03-11T19:39:35.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.315+0000 7f0a27578740 1 -- start start 2024-03-11T19:39:35.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.315+0000 7f0a27578740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7b270 0x559c74d7b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.315+0000 7f0a27578740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7bc60 0x559c74d80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.315+0000 7f0a27578740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d83260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.315+0000 7f0a27578740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c74d37960 con 0x559c74d7b270 2024-03-11T19:39:35.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.315+0000 7f0a27578740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c74ceb310 con 0x559c74d80dd0 2024-03-11T19:39:35.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.315+0000 7f0a27578740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c74ce8840 con 0x559c74d7bc60 2024-03-11T19:39:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.315+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7bc60 0x559c74d80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.315+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7bc60 0x559c74d80760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53210/0 (socket says 172.21.15.83:53210) 2024-03-11T19:39:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.315+0000 7f0a1dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7b270 0x559c74d7b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.315+0000 7f0a1e59c640 1 -- 172.21.15.83:0/2403132127 learned_addr learned my addr 172.21.15.83:0/2403132127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.316+0000 7f0a0f7fe640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a080a1c30 0x7f0a080a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.316+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/2403132127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d83260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.316+0000 7f0a0f7fe640 1 -- 172.21.15.83:0/3565509419 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0a18014070 con 0x7f0a20101c70 2024-03-11T19:39:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.316+0000 7f0a1e59c640 1 -- 172.21.15.83:0/2403132127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 msgr2=0x559c74d83260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.316+0000 7f0a1e59c640 1 --2- 172.21.15.83:0/2403132127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d83260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.316+0000 7f0a1e59c640 1 -- 172.21.15.83:0/2403132127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7b270 msgr2=0x559c74d7b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.316+0000 7f0a1e59c640 1 --2- 172.21.15.83:0/2403132127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7b270 0x559c74d7b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.316+0000 7f0a1e59c640 1 -- 172.21.15.83:0/2403132127 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c74ca6060 con 0x559c74d7bc60 2024-03-11T19:39:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.316+0000 7f0a1ed9d640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a080a1c30 0x7f0a080a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.316+0000 7f0a1e59c640 1 --2- 172.21.15.83:0/2403132127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7bc60 0x559c74d80760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a10032480 tx=0x7f0a100324b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19dfe2c30bc1b18f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.317+0000 7f0a0cff9640 1 -- 172.21.15.83:0/2403132127 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a100044b0 con 0x559c74d7bc60 2024-03-11T19:39:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.317+0000 7f0a0cff9640 1 -- 172.21.15.83:0/2403132127 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a10005500 con 0x559c74d7bc60 2024-03-11T19:39:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.317+0000 7f0a0cff9640 1 -- 172.21.15.83:0/2403132127 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1002c890 con 0x559c74d7bc60 2024-03-11T19:39:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.318+0000 7f0a27578740 1 -- 172.21.15.83:0/2403132127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7bc60 msgr2=0x559c74d80760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.318+0000 7f0a27578740 1 --2- 172.21.15.83:0/2403132127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7bc60 0x559c74d80760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a10032480 tx=0x7f0a100324b0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.318+0000 7f0a27578740 1 -- 172.21.15.83:0/2403132127 shutdown_connections 2024-03-11T19:39:35.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.318+0000 7f0a27578740 1 --2- 172.21.15.83:0/2403132127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d83260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.318+0000 7f0a27578740 1 --2- 172.21.15.83:0/2403132127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7bc60 0x559c74d80760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.318+0000 7f0a27578740 1 --2- 172.21.15.83:0/2403132127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7b270 0x559c74d7b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.318+0000 7f0a27578740 1 -- 172.21.15.83:0/2403132127 >> 172.21.15.83:0/2403132127 conn(0x559c74d767e0 msgr2=0x559c74d76be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.319+0000 7f0a1ed9d640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a080a1c30 0x7f0a080a40f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0a1809b590 tx=0x7f0a180a2040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.320+0000 7f0a27578740 1 -- 172.21.15.83:0/2403132127 shutdown_connections 2024-03-11T19:39:35.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.320+0000 7f0a27578740 1 -- 172.21.15.83:0/2403132127 wait complete. 2024-03-11T19:39:35.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.320+0000 7f0a27578740 1 -- start start 2024-03-11T19:39:35.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.320+0000 7f0a27578740 1 Processor -- start 2024-03-11T19:39:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.321+0000 7f0a27578740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 0x559c74d85330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.321+0000 7f0a27578740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d85870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.321+0000 7f0a1dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 0x559c74d85330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.321+0000 7f0a1dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 0x559c74d85330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53220/0 (socket says 172.21.15.83:53220) 2024-03-11T19:39:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.321+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d85870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.321+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d85870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39630/0 (socket says 172.21.15.83:39630) 2024-03-11T19:39:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.321+0000 7f0a1dd9b640 1 -- 172.21.15.83:0/1214270094 learned_addr learned my addr 172.21.15.83:0/1214270094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.321+0000 7f0a27578740 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d8c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.321+0000 7f0a27578740 1 -- 172.21.15.83:0/1214270094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c74ce4020 con 0x559c74d7bc60 2024-03-11T19:39:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.322+0000 7f0a27578740 1 -- 172.21.15.83:0/1214270094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c749b7430 con 0x559c74d80dd0 2024-03-11T19:39:35.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.322+0000 7f0a27578740 1 -- 172.21.15.83:0/1214270094 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c74973e00 con 0x559c74d7b270 2024-03-11T19:39:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.322+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d8c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.322+0000 7f0a1dd9b640 1 -- 172.21.15.83:0/1214270094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 msgr2=0x559c74d8c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.322+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d8c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.322+0000 7f0a1dd9b640 1 -- 172.21.15.83:0/1214270094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 msgr2=0x559c74d85870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.322+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d85870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.322+0000 7f0a1dd9b640 1 -- 172.21.15.83:0/1214270094 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a20108e00 con 0x559c74d7b270 2024-03-11T19:39:35.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.323+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 0x559c74d85330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0a14004470 tx=0x7f0a14018040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c28276df7c8e131d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.323+0000 7f0a05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:39:35.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.323+0000 7f0a05ffb640 1 -- 172.21.15.83:0/1214270094 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1401a070 con 0x559c74d7b270 2024-03-11T19:39:35.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.323+0000 7f0a05ffb640 1 -- 172.21.15.83:0/1214270094 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0a14005020 con 0x559c74d7b270 2024-03-11T19:39:35.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.323+0000 7f0a05ffb640 10 client.0 inactive, discarding osd_map(87..87 src has 1..87) v4 2024-03-11T19:39:35.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.323+0000 7f0a27578740 10 client.0 _ll_register_callbacks cb 0x559c74c8bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:35.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.323+0000 7f0a27578740 1 -- 172.21.15.83:0/1214270094 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c749e37e0 con 0x559c74d7b270 2024-03-11T19:39:35.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.324+0000 7f0a05ffb640 1 -- 172.21.15.83:0/1214270094 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a140051e0 con 0x559c74d7b270 2024-03-11T19:39:35.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.324+0000 7f0a27578740 10 client.25024 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:35.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.324+0000 7f0a27578740 1 -- 172.21.15.83:0/1214270094 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x559c74ce26a0 con 0x559c74d7b270 2024-03-11T19:39:35.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.325+0000 7f0a05ffb640 1 -- 172.21.15.83:0/1214270094 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0a1402d020 con 0x559c74d7b270 2024-03-11T19:39:35.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.325+0000 7f0a27578740 10 client.25024 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:35.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.325+0000 7f0a27578740 10 client.25024 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:39:35.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.325+0000 7f0a27578740 20 client.25024 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:35.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.325+0000 7f0a27578740 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559c74d91230 0x559c74d936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.325+0000 7f0a27578740 4 client.25024 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:39:35.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.325+0000 7f0a27578740 1 -- 172.21.15.83:0/1214270094 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c749e37e0 con 0x559c74d91230 2024-03-11T19:39:35.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.325+0000 7f0a1e59c640 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559c74d91230 0x559c74d936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.326+0000 7f0a1e59c640 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559c74d91230 0x559c74d936f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98701ccc6f276085 server_cookie=9ee8e4259c13d388 in_seq=0 out_seq=0 2024-03-11T19:39:35.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.326+0000 7f0a05ffb640 10 client.25024 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:39:35.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.326+0000 7f0a05ffb640 1 -- 172.21.15.83:0/1214270094 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c749e37e0 con 0x559c74d91230 2024-03-11T19:39:35.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.326+0000 7f0a05ffb640 10 client.25024 handle_command_reply: tid=0 2024-03-11T19:39:35.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.327+0000 7f0a27578740 1 client.25024 shutdown 2024-03-11T19:39:35.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.327+0000 7f0a27578740 1 -- 172.21.15.83:0/1214270094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 msgr2=0x559c74d85330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.327+0000 7f0a27578740 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 0x559c74d85330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0a14004470 tx=0x7f0a14018040 comp rx=0 tx=0).stop 2024-03-11T19:39:35.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.327+0000 7f0a27578740 1 -- 172.21.15.83:0/1214270094 shutdown_connections 2024-03-11T19:39:35.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.327+0000 7f0a27578740 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559c74d91230 0x559c74d936f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.327+0000 7f0a27578740 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d8c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.327+0000 7f0a27578740 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d85870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.327+0000 7f0a27578740 1 --2- 172.21.15.83:0/1214270094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 0x559c74d85330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.327+0000 7f0a27578740 1 -- 172.21.15.83:0/1214270094 >> 172.21.15.83:0/1214270094 conn(0x559c74d767e0 msgr2=0x559c74d75b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.327+0000 7f0a05ffb640 0 client.25024 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:39:35.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.328+0000 7f0a27578740 1 -- 172.21.15.83:0/1214270094 shutdown_connections 2024-03-11T19:39:35.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.329+0000 7f0a27578740 1 -- 172.21.15.83:0/1214270094 wait complete. 2024-03-11T19:39:35.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.329+0000 7f0a27578740 20 client.25024 trim_cache size 0 max 16384 2024-03-11T19:39:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.350+0000 7f0a27578740 1 Processor -- start 2024-03-11T19:39:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.351+0000 7f0a27578740 1 -- start start 2024-03-11T19:39:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.351+0000 7f0a27578740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d7b270 0x559c74d98090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.351+0000 7f0a27578740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d8c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.352+0000 7f0a27578740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d80dd0 0x559c74d7d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.352+0000 7f0a1dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d7b270 0x559c74d98090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.352+0000 7f0a1dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d7b270 0x559c74d98090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:53364/0 (socket says 172.21.15.83:53364) 2024-03-11T19:39:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.352+0000 7f0a1dd9b640 1 -- 172.21.15.83:0/1831928031 learned_addr learned my addr 172.21.15.83:0/1831928031 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.352+0000 7f0a27578740 1 -- 172.21.15.83:0/1831928031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c74c69130 con 0x559c74d7bc60 2024-03-11T19:39:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.352+0000 7f0a1e59c640 1 --2- 172.21.15.83:0/1831928031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d8c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.352+0000 7f0a27578740 1 -- 172.21.15.83:0/1831928031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c749e37e0 con 0x559c74d7b270 2024-03-11T19:39:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.352+0000 7f0a27578740 1 -- 172.21.15.83:0/1831928031 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c74d68a40 con 0x559c74d80dd0 2024-03-11T19:39:35.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.352+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1831928031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d80dd0 0x559c74d7d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.353+0000 7f0a1dd9b640 1 -- 172.21.15.83:0/1831928031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d80dd0 msgr2=0x559c74d7d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.353+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1831928031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d80dd0 0x559c74d7d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.353+0000 7f0a1dd9b640 1 -- 172.21.15.83:0/1831928031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 msgr2=0x559c74d8c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.353+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1831928031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d8c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.353+0000 7f0a1dd9b640 1 -- 172.21.15.83:0/1831928031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c74c7ec20 con 0x559c74d7b270 2024-03-11T19:39:35.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.353+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1831928031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d7b270 0x559c74d98090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400a010 tx=0x7f0a14022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=811d6c15e0d28eb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.354+0000 7f0a07fff640 1 -- 172.21.15.83:0/1831928031 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14003f00 con 0x559c74d7b270 2024-03-11T19:39:35.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.354+0000 7f0a07fff640 1 -- 172.21.15.83:0/1831928031 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a140055f0 con 0x559c74d7b270 2024-03-11T19:39:35.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.354+0000 7f0a07fff640 1 -- 172.21.15.83:0/1831928031 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14026480 con 0x559c74d7b270 2024-03-11T19:39:35.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.355+0000 7f0a27578740 1 -- 172.21.15.83:0/1831928031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d7b270 msgr2=0x559c74d98090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.355+0000 7f0a27578740 1 --2- 172.21.15.83:0/1831928031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d7b270 0x559c74d98090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400a010 tx=0x7f0a14022000 comp rx=0 tx=0).stop 2024-03-11T19:39:35.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.355+0000 7f0a27578740 1 -- 172.21.15.83:0/1831928031 shutdown_connections 2024-03-11T19:39:35.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.355+0000 7f0a27578740 1 --2- 172.21.15.83:0/1831928031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d80dd0 0x559c74d7d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.355+0000 7f0a27578740 1 --2- 172.21.15.83:0/1831928031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d8c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.355+0000 7f0a27578740 1 --2- 172.21.15.83:0/1831928031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d7b270 0x559c74d98090 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.355+0000 7f0a27578740 1 -- 172.21.15.83:0/1831928031 >> 172.21.15.83:0/1831928031 conn(0x559c74d767e0 msgr2=0x559c74d76bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.355+0000 7f0a27578740 1 -- 172.21.15.83:0/1831928031 shutdown_connections 2024-03-11T19:39:35.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.355+0000 7f0a27578740 1 -- 172.21.15.83:0/1831928031 wait complete. 2024-03-11T19:39:35.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.356+0000 7f0a27578740 1 -- start start 2024-03-11T19:39:35.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.356+0000 7f0a27578740 1 Processor -- start 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.356+0000 7f0a27578740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 0x559c74d8aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.356+0000 7f0a27578740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d8b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.356+0000 7f0a27578740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d8b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.356+0000 7f0a27578740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c74d8caf0 con 0x559c74d7bc60 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.356+0000 7f0a27578740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c74de7800 con 0x559c74d80dd0 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.356+0000 7f0a27578740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c74df8950 con 0x559c74d7b270 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.356+0000 7f0a27578740 10 client.0 _ll_register_callbacks cb 0x7f0a14005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.357+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d8b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.357+0000 7f0a1e59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d8b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39640/0 (socket says 172.21.15.83:39640) 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.357+0000 7f0a1e59c640 1 -- 172.21.15.83:0/1902911735 learned_addr learned my addr 172.21.15.83:0/1902911735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.357+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 0x559c74d8aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.357+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d8b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.357+0000 7f0a1dd9b640 1 -- 172.21.15.83:0/1902911735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 msgr2=0x559c74d8b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.357+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d8b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.357+0000 7f0a1dd9b640 1 -- 172.21.15.83:0/1902911735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 msgr2=0x559c74d8b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.357+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d8b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.357+0000 7f0a1dd9b640 1 -- 172.21.15.83:0/1902911735 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a14035040 con 0x559c74d7b270 2024-03-11T19:39:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.358+0000 7f0a1dd9b640 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 0x559c74d8aea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0a14018040 tx=0x7f0a14005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78101f43f4aa990e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.358+0000 7f0a05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:39:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.358+0000 7f0a05ffb640 1 -- 172.21.15.83:0/1902911735 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14024e00 con 0x559c74d7b270 2024-03-11T19:39:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.358+0000 7f0a05ffb640 1 -- 172.21.15.83:0/1902911735 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0a14000e40 con 0x559c74d7b270 2024-03-11T19:39:35.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.359+0000 7f0a27578740 1 -- 172.21.15.83:0/1902911735 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a10005500 con 0x559c74d7b270 2024-03-11T19:39:35.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.359+0000 7f0a05ffb640 1 -- 172.21.15.83:0/1902911735 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a14003e50 con 0x559c74d7b270 2024-03-11T19:39:35.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.359+0000 7f0a27578740 10 client.25030 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:35.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.359+0000 7f0a27578740 1 -- 172.21.15.83:0/1902911735 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559c74ca48c0 con 0x559c74d7b270 2024-03-11T19:39:35.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.359+0000 7f0a05ffb640 1 -- 172.21.15.83:0/1902911735 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a10005500 con 0x559c74d7b270 2024-03-11T19:39:35.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.361+0000 7f0a05ffb640 1 -- 172.21.15.83:0/1902911735 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0a1402d040 con 0x559c74d7b270 2024-03-11T19:39:35.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.361+0000 7f0a27578740 10 client.25030 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:35.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.361+0000 7f0a27578740 10 client.25030 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:39:35.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.361+0000 7f0a27578740 20 client.25030 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:35.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.361+0000 7f0a27578740 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559c74df9f60 0x559c74dfc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.361+0000 7f0a27578740 4 client.25030 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:39:35.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.361+0000 7f0a27578740 1 -- 172.21.15.83:0/1902911735 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559c74a20720 con 0x559c74df9f60 2024-03-11T19:39:35.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.361+0000 7f0a1e59c640 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559c74df9f60 0x559c74dfc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.366+0000 7f0a1e59c640 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559c74df9f60 0x559c74dfc380 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5643000c3865c832 server_cookie=e727bd9c27869d7 in_seq=0 out_seq=0 2024-03-11T19:39:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.369+0000 7f0a05ffb640 10 client.25030 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:39:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.369+0000 7f0a05ffb640 1 -- 172.21.15.83:0/1902911735 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559c74a20720 con 0x559c74df9f60 2024-03-11T19:39:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.369+0000 7f0a05ffb640 10 client.25030 handle_command_reply: tid=0 2024-03-11T19:39:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.369+0000 7f0a27578740 1 client.25030 shutdown 2024-03-11T19:39:35.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.369+0000 7f0a27578740 1 -- 172.21.15.83:0/1902911735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 msgr2=0x559c74d8aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.369+0000 7f0a27578740 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 0x559c74d8aea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0a14018040 tx=0x7f0a14005970 comp rx=0 tx=0).stop 2024-03-11T19:39:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.369+0000 7f0a27578740 1 -- 172.21.15.83:0/1902911735 shutdown_connections 2024-03-11T19:39:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.369+0000 7f0a27578740 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559c74df9f60 0x559c74dfc380 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.369+0000 7f0a27578740 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c74d80dd0 0x559c74d8b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.369+0000 7f0a27578740 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c74d7bc60 0x559c74d8b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.369+0000 7f0a27578740 1 --2- 172.21.15.83:0/1902911735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559c74d7b270 0x559c74d8aea0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.370+0000 7f0a27578740 1 -- 172.21.15.83:0/1902911735 >> 172.21.15.83:0/1902911735 conn(0x559c74d767e0 msgr2=0x559c74d98090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.371+0000 7f0a05ffb640 0 client.25030 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:39:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.371+0000 7f0a27578740 1 -- 172.21.15.83:0/1902911735 shutdown_connections 2024-03-11T19:39:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.371+0000 7f0a27578740 1 -- 172.21.15.83:0/1902911735 wait complete. 2024-03-11T19:39:35.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.371+0000 7f0a27578740 20 client.25030 trim_cache size 0 max 16384 2024-03-11T19:39:35.376 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:39:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.375+0000 7f0a249a1640 1 -- 172.21.15.83:0/3565509419 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a080a1c30 msgr2=0x7f0a080a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.375+0000 7f0a249a1640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a080a1c30 0x7f0a080a40f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0a1809b590 tx=0x7f0a180a2040 comp rx=0 tx=0).stop 2024-03-11T19:39:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.375+0000 7f0a249a1640 1 -- 172.21.15.83:0/3565509419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a20101c70 msgr2=0x7f0a201a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.376+0000 7f0a249a1640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a20101c70 0x7f0a201a0ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009fd0 tx=0x7f0a1800ee50 comp rx=0 tx=0).stop 2024-03-11T19:39:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.376+0000 7f0a249a1640 1 -- 172.21.15.83:0/3565509419 shutdown_connections 2024-03-11T19:39:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.376+0000 7f0a249a1640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a080a1c30 0x7f0a080a40f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.376+0000 7f0a249a1640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a20101c70 0x7f0a201a0ad0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.376+0000 7f0a249a1640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a201012b0 0x7f0a20199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.376+0000 7f0a249a1640 1 --2- 172.21.15.83:0/3565509419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a201000c0 0x7f0a20199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.376+0000 7f0a249a1640 1 -- 172.21.15.83:0/3565509419 >> 172.21.15.83:0/3565509419 conn(0x7f0a200fb630 msgr2=0x7f0a20069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.376+0000 7f0a249a1640 1 -- 172.21.15.83:0/3565509419 shutdown_connections 2024-03-11T19:39:35.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.376+0000 7f0a249a1640 1 -- 172.21.15.83:0/3565509419 wait complete. 2024-03-11T19:39:35.399 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:39:35.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:39:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.571+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/38177434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101c30 msgr2=0x7fd6ac1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.571+0000 7fd6b2b0d640 1 --2- 172.21.15.83:0/38177434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101c30 0x7fd6ac1084c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009ca0 tx=0x7fd6a402c120 comp rx=0 tx=0).stop 2024-03-11T19:39:35.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.571+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/38177434 shutdown_connections 2024-03-11T19:39:35.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.571+0000 7fd6b2b0d640 1 --2- 172.21.15.83:0/38177434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101c30 0x7fd6ac1084c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.571+0000 7fd6b2b0d640 1 --2- 172.21.15.83:0/38177434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac101270 0x7fd6ac1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.571+0000 7fd6b2b0d640 1 --2- 172.21.15.83:0/38177434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6ac100080 0x7fd6ac1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.571+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/38177434 >> 172.21.15.83:0/38177434 conn(0x7fd6ac0fb610 msgr2=0x7fd6ac0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.581+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/38177434 shutdown_connections 2024-03-11T19:39:35.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.581+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/38177434 wait complete. 2024-03-11T19:39:35.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.582+0000 7fd6b2b0d640 1 Processor -- start 2024-03-11T19:39:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.584+0000 7fd6b2b0d640 1 -- start start 2024-03-11T19:39:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.584+0000 7fd6b2b0d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6ac100080 0x7fd6ac194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.584+0000 7fd6b2b0d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101270 0x7fd6ac195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.584+0000 7fd6b2b0d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac101c30 0x7fd6ac19c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.584+0000 7fd6b2b0d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac19c8c0 con 0x7fd6ac101c30 2024-03-11T19:39:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.584+0000 7fd6b2b0d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac19ca30 con 0x7fd6ac100080 2024-03-11T19:39:35.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.584+0000 7fd6b2b0d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac19cbf0 con 0x7fd6ac101270 2024-03-11T19:39:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.584+0000 7fd6b10aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac101c30 0x7fd6ac19c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.584+0000 7fd6abfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101270 0x7fd6ac195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.584+0000 7fd6b10aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac101c30 0x7fd6ac19c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39662/0 (socket says 172.21.15.83:39662) 2024-03-11T19:39:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.584+0000 7fd6b10aa640 1 -- 172.21.15.83:0/1159905334 learned_addr learned my addr 172.21.15.83:0/1159905334 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.585+0000 7fd6abfff640 1 -- 172.21.15.83:0/1159905334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6ac100080 msgr2=0x7fd6ac194dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:39:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.585+0000 7fd6abfff640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6ac100080 0x7fd6ac194dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.585+0000 7fd6abfff640 1 -- 172.21.15.83:0/1159905334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac101c30 msgr2=0x7fd6ac19c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.585+0000 7fd6abfff640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac101c30 0x7fd6ac19c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.585+0000 7fd6abfff640 1 -- 172.21.15.83:0/1159905334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a4009900 con 0x7fd6ac101270 2024-03-11T19:39:35.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.585+0000 7fd6b10aa640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac101c30 0x7fd6ac19c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:39:35.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.585+0000 7fd6abfff640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101270 0x7fd6ac195310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00d6d0 tx=0x7fd69c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.586+0000 7fd6a9ffb640 1 -- 172.21.15.83:0/1159905334 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c005220 con 0x7fd6ac101270 2024-03-11T19:39:35.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.586+0000 7fd6a9ffb640 1 -- 172.21.15.83:0/1159905334 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd69c009d00 con 0x7fd6ac101270 2024-03-11T19:39:35.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.586+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/1159905334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac19ced0 con 0x7fd6ac101270 2024-03-11T19:39:35.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.587+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/1159905334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac19d3e0 con 0x7fd6ac101270 2024-03-11T19:39:35.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.587+0000 7fd6a9ffb640 1 -- 172.21.15.83:0/1159905334 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c005220 con 0x7fd6ac101270 2024-03-11T19:39:35.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.590+0000 7fd6a9ffb640 1 -- 172.21.15.83:0/1159905334 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd69c004bb0 con 0x7fd6ac101270 2024-03-11T19:39:35.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.590+0000 7fd6b57bb740 1 Processor -- start 2024-03-11T19:39:35.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.590+0000 7fd6b57bb740 1 -- start start 2024-03-11T19:39:35.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.590+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de371270 0x55d9de371690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.590+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 0x55d9de376760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.590+0000 7fd6a9ffb640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6940a1ae0 0x7fd6940a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.590+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de376dd0 0x55d9de379260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.590+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de32d960 con 0x55d9de371c60 2024-03-11T19:39:35.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.590+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de2e1310 con 0x55d9de371270 2024-03-11T19:39:35.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.591+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de2de840 con 0x55d9de376dd0 2024-03-11T19:39:35.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.591+0000 7fd6abfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 0x55d9de376760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.591+0000 7fd6abfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 0x55d9de376760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39664/0 (socket says 172.21.15.83:39664) 2024-03-11T19:39:35.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.591+0000 7fd6abfff640 1 -- 172.21.15.83:0/1431439293 learned_addr learned my addr 172.21.15.83:0/1431439293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.591+0000 7fd6b10aa640 1 --2- 172.21.15.83:0/1431439293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de371270 0x55d9de371690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.591+0000 7fd6b10aa640 1 --2- 172.21.15.83:0/1431439293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de376dd0 0x55d9de379260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.591+0000 7fd6a9ffb640 1 -- 172.21.15.83:0/1159905334 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd69c0ccb30 con 0x7fd6ac101270 2024-03-11T19:39:35.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.591+0000 7fd6b08a9640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6940a1ae0 0x7fd6940a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.592+0000 7fd6abfff640 1 -- 172.21.15.83:0/1431439293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de376dd0 msgr2=0x55d9de379260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.592+0000 7fd6abfff640 1 --2- 172.21.15.83:0/1431439293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de376dd0 0x55d9de379260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.592+0000 7fd6abfff640 1 -- 172.21.15.83:0/1431439293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de371270 msgr2=0x55d9de371690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.592+0000 7fd6abfff640 1 --2- 172.21.15.83:0/1431439293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de371270 0x55d9de371690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.592+0000 7fd6abfff640 1 -- 172.21.15.83:0/1431439293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9de29c060 con 0x55d9de371c60 2024-03-11T19:39:35.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.593+0000 7fd6b08a9640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6940a1ae0 0x7fd6940a3fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00058c0 tx=0x7fd6a0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.594+0000 7fd6abfff640 1 --2- 172.21.15.83:0/1431439293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 0x55d9de376760 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00d6a0 tx=0x7fd69c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fac13189588815 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.595+0000 7fd692ffd640 1 -- 172.21.15.83:0/1431439293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c005220 con 0x55d9de371c60 2024-03-11T19:39:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.595+0000 7fd692ffd640 1 -- 172.21.15.83:0/1431439293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd69c0994b0 con 0x55d9de371c60 2024-03-11T19:39:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.595+0000 7fd692ffd640 1 -- 172.21.15.83:0/1431439293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0a22f0 con 0x55d9de371c60 2024-03-11T19:39:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.596+0000 7fd6b57bb740 1 -- 172.21.15.83:0/1431439293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 msgr2=0x55d9de376760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.596+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/1431439293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 0x55d9de376760 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00d6a0 tx=0x7fd69c0048a0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.596+0000 7fd6b57bb740 1 -- 172.21.15.83:0/1431439293 shutdown_connections 2024-03-11T19:39:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.596+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/1431439293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de376dd0 0x55d9de379260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.596+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/1431439293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 0x55d9de376760 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.596+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/1431439293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de371270 0x55d9de371690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.596+0000 7fd6b57bb740 1 -- 172.21.15.83:0/1431439293 >> 172.21.15.83:0/1431439293 conn(0x55d9de36c7e0 msgr2=0x55d9de36cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.596+0000 7fd6b57bb740 1 -- 172.21.15.83:0/1431439293 shutdown_connections 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.596+0000 7fd6b57bb740 1 -- 172.21.15.83:0/1431439293 wait complete. 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.597+0000 7fd6b57bb740 1 -- start start 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.597+0000 7fd6b57bb740 1 Processor -- start 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.597+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371270 0x55d9de37b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.597+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 0x55d9de37b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.597+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de376dd0 0x55d9de382a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.597+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de2da020 con 0x55d9de371c60 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.597+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ddfad430 con 0x55d9de376dd0 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.597+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ddf69e00 con 0x55d9de371270 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.597+0000 7fd6b57bb740 10 client.0 _ll_register_callbacks cb 0x55d9de281cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.598+0000 7fd6b08a9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371270 0x55d9de37b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.598+0000 7fd6b08a9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371270 0x55d9de37b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53260/0 (socket says 172.21.15.83:53260) 2024-03-11T19:39:35.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.598+0000 7fd6b08a9640 1 -- 172.21.15.83:0/3556799084 learned_addr learned my addr 172.21.15.83:0/3556799084 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.598+0000 7fd6b10aa640 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de376dd0 0x55d9de382a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.598+0000 7fd6abfff640 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 0x55d9de37b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.598+0000 7fd6abfff640 1 -- 172.21.15.83:0/3556799084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371270 msgr2=0x55d9de37b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.598+0000 7fd6abfff640 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371270 0x55d9de37b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.598+0000 7fd6abfff640 1 -- 172.21.15.83:0/3556799084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de376dd0 msgr2=0x55d9de382a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.598+0000 7fd6abfff640 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de376dd0 0x55d9de382a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.598+0000 7fd6abfff640 1 -- 172.21.15.83:0/3556799084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac19d3e0 con 0x55d9de371c60 2024-03-11T19:39:35.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.598+0000 7fd6b08a9640 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371270 0x55d9de37b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:39:35.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.600+0000 7fd6abfff640 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 0x55d9de37b9c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005ca0 tx=0x7fd69c09a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88459df603f6e26f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.600+0000 7fd68bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:39:35.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.600+0000 7fd68bfff640 1 -- 172.21.15.83:0/3556799084 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0097c0 con 0x55d9de371c60 2024-03-11T19:39:35.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.600+0000 7fd68bfff640 1 -- 172.21.15.83:0/3556799084 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd69c0b6070 con 0x55d9de371c60 2024-03-11T19:39:35.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.600+0000 7fd6b57bb740 1 -- 172.21.15.83:0/3556799084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9ddfd97e0 con 0x55d9de371c60 2024-03-11T19:39:35.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.601+0000 7fd68bfff640 1 -- 172.21.15.83:0/3556799084 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd69c009920 con 0x55d9de371c60 2024-03-11T19:39:35.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.601+0000 7fd6b57bb740 10 client.15435 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:35.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.601+0000 7fd6b57bb740 1 -- 172.21.15.83:0/3556799084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d9de2d86a0 con 0x55d9de371c60 2024-03-11T19:39:35.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.601+0000 7fd68bfff640 1 -- 172.21.15.83:0/3556799084 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd69c098fa0 con 0x55d9de371c60 2024-03-11T19:39:35.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.601+0000 7fd68bfff640 1 -- 172.21.15.83:0/3556799084 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd69c0958a0 con 0x55d9de371c60 2024-03-11T19:39:35.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.601+0000 7fd6b57bb740 10 client.15435 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:35.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.601+0000 7fd6b57bb740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.601+0000 7fd6b57bb740 20 client.15435 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.601+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d9de3871f0 0x55d9de3895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.601+0000 7fd6b57bb740 4 client.15435 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:39:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.601+0000 7fd6b57bb740 1 -- 172.21.15.83:0/3556799084 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9ddfd97e0 con 0x55d9de3871f0 2024-03-11T19:39:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.602+0000 7fd6b08a9640 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d9de3871f0 0x55d9de3895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.602+0000 7fd6b08a9640 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d9de3871f0 0x55d9de3895e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1587ab58a611c11 server_cookie=e691eddad2e94f4d in_seq=0 out_seq=0 2024-03-11T19:39:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.603+0000 7fd68bfff640 10 client.15435 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.603+0000 7fd68bfff640 1 -- 172.21.15.83:0/3556799084 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d9ddfd97e0 con 0x55d9de3871f0 2024-03-11T19:39:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.603+0000 7fd68bfff640 10 client.15435 handle_command_reply: tid=0 2024-03-11T19:39:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.603+0000 7fd6b57bb740 1 client.15435 shutdown 2024-03-11T19:39:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.604+0000 7fd6b57bb740 1 -- 172.21.15.83:0/3556799084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 msgr2=0x55d9de37b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.604+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 0x55d9de37b9c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005ca0 tx=0x7fd69c09a250 comp rx=0 tx=0).stop 2024-03-11T19:39:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.604+0000 7fd6b57bb740 1 -- 172.21.15.83:0/3556799084 shutdown_connections 2024-03-11T19:39:35.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.604+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d9de3871f0 0x55d9de3895e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.604+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de376dd0 0x55d9de382a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.604+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371c60 0x55d9de37b9c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.604+0000 7fd68bfff640 0 client.15435 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:35.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.604+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/3556799084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371270 0x55d9de37b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.604+0000 7fd6b57bb740 1 -- 172.21.15.83:0/3556799084 >> 172.21.15.83:0/3556799084 conn(0x55d9de36c7e0 msgr2=0x55d9de36bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.604+0000 7fd6b57bb740 1 -- 172.21.15.83:0/3556799084 shutdown_connections 2024-03-11T19:39:35.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.605+0000 7fd6b57bb740 1 -- 172.21.15.83:0/3556799084 wait complete. 2024-03-11T19:39:35.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.605+0000 7fd6b57bb740 20 client.15435 trim_cache size 0 max 16384 2024-03-11T19:39:35.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.627+0000 7fd6b57bb740 1 Processor -- start 2024-03-11T19:39:35.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.627+0000 7fd6b57bb740 1 -- start start 2024-03-11T19:39:35.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371270 0x55d9de3731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371c60 0x55d9de3829c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de376dd0 0x55d9de3736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de25f130 con 0x55d9de371270 2024-03-11T19:39:35.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de385c10 con 0x55d9de376dd0 2024-03-11T19:39:35.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de379920 con 0x55d9de371c60 2024-03-11T19:39:35.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b08a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371270 0x55d9de3731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b08a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371270 0x55d9de3731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39694/0 (socket says 172.21.15.83:39694) 2024-03-11T19:39:35.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b08a9640 1 -- 172.21.15.83:0/2756173613 learned_addr learned my addr 172.21.15.83:0/2756173613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b10aa640 1 --2- 172.21.15.83:0/2756173613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de376dd0 0x55d9de3736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6abfff640 1 --2- 172.21.15.83:0/2756173613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371c60 0x55d9de3829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b08a9640 1 -- 172.21.15.83:0/2756173613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371c60 msgr2=0x55d9de3829c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b08a9640 1 --2- 172.21.15.83:0/2756173613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371c60 0x55d9de3829c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b08a9640 1 -- 172.21.15.83:0/2756173613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de376dd0 msgr2=0x55d9de3736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b08a9640 1 --2- 172.21.15.83:0/2756173613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de376dd0 0x55d9de3736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.628+0000 7fd6b08a9640 1 -- 172.21.15.83:0/2756173613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9de274c20 con 0x55d9de371270 2024-03-11T19:39:35.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.629+0000 7fd6b08a9640 1 --2- 172.21.15.83:0/2756173613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371270 0x55d9de3731b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000dc00 tx=0x7fd6a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cfb2032024cd2b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.629+0000 7fd6927fc640 1 -- 172.21.15.83:0/2756173613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0025d70 con 0x55d9de371270 2024-03-11T19:39:35.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.629+0000 7fd6927fc640 1 -- 172.21.15.83:0/2756173613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a0024040 con 0x55d9de371270 2024-03-11T19:39:35.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.629+0000 7fd6927fc640 1 -- 172.21.15.83:0/2756173613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a001e930 con 0x55d9de371270 2024-03-11T19:39:35.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.630+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2756173613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371270 msgr2=0x55d9de3731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.630+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/2756173613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371270 0x55d9de3731b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000dc00 tx=0x7fd6a0013000 comp rx=0 tx=0).stop 2024-03-11T19:39:35.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.631+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2756173613 shutdown_connections 2024-03-11T19:39:35.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.631+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/2756173613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de376dd0 0x55d9de3736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.631+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/2756173613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371c60 0x55d9de3829c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.631+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/2756173613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de371270 0x55d9de3731b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.631+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2756173613 >> 172.21.15.83:0/2756173613 conn(0x55d9de36c7e0 msgr2=0x55d9de379fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.631+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2756173613 shutdown_connections 2024-03-11T19:39:35.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.631+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2756173613 wait complete. 2024-03-11T19:39:35.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.631+0000 7fd6b57bb740 1 -- start start 2024-03-11T19:39:35.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.631+0000 7fd6b57bb740 1 Processor -- start 2024-03-11T19:39:35.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de371270 0x55d9de381270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371c60 0x55d9de3817b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6b57bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de376dd0 0x55d9de381cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de3dd6b0 con 0x55d9de376dd0 2024-03-11T19:39:35.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de3dd7f0 con 0x55d9de371270 2024-03-11T19:39:35.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6b57bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9de3ee940 con 0x55d9de371c60 2024-03-11T19:39:35.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6b57bb740 10 client.0 _ll_register_callbacks cb 0x7fd69c095750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6b10aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de376dd0 0x55d9de381cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6b10aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de376dd0 0x55d9de381cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39696/0 (socket says 172.21.15.83:39696) 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6b10aa640 1 -- 172.21.15.83:0/2931536753 learned_addr learned my addr 172.21.15.83:0/2931536753 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6abfff640 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371c60 0x55d9de3817b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.632+0000 7fd6b08a9640 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de371270 0x55d9de381270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.633+0000 7fd6b10aa640 1 -- 172.21.15.83:0/2931536753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371c60 msgr2=0x55d9de3817b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.633+0000 7fd6b10aa640 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371c60 0x55d9de3817b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.633+0000 7fd6b10aa640 1 -- 172.21.15.83:0/2931536753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de371270 msgr2=0x55d9de381270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.633+0000 7fd6b10aa640 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de371270 0x55d9de381270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.633+0000 7fd6b10aa640 1 -- 172.21.15.83:0/2931536753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a000d970 con 0x55d9de376dd0 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.633+0000 7fd6b10aa640 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de376dd0 0x55d9de381cf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4005e30 tx=0x7fd6a4034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269846d11442164d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:35.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.633+0000 7fd68bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:39:35.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.634+0000 7fd68bfff640 1 -- 172.21.15.83:0/2931536753 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4003b70 con 0x55d9de376dd0 2024-03-11T19:39:35.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.634+0000 7fd68bfff640 1 -- 172.21.15.83:0/2931536753 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd6a4003cd0 con 0x55d9de376dd0 2024-03-11T19:39:35.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.634+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2931536753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd69c0994b0 con 0x55d9de376dd0 2024-03-11T19:39:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.634+0000 7fd68bfff640 1 -- 172.21.15.83:0/2931536753 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6a4003e90 con 0x55d9de376dd0 2024-03-11T19:39:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.634+0000 7fd6b57bb740 10 client.15441 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.634+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2931536753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d9de29a8c0 con 0x55d9de376dd0 2024-03-11T19:39:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.635+0000 7fd68bfff640 1 -- 172.21.15.83:0/2931536753 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd69c0994b0 con 0x55d9de376dd0 2024-03-11T19:39:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.635+0000 7fd68bfff640 1 -- 172.21.15.83:0/2931536753 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd6a4042720 con 0x55d9de376dd0 2024-03-11T19:39:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.635+0000 7fd6b57bb740 10 client.15441 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.635+0000 7fd6b57bb740 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.635+0000 7fd6b57bb740 20 client.15441 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.635+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d9de3871f0 0x55d9de3f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.635+0000 7fd6b57bb740 4 client.15441 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:39:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.635+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2931536753 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d9de016720 con 0x55d9de3871f0 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.635+0000 7fd6b08a9640 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d9de3871f0 0x55d9de3f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd68bfff640 10 client.15441 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd6b08a9640 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d9de3871f0 0x55d9de3f24b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01bfa27d91bdb76 server_cookie=76dd7142acf9c88a in_seq=0 out_seq=0 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd68bfff640 1 -- 172.21.15.83:0/2931536753 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d9de016720 con 0x55d9de3871f0 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd68bfff640 10 client.15441 handle_command_reply: tid=0 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd6b57bb740 1 client.15441 shutdown 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2931536753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de376dd0 msgr2=0x55d9de381cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de376dd0 0x55d9de381cf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4005e30 tx=0x7fd6a4034670 comp rx=0 tx=0).stop 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2931536753 shutdown_connections 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d9de3871f0 0x55d9de3f24b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9de376dd0 0x55d9de381cf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9de371c60 0x55d9de3817b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd6b57bb740 1 --2- 172.21.15.83:0/2931536753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9de371270 0x55d9de381270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.636+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2931536753 >> 172.21.15.83:0/2931536753 conn(0x55d9de36c7e0 msgr2=0x55d9de37b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.637+0000 7fd68bfff640 0 client.15441 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:35.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.637+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2931536753 shutdown_connections 2024-03-11T19:39:35.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.637+0000 7fd6b57bb740 1 -- 172.21.15.83:0/2931536753 wait complete. 2024-03-11T19:39:35.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.637+0000 7fd6b57bb740 20 client.15441 trim_cache size 0 max 16384 2024-03-11T19:39:35.643 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:39:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.638+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/1159905334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6940a1ae0 msgr2=0x7fd6940a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.638+0000 7fd6b2b0d640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6940a1ae0 0x7fd6940a3fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00058c0 tx=0x7fd6a0005850 comp rx=0 tx=0).stop 2024-03-11T19:39:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.638+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/1159905334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101270 msgr2=0x7fd6ac195310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.638+0000 7fd6b2b0d640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101270 0x7fd6ac195310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00d6d0 tx=0x7fd69c00db90 comp rx=0 tx=0).stop 2024-03-11T19:39:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.638+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/1159905334 shutdown_connections 2024-03-11T19:39:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.638+0000 7fd6b2b0d640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd6940a1ae0 0x7fd6940a3fa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.639+0000 7fd6b2b0d640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6ac101c30 0x7fd6ac19c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.639+0000 7fd6b2b0d640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101270 0x7fd6ac195310 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.639+0000 7fd6b2b0d640 1 --2- 172.21.15.83:0/1159905334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6ac100080 0x7fd6ac194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.639+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/1159905334 >> 172.21.15.83:0/1159905334 conn(0x7fd6ac0fb610 msgr2=0x7fd6ac0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.639+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/1159905334 shutdown_connections 2024-03-11T19:39:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:35.639+0000 7fd6b2b0d640 1 -- 172.21.15.83:0/1159905334 wait complete. 2024-03-11T19:39:35.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:35 smithi083 ceph-mon[31864]: pgmap v360: 97 pgs: 97 active+clean; 961 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 958 op/s 2024-03-11T19:39:35.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:35 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4160314805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:39:35.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:35 smithi137 ceph-mon[36220]: pgmap v360: 97 pgs: 97 active+clean; 961 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 958 op/s 2024-03-11T19:39:35.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:35 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4160314805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:39:35.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:35 smithi204 ceph-mon[36739]: pgmap v360: 97 pgs: 97 active+clean; 961 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 958 op/s 2024-03-11T19:39:35.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:35 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4160314805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:39:36.661 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:39:36.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:39:36.662 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:39:36.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.840+0000 7f48c83c7640 1 -- 172.21.15.83:0/3247277611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48c0076d80 msgr2=0x7f48c00771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:36.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.840+0000 7f48c83c7640 1 --2- 172.21.15.83:0/3247277611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48c0076d80 0x7f48c00771e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009900 tx=0x7f48b002bc60 comp rx=0 tx=0).stop 2024-03-11T19:39:36.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.842+0000 7f48c83c7640 1 -- 172.21.15.83:0/3247277611 shutdown_connections 2024-03-11T19:39:36.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.842+0000 7f48c83c7640 1 --2- 172.21.15.83:0/3247277611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48c0077720 0x7f48c0077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.842+0000 7f48c83c7640 1 --2- 172.21.15.83:0/3247277611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48c0076d80 0x7f48c00771e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.842+0000 7f48c83c7640 1 --2- 172.21.15.83:0/3247277611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48c0075b40 0x7f48c0075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.842+0000 7f48c83c7640 1 -- 172.21.15.83:0/3247277611 >> 172.21.15.83:0/3247277611 conn(0x7f48c00fe930 msgr2=0x7f48c0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:36.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.845+0000 7f48c83c7640 1 -- 172.21.15.83:0/3247277611 shutdown_connections 2024-03-11T19:39:36.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.846+0000 7f48c83c7640 1 -- 172.21.15.83:0/3247277611 wait complete. 2024-03-11T19:39:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.847+0000 7f48c83c7640 1 Processor -- start 2024-03-11T19:39:36.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.847+0000 7f48c83c7640 1 -- start start 2024-03-11T19:39:36.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.847+0000 7f48c83c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48c0075b40 0x7f48c006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.847+0000 7f48c83c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48c0076d80 0x7f48c006fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.847+0000 7f48c83c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48c0077720 0x7f48c01adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.847+0000 7f48c83c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0072ef0 con 0x7f48c0075b40 2024-03-11T19:39:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.847+0000 7f48c83c7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0073060 con 0x7f48c0076d80 2024-03-11T19:39:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.847+0000 7f48c83c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0073220 con 0x7f48c0077720 2024-03-11T19:39:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.847+0000 7f48c6964640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48c0077720 0x7f48c01adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.848+0000 7f48c6964640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48c0077720 0x7f48c01adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53276/0 (socket says 172.21.15.83:53276) 2024-03-11T19:39:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.848+0000 7f48c6964640 1 -- 172.21.15.83:0/3063788334 learned_addr learned my addr 172.21.15.83:0/3063788334 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.848+0000 7f48c6163640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48c0075b40 0x7f48c006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.848+0000 7f48c6964640 1 -- 172.21.15.83:0/3063788334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48c0076d80 msgr2=0x7f48c006fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:36.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.848+0000 7f48c6964640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48c0076d80 0x7f48c006fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.848+0000 7f48c6964640 1 -- 172.21.15.83:0/3063788334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48c0075b40 msgr2=0x7f48c006f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:36.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.848+0000 7f48c6964640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48c0075b40 0x7f48c006f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.848+0000 7f48c6964640 1 -- 172.21.15.83:0/3063788334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b0009560 con 0x7f48c0077720 2024-03-11T19:39:36.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.848+0000 7f48c6964640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48c0077720 0x7f48c01adb00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00e970 tx=0x7f48bc00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:36.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.848+0000 7f48b77fe640 1 -- 172.21.15.83:0/3063788334 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc009720 con 0x7f48c0077720 2024-03-11T19:39:36.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.849+0000 7f48b77fe640 1 -- 172.21.15.83:0/3063788334 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48bc005500 con 0x7f48c0077720 2024-03-11T19:39:36.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.849+0000 7f48b77fe640 1 -- 172.21.15.83:0/3063788334 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc010430 con 0x7f48c0077720 2024-03-11T19:39:36.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.849+0000 7f48c83c7640 1 -- 172.21.15.83:0/3063788334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c0102f40 con 0x7f48c0077720 2024-03-11T19:39:36.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.849+0000 7f48c83c7640 1 -- 172.21.15.83:0/3063788334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c01033a0 con 0x7f48c0077720 2024-03-11T19:39:36.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.852+0000 7f48b77fe640 1 -- 172.21.15.83:0/3063788334 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f48bc005020 con 0x7f48c0077720 2024-03-11T19:39:36.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.853+0000 7f48b77fe640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f48ac0a1b00 0x7f48ac0a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.853+0000 7f48b77fe640 1 -- 172.21.15.83:0/3063788334 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f48bc014070 con 0x7f48c0077720 2024-03-11T19:39:36.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.853+0000 7f48c6163640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f48ac0a1b00 0x7f48ac0a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.854+0000 7f48caf99740 1 Processor -- start 2024-03-11T19:39:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.855+0000 7f48caf99740 1 -- start start 2024-03-11T19:39:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.855+0000 7f48caf99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 0x565231be48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.855+0000 7f48caf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be5690 0x565231be5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.855+0000 7f48c6163640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f48ac0a1b00 0x7f48ac0a3fc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f48b8005fd0 tx=0x7f48b8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.855+0000 7f48caf99740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.855+0000 7f48c6163640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 0x565231be48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48c6163640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 0x565231be48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53290/0 (socket says 172.21.15.83:53290) 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48c6964640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48c5962640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be5690 0x565231be5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48c6964640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bec8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:53446/0 (socket says 172.21.15.83:53446) 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48c5962640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be5690 0x565231be5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39716/0 (socket says 172.21.15.83:39716) 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48c6163640 1 -- 172.21.15.83:0/2492721178 learned_addr learned my addr 172.21.15.83:0/2492721178 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48caf99740 1 -- 172.21.15.83:0/2492721178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x565231b53f90 con 0x565231be5690 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48caf99740 1 -- 172.21.15.83:0/2492721178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x565231b514c0 con 0x565231be6050 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48caf99740 1 -- 172.21.15.83:0/2492721178 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565231b4cca0 con 0x565231be44c0 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48c6163640 1 -- 172.21.15.83:0/2492721178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 msgr2=0x565231bec8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48c6163640 1 --2- 172.21.15.83:0/2492721178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bec8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48c6163640 1 -- 172.21.15.83:0/2492721178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be5690 msgr2=0x565231be5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48c6163640 1 --2- 172.21.15.83:0/2492721178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be5690 0x565231be5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.856+0000 7f48c6163640 1 -- 172.21.15.83:0/2492721178 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565231b0f0b0 con 0x565231be44c0 2024-03-11T19:39:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.857+0000 7f48c6163640 1 --2- 172.21.15.83:0/2492721178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 0x565231be48e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f48b80115d0 tx=0x7f48b8012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be75e0e16029363d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:36.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.857+0000 7f48b4bfe640 1 -- 172.21.15.83:0/2492721178 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8010710 con 0x565231be44c0 2024-03-11T19:39:36.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.857+0000 7f48b4bfe640 1 -- 172.21.15.83:0/2492721178 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48b8010d50 con 0x565231be44c0 2024-03-11T19:39:36.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.857+0000 7f48b4bfe640 1 -- 172.21.15.83:0/2492721178 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b801d3b0 con 0x565231be44c0 2024-03-11T19:39:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.858+0000 7f48caf99740 1 -- 172.21.15.83:0/2492721178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 msgr2=0x565231be48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:36.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.858+0000 7f48caf99740 1 --2- 172.21.15.83:0/2492721178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 0x565231be48e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f48b80115d0 tx=0x7f48b8012000 comp rx=0 tx=0).stop 2024-03-11T19:39:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.858+0000 7f48caf99740 1 -- 172.21.15.83:0/2492721178 shutdown_connections 2024-03-11T19:39:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.858+0000 7f48caf99740 1 --2- 172.21.15.83:0/2492721178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.858+0000 7f48caf99740 1 --2- 172.21.15.83:0/2492721178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be5690 0x565231be5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.858+0000 7f48caf99740 1 --2- 172.21.15.83:0/2492721178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 0x565231be48e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.858+0000 7f48caf99740 1 -- 172.21.15.83:0/2492721178 >> 172.21.15.83:0/2492721178 conn(0x565231bdf6a0 msgr2=0x565231bdfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.858+0000 7f48caf99740 1 -- 172.21.15.83:0/2492721178 shutdown_connections 2024-03-11T19:39:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.858+0000 7f48caf99740 1 -- 172.21.15.83:0/2492721178 wait complete. 2024-03-11T19:39:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.859+0000 7f48caf99740 1 -- start start 2024-03-11T19:39:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.859+0000 7f48caf99740 1 Processor -- start 2024-03-11T19:39:36.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48caf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be44c0 0x565231bee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48caf99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 0x565231beeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48caf99740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bf5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48c6163640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be44c0 0x565231bee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48c5962640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 0x565231beeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48c5962640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 0x565231beeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53292/0 (socket says 172.21.15.83:53292) 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48c5962640 1 -- 172.21.15.83:0/2719999064 learned_addr learned my addr 172.21.15.83:0/2719999064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48c6964640 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bf5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48caf99740 1 -- 172.21.15.83:0/2719999064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x565231af1f40 con 0x565231be44c0 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48caf99740 1 -- 172.21.15.83:0/2719999064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652318204b0 con 0x565231be6050 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48caf99740 1 -- 172.21.15.83:0/2719999064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565231ad2670 con 0x565231be5690 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.860+0000 7f48caf99740 10 client.0 _ll_register_callbacks cb 0x565231b3ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.861+0000 7f48c5962640 1 -- 172.21.15.83:0/2719999064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 msgr2=0x565231bf5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:36.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.861+0000 7f48c5962640 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bf5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.861+0000 7f48c5962640 1 -- 172.21.15.83:0/2719999064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be44c0 msgr2=0x565231bee5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.861+0000 7f48c5962640 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be44c0 0x565231bee5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.861+0000 7f48c5962640 1 -- 172.21.15.83:0/2719999064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b800fe70 con 0x565231be5690 2024-03-11T19:39:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.861+0000 7f48c6163640 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be44c0 0x565231bee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:39:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.861+0000 7f48c5962640 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 0x565231beeaf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f48c0070e70 tx=0x7f48b00048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=859f01637daea5de server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.861+0000 7f48a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:39:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.862+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2719999064 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0005d30 con 0x565231be5690 2024-03-11T19:39:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.862+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2719999064 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f48b0004040 con 0x565231be5690 2024-03-11T19:39:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.862+0000 7f48caf99740 1 -- 172.21.15.83:0/2719999064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565231b26f40 con 0x565231be5690 2024-03-11T19:39:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.862+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2719999064 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48b0004200 con 0x565231be5690 2024-03-11T19:39:36.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.862+0000 7f48caf99740 10 client.25057 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:36.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.862+0000 7f48caf99740 1 -- 172.21.15.83:0/2719999064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x565231b4b320 con 0x565231be5690 2024-03-11T19:39:36.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.863+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2719999064 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x565231b26f40 con 0x565231be5690 2024-03-11T19:39:36.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.863+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2719999064 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f48b003c050 con 0x565231be5690 2024-03-11T19:39:36.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.863+0000 7f48caf99740 10 client.25057 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:36.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.863+0000 7f48caf99740 10 client.25057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:36.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.863+0000 7f48caf99740 20 client.25057 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:36.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.863+0000 7f48caf99740 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565231bfa7f0 0x565231bfcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.863+0000 7f48caf99740 4 client.25057 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:39:36.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.863+0000 7f48caf99740 1 -- 172.21.15.83:0/2719999064 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565231b43310 con 0x565231bfa7f0 2024-03-11T19:39:36.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.865+0000 7f48c6163640 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565231bfa7f0 0x565231bfcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.865+0000 7f48c6163640 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565231bfa7f0 0x565231bfcc30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8fe0ae91907c102 server_cookie=ec4edbf2295c2e7b in_seq=0 out_seq=0 2024-03-11T19:39:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.865+0000 7f48a9ffb640 10 client.25057 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.866+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2719999064 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565231b43310 con 0x565231bfa7f0 2024-03-11T19:39:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.866+0000 7f48a9ffb640 10 client.25057 handle_command_reply: tid=0 2024-03-11T19:39:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.866+0000 7f48caf99740 1 client.25057 shutdown 2024-03-11T19:39:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.867+0000 7f48caf99740 1 -- 172.21.15.83:0/2719999064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 msgr2=0x565231beeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.867+0000 7f48caf99740 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 0x565231beeaf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f48c0070e70 tx=0x7f48b00048d0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.867+0000 7f48caf99740 1 -- 172.21.15.83:0/2719999064 shutdown_connections 2024-03-11T19:39:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.867+0000 7f48caf99740 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565231bfa7f0 0x565231bfcc30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.867+0000 7f48caf99740 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bf5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.867+0000 7f48caf99740 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 0x565231beeaf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.867+0000 7f48caf99740 1 --2- 172.21.15.83:0/2719999064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be44c0 0x565231bee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.867+0000 7f48caf99740 1 -- 172.21.15.83:0/2719999064 >> 172.21.15.83:0/2719999064 conn(0x565231bdf6a0 msgr2=0x565231bde8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.867+0000 7f48a9ffb640 0 client.25057 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.868+0000 7f48caf99740 1 -- 172.21.15.83:0/2719999064 shutdown_connections 2024-03-11T19:39:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.868+0000 7f48caf99740 1 -- 172.21.15.83:0/2719999064 wait complete. 2024-03-11T19:39:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.868+0000 7f48caf99740 20 client.25057 trim_cache size 0 max 16384 2024-03-11T19:39:36.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.893+0000 7f48caf99740 1 Processor -- start 2024-03-11T19:39:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.893+0000 7f48caf99740 1 -- start start 2024-03-11T19:39:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48caf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be44c0 0x565231bee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48caf99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 0x565231beead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48caf99740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bf5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48caf99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x565231bec930 con 0x565231be44c0 2024-03-11T19:39:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48caf99740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x565231bd4350 con 0x565231be6050 2024-03-11T19:39:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48caf99740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565231b43310 con 0x565231be5690 2024-03-11T19:39:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48c5962640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 0x565231beead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48c6964640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bf5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48c6964640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bf5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:53454/0 (socket says 172.21.15.83:53454) 2024-03-11T19:39:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48c5962640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 0x565231beead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53294/0 (socket says 172.21.15.83:53294) 2024-03-11T19:39:36.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48c6964640 1 -- 172.21.15.83:0/3891777804 learned_addr learned my addr 172.21.15.83:0/3891777804 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:36.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48c6964640 1 -- 172.21.15.83:0/3891777804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 msgr2=0x565231beead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:36.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48c6964640 1 --2- 172.21.15.83:0/3891777804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 0x565231beead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48c6964640 1 -- 172.21.15.83:0/3891777804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be44c0 msgr2=0x565231bee590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:36.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48c6964640 1 --2- 172.21.15.83:0/3891777804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be44c0 0x565231bee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48c6964640 1 -- 172.21.15.83:0/3891777804 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565231ae7c00 con 0x565231be6050 2024-03-11T19:39:36.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.894+0000 7f48c6964640 1 --2- 172.21.15.83:0/3891777804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bf5af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48bc09a6f0 tx=0x7f48bc0ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4427e0a01b7a1ac2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.895+0000 7f48abfff640 1 -- 172.21.15.83:0/3891777804 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc09aeb0 con 0x565231be6050 2024-03-11T19:39:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.895+0000 7f48abfff640 1 -- 172.21.15.83:0/3891777804 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f48bc09a7b0 con 0x565231be6050 2024-03-11T19:39:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.895+0000 7f48abfff640 1 -- 172.21.15.83:0/3891777804 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc01fd50 con 0x565231be6050 2024-03-11T19:39:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.897+0000 7f48caf99740 1 -- 172.21.15.83:0/3891777804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 msgr2=0x565231bf5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.897+0000 7f48caf99740 1 --2- 172.21.15.83:0/3891777804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bf5af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48bc09a6f0 tx=0x7f48bc0ba040 comp rx=0 tx=0).stop 2024-03-11T19:39:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.897+0000 7f48caf99740 1 -- 172.21.15.83:0/3891777804 shutdown_connections 2024-03-11T19:39:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.897+0000 7f48caf99740 1 --2- 172.21.15.83:0/3891777804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be6050 0x565231bf5af0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.897+0000 7f48caf99740 1 --2- 172.21.15.83:0/3891777804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be5690 0x565231beead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.897+0000 7f48caf99740 1 --2- 172.21.15.83:0/3891777804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be44c0 0x565231bee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.897+0000 7f48caf99740 1 -- 172.21.15.83:0/3891777804 >> 172.21.15.83:0/3891777804 conn(0x565231bdf6a0 msgr2=0x565231bdfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.897+0000 7f48caf99740 1 -- 172.21.15.83:0/3891777804 shutdown_connections 2024-03-11T19:39:36.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.897+0000 7f48caf99740 1 -- 172.21.15.83:0/3891777804 wait complete. 2024-03-11T19:39:36.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.898+0000 7f48caf99740 1 -- start start 2024-03-11T19:39:36.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.898+0000 7f48caf99740 1 Processor -- start 2024-03-11T19:39:36.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.898+0000 7f48caf99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 0x565231bf42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.898+0000 7f48caf99740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be5690 0x565231bf47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.898+0000 7f48caf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be6050 0x565231bf4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48caf99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x565231bf5ef0 con 0x565231be6050 2024-03-11T19:39:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48c6163640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 0x565231bf42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48c6163640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 0x565231bf42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53306/0 (socket says 172.21.15.83:53306) 2024-03-11T19:39:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48c6163640 1 -- 172.21.15.83:0/2140913795 learned_addr learned my addr 172.21.15.83:0/2140913795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48c5962640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be5690 0x565231bf47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48caf99740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x565231c50d40 con 0x565231be5690 2024-03-11T19:39:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48caf99740 1 -- 172.21.15.83:0/2140913795 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565231c50e80 con 0x565231be44c0 2024-03-11T19:39:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48c6964640 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be6050 0x565231bf4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48caf99740 10 client.0 _ll_register_callbacks cb 0x7f48b8015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48c6964640 1 -- 172.21.15.83:0/2140913795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 msgr2=0x565231bf42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48c6964640 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 0x565231bf42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48c6964640 1 -- 172.21.15.83:0/2140913795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be5690 msgr2=0x565231bf47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48c6964640 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be5690 0x565231bf47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48c6964640 1 -- 172.21.15.83:0/2140913795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c01033a0 con 0x565231be6050 2024-03-11T19:39:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.899+0000 7f48c5962640 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be5690 0x565231bf47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:39:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.900+0000 7f48c6964640 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be6050 0x565231bf4d20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f48bc09a6f0 tx=0x7f48bc005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f29d20ea895296d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.900+0000 7f48a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:39:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.900+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2140913795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc0a2cc0 con 0x565231be6050 2024-03-11T19:39:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.900+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2140913795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f48bc0a2f50 con 0x565231be6050 2024-03-11T19:39:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.900+0000 7f48caf99740 1 -- 172.21.15.83:0/2140913795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565231b72900 con 0x565231be6050 2024-03-11T19:39:36.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.901+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2140913795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48bc0b32b0 con 0x565231be6050 2024-03-11T19:39:36.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.901+0000 7f48caf99740 10 client.15453 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:36.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.901+0000 7f48caf99740 1 -- 172.21.15.83:0/2140913795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x565231b0d910 con 0x565231be6050 2024-03-11T19:39:36.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.901+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2140913795 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x565231b72900 con 0x565231be6050 2024-03-11T19:39:36.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.901+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2140913795 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f48bc081020 con 0x565231be6050 2024-03-11T19:39:36.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.902+0000 7f48caf99740 10 client.15453 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:36.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.902+0000 7f48caf99740 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.902+0000 7f48caf99740 20 client.15453 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.902+0000 7f48caf99740 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565231c525f0 0x565231c54a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.902+0000 7f48caf99740 4 client.15453 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:39:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.902+0000 7f48caf99740 1 -- 172.21.15.83:0/2140913795 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56523184c860 con 0x565231c525f0 2024-03-11T19:39:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.902+0000 7f48c6163640 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565231c525f0 0x565231c54a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.903+0000 7f48c6163640 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565231c525f0 0x565231c54a10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6b01e89ee8c3a79 server_cookie=b1d7b9f4cc86603 in_seq=0 out_seq=0 2024-03-11T19:39:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.903+0000 7f48a9ffb640 10 client.15453 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:36.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48a9ffb640 1 -- 172.21.15.83:0/2140913795 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56523184c860 con 0x565231c525f0 2024-03-11T19:39:36.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48a9ffb640 10 client.15453 handle_command_reply: tid=0 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48caf99740 1 client.15453 shutdown 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48caf99740 1 -- 172.21.15.83:0/2140913795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be6050 msgr2=0x565231bf4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48caf99740 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be6050 0x565231bf4d20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f48bc09a6f0 tx=0x7f48bc005b40 comp rx=0 tx=0).stop 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48caf99740 1 -- 172.21.15.83:0/2140913795 shutdown_connections 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48caf99740 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565231c525f0 0x565231c54a10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48caf99740 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565231be6050 0x565231bf4d20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48caf99740 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565231be5690 0x565231bf47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48caf99740 1 --2- 172.21.15.83:0/2140913795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565231be44c0 0x565231bf42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48caf99740 1 -- 172.21.15.83:0/2140913795 >> 172.21.15.83:0/2140913795 conn(0x565231bdf6a0 msgr2=0x565231be9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48a9ffb640 0 client.15453 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.905+0000 7f48caf99740 1 -- 172.21.15.83:0/2140913795 shutdown_connections 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.906+0000 7f48caf99740 1 -- 172.21.15.83:0/2140913795 wait complete. 2024-03-11T19:39:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.906+0000 7f48caf99740 20 client.15453 trim_cache size 0 max 16384 2024-03-11T19:39:36.912 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:39:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:39:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "caa9652b-e90f-4d33-be49-62c7a7a69af3", 2024-03-11T19:39:36.912 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:39:36.912 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:39:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.906+0000 7f48c83c7640 1 -- 172.21.15.83:0/3063788334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f48ac0a1b00 msgr2=0x7f48ac0a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.906+0000 7f48c83c7640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f48ac0a1b00 0x7f48ac0a3fc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f48b8005fd0 tx=0x7f48b8009500 comp rx=0 tx=0).stop 2024-03-11T19:39:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.907+0000 7f48c83c7640 1 -- 172.21.15.83:0/3063788334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48c0077720 msgr2=0x7f48c01adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.907+0000 7f48c83c7640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48c0077720 0x7f48c01adb00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00e970 tx=0x7f48bc00ed70 comp rx=0 tx=0).stop 2024-03-11T19:39:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.908+0000 7f48c83c7640 1 -- 172.21.15.83:0/3063788334 shutdown_connections 2024-03-11T19:39:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.908+0000 7f48c83c7640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f48ac0a1b00 0x7f48ac0a3fc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.908+0000 7f48c83c7640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48c0077720 0x7f48c01adb00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.908+0000 7f48c83c7640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48c0076d80 0x7f48c006fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.908+0000 7f48c83c7640 1 --2- 172.21.15.83:0/3063788334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48c0075b40 0x7f48c006f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:36.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.908+0000 7f48c83c7640 1 -- 172.21.15.83:0/3063788334 >> 172.21.15.83:0/3063788334 conn(0x7f48c00fe930 msgr2=0x7f48c0100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:36.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.908+0000 7f48c83c7640 1 -- 172.21.15.83:0/3063788334 shutdown_connections 2024-03-11T19:39:36.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:36.908+0000 7f48c83c7640 1 -- 172.21.15.83:0/3063788334 wait complete. 2024-03-11T19:39:36.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:39:36.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:39:37.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.080+0000 7f58035ca640 1 -- 172.21.15.83:0/1683316409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc102e90 msgr2=0x7f57fc109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:37.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.080+0000 7f58035ca640 1 --2- 172.21.15.83:0/1683316409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc102e90 0x7f57fc109720 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009ca0 tx=0x7f57f402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 -- 172.21.15.83:0/1683316409 shutdown_connections 2024-03-11T19:39:37.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 --2- 172.21.15.83:0/1683316409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc102e90 0x7f57fc109720 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 --2- 172.21.15.83:0/1683316409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57fc1024d0 0x7f57fc102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 --2- 172.21.15.83:0/1683316409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57fc0ff460 0x7f57fc0ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 -- 172.21.15.83:0/1683316409 >> 172.21.15.83:0/1683316409 conn(0x7f57fc0fad90 msgr2=0x7f57fc0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:37.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 -- 172.21.15.83:0/1683316409 shutdown_connections 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 -- 172.21.15.83:0/1683316409 wait complete. 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 Processor -- start 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 -- start start 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc0ff460 0x7f57fc1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57fc1024d0 0x7f57fc197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57fc102e90 0x7f57fc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc19ecd0 con 0x7f57fc0ff460 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc19ee40 con 0x7f57fc102e90 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.081+0000 7f58035ca640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc19f000 con 0x7f57fc1024d0 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5801366640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc0ff460 0x7f57fc1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5801366640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc0ff460 0x7f57fc1971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39762/0 (socket says 172.21.15.83:39762) 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5801366640 1 -- 172.21.15.83:0/525170347 learned_addr learned my addr 172.21.15.83:0/525170347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:37.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5801b67640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57fc102e90 0x7f57fc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5800b65640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57fc1024d0 0x7f57fc197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5801366640 1 -- 172.21.15.83:0/525170347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57fc1024d0 msgr2=0x7f57fc197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5801366640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57fc1024d0 0x7f57fc197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5801366640 1 -- 172.21.15.83:0/525170347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57fc102e90 msgr2=0x7f57fc19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5801366640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57fc102e90 0x7f57fc19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5801366640 1 -- 172.21.15.83:0/525170347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4009900 con 0x7f57fc0ff460 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5801b67640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57fc102e90 0x7f57fc19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f5801366640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc0ff460 0x7f57fc1971e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f57f800e9c0 tx=0x7f57f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f57f27fc640 1 -- 172.21.15.83:0/525170347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8009700 con 0x7f57fc0ff460 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f58035ca640 1 -- 172.21.15.83:0/525170347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc19f2e0 con 0x7f57fc0ff460 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f57f27fc640 1 -- 172.21.15.83:0/525170347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57f800cd80 con 0x7f57fc0ff460 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f57f27fc640 1 -- 172.21.15.83:0/525170347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8010430 con 0x7f57fc0ff460 2024-03-11T19:39:37.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.082+0000 7f58035ca640 1 -- 172.21.15.83:0/525170347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc19f840 con 0x7f57fc0ff460 2024-03-11T19:39:37.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.084+0000 7f57f27fc640 1 -- 172.21.15.83:0/525170347 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f57f8010590 con 0x7f57fc0ff460 2024-03-11T19:39:37.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f580619d740 1 Processor -- start 2024-03-11T19:39:37.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f580619d740 1 -- start start 2024-03-11T19:39:37.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f580619d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879db62d0 0x563879db66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f580619d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db6cc0 0x563879dbb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f580619d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879dbbe30 0x563879dbe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f580619d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563879d26280 con 0x563879db6cc0 2024-03-11T19:39:37.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f57f27fc640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57e80a1a30 0x7f57e80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f580619d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563879d237b0 con 0x563879db62d0 2024-03-11T19:39:37.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f580619d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563879d1ef90 con 0x563879dbbe30 2024-03-11T19:39:37.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f5801366640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db6cc0 0x563879dbb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f5801b67640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57e80a1a30 0x7f57e80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f5801366640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db6cc0 0x563879dbb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39768/0 (socket says 172.21.15.83:39768) 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f5801366640 1 -- 172.21.15.83:0/1378592846 learned_addr learned my addr 172.21.15.83:0/1378592846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f5800b65640 1 --2- 172.21.15.83:0/1378592846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879dbbe30 0x563879dbe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f5801b67640 1 --2- 172.21.15.83:0/1378592846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879db62d0 0x563879db66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.085+0000 7f57f27fc640 1 -- 172.21.15.83:0/525170347 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f57f8014070 con 0x7f57fc0ff460 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.086+0000 7f5800b65640 1 -- 172.21.15.83:0/1378592846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879db62d0 msgr2=0x563879db66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.086+0000 7f5800b65640 1 --2- 172.21.15.83:0/1378592846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879db62d0 0x563879db66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.086+0000 7f5800b65640 1 -- 172.21.15.83:0/1378592846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db6cc0 msgr2=0x563879dbb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.086+0000 7f5800b65640 1 --2- 172.21.15.83:0/1378592846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db6cc0 0x563879dbb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.086+0000 7f5800b65640 1 -- 172.21.15.83:0/1378592846 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563879cdd570 con 0x563879dbbe30 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.086+0000 7f5801366640 1 --2- 172.21.15.83:0/1378592846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db6cc0 0x563879dbb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.086+0000 7f5801b67640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57e80a1a30 0x7f57e80a3ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f57f4005e30 tx=0x7f57f4005f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:37.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.086+0000 7f5800b65640 1 --2- 172.21.15.83:0/1378592846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879dbbe30 0x563879dbe2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f57ec005fd0 tx=0x7f57ec00bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37f423729ab6934a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.086+0000 7f57e77fe640 1 -- 172.21.15.83:0/1378592846 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec003bd0 con 0x563879dbbe30 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.086+0000 7f57e77fe640 1 -- 172.21.15.83:0/1378592846 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57ec003d30 con 0x563879dbbe30 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.086+0000 7f57e77fe640 1 -- 172.21.15.83:0/1378592846 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec004e40 con 0x563879dbbe30 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.087+0000 7f580619d740 1 -- 172.21.15.83:0/1378592846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879dbbe30 msgr2=0x563879dbe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.087+0000 7f580619d740 1 --2- 172.21.15.83:0/1378592846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879dbbe30 0x563879dbe2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f57ec005fd0 tx=0x7f57ec00bb20 comp rx=0 tx=0).stop 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.087+0000 7f580619d740 1 -- 172.21.15.83:0/1378592846 shutdown_connections 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.087+0000 7f580619d740 1 --2- 172.21.15.83:0/1378592846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879dbbe30 0x563879dbe2c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.087+0000 7f580619d740 1 --2- 172.21.15.83:0/1378592846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db6cc0 0x563879dbb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.087+0000 7f580619d740 1 --2- 172.21.15.83:0/1378592846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879db62d0 0x563879db66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.087+0000 7f580619d740 1 -- 172.21.15.83:0/1378592846 >> 172.21.15.83:0/1378592846 conn(0x563879db17f0 msgr2=0x563879db1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.087+0000 7f580619d740 1 -- 172.21.15.83:0/1378592846 shutdown_connections 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.087+0000 7f580619d740 1 -- 172.21.15.83:0/1378592846 wait complete. 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.087+0000 7f580619d740 1 -- start start 2024-03-11T19:39:37.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.087+0000 7f580619d740 1 Processor -- start 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f580619d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f580619d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db6cc0 0x563879dc07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f580619d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879dbbe30 0x563879dc7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f580619d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638799f2430 con 0x563879db62d0 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f580619d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638799aee00 con 0x563879dbbe30 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f580619d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563879ca3fd0 con 0x563879db6cc0 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f580619d740 10 client.0 _ll_register_callbacks cb 0x563879cf9fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f5801366640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f5801366640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39780/0 (socket says 172.21.15.83:39780) 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f5801366640 1 -- 172.21.15.83:0/667398965 learned_addr learned my addr 172.21.15.83:0/667398965 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f5800b65640 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db6cc0 0x563879dc07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f5801b67640 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879dbbe30 0x563879dc7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f5801366640 1 -- 172.21.15.83:0/667398965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db6cc0 msgr2=0x563879dc07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f5801366640 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db6cc0 0x563879dc07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f5801366640 1 -- 172.21.15.83:0/667398965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879dbbe30 msgr2=0x563879dc7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f5801366640 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879dbbe30 0x563879dc7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f5801366640 1 -- 172.21.15.83:0/667398965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec004290 con 0x563879db62d0 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f5801366640 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc0280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f57f8097190 tx=0x7f57f809a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1760bc754267fac server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f57e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f57e4ff9640 1 -- 172.21.15.83:0/667398965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f809aec0 con 0x563879db62d0 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f57e4ff9640 1 -- 172.21.15.83:0/667398965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f57f809a7c0 con 0x563879db62d0 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.088+0000 7f580619d740 1 -- 172.21.15.83:0/667398965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563879a1e7e0 con 0x563879db62d0 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f57e4ff9640 1 -- 172.21.15.83:0/667398965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57f809e910 con 0x563879db62d0 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f580619d740 10 client.15465 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f580619d740 1 -- 172.21.15.83:0/667398965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563879d1d610 con 0x563879db62d0 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f57e4ff9640 1 -- 172.21.15.83:0/667398965 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563879a1e7e0 con 0x563879db62d0 2024-03-11T19:39:37.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f57e4ff9640 1 -- 172.21.15.83:0/667398965 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f57f801f060 con 0x563879db62d0 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f580619d740 10 client.15465 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f580619d740 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f580619d740 20 client.15465 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f580619d740 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563879dcc540 0x563879dce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f580619d740 4 client.15465 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f580619d740 1 -- 172.21.15.83:0/667398965 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563879a5b720 con 0x563879dcc540 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.089+0000 7f5800b65640 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563879dcc540 0x563879dce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f5800b65640 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563879dcc540 0x563879dce980 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe174111590e3ea server_cookie=5cb466431c7cc80 in_seq=0 out_seq=0 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f57e4ff9640 10 client.15465 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f57e4ff9640 1 -- 172.21.15.83:0/667398965 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563879a5b720 con 0x563879dcc540 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f57e4ff9640 10 client.15465 handle_command_reply: tid=0 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f580619d740 1 client.15465 shutdown 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f580619d740 1 -- 172.21.15.83:0/667398965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 msgr2=0x563879dc0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:37.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f580619d740 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc0280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f57f8097190 tx=0x7f57f809a650 comp rx=0 tx=0).stop 2024-03-11T19:39:37.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f580619d740 1 -- 172.21.15.83:0/667398965 shutdown_connections 2024-03-11T19:39:37.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f580619d740 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563879dcc540 0x563879dce980 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f580619d740 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879dbbe30 0x563879dc7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f580619d740 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db6cc0 0x563879dc07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f57e4ff9640 0 client.15465 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:37.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f580619d740 1 --2- 172.21.15.83:0/667398965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc0280 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.090+0000 7f580619d740 1 -- 172.21.15.83:0/667398965 >> 172.21.15.83:0/667398965 conn(0x563879db17f0 msgr2=0x563879db0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:37.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.091+0000 7f580619d740 1 -- 172.21.15.83:0/667398965 shutdown_connections 2024-03-11T19:39:37.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.091+0000 7f580619d740 1 -- 172.21.15.83:0/667398965 wait complete. 2024-03-11T19:39:37.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.091+0000 7f580619d740 20 client.15465 trim_cache size 0 max 16384 2024-03-11T19:39:37.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.116+0000 7f580619d740 1 Processor -- start 2024-03-11T19:39:37.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.116+0000 7f580619d740 1 -- start start 2024-03-11T19:39:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.116+0000 7f580619d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db62d0 0x563879dbfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.116+0000 7f580619d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879db6cc0 0x563879dc7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.116+0000 7f580619d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879dbbe30 0x563879dc03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.116+0000 7f580619d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563879dad120 con 0x563879dbbe30 2024-03-11T19:39:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.116+0000 7f580619d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563879a1e7e0 con 0x563879db6cc0 2024-03-11T19:39:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.116+0000 7f580619d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563879dcc010 con 0x563879db62d0 2024-03-11T19:39:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.117+0000 7f5801b67640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879dbbe30 0x563879dc03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.117+0000 7f5801366640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db62d0 0x563879dbfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.117+0000 7f5801b67640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879dbbe30 0x563879dc03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39786/0 (socket says 172.21.15.83:39786) 2024-03-11T19:39:37.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.117+0000 7f5801b67640 1 -- 172.21.15.83:0/2536197105 learned_addr learned my addr 172.21.15.83:0/2536197105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:37.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.117+0000 7f5800b65640 1 --2- 172.21.15.83:0/2536197105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879db6cc0 0x563879dc7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.117+0000 7f5801b67640 1 -- 172.21.15.83:0/2536197105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db62d0 msgr2=0x563879dbfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:37.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.117+0000 7f5801b67640 1 --2- 172.21.15.83:0/2536197105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db62d0 0x563879dbfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.117+0000 7f5801b67640 1 -- 172.21.15.83:0/2536197105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879db6cc0 msgr2=0x563879dc7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:37.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.117+0000 7f5801b67640 1 --2- 172.21.15.83:0/2536197105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879db6cc0 0x563879dc7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.117+0000 7f5801b67640 1 -- 172.21.15.83:0/2536197105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563879cb9ac0 con 0x563879dbbe30 2024-03-11T19:39:37.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.118+0000 7f5801b67640 1 --2- 172.21.15.83:0/2536197105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879dbbe30 0x563879dc03d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f57f4036000 tx=0x7f57f404b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d66e1a1e568f66 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:37.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.118+0000 7f57e6ffd640 1 -- 172.21.15.83:0/2536197105 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4048070 con 0x563879dbbe30 2024-03-11T19:39:37.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.118+0000 7f57e6ffd640 1 -- 172.21.15.83:0/2536197105 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57f4004040 con 0x563879dbbe30 2024-03-11T19:39:37.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.118+0000 7f57e6ffd640 1 -- 172.21.15.83:0/2536197105 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f40506a0 con 0x563879dbbe30 2024-03-11T19:39:37.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.119+0000 7f580619d740 1 -- 172.21.15.83:0/2536197105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879dbbe30 msgr2=0x563879dc03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:37.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.119+0000 7f580619d740 1 --2- 172.21.15.83:0/2536197105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879dbbe30 0x563879dc03d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f57f4036000 tx=0x7f57f404b000 comp rx=0 tx=0).stop 2024-03-11T19:39:37.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.119+0000 7f580619d740 1 -- 172.21.15.83:0/2536197105 shutdown_connections 2024-03-11T19:39:37.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.119+0000 7f580619d740 1 --2- 172.21.15.83:0/2536197105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879dbbe30 0x563879dc03d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.119+0000 7f580619d740 1 --2- 172.21.15.83:0/2536197105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879db6cc0 0x563879dc7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.119+0000 7f580619d740 1 --2- 172.21.15.83:0/2536197105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db62d0 0x563879dbfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.119+0000 7f580619d740 1 -- 172.21.15.83:0/2536197105 >> 172.21.15.83:0/2536197105 conn(0x563879db17f0 msgr2=0x563879db1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:37.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.120+0000 7f580619d740 1 -- 172.21.15.83:0/2536197105 shutdown_connections 2024-03-11T19:39:37.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.120+0000 7f580619d740 1 -- 172.21.15.83:0/2536197105 wait complete. 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.120+0000 7f580619d740 1 -- start start 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.120+0000 7f580619d740 1 Processor -- start 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f580619d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f580619d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db6cc0 0x563879dc6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f580619d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879dbbe30 0x563879e2b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f580619d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563879e2b980 con 0x563879db62d0 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f580619d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563879e2bac0 con 0x563879dbbe30 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f580619d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563879e2bc30 con 0x563879db6cc0 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f580619d740 10 client.0 _ll_register_callbacks cb 0x7f57ec004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f5801366640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f5801366640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39794/0 (socket says 172.21.15.83:39794) 2024-03-11T19:39:37.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f5801366640 1 -- 172.21.15.83:0/2924823277 learned_addr learned my addr 172.21.15.83:0/2924823277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:39:37.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f5800b65640 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db6cc0 0x563879dc6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.121+0000 7f5801b67640 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879dbbe30 0x563879e2b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.122+0000 7f5800b65640 1 -- 172.21.15.83:0/2924823277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879dbbe30 msgr2=0x563879e2b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:37.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.122+0000 7f5800b65640 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879dbbe30 0x563879e2b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.122+0000 7f5800b65640 1 -- 172.21.15.83:0/2924823277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 msgr2=0x563879dc6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:39:37.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.122+0000 7f5800b65640 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.122+0000 7f5800b65640 1 -- 172.21.15.83:0/2924823277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc109ee0 con 0x563879db6cc0 2024-03-11T19:39:37.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.122+0000 7f5801366640 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:39:37.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.122+0000 7f5800b65640 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db6cc0 0x563879dc6eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00b490 tx=0x7f57ec005f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0eba79bbae609d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:39:37.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.122+0000 7f57e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.122+0000 7f57e4ff9640 1 -- 172.21.15.83:0/2924823277 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec027820 con 0x563879db6cc0 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.122+0000 7f57e4ff9640 1 -- 172.21.15.83:0/2924823277 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f57ec004680 con 0x563879db6cc0 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.122+0000 7f580619d740 1 -- 172.21.15.83:0/2924823277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57ec003d30 con 0x563879db6cc0 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.123+0000 7f57e4ff9640 1 -- 172.21.15.83:0/2924823277 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57ec036030 con 0x563879db6cc0 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.123+0000 7f580619d740 10 client.25081 fetch_fsmap learned FSMap version 24 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.123+0000 7f580619d740 1 -- 172.21.15.83:0/2924823277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563879cdf6a0 con 0x563879db6cc0 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.123+0000 7f57e4ff9640 1 -- 172.21.15.83:0/2924823277 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57ec003d30 con 0x563879db6cc0 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.123+0000 7f57e4ff9640 1 -- 172.21.15.83:0/2924823277 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f57ec011510 con 0x563879db6cc0 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.123+0000 7f580619d740 10 client.25081 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.123+0000 7f580619d740 10 client.25081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.123+0000 7f580619d740 20 client.25081 populate_metadata read hostname 'smithi083' 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.124+0000 7f580619d740 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563879e2d760 0x563879e2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:39:37.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.124+0000 7f580619d740 4 client.25081 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:39:37.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.124+0000 7f580619d740 1 -- 172.21.15.83:0/2924823277 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563879aaa890 con 0x563879e2d760 2024-03-11T19:39:37.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.124+0000 7f5801366640 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563879e2d760 0x563879e2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:39:37.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f5801366640 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563879e2d760 0x563879e2fb80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c26b4c3e9b0b5d7b server_cookie=82d2766903704716 in_seq=0 out_seq=0 2024-03-11T19:39:37.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f57e4ff9640 10 client.25081 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f57e4ff9640 1 -- 172.21.15.83:0/2924823277 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563879aaa890 con 0x563879e2d760 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f57e4ff9640 10 client.25081 handle_command_reply: tid=0 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f580619d740 1 client.25081 shutdown 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f580619d740 1 -- 172.21.15.83:0/2924823277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db6cc0 msgr2=0x563879dc6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f580619d740 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db6cc0 0x563879dc6eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00b490 tx=0x7f57ec005f40 comp rx=0 tx=0).stop 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f580619d740 1 -- 172.21.15.83:0/2924823277 shutdown_connections 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f580619d740 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563879e2d760 0x563879e2fb80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f580619d740 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563879dbbe30 0x563879e2b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f580619d740 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563879db6cc0 0x563879dc6eb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f580619d740 1 --2- 172.21.15.83:0/2924823277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563879db62d0 0x563879dc6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.127+0000 7f580619d740 1 -- 172.21.15.83:0/2924823277 >> 172.21.15.83:0/2924823277 conn(0x563879db17f0 msgr2=0x563879dbfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.128+0000 7f57e4ff9640 0 client.25081 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:39:37.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.128+0000 7f580619d740 1 -- 172.21.15.83:0/2924823277 shutdown_connections 2024-03-11T19:39:37.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.128+0000 7f580619d740 1 -- 172.21.15.83:0/2924823277 wait complete. 2024-03-11T19:39:37.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.128+0000 7f580619d740 20 client.25081 trim_cache size 0 max 16384 2024-03-11T19:39:37.133 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:39:37.133 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:39:37.133 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:39:37.133 INFO:teuthology.orchestra.run.smithi083.stdout: "caa9652b-e90f-4d33-be49-62c7a7a69af3": { 2024-03-11T19:39:37.133 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:39:37.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "caa9652b-e90f-4d33-be49-62c7a7a69af3", 2024-03-11T19:39:37.133 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:39:37.134 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:39:37.134 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:39:37.134 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:39:37.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.129+0000 7f58035ca640 1 -- 172.21.15.83:0/525170347 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57e80a1a30 msgr2=0x7f57e80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:37.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.129+0000 7f58035ca640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57e80a1a30 0x7f57e80a3ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f57f4005e30 tx=0x7f57f4005f80 comp rx=0 tx=0).stop 2024-03-11T19:39:37.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.129+0000 7f58035ca640 1 -- 172.21.15.83:0/525170347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc0ff460 msgr2=0x7f57fc1971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:39:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.129+0000 7f58035ca640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc0ff460 0x7f57fc1971e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f57f800e9c0 tx=0x7f57f800ee80 comp rx=0 tx=0).stop 2024-03-11T19:39:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.129+0000 7f58035ca640 1 -- 172.21.15.83:0/525170347 shutdown_connections 2024-03-11T19:39:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.129+0000 7f58035ca640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57e80a1a30 0x7f57e80a3ef0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.129+0000 7f58035ca640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57fc102e90 0x7f57fc19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.129+0000 7f58035ca640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57fc1024d0 0x7f57fc197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.129+0000 7f58035ca640 1 --2- 172.21.15.83:0/525170347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc0ff460 0x7f57fc1971e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:39:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.129+0000 7f58035ca640 1 -- 172.21.15.83:0/525170347 >> 172.21.15.83:0/525170347 conn(0x7f57fc0fad90 msgr2=0x7f57fc105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:39:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.130+0000 7f58035ca640 1 -- 172.21.15.83:0/525170347 shutdown_connections 2024-03-11T19:39:37.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:39:37.130+0000 7f58035ca640 1 -- 172.21.15.83:0/525170347 wait complete. 2024-03-11T19:39:37.152 INFO:tasks.cephfs.filesystem:scrub status for tag:caa9652b-e90f-4d33-be49-62c7a7a69af3 - {'path': '/', 'tag': 'caa9652b-e90f-4d33-be49-62c7a7a69af3', 'options': 'recursive,force'} 2024-03-11T19:39:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 20.0 s, 6270.3 tps, lat 15.917 ms stddev 16.420 2024-03-11T19:39:37.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:37 smithi137 ceph-mon[36220]: pgmap v361: 97 pgs: 97 active+clean; 973 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 991 op/s 2024-03-11T19:39:37.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:37 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:39:37.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:37 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:37.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:37 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:39:37.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:37 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:39:37.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:37 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:39:37.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:37 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:37.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:37 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:37.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:37 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:37 smithi204 ceph-mon[36739]: pgmap v361: 97 pgs: 97 active+clean; 973 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 991 op/s 2024-03-11T19:39:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:37 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:39:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:37 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:37 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:39:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:37 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:39:37.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:37 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:39:37.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:37 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:37.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:37 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:37.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:37 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:38.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:37 smithi083 ceph-mon[31864]: pgmap v361: 97 pgs: 97 active+clean; 973 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 MiB/s wr, 991 op/s 2024-03-11T19:39:38.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:37 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:39:38.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:37 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:38.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:37 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:39:38.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:37 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:39:38.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:37 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:39:38.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:37 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:38.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:37 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:38.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:37 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:39:39.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:39 smithi137 ceph-mon[36220]: pgmap v362: 97 pgs: 97 active+clean; 979 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-03-11T19:39:39.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:39 smithi204 ceph-mon[36739]: pgmap v362: 97 pgs: 97 active+clean; 979 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-03-11T19:39:40.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:39 smithi083 ceph-mon[31864]: pgmap v362: 97 pgs: 97 active+clean; 979 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-03-11T19:39:40.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:40 smithi137 ceph-mon[36220]: pgmap v363: 97 pgs: 97 active+clean; 985 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 968 op/s 2024-03-11T19:39:40.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:40 smithi204 ceph-mon[36739]: pgmap v363: 97 pgs: 97 active+clean; 985 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 968 op/s 2024-03-11T19:39:41.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:40 smithi083 ceph-mon[31864]: pgmap v363: 97 pgs: 97 active+clean; 985 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 968 op/s 2024-03-11T19:39:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 25.0 s, 6500.9 tps, lat 15.396 ms stddev 16.113 2024-03-11T19:39:43.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:43 smithi137 ceph-mon[36220]: pgmap v364: 97 pgs: 97 active+clean; 994 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-03-11T19:39:43.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:43 smithi204 ceph-mon[36739]: pgmap v364: 97 pgs: 97 active+clean; 994 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-03-11T19:39:44.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:43 smithi083 ceph-mon[31864]: pgmap v364: 97 pgs: 97 active+clean; 994 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-03-11T19:39:45.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:45 smithi137 ceph-mon[36220]: pgmap v365: 97 pgs: 97 active+clean; 999 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-03-11T19:39:45.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:45 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:39:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:45 smithi204 ceph-mon[36739]: pgmap v365: 97 pgs: 97 active+clean; 999 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-03-11T19:39:45.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:45 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:39:46.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:45 smithi083 ceph-mon[31864]: pgmap v365: 97 pgs: 97 active+clean; 999 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-03-11T19:39:46.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:45 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:39:46.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:46 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:39:46.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:46 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:39:46.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:46 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:39:46.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:46 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:39:47.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:46 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:39:47.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:46 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:39:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 30.0 s, 5724.5 tps, lat 17.476 ms stddev 17.549 2024-03-11T19:39:47.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:47 smithi137 ceph-mon[36220]: pgmap v366: 97 pgs: 97 active+clean; 1009 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-03-11T19:39:47.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:47 smithi204 ceph-mon[36739]: pgmap v366: 97 pgs: 97 active+clean; 1009 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-03-11T19:39:48.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:47 smithi083 ceph-mon[31864]: pgmap v366: 97 pgs: 97 active+clean; 1009 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-03-11T19:39:48.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:48 smithi137 ceph-mon[36220]: pgmap v367: 97 pgs: 97 active+clean; 1013 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-03-11T19:39:48.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:48 smithi204 ceph-mon[36739]: pgmap v367: 97 pgs: 97 active+clean; 1013 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-03-11T19:39:49.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:48 smithi083 ceph-mon[31864]: pgmap v367: 97 pgs: 97 active+clean; 1013 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-03-11T19:39:51.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:51 smithi137 ceph-mon[36220]: pgmap v368: 97 pgs: 97 active+clean; 1019 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 984 op/s 2024-03-11T19:39:51.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:51 smithi204 ceph-mon[36739]: pgmap v368: 97 pgs: 97 active+clean; 1019 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 984 op/s 2024-03-11T19:39:52.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:51 smithi083 ceph-mon[31864]: pgmap v368: 97 pgs: 97 active+clean; 1019 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 984 op/s 2024-03-11T19:39:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 35.0 s, 6192.6 tps, lat 16.112 ms stddev 18.465 2024-03-11T19:39:53.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:53 smithi083 ceph-mon[31864]: pgmap v369: 97 pgs: 97 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-11T19:39:53.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:53 smithi137 ceph-mon[36220]: pgmap v369: 97 pgs: 97 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-11T19:39:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:53 smithi204 ceph-mon[36739]: pgmap v369: 97 pgs: 97 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-03-11T19:39:55.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:55 smithi137 ceph-mon[36220]: pgmap v370: 97 pgs: 97 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 908 op/s 2024-03-11T19:39:55.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:55 smithi204 ceph-mon[36739]: pgmap v370: 97 pgs: 97 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 908 op/s 2024-03-11T19:39:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:55 smithi083 ceph-mon[31864]: pgmap v370: 97 pgs: 97 active+clean; 1.0 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 908 op/s 2024-03-11T19:39:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 40.0 s, 6104.8 tps, lat 16.419 ms stddev 17.285 2024-03-11T19:39:57.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:57 smithi137 ceph-mon[36220]: pgmap v371: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 MiB/s wr, 988 op/s 2024-03-11T19:39:57.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:57 smithi204 ceph-mon[36739]: pgmap v371: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 MiB/s wr, 988 op/s 2024-03-11T19:39:58.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:57 smithi083 ceph-mon[31864]: pgmap v371: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 MiB/s wr, 988 op/s 2024-03-11T19:39:59.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:39:59 smithi137 ceph-mon[36220]: pgmap v372: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2024-03-11T19:39:59.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:39:59 smithi204 ceph-mon[36739]: pgmap v372: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2024-03-11T19:40:00.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:39:59 smithi083 ceph-mon[31864]: pgmap v372: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2024-03-11T19:40:00.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:00 smithi137 ceph-mon[36220]: overall HEALTH_OK 2024-03-11T19:40:00.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:00 smithi204 ceph-mon[36739]: overall HEALTH_OK 2024-03-11T19:40:01.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:00 smithi083 ceph-mon[31864]: overall HEALTH_OK 2024-03-11T19:40:01.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:01 smithi137 ceph-mon[36220]: pgmap v373: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 822 op/s 2024-03-11T19:40:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:01 smithi204 ceph-mon[36739]: pgmap v373: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 822 op/s 2024-03-11T19:40:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:01 smithi083 ceph-mon[31864]: pgmap v373: 97 pgs: 97 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 822 op/s 2024-03-11T19:40:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 45.0 s, 5731.8 tps, lat 17.437 ms stddev 17.980 2024-03-11T19:40:02.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:02 smithi137 ceph-mon[36220]: pgmap v374: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 917 op/s 2024-03-11T19:40:02.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:02 smithi204 ceph-mon[36739]: pgmap v374: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 917 op/s 2024-03-11T19:40:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:02 smithi083 ceph-mon[31864]: pgmap v374: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 917 op/s 2024-03-11T19:40:05.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:05 smithi137 ceph-mon[36220]: pgmap v375: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-03-11T19:40:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:05 smithi204 ceph-mon[36739]: pgmap v375: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-03-11T19:40:06.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:05 smithi083 ceph-mon[31864]: pgmap v375: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-03-11T19:40:07.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:40:07.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:40:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.323+0000 7f34a6913640 1 -- 172.21.15.83:0/2961124612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0101c30 msgr2=0x7f34a01084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.323+0000 7f34a6913640 1 --2- 172.21.15.83:0/2961124612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0101c30 0x7f34a01084c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3490009900 tx=0x7f349002bc30 comp rx=0 tx=0).stop 2024-03-11T19:40:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.323+0000 7f34a6913640 1 -- 172.21.15.83:0/2961124612 shutdown_connections 2024-03-11T19:40:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.323+0000 7f34a6913640 1 --2- 172.21.15.83:0/2961124612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0101c30 0x7f34a01084c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.323+0000 7f34a6913640 1 --2- 172.21.15.83:0/2961124612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34a0101270 0x7f34a01016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.323+0000 7f34a6913640 1 --2- 172.21.15.83:0/2961124612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0100080 0x7f34a01004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.323+0000 7f34a6913640 1 -- 172.21.15.83:0/2961124612 >> 172.21.15.83:0/2961124612 conn(0x7f34a00fb5f0 msgr2=0x7f34a00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.323+0000 7f34a6913640 1 -- 172.21.15.83:0/2961124612 shutdown_connections 2024-03-11T19:40:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.323+0000 7f34a6913640 1 -- 172.21.15.83:0/2961124612 wait complete. 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.324+0000 7f34a6913640 1 Processor -- start 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.324+0000 7f34a6913640 1 -- start start 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.325+0000 7f34a6913640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0100080 0x7f34a019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.325+0000 7f34a6913640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34a0101270 0x7f34a019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.325+0000 7f34a6913640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0101c30 0x7f34a01a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.325+0000 7f34a6913640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a01a5280 con 0x7f34a0100080 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.325+0000 7f34a6913640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a01a53f0 con 0x7f34a0101270 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.325+0000 7f34a6913640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a01a55b0 con 0x7f34a0101c30 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.325+0000 7f34a4eb0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0101c30 0x7f34a01a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.325+0000 7f34a4eb0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0101c30 0x7f34a01a4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40050/0 (socket says 172.21.15.83:40050) 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.325+0000 7f34a4eb0640 1 -- 172.21.15.83:0/2144674982 learned_addr learned my addr 172.21.15.83:0/2144674982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.326+0000 7f349ffff640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0100080 0x7f34a019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.326+0000 7f34a4eb0640 1 -- 172.21.15.83:0/2144674982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34a0101270 msgr2=0x7f34a019dcd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.326+0000 7f34a4eb0640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34a0101270 0x7f34a019dcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.326+0000 7f34a4eb0640 1 -- 172.21.15.83:0/2144674982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0100080 msgr2=0x7f34a019d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.326+0000 7f34a4eb0640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0100080 0x7f34a019d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.326+0000 7f34a4eb0640 1 -- 172.21.15.83:0/2144674982 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3490009560 con 0x7f34a0101c30 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.326+0000 7f349ffff640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0100080 0x7f34a019d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.326+0000 7f34a4eb0640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0101c30 0x7f34a01a4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3490036040 tx=0x7f3490005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.326+0000 7f349d7fa640 1 -- 172.21.15.83:0/2144674982 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490004490 con 0x7f34a0101c30 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.327+0000 7f34a6913640 1 -- 172.21.15.83:0/2144674982 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a01a5830 con 0x7f34a0101c30 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.327+0000 7f34a6913640 1 -- 172.21.15.83:0/2144674982 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a01a5d90 con 0x7f34a0101c30 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.327+0000 7f349d7fa640 1 -- 172.21.15.83:0/2144674982 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34900052d0 con 0x7f34a0101c30 2024-03-11T19:40:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.327+0000 7f349d7fa640 1 -- 172.21.15.83:0/2144674982 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490003e30 con 0x7f34a0101c30 2024-03-11T19:40:07.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.329+0000 7f349d7fa640 1 -- 172.21.15.83:0/2144674982 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f349003e450 con 0x7f34a0101c30 2024-03-11T19:40:07.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.330+0000 7f34a95c9740 1 Processor -- start 2024-03-11T19:40:07.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.330+0000 7f34a95c9740 1 -- start start 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.330+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3f2d0 0x55e54df3f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.330+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df3fcc0 0x55e54df447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.330+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df44e30 0x55e54df472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.330+0000 7f34a95c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54deaf280 con 0x55e54df44e30 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.330+0000 7f34a95c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54deac7b0 con 0x55e54df3fcc0 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.330+0000 7f34a95c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54dea7f90 con 0x55e54df3f2d0 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.330+0000 7f349d7fa640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34880a1ab0 0x7f34880a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.331+0000 7f349f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3f2d0 0x55e54df3f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.331+0000 7f349f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3f2d0 0x55e54df3f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40052/0 (socket says 172.21.15.83:40052) 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.331+0000 7f349d7fa640 1 -- 172.21.15.83:0/2144674982 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f34900efbb0 con 0x7f34a0101c30 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.331+0000 7f349f7fe640 1 -- 172.21.15.83:0/4770147 learned_addr learned my addr 172.21.15.83:0/4770147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.331+0000 7f349f7fe640 1 --2- 172.21.15.83:0/4770147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df44e30 0x55e54df472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.331+0000 7f349f7fe640 1 -- 172.21.15.83:0/4770147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df3fcc0 msgr2=0x55e54df447c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:40:07.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.331+0000 7f349f7fe640 1 --2- 172.21.15.83:0/4770147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df3fcc0 0x55e54df447c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.332+0000 7f349f7fe640 1 -- 172.21.15.83:0/4770147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df44e30 msgr2=0x55e54df472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:07.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.332+0000 7f349f7fe640 1 --2- 172.21.15.83:0/4770147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df44e30 0x55e54df472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.332+0000 7f34a4eb0640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34880a1ab0 0x7f34880a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.332+0000 7f349f7fe640 1 -- 172.21.15.83:0/4770147 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e54de66570 con 0x55e54df3f2d0 2024-03-11T19:40:07.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.332+0000 7f349f7fe640 1 --2- 172.21.15.83:0/4770147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3f2d0 0x55e54df3f6f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3494009fb0 tx=0x7f3494009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f85b1159e437e2e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:07.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.333+0000 7f34a4eb0640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34880a1ab0 0x7f34880a3f70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f34900bcac0 tx=0x7f3490009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:07.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.333+0000 7f34867fc640 1 -- 172.21.15.83:0/4770147 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494013110 con 0x55e54df3f2d0 2024-03-11T19:40:07.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.333+0000 7f34867fc640 1 -- 172.21.15.83:0/4770147 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3494019030 con 0x55e54df3f2d0 2024-03-11T19:40:07.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.333+0000 7f34867fc640 1 -- 172.21.15.83:0/4770147 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349400edc0 con 0x55e54df3f2d0 2024-03-11T19:40:07.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.333+0000 7f34a95c9740 1 -- 172.21.15.83:0/4770147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3f2d0 msgr2=0x55e54df3f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.333+0000 7f34a95c9740 1 --2- 172.21.15.83:0/4770147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3f2d0 0x55e54df3f6f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3494009fb0 tx=0x7f3494009640 comp rx=0 tx=0).stop 2024-03-11T19:40:07.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.334+0000 7f34a95c9740 1 -- 172.21.15.83:0/4770147 shutdown_connections 2024-03-11T19:40:07.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.334+0000 7f34a95c9740 1 --2- 172.21.15.83:0/4770147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df44e30 0x55e54df472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.334+0000 7f34a95c9740 1 --2- 172.21.15.83:0/4770147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df3fcc0 0x55e54df447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.334+0000 7f34a95c9740 1 --2- 172.21.15.83:0/4770147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3f2d0 0x55e54df3f6f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.334+0000 7f34a95c9740 1 -- 172.21.15.83:0/4770147 >> 172.21.15.83:0/4770147 conn(0x55e54df3a7f0 msgr2=0x55e54df3abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:07.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.334+0000 7f34a95c9740 1 -- 172.21.15.83:0/4770147 shutdown_connections 2024-03-11T19:40:07.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.335+0000 7f34a95c9740 1 -- 172.21.15.83:0/4770147 wait complete. 2024-03-11T19:40:07.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.335+0000 7f34a95c9740 1 -- start start 2024-03-11T19:40:07.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.335+0000 7f34a95c9740 1 Processor -- start 2024-03-11T19:40:07.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df49310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df3fcc0 0x55e54df49850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 0x55e54df50890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f34a95c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54db7b430 con 0x55e54df3f2d0 2024-03-11T19:40:07.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f34a95c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54db37e00 con 0x55e54df3fcc0 2024-03-11T19:40:07.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f34a95c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54de2cfd0 con 0x55e54df44e30 2024-03-11T19:40:07.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f34a95c9740 10 client.0 _ll_register_callbacks cb 0x55e54de82fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f349f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df49310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f349f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df49310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37492/0 (socket says 172.21.15.83:37492) 2024-03-11T19:40:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f349f7fe640 1 -- 172.21.15.83:0/163803731 learned_addr learned my addr 172.21.15.83:0/163803731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f349f7fe640 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 0x55e54df50890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f349ffff640 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df3fcc0 0x55e54df49850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f349f7fe640 1 -- 172.21.15.83:0/163803731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 msgr2=0x55e54df50890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.336+0000 7f349f7fe640 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 0x55e54df50890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.337+0000 7f349f7fe640 1 -- 172.21.15.83:0/163803731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df3fcc0 msgr2=0x55e54df49850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.337+0000 7f349f7fe640 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df3fcc0 0x55e54df49850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.337+0000 7f349f7fe640 1 -- 172.21.15.83:0/163803731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3494013460 con 0x55e54df3f2d0 2024-03-11T19:40:07.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.337+0000 7f349ffff640 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df3fcc0 0x55e54df49850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:40:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.338+0000 7f349f7fe640 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df49310 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3494009f80 tx=0x7f3494005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3619cbccd6ecad3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.338+0000 7f347f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:40:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.338+0000 7f347f7fe640 1 -- 172.21.15.83:0/163803731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494013260 con 0x55e54df3f2d0 2024-03-11T19:40:07.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.338+0000 7f34a95c9740 1 -- 172.21.15.83:0/163803731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e54dba77e0 con 0x55e54df3f2d0 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.338+0000 7f347f7fe640 1 -- 172.21.15.83:0/163803731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3494004440 con 0x55e54df3f2d0 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.338+0000 7f347f7fe640 1 -- 172.21.15.83:0/163803731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34940035f0 con 0x55e54df3f2d0 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.338+0000 7f34a95c9740 10 client.15480 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.338+0000 7f34a95c9740 1 -- 172.21.15.83:0/163803731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e54dea6610 con 0x55e54df3f2d0 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.339+0000 7f347f7fe640 1 -- 172.21.15.83:0/163803731 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e54dba77e0 con 0x55e54df3f2d0 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.340+0000 7f347f7fe640 1 -- 172.21.15.83:0/163803731 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f349401e550 con 0x55e54df3f2d0 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.340+0000 7f34a95c9740 10 client.15480 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.340+0000 7f34a95c9740 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.340+0000 7f34a95c9740 20 client.15480 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.340+0000 7f34a95c9740 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e54df55390 0x55e54df577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.340+0000 7f34a95c9740 4 client.15480 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.340+0000 7f34a95c9740 1 -- 172.21.15.83:0/163803731 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e54dbe4720 con 0x55e54df55390 2024-03-11T19:40:07.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.341+0000 7f349ffff640 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e54df55390 0x55e54df577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.341+0000 7f349ffff640 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e54df55390 0x55e54df577b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ed3078945f0e25 server_cookie=c6605a5226156dad in_seq=0 out_seq=0 2024-03-11T19:40:07.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.342+0000 7f347f7fe640 10 client.15480 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.342+0000 7f347f7fe640 1 -- 172.21.15.83:0/163803731 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e54dbe4720 con 0x55e54df55390 2024-03-11T19:40:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.342+0000 7f347f7fe640 10 client.15480 handle_command_reply: tid=0 2024-03-11T19:40:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.343+0000 7f34a95c9740 1 client.15480 shutdown 2024-03-11T19:40:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.343+0000 7f34a95c9740 1 -- 172.21.15.83:0/163803731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 msgr2=0x55e54df49310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.343+0000 7f34a95c9740 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df49310 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3494009f80 tx=0x7f3494005f90 comp rx=0 tx=0).stop 2024-03-11T19:40:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.343+0000 7f34a95c9740 1 -- 172.21.15.83:0/163803731 shutdown_connections 2024-03-11T19:40:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.343+0000 7f34a95c9740 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e54df55390 0x55e54df577b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.343+0000 7f34a95c9740 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 0x55e54df50890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.343+0000 7f34a95c9740 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df3fcc0 0x55e54df49850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.343+0000 7f34a95c9740 1 --2- 172.21.15.83:0/163803731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df49310 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.343+0000 7f34a95c9740 1 -- 172.21.15.83:0/163803731 >> 172.21.15.83:0/163803731 conn(0x55e54df3a7f0 msgr2=0x55e54df39a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.343+0000 7f347f7fe640 0 client.15480 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.344+0000 7f34a95c9740 1 -- 172.21.15.83:0/163803731 shutdown_connections 2024-03-11T19:40:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.344+0000 7f34a95c9740 1 -- 172.21.15.83:0/163803731 wait complete. 2024-03-11T19:40:07.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.344+0000 7f34a95c9740 20 client.15480 trim_cache size 0 max 16384 2024-03-11T19:40:07.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.371+0000 7f34a95c9740 1 Processor -- start 2024-03-11T19:40:07.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.371+0000 7f34a95c9740 1 -- start start 2024-03-11T19:40:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.371+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df40f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.371+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3fcc0 0x55e54df50850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.371+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df44e30 0x55e54df41480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.371+0000 7f34a95c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54df51d80 con 0x55e54df3f2d0 2024-03-11T19:40:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.371+0000 7f34a95c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54df47e60 con 0x55e54df44e30 2024-03-11T19:40:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.371+0000 7f34a95c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54df54f00 con 0x55e54df3fcc0 2024-03-11T19:40:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.371+0000 7f349ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3fcc0 0x55e54df50850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.371+0000 7f349ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3fcc0 0x55e54df50850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40070/0 (socket says 172.21.15.83:40070) 2024-03-11T19:40:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.371+0000 7f349ffff640 1 -- 172.21.15.83:0/3066161931 learned_addr learned my addr 172.21.15.83:0/3066161931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.372+0000 7f349f7fe640 1 --2- 172.21.15.83:0/3066161931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df40f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.372+0000 7f349f7fe640 1 --2- 172.21.15.83:0/3066161931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df44e30 0x55e54df41480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.372+0000 7f349ffff640 1 -- 172.21.15.83:0/3066161931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df44e30 msgr2=0x55e54df41480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.372+0000 7f349ffff640 1 --2- 172.21.15.83:0/3066161931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df44e30 0x55e54df41480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.372+0000 7f349ffff640 1 -- 172.21.15.83:0/3066161931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 msgr2=0x55e54df40f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.372+0000 7f349ffff640 1 --2- 172.21.15.83:0/3066161931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df40f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.372+0000 7f349ffff640 1 -- 172.21.15.83:0/3066161931 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e54de42ac0 con 0x55e54df3fcc0 2024-03-11T19:40:07.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.373+0000 7f349ffff640 1 --2- 172.21.15.83:0/3066161931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3fcc0 0x55e54df50850 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34a01010d0 tx=0x7f349800b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2fd63e1275e6af3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:07.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.373+0000 7f3485ffb640 1 -- 172.21.15.83:0/3066161931 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498004e70 con 0x55e54df3fcc0 2024-03-11T19:40:07.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.373+0000 7f3485ffb640 1 -- 172.21.15.83:0/3066161931 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3498011070 con 0x55e54df3fcc0 2024-03-11T19:40:07.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.373+0000 7f3485ffb640 1 -- 172.21.15.83:0/3066161931 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34980056b0 con 0x55e54df3fcc0 2024-03-11T19:40:07.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.375+0000 7f34a95c9740 1 -- 172.21.15.83:0/3066161931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3fcc0 msgr2=0x55e54df50850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.375+0000 7f34a95c9740 1 --2- 172.21.15.83:0/3066161931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3fcc0 0x55e54df50850 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34a01010d0 tx=0x7f349800b710 comp rx=0 tx=0).stop 2024-03-11T19:40:07.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.375+0000 7f34a95c9740 1 -- 172.21.15.83:0/3066161931 shutdown_connections 2024-03-11T19:40:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.375+0000 7f34a95c9740 1 --2- 172.21.15.83:0/3066161931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df44e30 0x55e54df41480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.375+0000 7f34a95c9740 1 --2- 172.21.15.83:0/3066161931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df3fcc0 0x55e54df50850 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f34a01010d0 tx=0x7f349800b710 comp rx=0 tx=0).stop 2024-03-11T19:40:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.375+0000 7f34a95c9740 1 --2- 172.21.15.83:0/3066161931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df40f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.376+0000 7f34a95c9740 1 -- 172.21.15.83:0/3066161931 >> 172.21.15.83:0/3066161931 conn(0x55e54df3a7f0 msgr2=0x55e54df5c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.376+0000 7f34a95c9740 1 -- 172.21.15.83:0/3066161931 shutdown_connections 2024-03-11T19:40:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.376+0000 7f34a95c9740 1 -- 172.21.15.83:0/3066161931 wait complete. 2024-03-11T19:40:07.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.376+0000 7f34a95c9740 1 -- start start 2024-03-11T19:40:07.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.377+0000 7f34a95c9740 1 Processor -- start 2024-03-11T19:40:07.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.377+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.377+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 0x55e54df4eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.377+0000 7f34a95c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df4f400 0x55e54df4f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.377+0000 7f34a95c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54dfab900 con 0x55e54df3f2d0 2024-03-11T19:40:07.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.378+0000 7f34a95c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54dfaba70 con 0x55e54df4f400 2024-03-11T19:40:07.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.378+0000 7f34a95c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e54dfabc30 con 0x55e54df44e30 2024-03-11T19:40:07.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.378+0000 7f34a95c9740 10 client.0 _ll_register_callbacks cb 0x7f3494009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:07.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.378+0000 7f349ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 0x55e54df4eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.378+0000 7f349ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 0x55e54df4eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40076/0 (socket says 172.21.15.83:40076) 2024-03-11T19:40:07.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.378+0000 7f349ffff640 1 -- 172.21.15.83:0/471044140 learned_addr learned my addr 172.21.15.83:0/471044140 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:07.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.378+0000 7f349ffff640 1 -- 172.21.15.83:0/471044140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df4f400 msgr2=0x55e54df4f880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:40:07.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.378+0000 7f349ffff640 1 --2- 172.21.15.83:0/471044140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df4f400 0x55e54df4f880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.378+0000 7f349ffff640 1 -- 172.21.15.83:0/471044140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 msgr2=0x55e54df508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:07.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.378+0000 7f349ffff640 1 --2- 172.21.15.83:0/471044140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.378+0000 7f349ffff640 1 -- 172.21.15.83:0/471044140 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349800bd80 con 0x55e54df44e30 2024-03-11T19:40:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.379+0000 7f349ffff640 1 --2- 172.21.15.83:0/471044140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 0x55e54df4eec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3498004a80 tx=0x7f34980050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca9ea2063ec7480f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.379+0000 7f347f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:40:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.379+0000 7f347f7fe640 1 -- 172.21.15.83:0/471044140 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498028070 con 0x55e54df44e30 2024-03-11T19:40:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.379+0000 7f347f7fe640 1 -- 172.21.15.83:0/471044140 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3498005430 con 0x55e54df44e30 2024-03-11T19:40:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.379+0000 7f34a95c9740 1 -- 172.21.15.83:0/471044140 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3494019030 con 0x55e54df44e30 2024-03-11T19:40:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.379+0000 7f347f7fe640 1 -- 172.21.15.83:0/471044140 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34980055f0 con 0x55e54df44e30 2024-03-11T19:40:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.380+0000 7f34a95c9740 10 client.25099 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.380+0000 7f34a95c9740 1 -- 172.21.15.83:0/471044140 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e54de686a0 con 0x55e54df44e30 2024-03-11T19:40:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.381+0000 7f347f7fe640 1 -- 172.21.15.83:0/471044140 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3494019030 con 0x55e54df44e30 2024-03-11T19:40:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.381+0000 7f347f7fe640 1 -- 172.21.15.83:0/471044140 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3498023470 con 0x55e54df44e30 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.381+0000 7f34a95c9740 10 client.25099 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.381+0000 7f34a95c9740 10 client.25099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.381+0000 7f34a95c9740 20 client.25099 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.381+0000 7f34a95c9740 1 --2- 172.21.15.83:0/471044140 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e54dfad730 0x55e54dfafb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.381+0000 7f34a95c9740 4 client.25099 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.381+0000 7f34a95c9740 1 -- 172.21.15.83:0/471044140 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e54dc33890 con 0x55e54dfad730 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.381+0000 7f349f7fe640 1 --2- 172.21.15.83:0/471044140 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e54dfad730 0x55e54dfafb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.382+0000 7f347f7fe640 10 client.25099 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.382+0000 7f349f7fe640 1 --2- 172.21.15.83:0/471044140 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e54dfad730 0x55e54dfafb50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=338bc63eecda4362 server_cookie=ae74f51ec740781a in_seq=0 out_seq=0 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.383+0000 7f347f7fe640 1 -- 172.21.15.83:0/471044140 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e54dc33890 con 0x55e54dfad730 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.383+0000 7f347f7fe640 10 client.25099 handle_command_reply: tid=0 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.383+0000 7f34a95c9740 1 client.25099 shutdown 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a95c9740 1 -- 172.21.15.83:0/471044140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 msgr2=0x55e54df4eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a95c9740 1 --2- 172.21.15.83:0/471044140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 0x55e54df4eec0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3498004a80 tx=0x7f34980050b0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a95c9740 1 -- 172.21.15.83:0/471044140 shutdown_connections 2024-03-11T19:40:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a95c9740 1 --2- 172.21.15.83:0/471044140 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e54dfad730 0x55e54dfafb50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a95c9740 1 --2- 172.21.15.83:0/471044140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e54df4f400 0x55e54df4f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a95c9740 1 --2- 172.21.15.83:0/471044140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e54df44e30 0x55e54df4eec0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a95c9740 1 --2- 172.21.15.83:0/471044140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e54df3f2d0 0x55e54df508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f347f7fe640 0 client.25099 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a95c9740 1 -- 172.21.15.83:0/471044140 >> 172.21.15.83:0/471044140 conn(0x55e54df3a7f0 msgr2=0x55e54df40f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a95c9740 1 -- 172.21.15.83:0/471044140 shutdown_connections 2024-03-11T19:40:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a95c9740 1 -- 172.21.15.83:0/471044140 wait complete. 2024-03-11T19:40:07.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a95c9740 20 client.25099 trim_cache size 0 max 16384 2024-03-11T19:40:07.391 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:40:07.391 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:40:07.391 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:40:07.391 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:40:07.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a6913640 1 -- 172.21.15.83:0/2144674982 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34880a1ab0 msgr2=0x7f34880a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a6913640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34880a1ab0 0x7f34880a3f70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f34900bcac0 tx=0x7f3490009260 comp rx=0 tx=0).stop 2024-03-11T19:40:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a6913640 1 -- 172.21.15.83:0/2144674982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0101c30 msgr2=0x7f34a01a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.384+0000 7f34a6913640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0101c30 0x7f34a01a4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3490036040 tx=0x7f3490005e20 comp rx=0 tx=0).stop 2024-03-11T19:40:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.385+0000 7f34a6913640 1 -- 172.21.15.83:0/2144674982 shutdown_connections 2024-03-11T19:40:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.385+0000 7f34a6913640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f34880a1ab0 0x7f34880a3f70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.385+0000 7f34a6913640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0101c30 0x7f34a01a4d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.385+0000 7f34a6913640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34a0101270 0x7f34a019dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.385+0000 7f34a6913640 1 --2- 172.21.15.83:0/2144674982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34a0100080 0x7f34a019d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.385+0000 7f34a6913640 1 -- 172.21.15.83:0/2144674982 >> 172.21.15.83:0/2144674982 conn(0x7f34a00fb5f0 msgr2=0x7f34a0102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.387+0000 7f34a6913640 1 -- 172.21.15.83:0/2144674982 shutdown_connections 2024-03-11T19:40:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.387+0000 7f34a6913640 1 -- 172.21.15.83:0/2144674982 wait complete. 2024-03-11T19:40:07.409 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:40:07.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:40:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 50.0 s, 6487.2 tps, lat 15.390 ms stddev 16.402 2024-03-11T19:40:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.567+0000 7f71c78c9640 1 -- 172.21.15.83:0/3864612945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c0108180 msgr2=0x7f71c010a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.567+0000 7f71c78c9640 1 --2- 172.21.15.83:0/3864612945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c0108180 0x7f71c010a5b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009900 tx=0x7f71bc02bc40 comp rx=0 tx=0).stop 2024-03-11T19:40:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.567+0000 7f71c78c9640 1 -- 172.21.15.83:0/3864612945 shutdown_connections 2024-03-11T19:40:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.567+0000 7f71c78c9640 1 --2- 172.21.15.83:0/3864612945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71c010d4c0 0x7f71c010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.567+0000 7f71c78c9640 1 --2- 172.21.15.83:0/3864612945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c010aaf0 0x7f71c010cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.567+0000 7f71c78c9640 1 --2- 172.21.15.83:0/3864612945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c0108180 0x7f71c010a5b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.567+0000 7f71c78c9640 1 -- 172.21.15.83:0/3864612945 >> 172.21.15.83:0/3864612945 conn(0x7f71c006dff0 msgr2=0x7f71c0070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.567+0000 7f71c78c9640 1 -- 172.21.15.83:0/3864612945 shutdown_connections 2024-03-11T19:40:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.567+0000 7f71c78c9640 1 -- 172.21.15.83:0/3864612945 wait complete. 2024-03-11T19:40:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.568+0000 7f71c78c9640 1 Processor -- start 2024-03-11T19:40:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.568+0000 7f71c78c9640 1 -- start start 2024-03-11T19:40:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.568+0000 7f71c78c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c0108180 0x7f71c01a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.569+0000 7f71c78c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c010aaf0 0x7f71c01a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.569+0000 7f71c78c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71c010d4c0 0x7f71c01ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.569+0000 7f71c5665640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c0108180 0x7f71c01a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.569+0000 7f71c5665640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c0108180 0x7f71c01a5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37544/0 (socket says 172.21.15.83:37544) 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.569+0000 7f71c5665640 1 -- 172.21.15.83:0/855895418 learned_addr learned my addr 172.21.15.83:0/855895418 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.569+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c01ad9c0 con 0x7f71c0108180 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.569+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c01adb30 con 0x7f71c010d4c0 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.569+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c01adcf0 con 0x7f71c010aaf0 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.569+0000 7f71c5e66640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71c010d4c0 0x7f71c01ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.570+0000 7f71c4e64640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c010aaf0 0x7f71c01a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.570+0000 7f71c5e66640 1 -- 172.21.15.83:0/855895418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c010aaf0 msgr2=0x7f71c01a6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.570+0000 7f71c5e66640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c010aaf0 0x7f71c01a6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.570+0000 7f71c5e66640 1 -- 172.21.15.83:0/855895418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c0108180 msgr2=0x7f71c01a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.570+0000 7f71c5e66640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c0108180 0x7f71c01a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.570+0000 7f71c5e66640 1 -- 172.21.15.83:0/855895418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc009560 con 0x7f71c010d4c0 2024-03-11T19:40:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.570+0000 7f71c4e64640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c010aaf0 0x7f71c01a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:40:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.570+0000 7f71c5e66640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71c010d4c0 0x7f71c01ad480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f71b8005fd0 tx=0x7f71b800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.570+0000 7f71b67fc640 1 -- 172.21.15.83:0/855895418 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800cdf0 con 0x7f71c010d4c0 2024-03-11T19:40:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.570+0000 7f71b67fc640 1 -- 172.21.15.83:0/855895418 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71b8005020 con 0x7f71c010d4c0 2024-03-11T19:40:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.570+0000 7f71b67fc640 1 -- 172.21.15.83:0/855895418 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8010690 con 0x7f71c010d4c0 2024-03-11T19:40:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.571+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c01adfd0 con 0x7f71c010d4c0 2024-03-11T19:40:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.571+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c01ae3a0 con 0x7f71c010d4c0 2024-03-11T19:40:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.572+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c0070af0 con 0x7f71c010d4c0 2024-03-11T19:40:07.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.574+0000 7f71b67fc640 1 -- 172.21.15.83:0/855895418 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f71b8005190 con 0x7f71c010d4c0 2024-03-11T19:40:07.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.575+0000 7f71b67fc640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71ac0a1b30 0x7f71ac0a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:07.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.575+0000 7f71b67fc640 1 -- 172.21.15.83:0/855895418 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f71b8014070 con 0x7f71c010d4c0 2024-03-11T19:40:07.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.575+0000 7f71c5665640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71ac0a1b30 0x7f71ac0a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:07.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.576+0000 7f71c5665640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71ac0a1b30 0x7f71ac0a3ff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f71bc02c180 tx=0x7f71bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:07.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.578+0000 7f71b67fc640 1 -- 172.21.15.83:0/855895418 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f71b809a800 con 0x7f71c010d4c0 2024-03-11T19:40:07.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:07 smithi083 ceph-mon[31864]: pgmap v376: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-03-11T19:40:07.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:07 smithi137 ceph-mon[36220]: pgmap v376: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-03-11T19:40:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:07 smithi204 ceph-mon[36739]: pgmap v376: 97 pgs: 97 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-03-11T19:40:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.970+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71c011ba20 con 0x7f71c010d4c0 2024-03-11T19:40:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.972+0000 7f71b67fc640 1 -- 172.21.15.83:0/855895418 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11352 (secure 0 0 0) 0x7f71b809a390 con 0x7f71c010d4c0 2024-03-11T19:40:07.976 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:40:07.976 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:39:20.446302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:40:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-03-11T19:40:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.975+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71ac0a1b30 msgr2=0x7f71ac0a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.975+0000 7f71c78c9640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71ac0a1b30 0x7f71ac0a3ff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f71bc02c180 tx=0x7f71bc000bc0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.975+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71c010d4c0 msgr2=0x7f71c01ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.975+0000 7f71c78c9640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71c010d4c0 0x7f71c01ad480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f71b8005fd0 tx=0x7f71b800ef10 comp rx=0 tx=0).stop 2024-03-11T19:40:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.975+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 shutdown_connections 2024-03-11T19:40:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.975+0000 7f71c78c9640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f71ac0a1b30 0x7f71ac0a3ff0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.976+0000 7f71c78c9640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71c010d4c0 0x7f71c01ad480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.976+0000 7f71c78c9640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c010aaf0 0x7f71c01a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.976+0000 7f71c78c9640 1 --2- 172.21.15.83:0/855895418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71c0108180 0x7f71c01a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.976+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 >> 172.21.15.83:0/855895418 conn(0x7f71c006dff0 msgr2=0x7f71c010b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.976+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 shutdown_connections 2024-03-11T19:40:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:07.976+0000 7f71c78c9640 1 -- 172.21.15.83:0/855895418 wait complete. 2024-03-11T19:40:07.998 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:40:07.999 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:40:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.167+0000 7f0b4929a640 1 -- 172.21.15.83:0/3653191291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b44104f70 msgr2=0x7f0b4410b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.167+0000 7f0b4929a640 1 --2- 172.21.15.83:0/3653191291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b44104f70 0x7f0b4410b800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009be0 tx=0x7f0b3c02bef0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.167+0000 7f0b4929a640 1 -- 172.21.15.83:0/3653191291 shutdown_connections 2024-03-11T19:40:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.167+0000 7f0b4929a640 1 --2- 172.21.15.83:0/3653191291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b44104f70 0x7f0b4410b800 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.167+0000 7f0b4929a640 1 --2- 172.21.15.83:0/3653191291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b441045b0 0x7f0b44104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.167+0000 7f0b4929a640 1 --2- 172.21.15.83:0/3653191291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b441033c0 0x7f0b441037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.167+0000 7f0b4929a640 1 -- 172.21.15.83:0/3653191291 >> 172.21.15.83:0/3653191291 conn(0x7f0b440fe930 msgr2=0x7f0b44100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.167+0000 7f0b4929a640 1 -- 172.21.15.83:0/3653191291 shutdown_connections 2024-03-11T19:40:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.167+0000 7f0b4929a640 1 -- 172.21.15.83:0/3653191291 wait complete. 2024-03-11T19:40:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.168+0000 7f0b4929a640 1 Processor -- start 2024-03-11T19:40:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.168+0000 7f0b4929a640 1 -- start start 2024-03-11T19:40:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.169+0000 7f0b4929a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b441033c0 0x7f0b44199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.169+0000 7f0b4929a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b441045b0 0x7f0b44199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.169+0000 7f0b4929a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b44104f70 0x7f0b441a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.169+0000 7f0b4929a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b441a0f10 con 0x7f0b441033c0 2024-03-11T19:40:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.169+0000 7f0b4929a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b441a1080 con 0x7f0b441045b0 2024-03-11T19:40:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.169+0000 7f0b4929a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b441a1240 con 0x7f0b44104f70 2024-03-11T19:40:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.170+0000 7f0b42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b441033c0 0x7f0b44199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.170+0000 7f0b437fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b44104f70 0x7f0b441a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.170+0000 7f0b437fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b44104f70 0x7f0b441a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40116/0 (socket says 172.21.15.83:40116) 2024-03-11T19:40:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.170+0000 7f0b437fe640 1 -- 172.21.15.83:0/528204239 learned_addr learned my addr 172.21.15.83:0/528204239 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.170+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b441045b0 0x7f0b44199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.170+0000 7f0b437fe640 1 -- 172.21.15.83:0/528204239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b441045b0 msgr2=0x7f0b44199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.170+0000 7f0b437fe640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b441045b0 0x7f0b44199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.170+0000 7f0b437fe640 1 -- 172.21.15.83:0/528204239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b441033c0 msgr2=0x7f0b44199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.170+0000 7f0b437fe640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b441033c0 0x7f0b44199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.170+0000 7f0b437fe640 1 -- 172.21.15.83:0/528204239 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34009560 con 0x7f0b44104f70 2024-03-11T19:40:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.171+0000 7f0b437fe640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b44104f70 0x7f0b441a09d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c033420 tx=0x7f0b3c033cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.171+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b441045b0 0x7f0b44199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:40:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.171+0000 7f0b40ff9640 1 -- 172.21.15.83:0/528204239 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c003b80 con 0x7f0b44104f70 2024-03-11T19:40:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.171+0000 7f0b40ff9640 1 -- 172.21.15.83:0/528204239 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b3c003ce0 con 0x7f0b44104f70 2024-03-11T19:40:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.171+0000 7f0b40ff9640 1 -- 172.21.15.83:0/528204239 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c034960 con 0x7f0b44104f70 2024-03-11T19:40:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.171+0000 7f0b4929a640 1 -- 172.21.15.83:0/528204239 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c009840 con 0x7f0b44104f70 2024-03-11T19:40:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.171+0000 7f0b4929a640 1 -- 172.21.15.83:0/528204239 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b441a1790 con 0x7f0b44104f70 2024-03-11T19:40:08.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.174+0000 7f0b40ff9640 1 -- 172.21.15.83:0/528204239 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0b3c034ac0 con 0x7f0b44104f70 2024-03-11T19:40:08.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.175+0000 7f0b4be6d740 1 Processor -- start 2024-03-11T19:40:08.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.175+0000 7f0b4be6d740 1 -- start start 2024-03-11T19:40:08.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.175+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589d9a80 0x55e8589d9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.175+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589da470 0x55e8589def70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.175+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589df5e0 0x55e8589e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.175+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85898d960 con 0x55e8589da470 2024-03-11T19:40:08.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.175+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e858941310 con 0x55e8589d9a80 2024-03-11T19:40:08.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.175+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85893e840 con 0x55e8589df5e0 2024-03-11T19:40:08.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.176+0000 7f0b3bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589df5e0 0x55e8589e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.176+0000 7f0b40ff9640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b300a1b30 0x7f0b300a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.176+0000 7f0b42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589da470 0x55e8589def70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.176+0000 7f0b42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589da470 0x55e8589def70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37570/0 (socket says 172.21.15.83:37570) 2024-03-11T19:40:08.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.176+0000 7f0b42ffd640 1 -- 172.21.15.83:0/732981209 learned_addr learned my addr 172.21.15.83:0/732981209 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.176+0000 7f0b437fe640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b300a1b30 0x7f0b300a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.176+0000 7f0b42ffd640 1 -- 172.21.15.83:0/732981209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589df5e0 msgr2=0x55e8589e1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.176+0000 7f0b42ffd640 1 --2- 172.21.15.83:0/732981209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589df5e0 0x55e8589e1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.176+0000 7f0b42ffd640 1 -- 172.21.15.83:0/732981209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589d9a80 msgr2=0x55e8589d9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.176+0000 7f0b42ffd640 1 --2- 172.21.15.83:0/732981209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589d9a80 0x55e8589d9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.176+0000 7f0b42ffd640 1 -- 172.21.15.83:0/732981209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8588fc060 con 0x55e8589da470 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.177+0000 7f0b40ff9640 1 -- 172.21.15.83:0/528204239 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0b3c0efbf0 con 0x7f0b44104f70 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.177+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/732981209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589df5e0 0x55e8589e1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.177+0000 7f0b42ffd640 1 --2- 172.21.15.83:0/732981209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589da470 0x55e8589def70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009e10 tx=0x7f0b34011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e0af6f24bae443 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.177+0000 7f0b437fe640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b300a1b30 0x7f0b300a3ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0bc870 tx=0x7f0b3c099040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.177+0000 7f0b397fa640 1 -- 172.21.15.83:0/732981209 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34005220 con 0x55e8589da470 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.177+0000 7f0b397fa640 1 -- 172.21.15.83:0/732981209 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b34004c80 con 0x55e8589da470 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.177+0000 7f0b397fa640 1 -- 172.21.15.83:0/732981209 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34005220 con 0x55e8589da470 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.178+0000 7f0b4be6d740 1 -- 172.21.15.83:0/732981209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589da470 msgr2=0x55e8589def70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.178+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/732981209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589da470 0x55e8589def70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009e10 tx=0x7f0b34011810 comp rx=0 tx=0).stop 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.178+0000 7f0b4be6d740 1 -- 172.21.15.83:0/732981209 shutdown_connections 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.178+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/732981209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589df5e0 0x55e8589e1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.179+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/732981209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589da470 0x55e8589def70 secure :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009e10 tx=0x7f0b34011810 comp rx=0 tx=0).stop 2024-03-11T19:40:08.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.179+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/732981209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589d9a80 0x55e8589d9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.179+0000 7f0b4be6d740 1 -- 172.21.15.83:0/732981209 >> 172.21.15.83:0/732981209 conn(0x55e8589cc7e0 msgr2=0x55e8589ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.179+0000 7f0b4be6d740 1 -- 172.21.15.83:0/732981209 shutdown_connections 2024-03-11T19:40:08.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.179+0000 7f0b4be6d740 1 -- 172.21.15.83:0/732981209 wait complete. 2024-03-11T19:40:08.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.180+0000 7f0b4be6d740 1 -- start start 2024-03-11T19:40:08.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.180+0000 7f0b4be6d740 1 Processor -- start 2024-03-11T19:40:08.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.181+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589d9a80 0x55e8589e3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.181+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589df5e0 0x55e8589e4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.181+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589e92c0 0x55e8589eb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.181+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85893a020 con 0x55e8589d9a80 2024-03-11T19:40:08.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.181+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e85860d430 con 0x55e8589e92c0 2024-03-11T19:40:08.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.181+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8585c9e00 con 0x55e8589df5e0 2024-03-11T19:40:08.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.181+0000 7f0b3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589d9a80 0x55e8589e3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.181+0000 7f0b3bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589d9a80 0x55e8589e3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37576/0 (socket says 172.21.15.83:37576) 2024-03-11T19:40:08.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.181+0000 7f0b3bfff640 1 -- 172.21.15.83:0/596337849 learned_addr learned my addr 172.21.15.83:0/596337849 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.182+0000 7f0b4be6d740 10 client.0 _ll_register_callbacks cb 0x55e8588e1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:08.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.182+0000 7f0b3bfff640 1 -- 172.21.15.83:0/596337849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589df5e0 msgr2=0x55e8589e4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.182+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/596337849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589df5e0 0x55e8589e4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.182+0000 7f0b3bfff640 1 -- 172.21.15.83:0/596337849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589e92c0 msgr2=0x55e8589eb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.182+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/596337849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589e92c0 0x55e8589eb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.182+0000 7f0b3bfff640 1 -- 172.21.15.83:0/596337849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b34009a40 con 0x55e8589d9a80 2024-03-11T19:40:08.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.183+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/596337849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589d9a80 0x55e8589e3d50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00bf50 tx=0x7f0b2c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863077b197464cf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.183+0000 7f0b2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:40:08.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.183+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/596337849 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c011850 con 0x55e8589d9a80 2024-03-11T19:40:08.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.183+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/596337849 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0b2c0119b0 con 0x55e8589d9a80 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.183+0000 7f0b4be6d740 1 -- 172.21.15.83:0/596337849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8586397e0 con 0x55e8589d9a80 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.184+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/596337849 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b2c011b70 con 0x55e8589d9a80 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.184+0000 7f0b4be6d740 10 client.15501 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.184+0000 7f0b4be6d740 1 -- 172.21.15.83:0/596337849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e8589386a0 con 0x55e8589d9a80 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.184+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/596337849 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e8586397e0 con 0x55e8589d9a80 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.184+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/596337849 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0b2c024420 con 0x55e8589d9a80 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.184+0000 7f0b4be6d740 10 client.15501 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.184+0000 7f0b4be6d740 10 client.15501 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.184+0000 7f0b4be6d740 20 client.15501 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.184+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/596337849 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e8589f0290 0x55e8589f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.184+0000 7f0b4be6d740 4 client.15501 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:08.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.184+0000 7f0b4be6d740 1 -- 172.21.15.83:0/596337849 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e858676720 con 0x55e8589f0290 2024-03-11T19:40:08.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.186+0000 7f0b42ffd640 1 --2- 172.21.15.83:0/596337849 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e8589f0290 0x55e8589f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.187+0000 7f0b42ffd640 1 --2- 172.21.15.83:0/596337849 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e8589f0290 0x55e8589f26d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a668fdd85441a052 server_cookie=8586faadab4d9329 in_seq=0 out_seq=0 2024-03-11T19:40:08.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.187+0000 7f0b2a7fc640 10 client.15501 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:40:08.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.188+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/596337849 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e858676720 con 0x55e8589f0290 2024-03-11T19:40:08.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.188+0000 7f0b2a7fc640 10 client.15501 handle_command_reply: tid=0 2024-03-11T19:40:08.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.188+0000 7f0b4be6d740 1 client.15501 shutdown 2024-03-11T19:40:08.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.188+0000 7f0b4be6d740 1 -- 172.21.15.83:0/596337849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589d9a80 msgr2=0x55e8589e3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.188+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/596337849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589d9a80 0x55e8589e3d50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00bf50 tx=0x7f0b2c00f7f0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.188+0000 7f0b4be6d740 1 -- 172.21.15.83:0/596337849 shutdown_connections 2024-03-11T19:40:08.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.189+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/596337849 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e8589f0290 0x55e8589f26d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.189+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/596337849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589e92c0 0x55e8589eb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.189+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/596337849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589df5e0 0x55e8589e4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.189+0000 7f0b2a7fc640 0 client.15501 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:40:08.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.189+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/596337849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589d9a80 0x55e8589e3d50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.189+0000 7f0b4be6d740 1 -- 172.21.15.83:0/596337849 >> 172.21.15.83:0/596337849 conn(0x55e8589cc7e0 msgr2=0x55e8589cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.189+0000 7f0b4be6d740 1 -- 172.21.15.83:0/596337849 shutdown_connections 2024-03-11T19:40:08.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.189+0000 7f0b4be6d740 1 -- 172.21.15.83:0/596337849 wait complete. 2024-03-11T19:40:08.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.189+0000 7f0b4be6d740 20 client.15501 trim_cache size 0 max 16384 2024-03-11T19:40:08.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.211+0000 7f0b4be6d740 1 Processor -- start 2024-03-11T19:40:08.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.211+0000 7f0b4be6d740 1 -- start start 2024-03-11T19:40:08.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.211+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a005b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589df5e0 0x55e858a00af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589e92c0 0x55e8589e3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8588bf130 con 0x55e8589e92c0 2024-03-11T19:40:08.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8589c8080 con 0x55e8589df5e0 2024-03-11T19:40:08.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8589bea40 con 0x55e8589d9a80 2024-03-11T19:40:08.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b3bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b3bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a005b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40140/0 (socket says 172.21.15.83:40140) 2024-03-11T19:40:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b3bfff640 1 -- 172.21.15.83:0/1520136860 learned_addr learned my addr 172.21.15.83:0/1520136860 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/1520136860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589e92c0 0x55e8589e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b42ffd640 1 --2- 172.21.15.83:0/1520136860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589df5e0 0x55e858a00af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b3bfff640 1 -- 172.21.15.83:0/1520136860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589df5e0 msgr2=0x55e858a00af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/1520136860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589df5e0 0x55e858a00af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b3bfff640 1 -- 172.21.15.83:0/1520136860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589e92c0 msgr2=0x55e8589e3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/1520136860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589e92c0 0x55e8589e3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.212+0000 7f0b3bfff640 1 -- 172.21.15.83:0/1520136860 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8588d4c20 con 0x55e8589d9a80 2024-03-11T19:40:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.213+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/1520136860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a005b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00bc30 tx=0x7f0b2c005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d3799f0a5d994c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.213+0000 7f0b38ff9640 1 -- 172.21.15.83:0/1520136860 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0249a0 con 0x55e8589d9a80 2024-03-11T19:40:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.213+0000 7f0b38ff9640 1 -- 172.21.15.83:0/1520136860 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b2c024b00 con 0x55e8589d9a80 2024-03-11T19:40:08.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.213+0000 7f0b38ff9640 1 -- 172.21.15.83:0/1520136860 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c004c20 con 0x55e8589d9a80 2024-03-11T19:40:08.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.214+0000 7f0b4be6d740 1 -- 172.21.15.83:0/1520136860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 msgr2=0x55e858a005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.214+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/1520136860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a005b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00bc30 tx=0x7f0b2c005c70 comp rx=0 tx=0).stop 2024-03-11T19:40:08.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.214+0000 7f0b4be6d740 1 -- 172.21.15.83:0/1520136860 shutdown_connections 2024-03-11T19:40:08.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.214+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/1520136860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589e92c0 0x55e8589e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.214+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/1520136860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589df5e0 0x55e858a00af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.214+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/1520136860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a005b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.215+0000 7f0b4be6d740 1 -- 172.21.15.83:0/1520136860 >> 172.21.15.83:0/1520136860 conn(0x55e8589cc7e0 msgr2=0x55e8589ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.215+0000 7f0b4be6d740 1 -- 172.21.15.83:0/1520136860 shutdown_connections 2024-03-11T19:40:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.215+0000 7f0b4be6d740 1 -- 172.21.15.83:0/1520136860 wait complete. 2024-03-11T19:40:08.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.216+0000 7f0b4be6d740 1 -- start start 2024-03-11T19:40:08.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.216+0000 7f0b4be6d740 1 Processor -- start 2024-03-11T19:40:08.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a00940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589df5e0 0x55e858a43260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b4be6d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589e92c0 0x55e858a457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e858a45df0 con 0x55e8589e92c0 2024-03-11T19:40:08.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e858a45f60 con 0x55e8589df5e0 2024-03-11T19:40:08.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b4be6d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e858a46120 con 0x55e8589d9a80 2024-03-11T19:40:08.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b4be6d740 10 client.0 _ll_register_callbacks cb 0x7f0b34009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b3bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a00940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b3bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a00940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40146/0 (socket says 172.21.15.83:40146) 2024-03-11T19:40:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b3bfff640 1 -- 172.21.15.83:0/2893120433 learned_addr learned my addr 172.21.15.83:0/2893120433 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b42ffd640 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589df5e0 0x55e858a43260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b3bfff640 1 -- 172.21.15.83:0/2893120433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589df5e0 msgr2=0x55e858a43260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.217+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589df5e0 0x55e858a43260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.218+0000 7f0b3bfff640 1 -- 172.21.15.83:0/2893120433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589e92c0 msgr2=0x55e858a457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.218+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589e92c0 0x55e858a457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.218+0000 7f0b3bfff640 1 -- 172.21.15.83:0/2893120433 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8589386a0 con 0x55e8589d9a80 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.218+0000 7f0b3bfff640 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a00940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00bca0 tx=0x7f0b2c00b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc9d25c4afc42cbb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.218+0000 7f0b2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.218+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/2893120433 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0249a0 con 0x55e8589d9a80 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.218+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/2893120433 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0b2c02c070 con 0x55e8589d9a80 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.218+0000 7f0b4be6d740 1 -- 172.21.15.83:0/2893120433 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b34004c80 con 0x55e8589d9a80 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.218+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/2893120433 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b2c00bee0 con 0x55e8589d9a80 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.218+0000 7f0b4be6d740 10 client.25114 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.218+0000 7f0b4be6d740 1 -- 172.21.15.83:0/2893120433 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e8588fa8c0 con 0x55e8589d9a80 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.219+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/2893120433 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b34004c80 con 0x55e8589d9a80 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.219+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/2893120433 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0b2c032040 con 0x55e8589d9a80 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.219+0000 7f0b4be6d740 10 client.25114 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.219+0000 7f0b4be6d740 10 client.25114 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:40:08.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.219+0000 7f0b4be6d740 20 client.25114 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.219+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e858a47730 0x55e858a49b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.219+0000 7f0b4be6d740 4 client.25114 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:40:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.219+0000 7f0b4be6d740 1 -- 172.21.15.83:0/2893120433 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e8586c5890 con 0x55e858a47730 2024-03-11T19:40:08.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.219+0000 7f0b42ffd640 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e858a47730 0x55e858a49b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.220+0000 7f0b42ffd640 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e858a47730 0x55e858a49b50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3bdcebaada756122 server_cookie=52c467286d90e6b4 in_seq=0 out_seq=0 2024-03-11T19:40:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.220+0000 7f0b2a7fc640 10 client.25114 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:40:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.220+0000 7f0b2a7fc640 1 -- 172.21.15.83:0/2893120433 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e8586c5890 con 0x55e858a47730 2024-03-11T19:40:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.220+0000 7f0b2a7fc640 10 client.25114 handle_command_reply: tid=0 2024-03-11T19:40:08.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.220+0000 7f0b4be6d740 1 client.25114 shutdown 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.221+0000 7f0b4be6d740 1 -- 172.21.15.83:0/2893120433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 msgr2=0x55e858a00940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.221+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a00940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00bca0 tx=0x7f0b2c00b940 comp rx=0 tx=0).stop 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.221+0000 7f0b4be6d740 1 -- 172.21.15.83:0/2893120433 shutdown_connections 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.221+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e858a47730 0x55e858a49b50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.221+0000 7f0b2a7fc640 0 client.25114 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.221+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8589e92c0 0x55e858a457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.221+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8589df5e0 0x55e858a43260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.221+0000 7f0b4be6d740 1 --2- 172.21.15.83:0/2893120433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8589d9a80 0x55e858a00940 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.221+0000 7f0b4be6d740 1 -- 172.21.15.83:0/2893120433 >> 172.21.15.83:0/2893120433 conn(0x55e8589cc7e0 msgr2=0x55e8589e3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.221+0000 7f0b4be6d740 1 -- 172.21.15.83:0/2893120433 shutdown_connections 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.221+0000 7f0b4be6d740 1 -- 172.21.15.83:0/2893120433 wait complete. 2024-03-11T19:40:08.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.222+0000 7f0b4be6d740 20 client.25114 trim_cache size 0 max 16384 2024-03-11T19:40:08.228 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:40:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.222+0000 7f0b4929a640 1 -- 172.21.15.83:0/528204239 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b300a1b30 msgr2=0x7f0b300a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.222+0000 7f0b4929a640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b300a1b30 0x7f0b300a3ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0bc870 tx=0x7f0b3c099040 comp rx=0 tx=0).stop 2024-03-11T19:40:08.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.222+0000 7f0b4929a640 1 -- 172.21.15.83:0/528204239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b44104f70 msgr2=0x7f0b441a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.222+0000 7f0b4929a640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b44104f70 0x7f0b441a09d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c033420 tx=0x7f0b3c033cb0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.223+0000 7f0b4929a640 1 -- 172.21.15.83:0/528204239 shutdown_connections 2024-03-11T19:40:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.223+0000 7f0b4929a640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b300a1b30 0x7f0b300a3ff0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.223+0000 7f0b4929a640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b44104f70 0x7f0b441a09d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.223+0000 7f0b4929a640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b441045b0 0x7f0b44199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.223+0000 7f0b4929a640 1 --2- 172.21.15.83:0/528204239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b441033c0 0x7f0b44199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.223+0000 7f0b4929a640 1 -- 172.21.15.83:0/528204239 >> 172.21.15.83:0/528204239 conn(0x7f0b440fe930 msgr2=0x7f0b44107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.223+0000 7f0b4929a640 1 -- 172.21.15.83:0/528204239 shutdown_connections 2024-03-11T19:40:08.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.223+0000 7f0b4929a640 1 -- 172.21.15.83:0/528204239 wait complete. 2024-03-11T19:40:08.244 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:40:08.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:40:08.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.407+0000 7f11b913a640 1 -- 172.21.15.83:0/3669893294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11b4106e60 msgr2=0x7f11b41092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.407+0000 7f11b913a640 1 --2- 172.21.15.83:0/3669893294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11b4106e60 0x7f11b41092b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009ca0 tx=0x7f11ac02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.407+0000 7f11b913a640 1 -- 172.21.15.83:0/3669893294 shutdown_connections 2024-03-11T19:40:08.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.407+0000 7f11b913a640 1 --2- 172.21.15.83:0/3669893294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11b4106e60 0x7f11b41092b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.407+0000 7f11b913a640 1 --2- 172.21.15.83:0/3669893294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11b4104530 0x7f11b4106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.407+0000 7f11b913a640 1 --2- 172.21.15.83:0/3669893294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b4101c00 0x7f11b4103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.408+0000 7f11b913a640 1 -- 172.21.15.83:0/3669893294 >> 172.21.15.83:0/3669893294 conn(0x7f11b40fb610 msgr2=0x7f11b40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.408+0000 7f11b913a640 1 -- 172.21.15.83:0/3669893294 shutdown_connections 2024-03-11T19:40:08.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.408+0000 7f11b913a640 1 -- 172.21.15.83:0/3669893294 wait complete. 2024-03-11T19:40:08.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.409+0000 7f11b913a640 1 Processor -- start 2024-03-11T19:40:08.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.410+0000 7f11b913a640 1 -- start start 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.410+0000 7f11b913a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11b4101c00 0x7f11b4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.410+0000 7f11b913a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b4104530 0x7f11b4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.410+0000 7f11b913a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11b4106e60 0x7f11b419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.410+0000 7f11b913a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b419ed40 con 0x7f11b4106e60 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.410+0000 7f11b913a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b419eeb0 con 0x7f11b4101c00 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.410+0000 7f11b913a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b419f070 con 0x7f11b4104530 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11aa59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b4104530 0x7f11b4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11b359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11b4106e60 0x7f11b419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11aa59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b4104530 0x7f11b4197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40174/0 (socket says 172.21.15.83:40174) 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11b359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11b4106e60 0x7f11b419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37622/0 (socket says 172.21.15.83:37622) 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11aa59c640 1 -- 172.21.15.83:0/3269765602 learned_addr learned my addr 172.21.15.83:0/3269765602 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11b4101c00 0x7f11b4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11aa59c640 1 -- 172.21.15.83:0/3269765602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11b4101c00 msgr2=0x7f11b4197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11aa59c640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11b4101c00 0x7f11b4197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11aa59c640 1 -- 172.21.15.83:0/3269765602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11b4106e60 msgr2=0x7f11b419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11aa59c640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11b4106e60 0x7f11b419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11aa59c640 1 -- 172.21.15.83:0/3269765602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac009900 con 0x7f11b4104530 2024-03-11T19:40:08.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11aa59c640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b4104530 0x7f11b4197790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f119c005fd0 tx=0x7f119c0071c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11b0d99640 1 -- 172.21.15.83:0/3269765602 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c014070 con 0x7f11b4104530 2024-03-11T19:40:08.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11b0d99640 1 -- 172.21.15.83:0/3269765602 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f119c00f020 con 0x7f11b4104530 2024-03-11T19:40:08.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11b0d99640 1 -- 172.21.15.83:0/3269765602 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c00e660 con 0x7f11b4104530 2024-03-11T19:40:08.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.411+0000 7f11b913a640 1 -- 172.21.15.83:0/3269765602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b40ffa50 con 0x7f11b4104530 2024-03-11T19:40:08.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.412+0000 7f11b913a640 1 -- 172.21.15.83:0/3269765602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b40fffb0 con 0x7f11b4104530 2024-03-11T19:40:08.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.415+0000 7f11bbde2740 1 Processor -- start 2024-03-11T19:40:08.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.415+0000 7f11bbde2740 1 -- start start 2024-03-11T19:40:08.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.416+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565405298ac0 0x565405298ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.416+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654052994b0 0x56540529dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.416+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56540529e620 0x5654052a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.416+0000 7f11bbde2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540524c960 con 0x5654052994b0 2024-03-11T19:40:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.416+0000 7f11bbde2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x565405200310 con 0x565405298ac0 2024-03-11T19:40:08.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.416+0000 7f11bbde2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654051fd840 con 0x56540529e620 2024-03-11T19:40:08.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.416+0000 7f11b359e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565405298ac0 0x565405298ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.416+0000 7f11b2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654052994b0 0x56540529dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.416+0000 7f11b359e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565405298ac0 0x565405298ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:38238/0 (socket says 172.21.15.83:38238) 2024-03-11T19:40:08.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.416+0000 7f11b359e640 1 -- 172.21.15.83:0/584683707 learned_addr learned my addr 172.21.15.83:0/584683707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.416+0000 7f11b0d99640 1 -- 172.21.15.83:0/3269765602 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f119c018070 con 0x7f11b4104530 2024-03-11T19:40:08.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.417+0000 7f11b2d9d640 1 -- 172.21.15.83:0/584683707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56540529e620 msgr2=0x5654052a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.417+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/584683707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56540529e620 0x5654052a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.417+0000 7f11b2d9d640 1 -- 172.21.15.83:0/584683707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565405298ac0 msgr2=0x565405298ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.417+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/584683707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565405298ac0 0x565405298ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.417+0000 7f11b2d9d640 1 -- 172.21.15.83:0/584683707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654051bb060 con 0x5654052994b0 2024-03-11T19:40:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.417+0000 7f11b359e640 1 --2- 172.21.15.83:0/584683707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565405298ac0 0x565405298ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.417+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/584683707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654052994b0 0x56540529dfb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009c00 tx=0x7f11a4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e695364b02031a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.417+0000 7f11a959a640 1 -- 172.21.15.83:0/584683707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4004230 con 0x5654052994b0 2024-03-11T19:40:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.417+0000 7f11a959a640 1 -- 172.21.15.83:0/584683707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11a4009e70 con 0x5654052994b0 2024-03-11T19:40:08.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.417+0000 7f11a959a640 1 -- 172.21.15.83:0/584683707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4004c40 con 0x5654052994b0 2024-03-11T19:40:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.418+0000 7f11b0d99640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11a00a1b80 0x7f11a00a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.418+0000 7f11b0d99640 1 -- 172.21.15.83:0/3269765602 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f119c0cc030 con 0x7f11b4104530 2024-03-11T19:40:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.418+0000 7f11b359e640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11a00a1b80 0x7f11a00a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.418+0000 7f11bbde2740 1 -- 172.21.15.83:0/584683707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654052994b0 msgr2=0x56540529dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.418+0000 7f11bbde2740 1 --2- 172.21.15.83:0/584683707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654052994b0 0x56540529dfb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009c00 tx=0x7f11a4011810 comp rx=0 tx=0).stop 2024-03-11T19:40:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.419+0000 7f11bbde2740 1 -- 172.21.15.83:0/584683707 shutdown_connections 2024-03-11T19:40:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.419+0000 7f11bbde2740 1 --2- 172.21.15.83:0/584683707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56540529e620 0x5654052a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.419+0000 7f11bbde2740 1 --2- 172.21.15.83:0/584683707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654052994b0 0x56540529dfb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.419+0000 7f11bbde2740 1 --2- 172.21.15.83:0/584683707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565405298ac0 0x565405298ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.419+0000 7f11bbde2740 1 -- 172.21.15.83:0/584683707 >> 172.21.15.83:0/584683707 conn(0x565405294030 msgr2=0x565405294430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.419+0000 7f11b359e640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11a00a1b80 0x7f11a00a4040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x565405299d10 tx=0x7f11ac0323a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.420+0000 7f11bbde2740 1 -- 172.21.15.83:0/584683707 shutdown_connections 2024-03-11T19:40:08.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.420+0000 7f11bbde2740 1 -- 172.21.15.83:0/584683707 wait complete. 2024-03-11T19:40:08.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.420+0000 7f11bbde2740 1 -- start start 2024-03-11T19:40:08.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.420+0000 7f11bbde2740 1 Processor -- start 2024-03-11T19:40:08.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.421+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 0x5654052a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.421+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654052994b0 0x5654052a3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56540529e620 0x5654052aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11aa59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654052994b0 0x5654052a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11aa59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654052994b0 0x5654052a3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:38240/0 (socket says 172.21.15.83:38240) 2024-03-11T19:40:08.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11b2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 0x5654052a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11b359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56540529e620 0x5654052aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11b2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 0x5654052a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37638/0 (socket says 172.21.15.83:37638) 2024-03-11T19:40:08.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11b359e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56540529e620 0x5654052aa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40192/0 (socket says 172.21.15.83:40192) 2024-03-11T19:40:08.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11aa59c640 1 -- 172.21.15.83:0/4267162150 learned_addr learned my addr 172.21.15.83:0/4267162150 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11bbde2740 1 -- 172.21.15.83:0/4267162150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654051f9020 con 0x565405298ac0 2024-03-11T19:40:08.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11bbde2740 1 -- 172.21.15.83:0/4267162150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x565404ecc430 con 0x5654052994b0 2024-03-11T19:40:08.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11bbde2740 1 -- 172.21.15.83:0/4267162150 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565404e88e00 con 0x56540529e620 2024-03-11T19:40:08.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.422+0000 7f11bbde2740 10 client.0 _ll_register_callbacks cb 0x5654051a0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:08.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.423+0000 7f11b2d9d640 1 -- 172.21.15.83:0/4267162150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56540529e620 msgr2=0x5654052aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.423+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56540529e620 0x5654052aa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.423+0000 7f11b2d9d640 1 -- 172.21.15.83:0/4267162150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654052994b0 msgr2=0x5654052a3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.423+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654052994b0 0x5654052a3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.423+0000 7f11b2d9d640 1 -- 172.21.15.83:0/4267162150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a4009830 con 0x565405298ac0 2024-03-11T19:40:08.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.423+0000 7f11b359e640 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56540529e620 0x5654052aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:08.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.423+0000 7f11aa59c640 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654052994b0 0x5654052a3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.423+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 0x5654052a2b20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x565405294fa0 tx=0x7f11a4019d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f70547973880 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.423+0000 7f119a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.424+0000 7f119a7fc640 1 -- 172.21.15.83:0/4267162150 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4024670 con 0x565405298ac0 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.424+0000 7f119a7fc640 1 -- 172.21.15.83:0/4267162150 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f11a40247d0 con 0x565405298ac0 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.424+0000 7f11bbde2740 1 -- 172.21.15.83:0/4267162150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565404ef87e0 con 0x565405298ac0 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.424+0000 7f119a7fc640 1 -- 172.21.15.83:0/4267162150 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11a4024990 con 0x565405298ac0 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.425+0000 7f11bbde2740 10 client.15516 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.425+0000 7f11bbde2740 1 -- 172.21.15.83:0/4267162150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5654051f76a0 con 0x565405298ac0 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.425+0000 7f119a7fc640 1 -- 172.21.15.83:0/4267162150 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11a40233d0 con 0x565405298ac0 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.425+0000 7f119a7fc640 1 -- 172.21.15.83:0/4267162150 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f11a4018390 con 0x565405298ac0 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.425+0000 7f11bbde2740 10 client.15516 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:08.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.425+0000 7f11bbde2740 10 client.15516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:40:08.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.426+0000 7f11bbde2740 20 client.15516 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:08.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.426+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5654052aea40 0x5654052b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.426+0000 7f11bbde2740 4 client.15516 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:08.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.426+0000 7f11bbde2740 1 -- 172.21.15.83:0/4267162150 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565404ef87e0 con 0x5654052aea40 2024-03-11T19:40:08.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.427+0000 7f11aa59c640 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5654052aea40 0x5654052b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.427+0000 7f119a7fc640 10 client.15516 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:40:08.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.427+0000 7f11aa59c640 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5654052aea40 0x5654052b0e90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d15df4bd998bbe server_cookie=393b6a54f296f993 in_seq=0 out_seq=0 2024-03-11T19:40:08.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.428+0000 7f119a7fc640 1 -- 172.21.15.83:0/4267162150 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565404ef87e0 con 0x5654052aea40 2024-03-11T19:40:08.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.428+0000 7f119a7fc640 10 client.15516 handle_command_reply: tid=0 2024-03-11T19:40:08.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.428+0000 7f11bbde2740 1 client.15516 shutdown 2024-03-11T19:40:08.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.429+0000 7f11bbde2740 1 -- 172.21.15.83:0/4267162150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 msgr2=0x5654052a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.429+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 0x5654052a2b20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x565405294fa0 tx=0x7f11a4019d90 comp rx=0 tx=0).stop 2024-03-11T19:40:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.429+0000 7f11bbde2740 1 -- 172.21.15.83:0/4267162150 shutdown_connections 2024-03-11T19:40:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.430+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5654052aea40 0x5654052b0e90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.430+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56540529e620 0x5654052aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.430+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654052994b0 0x5654052a3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.430+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4267162150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 0x5654052a2b20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.430+0000 7f119a7fc640 0 client.15516 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:40:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.430+0000 7f11bbde2740 1 -- 172.21.15.83:0/4267162150 >> 172.21.15.83:0/4267162150 conn(0x565405294030 msgr2=0x56540528ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.430+0000 7f11bbde2740 1 -- 172.21.15.83:0/4267162150 shutdown_connections 2024-03-11T19:40:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.430+0000 7f11bbde2740 1 -- 172.21.15.83:0/4267162150 wait complete. 2024-03-11T19:40:08.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.430+0000 7f11bbde2740 20 client.15516 trim_cache size 0 max 16384 2024-03-11T19:40:08.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.451+0000 7f11bbde2740 1 Processor -- start 2024-03-11T19:40:08.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.452+0000 7f11bbde2740 1 -- start start 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.452+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565405298ac0 0x56540528a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.452+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654052994b0 0x5654052aa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.452+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56540529e620 0x56540528a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11b2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565405298ac0 0x56540528a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11bbde2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56540517e130 con 0x5654052994b0 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11bbde2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x565405287610 con 0x56540529e620 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11bbde2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565404ef87e0 con 0x565405298ac0 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11aa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654052994b0 0x5654052aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11b2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565405298ac0 0x56540528a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40200/0 (socket says 172.21.15.83:40200) 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11b2d9d640 1 -- 172.21.15.83:0/1927457128 learned_addr learned my addr 172.21.15.83:0/1927457128 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11b2d9d640 1 -- 172.21.15.83:0/1927457128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56540529e620 msgr2=0x56540528a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/1927457128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56540529e620 0x56540528a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11b2d9d640 1 -- 172.21.15.83:0/1927457128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654052994b0 msgr2=0x5654052aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/1927457128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654052994b0 0x5654052aa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11b2d9d640 1 -- 172.21.15.83:0/1927457128 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565405193c20 con 0x565405298ac0 2024-03-11T19:40:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/1927457128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565405298ac0 0x56540528a2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f11a4011d10 tx=0x7f11a4009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad72c6b7dee0584b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.453+0000 7f11a8d99640 1 -- 172.21.15.83:0/1927457128 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4009a50 con 0x565405298ac0 2024-03-11T19:40:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.454+0000 7f11a8d99640 1 -- 172.21.15.83:0/1927457128 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11a4018e40 con 0x565405298ac0 2024-03-11T19:40:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.454+0000 7f11a8d99640 1 -- 172.21.15.83:0/1927457128 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4024cd0 con 0x565405298ac0 2024-03-11T19:40:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.455+0000 7f11bbde2740 1 -- 172.21.15.83:0/1927457128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565405298ac0 msgr2=0x56540528a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.455+0000 7f11bbde2740 1 --2- 172.21.15.83:0/1927457128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565405298ac0 0x56540528a2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f11a4011d10 tx=0x7f11a4009620 comp rx=0 tx=0).stop 2024-03-11T19:40:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.455+0000 7f11bbde2740 1 -- 172.21.15.83:0/1927457128 shutdown_connections 2024-03-11T19:40:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.455+0000 7f11bbde2740 1 --2- 172.21.15.83:0/1927457128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56540529e620 0x56540528a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.455+0000 7f11bbde2740 1 --2- 172.21.15.83:0/1927457128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654052994b0 0x5654052aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.455+0000 7f11bbde2740 1 --2- 172.21.15.83:0/1927457128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565405298ac0 0x56540528a2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.455+0000 7f11bbde2740 1 -- 172.21.15.83:0/1927457128 >> 172.21.15.83:0/1927457128 conn(0x565405294030 msgr2=0x56540529a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.455+0000 7f11bbde2740 1 -- 172.21.15.83:0/1927457128 shutdown_connections 2024-03-11T19:40:08.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.455+0000 7f11bbde2740 1 -- 172.21.15.83:0/1927457128 wait complete. 2024-03-11T19:40:08.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.455+0000 7f11bbde2740 1 -- start start 2024-03-11T19:40:08.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.456+0000 7f11bbde2740 1 Processor -- start 2024-03-11T19:40:08.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.456+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 0x5654052a7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.456+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654052994b0 0x5654052a8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.456+0000 7f11bbde2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56540529e620 0x5654052a9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.457+0000 7f11aa59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654052994b0 0x5654052a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.457+0000 7f11aa59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654052994b0 0x5654052a8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40212/0 (socket says 172.21.15.83:40212) 2024-03-11T19:40:08.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.457+0000 7f11aa59c640 1 -- 172.21.15.83:0/4049023659 learned_addr learned my addr 172.21.15.83:0/4049023659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.457+0000 7f11bbde2740 1 -- 172.21.15.83:0/4049023659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654052a98d0 con 0x565405298ac0 2024-03-11T19:40:08.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.457+0000 7f11bbde2740 1 -- 172.21.15.83:0/4049023659 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654052a9a40 con 0x56540529e620 2024-03-11T19:40:08.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.457+0000 7f11b359e640 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56540529e620 0x5654052a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.457+0000 7f11bbde2740 1 -- 172.21.15.83:0/4049023659 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654052a9c00 con 0x5654052994b0 2024-03-11T19:40:08.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.457+0000 7f11bbde2740 10 client.0 _ll_register_callbacks cb 0x7f11a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:08.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.458+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 0x5654052a7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.458+0000 7f11aa59c640 1 -- 172.21.15.83:0/4049023659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56540529e620 msgr2=0x5654052a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.458+0000 7f11aa59c640 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56540529e620 0x5654052a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.458+0000 7f11aa59c640 1 -- 172.21.15.83:0/4049023659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 msgr2=0x5654052a7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.458+0000 7f11aa59c640 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 0x5654052a7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.458+0000 7f11aa59c640 1 -- 172.21.15.83:0/4049023659 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654051f76a0 con 0x5654052994b0 2024-03-11T19:40:08.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.458+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 0x5654052a7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:40:08.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.458+0000 7f11b359e640 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56540529e620 0x5654052a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:08.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.458+0000 7f11aa59c640 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654052994b0 0x5654052a8380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f119c007a20 tx=0x7f119c098e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30327e1f45234858 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.458+0000 7f119a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:40:08.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.459+0000 7f119a7fc640 1 -- 172.21.15.83:0/4049023659 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c0a1750 con 0x5654052994b0 2024-03-11T19:40:08.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.459+0000 7f119a7fc640 1 -- 172.21.15.83:0/4049023659 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f119c0a12c0 con 0x5654052994b0 2024-03-11T19:40:08.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.459+0000 7f11bbde2740 1 -- 172.21.15.83:0/4049023659 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11a4009e70 con 0x5654052994b0 2024-03-11T19:40:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.460+0000 7f119a7fc640 1 -- 172.21.15.83:0/4049023659 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f119c0a1480 con 0x5654052994b0 2024-03-11T19:40:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.461+0000 7f11bbde2740 10 client.25129 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.461+0000 7f11bbde2740 1 -- 172.21.15.83:0/4049023659 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5654051b98c0 con 0x5654052994b0 2024-03-11T19:40:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.461+0000 7f119a7fc640 1 -- 172.21.15.83:0/4049023659 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f119c0b7240 con 0x5654052994b0 2024-03-11T19:40:08.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.462+0000 7f119a7fc640 1 -- 172.21.15.83:0/4049023659 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f119c07db20 con 0x5654052994b0 2024-03-11T19:40:08.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.462+0000 7f11bbde2740 10 client.25129 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:08.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.462+0000 7f11bbde2740 10 client.25129 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:40:08.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.462+0000 7f11bbde2740 20 client.25129 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:08.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.462+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5654053060d0 0x5654053084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.462+0000 7f11bbde2740 4 client.25129 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:40:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.462+0000 7f11bbde2740 1 -- 172.21.15.83:0/4049023659 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f11a4009e70 con 0x5654053060d0 2024-03-11T19:40:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.463+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5654053060d0 0x5654053084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.463+0000 7f11b2d9d640 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5654053060d0 0x5654053084f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3743beb8a42e8877 server_cookie=9d879adced744803 in_seq=0 out_seq=0 2024-03-11T19:40:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.463+0000 7f119a7fc640 10 client.25129 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:40:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.463+0000 7f119a7fc640 1 -- 172.21.15.83:0/4049023659 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f11a4009e70 con 0x5654053060d0 2024-03-11T19:40:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.463+0000 7f119a7fc640 10 client.25129 handle_command_reply: tid=0 2024-03-11T19:40:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.463+0000 7f11bbde2740 1 client.25129 shutdown 2024-03-11T19:40:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.465+0000 7f11bbde2740 1 -- 172.21.15.83:0/4049023659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654052994b0 msgr2=0x5654052a8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.465+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654052994b0 0x5654052a8380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f119c007a20 tx=0x7f119c098e40 comp rx=0 tx=0).stop 2024-03-11T19:40:08.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.465+0000 7f11bbde2740 1 -- 172.21.15.83:0/4049023659 shutdown_connections 2024-03-11T19:40:08.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.465+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5654053060d0 0x5654053084f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.465+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56540529e620 0x5654052a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.465+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5654052994b0 0x5654052a8380 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f119c007a20 tx=0x7f119c098e40 comp rx=0 tx=0).stop 2024-03-11T19:40:08.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.465+0000 7f119a7fc640 0 client.25129 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:40:08.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.465+0000 7f11bbde2740 1 --2- 172.21.15.83:0/4049023659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565405298ac0 0x5654052a7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.465+0000 7f11bbde2740 1 -- 172.21.15.83:0/4049023659 >> 172.21.15.83:0/4049023659 conn(0x565405294030 msgr2=0x56540528a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.466+0000 7f11bbde2740 1 -- 172.21.15.83:0/4049023659 shutdown_connections 2024-03-11T19:40:08.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.466+0000 7f11bbde2740 1 -- 172.21.15.83:0/4049023659 wait complete. 2024-03-11T19:40:08.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.466+0000 7f11bbde2740 20 client.25129 trim_cache size 0 max 16384 2024-03-11T19:40:08.471 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:40:08.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 -- 172.21.15.83:0/3269765602 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11a00a1b80 msgr2=0x7f11a00a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11a00a1b80 0x7f11a00a4040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x565405299d10 tx=0x7f11ac0323a0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 -- 172.21.15.83:0/3269765602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b4104530 msgr2=0x7f11b4197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b4104530 0x7f11b4197790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f119c005fd0 tx=0x7f119c0071c0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 -- 172.21.15.83:0/3269765602 shutdown_connections 2024-03-11T19:40:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11a00a1b80 0x7f11a00a4040 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11b4106e60 0x7f11b419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b4104530 0x7f11b4197790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 --2- 172.21.15.83:0/3269765602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11b4101c00 0x7f11b4197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 -- 172.21.15.83:0/3269765602 >> 172.21.15.83:0/3269765602 conn(0x7f11b40fb610 msgr2=0x7f11b4102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 -- 172.21.15.83:0/3269765602 shutdown_connections 2024-03-11T19:40:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.467+0000 7f11b913a640 1 -- 172.21.15.83:0/3269765602 wait complete. 2024-03-11T19:40:08.491 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:40:08.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:40:08.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.655+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3513490394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3810bbc0 msgr2=0x7f5d3810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.655+0000 7f5d3f81c640 1 --2- 172.21.15.83:0/3513490394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3810bbc0 0x7f5d3810dfb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800b880 tx=0x7f5d28005f90 comp rx=0 tx=0).stop 2024-03-11T19:40:08.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.656+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3513490394 shutdown_connections 2024-03-11T19:40:08.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.656+0000 7f5d3f81c640 1 --2- 172.21.15.83:0/3513490394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3810e580 0x7f5d38110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.656+0000 7f5d3f81c640 1 --2- 172.21.15.83:0/3513490394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d3810bbc0 0x7f5d3810dfb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.656+0000 7f5d3f81c640 1 --2- 172.21.15.83:0/3513490394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d38109290 0x7f5d3810b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.656+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3513490394 >> 172.21.15.83:0/3513490394 conn(0x7f5d3806e380 msgr2=0x7f5d380707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.656+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3513490394 shutdown_connections 2024-03-11T19:40:08.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.657+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3513490394 wait complete. 2024-03-11T19:40:08.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.658+0000 7f5d3f81c640 1 Processor -- start 2024-03-11T19:40:08.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.658+0000 7f5d3f81c640 1 -- start start 2024-03-11T19:40:08.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.658+0000 7f5d3f81c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38109290 0x7f5d3819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.658+0000 7f5d3f81c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d3810bbc0 0x7f5d3819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.658+0000 7f5d3f81c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3810e580 0x7f5d381a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.658+0000 7f5d3f81c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d381a51b0 con 0x7f5d3810e580 2024-03-11T19:40:08.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.658+0000 7f5d3f81c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d381a5320 con 0x7f5d3810bbc0 2024-03-11T19:40:08.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.658+0000 7f5d3f81c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d381a54e0 con 0x7f5d38109290 2024-03-11T19:40:08.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.659+0000 7f5d3ddb9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3810e580 0x7f5d381a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.659+0000 7f5d3d5b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38109290 0x7f5d3819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.659+0000 7f5d3d5b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38109290 0x7f5d3819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40230/0 (socket says 172.21.15.83:40230) 2024-03-11T19:40:08.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.659+0000 7f5d3d5b8640 1 -- 172.21.15.83:0/3486302428 learned_addr learned my addr 172.21.15.83:0/3486302428 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.659+0000 7f5d3cdb7640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d3810bbc0 0x7f5d3819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.659+0000 7f5d3d5b8640 1 -- 172.21.15.83:0/3486302428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d3810bbc0 msgr2=0x7f5d3819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.659+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d3810bbc0 0x7f5d3819dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.659+0000 7f5d3d5b8640 1 -- 172.21.15.83:0/3486302428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3810e580 msgr2=0x7f5d381a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.659+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3810e580 0x7f5d381a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.659+0000 7f5d3d5b8640 1 -- 172.21.15.83:0/3486302428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2800b4b0 con 0x7f5d38109290 2024-03-11T19:40:08.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.660+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38109290 0x7f5d3819d6c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400bd40 tx=0x7f5d34008600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.661+0000 7f5d2e7fc640 1 -- 172.21.15.83:0/3486302428 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400de80 con 0x7f5d38109290 2024-03-11T19:40:08.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.661+0000 7f5d2e7fc640 1 -- 172.21.15.83:0/3486302428 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d3400be90 con 0x7f5d38109290 2024-03-11T19:40:08.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.661+0000 7f5d2e7fc640 1 -- 172.21.15.83:0/3486302428 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34014830 con 0x7f5d38109290 2024-03-11T19:40:08.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.661+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3486302428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d381a57c0 con 0x7f5d38109290 2024-03-11T19:40:08.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.661+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3486302428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d381a5c50 con 0x7f5d38109290 2024-03-11T19:40:08.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.664+0000 7f5d2e7fc640 1 -- 172.21.15.83:0/3486302428 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5d34015b70 con 0x7f5d38109290 2024-03-11T19:40:08.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d423f5740 1 Processor -- start 2024-03-11T19:40:08.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d423f5740 1 -- start start 2024-03-11T19:40:08.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d2e7fc640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5d240a1ab0 0x7f5d240a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059270 0x558605059690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059c60 0x55860505e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55860505edd0 0x558605061260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d423f5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558605015960 con 0x55860505edd0 2024-03-11T19:40:08.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d423f5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558604fc9310 con 0x558605059c60 2024-03-11T19:40:08.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d423f5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558604fc6840 con 0x558605059270 2024-03-11T19:40:08.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d3ddb9640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5d240a1ab0 0x7f5d240a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d3d5b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059270 0x558605059690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d3ddb9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55860505edd0 0x558605061260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d3d5b8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059270 0x558605059690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40240/0 (socket says 172.21.15.83:40240) 2024-03-11T19:40:08.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d3d5b8640 1 -- 172.21.15.83:0/3683174382 learned_addr learned my addr 172.21.15.83:0/3683174382 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.665+0000 7f5d3cdb7640 1 --2- 172.21.15.83:0/3683174382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059c60 0x55860505e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.666+0000 7f5d3d5b8640 1 -- 172.21.15.83:0/3683174382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059c60 msgr2=0x55860505e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.666+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/3683174382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059c60 0x55860505e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.666+0000 7f5d3d5b8640 1 -- 172.21.15.83:0/3683174382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55860505edd0 msgr2=0x558605061260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.666+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/3683174382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55860505edd0 0x558605061260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.666+0000 7f5d2e7fc640 1 -- 172.21.15.83:0/3486302428 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5d340cd720 con 0x7f5d38109290 2024-03-11T19:40:08.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.666+0000 7f5d3d5b8640 1 -- 172.21.15.83:0/3683174382 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558604f84060 con 0x558605059270 2024-03-11T19:40:08.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.666+0000 7f5d3ddb9640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5d240a1ab0 0x7f5d240a3f70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5d30013ce0 tx=0x7f5d300041b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.667+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/3683174382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059270 0x558605059690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5d34097110 tx=0x7f5d3406da40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a8561748b6d7817 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.667+0000 7f5d237fe640 1 -- 172.21.15.83:0/3683174382 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3409ae40 con 0x558605059270 2024-03-11T19:40:08.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.667+0000 7f5d237fe640 1 -- 172.21.15.83:0/3683174382 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d34065f30 con 0x558605059270 2024-03-11T19:40:08.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.667+0000 7f5d237fe640 1 -- 172.21.15.83:0/3683174382 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3409bca0 con 0x558605059270 2024-03-11T19:40:08.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.668+0000 7f5d423f5740 1 -- 172.21.15.83:0/3683174382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059270 msgr2=0x558605059690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.668+0000 7f5d423f5740 1 --2- 172.21.15.83:0/3683174382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059270 0x558605059690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5d34097110 tx=0x7f5d3406da40 comp rx=0 tx=0).stop 2024-03-11T19:40:08.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.668+0000 7f5d423f5740 1 -- 172.21.15.83:0/3683174382 shutdown_connections 2024-03-11T19:40:08.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.668+0000 7f5d423f5740 1 --2- 172.21.15.83:0/3683174382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55860505edd0 0x558605061260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.668+0000 7f5d423f5740 1 --2- 172.21.15.83:0/3683174382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059c60 0x55860505e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.668+0000 7f5d423f5740 1 --2- 172.21.15.83:0/3683174382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059270 0x558605059690 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.668+0000 7f5d423f5740 1 -- 172.21.15.83:0/3683174382 >> 172.21.15.83:0/3683174382 conn(0x5586050547e0 msgr2=0x558605054be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.669+0000 7f5d423f5740 1 -- 172.21.15.83:0/3683174382 shutdown_connections 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.669+0000 7f5d423f5740 1 -- 172.21.15.83:0/3683174382 wait complete. 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.670+0000 7f5d423f5740 1 -- start start 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.670+0000 7f5d423f5740 1 Processor -- start 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.670+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059270 0x5586050633f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 0x558605063930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d3d5b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059270 0x5586050633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55860505edd0 0x55860506a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d3d5b8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059270 0x5586050633f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37676/0 (socket says 172.21.15.83:37676) 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d3cdb7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 0x558605063930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d3cdb7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 0x558605063930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40254/0 (socket says 172.21.15.83:40254) 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d423f5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558604fc2020 con 0x558605059270 2024-03-11T19:40:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d3d5b8640 1 -- 172.21.15.83:0/4039712777 learned_addr learned my addr 172.21.15.83:0/4039712777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d3ddb9640 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55860505edd0 0x55860506a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d423f5740 1 -- 172.21.15.83:0/4039712777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558604c95430 con 0x55860505edd0 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d423f5740 1 -- 172.21.15.83:0/4039712777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558604c51e00 con 0x558605059c60 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.671+0000 7f5d423f5740 10 client.0 _ll_register_callbacks cb 0x558604f69cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.672+0000 7f5d3ddb9640 1 -- 172.21.15.83:0/4039712777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 msgr2=0x558605063930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.672+0000 7f5d3ddb9640 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 0x558605063930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.672+0000 7f5d3ddb9640 1 -- 172.21.15.83:0/4039712777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059270 msgr2=0x5586050633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.672+0000 7f5d3ddb9640 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059270 0x5586050633f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.672+0000 7f5d3ddb9640 1 -- 172.21.15.83:0/4039712777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d381a5c50 con 0x55860505edd0 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.672+0000 7f5d3ddb9640 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55860505edd0 0x55860506a970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d30005fd0 tx=0x7f5d30015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd40cfd2df4fc2b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.672+0000 7f5d20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.672+0000 7f5d20ff9640 1 -- 172.21.15.83:0/4039712777 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30018040 con 0x55860505edd0 2024-03-11T19:40:08.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.673+0000 7f5d423f5740 1 -- 172.21.15.83:0/4039712777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558604cc17e0 con 0x55860505edd0 2024-03-11T19:40:08.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.673+0000 7f5d20ff9640 1 -- 172.21.15.83:0/4039712777 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5d30029070 con 0x55860505edd0 2024-03-11T19:40:08.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.673+0000 7f5d20ff9640 1 -- 172.21.15.83:0/4039712777 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d30013d50 con 0x55860505edd0 2024-03-11T19:40:08.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.673+0000 7f5d423f5740 10 client.25031 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:08.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.673+0000 7f5d423f5740 1 -- 172.21.15.83:0/4039712777 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558604fc06a0 con 0x55860505edd0 2024-03-11T19:40:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.674+0000 7f5d20ff9640 1 -- 172.21.15.83:0/4039712777 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d3002db50 con 0x55860505edd0 2024-03-11T19:40:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.674+0000 7f5d20ff9640 1 -- 172.21.15.83:0/4039712777 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5d30023870 con 0x55860505edd0 2024-03-11T19:40:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.675+0000 7f5d423f5740 10 client.25031 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.675+0000 7f5d423f5740 10 client.25031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.675+0000 7f5d423f5740 20 client.25031 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.675+0000 7f5d423f5740 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55860506f5a0 0x558605071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.675+0000 7f5d423f5740 4 client.25031 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.675+0000 7f5d423f5740 1 -- 172.21.15.83:0/4039712777 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558604cc17e0 con 0x55860506f5a0 2024-03-11T19:40:08.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.676+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55860506f5a0 0x558605071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.678+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55860506f5a0 0x558605071a00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b942a70628bf669a server_cookie=d5ae5d4afe27c45a in_seq=0 out_seq=0 2024-03-11T19:40:08.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.678+0000 7f5d20ff9640 10 client.25031 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:08.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.678+0000 7f5d20ff9640 1 -- 172.21.15.83:0/4039712777 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558604cc17e0 con 0x55860506f5a0 2024-03-11T19:40:08.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.678+0000 7f5d20ff9640 10 client.25031 handle_command_reply: tid=0 2024-03-11T19:40:08.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.678+0000 7f5d423f5740 1 client.25031 shutdown 2024-03-11T19:40:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.679+0000 7f5d423f5740 1 -- 172.21.15.83:0/4039712777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55860505edd0 msgr2=0x55860506a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.679+0000 7f5d423f5740 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55860505edd0 0x55860506a970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d30005fd0 tx=0x7f5d30015040 comp rx=0 tx=0).stop 2024-03-11T19:40:08.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.679+0000 7f5d423f5740 1 -- 172.21.15.83:0/4039712777 shutdown_connections 2024-03-11T19:40:08.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.679+0000 7f5d423f5740 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55860506f5a0 0x558605071a00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.679+0000 7f5d423f5740 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55860505edd0 0x55860506a970 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.679+0000 7f5d20ff9640 0 client.25031 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:08.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.679+0000 7f5d423f5740 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 0x558605063930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.679+0000 7f5d423f5740 1 --2- 172.21.15.83:0/4039712777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059270 0x5586050633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.679+0000 7f5d423f5740 1 -- 172.21.15.83:0/4039712777 >> 172.21.15.83:0/4039712777 conn(0x5586050547e0 msgr2=0x558605053b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.680+0000 7f5d423f5740 1 -- 172.21.15.83:0/4039712777 shutdown_connections 2024-03-11T19:40:08.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.680+0000 7f5d423f5740 1 -- 172.21.15.83:0/4039712777 wait complete. 2024-03-11T19:40:08.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.681+0000 7f5d423f5740 20 client.25031 trim_cache size 0 max 16384 2024-03-11T19:40:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.711+0000 7f5d423f5740 1 Processor -- start 2024-03-11T19:40:08.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.713+0000 7f5d423f5740 1 -- start start 2024-03-11T19:40:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.713+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059270 0x55860505b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.713+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059c60 0x55860505b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.713+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860505edd0 0x55860506a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.713+0000 7f5d423f5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558604f47130 con 0x558605059c60 2024-03-11T19:40:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.713+0000 7f5d423f5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558605064460 con 0x558605059270 2024-03-11T19:40:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.713+0000 7f5d423f5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558604cc17e0 con 0x55860505edd0 2024-03-11T19:40:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.713+0000 7f5d3cdb7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059c60 0x55860505b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.713+0000 7f5d3cdb7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059c60 0x55860505b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37682/0 (socket says 172.21.15.83:37682) 2024-03-11T19:40:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.713+0000 7f5d3cdb7640 1 -- 172.21.15.83:0/324785769 learned_addr learned my addr 172.21.15.83:0/324785769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.713+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/324785769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059270 0x55860505b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.714+0000 7f5d3ddb9640 1 --2- 172.21.15.83:0/324785769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860505edd0 0x55860506a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.714+0000 7f5d3cdb7640 1 -- 172.21.15.83:0/324785769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860505edd0 msgr2=0x55860506a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.714+0000 7f5d3cdb7640 1 --2- 172.21.15.83:0/324785769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860505edd0 0x55860506a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.714+0000 7f5d3cdb7640 1 -- 172.21.15.83:0/324785769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059270 msgr2=0x55860505b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.714+0000 7f5d3cdb7640 1 --2- 172.21.15.83:0/324785769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059270 0x55860505b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.714+0000 7f5d3cdb7640 1 -- 172.21.15.83:0/324785769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558604f5cc20 con 0x558605059c60 2024-03-11T19:40:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.714+0000 7f5d3ddb9640 1 --2- 172.21.15.83:0/324785769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860505edd0 0x55860506a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:40:08.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.714+0000 7f5d3cdb7640 1 --2- 172.21.15.83:0/324785769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059c60 0x55860505b6f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800b880 tx=0x7f5d2800b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0c6512114f796a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.715+0000 7f5d22ffd640 1 -- 172.21.15.83:0/324785769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28004450 con 0x558605059c60 2024-03-11T19:40:08.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.715+0000 7f5d22ffd640 1 -- 172.21.15.83:0/324785769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d2800acd0 con 0x558605059c60 2024-03-11T19:40:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.716+0000 7f5d423f5740 1 -- 172.21.15.83:0/324785769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059c60 msgr2=0x55860505b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.716+0000 7f5d423f5740 1 --2- 172.21.15.83:0/324785769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059c60 0x55860505b6f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800b880 tx=0x7f5d2800b3f0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.717+0000 7f5d423f5740 1 -- 172.21.15.83:0/324785769 shutdown_connections 2024-03-11T19:40:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.717+0000 7f5d423f5740 1 --2- 172.21.15.83:0/324785769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55860505edd0 0x55860506a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.717+0000 7f5d423f5740 1 --2- 172.21.15.83:0/324785769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558605059c60 0x55860505b6f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.717+0000 7f5d423f5740 1 --2- 172.21.15.83:0/324785769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059270 0x55860505b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.717+0000 7f5d423f5740 1 -- 172.21.15.83:0/324785769 >> 172.21.15.83:0/324785769 conn(0x5586050547e0 msgr2=0x558605061ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.717+0000 7f5d423f5740 1 -- 172.21.15.83:0/324785769 shutdown_connections 2024-03-11T19:40:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.717+0000 7f5d423f5740 1 -- 172.21.15.83:0/324785769 wait complete. 2024-03-11T19:40:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.717+0000 7f5d423f5740 1 -- start start 2024-03-11T19:40:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.717+0000 7f5d423f5740 1 Processor -- start 2024-03-11T19:40:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.718+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059270 0x55860506a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.718+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 0x558605068e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.718+0000 7f5d423f5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55860505edd0 0x5586050693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.718+0000 7f5d3cdb7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 0x558605068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.718+0000 7f5d3cdb7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 0x558605068e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:40272/0 (socket says 172.21.15.83:40272) 2024-03-11T19:40:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.718+0000 7f5d3cdb7640 1 -- 172.21.15.83:0/2401737115 learned_addr learned my addr 172.21.15.83:0/2401737115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.718+0000 7f5d423f5740 1 -- 172.21.15.83:0/2401737115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558605069aa0 con 0x55860505edd0 2024-03-11T19:40:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.719+0000 7f5d423f5740 1 -- 172.21.15.83:0/2401737115 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558605069c10 con 0x558605059270 2024-03-11T19:40:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.719+0000 7f5d423f5740 1 -- 172.21.15.83:0/2401737115 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586050c5b10 con 0x558605059c60 2024-03-11T19:40:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.719+0000 7f5d423f5740 10 client.0 _ll_register_callbacks cb 0x7f5d34096a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.719+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059270 0x55860506a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.719+0000 7f5d3ddb9640 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55860505edd0 0x5586050693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.720+0000 7f5d3cdb7640 1 -- 172.21.15.83:0/2401737115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059270 msgr2=0x55860506a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.720+0000 7f5d3cdb7640 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059270 0x55860506a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.720+0000 7f5d3cdb7640 1 -- 172.21.15.83:0/2401737115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55860505edd0 msgr2=0x5586050693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.720+0000 7f5d3cdb7640 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55860505edd0 0x5586050693c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.720+0000 7f5d3cdb7640 1 -- 172.21.15.83:0/2401737115 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d38111600 con 0x558605059c60 2024-03-11T19:40:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.720+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059270 0x55860506a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:40:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.720+0000 7f5d3cdb7640 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 0x558605068e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d28035040 tx=0x7f5d2800a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56125e3e478a9874 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.721+0000 7f5d20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:40:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.721+0000 7f5d20ff9640 1 -- 172.21.15.83:0/2401737115 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800a7f0 con 0x558605059c60 2024-03-11T19:40:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.721+0000 7f5d20ff9640 1 -- 172.21.15.83:0/2401737115 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5d2800a950 con 0x558605059c60 2024-03-11T19:40:08.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.721+0000 7f5d423f5740 1 -- 172.21.15.83:0/2401737115 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d34065f30 con 0x558605059c60 2024-03-11T19:40:08.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.721+0000 7f5d20ff9640 1 -- 172.21.15.83:0/2401737115 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d2800ab10 con 0x558605059c60 2024-03-11T19:40:08.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.721+0000 7f5d423f5740 10 client.25147 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:08.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.721+0000 7f5d423f5740 1 -- 172.21.15.83:0/2401737115 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558604f828c0 con 0x558605059c60 2024-03-11T19:40:08.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.722+0000 7f5d20ff9640 1 -- 172.21.15.83:0/2401737115 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d28005500 con 0x558605059c60 2024-03-11T19:40:08.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.723+0000 7f5d20ff9640 1 -- 172.21.15.83:0/2401737115 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5d2803d450 con 0x558605059c60 2024-03-11T19:40:08.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.723+0000 7f5d423f5740 10 client.25147 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:08.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.723+0000 7f5d423f5740 10 client.25147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:08.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.723+0000 7f5d423f5740 20 client.25147 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:08.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.723+0000 7f5d423f5740 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55860506f5a0 0x5586050c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:08.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.723+0000 7f5d423f5740 4 client.25147 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:40:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.723+0000 7f5d423f5740 1 -- 172.21.15.83:0/2401737115 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f5d34065f30 con 0x55860506f5a0 2024-03-11T19:40:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.723+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55860506f5a0 0x5586050c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.724+0000 7f5d20ff9640 10 client.25147 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.724+0000 7f5d3d5b8640 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55860506f5a0 0x5586050c9340 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b637faa051715bff server_cookie=1d243ea27887a204 in_seq=0 out_seq=0 2024-03-11T19:40:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d20ff9640 1 -- 172.21.15.83:0/2401737115 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f5d34065f30 con 0x55860506f5a0 2024-03-11T19:40:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d20ff9640 10 client.25147 handle_command_reply: tid=0 2024-03-11T19:40:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 1 client.25147 shutdown 2024-03-11T19:40:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 1 -- 172.21.15.83:0/2401737115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 msgr2=0x558605068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 0x558605068e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5d28035040 tx=0x7f5d2800a5b0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 1 -- 172.21.15.83:0/2401737115 shutdown_connections 2024-03-11T19:40:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55860506f5a0 0x5586050c9340 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55860505edd0 0x5586050693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558605059c60 0x558605068e80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 1 --2- 172.21.15.83:0/2401737115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558605059270 0x55860506a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d20ff9640 0 client.25147 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 1 -- 172.21.15.83:0/2401737115 >> 172.21.15.83:0/2401737115 conn(0x5586050547e0 msgr2=0x558605063390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 1 -- 172.21.15.83:0/2401737115 shutdown_connections 2024-03-11T19:40:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 1 -- 172.21.15.83:0/2401737115 wait complete. 2024-03-11T19:40:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.818+0000 7f5d423f5740 20 client.25147 trim_cache size 0 max 16384 2024-03-11T19:40:08.824 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:40:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3486302428 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5d240a1ab0 msgr2=0x7f5d240a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5d240a1ab0 0x7f5d240a3f70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5d30013ce0 tx=0x7f5d300041b0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3486302428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38109290 msgr2=0x7f5d3819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38109290 0x7f5d3819d6c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400bd40 tx=0x7f5d34008600 comp rx=0 tx=0).stop 2024-03-11T19:40:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3486302428 shutdown_connections 2024-03-11T19:40:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5d240a1ab0 0x7f5d240a3f70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3810e580 0x7f5d381a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d3810bbc0 0x7f5d3819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 --2- 172.21.15.83:0/3486302428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d38109290 0x7f5d3819d6c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:08.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3486302428 >> 172.21.15.83:0/3486302428 conn(0x7f5d3806e380 msgr2=0x7f5d3810c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:08.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3486302428 shutdown_connections 2024-03-11T19:40:08.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:08.819+0000 7f5d3f81c640 1 -- 172.21.15.83:0/3486302428 wait complete. 2024-03-11T19:40:08.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:08 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/855895418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:40:08.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:08 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/855895418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:40:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:08 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/855895418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:40:09.841 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:40:09.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:40:09.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:40:09.876 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:09 smithi083 ceph-mon[31864]: pgmap v377: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2024-03-11T19:40:09.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:09 smithi137 ceph-mon[36220]: pgmap v377: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2024-03-11T19:40:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:09 smithi204 ceph-mon[36739]: pgmap v377: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2024-03-11T19:40:10.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.998+0000 7f1c67a15640 1 -- 172.21.15.83:0/3589122874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6010fa00 msgr2=0x7f1c60111df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.998+0000 7f1c67a15640 1 --2- 172.21.15.83:0/3589122874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6010fa00 0x7f1c60111df0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009900 tx=0x7f1c5c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:40:10.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.998+0000 7f1c67a15640 1 -- 172.21.15.83:0/3589122874 shutdown_connections 2024-03-11T19:40:10.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.998+0000 7f1c67a15640 1 --2- 172.21.15.83:0/3589122874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6010fa00 0x7f1c60111df0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.998+0000 7f1c67a15640 1 --2- 172.21.15.83:0/3589122874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c60108ae0 0x7f1c6010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.998+0000 7f1c67a15640 1 --2- 172.21.15.83:0/3589122874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c60108180 0x7f1c601085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.998+0000 7f1c67a15640 1 -- 172.21.15.83:0/3589122874 >> 172.21.15.83:0/3589122874 conn(0x7f1c6006dfe0 msgr2=0x7f1c60070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.998+0000 7f1c67a15640 1 -- 172.21.15.83:0/3589122874 shutdown_connections 2024-03-11T19:40:10.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.998+0000 7f1c67a15640 1 -- 172.21.15.83:0/3589122874 wait complete. 2024-03-11T19:40:10.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.999+0000 7f1c67a15640 1 Processor -- start 2024-03-11T19:40:10.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.999+0000 7f1c67a15640 1 -- start start 2024-03-11T19:40:10.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:09.999+0000 7f1c67a15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c60108180 0x7f1c601a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c67a15640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c60108ae0 0x7f1c601a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c67a15640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6010fa00 0x7f1c601ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c67a15640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c601adb40 con 0x7f1c60108ae0 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c67a15640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c601adcb0 con 0x7f1c60108180 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c67a15640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c601ade70 con 0x7f1c6010fa00 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c657b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c60108180 0x7f1c601a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c657b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c60108180 0x7f1c601a6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:44694/0 (socket says 172.21.15.83:44694) 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c657b1640 1 -- 172.21.15.83:0/973595017 learned_addr learned my addr 172.21.15.83:0/973595017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c64fb0640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c60108ae0 0x7f1c601a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6010fa00 0x7f1c601ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c657b1640 1 -- 172.21.15.83:0/973595017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6010fa00 msgr2=0x7f1c601ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c657b1640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6010fa00 0x7f1c601ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c657b1640 1 -- 172.21.15.83:0/973595017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c60108ae0 msgr2=0x7f1c601a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c657b1640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c60108ae0 0x7f1c601a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c657b1640 1 -- 172.21.15.83:0/973595017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c009560 con 0x7f1c60108180 2024-03-11T19:40:10.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.000+0000 7f1c657b1640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c60108180 0x7f1c601a6050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800eb90 tx=0x7f1c5800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.001+0000 7f1c567fc640 1 -- 172.21.15.83:0/973595017 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c580099b0 con 0x7f1c60108180 2024-03-11T19:40:10.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.001+0000 7f1c67a15640 1 -- 172.21.15.83:0/973595017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c601ae150 con 0x7f1c60108180 2024-03-11T19:40:10.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.001+0000 7f1c67a15640 1 -- 172.21.15.83:0/973595017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c601ae660 con 0x7f1c60108180 2024-03-11T19:40:10.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.001+0000 7f1c567fc640 1 -- 172.21.15.83:0/973595017 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c5800ee90 con 0x7f1c60108180 2024-03-11T19:40:10.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.001+0000 7f1c567fc640 1 -- 172.21.15.83:0/973595017 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c580107a0 con 0x7f1c60108180 2024-03-11T19:40:10.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.003+0000 7f1c567fc640 1 -- 172.21.15.83:0/973595017 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1c58010900 con 0x7f1c60108180 2024-03-11T19:40:10.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.004+0000 7f1c6a5ed740 1 Processor -- start 2024-03-11T19:40:10.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.004+0000 7f1c567fc640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c4c0a1b30 0x7f1c4c0a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.004+0000 7f1c6a5ed740 1 -- start start 2024-03-11T19:40:10.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.004+0000 7f1c567fc640 1 -- 172.21.15.83:0/973595017 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1c58014070 con 0x7f1c60108180 2024-03-11T19:40:10.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.004+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c4c0a1b30 0x7f1c4c0a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.004+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498a4c0 0x55636498a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.005+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x55636498bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.005+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498c050 0x5563649928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.005+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563648f9f90 con 0x55636498c050 2024-03-11T19:40:10.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.005+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563648f74c0 con 0x55636498a4c0 2024-03-11T19:40:10.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.005+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563648f2ca0 con 0x55636498b690 2024-03-11T19:40:10.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.005+0000 7f1c64fb0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x55636498bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.005+0000 7f1c64fb0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x55636498bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:39396/0 (socket says 172.21.15.83:39396) 2024-03-11T19:40:10.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.005+0000 7f1c64fb0640 1 -- 172.21.15.83:0/1965012472 learned_addr learned my addr 172.21.15.83:0/1965012472 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:10.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.005+0000 7f1c657b1640 1 --2- 172.21.15.83:0/1965012472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498a4c0 0x55636498a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.005+0000 7f1c657b1640 1 -- 172.21.15.83:0/1965012472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 msgr2=0x55636498bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.006+0000 7f1c657b1640 1 --2- 172.21.15.83:0/1965012472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x55636498bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.006+0000 7f1c657b1640 1 -- 172.21.15.83:0/1965012472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498c050 msgr2=0x5563649928e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:40:10.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.006+0000 7f1c657b1640 1 --2- 172.21.15.83:0/1965012472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498c050 0x5563649928e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.006+0000 7f1c657b1640 1 -- 172.21.15.83:0/1965012472 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563648b50b0 con 0x55636498a4c0 2024-03-11T19:40:10.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.006+0000 7f1c64fb0640 1 --2- 172.21.15.83:0/1965012472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x55636498bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:10.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.006+0000 7f1c657b1640 1 --2- 172.21.15.83:0/1965012472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498a4c0 0x55636498a8e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800cba0 tx=0x7f1c5809b130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6598039e27cc9620 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.006+0000 7f1c4b7fe640 1 -- 172.21.15.83:0/1965012472 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5809eb80 con 0x55636498a4c0 2024-03-11T19:40:10.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.006+0000 7f1c4b7fe640 1 -- 172.21.15.83:0/1965012472 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c580a31d0 con 0x55636498a4c0 2024-03-11T19:40:10.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.006+0000 7f1c4b7fe640 1 -- 172.21.15.83:0/1965012472 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5809a0e0 con 0x55636498a4c0 2024-03-11T19:40:10.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.007+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1965012472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498a4c0 msgr2=0x55636498a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.007+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1965012472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498a4c0 0x55636498a8e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800cba0 tx=0x7f1c5809b130 comp rx=0 tx=0).stop 2024-03-11T19:40:10.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.008+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c4c0a1b30 0x7f1c4c0a3ff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c02c170 tx=0x7f1c5c02c810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.008+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1965012472 shutdown_connections 2024-03-11T19:40:10.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.008+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1965012472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498c050 0x5563649928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.008+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1965012472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x55636498bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.008+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1965012472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498a4c0 0x55636498a8e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.008+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1965012472 >> 172.21.15.83:0/1965012472 conn(0x5563649856a0 msgr2=0x556364985aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.008+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1965012472 shutdown_connections 2024-03-11T19:40:10.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.008+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1965012472 wait complete. 2024-03-11T19:40:10.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c6a5ed740 1 -- start start 2024-03-11T19:40:10.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c6a5ed740 1 Processor -- start 2024-03-11T19:40:10.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x5563649946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498b690 0x556364994c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 0x55636499bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556364897f40 con 0x55636498a4c0 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563645c64b0 con 0x55636498b690 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556364878670 con 0x55636498c050 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c6a5ed740 10 client.0 _ll_register_callbacks cb 0x5563648e5f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c657b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x5563649946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c64fb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498b690 0x556364994c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c657b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x5563649946d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51474/0 (socket says 172.21.15.83:51474) 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.009+0000 7f1c657b1640 1 -- 172.21.15.83:0/1527204825 learned_addr learned my addr 172.21.15.83:0/1527204825 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.010+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 0x55636499bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.010+0000 7f1c65fb2640 1 -- 172.21.15.83:0/1527204825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498b690 msgr2=0x556364994c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.010+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498b690 0x556364994c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.010+0000 7f1c65fb2640 1 -- 172.21.15.83:0/1527204825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 msgr2=0x5563649946d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.010+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x5563649946d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.010+0000 7f1c65fb2640 1 -- 172.21.15.83:0/1527204825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c601ae660 con 0x55636498c050 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.011+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 0x55636499bc50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c02c240 tx=0x7f1c5c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13d8dd2b35dd8716 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.011+0000 7f1c48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.011+0000 7f1c48ff9640 1 -- 172.21.15.83:0/1527204825 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c02dea0 con 0x55636498c050 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.011+0000 7f1c48ff9640 1 -- 172.21.15.83:0/1527204825 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1c5c047040 con 0x55636498c050 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.011+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1527204825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563648ccf40 con 0x55636498c050 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.011+0000 7f1c48ff9640 1 -- 172.21.15.83:0/1527204825 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c5c004450 con 0x55636498c050 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.011+0000 7f1c6a5ed740 10 client.25159 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.011+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1527204825 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5563648f1320 con 0x55636498c050 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.012+0000 7f1c48ff9640 1 -- 172.21.15.83:0/1527204825 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5563648ccf40 con 0x55636498c050 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.012+0000 7f1c48ff9640 1 -- 172.21.15.83:0/1527204825 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1c5c005060 con 0x55636498c050 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.012+0000 7f1c6a5ed740 10 client.25159 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.012+0000 7f1c6a5ed740 10 client.25159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:10.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.012+0000 7f1c6a5ed740 20 client.25159 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:10.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.012+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5563649a0770 0x5563649a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.012+0000 7f1c6a5ed740 4 client.25159 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:10.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.012+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1527204825 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563648e9310 con 0x5563649a0770 2024-03-11T19:40:10.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.012+0000 7f1c657b1640 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5563649a0770 0x5563649a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.012+0000 7f1c657b1640 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5563649a0770 0x5563649a2b90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4ca34447b35940f server_cookie=5281094d6110b421 in_seq=0 out_seq=0 2024-03-11T19:40:10.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.012+0000 7f1c48ff9640 10 client.25159 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:10.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.013+0000 7f1c48ff9640 1 -- 172.21.15.83:0/1527204825 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5563648e9310 con 0x5563649a0770 2024-03-11T19:40:10.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.013+0000 7f1c48ff9640 10 client.25159 handle_command_reply: tid=0 2024-03-11T19:40:10.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.013+0000 7f1c6a5ed740 1 client.25159 shutdown 2024-03-11T19:40:10.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1527204825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 msgr2=0x55636499bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 0x55636499bc50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c02c240 tx=0x7f1c5c009260 comp rx=0 tx=0).stop 2024-03-11T19:40:10.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1527204825 shutdown_connections 2024-03-11T19:40:10.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5563649a0770 0x5563649a2b90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 0x55636499bc50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498b690 0x556364994c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1527204825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x5563649946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c48ff9640 0 client.25159 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:10.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1527204825 >> 172.21.15.83:0/1527204825 conn(0x5563649856a0 msgr2=0x5563649848e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1527204825 shutdown_connections 2024-03-11T19:40:10.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1527204825 wait complete. 2024-03-11T19:40:10.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.014+0000 7f1c6a5ed740 20 client.25159 trim_cache size 0 max 16384 2024-03-11T19:40:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.040+0000 7f1c6a5ed740 1 Processor -- start 2024-03-11T19:40:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.040+0000 7f1c6a5ed740 1 -- start start 2024-03-11T19:40:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.040+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x5563649946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.040+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498b690 0x55636499bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.040+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 0x556364994c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.040+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556364993820 con 0x55636498a4c0 2024-03-11T19:40:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.040+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556364994510 con 0x55636498b690 2024-03-11T19:40:10.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.040+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563648e9310 con 0x55636498c050 2024-03-11T19:40:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.040+0000 7f1c64fb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498b690 0x55636499bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.040+0000 7f1c64fb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498b690 0x55636499bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:44724/0 (socket says 172.21.15.83:44724) 2024-03-11T19:40:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.040+0000 7f1c64fb0640 1 -- 172.21.15.83:0/1256532144 learned_addr learned my addr 172.21.15.83:0/1256532144 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.041+0000 7f1c657b1640 1 --2- 172.21.15.83:0/1256532144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x5563649946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.041+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/1256532144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 0x556364994c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.041+0000 7f1c65fb2640 1 -- 172.21.15.83:0/1256532144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498b690 msgr2=0x55636499bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.041+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/1256532144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498b690 0x55636499bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.041+0000 7f1c65fb2640 1 -- 172.21.15.83:0/1256532144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 msgr2=0x5563649946d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.041+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/1256532144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x5563649946d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.041+0000 7f1c65fb2640 1 -- 172.21.15.83:0/1256532144 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55636488dc00 con 0x55636498c050 2024-03-11T19:40:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.041+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/1256532144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 0x556364994c10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c005490 tx=0x7f1c5c005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec735c4161be92f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.042+0000 7f1c4affd640 1 -- 172.21.15.83:0/1256532144 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c005600 con 0x55636498c050 2024-03-11T19:40:10.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.042+0000 7f1c4affd640 1 -- 172.21.15.83:0/1256532144 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c5c02ce90 con 0x55636498c050 2024-03-11T19:40:10.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.042+0000 7f1c4affd640 1 -- 172.21.15.83:0/1256532144 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c060480 con 0x55636498c050 2024-03-11T19:40:10.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.043+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1256532144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 msgr2=0x556364994c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.043+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1256532144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 0x556364994c10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c005490 tx=0x7f1c5c005a20 comp rx=0 tx=0).stop 2024-03-11T19:40:10.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.043+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1256532144 shutdown_connections 2024-03-11T19:40:10.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.043+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1256532144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498c050 0x556364994c10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.043+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1256532144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498b690 0x55636499bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.043+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/1256532144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x5563649946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.043+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1256532144 >> 172.21.15.83:0/1256532144 conn(0x5563649856a0 msgr2=0x5563649a74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.043+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1256532144 shutdown_connections 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.043+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/1256532144 wait complete. 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.044+0000 7f1c6a5ed740 1 -- start start 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.044+0000 7f1c6a5ed740 1 Processor -- start 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.044+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x556364999a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.044+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x556364999f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.044+0000 7f1c6a5ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498c050 0x55636499af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.044+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636499b490 con 0x55636498a4c0 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.044+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636499b600 con 0x55636498c050 2024-03-11T19:40:10.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.044+0000 7f1c6a5ed740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55636499b7c0 con 0x55636498b690 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.044+0000 7f1c6a5ed740 10 client.0 _ll_register_callbacks cb 0x7f1c58004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c64fb0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x556364999f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c64fb0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x556364999f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:39444/0 (socket says 172.21.15.83:39444) 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c64fb0640 1 -- 172.21.15.83:0/2537157558 learned_addr learned my addr 172.21.15.83:0/2537157558 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c65fb2640 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498c050 0x55636499af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c657b1640 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x556364999a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c64fb0640 1 -- 172.21.15.83:0/2537157558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498c050 msgr2=0x55636499af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c64fb0640 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498c050 0x55636499af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c64fb0640 1 -- 172.21.15.83:0/2537157558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 msgr2=0x556364999a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c64fb0640 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x556364999a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c64fb0640 1 -- 172.21.15.83:0/2537157558 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563648f1320 con 0x55636498b690 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c64fb0640 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x556364999f40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1c50010840 tx=0x7f1c50010d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98b04a8d591e5d91 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:40:10.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c48ff9640 1 -- 172.21.15.83:0/2537157558 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50004460 con 0x55636498b690 2024-03-11T19:40:10.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.045+0000 7f1c48ff9640 1 -- 172.21.15.83:0/2537157558 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1c5000f920 con 0x55636498b690 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.046+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/2537157558 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556364918900 con 0x55636498b690 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.046+0000 7f1c48ff9640 1 -- 172.21.15.83:0/2537157558 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c5000fae0 con 0x55636498b690 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.046+0000 7f1c6a5ed740 10 client.25165 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.046+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/2537157558 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5563648b3910 con 0x55636498b690 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.046+0000 7f1c48ff9640 1 -- 172.21.15.83:0/2537157558 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556364918900 con 0x55636498b690 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.046+0000 7f1c48ff9640 1 -- 172.21.15.83:0/2537157558 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1c50025070 con 0x55636498b690 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.046+0000 7f1c6a5ed740 10 client.25165 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.046+0000 7f1c6a5ed740 10 client.25165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.046+0000 7f1c6a5ed740 20 client.25165 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.046+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5563649f7db0 0x5563649fa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.047+0000 7f1c6a5ed740 4 client.25165 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:40:10.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.047+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/2537157558 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5563645f2860 con 0x5563649f7db0 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.047+0000 7f1c657b1640 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5563649f7db0 0x5563649fa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.047+0000 7f1c48ff9640 10 client.25165 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.048+0000 7f1c657b1640 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5563649f7db0 0x5563649fa1d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d5cf4e64e3d0a9 server_cookie=d7cdf93d4817f221 in_seq=0 out_seq=0 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c48ff9640 1 -- 172.21.15.83:0/2537157558 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5563645f2860 con 0x5563649f7db0 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c48ff9640 10 client.25165 handle_command_reply: tid=0 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c6a5ed740 1 client.25165 shutdown 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/2537157558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 msgr2=0x556364999f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x556364999f40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1c50010840 tx=0x7f1c50010d00 comp rx=0 tx=0).stop 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/2537157558 shutdown_connections 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5563649f7db0 0x5563649fa1d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55636498c050 0x55636499af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55636498b690 0x556364999f40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c48ff9640 0 client.25165 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:10.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c6a5ed740 1 --2- 172.21.15.83:0/2537157558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55636498a4c0 0x556364999a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.049+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/2537157558 >> 172.21.15.83:0/2537157558 conn(0x5563649856a0 msgr2=0x55636498f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.050+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/2537157558 shutdown_connections 2024-03-11T19:40:10.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.050+0000 7f1c6a5ed740 1 -- 172.21.15.83:0/2537157558 wait complete. 2024-03-11T19:40:10.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.050+0000 7f1c6a5ed740 20 client.25165 trim_cache size 0 max 16384 2024-03-11T19:40:10.062 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:40:10.062 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:40:10.062 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "d13b35fe-2f7c-491e-83d4-b205377d3c7b", 2024-03-11T19:40:10.062 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:40:10.062 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:40:10.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.051+0000 7f1c67a15640 1 -- 172.21.15.83:0/973595017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c4c0a1b30 msgr2=0x7f1c4c0a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.051+0000 7f1c67a15640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c4c0a1b30 0x7f1c4c0a3ff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c02c170 tx=0x7f1c5c02c810 comp rx=0 tx=0).stop 2024-03-11T19:40:10.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.052+0000 7f1c67a15640 1 -- 172.21.15.83:0/973595017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c60108180 msgr2=0x7f1c601a6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.052+0000 7f1c67a15640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c60108180 0x7f1c601a6050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800eb90 tx=0x7f1c5800c6a0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.052+0000 7f1c67a15640 1 -- 172.21.15.83:0/973595017 shutdown_connections 2024-03-11T19:40:10.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.052+0000 7f1c67a15640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c4c0a1b30 0x7f1c4c0a3ff0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.052+0000 7f1c67a15640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6010fa00 0x7f1c601ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.052+0000 7f1c67a15640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c60108ae0 0x7f1c601a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.052+0000 7f1c67a15640 1 --2- 172.21.15.83:0/973595017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c60108180 0x7f1c601a6050 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.052+0000 7f1c67a15640 1 -- 172.21.15.83:0/973595017 >> 172.21.15.83:0/973595017 conn(0x7f1c6006dfe0 msgr2=0x7f1c60115d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.052+0000 7f1c67a15640 1 -- 172.21.15.83:0/973595017 shutdown_connections 2024-03-11T19:40:10.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.052+0000 7f1c67a15640 1 -- 172.21.15.83:0/973595017 wait complete. 2024-03-11T19:40:10.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:40:10.072 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:40:10.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.272+0000 7f224aaa3640 1 -- 172.21.15.83:0/2631957435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101cb0 msgr2=0x7f2244108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.272+0000 7f224aaa3640 1 --2- 172.21.15.83:0/2631957435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101cb0 0x7f2244108540 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f223c009ca0 tx=0x7f223c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.272+0000 7f224aaa3640 1 -- 172.21.15.83:0/2631957435 shutdown_connections 2024-03-11T19:40:10.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.272+0000 7f224aaa3640 1 --2- 172.21.15.83:0/2631957435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101cb0 0x7f2244108540 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.272+0000 7f224aaa3640 1 --2- 172.21.15.83:0/2631957435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22441012f0 0x7f2244101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.272+0000 7f224aaa3640 1 --2- 172.21.15.83:0/2631957435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100100 0x7f2244100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.272+0000 7f224aaa3640 1 -- 172.21.15.83:0/2631957435 >> 172.21.15.83:0/2631957435 conn(0x7f22440fb670 msgr2=0x7f22440fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.272+0000 7f224aaa3640 1 -- 172.21.15.83:0/2631957435 shutdown_connections 2024-03-11T19:40:10.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.272+0000 7f224aaa3640 1 -- 172.21.15.83:0/2631957435 wait complete. 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f224aaa3640 1 Processor -- start 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f224aaa3640 1 -- start start 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f224aaa3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100100 0x7f22441a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f224aaa3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22441012f0 0x7f22441a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f224aaa3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101cb0 0x7f22441a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f224aaa3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22441a5300 con 0x7f2244101cb0 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f224aaa3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22441a5470 con 0x7f22441012f0 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f224aaa3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22441a5630 con 0x7f2244100100 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f224883f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100100 0x7f22441a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f2249040640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101cb0 0x7f22441a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f2243fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22441012f0 0x7f22441a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f224883f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100100 0x7f22441a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:39466/0 (socket says 172.21.15.83:39466) 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f224883f640 1 -- 172.21.15.83:0/2979743015 learned_addr learned my addr 172.21.15.83:0/2979743015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.273+0000 7f2249040640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101cb0 0x7f22441a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51494/0 (socket says 172.21.15.83:51494) 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.274+0000 7f2243fff640 1 -- 172.21.15.83:0/2979743015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100100 msgr2=0x7f22441a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.274+0000 7f2243fff640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100100 0x7f22441a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.274+0000 7f2243fff640 1 -- 172.21.15.83:0/2979743015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101cb0 msgr2=0x7f22441a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.274+0000 7f2243fff640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101cb0 0x7f22441a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.274+0000 7f2243fff640 1 -- 172.21.15.83:0/2979743015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c009900 con 0x7f22441012f0 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.274+0000 7f2243fff640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22441012f0 0x7f22441a25a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f223000e9f0 tx=0x7f223000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.274+0000 7f2241ffb640 1 -- 172.21.15.83:0/2979743015 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223000ccc0 con 0x7f22441012f0 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.274+0000 7f2241ffb640 1 -- 172.21.15.83:0/2979743015 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f223000ce20 con 0x7f22441012f0 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.274+0000 7f2241ffb640 1 -- 172.21.15.83:0/2979743015 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230018920 con 0x7f22441012f0 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.274+0000 7f224aaa3640 1 -- 172.21.15.83:0/2979743015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22441a5910 con 0x7f22441012f0 2024-03-11T19:40:10.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.275+0000 7f224aaa3640 1 -- 172.21.15.83:0/2979743015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22441a5ce0 con 0x7f22441012f0 2024-03-11T19:40:10.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.278+0000 7f2241ffb640 1 -- 172.21.15.83:0/2979743015 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2230018a80 con 0x7f22441012f0 2024-03-11T19:40:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.278+0000 7f2241ffb640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22340a1ab0 0x7f22340a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.278+0000 7f2241ffb640 1 -- 172.21.15.83:0/2979743015 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2230014070 con 0x7f22441012f0 2024-03-11T19:40:10.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.278+0000 7f224883f640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22340a1ab0 0x7f22340a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.279+0000 7f224883f640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22340a1ab0 0x7f22340a3f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2244108910 tx=0x7f2238005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.279+0000 7f224d670740 1 Processor -- start 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.279+0000 7f224d670740 1 -- start start 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.279+0000 7f224d670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f22466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224d670740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f224b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224d670740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 0x5604f224e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224d670740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f21b6280 con 0x5604f22462d0 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224883f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f22466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224883f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f22466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51508/0 (socket says 172.21.15.83:51508) 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224d670740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f21b37b0 con 0x5604f224be30 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224883f640 1 -- 172.21.15.83:0/2846187390 learned_addr learned my addr 172.21.15.83:0/2846187390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224d670740 1 -- 172.21.15.83:0/2846187390 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f21aef90 con 0x5604f2246cc0 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f2249040640 1 --2- 172.21.15.83:0/2846187390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 0x5604f224e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f2243fff640 1 --2- 172.21.15.83:0/2846187390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f224b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224883f640 1 -- 172.21.15.83:0/2846187390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 msgr2=0x5604f224b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224883f640 1 --2- 172.21.15.83:0/2846187390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f224b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224883f640 1 -- 172.21.15.83:0/2846187390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 msgr2=0x5604f224e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224883f640 1 --2- 172.21.15.83:0/2846187390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 0x5604f224e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224883f640 1 -- 172.21.15.83:0/2846187390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604f216d570 con 0x5604f22462d0 2024-03-11T19:40:10.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f2249040640 1 --2- 172.21.15.83:0/2846187390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 0x5604f224e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:10.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f224883f640 1 --2- 172.21.15.83:0/2846187390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f22466f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2238009f60 tx=0x7f2238014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce95023bf73f5b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f222effd640 1 -- 172.21.15.83:0/2846187390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223800ed70 con 0x5604f22462d0 2024-03-11T19:40:10.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f222effd640 1 -- 172.21.15.83:0/2846187390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2238021070 con 0x5604f22462d0 2024-03-11T19:40:10.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.280+0000 7f222effd640 1 -- 172.21.15.83:0/2846187390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223801d670 con 0x5604f22462d0 2024-03-11T19:40:10.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.281+0000 7f224d670740 1 -- 172.21.15.83:0/2846187390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 msgr2=0x5604f22466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.281+0000 7f224d670740 1 --2- 172.21.15.83:0/2846187390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f22466f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2238009f60 tx=0x7f2238014000 comp rx=0 tx=0).stop 2024-03-11T19:40:10.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.281+0000 7f224d670740 1 -- 172.21.15.83:0/2846187390 shutdown_connections 2024-03-11T19:40:10.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.281+0000 7f224d670740 1 --2- 172.21.15.83:0/2846187390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 0x5604f224e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.281+0000 7f224d670740 1 --2- 172.21.15.83:0/2846187390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f224b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.281+0000 7f224d670740 1 --2- 172.21.15.83:0/2846187390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f22466f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.281+0000 7f224d670740 1 -- 172.21.15.83:0/2846187390 >> 172.21.15.83:0/2846187390 conn(0x5604f22417f0 msgr2=0x5604f2241bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.282+0000 7f224d670740 1 -- 172.21.15.83:0/2846187390 shutdown_connections 2024-03-11T19:40:10.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.283+0000 7f224d670740 1 -- 172.21.15.83:0/2846187390 wait complete. 2024-03-11T19:40:10.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.283+0000 7f224d670740 1 -- start start 2024-03-11T19:40:10.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.284+0000 7f224d670740 1 Processor -- start 2024-03-11T19:40:10.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.284+0000 7f224d670740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f22462d0 0x5604f22610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.284+0000 7f224d670740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f2261620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.284+0000 7f224d670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f224be30 0x5604f22685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.284+0000 7f224d670740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f1e82430 con 0x5604f224be30 2024-03-11T19:40:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.284+0000 7f224d670740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f1e3ee00 con 0x5604f22462d0 2024-03-11T19:40:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.284+0000 7f224883f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f22462d0 0x5604f22610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.284+0000 7f224883f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f22462d0 0x5604f22610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:44766/0 (socket says 172.21.15.83:44766) 2024-03-11T19:40:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.284+0000 7f224883f640 1 -- 172.21.15.83:0/2823422937 learned_addr learned my addr 172.21.15.83:0/2823422937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.284+0000 7f224d670740 1 -- 172.21.15.83:0/2823422937 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f2133fd0 con 0x5604f2246cc0 2024-03-11T19:40:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.284+0000 7f2249040640 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f224be30 0x5604f22685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.285+0000 7f224883f640 1 -- 172.21.15.83:0/2823422937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 msgr2=0x5604f2261620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:40:10.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.285+0000 7f224d670740 10 client.0 _ll_register_callbacks cb 0x5604f2189fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.285+0000 7f224883f640 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f2261620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.285+0000 7f224883f640 1 -- 172.21.15.83:0/2823422937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f224be30 msgr2=0x5604f22685f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.285+0000 7f224883f640 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f224be30 0x5604f22685f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.285+0000 7f224883f640 1 -- 172.21.15.83:0/2823422937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2238017070 con 0x5604f22462d0 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.285+0000 7f224d670740 1 -- 172.21.15.83:0/2823422937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604f1eae7e0 con 0x5604f22462d0 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.285+0000 7f224883f640 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f22462d0 0x5604f22610e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2238009760 tx=0x7f223800eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69b5476351f0004a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.285+0000 7f2223fff640 10 client.25067 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.285+0000 7f2223fff640 1 -- 172.21.15.83:0/2823422937 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238005320 con 0x5604f22462d0 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.285+0000 7f2223fff640 1 -- 172.21.15.83:0/2823422937 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2238005480 con 0x5604f22462d0 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.286+0000 7f2223fff640 1 -- 172.21.15.83:0/2823422937 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f223801c510 con 0x5604f22462d0 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.286+0000 7f224d670740 10 client.25067 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:10.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.286+0000 7f224d670740 1 -- 172.21.15.83:0/2823422937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x5604f21ad610 con 0x5604f22462d0 2024-03-11T19:40:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.286+0000 7f2223fff640 1 -- 172.21.15.83:0/2823422937 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f223801c670 con 0x5604f22462d0 2024-03-11T19:40:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.286+0000 7f224d670740 10 client.25067 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.286+0000 7f224d670740 10 client.25067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.286+0000 7f224d670740 20 client.25067 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.286+0000 7f224d670740 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5604f226cfe0 0x5604f226f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.286+0000 7f224d670740 4 client.25067 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.286+0000 7f224d670740 1 -- 172.21.15.83:0/2823422937 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604f1eeb720 con 0x5604f226cfe0 2024-03-11T19:40:10.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.286+0000 7f2243fff640 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5604f226cfe0 0x5604f226f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.287+0000 7f2243fff640 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5604f226cfe0 0x5604f226f440 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e4f36caebf06a7b server_cookie=d80b9ab46387bee in_seq=0 out_seq=0 2024-03-11T19:40:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.287+0000 7f2223fff640 10 client.25067 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.288+0000 7f2223fff640 1 -- 172.21.15.83:0/2823422937 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604f1eeb720 con 0x5604f226cfe0 2024-03-11T19:40:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.288+0000 7f2223fff640 10 client.25067 handle_command_reply: tid=0 2024-03-11T19:40:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.288+0000 7f224d670740 1 client.25067 shutdown 2024-03-11T19:40:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f224d670740 1 -- 172.21.15.83:0/2823422937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f22462d0 msgr2=0x5604f22610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f224d670740 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f22462d0 0x5604f22610e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2238009760 tx=0x7f223800eed0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f224d670740 1 -- 172.21.15.83:0/2823422937 shutdown_connections 2024-03-11T19:40:10.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f224d670740 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5604f226cfe0 0x5604f226f440 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f224d670740 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f224be30 0x5604f22685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f224d670740 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f2261620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f224d670740 1 --2- 172.21.15.83:0/2823422937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f22462d0 0x5604f22610e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f224d670740 1 -- 172.21.15.83:0/2823422937 >> 172.21.15.83:0/2823422937 conn(0x5604f22417f0 msgr2=0x5604f2240a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f2223fff640 0 client.25067 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f224d670740 1 -- 172.21.15.83:0/2823422937 shutdown_connections 2024-03-11T19:40:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f224d670740 1 -- 172.21.15.83:0/2823422937 wait complete. 2024-03-11T19:40:10.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.289+0000 7f224d670740 20 client.25067 trim_cache size 0 max 16384 2024-03-11T19:40:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f224d670740 1 Processor -- start 2024-03-11T19:40:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f224d670740 1 -- start start 2024-03-11T19:40:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f224d670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f2260cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f224d670740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f2246cc0 0x5604f22685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f224d670740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f224be30 0x5604f2261230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f224d670740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f2261e10 con 0x5604f22462d0 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f224d670740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f2247960 con 0x5604f2246cc0 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f224d670740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f2246a60 con 0x5604f224be30 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f224883f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f2260cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f224883f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f2260cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51528/0 (socket says 172.21.15.83:51528) 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f224883f640 1 -- 172.21.15.83:0/1584662623 learned_addr learned my addr 172.21.15.83:0/1584662623 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f2243fff640 1 --2- 172.21.15.83:0/1584662623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f2246cc0 0x5604f22685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.314+0000 7f2249040640 1 --2- 172.21.15.83:0/1584662623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f224be30 0x5604f2261230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.315+0000 7f224883f640 1 -- 172.21.15.83:0/1584662623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f224be30 msgr2=0x5604f2261230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.315+0000 7f224883f640 1 --2- 172.21.15.83:0/1584662623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f224be30 0x5604f2261230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.315+0000 7f224883f640 1 -- 172.21.15.83:0/1584662623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f2246cc0 msgr2=0x5604f22685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.315+0000 7f224883f640 1 --2- 172.21.15.83:0/1584662623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f2246cc0 0x5604f22685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.315+0000 7f224883f640 1 -- 172.21.15.83:0/1584662623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604f2149ac0 con 0x5604f22462d0 2024-03-11T19:40:10.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.315+0000 7f2249040640 1 --2- 172.21.15.83:0/1584662623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f224be30 0x5604f2261230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:40:10.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.315+0000 7f224883f640 1 --2- 172.21.15.83:0/1584662623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f2260cf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f223800e840 tx=0x7f223801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f28712cb3c25be server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.315+0000 7f222e7fc640 1 -- 172.21.15.83:0/1584662623 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238009e60 con 0x5604f22462d0 2024-03-11T19:40:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.315+0000 7f222e7fc640 1 -- 172.21.15.83:0/1584662623 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5604f1eae7e0 con 0x5604f22462d0 2024-03-11T19:40:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.315+0000 7f222e7fc640 1 -- 172.21.15.83:0/1584662623 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238004a00 con 0x5604f22462d0 2024-03-11T19:40:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.316+0000 7f224d670740 1 -- 172.21.15.83:0/1584662623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 msgr2=0x5604f2260cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.316+0000 7f224d670740 1 --2- 172.21.15.83:0/1584662623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f2260cf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f223800e840 tx=0x7f223801f000 comp rx=0 tx=0).stop 2024-03-11T19:40:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.316+0000 7f224d670740 1 -- 172.21.15.83:0/1584662623 shutdown_connections 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.316+0000 7f224d670740 1 --2- 172.21.15.83:0/1584662623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f224be30 0x5604f2261230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.316+0000 7f224d670740 1 --2- 172.21.15.83:0/1584662623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f2246cc0 0x5604f22685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.316+0000 7f224d670740 1 --2- 172.21.15.83:0/1584662623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f2260cf0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.316+0000 7f224d670740 1 -- 172.21.15.83:0/1584662623 >> 172.21.15.83:0/1584662623 conn(0x5604f22417f0 msgr2=0x5604f2247e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.316+0000 7f224d670740 1 -- 172.21.15.83:0/1584662623 shutdown_connections 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.316+0000 7f224d670740 1 -- 172.21.15.83:0/1584662623 wait complete. 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.317+0000 7f224d670740 1 -- start start 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.317+0000 7f224d670740 1 Processor -- start 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.317+0000 7f224d670740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f2267910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.317+0000 7f224d670740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f2267e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.317+0000 7f224d670740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 0x5604f22c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.317+0000 7f224d670740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f22c3e00 con 0x5604f22462d0 2024-03-11T19:40:10.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.317+0000 7f224d670740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f22c3f40 con 0x5604f224be30 2024-03-11T19:40:10.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.317+0000 7f224d670740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f22c40d0 con 0x5604f2246cc0 2024-03-11T19:40:10.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f224d670740 10 client.0 _ll_register_callbacks cb 0x7f2238009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:10.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2249040640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 0x5604f22c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2249040640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 0x5604f22c38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:44790/0 (socket says 172.21.15.83:44790) 2024-03-11T19:40:10.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2249040640 1 -- 172.21.15.83:0/619999017 learned_addr learned my addr 172.21.15.83:0/619999017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:10.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2243fff640 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f2267e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f224883f640 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f2267910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2249040640 1 -- 172.21.15.83:0/619999017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 msgr2=0x5604f2267e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2249040640 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f2267e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2249040640 1 -- 172.21.15.83:0/619999017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 msgr2=0x5604f2267910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2249040640 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f2267910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2249040640 1 -- 172.21.15.83:0/619999017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604f21ad610 con 0x5604f224be30 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2243fff640 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f2267e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2249040640 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 0x5604f22c38c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f223c02c700 tx=0x7f223c032fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=286f46472121524c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.318+0000 7f2223fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.319+0000 7f2223fff640 1 -- 172.21.15.83:0/619999017 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c045040 con 0x5604f224be30 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.319+0000 7f2223fff640 1 -- 172.21.15.83:0/619999017 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f223c034a90 con 0x5604f224be30 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.319+0000 7f224d670740 1 -- 172.21.15.83:0/619999017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2238021070 con 0x5604f224be30 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.319+0000 7f2223fff640 1 -- 172.21.15.83:0/619999017 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f223c034c50 con 0x5604f224be30 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.319+0000 7f224d670740 10 client.25073 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.319+0000 7f224d670740 1 -- 172.21.15.83:0/619999017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5604f216f6a0 con 0x5604f224be30 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.319+0000 7f2223fff640 1 -- 172.21.15.83:0/619999017 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2238021070 con 0x5604f224be30 2024-03-11T19:40:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.319+0000 7f2223fff640 1 -- 172.21.15.83:0/619999017 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f223c004470 con 0x5604f224be30 2024-03-11T19:40:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.319+0000 7f224d670740 10 client.25073 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.320+0000 7f224d670740 10 client.25073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.320+0000 7f224d670740 20 client.25073 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.320+0000 7f224d670740 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5604f22c5ba0 0x5604f22c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.320+0000 7f224d670740 4 client.25073 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:40:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.320+0000 7f224d670740 1 -- 172.21.15.83:0/619999017 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5604f1f3a890 con 0x5604f22c5ba0 2024-03-11T19:40:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.320+0000 7f224883f640 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5604f22c5ba0 0x5604f22c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.320+0000 7f2223fff640 10 client.25073 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.320+0000 7f224883f640 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5604f22c5ba0 0x5604f22c7fc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec9b6bf1ed646df server_cookie=42a907e343e6bb97 in_seq=0 out_seq=0 2024-03-11T19:40:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f2223fff640 1 -- 172.21.15.83:0/619999017 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5604f1f3a890 con 0x5604f22c5ba0 2024-03-11T19:40:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f2223fff640 10 client.25073 handle_command_reply: tid=0 2024-03-11T19:40:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f224d670740 1 client.25073 shutdown 2024-03-11T19:40:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f224d670740 1 -- 172.21.15.83:0/619999017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 msgr2=0x5604f22c38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f224d670740 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 0x5604f22c38c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f223c02c700 tx=0x7f223c032fa0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f224d670740 1 -- 172.21.15.83:0/619999017 shutdown_connections 2024-03-11T19:40:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f224d670740 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5604f22c5ba0 0x5604f22c7fc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f224d670740 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f224be30 0x5604f22c38c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f224d670740 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f2246cc0 0x5604f2267e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f224d670740 1 --2- 172.21.15.83:0/619999017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f22462d0 0x5604f2267910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f2223fff640 0 client.25073 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:10.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f224d670740 1 -- 172.21.15.83:0/619999017 >> 172.21.15.83:0/619999017 conn(0x5604f22417f0 msgr2=0x5604f2260cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f224d670740 1 -- 172.21.15.83:0/619999017 shutdown_connections 2024-03-11T19:40:10.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.321+0000 7f224d670740 1 -- 172.21.15.83:0/619999017 wait complete. 2024-03-11T19:40:10.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224d670740 20 client.25073 trim_cache size 0 max 16384 2024-03-11T19:40:10.332 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:40:10.332 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:40:10.332 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:40:10.332 INFO:teuthology.orchestra.run.smithi083.stdout: "d13b35fe-2f7c-491e-83d4-b205377d3c7b": { 2024-03-11T19:40:10.332 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:40:10.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "d13b35fe-2f7c-491e-83d4-b205377d3c7b", 2024-03-11T19:40:10.333 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:40:10.333 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:40:10.333 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:40:10.333 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224aaa3640 1 -- 172.21.15.83:0/2979743015 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22340a1ab0 msgr2=0x7f22340a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224aaa3640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22340a1ab0 0x7f22340a3f70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2244108910 tx=0x7f2238005f90 comp rx=0 tx=0).stop 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224aaa3640 1 -- 172.21.15.83:0/2979743015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22441012f0 msgr2=0x7f22441a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224aaa3640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22441012f0 0x7f22441a25a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f223000e9f0 tx=0x7f223000eeb0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224aaa3640 1 -- 172.21.15.83:0/2979743015 shutdown_connections 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224aaa3640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22340a1ab0 0x7f22340a3f70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224aaa3640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101cb0 0x7f22441a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224aaa3640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22441012f0 0x7f22441a25a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224aaa3640 1 --2- 172.21.15.83:0/2979743015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244100100 0x7f22441a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224aaa3640 1 -- 172.21.15.83:0/2979743015 >> 172.21.15.83:0/2979743015 conn(0x7f22440fb670 msgr2=0x7f224410d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.322+0000 7f224aaa3640 1 -- 172.21.15.83:0/2979743015 shutdown_connections 2024-03-11T19:40:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:10.323+0000 7f224aaa3640 1 -- 172.21.15.83:0/2979743015 wait complete. 2024-03-11T19:40:10.343 INFO:tasks.cephfs.filesystem:scrub status for tag:d13b35fe-2f7c-491e-83d4-b205377d3c7b - {'path': '/', 'tag': 'd13b35fe-2f7c-491e-83d4-b205377d3c7b', 'options': 'recursive,force'} 2024-03-11T19:40:10.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:10 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:40:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:10 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:10 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:40:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:10 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:40:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:10 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:10 smithi137 ceph-mon[36220]: pgmap v378: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 910 op/s 2024-03-11T19:40:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:10 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:40:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:10 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:10 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:40:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:10 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:40:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:10 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:10 smithi204 ceph-mon[36739]: pgmap v378: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 910 op/s 2024-03-11T19:40:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:10 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:40:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:10 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:10 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:40:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:10 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:40:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:10 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:10 smithi083 ceph-mon[31864]: pgmap v378: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 910 op/s 2024-03-11T19:40:11.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:11 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:40:11.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:11 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:11.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:11 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:40:11.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:11 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:11.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:11 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:11 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:40:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:11 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:11 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:40:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:11 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:11 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:12.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:11 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:40:12.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:11 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:12.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:11 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:40:12.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:11 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:12.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:11 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 55.0 s, 4950.1 tps, lat 20.184 ms stddev 20.931 2024-03-11T19:40:12.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:12 smithi204 ceph-mon[36739]: pgmap v379: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-03-11T19:40:13.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:12 smithi083 ceph-mon[31864]: pgmap v379: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-03-11T19:40:13.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:12 smithi137 ceph-mon[36220]: pgmap v379: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-03-11T19:40:15.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:15 smithi137 ceph-mon[36220]: pgmap v380: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-03-11T19:40:15.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:15 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:40:15.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:15 smithi204 ceph-mon[36739]: pgmap v380: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-03-11T19:40:15.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:15 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:40:16.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:15 smithi083 ceph-mon[31864]: pgmap v380: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-03-11T19:40:16.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:15 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:40:16.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:16 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:40:16.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:16 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:40:16.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:16 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:40:16.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:16 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:40:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:16 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:40:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:16 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:40:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 60.0 s, 5020.9 tps, lat 19.926 ms stddev 20.750 2024-03-11T19:40:17.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:17 smithi137 ceph-mon[36220]: pgmap v381: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 907 op/s 2024-03-11T19:40:17.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:17 smithi204 ceph-mon[36739]: pgmap v381: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 907 op/s 2024-03-11T19:40:18.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:17 smithi083 ceph-mon[31864]: pgmap v381: 97 pgs: 97 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 907 op/s 2024-03-11T19:40:19.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:19 smithi137 ceph-mon[36220]: pgmap v382: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 14 MiB/s wr, 997 op/s 2024-03-11T19:40:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:19 smithi204 ceph-mon[36739]: pgmap v382: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 14 MiB/s wr, 997 op/s 2024-03-11T19:40:20.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:19 smithi083 ceph-mon[31864]: pgmap v382: 97 pgs: 97 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 14 MiB/s wr, 997 op/s 2024-03-11T19:40:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:20 smithi083 ceph-mon[31864]: pgmap v383: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 11 MiB/s wr, 813 op/s 2024-03-11T19:40:21.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:20 smithi137 ceph-mon[36220]: pgmap v383: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 11 MiB/s wr, 813 op/s 2024-03-11T19:40:21.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:20 smithi204 ceph-mon[36739]: pgmap v383: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 11 MiB/s wr, 813 op/s 2024-03-11T19:40:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 65.0 s, 5512.4 tps, lat 18.117 ms stddev 21.262 2024-03-11T19:40:23.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:23 smithi137 ceph-mon[36220]: pgmap v384: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 12 MiB/s wr, 813 op/s 2024-03-11T19:40:23.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:23 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:40:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:23 smithi204 ceph-mon[36739]: pgmap v384: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 12 MiB/s wr, 813 op/s 2024-03-11T19:40:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:23 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:40:24.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:23 smithi083 ceph-mon[31864]: pgmap v384: 97 pgs: 97 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 12 MiB/s wr, 813 op/s 2024-03-11T19:40:24.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:23 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:40:24.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:24 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:40:24.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:24 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:40:24.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:24 smithi137 ceph-mon[36220]: pgmap v385: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 866 op/s 2024-03-11T19:40:24.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:24 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:40:24.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:24 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:40:24.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:24 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:40:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:24 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:40:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:24 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:40:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:24 smithi204 ceph-mon[36739]: pgmap v385: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 866 op/s 2024-03-11T19:40:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:24 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:40:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:24 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:40:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:24 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:40:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:40:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:40:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:24 smithi083 ceph-mon[31864]: pgmap v385: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 14 MiB/s wr, 866 op/s 2024-03-11T19:40:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:40:25.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:40:25.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:24 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:40:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 70.0 s, 4204.6 tps, lat 23.810 ms stddev 24.890 2024-03-11T19:40:27.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:27 smithi137 ceph-mon[36220]: pgmap v386: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 851 op/s 2024-03-11T19:40:27.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:27 smithi204 ceph-mon[36739]: pgmap v386: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 851 op/s 2024-03-11T19:40:28.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:27 smithi083 ceph-mon[31864]: pgmap v386: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 14 MiB/s wr, 851 op/s 2024-03-11T19:40:29.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:29 smithi137 ceph-mon[36220]: pgmap v387: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 15 MiB/s wr, 951 op/s 2024-03-11T19:40:29.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:29 smithi204 ceph-mon[36739]: pgmap v387: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 15 MiB/s wr, 951 op/s 2024-03-11T19:40:30.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:29 smithi083 ceph-mon[31864]: pgmap v387: 97 pgs: 97 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 15 MiB/s wr, 951 op/s 2024-03-11T19:40:31.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:31 smithi137 ceph-mon[36220]: pgmap v388: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 780 op/s 2024-03-11T19:40:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:31 smithi204 ceph-mon[36739]: pgmap v388: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 780 op/s 2024-03-11T19:40:32.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:31 smithi083 ceph-mon[31864]: pgmap v388: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 12 MiB/s wr, 780 op/s 2024-03-11T19:40:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 75.0 s, 4868.8 tps, lat 20.540 ms stddev 21.098 2024-03-11T19:40:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:32 smithi083 ceph-mon[31864]: pgmap v389: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 868 op/s 2024-03-11T19:40:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:32 smithi137 ceph-mon[36220]: pgmap v389: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 868 op/s 2024-03-11T19:40:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:32 smithi204 ceph-mon[36739]: pgmap v389: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 868 op/s 2024-03-11T19:40:35.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:35 smithi137 ceph-mon[36220]: pgmap v390: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 951 op/s 2024-03-11T19:40:35.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:35 smithi204 ceph-mon[36739]: pgmap v390: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 951 op/s 2024-03-11T19:40:36.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:35 smithi083 ceph-mon[31864]: pgmap v390: 97 pgs: 97 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 13 MiB/s wr, 951 op/s 2024-03-11T19:40:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 80.0 s, 4897.4 tps, lat 20.411 ms stddev 23.067 2024-03-11T19:40:37.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:37 smithi137 ceph-mon[36220]: pgmap v391: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 911 op/s 2024-03-11T19:40:37.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:37 smithi204 ceph-mon[36739]: pgmap v391: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 911 op/s 2024-03-11T19:40:38.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:37 smithi083 ceph-mon[31864]: pgmap v391: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 12 MiB/s wr, 911 op/s 2024-03-11T19:40:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:38 smithi083 ceph-mon[31864]: pgmap v392: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 897 op/s 2024-03-11T19:40:39.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:38 smithi137 ceph-mon[36220]: pgmap v392: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 897 op/s 2024-03-11T19:40:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:38 smithi204 ceph-mon[36739]: pgmap v392: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 897 op/s 2024-03-11T19:40:40.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:40:40.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:40:40.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.499+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/3619818766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a78101c30 msgr2=0x7f8a781084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:40.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.499+0000 7f8a7e0c0640 1 --2- 172.21.15.83:0/3619818766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a78101c30 0x7f8a781084c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a70009be0 tx=0x7f8a7002de80 comp rx=0 tx=0).stop 2024-03-11T19:40:40.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.499+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/3619818766 shutdown_connections 2024-03-11T19:40:40.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.499+0000 7f8a7e0c0640 1 --2- 172.21.15.83:0/3619818766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a78101c30 0x7f8a781084c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.499+0000 7f8a7e0c0640 1 --2- 172.21.15.83:0/3619818766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a78101270 0x7f8a781016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.499+0000 7f8a7e0c0640 1 --2- 172.21.15.83:0/3619818766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a78100080 0x7f8a781004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.499+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/3619818766 >> 172.21.15.83:0/3619818766 conn(0x7f8a780fb5f0 msgr2=0x7f8a780fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:40.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.499+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/3619818766 shutdown_connections 2024-03-11T19:40:40.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.499+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/3619818766 wait complete. 2024-03-11T19:40:40.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.500+0000 7f8a7e0c0640 1 Processor -- start 2024-03-11T19:40:40.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.500+0000 7f8a7e0c0640 1 -- start start 2024-03-11T19:40:40.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.500+0000 7f8a7e0c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a78100080 0x7f8a781993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.500+0000 7f8a7e0c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a78101270 0x7f8a781998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.500+0000 7f8a7e0c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a78101c30 0x7f8a781a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.500+0000 7f8a7e0c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a781a0ea0 con 0x7f8a78100080 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.500+0000 7f8a7e0c0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a781a1010 con 0x7f8a78101270 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.500+0000 7f8a7e0c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a781a11d0 con 0x7f8a78101c30 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.500+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a78101c30 0x7f8a781a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.501+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a78101c30 0x7f8a781a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56644/0 (socket says 172.21.15.83:56644) 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.501+0000 7f8a777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a78100080 0x7f8a781993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.501+0000 7f8a77fff640 1 -- 172.21.15.83:0/1892732892 learned_addr learned my addr 172.21.15.83:0/1892732892 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.501+0000 7f8a77fff640 1 -- 172.21.15.83:0/1892732892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a78101270 msgr2=0x7f8a781998f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.501+0000 7f8a77fff640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a78101270 0x7f8a781998f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.501+0000 7f8a77fff640 1 -- 172.21.15.83:0/1892732892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a78100080 msgr2=0x7f8a781993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.501+0000 7f8a77fff640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a78100080 0x7f8a781993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.501+0000 7f8a77fff640 1 -- 172.21.15.83:0/1892732892 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a70009840 con 0x7f8a78101c30 2024-03-11T19:40:40.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.502+0000 7f8a77fff640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a78101c30 0x7f8a781a0960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8a7002e3c0 tx=0x7f8a7002eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:40.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.503+0000 7f8a74ff9640 1 -- 172.21.15.83:0/1892732892 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70003970 con 0x7f8a78101c30 2024-03-11T19:40:40.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.503+0000 7f8a74ff9640 1 -- 172.21.15.83:0/1892732892 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a70004040 con 0x7f8a78101c30 2024-03-11T19:40:40.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.503+0000 7f8a74ff9640 1 -- 172.21.15.83:0/1892732892 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70036790 con 0x7f8a78101c30 2024-03-11T19:40:40.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.504+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/1892732892 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a781a1450 con 0x7f8a78101c30 2024-03-11T19:40:40.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.504+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/1892732892 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a781a18b0 con 0x7f8a78101c30 2024-03-11T19:40:40.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.507+0000 7f8a74ff9640 1 -- 172.21.15.83:0/1892732892 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8a700041b0 con 0x7f8a78101c30 2024-03-11T19:40:40.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.509+0000 7f8a80d6e740 1 Processor -- start 2024-03-11T19:40:40.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.509+0000 7f8a80d6e740 1 -- start start 2024-03-11T19:40:40.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.509+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 0x557c1d0cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.509+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cdcc0 0x557c1d0d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.509+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0d2e30 0x557c1d0d52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.509+0000 7f8a80d6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d03d280 con 0x557c1d0d2e30 2024-03-11T19:40:40.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.509+0000 7f8a80d6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d03a7b0 con 0x557c1d0cd2d0 2024-03-11T19:40:40.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.509+0000 7f8a80d6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d035f90 con 0x557c1d0cdcc0 2024-03-11T19:40:40.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.510+0000 7f8a76ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 0x557c1d0cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.510+0000 7f8a76ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 0x557c1d0cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40352/0 (socket says 172.21.15.83:40352) 2024-03-11T19:40:40.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.510+0000 7f8a76ffd640 1 -- 172.21.15.83:0/782252154 learned_addr learned my addr 172.21.15.83:0/782252154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:40.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.511+0000 7f8a777fe640 1 --2- 172.21.15.83:0/782252154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cdcc0 0x557c1d0d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.511+0000 7f8a76ffd640 1 --2- 172.21.15.83:0/782252154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0d2e30 0x557c1d0d52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.511+0000 7f8a777fe640 1 -- 172.21.15.83:0/782252154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 msgr2=0x557c1d0cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:40.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.511+0000 7f8a777fe640 1 --2- 172.21.15.83:0/782252154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 0x557c1d0cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.512+0000 7f8a777fe640 1 -- 172.21.15.83:0/782252154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0d2e30 msgr2=0x557c1d0d52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:40.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.512+0000 7f8a777fe640 1 --2- 172.21.15.83:0/782252154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0d2e30 0x557c1d0d52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.512+0000 7f8a777fe640 1 -- 172.21.15.83:0/782252154 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c1cff4570 con 0x557c1d0cdcc0 2024-03-11T19:40:40.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.512+0000 7f8a76ffd640 1 --2- 172.21.15.83:0/782252154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 0x557c1d0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:40.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.512+0000 7f8a74ff9640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8a600a1c00 0x7f8a600a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.512+0000 7f8a76ffd640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8a600a1c00 0x7f8a600a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.512+0000 7f8a777fe640 1 --2- 172.21.15.83:0/782252154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cdcc0 0x557c1d0d27c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a781010d0 tx=0x7f8a6c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=815d899742745ae3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:40.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.512+0000 7f8a74ff9640 1 -- 172.21.15.83:0/1892732892 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8a700efe50 con 0x7f8a78101c30 2024-03-11T19:40:40.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.512+0000 7f8a5dffb640 1 -- 172.21.15.83:0/782252154 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c003dd0 con 0x557c1d0cdcc0 2024-03-11T19:40:40.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.512+0000 7f8a5dffb640 1 -- 172.21.15.83:0/782252154 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a6c00be60 con 0x557c1d0cdcc0 2024-03-11T19:40:40.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.513+0000 7f8a80d6e740 1 -- 172.21.15.83:0/782252154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cdcc0 msgr2=0x557c1d0d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:40.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.513+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/782252154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cdcc0 0x557c1d0d27c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8a781010d0 tx=0x7f8a6c004610 comp rx=0 tx=0).stop 2024-03-11T19:40:40.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.513+0000 7f8a76ffd640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8a600a1c00 0x7f8a600a40c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x557c1d0ce520 tx=0x7f8a68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:40.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.514+0000 7f8a80d6e740 1 -- 172.21.15.83:0/782252154 shutdown_connections 2024-03-11T19:40:40.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.514+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/782252154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0d2e30 0x557c1d0d52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.514+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/782252154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cdcc0 0x557c1d0d27c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.514+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/782252154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 0x557c1d0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.514+0000 7f8a80d6e740 1 -- 172.21.15.83:0/782252154 >> 172.21.15.83:0/782252154 conn(0x557c1d0c87f0 msgr2=0x557c1d0c8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.514+0000 7f8a80d6e740 1 -- 172.21.15.83:0/782252154 shutdown_connections 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.514+0000 7f8a80d6e740 1 -- 172.21.15.83:0/782252154 wait complete. 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.514+0000 7f8a80d6e740 1 -- start start 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.514+0000 7f8a80d6e740 1 Processor -- start 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 0x557c1d0dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0d2e30 0x557c1d0e7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a777fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 0x557c1d0dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a777fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 0x557c1d0dfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40364/0 (socket says 172.21.15.83:40364) 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0d2e30 0x557c1d0e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0d2e30 0x557c1d0e7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56666/0 (socket says 172.21.15.83:56666) 2024-03-11T19:40:40.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a777fe640 1 -- 172.21.15.83:0/81732293 learned_addr learned my addr 172.21.15.83:0/81732293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:40.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a80d6e740 1 -- 172.21.15.83:0/81732293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cd09430 con 0x557c1d0cdcc0 2024-03-11T19:40:40.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a80d6e740 1 -- 172.21.15.83:0/81732293 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1ccc5e00 con 0x557c1d0cd2d0 2024-03-11T19:40:40.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a80d6e740 1 -- 172.21.15.83:0/81732293 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1cfbafd0 con 0x557c1d0d2e30 2024-03-11T19:40:40.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.515+0000 7f8a80d6e740 10 client.0 _ll_register_callbacks cb 0x557c1d010fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:40.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.516+0000 7f8a76ffd640 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.516+0000 7f8a77fff640 1 -- 172.21.15.83:0/81732293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 msgr2=0x557c1d0dfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.516+0000 7f8a77fff640 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 0x557c1d0dfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.516+0000 7f8a77fff640 1 -- 172.21.15.83:0/81732293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 msgr2=0x557c1d0e0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.516+0000 7f8a77fff640 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.516+0000 7f8a77fff640 1 -- 172.21.15.83:0/81732293 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c004270 con 0x557c1d0d2e30 2024-03-11T19:40:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.516+0000 7f8a76ffd640 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:40:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.516+0000 7f8a77fff640 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0d2e30 0x557c1d0e7370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a700ed960 tx=0x7f8a700bca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=553ddcef4bf51f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.516+0000 7f8a56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:40:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.516+0000 7f8a56ffd640 1 -- 172.21.15.83:0/81732293 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700c0f20 con 0x557c1d0d2e30 2024-03-11T19:40:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.517+0000 7f8a80d6e740 1 -- 172.21.15.83:0/81732293 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c1cd72720 con 0x557c1d0d2e30 2024-03-11T19:40:40.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.517+0000 7f8a56ffd640 1 -- 172.21.15.83:0/81732293 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8a700c5570 con 0x557c1d0d2e30 2024-03-11T19:40:40.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.517+0000 7f8a56ffd640 1 -- 172.21.15.83:0/81732293 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a700b9440 con 0x557c1d0d2e30 2024-03-11T19:40:40.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.517+0000 7f8a80d6e740 10 client.25192 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:40.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.517+0000 7f8a80d6e740 1 -- 172.21.15.83:0/81732293 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557c1d034610 con 0x557c1d0d2e30 2024-03-11T19:40:40.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.517+0000 7f8a56ffd640 1 -- 172.21.15.83:0/81732293 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a700d9030 con 0x557c1d0d2e30 2024-03-11T19:40:40.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.518+0000 7f8a56ffd640 1 -- 172.21.15.83:0/81732293 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a700bc220 con 0x557c1d0d2e30 2024-03-11T19:40:40.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.518+0000 7f8a80d6e740 10 client.25192 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:40.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.518+0000 7f8a80d6e740 10 client.25192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:40.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.518+0000 7f8a80d6e740 20 client.25192 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:40.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.518+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557c1d0ebe00 0x557c1d0ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.518+0000 7f8a80d6e740 4 client.25192 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:40.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.518+0000 7f8a80d6e740 1 -- 172.21.15.83:0/81732293 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c1cdc1890 con 0x557c1d0ebe00 2024-03-11T19:40:40.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.518+0000 7f8a777fe640 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557c1d0ebe00 0x557c1d0ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.519+0000 7f8a777fe640 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557c1d0ebe00 0x557c1d0ee1f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baabc9ab5ddddabe server_cookie=e8d365bafa2dda03 in_seq=0 out_seq=0 2024-03-11T19:40:40.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.519+0000 7f8a56ffd640 10 client.25192 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:40.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.520+0000 7f8a56ffd640 1 -- 172.21.15.83:0/81732293 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c1cdc1890 con 0x557c1d0ebe00 2024-03-11T19:40:40.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.520+0000 7f8a56ffd640 10 client.25192 handle_command_reply: tid=0 2024-03-11T19:40:40.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.520+0000 7f8a80d6e740 1 client.25192 shutdown 2024-03-11T19:40:40.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.521+0000 7f8a80d6e740 1 -- 172.21.15.83:0/81732293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0d2e30 msgr2=0x557c1d0e7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:40.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.521+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0d2e30 0x557c1d0e7370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a700ed960 tx=0x7f8a700bca00 comp rx=0 tx=0).stop 2024-03-11T19:40:40.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.521+0000 7f8a80d6e740 1 -- 172.21.15.83:0/81732293 shutdown_connections 2024-03-11T19:40:40.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.521+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557c1d0ebe00 0x557c1d0ee1f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.521+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0d2e30 0x557c1d0e7370 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.521+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.521+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/81732293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0cd2d0 0x557c1d0dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.521+0000 7f8a80d6e740 1 -- 172.21.15.83:0/81732293 >> 172.21.15.83:0/81732293 conn(0x557c1d0c87f0 msgr2=0x557c1d0c7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:40.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.522+0000 7f8a56ffd640 0 client.25192 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:40.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.522+0000 7f8a80d6e740 1 -- 172.21.15.83:0/81732293 shutdown_connections 2024-03-11T19:40:40.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.522+0000 7f8a80d6e740 1 -- 172.21.15.83:0/81732293 wait complete. 2024-03-11T19:40:40.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.522+0000 7f8a80d6e740 20 client.25192 trim_cache size 0 max 16384 2024-03-11T19:40:40.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.552+0000 7f8a80d6e740 1 Processor -- start 2024-03-11T19:40:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.553+0000 7f8a80d6e740 1 -- start start 2024-03-11T19:40:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.553+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cd2d0 0x557c1d0f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.553+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.553+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 0x557c1d0dfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.553+0000 7f8a80d6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d0e7ff0 con 0x557c1d0cdcc0 2024-03-11T19:40:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.553+0000 7f8a80d6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d0e0cf0 con 0x557c1d0d2e30 2024-03-11T19:40:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.553+0000 7f8a80d6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d0c9240 con 0x557c1d0cd2d0 2024-03-11T19:40:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.553+0000 7f8a777fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cd2d0 0x557c1d0f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.553+0000 7f8a777fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cd2d0 0x557c1d0f2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56678/0 (socket says 172.21.15.83:56678) 2024-03-11T19:40:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.553+0000 7f8a777fe640 1 -- 172.21.15.83:0/2784785090 learned_addr learned my addr 172.21.15.83:0/2784785090 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.554+0000 7f8a76ffd640 1 --2- 172.21.15.83:0/2784785090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.553+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 0x557c1d0dfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.554+0000 7f8a77fff640 1 -- 172.21.15.83:0/2784785090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cd2d0 msgr2=0x557c1d0f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.554+0000 7f8a77fff640 1 --2- 172.21.15.83:0/2784785090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cd2d0 0x557c1d0f2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.554+0000 7f8a77fff640 1 -- 172.21.15.83:0/2784785090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 msgr2=0x557c1d0e7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.554+0000 7f8a77fff640 1 --2- 172.21.15.83:0/2784785090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.554+0000 7f8a77fff640 1 -- 172.21.15.83:0/2784785090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c1cfd0ac0 con 0x557c1d0d2e30 2024-03-11T19:40:40.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.554+0000 7f8a77fff640 1 --2- 172.21.15.83:0/2784785090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 0x557c1d0dfdb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a700c5040 tx=0x7f8a700bcf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c19c13f0d56e0c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:40.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.555+0000 7f8a5d7fa640 1 -- 172.21.15.83:0/2784785090 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700bd4d0 con 0x557c1d0d2e30 2024-03-11T19:40:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.555+0000 7f8a5d7fa640 1 -- 172.21.15.83:0/2784785090 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a700d9070 con 0x557c1d0d2e30 2024-03-11T19:40:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.555+0000 7f8a5d7fa640 1 -- 172.21.15.83:0/2784785090 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700bc330 con 0x557c1d0d2e30 2024-03-11T19:40:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.556+0000 7f8a80d6e740 1 -- 172.21.15.83:0/2784785090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 msgr2=0x557c1d0dfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.556+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/2784785090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 0x557c1d0dfdb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a700c5040 tx=0x7f8a700bcf50 comp rx=0 tx=0).stop 2024-03-11T19:40:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.556+0000 7f8a80d6e740 1 -- 172.21.15.83:0/2784785090 shutdown_connections 2024-03-11T19:40:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.556+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/2784785090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 0x557c1d0dfdb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.556+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/2784785090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.556+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/2784785090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cd2d0 0x557c1d0f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.556+0000 7f8a80d6e740 1 -- 172.21.15.83:0/2784785090 >> 172.21.15.83:0/2784785090 conn(0x557c1d0c87f0 msgr2=0x557c1d0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.557+0000 7f8a80d6e740 1 -- 172.21.15.83:0/2784785090 shutdown_connections 2024-03-11T19:40:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.557+0000 7f8a80d6e740 1 -- 172.21.15.83:0/2784785090 wait complete. 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.557+0000 7f8a80d6e740 1 -- start start 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.558+0000 7f8a80d6e740 1 Processor -- start 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.558+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cd2d0 0x557c1d0e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.558+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.558+0000 7f8a80d6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 0x557c1d0e5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.558+0000 7f8a80d6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d0ded90 con 0x557c1d0cdcc0 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.558+0000 7f8a80d6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d0d5450 con 0x557c1d0d2e30 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.558+0000 7f8a80d6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c1d0e57d0 con 0x557c1d0cd2d0 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.558+0000 7f8a80d6e740 10 client.0 _ll_register_callbacks cb 0x7f8a6c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.559+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 0x557c1d0e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.559+0000 7f8a77fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 0x557c1d0e5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40374/0 (socket says 172.21.15.83:40374) 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.559+0000 7f8a77fff640 1 -- 172.21.15.83:0/3949855449 learned_addr learned my addr 172.21.15.83:0/3949855449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.559+0000 7f8a76ffd640 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.559+0000 7f8a777fe640 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cd2d0 0x557c1d0e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.559+0000 7f8a77fff640 1 -- 172.21.15.83:0/3949855449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cd2d0 msgr2=0x557c1d0e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.559+0000 7f8a77fff640 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cd2d0 0x557c1d0e6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.559+0000 7f8a77fff640 1 -- 172.21.15.83:0/3949855449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 msgr2=0x557c1d0e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.559+0000 7f8a77fff640 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.559+0000 7f8a77fff640 1 -- 172.21.15.83:0/3949855449 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c004270 con 0x557c1d0d2e30 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.560+0000 7f8a77fff640 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 0x557c1d0e5210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8a700ed9b0 tx=0x7f8a700bca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f6609c1ea7ed297 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.560+0000 7f8a56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.560+0000 7f8a56ffd640 1 -- 172.21.15.83:0/3949855449 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700c5570 con 0x557c1d0d2e30 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.560+0000 7f8a80d6e740 1 -- 172.21.15.83:0/3949855449 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c1cd357e0 con 0x557c1d0d2e30 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.560+0000 7f8a56ffd640 1 -- 172.21.15.83:0/3949855449 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8a700bd4d0 con 0x557c1d0d2e30 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.560+0000 7f8a56ffd640 1 -- 172.21.15.83:0/3949855449 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a70004b40 con 0x557c1d0d2e30 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.560+0000 7f8a80d6e740 10 client.25088 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.560+0000 7f8a80d6e740 1 -- 172.21.15.83:0/3949855449 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557c1cff66a0 con 0x557c1d0d2e30 2024-03-11T19:40:40.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.561+0000 7f8a56ffd640 1 -- 172.21.15.83:0/3949855449 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557c1cd357e0 con 0x557c1d0d2e30 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.561+0000 7f8a56ffd640 1 -- 172.21.15.83:0/3949855449 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8a700368f0 con 0x557c1d0d2e30 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.561+0000 7f8a80d6e740 10 client.25088 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.561+0000 7f8a80d6e740 10 client.25088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.561+0000 7f8a80d6e740 20 client.25088 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.561+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557c1d142d70 0x557c1d145160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.561+0000 7f8a80d6e740 4 client.25088 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.561+0000 7f8a80d6e740 1 -- 172.21.15.83:0/3949855449 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557c1cd244e0 con 0x557c1d142d70 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.561+0000 7f8a777fe640 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557c1d142d70 0x557c1d145160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.562+0000 7f8a777fe640 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557c1d142d70 0x557c1d145160 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4039081bc243d4 server_cookie=16bb202853b8fe in_seq=0 out_seq=0 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.562+0000 7f8a56ffd640 10 client.25088 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.562+0000 7f8a56ffd640 1 -- 172.21.15.83:0/3949855449 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557c1cd244e0 con 0x557c1d142d70 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.562+0000 7f8a56ffd640 10 client.25088 handle_command_reply: tid=0 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.562+0000 7f8a80d6e740 1 client.25088 shutdown 2024-03-11T19:40:40.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.562+0000 7f8a80d6e740 1 -- 172.21.15.83:0/3949855449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 msgr2=0x557c1d0e5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.562+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 0x557c1d0e5210 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8a700ed9b0 tx=0x7f8a700bca00 comp rx=0 tx=0).stop 2024-03-11T19:40:40.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.563+0000 7f8a80d6e740 1 -- 172.21.15.83:0/3949855449 shutdown_connections 2024-03-11T19:40:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.563+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557c1d142d70 0x557c1d145160 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.563+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557c1d0d2e30 0x557c1d0e5210 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.563+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c1d0cdcc0 0x557c1d0e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.563+0000 7f8a80d6e740 1 --2- 172.21.15.83:0/3949855449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c1d0cd2d0 0x557c1d0e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.563+0000 7f8a56ffd640 0 client.25088 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.563+0000 7f8a80d6e740 1 -- 172.21.15.83:0/3949855449 >> 172.21.15.83:0/3949855449 conn(0x557c1d0c87f0 msgr2=0x557c1d0f2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.563+0000 7f8a80d6e740 1 -- 172.21.15.83:0/3949855449 shutdown_connections 2024-03-11T19:40:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.563+0000 7f8a80d6e740 1 -- 172.21.15.83:0/3949855449 wait complete. 2024-03-11T19:40:40.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.564+0000 7f8a80d6e740 20 client.25088 trim_cache size 0 max 16384 2024-03-11T19:40:40.573 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:40:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:40:40.574 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:40:40.574 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.564+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/1892732892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8a600a1c00 msgr2=0x7f8a600a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.564+0000 7f8a7e0c0640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8a600a1c00 0x7f8a600a40c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x557c1d0ce520 tx=0x7f8a68005f90 comp rx=0 tx=0).stop 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.564+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/1892732892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a78101c30 msgr2=0x7f8a781a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.564+0000 7f8a7e0c0640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a78101c30 0x7f8a781a0960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8a7002e3c0 tx=0x7f8a7002eac0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.565+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/1892732892 shutdown_connections 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.565+0000 7f8a7e0c0640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8a600a1c00 0x7f8a600a40c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.565+0000 7f8a7e0c0640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a78101c30 0x7f8a781a0960 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.565+0000 7f8a7e0c0640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a78101270 0x7f8a781998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.565+0000 7f8a7e0c0640 1 --2- 172.21.15.83:0/1892732892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a78100080 0x7f8a781993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.565+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/1892732892 >> 172.21.15.83:0/1892732892 conn(0x7f8a780fb5f0 msgr2=0x7f8a78104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.566+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/1892732892 shutdown_connections 2024-03-11T19:40:40.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.566+0000 7f8a7e0c0640 1 -- 172.21.15.83:0/1892732892 wait complete. 2024-03-11T19:40:40.589 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:40:40.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:40:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.764+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/4105196712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e84102d70 msgr2=0x7f0e84109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.764+0000 7f0e8aa0e640 1 --2- 172.21.15.83:0/4105196712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e84102d70 0x7f0e84109600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009ca0 tx=0x7f0e7c02c100 comp rx=0 tx=0).stop 2024-03-11T19:40:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.764+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/4105196712 shutdown_connections 2024-03-11T19:40:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.764+0000 7f0e8aa0e640 1 --2- 172.21.15.83:0/4105196712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e84102d70 0x7f0e84109600 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.764+0000 7f0e8aa0e640 1 --2- 172.21.15.83:0/4105196712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e841023b0 0x7f0e84102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.764+0000 7f0e8aa0e640 1 --2- 172.21.15.83:0/4105196712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e841011c0 0x7f0e841015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.764+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/4105196712 >> 172.21.15.83:0/4105196712 conn(0x7f0e840fc730 msgr2=0x7f0e840feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:40.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.765+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/4105196712 shutdown_connections 2024-03-11T19:40:40.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.765+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/4105196712 wait complete. 2024-03-11T19:40:40.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.766+0000 7f0e8aa0e640 1 Processor -- start 2024-03-11T19:40:40.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.766+0000 7f0e8aa0e640 1 -- start start 2024-03-11T19:40:40.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.766+0000 7f0e8aa0e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e841011c0 0x7f0e84194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.766+0000 7f0e8aa0e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e841023b0 0x7f0e8419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.766+0000 7f0e8aa0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e84102d70 0x7f0e84195420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.766+0000 7f0e8aa0e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e841959f0 con 0x7f0e84102d70 2024-03-11T19:40:40.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.766+0000 7f0e8aa0e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84195b60 con 0x7f0e841011c0 2024-03-11T19:40:40.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.766+0000 7f0e8aa0e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e841add10 con 0x7f0e841023b0 2024-03-11T19:40:40.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e88fab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e84102d70 0x7f0e84195420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e88fab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e84102d70 0x7f0e84195420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45320/0 (socket says 172.21.15.83:45320) 2024-03-11T19:40:40.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e88fab640 1 -- 172.21.15.83:0/2352227085 learned_addr learned my addr 172.21.15.83:0/2352227085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e837fe640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e841023b0 0x7f0e8419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e83fff640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e841011c0 0x7f0e84194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e837fe640 1 -- 172.21.15.83:0/2352227085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e841011c0 msgr2=0x7f0e84194ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e837fe640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e841011c0 0x7f0e84194ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e837fe640 1 -- 172.21.15.83:0/2352227085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e84102d70 msgr2=0x7f0e84195420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e837fe640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e84102d70 0x7f0e84195420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e837fe640 1 -- 172.21.15.83:0/2352227085 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c009900 con 0x7f0e841023b0 2024-03-11T19:40:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e88fab640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e84102d70 0x7f0e84195420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:40.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.767+0000 7f0e837fe640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e841023b0 0x7f0e8419a480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800e9f0 tx=0x7f0e7800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.768+0000 7f0e817fa640 1 -- 172.21.15.83:0/2352227085 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78009640 con 0x7f0e841023b0 2024-03-11T19:40:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.768+0000 7f0e817fa640 1 -- 172.21.15.83:0/2352227085 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e7800ccc0 con 0x7f0e841023b0 2024-03-11T19:40:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.768+0000 7f0e817fa640 1 -- 172.21.15.83:0/2352227085 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78018750 con 0x7f0e841023b0 2024-03-11T19:40:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.768+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/2352227085 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e841adff0 con 0x7f0e841023b0 2024-03-11T19:40:40.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.768+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/2352227085 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e841ae420 con 0x7f0e841023b0 2024-03-11T19:40:40.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.772+0000 7f0e817fa640 1 -- 172.21.15.83:0/2352227085 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0e780109a0 con 0x7f0e841023b0 2024-03-11T19:40:40.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.772+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/2352227085 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e84106400 con 0x7f0e841023b0 2024-03-11T19:40:40.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.772+0000 7f0e817fa640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0e6c0a1b80 0x7f0e6c0a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:40.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.773+0000 7f0e817fa640 1 -- 172.21.15.83:0/2352227085 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0e78014070 con 0x7f0e841023b0 2024-03-11T19:40:40.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.773+0000 7f0e83fff640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0e6c0a1b80 0x7f0e6c0a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:40.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.774+0000 7f0e83fff640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0e6c0a1b80 0x7f0e6c0a4040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0e84102210 tx=0x7f0e74009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:40.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:40.779+0000 7f0e817fa640 1 -- 172.21.15.83:0/2352227085 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0e78052d20 con 0x7f0e841023b0 2024-03-11T19:40:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.206+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/2352227085 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e8402ce40 con 0x7f0e841023b0 2024-03-11T19:40:41.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.207+0000 7f0e817fa640 1 -- 172.21.15.83:0/2352227085 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11352 (secure 0 0 0) 0x7f0e78096830 con 0x7f0e841023b0 2024-03-11T19:40:41.213 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:40:41.214 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:39:20.446302+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:40:41.214 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-03-11T19:40:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.211+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/2352227085 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0e6c0a1b80 msgr2=0x7f0e6c0a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.211+0000 7f0e8aa0e640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0e6c0a1b80 0x7f0e6c0a4040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0e84102210 tx=0x7f0e74009110 comp rx=0 tx=0).stop 2024-03-11T19:40:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.211+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/2352227085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e841023b0 msgr2=0x7f0e8419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.211+0000 7f0e8aa0e640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e841023b0 0x7f0e8419a480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800e9f0 tx=0x7f0e7800edf0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.212+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/2352227085 shutdown_connections 2024-03-11T19:40:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.212+0000 7f0e8aa0e640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0e6c0a1b80 0x7f0e6c0a4040 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.212+0000 7f0e8aa0e640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e84102d70 0x7f0e84195420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.212+0000 7f0e8aa0e640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e841023b0 0x7f0e8419a480 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.212+0000 7f0e8aa0e640 1 --2- 172.21.15.83:0/2352227085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e841011c0 0x7f0e84194ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.212+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/2352227085 >> 172.21.15.83:0/2352227085 conn(0x7f0e840fc730 msgr2=0x7f0e841035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.212+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/2352227085 shutdown_connections 2024-03-11T19:40:41.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.212+0000 7f0e8aa0e640 1 -- 172.21.15.83:0/2352227085 wait complete. 2024-03-11T19:40:41.236 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:40:41.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:40:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.421+0000 7f092f5bc640 1 -- 172.21.15.83:0/3109393718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09280ff9f0 msgr2=0x7f0928101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.421+0000 7f092f5bc640 1 --2- 172.21.15.83:0/3109393718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09280ff9f0 0x7f0928101e20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0918009900 tx=0x7f091802bc50 comp rx=0 tx=0).stop 2024-03-11T19:40:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.421+0000 7f092f5bc640 1 -- 172.21.15.83:0/3109393718 shutdown_connections 2024-03-11T19:40:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.421+0000 7f092f5bc640 1 --2- 172.21.15.83:0/3109393718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0928104d10 0x7f0928105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.421+0000 7f092f5bc640 1 --2- 172.21.15.83:0/3109393718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0928102360 0x7f09281047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.421+0000 7f092f5bc640 1 --2- 172.21.15.83:0/3109393718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09280ff9f0 0x7f0928101e20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.421+0000 7f092f5bc640 1 -- 172.21.15.83:0/3109393718 >> 172.21.15.83:0/3109393718 conn(0x7f09280fb610 msgr2=0x7f09280fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.421+0000 7f092f5bc640 1 -- 172.21.15.83:0/3109393718 shutdown_connections 2024-03-11T19:40:41.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.422+0000 7f092f5bc640 1 -- 172.21.15.83:0/3109393718 wait complete. 2024-03-11T19:40:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092f5bc640 1 Processor -- start 2024-03-11T19:40:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092f5bc640 1 -- start start 2024-03-11T19:40:41.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092f5bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09280ff9f0 0x7f092819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092f5bc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0928102360 0x7f092819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092f5bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0928104d10 0x7f09281a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092f5bc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092806aba0 con 0x7f0928104d10 2024-03-11T19:40:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092f5bc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092806ad10 con 0x7f09280ff9f0 2024-03-11T19:40:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092f5bc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092806aed0 con 0x7f0928102360 2024-03-11T19:40:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092db59640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0928104d10 0x7f09281a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092db59640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0928104d10 0x7f09281a4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45334/0 (socket says 172.21.15.83:45334) 2024-03-11T19:40:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092db59640 1 -- 172.21.15.83:0/1164210544 learned_addr learned my addr 172.21.15.83:0/1164210544 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.423+0000 7f092d358640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09280ff9f0 0x7f092819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.424+0000 7f092cb57640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0928102360 0x7f092819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.424+0000 7f092db59640 1 -- 172.21.15.83:0/1164210544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0928102360 msgr2=0x7f092819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.424+0000 7f092db59640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0928102360 0x7f092819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.424+0000 7f092db59640 1 -- 172.21.15.83:0/1164210544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09280ff9f0 msgr2=0x7f092819d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.424+0000 7f092db59640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09280ff9f0 0x7f092819d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.424+0000 7f092db59640 1 -- 172.21.15.83:0/1164210544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0918009560 con 0x7f0928104d10 2024-03-11T19:40:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.424+0000 7f092cb57640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0928102360 0x7f092819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:40:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.425+0000 7f092db59640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0928104d10 0x7f09281a4eb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f092000cad0 tx=0x7f092000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.425+0000 7f091e7fc640 1 -- 172.21.15.83:0/1164210544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092000ae20 con 0x7f0928104d10 2024-03-11T19:40:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.425+0000 7f092f5bc640 1 -- 172.21.15.83:0/1164210544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092806b1b0 con 0x7f0928104d10 2024-03-11T19:40:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.425+0000 7f091e7fc640 1 -- 172.21.15.83:0/1164210544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0920005020 con 0x7f0928104d10 2024-03-11T19:40:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.426+0000 7f092f5bc640 1 -- 172.21.15.83:0/1164210544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092806b740 con 0x7f0928104d10 2024-03-11T19:40:41.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.427+0000 7f091e7fc640 1 -- 172.21.15.83:0/1164210544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092000f6b0 con 0x7f0928104d10 2024-03-11T19:40:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.430+0000 7f093218d740 1 Processor -- start 2024-03-11T19:40:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.430+0000 7f091e7fc640 1 -- 172.21.15.83:0/1164210544 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0920004a00 con 0x7f0928104d10 2024-03-11T19:40:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.430+0000 7f093218d740 1 -- start start 2024-03-11T19:40:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.430+0000 7f093218d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b5d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.430+0000 7f093218d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b62760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.430+0000 7f093218d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.430+0000 7f093218d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966b19960 con 0x559966b5dc60 2024-03-11T19:40:41.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.430+0000 7f093218d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966acd310 con 0x559966b5d270 2024-03-11T19:40:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.430+0000 7f093218d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966aca840 con 0x559966b62dd0 2024-03-11T19:40:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.431+0000 7f092cb57640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.431+0000 7f092cb57640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b65260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56722/0 (socket says 172.21.15.83:56722) 2024-03-11T19:40:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.431+0000 7f092cb57640 1 -- 172.21.15.83:0/905794017 learned_addr learned my addr 172.21.15.83:0/905794017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.431+0000 7f091e7fc640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09140a1aa0 0x7f09140a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.431+0000 7f092cb57640 1 --2- 172.21.15.83:0/905794017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b5d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.431+0000 7f092db59640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09140a1aa0 0x7f09140a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.431+0000 7f092d358640 1 --2- 172.21.15.83:0/905794017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b62760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f091e7fc640 1 -- 172.21.15.83:0/1164210544 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f09200cd260 con 0x7f0928104d10 2024-03-11T19:40:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f092cb57640 1 -- 172.21.15.83:0/905794017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 msgr2=0x559966b5d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f092cb57640 1 --2- 172.21.15.83:0/905794017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b5d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f092cb57640 1 -- 172.21.15.83:0/905794017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 msgr2=0x559966b62760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f092cb57640 1 --2- 172.21.15.83:0/905794017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b62760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f092cb57640 1 -- 172.21.15.83:0/905794017 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559966a88060 con 0x559966b62dd0 2024-03-11T19:40:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f092d358640 1 --2- 172.21.15.83:0/905794017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b62760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:40:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f092cb57640 1 --2- 172.21.15.83:0/905794017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b65260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09240155d0 tx=0x7f092400a170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32fa3909478510d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f092db59640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09140a1aa0 0x7f09140a3f60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0920007850 tx=0x7f092008c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f09137fe640 1 -- 172.21.15.83:0/905794017 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924004220 con 0x559966b62dd0 2024-03-11T19:40:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f09137fe640 1 -- 172.21.15.83:0/905794017 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f092400c070 con 0x559966b62dd0 2024-03-11T19:40:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.432+0000 7f09137fe640 1 -- 172.21.15.83:0/905794017 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240186d0 con 0x559966b62dd0 2024-03-11T19:40:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.434+0000 7f093218d740 1 -- 172.21.15.83:0/905794017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 msgr2=0x559966b65260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.434+0000 7f093218d740 1 --2- 172.21.15.83:0/905794017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b65260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09240155d0 tx=0x7f092400a170 comp rx=0 tx=0).stop 2024-03-11T19:40:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.434+0000 7f093218d740 1 -- 172.21.15.83:0/905794017 shutdown_connections 2024-03-11T19:40:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.434+0000 7f093218d740 1 --2- 172.21.15.83:0/905794017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b65260 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.434+0000 7f093218d740 1 --2- 172.21.15.83:0/905794017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b62760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.434+0000 7f093218d740 1 --2- 172.21.15.83:0/905794017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b5d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.434+0000 7f093218d740 1 -- 172.21.15.83:0/905794017 >> 172.21.15.83:0/905794017 conn(0x559966b587e0 msgr2=0x559966b58be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.434+0000 7f093218d740 1 -- 172.21.15.83:0/905794017 shutdown_connections 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.435+0000 7f093218d740 1 -- 172.21.15.83:0/905794017 wait complete. 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.436+0000 7f093218d740 1 -- start start 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.436+0000 7f093218d740 1 Processor -- start 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f093218d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f093218d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b67910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f093218d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b6e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f092cb57640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f092cb57640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b673d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40418/0 (socket says 172.21.15.83:40418) 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f092cb57640 1 -- 172.21.15.83:0/2224850986 learned_addr learned my addr 172.21.15.83:0/2224850986 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f092d358640 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b67910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f093218d740 1 -- 172.21.15.83:0/2224850986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966ac6020 con 0x559966b5dc60 2024-03-11T19:40:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f093218d740 1 -- 172.21.15.83:0/2224850986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966799430 con 0x559966b5d270 2024-03-11T19:40:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f093218d740 1 -- 172.21.15.83:0/2224850986 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966755e00 con 0x559966b62dd0 2024-03-11T19:40:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f093218d740 10 client.0 _ll_register_callbacks cb 0x559966a6dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:41.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.437+0000 7f092cb57640 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b6e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.438+0000 7f092d358640 1 -- 172.21.15.83:0/2224850986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 msgr2=0x559966b6e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.438+0000 7f092d358640 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b6e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.438+0000 7f092d358640 1 -- 172.21.15.83:0/2224850986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 msgr2=0x559966b673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.438+0000 7f092d358640 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.438+0000 7f092d358640 1 -- 172.21.15.83:0/2224850986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0924015300 con 0x559966b5dc60 2024-03-11T19:40:41.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.438+0000 7f092cb57640 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:41.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.438+0000 7f092d358640 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b67910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0918009850 tx=0x7f09180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f038eb32940b53 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.438+0000 7f0910ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:40:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.439+0000 7f0910ff9640 1 -- 172.21.15.83:0/2224850986 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918004250 con 0x559966b5dc60 2024-03-11T19:40:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.439+0000 7f093218d740 1 -- 172.21.15.83:0/2224850986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599667c57e0 con 0x559966b5dc60 2024-03-11T19:40:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.439+0000 7f0910ff9640 1 -- 172.21.15.83:0/2224850986 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0918045070 con 0x559966b5dc60 2024-03-11T19:40:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.439+0000 7f0910ff9640 1 -- 172.21.15.83:0/2224850986 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09180043b0 con 0x559966b5dc60 2024-03-11T19:40:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.439+0000 7f093218d740 10 client.15600 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.439+0000 7f093218d740 1 -- 172.21.15.83:0/2224850986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559966ac46a0 con 0x559966b5dc60 2024-03-11T19:40:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.440+0000 7f0910ff9640 1 -- 172.21.15.83:0/2224850986 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0918043070 con 0x559966b5dc60 2024-03-11T19:40:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.440+0000 7f0910ff9640 1 -- 172.21.15.83:0/2224850986 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f091803c040 con 0x559966b5dc60 2024-03-11T19:40:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.440+0000 7f093218d740 10 client.15600 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.440+0000 7f093218d740 10 client.15600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:40:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.440+0000 7f093218d740 20 client.15600 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.440+0000 7f093218d740 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559966b734e0 0x559966b75960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.440+0000 7f093218d740 4 client.15600 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.440+0000 7f093218d740 1 -- 172.21.15.83:0/2224850986 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599667c57e0 con 0x559966b734e0 2024-03-11T19:40:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.441+0000 7f092cb57640 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559966b734e0 0x559966b75960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.442+0000 7f092cb57640 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559966b734e0 0x559966b75960 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=568e536ad9d53d50 server_cookie=8bb792f1d344aad0 in_seq=0 out_seq=0 2024-03-11T19:40:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.442+0000 7f0910ff9640 10 client.15600 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:40:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.443+0000 7f0910ff9640 1 -- 172.21.15.83:0/2224850986 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599667c57e0 con 0x559966b734e0 2024-03-11T19:40:41.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.443+0000 7f0910ff9640 10 client.15600 handle_command_reply: tid=0 2024-03-11T19:40:41.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.444+0000 7f093218d740 1 client.15600 shutdown 2024-03-11T19:40:41.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.445+0000 7f093218d740 1 -- 172.21.15.83:0/2224850986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 msgr2=0x559966b67910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.445+0000 7f093218d740 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b67910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0918009850 tx=0x7f09180048f0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.447+0000 7f093218d740 1 -- 172.21.15.83:0/2224850986 shutdown_connections 2024-03-11T19:40:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.447+0000 7f093218d740 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559966b734e0 0x559966b75960 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.447+0000 7f093218d740 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b6e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.447+0000 7f093218d740 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b67910 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.447+0000 7f093218d740 1 --2- 172.21.15.83:0/2224850986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.447+0000 7f093218d740 1 -- 172.21.15.83:0/2224850986 >> 172.21.15.83:0/2224850986 conn(0x559966b587e0 msgr2=0x559966b57b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.447+0000 7f0910ff9640 0 client.15600 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:40:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.451+0000 7f093218d740 1 -- 172.21.15.83:0/2224850986 shutdown_connections 2024-03-11T19:40:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.453+0000 7f093218d740 1 -- 172.21.15.83:0/2224850986 wait complete. 2024-03-11T19:40:41.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.454+0000 7f093218d740 20 client.15600 trim_cache size 0 max 16384 2024-03-11T19:40:41.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.485+0000 7f093218d740 1 Processor -- start 2024-03-11T19:40:41.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.486+0000 7f093218d740 1 -- start start 2024-03-11T19:40:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.486+0000 7f093218d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b58bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.486+0000 7f093218d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b6e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.486+0000 7f092cb57640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b58bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.486+0000 7f092cb57640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b58bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40424/0 (socket says 172.21.15.83:40424) 2024-03-11T19:40:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.486+0000 7f092cb57640 1 -- 172.21.15.83:0/1146740924 learned_addr learned my addr 172.21.15.83:0/1146740924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.487+0000 7f093218d740 1 --2- 172.21.15.83:0/1146740924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.487+0000 7f093218d740 1 -- 172.21.15.83:0/1146740924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966a4b130 con 0x559966b5dc60 2024-03-11T19:40:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.487+0000 7f093218d740 1 -- 172.21.15.83:0/1146740924 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966b4aa40 con 0x559966b5d270 2024-03-11T19:40:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.487+0000 7f093218d740 1 -- 172.21.15.83:0/1146740924 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966b5e520 con 0x559966b62dd0 2024-03-11T19:40:41.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.487+0000 7f092d358640 1 --2- 172.21.15.83:0/1146740924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b6e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.487+0000 7f092cb57640 1 --2- 172.21.15.83:0/1146740924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.487+0000 7f092cb57640 1 -- 172.21.15.83:0/1146740924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 msgr2=0x559966b673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.487+0000 7f092cb57640 1 --2- 172.21.15.83:0/1146740924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.487+0000 7f092cb57640 1 -- 172.21.15.83:0/1146740924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 msgr2=0x559966b6e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.487+0000 7f092cb57640 1 --2- 172.21.15.83:0/1146740924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b6e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.487+0000 7f092cb57640 1 -- 172.21.15.83:0/1146740924 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559966a60c20 con 0x559966b5d270 2024-03-11T19:40:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.488+0000 7f092cb57640 1 --2- 172.21.15.83:0/1146740924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b58bc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09240155d0 tx=0x7f092400a500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bdc40544297ffc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.488+0000 7f0912ffd640 1 -- 172.21.15.83:0/1146740924 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924003ac0 con 0x559966b5d270 2024-03-11T19:40:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.488+0000 7f0912ffd640 1 -- 172.21.15.83:0/1146740924 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0924003c20 con 0x559966b5d270 2024-03-11T19:40:41.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.488+0000 7f0912ffd640 1 -- 172.21.15.83:0/1146740924 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092402e4e0 con 0x559966b5d270 2024-03-11T19:40:41.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.489+0000 7f093218d740 1 -- 172.21.15.83:0/1146740924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 msgr2=0x559966b58bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.489+0000 7f093218d740 1 --2- 172.21.15.83:0/1146740924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b58bc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09240155d0 tx=0x7f092400a500 comp rx=0 tx=0).stop 2024-03-11T19:40:41.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.489+0000 7f093218d740 1 -- 172.21.15.83:0/1146740924 shutdown_connections 2024-03-11T19:40:41.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.489+0000 7f093218d740 1 --2- 172.21.15.83:0/1146740924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b62dd0 0x559966b673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.489+0000 7f093218d740 1 --2- 172.21.15.83:0/1146740924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b5dc60 0x559966b6e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.489+0000 7f093218d740 1 --2- 172.21.15.83:0/1146740924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5d270 0x559966b58bc0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09240155d0 tx=0x7f092400a500 comp rx=0 tx=0).stop 2024-03-11T19:40:41.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.489+0000 7f093218d740 1 -- 172.21.15.83:0/1146740924 >> 172.21.15.83:0/1146740924 conn(0x559966b587e0 msgr2=0x559966b53a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.489+0000 7f093218d740 1 -- 172.21.15.83:0/1146740924 shutdown_connections 2024-03-11T19:40:41.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.490+0000 7f093218d740 1 -- 172.21.15.83:0/1146740924 wait complete. 2024-03-11T19:40:41.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.490+0000 7f093218d740 1 -- start start 2024-03-11T19:40:41.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.490+0000 7f093218d740 1 Processor -- start 2024-03-11T19:40:41.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.491+0000 7f093218d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5dc60 0x559966b6e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.491+0000 7f093218d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b62dd0 0x559966b6ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.491+0000 7f093218d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b6e9c0 0x559966b6cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.491+0000 7f093218d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966b6d6d0 con 0x559966b62dd0 2024-03-11T19:40:41.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.491+0000 7f093218d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966b6d840 con 0x559966b5dc60 2024-03-11T19:40:41.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.491+0000 7f093218d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559966bc9a70 con 0x559966b6e9c0 2024-03-11T19:40:41.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.491+0000 7f092cb57640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5dc60 0x559966b6e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.491+0000 7f093218d740 10 client.0 _ll_register_callbacks cb 0x7f092400a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:41.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.491+0000 7f092cb57640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5dc60 0x559966b6e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40428/0 (socket says 172.21.15.83:40428) 2024-03-11T19:40:41.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.491+0000 7f092cb57640 1 -- 172.21.15.83:0/3286468007 learned_addr learned my addr 172.21.15.83:0/3286468007 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.491+0000 7f092cb57640 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b6e9c0 0x559966b6cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.492+0000 7f092cb57640 1 -- 172.21.15.83:0/3286468007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b6e9c0 msgr2=0x559966b6cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.492+0000 7f092cb57640 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b6e9c0 0x559966b6cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.492+0000 7f092cb57640 1 -- 172.21.15.83:0/3286468007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b62dd0 msgr2=0x559966b6ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.492+0000 7f092cb57640 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b62dd0 0x559966b6ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.492+0000 7f092cb57640 1 -- 172.21.15.83:0/3286468007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559966a88060 con 0x559966b5dc60 2024-03-11T19:40:41.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.492+0000 7f092cb57640 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5dc60 0x559966b6e430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0924006d70 tx=0x7f0924006cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4890a4dffe2f99c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.492+0000 7f0910ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:40:41.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.492+0000 7f0910ff9640 1 -- 172.21.15.83:0/3286468007 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924004280 con 0x559966b5dc60 2024-03-11T19:40:41.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.492+0000 7f0910ff9640 1 -- 172.21.15.83:0/3286468007 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0924018470 con 0x559966b5dc60 2024-03-11T19:40:41.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.492+0000 7f093218d740 1 -- 172.21.15.83:0/3286468007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f092400c070 con 0x559966b5dc60 2024-03-11T19:40:41.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.493+0000 7f0910ff9640 1 -- 172.21.15.83:0/3286468007 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09240043e0 con 0x559966b5dc60 2024-03-11T19:40:41.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.493+0000 7f093218d740 10 client.25109 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:41.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.493+0000 7f093218d740 1 -- 172.21.15.83:0/3286468007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559966a868c0 con 0x559966b5dc60 2024-03-11T19:40:41.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.493+0000 7f0910ff9640 1 -- 172.21.15.83:0/3286468007 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f092400c070 con 0x559966b5dc60 2024-03-11T19:40:41.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.494+0000 7f0910ff9640 1 -- 172.21.15.83:0/3286468007 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0924020040 con 0x559966b5dc60 2024-03-11T19:40:41.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.494+0000 7f093218d740 10 client.25109 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:41.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.494+0000 7f093218d740 10 client.25109 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:40:41.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.494+0000 7f093218d740 20 client.25109 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:41.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.494+0000 7f093218d740 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559966bcb090 0x559966bcd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.494+0000 7f093218d740 4 client.25109 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:40:41.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.494+0000 7f093218d740 1 -- 172.21.15.83:0/3286468007 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559966802720 con 0x559966bcb090 2024-03-11T19:40:41.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.494+0000 7f092d358640 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559966bcb090 0x559966bcd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.496+0000 7f092d358640 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559966bcb090 0x559966bcd4b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=373a8a1eeb9a85e9 server_cookie=8c692f6d69af19b9 in_seq=0 out_seq=0 2024-03-11T19:40:41.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.496+0000 7f0910ff9640 10 client.25109 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:40:41.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f0910ff9640 1 -- 172.21.15.83:0/3286468007 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559966802720 con 0x559966bcb090 2024-03-11T19:40:41.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f0910ff9640 10 client.25109 handle_command_reply: tid=0 2024-03-11T19:40:41.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f093218d740 1 client.25109 shutdown 2024-03-11T19:40:41.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f093218d740 1 -- 172.21.15.83:0/3286468007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5dc60 msgr2=0x559966b6e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f093218d740 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5dc60 0x559966b6e430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0924006d70 tx=0x7f0924006cb0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f093218d740 1 -- 172.21.15.83:0/3286468007 shutdown_connections 2024-03-11T19:40:41.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f093218d740 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559966bcb090 0x559966bcd4b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f093218d740 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559966b6e9c0 0x559966b6cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f093218d740 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559966b62dd0 0x559966b6ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f093218d740 1 --2- 172.21.15.83:0/3286468007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559966b5dc60 0x559966b6e430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f093218d740 1 -- 172.21.15.83:0/3286468007 >> 172.21.15.83:0/3286468007 conn(0x559966b587e0 msgr2=0x559966b58bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f0910ff9640 0 client.25109 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:40:41.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.497+0000 7f093218d740 1 -- 172.21.15.83:0/3286468007 shutdown_connections 2024-03-11T19:40:41.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.498+0000 7f093218d740 1 -- 172.21.15.83:0/3286468007 wait complete. 2024-03-11T19:40:41.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.498+0000 7f093218d740 20 client.25109 trim_cache size 0 max 16384 2024-03-11T19:40:41.507 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:40:41.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.498+0000 7f092f5bc640 1 -- 172.21.15.83:0/1164210544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09140a1aa0 msgr2=0x7f09140a3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.499+0000 7f092f5bc640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09140a1aa0 0x7f09140a3f60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0920007850 tx=0x7f092008c040 comp rx=0 tx=0).stop 2024-03-11T19:40:41.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.499+0000 7f092f5bc640 1 -- 172.21.15.83:0/1164210544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0928104d10 msgr2=0x7f09281a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.499+0000 7f092f5bc640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0928104d10 0x7f09281a4eb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f092000cad0 tx=0x7f092000cf90 comp rx=0 tx=0).stop 2024-03-11T19:40:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.500+0000 7f092f5bc640 1 -- 172.21.15.83:0/1164210544 shutdown_connections 2024-03-11T19:40:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.500+0000 7f092f5bc640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f09140a1aa0 0x7f09140a3f60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.500+0000 7f092f5bc640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0928104d10 0x7f09281a4eb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.500+0000 7f092f5bc640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0928102360 0x7f092819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.500+0000 7f092f5bc640 1 --2- 172.21.15.83:0/1164210544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09280ff9f0 0x7f092819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.500+0000 7f092f5bc640 1 -- 172.21.15.83:0/1164210544 >> 172.21.15.83:0/1164210544 conn(0x7f09280fb610 msgr2=0x7f0928102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.501+0000 7f092f5bc640 1 -- 172.21.15.83:0/1164210544 shutdown_connections 2024-03-11T19:40:41.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.501+0000 7f092f5bc640 1 -- 172.21.15.83:0/1164210544 wait complete. 2024-03-11T19:40:41.523 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:40:41.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:40:41.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.697+0000 7f2fbb423640 1 -- 172.21.15.83:0/3064157082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb4072760 msgr2=0x7f2fb4108570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.697+0000 7f2fbb423640 1 --2- 172.21.15.83:0/3064157082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb4072760 0x7f2fb4108570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009860 tx=0x7f2fb002bb70 comp rx=0 tx=0).stop 2024-03-11T19:40:41.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.697+0000 7f2fbb423640 1 -- 172.21.15.83:0/3064157082 shutdown_connections 2024-03-11T19:40:41.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.697+0000 7f2fbb423640 1 --2- 172.21.15.83:0/3064157082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb4109230 0x7f2fb41096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.697+0000 7f2fbb423640 1 --2- 172.21.15.83:0/3064157082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4072c10 0x7f2fb4108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.697+0000 7f2fbb423640 1 --2- 172.21.15.83:0/3064157082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb4072760 0x7f2fb4108570 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009860 tx=0x7f2fb002bb70 comp rx=0 tx=0).stop 2024-03-11T19:40:41.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.697+0000 7f2fbb423640 1 -- 172.21.15.83:0/3064157082 >> 172.21.15.83:0/3064157082 conn(0x7f2fb406e380 msgr2=0x7f2fb40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.697+0000 7f2fbb423640 1 -- 172.21.15.83:0/3064157082 shutdown_connections 2024-03-11T19:40:41.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.698+0000 7f2fbb423640 1 -- 172.21.15.83:0/3064157082 wait complete. 2024-03-11T19:40:41.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.699+0000 7f2fbb423640 1 Processor -- start 2024-03-11T19:40:41.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.699+0000 7f2fbb423640 1 -- start start 2024-03-11T19:40:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.699+0000 7f2fbb423640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4072c10 0x7f2fb41a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.699+0000 7f2fbb423640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb4109230 0x7f2fb41a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.699+0000 7f2fbb423640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb41ab860 0x7f2fb41adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.699+0000 7f2fbb423640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb41ae190 con 0x7f2fb4072c10 2024-03-11T19:40:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.699+0000 7f2fbb423640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb41ae300 con 0x7f2fb4109230 2024-03-11T19:40:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.699+0000 7f2fbb423640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb41ae4c0 con 0x7f2fb41ab860 2024-03-11T19:40:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb91bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4072c10 0x7f2fb41a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb91bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4072c10 0x7f2fb41a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45394/0 (socket says 172.21.15.83:45394) 2024-03-11T19:40:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb91bf640 1 -- 172.21.15.83:0/4105055854 learned_addr learned my addr 172.21.15.83:0/4105055854 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb89be640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb4109230 0x7f2fb41a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb99c0640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb41ab860 0x7f2fb41adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb99c0640 1 -- 172.21.15.83:0/4105055854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb4109230 msgr2=0x7f2fb41a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb99c0640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb4109230 0x7f2fb41a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb99c0640 1 -- 172.21.15.83:0/4105055854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4072c10 msgr2=0x7f2fb41a6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb99c0640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4072c10 0x7f2fb41a6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb99c0640 1 -- 172.21.15.83:0/4105055854 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac009900 con 0x7f2fb41ab860 2024-03-11T19:40:41.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb89be640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb4109230 0x7f2fb41a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:41.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb91bf640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4072c10 0x7f2fb41a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:41.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.700+0000 7f2fb99c0640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb41ab860 0x7f2fb41adc50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00efc0 tx=0x7f2fac00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.701+0000 7f2faa7fc640 1 -- 172.21.15.83:0/4105055854 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac003970 con 0x7f2fb41ab860 2024-03-11T19:40:41.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.701+0000 7f2faa7fc640 1 -- 172.21.15.83:0/4105055854 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fac004020 con 0x7f2fb41ab860 2024-03-11T19:40:41.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.701+0000 7f2faa7fc640 1 -- 172.21.15.83:0/4105055854 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00b960 con 0x7f2fb41ab860 2024-03-11T19:40:41.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.701+0000 7f2fbb423640 1 -- 172.21.15.83:0/4105055854 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb0009490 con 0x7f2fb41ab860 2024-03-11T19:40:41.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.701+0000 7f2fbb423640 1 -- 172.21.15.83:0/4105055854 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb41aead0 con 0x7f2fb41ab860 2024-03-11T19:40:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.705+0000 7f2fbdff7740 1 Processor -- start 2024-03-11T19:40:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.705+0000 7f2faa7fc640 1 -- 172.21.15.83:0/4105055854 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2fac00a040 con 0x7f2fb41ab860 2024-03-11T19:40:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fbdff7740 1 -- start start 2024-03-11T19:40:41.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 0x558f04a5b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a60760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a60dd0 0x558f04a63260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fbdff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f04a17960 con 0x558f04a60dd0 2024-03-11T19:40:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fbdff7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f049cb310 con 0x558f04a5b270 2024-03-11T19:40:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fbdff7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f049c8840 con 0x558f04a5bc60 2024-03-11T19:40:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fb91bf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a60760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fb91bf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a60760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56778/0 (socket says 172.21.15.83:56778) 2024-03-11T19:40:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fb91bf640 1 -- 172.21.15.83:0/1495195855 learned_addr learned my addr 172.21.15.83:0/1495195855 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fb89be640 1 --2- 172.21.15.83:0/1495195855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 0x558f04a5b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2fb89be640 1 --2- 172.21.15.83:0/1495195855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a60dd0 0x558f04a63260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.706+0000 7f2faa7fc640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2fa00a1c00 0x7f2fa00a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.707+0000 7f2fb89be640 1 -- 172.21.15.83:0/1495195855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 msgr2=0x558f04a60760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.707+0000 7f2fb89be640 1 --2- 172.21.15.83:0/1495195855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a60760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.707+0000 7f2fb89be640 1 -- 172.21.15.83:0/1495195855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a60dd0 msgr2=0x558f04a63260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.707+0000 7f2fb89be640 1 --2- 172.21.15.83:0/1495195855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a60dd0 0x558f04a63260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.707+0000 7f2fb89be640 1 -- 172.21.15.83:0/1495195855 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f04986060 con 0x558f04a5b270 2024-03-11T19:40:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.707+0000 7f2faa7fc640 1 -- 172.21.15.83:0/4105055854 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2fac0cd350 con 0x7f2fb41ab860 2024-03-11T19:40:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.707+0000 7f2fb99c0640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2fa00a1c00 0x7f2fa00a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.708+0000 7f2fb89be640 1 --2- 172.21.15.83:0/1495195855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 0x558f04a5b690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2fb41a77d0 tx=0x7f2fa4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ca1c0b88c607dda server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.708+0000 7f2f9f7fe640 1 -- 172.21.15.83:0/1495195855 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4011050 con 0x558f04a5b270 2024-03-11T19:40:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.708+0000 7f2f9f7fe640 1 -- 172.21.15.83:0/1495195855 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fa400fdf0 con 0x558f04a5b270 2024-03-11T19:40:41.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.708+0000 7f2f9f7fe640 1 -- 172.21.15.83:0/1495195855 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40055d0 con 0x558f04a5b270 2024-03-11T19:40:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.709+0000 7f2fbdff7740 1 -- 172.21.15.83:0/1495195855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 msgr2=0x558f04a5b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.709+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/1495195855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 0x558f04a5b690 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2fb41a77d0 tx=0x7f2fa4005850 comp rx=0 tx=0).stop 2024-03-11T19:40:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.709+0000 7f2fb99c0640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2fa00a1c00 0x7f2fa00a40c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2fac096ca0 tx=0x7f2fac004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.709+0000 7f2fbdff7740 1 -- 172.21.15.83:0/1495195855 shutdown_connections 2024-03-11T19:40:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.709+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/1495195855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a60dd0 0x558f04a63260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.709+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/1495195855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a60760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.709+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/1495195855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 0x558f04a5b690 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.709+0000 7f2fbdff7740 1 -- 172.21.15.83:0/1495195855 >> 172.21.15.83:0/1495195855 conn(0x558f04a567e0 msgr2=0x558f04a56be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.710+0000 7f2fbdff7740 1 -- 172.21.15.83:0/1495195855 shutdown_connections 2024-03-11T19:40:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.710+0000 7f2fbdff7740 1 -- 172.21.15.83:0/1495195855 wait complete. 2024-03-11T19:40:41.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.710+0000 7f2fbdff7740 1 -- start start 2024-03-11T19:40:41.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fbdff7740 1 Processor -- start 2024-03-11T19:40:41.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 0x558f04a65390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a60dd0 0x558f04a6c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fbdff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f049c4020 con 0x558f04a60dd0 2024-03-11T19:40:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fbdff7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f04697430 con 0x558f04a5b270 2024-03-11T19:40:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fbdff7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f04653e00 con 0x558f04a5bc60 2024-03-11T19:40:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fbdff7740 10 client.0 _ll_register_callbacks cb 0x558f0496bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fb91bf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fb89be640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 0x558f04a65390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fb89be640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 0x558f04a65390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40454/0 (socket says 172.21.15.83:40454) 2024-03-11T19:40:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.711+0000 7f2fb89be640 1 -- 172.21.15.83:0/2572764308 learned_addr learned my addr 172.21.15.83:0/2572764308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.712+0000 7f2fb89be640 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a60dd0 0x558f04a6c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.712+0000 7f2fb91bf640 1 -- 172.21.15.83:0/2572764308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 msgr2=0x558f04a65390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.712+0000 7f2fb91bf640 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 0x558f04a65390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.712+0000 7f2fb91bf640 1 -- 172.21.15.83:0/2572764308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a60dd0 msgr2=0x558f04a6c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.712+0000 7f2fb91bf640 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a60dd0 0x558f04a6c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.712+0000 7f2fb91bf640 1 -- 172.21.15.83:0/2572764308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa4019040 con 0x558f04a5bc60 2024-03-11T19:40:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.713+0000 7f2fb91bf640 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a658d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x558f04a5c6d0 tx=0x7f2fb00042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e720448e526c3a42 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.713+0000 7f2f9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:40:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.714+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2572764308 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0034640 con 0x558f04a5bc60 2024-03-11T19:40:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.714+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2572764308 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2fb00347a0 con 0x558f04a5bc60 2024-03-11T19:40:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.714+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2572764308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f046c37e0 con 0x558f04a5bc60 2024-03-11T19:40:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.714+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2572764308 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fb00035f0 con 0x558f04a5bc60 2024-03-11T19:40:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.715+0000 7f2fbdff7740 10 client.25228 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.715+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2572764308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558f049c26a0 con 0x558f04a5bc60 2024-03-11T19:40:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.715+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2572764308 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558f046c37e0 con 0x558f04a5bc60 2024-03-11T19:40:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.715+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2572764308 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2fb0050070 con 0x558f04a5bc60 2024-03-11T19:40:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.715+0000 7f2fbdff7740 10 client.25228 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.715+0000 7f2fbdff7740 10 client.25228 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:40:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.715+0000 7f2fbdff7740 20 client.25228 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.715+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558f04a71440 0x558f04a73830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.715+0000 7f2fbdff7740 4 client.25228 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:41.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.715+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2572764308 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f04700720 con 0x558f04a71440 2024-03-11T19:40:41.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.716+0000 7f2fb89be640 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558f04a71440 0x558f04a73830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.716+0000 7f2f9cff9640 10 client.25228 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:40:41.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.717+0000 7f2fb89be640 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558f04a71440 0x558f04a73830 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d022c6fbf5eea41 server_cookie=81845f0a2ce7e749 in_seq=0 out_seq=0 2024-03-11T19:40:41.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.718+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2572764308 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f04700720 con 0x558f04a71440 2024-03-11T19:40:41.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.718+0000 7f2f9cff9640 10 client.25228 handle_command_reply: tid=0 2024-03-11T19:40:41.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.718+0000 7f2fbdff7740 1 client.25228 shutdown 2024-03-11T19:40:41.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2572764308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 msgr2=0x558f04a658d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a658d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x558f04a5c6d0 tx=0x7f2fb00042e0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2572764308 shutdown_connections 2024-03-11T19:40:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558f04a71440 0x558f04a73830 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a60dd0 0x558f04a6c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a658d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2f9cff9640 0 client.25228 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:40:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2572764308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5b270 0x558f04a65390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2572764308 >> 172.21.15.83:0/2572764308 conn(0x558f04a567e0 msgr2=0x558f04a55b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2572764308 shutdown_connections 2024-03-11T19:40:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2572764308 wait complete. 2024-03-11T19:40:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.719+0000 7f2fbdff7740 20 client.25228 trim_cache size 0 max 16384 2024-03-11T19:40:41.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.743+0000 7f2fbdff7740 1 Processor -- start 2024-03-11T19:40:41.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.743+0000 7f2fbdff7740 1 -- start start 2024-03-11T19:40:41.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.743+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 0x558f04a5c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.743+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5bc60 0x558f04a6c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.743+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a60dd0 0x558f04a60690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fb89be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 0x558f04a5c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fbdff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f04949130 con 0x558f04a5b270 2024-03-11T19:40:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fb89be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 0x558f04a5c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45428/0 (socket says 172.21.15.83:45428) 2024-03-11T19:40:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fb89be640 1 -- 172.21.15.83:0/643702196 learned_addr learned my addr 172.21.15.83:0/643702196 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fbdff7740 1 -- 172.21.15.83:0/643702196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f04a5c4a0 con 0x558f04a5bc60 2024-03-11T19:40:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fb91bf640 1 --2- 172.21.15.83:0/643702196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5bc60 0x558f04a6c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fbdff7740 1 -- 172.21.15.83:0/643702196 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f04a52320 con 0x558f04a60dd0 2024-03-11T19:40:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fb89be640 1 --2- 172.21.15.83:0/643702196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a60dd0 0x558f04a60690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fb91bf640 1 -- 172.21.15.83:0/643702196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a60dd0 msgr2=0x558f04a60690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fb91bf640 1 --2- 172.21.15.83:0/643702196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a60dd0 0x558f04a60690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fb91bf640 1 -- 172.21.15.83:0/643702196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 msgr2=0x558f04a5c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fb91bf640 1 --2- 172.21.15.83:0/643702196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 0x558f04a5c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.744+0000 7f2fb91bf640 1 -- 172.21.15.83:0/643702196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f0495ec20 con 0x558f04a5bc60 2024-03-11T19:40:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.745+0000 7f2fb91bf640 1 --2- 172.21.15.83:0/643702196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5bc60 0x558f04a6c860 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0000c00 tx=0x7f2fb00096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3645f9b17f00bcc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.745+0000 7f2f9effd640 1 -- 172.21.15.83:0/643702196 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb003f650 con 0x558f04a5bc60 2024-03-11T19:40:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.745+0000 7f2f9effd640 1 -- 172.21.15.83:0/643702196 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fb004d070 con 0x558f04a5bc60 2024-03-11T19:40:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.745+0000 7f2f9effd640 1 -- 172.21.15.83:0/643702196 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0003850 con 0x558f04a5bc60 2024-03-11T19:40:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.746+0000 7f2fbdff7740 1 -- 172.21.15.83:0/643702196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5bc60 msgr2=0x558f04a6c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.746+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/643702196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5bc60 0x558f04a6c860 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0000c00 tx=0x7f2fb00096b0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.746+0000 7f2fbdff7740 1 -- 172.21.15.83:0/643702196 shutdown_connections 2024-03-11T19:40:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.746+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/643702196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a60dd0 0x558f04a60690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.746+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/643702196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a5bc60 0x558f04a6c860 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.746+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/643702196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 0x558f04a5c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.746+0000 7f2fbdff7740 1 -- 172.21.15.83:0/643702196 >> 172.21.15.83:0/643702196 conn(0x558f04a567e0 msgr2=0x558f04a56bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.746+0000 7f2fbdff7740 1 -- 172.21.15.83:0/643702196 shutdown_connections 2024-03-11T19:40:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.746+0000 7f2fbdff7740 1 -- 172.21.15.83:0/643702196 wait complete. 2024-03-11T19:40:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.746+0000 7f2fbdff7740 1 -- start start 2024-03-11T19:40:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.747+0000 7f2fbdff7740 1 Processor -- start 2024-03-11T19:40:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.747+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 0x558f04a6c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.747+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a6ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.747+0000 7f2fbdff7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a60dd0 0x558f04a6b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.747+0000 7f2fbdff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f04a6b610 con 0x558f04a5b270 2024-03-11T19:40:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.747+0000 7f2fbdff7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f04a6b750 con 0x558f04a60dd0 2024-03-11T19:40:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.747+0000 7f2fbdff7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f04a6b910 con 0x558f04a5bc60 2024-03-11T19:40:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.747+0000 7f2fbdff7740 10 client.0 _ll_register_callbacks cb 0x7f2fa4009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.747+0000 7f2fb89be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 0x558f04a6c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.747+0000 7f2fb89be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 0x558f04a6c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45438/0 (socket says 172.21.15.83:45438) 2024-03-11T19:40:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.747+0000 7f2fb89be640 1 -- 172.21.15.83:0/2180190905 learned_addr learned my addr 172.21.15.83:0/2180190905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.748+0000 7f2fb91bf640 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a6ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.748+0000 7f2fb89be640 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a60dd0 0x558f04a6b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.748+0000 7f2fb89be640 1 -- 172.21.15.83:0/2180190905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 msgr2=0x558f04a6ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.748+0000 7f2fb89be640 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a6ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.748+0000 7f2fb89be640 1 -- 172.21.15.83:0/2180190905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 msgr2=0x558f04a6c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.748+0000 7f2fb89be640 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 0x558f04a6c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.748+0000 7f2fb89be640 1 -- 172.21.15.83:0/2180190905 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f049c26a0 con 0x558f04a60dd0 2024-03-11T19:40:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.748+0000 7f2fb89be640 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a60dd0 0x558f04a6b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4005100 tx=0x7f2fb0005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dc9153161fad1ca server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.748+0000 7f2f9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:40:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.749+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2180190905 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4011140 con 0x558f04a60dd0 2024-03-11T19:40:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.749+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2180190905 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2fa40112a0 con 0x558f04a60dd0 2024-03-11T19:40:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.749+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2180190905 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fa400fdf0 con 0x558f04a60dd0 2024-03-11T19:40:41.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.749+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2180190905 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa400f910 con 0x558f04a60dd0 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.750+0000 7f2fbdff7740 10 client.25127 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.750+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2180190905 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558f049848c0 con 0x558f04a60dd0 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.750+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2180190905 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fa400fdf0 con 0x558f04a60dd0 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.750+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2180190905 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2fa40038f0 con 0x558f04a60dd0 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.750+0000 7f2fbdff7740 10 client.25127 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.750+0000 7f2fbdff7740 10 client.25127 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.750+0000 7f2fbdff7740 20 client.25127 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.750+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558f04ada060 0x558f04adc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.750+0000 7f2fbdff7740 4 client.25127 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.750+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2180190905 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558f0474f890 con 0x558f04ada060 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.750+0000 7f2fb91bf640 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558f04ada060 0x558f04adc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.751+0000 7f2fb91bf640 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558f04ada060 0x558f04adc480 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e95e4b17b5b27cce server_cookie=1d9545e0efe8b609 in_seq=0 out_seq=0 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.751+0000 7f2f9cff9640 10 client.25127 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2f9cff9640 1 -- 172.21.15.83:0/2180190905 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558f0474f890 con 0x558f04ada060 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2f9cff9640 10 client.25127 handle_command_reply: tid=0 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2fbdff7740 1 client.25127 shutdown 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2180190905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a60dd0 msgr2=0x558f04a6b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a60dd0 0x558f04a6b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4005100 tx=0x7f2fb0005a90 comp rx=0 tx=0).stop 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2180190905 shutdown_connections 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558f04ada060 0x558f04adc480 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f04a60dd0 0x558f04a6b0d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f04a5bc60 0x558f04a6ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2fbdff7740 1 --2- 172.21.15.83:0/2180190905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f04a5b270 0x558f04a6c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2180190905 >> 172.21.15.83:0/2180190905 conn(0x558f04a567e0 msgr2=0x558f04a652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2f9cff9640 0 client.25127 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:40:41.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.752+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2180190905 shutdown_connections 2024-03-11T19:40:41.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.753+0000 7f2fbdff7740 1 -- 172.21.15.83:0/2180190905 wait complete. 2024-03-11T19:40:41.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.753+0000 7f2fbdff7740 20 client.25127 trim_cache size 0 max 16384 2024-03-11T19:40:41.762 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:40:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.755+0000 7f2fbb423640 1 -- 172.21.15.83:0/4105055854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2fa00a1c00 msgr2=0x7f2fa00a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.755+0000 7f2fbb423640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2fa00a1c00 0x7f2fa00a40c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2fac096ca0 tx=0x7f2fac004a40 comp rx=0 tx=0).stop 2024-03-11T19:40:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.755+0000 7f2fbb423640 1 -- 172.21.15.83:0/4105055854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb41ab860 msgr2=0x7f2fb41adc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.755+0000 7f2fbb423640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb41ab860 0x7f2fb41adc50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00efc0 tx=0x7f2fac00c800 comp rx=0 tx=0).stop 2024-03-11T19:40:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.755+0000 7f2fbb423640 1 -- 172.21.15.83:0/4105055854 shutdown_connections 2024-03-11T19:40:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.755+0000 7f2fbb423640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2fa00a1c00 0x7f2fa00a40c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.755+0000 7f2fbb423640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb41ab860 0x7f2fb41adc50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.755+0000 7f2fbb423640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb4109230 0x7f2fb41a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.755+0000 7f2fbb423640 1 --2- 172.21.15.83:0/4105055854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4072c10 0x7f2fb41a6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.755+0000 7f2fbb423640 1 -- 172.21.15.83:0/4105055854 >> 172.21.15.83:0/4105055854 conn(0x7f2fb406e380 msgr2=0x7f2fb406fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.755+0000 7f2fbb423640 1 -- 172.21.15.83:0/4105055854 shutdown_connections 2024-03-11T19:40:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.756+0000 7f2fbb423640 1 -- 172.21.15.83:0/4105055854 wait complete. 2024-03-11T19:40:41.777 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:41 smithi083 ceph-mon[31864]: pgmap v393: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-03-11T19:40:41.778 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:41 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2352227085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:40:41.779 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:40:41.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:40:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:41 smithi137 ceph-mon[36220]: pgmap v393: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-03-11T19:40:41.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:41 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2352227085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:40:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:41 smithi204 ceph-mon[36739]: pgmap v393: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-03-11T19:40:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:41 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2352227085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:40:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.948+0000 7f6f9e70c640 1 -- 172.21.15.83:0/3504829122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f98108180 msgr2=0x7f6f9810a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.948+0000 7f6f9e70c640 1 --2- 172.21.15.83:0/3504829122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f98108180 0x7f6f9810a5b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009900 tx=0x7f6f8802bc40 comp rx=0 tx=0).stop 2024-03-11T19:40:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.948+0000 7f6f9e70c640 1 -- 172.21.15.83:0/3504829122 shutdown_connections 2024-03-11T19:40:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.948+0000 7f6f9e70c640 1 --2- 172.21.15.83:0/3504829122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9810d4a0 0x7f6f9810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.948+0000 7f6f9e70c640 1 --2- 172.21.15.83:0/3504829122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f9810aaf0 0x7f6f9810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.948+0000 7f6f9e70c640 1 --2- 172.21.15.83:0/3504829122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f98108180 0x7f6f9810a5b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.948+0000 7f6f9e70c640 1 -- 172.21.15.83:0/3504829122 >> 172.21.15.83:0/3504829122 conn(0x7f6f9806dfe0 msgr2=0x7f6f98070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.949+0000 7f6f9e70c640 1 -- 172.21.15.83:0/3504829122 shutdown_connections 2024-03-11T19:40:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.949+0000 7f6f9e70c640 1 -- 172.21.15.83:0/3504829122 wait complete. 2024-03-11T19:40:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.950+0000 7f6f9e70c640 1 Processor -- start 2024-03-11T19:40:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.950+0000 7f6f9e70c640 1 -- start start 2024-03-11T19:40:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.951+0000 7f6f9e70c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f98108180 0x7f6f981a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.951+0000 7f6f9e70c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f9810aaf0 0x7f6f981a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.951+0000 7f6f9e70c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9810d4a0 0x7f6f981ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.951+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f98108180 0x7f6f981a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.951+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f98108180 0x7f6f981a5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45446/0 (socket says 172.21.15.83:45446) 2024-03-11T19:40:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.951+0000 7f6f97fff640 1 -- 172.21.15.83:0/1981059768 learned_addr learned my addr 172.21.15.83:0/1981059768 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.951+0000 7f6f977fe640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f9810aaf0 0x7f6f981a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.951+0000 7f6f9cca9640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9810d4a0 0x7f6f981ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.951+0000 7f6f9e70c640 1 -- 172.21.15.83:0/1981059768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981ad920 con 0x7f6f98108180 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.951+0000 7f6f9e70c640 1 -- 172.21.15.83:0/1981059768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981ada90 con 0x7f6f9810aaf0 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.951+0000 7f6f9e70c640 1 -- 172.21.15.83:0/1981059768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f981adc50 con 0x7f6f9810d4a0 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.952+0000 7f6f9cca9640 1 -- 172.21.15.83:0/1981059768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f9810aaf0 msgr2=0x7f6f981a63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.952+0000 7f6f9cca9640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f9810aaf0 0x7f6f981a63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.952+0000 7f6f9cca9640 1 -- 172.21.15.83:0/1981059768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f98108180 msgr2=0x7f6f981a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.952+0000 7f6f9cca9640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f98108180 0x7f6f981a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.952+0000 7f6f9cca9640 1 -- 172.21.15.83:0/1981059768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f90009840 con 0x7f6f9810d4a0 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.952+0000 7f6f9cca9640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9810d4a0 0x7f6f981ad3e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6f90005fd0 tx=0x7f6f9000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.952+0000 7f6f957fa640 1 -- 172.21.15.83:0/1981059768 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90003b70 con 0x7f6f9810d4a0 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.952+0000 7f6f957fa640 1 -- 172.21.15.83:0/1981059768 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f9000ce70 con 0x7f6f9810d4a0 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.953+0000 7f6f957fa640 1 -- 172.21.15.83:0/1981059768 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90010630 con 0x7f6f9810d4a0 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.953+0000 7f6f9e70c640 1 -- 172.21.15.83:0/1981059768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f88009560 con 0x7f6f9810d4a0 2024-03-11T19:40:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.953+0000 7f6f9e70c640 1 -- 172.21.15.83:0/1981059768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f981ae1a0 con 0x7f6f9810d4a0 2024-03-11T19:40:41.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.956+0000 7f6fa13c0740 1 Processor -- start 2024-03-11T19:40:41.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6f957fa640 1 -- 172.21.15.83:0/1981059768 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6f90004020 con 0x7f6f9810d4a0 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6fa13c0740 1 -- start start 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6fa13c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca2270 0x55cd15ca2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6fa13c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 0x55cd15ca7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6fa13c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca7dd0 0x55cd15caa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6fa13c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd15c5e960 con 0x55cd15ca2270 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6fa13c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd15c12310 con 0x55cd15ca7dd0 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6fa13c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd15c0f840 con 0x55cd15ca2c60 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 0x55cd15ca7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 0x55cd15ca7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56828/0 (socket says 172.21.15.83:56828) 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6f97fff640 1 -- 172.21.15.83:0/915796018 learned_addr learned my addr 172.21.15.83:0/915796018 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.957+0000 7f6f977fe640 1 --2- 172.21.15.83:0/915796018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca2270 0x55cd15ca2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.958+0000 7f6f977fe640 1 --2- 172.21.15.83:0/915796018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca7dd0 0x55cd15caa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.958+0000 7f6f97fff640 1 -- 172.21.15.83:0/915796018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca7dd0 msgr2=0x55cd15caa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.958+0000 7f6f97fff640 1 --2- 172.21.15.83:0/915796018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca7dd0 0x55cd15caa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.958+0000 7f6f97fff640 1 -- 172.21.15.83:0/915796018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca2270 msgr2=0x55cd15ca2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.958+0000 7f6f97fff640 1 --2- 172.21.15.83:0/915796018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca2270 0x55cd15ca2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.958+0000 7f6f97fff640 1 -- 172.21.15.83:0/915796018 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd15bcd060 con 0x55cd15ca2c60 2024-03-11T19:40:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.958+0000 7f6f977fe640 1 --2- 172.21.15.83:0/915796018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca7dd0 0x55cd15caa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:40:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.959+0000 7f6f957fa640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f840a1b30 0x7f6f840a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.959+0000 7f6f97fff640 1 --2- 172.21.15.83:0/915796018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 0x55cd15ca7760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6f8802c9c0 tx=0x7f6f8802c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e0936f965cbf115 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.959+0000 7f6f957fa640 1 -- 172.21.15.83:0/1981059768 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6f90014070 con 0x7f6f9810d4a0 2024-03-11T19:40:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.959+0000 7f6f977fe640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f840a1b30 0x7f6f840a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.959+0000 7f6f8e7fc640 1 -- 172.21.15.83:0/915796018 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880048d0 con 0x55cd15ca2c60 2024-03-11T19:40:41.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.959+0000 7f6f8e7fc640 1 -- 172.21.15.83:0/915796018 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f88004a30 con 0x55cd15ca2c60 2024-03-11T19:40:41.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.959+0000 7f6f8e7fc640 1 -- 172.21.15.83:0/915796018 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880035f0 con 0x55cd15ca2c60 2024-03-11T19:40:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.960+0000 7f6fa13c0740 1 -- 172.21.15.83:0/915796018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 msgr2=0x55cd15ca7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.960+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/915796018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 0x55cd15ca7760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6f8802c9c0 tx=0x7f6f8802c9f0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.960+0000 7f6f977fe640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f840a1b30 0x7f6f840a3ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55cd15ca34c0 tx=0x7f6f8000a690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.961+0000 7f6fa13c0740 1 -- 172.21.15.83:0/915796018 shutdown_connections 2024-03-11T19:40:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.961+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/915796018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca7dd0 0x55cd15caa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.961+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/915796018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 0x55cd15ca7760 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.961+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/915796018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca2270 0x55cd15ca2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.961+0000 7f6fa13c0740 1 -- 172.21.15.83:0/915796018 >> 172.21.15.83:0/915796018 conn(0x55cd15c9d7e0 msgr2=0x55cd15c9dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.961+0000 7f6fa13c0740 1 -- 172.21.15.83:0/915796018 shutdown_connections 2024-03-11T19:40:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.961+0000 7f6fa13c0740 1 -- 172.21.15.83:0/915796018 wait complete. 2024-03-11T19:40:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.961+0000 7f6fa13c0740 1 -- start start 2024-03-11T19:40:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.962+0000 7f6fa13c0740 1 Processor -- start 2024-03-11T19:40:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.962+0000 7f6fa13c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2270 0x55cd15cac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.962+0000 7f6fa13c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2c60 0x55cd15cac840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.962+0000 7f6fa13c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 0x55cd15cb3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.962+0000 7f6fa13c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd15c0b020 con 0x55cd15ca7dd0 2024-03-11T19:40:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.962+0000 7f6fa13c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd158de430 con 0x55cd15ca2c60 2024-03-11T19:40:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.962+0000 7f6fa13c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd1589ae00 con 0x55cd15ca2270 2024-03-11T19:40:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.962+0000 7f6fa13c0740 10 client.0 _ll_register_callbacks cb 0x55cd15bb2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.963+0000 7f6f977fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2c60 0x55cd15cac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.963+0000 7f6f977fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2c60 0x55cd15cac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40500/0 (socket says 172.21.15.83:40500) 2024-03-11T19:40:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.963+0000 7f6f977fe640 1 -- 172.21.15.83:0/3628743195 learned_addr learned my addr 172.21.15.83:0/3628743195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.963+0000 7f6f9cca9640 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 0x55cd15cb3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.963+0000 7f6f97fff640 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2270 0x55cd15cac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.963+0000 7f6f977fe640 1 -- 172.21.15.83:0/3628743195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2270 msgr2=0x55cd15cac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.963+0000 7f6f977fe640 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2270 0x55cd15cac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.963+0000 7f6f977fe640 1 -- 172.21.15.83:0/3628743195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 msgr2=0x55cd15cb3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.963+0000 7f6f977fe640 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 0x55cd15cb3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.963+0000 7f6f977fe640 1 -- 172.21.15.83:0/3628743195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9810e4b0 con 0x55cd15ca2c60 2024-03-11T19:40:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.963+0000 7f6f97fff640 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2270 0x55cd15cac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:40:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.964+0000 7f6f977fe640 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2c60 0x55cd15cac840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f80011e10 tx=0x7f6f8000a2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e42af607f06ccbd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.964+0000 7f6f777fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:40:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.964+0000 7f6f777fe640 1 -- 172.21.15.83:0/3628743195 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80010bc0 con 0x55cd15ca2c60 2024-03-11T19:40:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.964+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3628743195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd1590a7e0 con 0x55cd15ca2c60 2024-03-11T19:40:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.965+0000 7f6f777fe640 1 -- 172.21.15.83:0/3628743195 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6f80010d20 con 0x55cd15ca2c60 2024-03-11T19:40:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.965+0000 7f6f777fe640 1 -- 172.21.15.83:0/3628743195 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f8002e030 con 0x55cd15ca2c60 2024-03-11T19:40:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.965+0000 7f6fa13c0740 10 client.25133 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.965+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3628743195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cd15c096a0 con 0x55cd15ca2c60 2024-03-11T19:40:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.966+0000 7f6f777fe640 1 -- 172.21.15.83:0/3628743195 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cd1590a7e0 con 0x55cd15ca2c60 2024-03-11T19:40:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.966+0000 7f6f777fe640 1 -- 172.21.15.83:0/3628743195 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6f80004d20 con 0x55cd15ca2c60 2024-03-11T19:40:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.966+0000 7f6fa13c0740 10 client.25133 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.966+0000 7f6fa13c0740 10 client.25133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.966+0000 7f6fa13c0740 20 client.25133 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.966+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd15cc9620 0x55cd15ccba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.966+0000 7f6fa13c0740 4 client.25133 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.966+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3628743195 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd15947720 con 0x55cd15cc9620 2024-03-11T19:40:41.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.966+0000 7f6f97fff640 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd15cc9620 0x55cd15ccba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.967+0000 7f6f97fff640 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd15cc9620 0x55cd15ccba40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d530b91d922b47 server_cookie=cd4b217cc9ddc68e in_seq=0 out_seq=0 2024-03-11T19:40:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.968+0000 7f6f777fe640 10 client.25133 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.968+0000 7f6f777fe640 1 -- 172.21.15.83:0/3628743195 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd15947720 con 0x55cd15cc9620 2024-03-11T19:40:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.968+0000 7f6f777fe640 10 client.25133 handle_command_reply: tid=0 2024-03-11T19:40:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.968+0000 7f6fa13c0740 1 client.25133 shutdown 2024-03-11T19:40:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.969+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3628743195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2c60 msgr2=0x55cd15cac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.969+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2c60 0x55cd15cac840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f80011e10 tx=0x7f6f8000a2d0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.969+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3628743195 shutdown_connections 2024-03-11T19:40:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.969+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd15cc9620 0x55cd15ccba40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.970+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 0x55cd15cb3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.970+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2c60 0x55cd15cac840 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.970+0000 7f6f777fe640 0 client.25133 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.970+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3628743195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2270 0x55cd15cac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.970+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3628743195 >> 172.21.15.83:0/3628743195 conn(0x55cd15c9d7e0 msgr2=0x55cd15c9cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.970+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3628743195 shutdown_connections 2024-03-11T19:40:41.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.970+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3628743195 wait complete. 2024-03-11T19:40:41.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.970+0000 7f6fa13c0740 20 client.25133 trim_cache size 0 max 16384 2024-03-11T19:40:41.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.992+0000 7f6fa13c0740 1 Processor -- start 2024-03-11T19:40:41.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.992+0000 7f6fa13c0740 1 -- start start 2024-03-11T19:40:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.993+0000 7f6fa13c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2270 0x55cd15cac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.993+0000 7f6fa13c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 0x55cd15cb3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.993+0000 7f6fa13c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 0x55cd15cac840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.993+0000 7f6fa13c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd15b90130 con 0x55cd15ca7dd0 2024-03-11T19:40:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.993+0000 7f6fa13c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd15947720 con 0x55cd15ca2270 2024-03-11T19:40:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.993+0000 7f6fa13c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd1590a7e0 con 0x55cd15ca2c60 2024-03-11T19:40:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.994+0000 7f6f9cca9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 0x55cd15cac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.994+0000 7f6f9cca9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 0x55cd15cac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45488/0 (socket says 172.21.15.83:45488) 2024-03-11T19:40:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.994+0000 7f6f9cca9640 1 -- 172.21.15.83:0/1460410775 learned_addr learned my addr 172.21.15.83:0/1460410775 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.994+0000 7f6f977fe640 1 --2- 172.21.15.83:0/1460410775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 0x55cd15cb3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.994+0000 7f6f9cca9640 1 -- 172.21.15.83:0/1460410775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 msgr2=0x55cd15cb3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.994+0000 7f6f9cca9640 1 --2- 172.21.15.83:0/1460410775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 0x55cd15cb3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.994+0000 7f6f9cca9640 1 -- 172.21.15.83:0/1460410775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2270 msgr2=0x55cd15cac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.994+0000 7f6f9cca9640 1 --2- 172.21.15.83:0/1460410775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2270 0x55cd15cac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.994+0000 7f6f9cca9640 1 -- 172.21.15.83:0/1460410775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd15ba5c20 con 0x55cd15ca7dd0 2024-03-11T19:40:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.994+0000 7f6f977fe640 1 --2- 172.21.15.83:0/1460410775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 0x55cd15cb3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.995+0000 7f6f9cca9640 1 --2- 172.21.15.83:0/1460410775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 0x55cd15cac840 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6f9009a6b0 tx=0x7f6f9009a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a020569ad5acc999 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.995+0000 7f6f8dffb640 1 -- 172.21.15.83:0/1460410775 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9009eb00 con 0x55cd15ca7dd0 2024-03-11T19:40:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.995+0000 7f6f8dffb640 1 -- 172.21.15.83:0/1460410775 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f900a3150 con 0x55cd15ca7dd0 2024-03-11T19:40:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.995+0000 7f6f8dffb640 1 -- 172.21.15.83:0/1460410775 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f900b3220 con 0x55cd15ca7dd0 2024-03-11T19:40:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.996+0000 7f6fa13c0740 1 -- 172.21.15.83:0/1460410775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 msgr2=0x55cd15cac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.996+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/1460410775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 0x55cd15cac840 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6f9009a6b0 tx=0x7f6f9009a410 comp rx=0 tx=0).stop 2024-03-11T19:40:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.996+0000 7f6fa13c0740 1 -- 172.21.15.83:0/1460410775 shutdown_connections 2024-03-11T19:40:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.996+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/1460410775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca7dd0 0x55cd15cac840 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.997+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/1460410775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca2c60 0x55cd15cb3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.997+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/1460410775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2270 0x55cd15cac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.997+0000 7f6fa13c0740 1 -- 172.21.15.83:0/1460410775 >> 172.21.15.83:0/1460410775 conn(0x55cd15c9d7e0 msgr2=0x55cd15c9dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.997+0000 7f6fa13c0740 1 -- 172.21.15.83:0/1460410775 shutdown_connections 2024-03-11T19:40:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.997+0000 7f6fa13c0740 1 -- 172.21.15.83:0/1460410775 wait complete. 2024-03-11T19:40:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.997+0000 7f6fa13c0740 1 -- start start 2024-03-11T19:40:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.997+0000 7f6fa13c0740 1 Processor -- start 2024-03-11T19:40:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.998+0000 7f6fa13c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2270 0x55cd15cb22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.998+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2270 0x55cd15cb22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.998+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2270 0x55cd15cb22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40516/0 (socket says 172.21.15.83:40516) 2024-03-11T19:40:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.998+0000 7f6fa13c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca2c60 0x55cd15cb2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.998+0000 7f6f97fff640 1 -- 172.21.15.83:0/3489489044 learned_addr learned my addr 172.21.15.83:0/3489489044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.998+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca7dd0 0x55cd15cb2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.998+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3489489044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd15d1fc80 con 0x55cd15ca2c60 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.998+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3489489044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd15d1fdf0 con 0x55cd15ca2270 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.998+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3489489044 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd15d1ffb0 con 0x55cd15ca7dd0 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.999+0000 7f6fa13c0740 10 client.0 _ll_register_callbacks cb 0x7f6f88009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.999+0000 7f6f97fff640 1 -- 172.21.15.83:0/3489489044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca7dd0 msgr2=0x55cd15cb2d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.999+0000 7f6f97fff640 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca7dd0 0x55cd15cb2d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.999+0000 7f6f97fff640 1 -- 172.21.15.83:0/3489489044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca2c60 msgr2=0x55cd15cb2830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.999+0000 7f6f97fff640 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca2c60 0x55cd15cb2830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.999+0000 7f6f97fff640 1 -- 172.21.15.83:0/3489489044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd15c096a0 con 0x55cd15ca2270 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.999+0000 7f6f97fff640 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2270 0x55cd15cb22f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6f8802c220 tx=0x7f6f8802c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28af1d49de9c026c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.999+0000 7f6f777fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:40:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.999+0000 7f6f777fe640 1 -- 172.21.15.83:0/3489489044 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880048d0 con 0x55cd15ca2270 2024-03-11T19:40:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:41.999+0000 7f6f777fe640 1 -- 172.21.15.83:0/3489489044 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6f8804f070 con 0x55cd15ca2270 2024-03-11T19:40:42.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.000+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3489489044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f88004a30 con 0x55cd15ca2270 2024-03-11T19:40:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.000+0000 7f6f777fe640 1 -- 172.21.15.83:0/3489489044 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f88005600 con 0x55cd15ca2270 2024-03-11T19:40:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.000+0000 7f6fa13c0740 10 client.25136 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.000+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3489489044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f6f981ae1a0 con 0x55cd15ca2270 2024-03-11T19:40:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.000+0000 7f6f777fe640 1 -- 172.21.15.83:0/3489489044 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f88004a30 con 0x55cd15ca2270 2024-03-11T19:40:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.001+0000 7f6f777fe640 1 -- 172.21.15.83:0/3489489044 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6f8803c050 con 0x55cd15ca2270 2024-03-11T19:40:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.001+0000 7f6fa13c0740 10 client.25136 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.001+0000 7f6fa13c0740 10 client.25136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.001+0000 7f6fa13c0740 20 client.25136 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.001+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd15d21670 0x55cd15d23a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.001+0000 7f6fa13c0740 4 client.25136 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.001+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3489489044 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cd15996890 con 0x55cd15d21670 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.001+0000 7f6f977fe640 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd15d21670 0x55cd15d23a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.002+0000 7f6f777fe640 10 client.25136 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.002+0000 7f6f977fe640 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd15d21670 0x55cd15d23a90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e9d14a3bb059a3 server_cookie=b243bd20315ecf4b in_seq=0 out_seq=0 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.002+0000 7f6f777fe640 1 -- 172.21.15.83:0/3489489044 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cd15996890 con 0x55cd15d21670 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.002+0000 7f6f777fe640 10 client.25136 handle_command_reply: tid=0 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.002+0000 7f6fa13c0740 1 client.25136 shutdown 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.002+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3489489044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2270 msgr2=0x55cd15cb22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:42.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.002+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2270 0x55cd15cb22f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6f8802c220 tx=0x7f6f8802c9f0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.003+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3489489044 shutdown_connections 2024-03-11T19:40:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.003+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd15d21670 0x55cd15d23a90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.003+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd15ca7dd0 0x55cd15cb2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.003+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd15ca2c60 0x55cd15cb2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.003+0000 7f6fa13c0740 1 --2- 172.21.15.83:0/3489489044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd15ca2270 0x55cd15cb22f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.003+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3489489044 >> 172.21.15.83:0/3489489044 conn(0x55cd15c9d7e0 msgr2=0x55cd15cac2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:42.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.003+0000 7f6f777fe640 0 client.25136 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:42.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.004+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3489489044 shutdown_connections 2024-03-11T19:40:42.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.004+0000 7f6fa13c0740 1 -- 172.21.15.83:0/3489489044 wait complete. 2024-03-11T19:40:42.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.004+0000 7f6fa13c0740 20 client.25136 trim_cache size 0 max 16384 2024-03-11T19:40:42.010 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:40:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.005+0000 7f6f9e70c640 1 -- 172.21.15.83:0/1981059768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f840a1b30 msgr2=0x7f6f840a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.005+0000 7f6f9e70c640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f840a1b30 0x7f6f840a3ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55cd15ca34c0 tx=0x7f6f8000a690 comp rx=0 tx=0).stop 2024-03-11T19:40:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.005+0000 7f6f9e70c640 1 -- 172.21.15.83:0/1981059768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9810d4a0 msgr2=0x7f6f981ad3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.005+0000 7f6f9e70c640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9810d4a0 0x7f6f981ad3e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6f90005fd0 tx=0x7f6f9000c6a0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.006+0000 7f6f9e70c640 1 -- 172.21.15.83:0/1981059768 shutdown_connections 2024-03-11T19:40:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.006+0000 7f6f9e70c640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f840a1b30 0x7f6f840a3ff0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.006+0000 7f6f9e70c640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f9810d4a0 0x7f6f981ad3e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.006+0000 7f6f9e70c640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f9810aaf0 0x7f6f981a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.006+0000 7f6f9e70c640 1 --2- 172.21.15.83:0/1981059768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f98108180 0x7f6f981a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.006+0000 7f6f9e70c640 1 -- 172.21.15.83:0/1981059768 >> 172.21.15.83:0/1981059768 conn(0x7f6f9806dfe0 msgr2=0x7f6f9810b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.006+0000 7f6f9e70c640 1 -- 172.21.15.83:0/1981059768 shutdown_connections 2024-03-11T19:40:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:42.006+0000 7f6f9e70c640 1 -- 172.21.15.83:0/1981059768 wait complete. 2024-03-11T19:40:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 85.0 s, 5861.6 tps, lat 17.074 ms stddev 18.328 2024-03-11T19:40:43.028 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:40:43.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:40:43.029 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:40:43.062 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:42 smithi083 ceph-mon[31864]: pgmap v394: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 MiB/s wr, 869 op/s 2024-03-11T19:40:43.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:42 smithi137 ceph-mon[36220]: pgmap v394: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 MiB/s wr, 869 op/s 2024-03-11T19:40:43.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:42 smithi204 ceph-mon[36739]: pgmap v394: 97 pgs: 97 active+clean; 1.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 MiB/s wr, 869 op/s 2024-03-11T19:40:43.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.187+0000 7facbb78c640 1 -- 172.21.15.83:0/47659847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb41077a0 msgr2=0x7facb4107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.187+0000 7facbb78c640 1 --2- 172.21.15.83:0/47659847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb41077a0 0x7facb4107bc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7facb0009900 tx=0x7facb002bc30 comp rx=0 tx=0).stop 2024-03-11T19:40:43.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.188+0000 7facbb78c640 1 -- 172.21.15.83:0/47659847 shutdown_connections 2024-03-11T19:40:43.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.188+0000 7facbb78c640 1 --2- 172.21.15.83:0/47659847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facb4109350 0x7facb410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.188+0000 7facbb78c640 1 --2- 172.21.15.83:0/47659847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb4108990 0x7facb4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.188+0000 7facbb78c640 1 --2- 172.21.15.83:0/47659847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb41077a0 0x7facb4107bc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.188+0000 7facbb78c640 1 -- 172.21.15.83:0/47659847 >> 172.21.15.83:0/47659847 conn(0x7facb4076740 msgr2=0x7facb4078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.188+0000 7facbb78c640 1 -- 172.21.15.83:0/47659847 shutdown_connections 2024-03-11T19:40:43.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.189+0000 7facbb78c640 1 -- 172.21.15.83:0/47659847 wait complete. 2024-03-11T19:40:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facbb78c640 1 Processor -- start 2024-03-11T19:40:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facbb78c640 1 -- start start 2024-03-11T19:40:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facbb78c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb41077a0 0x7facb419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facbb78c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facb4108990 0x7facb419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facbb78c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb4109350 0x7facb41a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facbb78c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb41a5370 con 0x7facb4109350 2024-03-11T19:40:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facbb78c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb41a54e0 con 0x7facb4108990 2024-03-11T19:40:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facbb78c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb41a56a0 con 0x7facb41077a0 2024-03-11T19:40:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb9d29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb4109350 0x7facb41a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb9528640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb41077a0 0x7facb419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb8d27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facb4108990 0x7facb419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb9d29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb4109350 0x7facb41a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45508/0 (socket says 172.21.15.83:45508) 2024-03-11T19:40:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb9d29640 1 -- 172.21.15.83:0/2679529358 learned_addr learned my addr 172.21.15.83:0/2679529358 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb9528640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb41077a0 0x7facb419d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56856/0 (socket says 172.21.15.83:56856) 2024-03-11T19:40:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb9d29640 1 -- 172.21.15.83:0/2679529358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb41077a0 msgr2=0x7facb419d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb9d29640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb41077a0 0x7facb419d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb9d29640 1 -- 172.21.15.83:0/2679529358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facb4108990 msgr2=0x7facb419ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb9d29640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facb4108990 0x7facb419ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb9d29640 1 -- 172.21.15.83:0/2679529358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb0009560 con 0x7facb4109350 2024-03-11T19:40:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.190+0000 7facb8d27640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facb4108990 0x7facb419ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.191+0000 7facb9d29640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb4109350 0x7facb41a4e30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7facac00eb10 tx=0x7facac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.191+0000 7facaa7fc640 1 -- 172.21.15.83:0/2679529358 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac004b20 con 0x7facb4109350 2024-03-11T19:40:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.191+0000 7facaa7fc640 1 -- 172.21.15.83:0/2679529358 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facac004c80 con 0x7facb4109350 2024-03-11T19:40:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.191+0000 7facbb78c640 1 -- 172.21.15.83:0/2679529358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb41a5980 con 0x7facb4109350 2024-03-11T19:40:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.191+0000 7facaa7fc640 1 -- 172.21.15.83:0/2679529358 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac010650 con 0x7facb4109350 2024-03-11T19:40:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.191+0000 7facbb78c640 1 -- 172.21.15.83:0/2679529358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb41a5e90 con 0x7facb4109350 2024-03-11T19:40:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.196+0000 7facbe366740 1 Processor -- start 2024-03-11T19:40:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.196+0000 7facaa7fc640 1 -- 172.21.15.83:0/2679529358 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7facac010810 con 0x7facb4109350 2024-03-11T19:40:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.196+0000 7facbe366740 1 -- start start 2024-03-11T19:40:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.197+0000 7facaa7fc640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faca00a1b80 0x7faca00a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facaa7fc640 1 -- 172.21.15.83:0/2679529358 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7facac014070 con 0x7facb4109350 2024-03-11T19:40:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facbe366740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a909f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facbe366740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a909f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facbe366740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a909fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facbe366740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a90964f90 con 0x559a909f7050 2024-03-11T19:40:43.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facbe366740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a909624c0 con 0x559a909f6690 2024-03-11T19:40:43.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facbe366740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9095dca0 con 0x559a909f54c0 2024-03-11T19:40:43.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb8d27640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faca00a1b80 0x7faca00a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb9528640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a909f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb9528640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a909f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56868/0 (socket says 172.21.15.83:56868) 2024-03-11T19:40:43.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb9528640 1 -- 172.21.15.83:0/1147058110 learned_addr learned my addr 172.21.15.83:0/1147058110 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:43.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb8d27640 1 --2- 172.21.15.83:0/1147058110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a909f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb9d29640 1 --2- 172.21.15.83:0/1147058110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a909fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb9528640 1 -- 172.21.15.83:0/1147058110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 msgr2=0x559a909f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb9528640 1 --2- 172.21.15.83:0/1147058110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a909f6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb9528640 1 -- 172.21.15.83:0/1147058110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 msgr2=0x559a909fd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb9528640 1 --2- 172.21.15.83:0/1147058110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a909fd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb9528640 1 -- 172.21.15.83:0/1147058110 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a909200b0 con 0x559a909f54c0 2024-03-11T19:40:43.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb8d27640 1 --2- 172.21.15.83:0/1147058110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a909f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:40:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb9528640 1 --2- 172.21.15.83:0/1147058110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a909f58e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7facb0009900 tx=0x7facb0005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=763d45c18869eff6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.198+0000 7facb8d27640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faca00a1b80 0x7faca00a4040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7faca4009fd0 tx=0x7faca4009d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.199+0000 7fac9f7fe640 1 -- 172.21.15.83:0/1147058110 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb002ca20 con 0x559a909f54c0 2024-03-11T19:40:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.199+0000 7fac9f7fe640 1 -- 172.21.15.83:0/1147058110 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facb002cb80 con 0x559a909f54c0 2024-03-11T19:40:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.199+0000 7fac9f7fe640 1 -- 172.21.15.83:0/1147058110 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb00051b0 con 0x559a909f54c0 2024-03-11T19:40:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.199+0000 7facbe366740 1 -- 172.21.15.83:0/1147058110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 msgr2=0x559a909f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.199+0000 7facbe366740 1 --2- 172.21.15.83:0/1147058110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a909f58e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7facb0009900 tx=0x7facb0005d50 comp rx=0 tx=0).stop 2024-03-11T19:40:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.200+0000 7facbe366740 1 -- 172.21.15.83:0/1147058110 shutdown_connections 2024-03-11T19:40:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.200+0000 7facbe366740 1 --2- 172.21.15.83:0/1147058110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a909fd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.200+0000 7facbe366740 1 --2- 172.21.15.83:0/1147058110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a909f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.200+0000 7facbe366740 1 --2- 172.21.15.83:0/1147058110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a909f58e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.200+0000 7facbe366740 1 -- 172.21.15.83:0/1147058110 >> 172.21.15.83:0/1147058110 conn(0x559a909f06a0 msgr2=0x559a909f0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.200+0000 7facbe366740 1 -- 172.21.15.83:0/1147058110 shutdown_connections 2024-03-11T19:40:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.200+0000 7facbe366740 1 -- 172.21.15.83:0/1147058110 wait complete. 2024-03-11T19:40:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.200+0000 7facbe366740 1 -- start start 2024-03-11T19:40:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facbe366740 1 Processor -- start 2024-03-11T19:40:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facbe366740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f54c0 0x559a909ff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facbe366740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f6690 0x559a909ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facbe366740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a90a06bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facbe366740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a90902f40 con 0x559a909f7050 2024-03-11T19:40:43.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facbe366740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a906314b0 con 0x559a909f54c0 2024-03-11T19:40:43.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facbe366740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a908e3670 con 0x559a909f6690 2024-03-11T19:40:43.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facbe366740 10 client.0 _ll_register_callbacks cb 0x559a90950f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:43.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facb9528640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f54c0 0x559a909ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facb9d29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a90a06bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facb8d27640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f6690 0x559a909ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facb9528640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f54c0 0x559a909ff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40560/0 (socket says 172.21.15.83:40560) 2024-03-11T19:40:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facb9528640 1 -- 172.21.15.83:0/4257446389 learned_addr learned my addr 172.21.15.83:0/4257446389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.201+0000 7facb8d27640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f6690 0x559a909ffb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56884/0 (socket says 172.21.15.83:56884) 2024-03-11T19:40:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.202+0000 7facb9528640 1 -- 172.21.15.83:0/4257446389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f6690 msgr2=0x559a909ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.202+0000 7facb9528640 1 --2- 172.21.15.83:0/4257446389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f6690 0x559a909ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.202+0000 7facb9528640 1 -- 172.21.15.83:0/4257446389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 msgr2=0x559a90a06bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.202+0000 7facb9528640 1 --2- 172.21.15.83:0/4257446389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a90a06bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.202+0000 7facb9528640 1 -- 172.21.15.83:0/4257446389 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb41104e0 con 0x559a909f54c0 2024-03-11T19:40:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.202+0000 7facb9528640 1 --2- 172.21.15.83:0/4257446389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f54c0 0x559a909ff640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facb002c170 tx=0x7facb0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c3255a0e869890 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.202+0000 7fac9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:40:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.203+0000 7fac9cff9640 1 -- 172.21.15.83:0/4257446389 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb00053c0 con 0x559a909f54c0 2024-03-11T19:40:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.203+0000 7fac9cff9640 1 -- 172.21.15.83:0/4257446389 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7facb004d070 con 0x559a909f54c0 2024-03-11T19:40:43.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.203+0000 7facbe366740 1 -- 172.21.15.83:0/4257446389 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a90937f40 con 0x559a909f54c0 2024-03-11T19:40:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.203+0000 7fac9cff9640 1 -- 172.21.15.83:0/4257446389 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7facb0057030 con 0x559a909f54c0 2024-03-11T19:40:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.203+0000 7facbe366740 10 client.25148 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.203+0000 7facbe366740 1 -- 172.21.15.83:0/4257446389 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559a9095c320 con 0x559a909f54c0 2024-03-11T19:40:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.204+0000 7fac9cff9640 1 -- 172.21.15.83:0/4257446389 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559a90937f40 con 0x559a909f54c0 2024-03-11T19:40:43.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.204+0000 7fac9cff9640 1 -- 172.21.15.83:0/4257446389 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7facb003c050 con 0x559a909f54c0 2024-03-11T19:40:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.204+0000 7facbe366740 10 client.25148 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.204+0000 7facbe366740 10 client.25148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.204+0000 7facbe366740 20 client.25148 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.204+0000 7facbe366740 1 --2- 172.21.15.83:0/4257446389 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559a90a0b4f0 0x559a90a0d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.204+0000 7facbe366740 4 client.25148 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.204+0000 7facbe366740 1 -- 172.21.15.83:0/4257446389 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a90954310 con 0x559a90a0b4f0 2024-03-11T19:40:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.204+0000 7facb8d27640 1 --2- 172.21.15.83:0/4257446389 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559a90a0b4f0 0x559a90a0d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.204+0000 7fac9cff9640 10 client.25148 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.205+0000 7facb8d27640 1 --2- 172.21.15.83:0/4257446389 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559a90a0b4f0 0x559a90a0d910 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e846a16349041002 server_cookie=99ee1eecd8aaa0f2 in_seq=0 out_seq=0 2024-03-11T19:40:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.205+0000 7fac9cff9640 1 -- 172.21.15.83:0/4257446389 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559a90954310 con 0x559a90a0b4f0 2024-03-11T19:40:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.205+0000 7fac9cff9640 10 client.25148 handle_command_reply: tid=0 2024-03-11T19:40:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.205+0000 7facbe366740 1 client.25148 shutdown 2024-03-11T19:40:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.206+0000 7facbe366740 1 -- 172.21.15.83:0/4257446389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f54c0 msgr2=0x559a909ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.206+0000 7facbe366740 1 --2- 172.21.15.83:0/4257446389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f54c0 0x559a909ff640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facb002c170 tx=0x7facb0004a40 comp rx=0 tx=0).stop 2024-03-11T19:40:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.206+0000 7facbe366740 1 -- 172.21.15.83:0/4257446389 shutdown_connections 2024-03-11T19:40:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.206+0000 7facbe366740 1 --2- 172.21.15.83:0/4257446389 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559a90a0b4f0 0x559a90a0d910 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.206+0000 7facbe366740 1 --2- 172.21.15.83:0/4257446389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a90a06bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.206+0000 7facbe366740 1 --2- 172.21.15.83:0/4257446389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f6690 0x559a909ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.206+0000 7fac9cff9640 0 client.25148 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:43.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.206+0000 7facbe366740 1 --2- 172.21.15.83:0/4257446389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f54c0 0x559a909ff640 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.206+0000 7facbe366740 1 -- 172.21.15.83:0/4257446389 >> 172.21.15.83:0/4257446389 conn(0x559a909f06a0 msgr2=0x559a909ef8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.206+0000 7facbe366740 1 -- 172.21.15.83:0/4257446389 shutdown_connections 2024-03-11T19:40:43.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.206+0000 7facbe366740 1 -- 172.21.15.83:0/4257446389 wait complete. 2024-03-11T19:40:43.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.207+0000 7facbe366740 20 client.25148 trim_cache size 0 max 16384 2024-03-11T19:40:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.230+0000 7facbe366740 1 Processor -- start 2024-03-11T19:40:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.230+0000 7facbe366740 1 -- start start 2024-03-11T19:40:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.230+0000 7facbe366740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a90a12270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.230+0000 7facbe366740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a06ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.230+0000 7facbe366740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a909fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facb8d27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a06ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facbe366740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a90a00830 con 0x559a909f7050 2024-03-11T19:40:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facb8d27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a06ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40568/0 (socket says 172.21.15.83:40568) 2024-03-11T19:40:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facb8d27640 1 -- 172.21.15.83:0/1000737009 learned_addr learned my addr 172.21.15.83:0/1000737009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facbe366740 1 -- 172.21.15.83:0/1000737009 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a909f6170 con 0x559a909f6690 2024-03-11T19:40:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facbe366740 1 -- 172.21.15.83:0/1000737009 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a90a080b0 con 0x559a909f54c0 2024-03-11T19:40:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facb9d29640 1 --2- 172.21.15.83:0/1000737009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a909fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facb8d27640 1 -- 172.21.15.83:0/1000737009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 msgr2=0x559a90a12270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facb8d27640 1 --2- 172.21.15.83:0/1000737009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a90a12270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facb8d27640 1 -- 172.21.15.83:0/1000737009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 msgr2=0x559a909fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facb8d27640 1 --2- 172.21.15.83:0/1000737009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a909fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facb8d27640 1 -- 172.21.15.83:0/1000737009 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a908f8c00 con 0x559a909f6690 2024-03-11T19:40:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.231+0000 7facb8d27640 1 --2- 172.21.15.83:0/1000737009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a06ae0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7facb419ed90 tx=0x7faca4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cea671eae87bfcf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.232+0000 7fac9effd640 1 -- 172.21.15.83:0/1000737009 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca4026040 con 0x559a909f6690 2024-03-11T19:40:43.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.232+0000 7fac9effd640 1 -- 172.21.15.83:0/1000737009 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faca4028040 con 0x559a909f6690 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.232+0000 7fac9effd640 1 -- 172.21.15.83:0/1000737009 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca402c7e0 con 0x559a909f6690 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.233+0000 7facbe366740 1 -- 172.21.15.83:0/1000737009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 msgr2=0x559a90a06ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.233+0000 7facbe366740 1 --2- 172.21.15.83:0/1000737009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a06ae0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7facb419ed90 tx=0x7faca4004730 comp rx=0 tx=0).stop 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.233+0000 7facbe366740 1 -- 172.21.15.83:0/1000737009 shutdown_connections 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.233+0000 7facbe366740 1 --2- 172.21.15.83:0/1000737009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a909fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.233+0000 7facbe366740 1 --2- 172.21.15.83:0/1000737009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a06ae0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.233+0000 7facbe366740 1 --2- 172.21.15.83:0/1000737009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a90a12270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.233+0000 7facbe366740 1 -- 172.21.15.83:0/1000737009 >> 172.21.15.83:0/1000737009 conn(0x559a909f06a0 msgr2=0x559a909f0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.233+0000 7facbe366740 1 -- 172.21.15.83:0/1000737009 shutdown_connections 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.233+0000 7facbe366740 1 -- 172.21.15.83:0/1000737009 wait complete. 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.233+0000 7facbe366740 1 -- start start 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.233+0000 7facbe366740 1 Processor -- start 2024-03-11T19:40:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facbe366740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a90a053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facbe366740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a05930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facbe366740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a90a05e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facbe366740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a90a72a20 con 0x559a909f7050 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facbe366740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a90a72b90 con 0x559a909f6690 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facbe366740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a90a72d50 con 0x559a909f54c0 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facbe366740 10 client.0 _ll_register_callbacks cb 0x7faca4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb8d27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a05930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb8d27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a05930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40584/0 (socket says 172.21.15.83:40584) 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb9528640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a90a053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb8d27640 1 -- 172.21.15.83:0/2781140962 learned_addr learned my addr 172.21.15.83:0/2781140962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb9d29640 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a90a05e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb8d27640 1 -- 172.21.15.83:0/2781140962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 msgr2=0x559a90a053f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb8d27640 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a90a053f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb8d27640 1 -- 172.21.15.83:0/2781140962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 msgr2=0x559a90a05e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb8d27640 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a90a05e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb8d27640 1 -- 172.21.15.83:0/2781140962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca401c070 con 0x559a909f6690 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb9528640 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a90a053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7facb8d27640 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a05930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faca4009460 tx=0x7faca4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc8ca35ff8452ef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.234+0000 7fac9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.235+0000 7fac9cff9640 1 -- 172.21.15.83:0/2781140962 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca4026040 con 0x559a909f6690 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.235+0000 7fac9cff9640 1 -- 172.21.15.83:0/2781140962 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7faca402d780 con 0x559a909f6690 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.235+0000 7facbe366740 1 -- 172.21.15.83:0/2781140962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a90983900 con 0x559a909f6690 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.235+0000 7fac9cff9640 1 -- 172.21.15.83:0/2781140962 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faca4021dc0 con 0x559a909f6690 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.235+0000 7facbe366740 10 client.25154 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.235+0000 7facbe366740 1 -- 172.21.15.83:0/2781140962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559a9091e910 con 0x559a909f6690 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.236+0000 7fac9cff9640 1 -- 172.21.15.83:0/2781140962 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faca4046030 con 0x559a909f6690 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.236+0000 7fac9cff9640 1 -- 172.21.15.83:0/2781140962 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faca403d020 con 0x559a909f6690 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.236+0000 7facbe366740 10 client.25154 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.236+0000 7facbe366740 10 client.25154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.236+0000 7facbe366740 20 client.25154 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.236+0000 7facbe366740 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559a90a0b4f0 0x559a90a76820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.236+0000 7facbe366740 4 client.25154 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.236+0000 7facbe366740 1 -- 172.21.15.83:0/2781140962 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559a90983900 con 0x559a90a0b4f0 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.236+0000 7facb9528640 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559a90a0b4f0 0x559a90a76820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.236+0000 7fac9cff9640 10 client.25154 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.237+0000 7facb9528640 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559a90a0b4f0 0x559a90a76820 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0c7522f98b78ad server_cookie=79dcc95b34ddcc32 in_seq=0 out_seq=0 2024-03-11T19:40:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.238+0000 7fac9cff9640 1 -- 172.21.15.83:0/2781140962 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559a90983900 con 0x559a90a0b4f0 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.238+0000 7fac9cff9640 10 client.25154 handle_command_reply: tid=0 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.238+0000 7facbe366740 1 client.25154 shutdown 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.238+0000 7facbe366740 1 -- 172.21.15.83:0/2781140962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 msgr2=0x559a90a05930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.238+0000 7facbe366740 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a05930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faca4009460 tx=0x7faca4005fd0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7facbe366740 1 -- 172.21.15.83:0/2781140962 shutdown_connections 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7facbe366740 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559a90a0b4f0 0x559a90a76820 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7facbe366740 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a909f7050 0x559a90a05e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7facbe366740 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a909f6690 0x559a90a05930 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7facbe366740 1 --2- 172.21.15.83:0/2781140962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a909f54c0 0x559a90a053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7fac9cff9640 0 client.25154 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7facbe366740 1 -- 172.21.15.83:0/2781140962 >> 172.21.15.83:0/2781140962 conn(0x559a909f06a0 msgr2=0x559a90a12270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7facbe366740 1 -- 172.21.15.83:0/2781140962 shutdown_connections 2024-03-11T19:40:43.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7facbe366740 1 -- 172.21.15.83:0/2781140962 wait complete. 2024-03-11T19:40:43.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7facbe366740 20 client.25154 trim_cache size 0 max 16384 2024-03-11T19:40:43.250 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:40:43.250 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:40:43.250 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "93b8a617-0262-4193-885f-97c0ed95dbef", 2024-03-11T19:40:43.250 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:40:43.250 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:40:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7facbb78c640 1 -- 172.21.15.83:0/2679529358 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faca00a1b80 msgr2=0x7faca00a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.239+0000 7facbb78c640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faca00a1b80 0x7faca00a4040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7faca4009fd0 tx=0x7faca4009d30 comp rx=0 tx=0).stop 2024-03-11T19:40:43.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.240+0000 7facbb78c640 1 -- 172.21.15.83:0/2679529358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb4109350 msgr2=0x7facb41a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.240+0000 7facbb78c640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb4109350 0x7facb41a4e30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7facac00eb10 tx=0x7facac00c2d0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.240+0000 7facbb78c640 1 -- 172.21.15.83:0/2679529358 shutdown_connections 2024-03-11T19:40:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.240+0000 7facbb78c640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faca00a1b80 0x7faca00a4040 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.240+0000 7facbb78c640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facb4109350 0x7facb41a4e30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.240+0000 7facbb78c640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7facb4108990 0x7facb419ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.240+0000 7facbb78c640 1 --2- 172.21.15.83:0/2679529358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb41077a0 0x7facb419d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.240+0000 7facbb78c640 1 -- 172.21.15.83:0/2679529358 >> 172.21.15.83:0/2679529358 conn(0x7facb4076740 msgr2=0x7facb410bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.240+0000 7facbb78c640 1 -- 172.21.15.83:0/2679529358 shutdown_connections 2024-03-11T19:40:43.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.240+0000 7facbb78c640 1 -- 172.21.15.83:0/2679529358 wait complete. 2024-03-11T19:40:43.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:40:43.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:40:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.458+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3715951943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6980ffa10 msgr2=0x7ff698101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.458+0000 7ff69ca0e640 1 --2- 172.21.15.83:0/3715951943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6980ffa10 0x7ff698101e40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff688009900 tx=0x7ff68802bc30 comp rx=0 tx=0).stop 2024-03-11T19:40:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.458+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3715951943 shutdown_connections 2024-03-11T19:40:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.458+0000 7ff69ca0e640 1 --2- 172.21.15.83:0/3715951943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff698104d30 0x7ff6981051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.458+0000 7ff69ca0e640 1 --2- 172.21.15.83:0/3715951943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff698102380 0x7ff6981047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.458+0000 7ff69ca0e640 1 --2- 172.21.15.83:0/3715951943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6980ffa10 0x7ff698101e40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.458+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3715951943 >> 172.21.15.83:0/3715951943 conn(0x7ff6980fb650 msgr2=0x7ff6980fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.458+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3715951943 shutdown_connections 2024-03-11T19:40:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.458+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3715951943 wait complete. 2024-03-11T19:40:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.459+0000 7ff69ca0e640 1 Processor -- start 2024-03-11T19:40:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.459+0000 7ff69ca0e640 1 -- start start 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.459+0000 7ff69ca0e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6980ffa10 0x7ff69819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.460+0000 7ff69ca0e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff698102380 0x7ff69819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.460+0000 7ff69ca0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff698104d30 0x7ff6981a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.460+0000 7ff68dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff698102380 0x7ff69819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.460+0000 7ff68dd9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff698102380 0x7ff69819dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40594/0 (socket says 172.21.15.83:40594) 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.460+0000 7ff69659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6980ffa10 0x7ff69819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.460+0000 7ff69659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6980ffa10 0x7ff69819d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56932/0 (socket says 172.21.15.83:56932) 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.460+0000 7ff68dd9b640 1 -- 172.21.15.83:0/3999768813 learned_addr learned my addr 172.21.15.83:0/3999768813 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.460+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3999768813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6981a5340 con 0x7ff698104d30 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.460+0000 7ff696d9d640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff698104d30 0x7ff6981a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.460+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3999768813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6981a54b0 con 0x7ff698102380 2024-03-11T19:40:43.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.461+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3999768813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6981a5670 con 0x7ff6980ffa10 2024-03-11T19:40:43.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.461+0000 7ff68dd9b640 1 -- 172.21.15.83:0/3999768813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6980ffa10 msgr2=0x7ff69819d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.461+0000 7ff68dd9b640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6980ffa10 0x7ff69819d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.461+0000 7ff68dd9b640 1 -- 172.21.15.83:0/3999768813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff698104d30 msgr2=0x7ff6981a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.461+0000 7ff68dd9b640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff698104d30 0x7ff6981a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.461+0000 7ff68dd9b640 1 -- 172.21.15.83:0/3999768813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688009560 con 0x7ff698102380 2024-03-11T19:40:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.461+0000 7ff69659c640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6980ffa10 0x7ff69819d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.462+0000 7ff68dd9b640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff698102380 0x7ff69819dd90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff68000b4a0 tx=0x7ff68000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.462+0000 7ff68ffff640 1 -- 172.21.15.83:0/3999768813 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680010610 con 0x7ff698102380 2024-03-11T19:40:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.462+0000 7ff68ffff640 1 -- 172.21.15.83:0/3999768813 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff680010c50 con 0x7ff698102380 2024-03-11T19:40:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.462+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3999768813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6981a5950 con 0x7ff698102380 2024-03-11T19:40:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.462+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3999768813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6981a5e00 con 0x7ff698102380 2024-03-11T19:40:43.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.463+0000 7ff68ffff640 1 -- 172.21.15.83:0/3999768813 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680017e20 con 0x7ff698102380 2024-03-11T19:40:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.465+0000 7ff68ffff640 1 -- 172.21.15.83:0/3999768813 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff68000f360 con 0x7ff698102380 2024-03-11T19:40:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.465+0000 7ff69f5e4740 1 Processor -- start 2024-03-11T19:40:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff69f5e4740 1 -- start start 2024-03-11T19:40:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed3e2d0 0x55720ed3e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3ecc0 0x55720ed437c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed43e30 0x55720ed462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720ecae280 con 0x55720ed43e30 2024-03-11T19:40:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720ecab7b0 con 0x55720ed3ecc0 2024-03-11T19:40:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720eca6f90 con 0x55720ed3e2d0 2024-03-11T19:40:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff696d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed3e2d0 0x55720ed3e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff696d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed3e2d0 0x55720ed3e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56934/0 (socket says 172.21.15.83:56934) 2024-03-11T19:40:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff696d9d640 1 -- 172.21.15.83:0/35251372 learned_addr learned my addr 172.21.15.83:0/35251372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff69659c640 1 --2- 172.21.15.83:0/35251372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3ecc0 0x55720ed437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.466+0000 7ff68dd9b640 1 --2- 172.21.15.83:0/35251372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed43e30 0x55720ed462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.467+0000 7ff69659c640 1 -- 172.21.15.83:0/35251372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed3e2d0 msgr2=0x55720ed3e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.467+0000 7ff69659c640 1 --2- 172.21.15.83:0/35251372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed3e2d0 0x55720ed3e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.467+0000 7ff69659c640 1 -- 172.21.15.83:0/35251372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed43e30 msgr2=0x55720ed462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.467+0000 7ff69659c640 1 --2- 172.21.15.83:0/35251372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed43e30 0x55720ed462c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.467+0000 7ff69659c640 1 -- 172.21.15.83:0/35251372 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55720ec65570 con 0x55720ed3ecc0 2024-03-11T19:40:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.467+0000 7ff696d9d640 1 --2- 172.21.15.83:0/35251372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed3e2d0 0x55720ed3e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:40:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.467+0000 7ff68ffff640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6840a1bb0 0x7ff6840a4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.467+0000 7ff69659c640 1 --2- 172.21.15.83:0/35251372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3ecc0 0x55720ed437c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff688009850 tx=0x7ff688004820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d3e36b12a2344 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.467+0000 7ff68ffff640 1 -- 172.21.15.83:0/3999768813 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff6800cc800 con 0x7ff698102380 2024-03-11T19:40:43.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.468+0000 7ff696d9d640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6840a1bb0 0x7ff6840a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.468+0000 7ff68cd99640 1 -- 172.21.15.83:0/35251372 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688004990 con 0x55720ed3ecc0 2024-03-11T19:40:43.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.468+0000 7ff68cd99640 1 -- 172.21.15.83:0/35251372 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff688004af0 con 0x55720ed3ecc0 2024-03-11T19:40:43.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.468+0000 7ff68cd99640 1 -- 172.21.15.83:0/35251372 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6880035f0 con 0x55720ed3ecc0 2024-03-11T19:40:43.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff69f5e4740 1 -- 172.21.15.83:0/35251372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3ecc0 msgr2=0x55720ed437c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff696d9d640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6840a1bb0 0x7ff6840a4070 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55720ed3f520 tx=0x7ff6900096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/35251372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3ecc0 0x55720ed437c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff688009850 tx=0x7ff688004820 comp rx=0 tx=0).stop 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff69f5e4740 1 -- 172.21.15.83:0/35251372 shutdown_connections 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/35251372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed43e30 0x55720ed462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/35251372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3ecc0 0x55720ed437c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/35251372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed3e2d0 0x55720ed3e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff69f5e4740 1 -- 172.21.15.83:0/35251372 >> 172.21.15.83:0/35251372 conn(0x55720ed397f0 msgr2=0x55720ed39bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff69f5e4740 1 -- 172.21.15.83:0/35251372 shutdown_connections 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff69f5e4740 1 -- 172.21.15.83:0/35251372 wait complete. 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff69f5e4740 1 -- start start 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.469+0000 7ff69f5e4740 1 Processor -- start 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3e2d0 0x55720ed48240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3ecc0 0x55720ed48780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed4f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720e97a430 con 0x55720ed3e2d0 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720e936e00 con 0x55720ed3ecc0 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720ec2bfd0 con 0x55720ed43e30 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69f5e4740 10 client.0 _ll_register_callbacks cb 0x55720ec81fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3e2d0 0x55720ed48240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3e2d0 0x55720ed48240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45580/0 (socket says 172.21.15.83:45580) 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69659c640 1 -- 172.21.15.83:0/1020071450 learned_addr learned my addr 172.21.15.83:0/1020071450 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff68dd9b640 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3ecc0 0x55720ed48780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69659c640 1 -- 172.21.15.83:0/1020071450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 msgr2=0x55720ed4f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69659c640 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed4f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69659c640 1 -- 172.21.15.83:0/1020071450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3ecc0 msgr2=0x55720ed48780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69659c640 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3ecc0 0x55720ed48780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.470+0000 7ff69659c640 1 -- 172.21.15.83:0/1020071450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff698105ca0 con 0x55720ed3e2d0 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.471+0000 7ff69659c640 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3e2d0 0x55720ed48240 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff688000c00 tx=0x7ff688005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1e761b61419957 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.471+0000 7ff67dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.471+0000 7ff67dffb640 1 -- 172.21.15.83:0/1020071450 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68804a860 con 0x55720ed3e2d0 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.471+0000 7ff67dffb640 1 -- 172.21.15.83:0/1020071450 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff68804a9c0 con 0x55720ed3e2d0 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.471+0000 7ff69f5e4740 1 -- 172.21.15.83:0/1020071450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55720e9a67e0 con 0x55720ed3e2d0 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.471+0000 7ff67dffb640 1 -- 172.21.15.83:0/1020071450 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff68804ab80 con 0x55720ed3e2d0 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.471+0000 7ff69f5e4740 10 client.15663 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.471+0000 7ff69f5e4740 1 -- 172.21.15.83:0/1020071450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55720eca5610 con 0x55720ed3e2d0 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.472+0000 7ff67dffb640 1 -- 172.21.15.83:0/1020071450 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55720e9a67e0 con 0x55720ed3e2d0 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.472+0000 7ff67dffb640 1 -- 172.21.15.83:0/1020071450 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff68805d070 con 0x55720ed3e2d0 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.472+0000 7ff69f5e4740 10 client.15663 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:43.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.472+0000 7ff69f5e4740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.472+0000 7ff69f5e4740 20 client.15663 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.472+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55720ed53fe0 0x55720ed56400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.472+0000 7ff69f5e4740 4 client.15663 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.472+0000 7ff69f5e4740 1 -- 172.21.15.83:0/1020071450 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55720e9e3720 con 0x55720ed53fe0 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.472+0000 7ff68dd9b640 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55720ed53fe0 0x55720ed56400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.473+0000 7ff67dffb640 10 client.15663 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.473+0000 7ff68dd9b640 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55720ed53fe0 0x55720ed56400 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c793794c27cb586 server_cookie=31b84deb2f267d3a in_seq=0 out_seq=0 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff67dffb640 1 -- 172.21.15.83:0/1020071450 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55720e9e3720 con 0x55720ed53fe0 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff67dffb640 10 client.15663 handle_command_reply: tid=0 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 1 client.15663 shutdown 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 1 -- 172.21.15.83:0/1020071450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3e2d0 msgr2=0x55720ed48240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3e2d0 0x55720ed48240 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff688000c00 tx=0x7ff688005740 comp rx=0 tx=0).stop 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 1 -- 172.21.15.83:0/1020071450 shutdown_connections 2024-03-11T19:40:43.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55720ed53fe0 0x55720ed56400 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed4f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff67dffb640 0 client.15663 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:43.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3ecc0 0x55720ed48780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/1020071450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3e2d0 0x55720ed48240 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 1 -- 172.21.15.83:0/1020071450 >> 172.21.15.83:0/1020071450 conn(0x55720ed397f0 msgr2=0x55720ed38a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 1 -- 172.21.15.83:0/1020071450 shutdown_connections 2024-03-11T19:40:43.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 1 -- 172.21.15.83:0/1020071450 wait complete. 2024-03-11T19:40:43.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.474+0000 7ff69f5e4740 20 client.15663 trim_cache size 0 max 16384 2024-03-11T19:40:43.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.500+0000 7ff69f5e4740 1 Processor -- start 2024-03-11T19:40:43.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69f5e4740 1 -- start start 2024-03-11T19:40:43.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 0x55720ed43570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3ecc0 0x55720ed4f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720ed3f550 con 0x55720ed3ecc0 2024-03-11T19:40:43.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720ed49290 con 0x55720ed3e2d0 2024-03-11T19:40:43.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720ed50c40 con 0x55720ed43e30 2024-03-11T19:40:43.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff68dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3ecc0 0x55720ed4f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff696d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 0x55720ed43570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff696d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed381c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:56950/0 (socket says 172.21.15.83:56950) 2024-03-11T19:40:43.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff696d9d640 1 -- 172.21.15.83:0/3965883619 learned_addr learned my addr 172.21.15.83:0/3965883619 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:43.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69659c640 1 -- 172.21.15.83:0/3965883619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 msgr2=0x55720ed381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69659c640 1 --2- 172.21.15.83:0/3965883619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69659c640 1 -- 172.21.15.83:0/3965883619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3ecc0 msgr2=0x55720ed4f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69659c640 1 --2- 172.21.15.83:0/3965883619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3ecc0 0x55720ed4f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.501+0000 7ff69659c640 1 -- 172.21.15.83:0/3965883619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55720ec41ac0 con 0x55720ed3e2d0 2024-03-11T19:40:43.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.502+0000 7ff69659c640 1 --2- 172.21.15.83:0/3965883619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 0x55720ed43570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff68802c840 tx=0x7ff688005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0109c3be4c29f22 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.502+0000 7ff67ffff640 1 -- 172.21.15.83:0/3965883619 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688004990 con 0x55720ed3e2d0 2024-03-11T19:40:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.502+0000 7ff67ffff640 1 -- 172.21.15.83:0/3965883619 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff68805a070 con 0x55720ed3e2d0 2024-03-11T19:40:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.502+0000 7ff67ffff640 1 -- 172.21.15.83:0/3965883619 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68804a9d0 con 0x55720ed3e2d0 2024-03-11T19:40:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.503+0000 7ff69f5e4740 1 -- 172.21.15.83:0/3965883619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 msgr2=0x55720ed43570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.503+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/3965883619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 0x55720ed43570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff68802c840 tx=0x7ff688005bd0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.503+0000 7ff69f5e4740 1 -- 172.21.15.83:0/3965883619 shutdown_connections 2024-03-11T19:40:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.503+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/3965883619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.503+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/3965883619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3ecc0 0x55720ed4f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.503+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/3965883619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 0x55720ed43570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.503+0000 7ff69f5e4740 1 -- 172.21.15.83:0/3965883619 >> 172.21.15.83:0/3965883619 conn(0x55720ed397f0 msgr2=0x55720ed39bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.503+0000 7ff69f5e4740 1 -- 172.21.15.83:0/3965883619 shutdown_connections 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.503+0000 7ff69f5e4740 1 -- 172.21.15.83:0/3965883619 wait complete. 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.503+0000 7ff69f5e4740 1 -- start start 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.503+0000 7ff69f5e4740 1 Processor -- start 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 0x55720ed4f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3ecc0 0x55720ed4f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69f5e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed4d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720ed4df20 con 0x55720ed3ecc0 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720ed4e090 con 0x55720ed3e2d0 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69f5e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55720ed4e250 con 0x55720ed43e30 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69f5e4740 10 client.0 _ll_register_callbacks cb 0x7ff6800d2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 0x55720ed4f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 0x55720ed4f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40624/0 (socket says 172.21.15.83:40624) 2024-03-11T19:40:43.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69659c640 1 -- 172.21.15.83:0/882274023 learned_addr learned my addr 172.21.15.83:0/882274023 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:40:43.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff696d9d640 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed4d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69659c640 1 -- 172.21.15.83:0/882274023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 msgr2=0x55720ed4d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69659c640 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed4d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69659c640 1 -- 172.21.15.83:0/882274023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3ecc0 msgr2=0x55720ed4f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:40:43.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69659c640 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3ecc0 0x55720ed4f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.504+0000 7ff69659c640 1 -- 172.21.15.83:0/882274023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55720eca5610 con 0x55720ed3e2d0 2024-03-11T19:40:43.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.505+0000 7ff69659c640 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 0x55720ed4f170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff6880344b0 tx=0x7ff688004a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5010bafdb4c3e53 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:40:43.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.505+0000 7ff67dffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:40:43.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.505+0000 7ff67dffb640 1 -- 172.21.15.83:0/882274023 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68806c070 con 0x55720ed3e2d0 2024-03-11T19:40:43.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.505+0000 7ff67dffb640 1 -- 172.21.15.83:0/882274023 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff688000e40 con 0x55720ed3e2d0 2024-03-11T19:40:43.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.505+0000 7ff69f5e4740 1 -- 172.21.15.83:0/882274023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff688004af0 con 0x55720ed3e2d0 2024-03-11T19:40:43.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.506+0000 7ff67dffb640 1 -- 172.21.15.83:0/882274023 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff688004040 con 0x55720ed3e2d0 2024-03-11T19:40:43.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.506+0000 7ff69f5e4740 10 client.25169 fetch_fsmap learned FSMap version 24 2024-03-11T19:40:43.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.506+0000 7ff69f5e4740 1 -- 172.21.15.83:0/882274023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55720ec676a0 con 0x55720ed3e2d0 2024-03-11T19:40:43.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.506+0000 7ff67dffb640 1 -- 172.21.15.83:0/882274023 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff688004af0 con 0x55720ed3e2d0 2024-03-11T19:40:43.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.507+0000 7ff67dffb640 1 -- 172.21.15.83:0/882274023 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff68806f020 con 0x55720ed3e2d0 2024-03-11T19:40:43.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.507+0000 7ff69f5e4740 10 client.25169 fetch_fsmap finished waiting for FSMap version 24 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.507+0000 7ff69f5e4740 10 client.25169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.507+0000 7ff69f5e4740 20 client.25169 populate_metadata read hostname 'smithi083' 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.507+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55720edab510 0x55720edad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.507+0000 7ff69f5e4740 4 client.25169 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.507+0000 7ff69f5e4740 1 -- 172.21.15.83:0/882274023 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55720ea32890 con 0x55720edab510 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.507+0000 7ff68dd9b640 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55720edab510 0x55720edad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.507+0000 7ff67dffb640 10 client.25169 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff68dd9b640 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55720edab510 0x55720edad900 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f613603a0bdb4d50 server_cookie=f3f47f0c21b3a9d7 in_seq=0 out_seq=0 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff67dffb640 1 -- 172.21.15.83:0/882274023 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55720ea32890 con 0x55720edab510 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff67dffb640 10 client.25169 handle_command_reply: tid=0 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 1 client.25169 shutdown 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 1 -- 172.21.15.83:0/882274023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 msgr2=0x55720ed4f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 0x55720ed4f170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff6880344b0 tx=0x7ff688004a20 comp rx=0 tx=0).stop 2024-03-11T19:40:43.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 1 -- 172.21.15.83:0/882274023 shutdown_connections 2024-03-11T19:40:43.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55720edab510 0x55720edad900 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55720ed43e30 0x55720ed4d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55720ed3ecc0 0x55720ed4f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff67dffb640 0 client.25169 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:40:43.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 1 --2- 172.21.15.83:0/882274023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55720ed3e2d0 0x55720ed4f170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 1 -- 172.21.15.83:0/882274023 >> 172.21.15.83:0/882274023 conn(0x55720ed397f0 msgr2=0x55720ed381c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 1 -- 172.21.15.83:0/882274023 shutdown_connections 2024-03-11T19:40:43.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 1 -- 172.21.15.83:0/882274023 wait complete. 2024-03-11T19:40:43.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.508+0000 7ff69f5e4740 20 client.25169 trim_cache size 0 max 16384 2024-03-11T19:40:43.518 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:40:43.518 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:40:43.518 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:40:43.518 INFO:teuthology.orchestra.run.smithi083.stdout: "93b8a617-0262-4193-885f-97c0ed95dbef": { 2024-03-11T19:40:43.518 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:40:43.518 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "93b8a617-0262-4193-885f-97c0ed95dbef", 2024-03-11T19:40:43.518 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:40:43.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:40:43.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:40:43.519 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:40:43.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3999768813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6840a1bb0 msgr2=0x7ff6840a4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6840a1bb0 0x7ff6840a4070 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55720ed3f520 tx=0x7ff6900096a0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3999768813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff698102380 msgr2=0x7ff69819dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:40:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff698102380 0x7ff69819dd90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff68000b4a0 tx=0x7ff68000b960 comp rx=0 tx=0).stop 2024-03-11T19:40:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3999768813 shutdown_connections 2024-03-11T19:40:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6840a1bb0 0x7ff6840a4070 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff698104d30 0x7ff6981a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff698102380 0x7ff69819dd90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 --2- 172.21.15.83:0/3999768813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6980ffa10 0x7ff69819d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:40:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3999768813 >> 172.21.15.83:0/3999768813 conn(0x7ff6980fb650 msgr2=0x7ff698102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:40:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3999768813 shutdown_connections 2024-03-11T19:40:43.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:40:43.509+0000 7ff69ca0e640 1 -- 172.21.15.83:0/3999768813 wait complete. 2024-03-11T19:40:43.529 INFO:tasks.cephfs.filesystem:scrub status for tag:93b8a617-0262-4193-885f-97c0ed95dbef - {'path': '/', 'tag': '93b8a617-0262-4193-885f-97c0ed95dbef', 'options': 'recursive,force'} 2024-03-11T19:40:44.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:43 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:40:44.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:43 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:44.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:43 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:40:44.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:43 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:44.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:43 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:40:44.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:43 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:44.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:43 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:40:44.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:43 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:44.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:43 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:40:44.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:43 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:44.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:43 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:40:44.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:43 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:44 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:40:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:44 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:44 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:40:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:44 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:40:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:44 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:44 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:44 smithi083 ceph-mon[31864]: pgmap v395: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 12 MiB/s wr, 902 op/s 2024-03-11T19:40:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:44 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:40:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:44 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:45.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:44 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:40:45.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:44 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:40:45.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:44 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:45.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:44 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:45.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:44 smithi137 ceph-mon[36220]: pgmap v395: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 12 MiB/s wr, 902 op/s 2024-03-11T19:40:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:44 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:40:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:44 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:44 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:40:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:44 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:40:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:44 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:44 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:40:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:44 smithi204 ceph-mon[36739]: pgmap v395: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 12 MiB/s wr, 902 op/s 2024-03-11T19:40:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 90.0 s, 5223.8 tps, lat 19.142 ms stddev 20.439 2024-03-11T19:40:47.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:47 smithi137 ceph-mon[36220]: pgmap v396: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 916 op/s 2024-03-11T19:40:47.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:47 smithi204 ceph-mon[36739]: pgmap v396: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 916 op/s 2024-03-11T19:40:48.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:47 smithi083 ceph-mon[31864]: pgmap v396: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 916 op/s 2024-03-11T19:40:49.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:48 smithi083 ceph-mon[31864]: pgmap v397: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 938 op/s 2024-03-11T19:40:49.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:48 smithi137 ceph-mon[36220]: pgmap v397: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 938 op/s 2024-03-11T19:40:49.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:48 smithi204 ceph-mon[36739]: pgmap v397: 97 pgs: 97 active+clean; 1.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 938 op/s 2024-03-11T19:40:50.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:50 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:40:50.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:50 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:40:50.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:50 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:40:51.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:51 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:40:51.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:51 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:40:51.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:51 smithi083 ceph-mon[31864]: pgmap v398: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 892 op/s 2024-03-11T19:40:51.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:51 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:40:51.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:51 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:40:51.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:51 smithi137 ceph-mon[36220]: pgmap v398: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 892 op/s 2024-03-11T19:40:51.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:51 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:40:51.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:51 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:40:51.909 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:51 smithi204 ceph-mon[36739]: pgmap v398: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 892 op/s 2024-03-11T19:40:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 95.0 s, 6094.6 tps, lat 16.423 ms stddev 17.857 2024-03-11T19:40:53.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:53 smithi137 ceph-mon[36220]: pgmap v399: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 831 op/s 2024-03-11T19:40:53.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:53 smithi204 ceph-mon[36739]: pgmap v399: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 831 op/s 2024-03-11T19:40:54.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:53 smithi083 ceph-mon[31864]: pgmap v399: 97 pgs: 97 active+clean; 1.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 831 op/s 2024-03-11T19:40:54.997 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:54 smithi204 ceph-mon[36739]: pgmap v400: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 970 op/s 2024-03-11T19:40:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:54 smithi083 ceph-mon[31864]: pgmap v400: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 970 op/s 2024-03-11T19:40:55.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:54 smithi137 ceph-mon[36220]: pgmap v400: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 970 op/s 2024-03-11T19:40:57.498 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 100.0 s, 6401.0 tps, lat 15.559 ms stddev 18.092 2024-03-11T19:40:57.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:57 smithi137 ceph-mon[36220]: pgmap v401: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-03-11T19:40:57.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:57 smithi204 ceph-mon[36739]: pgmap v401: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-03-11T19:40:58.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:57 smithi083 ceph-mon[31864]: pgmap v401: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-03-11T19:40:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:40:58 smithi083 ceph-mon[31864]: pgmap v402: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-03-11T19:40:59.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:40:58 smithi137 ceph-mon[36220]: pgmap v402: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-03-11T19:40:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:40:58 smithi204 ceph-mon[36739]: pgmap v402: 97 pgs: 97 active+clean; 1.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-03-11T19:41:01.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:00 smithi083 ceph-mon[31864]: pgmap v403: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 965 op/s 2024-03-11T19:41:01.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:00 smithi137 ceph-mon[36220]: pgmap v403: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 965 op/s 2024-03-11T19:41:01.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:00 smithi204 ceph-mon[36739]: pgmap v403: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 965 op/s 2024-03-11T19:41:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 105.0 s, 4246.0 tps, lat 23.594 ms stddev 29.431 2024-03-11T19:41:03.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:03 smithi204 ceph-mon[36739]: pgmap v404: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 999 op/s 2024-03-11T19:41:04.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:03 smithi083 ceph-mon[31864]: pgmap v404: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 999 op/s 2024-03-11T19:41:04.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:03 smithi137 ceph-mon[36220]: pgmap v404: 97 pgs: 97 active+clean; 1.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 999 op/s 2024-03-11T19:41:04.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:04 smithi204 ceph-mon[36739]: pgmap v405: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 1.13k op/s 2024-03-11T19:41:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:04 smithi083 ceph-mon[31864]: pgmap v405: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 1.13k op/s 2024-03-11T19:41:05.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:04 smithi137 ceph-mon[36220]: pgmap v405: 97 pgs: 97 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 1.13k op/s 2024-03-11T19:41:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:06 smithi083 ceph-mon[31864]: pgmap v406: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 1.03k op/s 2024-03-11T19:41:07.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:06 smithi137 ceph-mon[36220]: pgmap v406: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 1.03k op/s 2024-03-11T19:41:07.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:06 smithi204 ceph-mon[36739]: pgmap v406: 97 pgs: 97 active+clean; 1.3 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 MiB/s wr, 1.03k op/s 2024-03-11T19:41:07.497 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 110.0 s, 4194.4 tps, lat 23.837 ms stddev 28.737 2024-03-11T19:41:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:08 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:41:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:08 smithi083 ceph-mon[31864]: pgmap v407: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 43 MiB/s wr, 989 op/s 2024-03-11T19:41:09.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:08 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:41:09.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:08 smithi137 ceph-mon[36220]: pgmap v407: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 43 MiB/s wr, 989 op/s 2024-03-11T19:41:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:08 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:41:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:08 smithi204 ceph-mon[36739]: pgmap v407: 97 pgs: 97 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 43 MiB/s wr, 989 op/s 2024-03-11T19:41:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:10 smithi083 ceph-mon[31864]: pgmap v408: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 46 MiB/s wr, 921 op/s 2024-03-11T19:41:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:10 smithi137 ceph-mon[36220]: pgmap v408: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 46 MiB/s wr, 921 op/s 2024-03-11T19:41:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:10 smithi204 ceph-mon[36739]: pgmap v408: 97 pgs: 97 active+clean; 1.4 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 46 MiB/s wr, 921 op/s 2024-03-11T19:41:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 115.0 s, 4746.2 tps, lat 21.051 ms stddev 24.605 2024-03-11T19:41:13.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:12 smithi083 ceph-mon[31864]: pgmap v409: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 MiB/s wr, 881 op/s 2024-03-11T19:41:13.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:12 smithi137 ceph-mon[36220]: pgmap v409: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 MiB/s wr, 881 op/s 2024-03-11T19:41:13.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:12 smithi204 ceph-mon[36739]: pgmap v409: 97 pgs: 97 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 MiB/s wr, 881 op/s 2024-03-11T19:41:13.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:41:13.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:41:13.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.661+0000 7fc53e74f640 1 -- 172.21.15.83:0/2841385253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc538101270 msgr2=0x7fc5381016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.661+0000 7fc53e74f640 1 --2- 172.21.15.83:0/2841385253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc538101270 0x7fc5381016f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009900 tx=0x7fc52c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:41:13.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.662+0000 7fc53e74f640 1 -- 172.21.15.83:0/2841385253 shutdown_connections 2024-03-11T19:41:13.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.662+0000 7fc53e74f640 1 --2- 172.21.15.83:0/2841385253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc538101c30 0x7fc5381084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.662+0000 7fc53e74f640 1 --2- 172.21.15.83:0/2841385253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc538101270 0x7fc5381016f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.662+0000 7fc53e74f640 1 --2- 172.21.15.83:0/2841385253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538100080 0x7fc5381004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.662+0000 7fc53e74f640 1 -- 172.21.15.83:0/2841385253 >> 172.21.15.83:0/2841385253 conn(0x7fc5380fb5f0 msgr2=0x7fc5380fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:13.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.663+0000 7fc53e74f640 1 -- 172.21.15.83:0/2841385253 shutdown_connections 2024-03-11T19:41:13.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.663+0000 7fc53e74f640 1 -- 172.21.15.83:0/2841385253 wait complete. 2024-03-11T19:41:13.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc53e74f640 1 Processor -- start 2024-03-11T19:41:13.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc53e74f640 1 -- start start 2024-03-11T19:41:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc53e74f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538100080 0x7fc53819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc53e74f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc538101270 0x7fc53819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc53e74f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc538101c30 0x7fc5381a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc53e74f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5381a52d0 con 0x7fc538101c30 2024-03-11T19:41:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc53e74f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5381a5440 con 0x7fc538100080 2024-03-11T19:41:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc53e74f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5381a5600 con 0x7fc538101270 2024-03-11T19:41:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc53ccec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc538101c30 0x7fc5381a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc53ccec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc538101c30 0x7fc5381a4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39198/0 (socket says 172.21.15.83:39198) 2024-03-11T19:41:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc537fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538100080 0x7fc53819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc53ccec640 1 -- 172.21.15.83:0/2184480036 learned_addr learned my addr 172.21.15.83:0/2184480036 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:13.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.664+0000 7fc5377fe640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc538101270 0x7fc53819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.665+0000 7fc537fff640 1 -- 172.21.15.83:0/2184480036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc538101270 msgr2=0x7fc53819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.665+0000 7fc537fff640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc538101270 0x7fc53819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.665+0000 7fc537fff640 1 -- 172.21.15.83:0/2184480036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc538101c30 msgr2=0x7fc5381a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.665+0000 7fc537fff640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc538101c30 0x7fc5381a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.665+0000 7fc537fff640 1 -- 172.21.15.83:0/2184480036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c009560 con 0x7fc538100080 2024-03-11T19:41:13.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.665+0000 7fc537fff640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538100080 0x7fc53819d7e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc530005fd0 tx=0x7fc53000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:13.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.665+0000 7fc5357fa640 1 -- 172.21.15.83:0/2184480036 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53000cea0 con 0x7fc538100080 2024-03-11T19:41:13.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.665+0000 7fc53e74f640 1 -- 172.21.15.83:0/2184480036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5381a58e0 con 0x7fc538100080 2024-03-11T19:41:13.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.665+0000 7fc53e74f640 1 -- 172.21.15.83:0/2184480036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5381a5e10 con 0x7fc538100080 2024-03-11T19:41:13.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.666+0000 7fc5357fa640 1 -- 172.21.15.83:0/2184480036 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc530003e50 con 0x7fc538100080 2024-03-11T19:41:13.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.666+0000 7fc5357fa640 1 -- 172.21.15.83:0/2184480036 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530010720 con 0x7fc538100080 2024-03-11T19:41:13.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.668+0000 7fc5357fa640 1 -- 172.21.15.83:0/2184480036 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc530004380 con 0x7fc538100080 2024-03-11T19:41:13.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc541403740 1 Processor -- start 2024-03-11T19:41:13.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc541403740 1 -- start start 2024-03-11T19:41:13.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc5357fa640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5200a1b30 0x7fc5200a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc541403740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5f2d0 0x564eb3f5f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc541403740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 0x564eb3f647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc541403740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f64e30 0x564eb3f672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc541403740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3ecf280 con 0x564eb3f5f2d0 2024-03-11T19:41:13.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc541403740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3ecc7b0 con 0x564eb3f64e30 2024-03-11T19:41:13.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc541403740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3ec7f90 con 0x564eb3f5fcc0 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc5357fa640 1 -- 172.21.15.83:0/2184480036 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc530014070 con 0x7fc538100080 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc53ccec640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5200a1b30 0x7fc5200a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc53ccec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f64e30 0x564eb3f672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc5377fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 0x564eb3f647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc537fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5f2d0 0x564eb3f5f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc5377fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 0x564eb3f647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52574/0 (socket says 172.21.15.83:52574) 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.670+0000 7fc5377fe640 1 -- 172.21.15.83:0/3242169308 learned_addr learned my addr 172.21.15.83:0/3242169308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.671+0000 7fc5377fe640 1 -- 172.21.15.83:0/3242169308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f64e30 msgr2=0x564eb3f672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.671+0000 7fc5377fe640 1 --2- 172.21.15.83:0/3242169308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f64e30 0x564eb3f672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.671+0000 7fc5377fe640 1 -- 172.21.15.83:0/3242169308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5f2d0 msgr2=0x564eb3f5f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.671+0000 7fc5377fe640 1 --2- 172.21.15.83:0/3242169308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5f2d0 0x564eb3f5f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.671+0000 7fc5377fe640 1 -- 172.21.15.83:0/3242169308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eb3e86570 con 0x564eb3f5fcc0 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.671+0000 7fc53ccec640 1 --2- 172.21.15.83:0/3242169308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f64e30 0x564eb3f672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:13.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.671+0000 7fc5377fe640 1 --2- 172.21.15.83:0/3242169308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 0x564eb3f647c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009900 tx=0x7fc52c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=593d249d956a9029 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:13.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.671+0000 7fc53ccec640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5200a1b30 0x7fc5200a3ff0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc528009810 tx=0x7fc528009b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:13.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.671+0000 7fc51e7fc640 1 -- 172.21.15.83:0/3242169308 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c02ca20 con 0x564eb3f5fcc0 2024-03-11T19:41:13.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.671+0000 7fc51e7fc640 1 -- 172.21.15.83:0/3242169308 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc52c02cb80 con 0x564eb3f5fcc0 2024-03-11T19:41:13.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.671+0000 7fc51e7fc640 1 -- 172.21.15.83:0/3242169308 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0051b0 con 0x564eb3f5fcc0 2024-03-11T19:41:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.672+0000 7fc541403740 1 -- 172.21.15.83:0/3242169308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 msgr2=0x564eb3f647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.672+0000 7fc541403740 1 --2- 172.21.15.83:0/3242169308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 0x564eb3f647c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009900 tx=0x7fc52c005d50 comp rx=0 tx=0).stop 2024-03-11T19:41:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.672+0000 7fc541403740 1 -- 172.21.15.83:0/3242169308 shutdown_connections 2024-03-11T19:41:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.672+0000 7fc541403740 1 --2- 172.21.15.83:0/3242169308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f64e30 0x564eb3f672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.672+0000 7fc541403740 1 --2- 172.21.15.83:0/3242169308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 0x564eb3f647c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.672+0000 7fc541403740 1 --2- 172.21.15.83:0/3242169308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5f2d0 0x564eb3f5f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.672+0000 7fc541403740 1 -- 172.21.15.83:0/3242169308 >> 172.21.15.83:0/3242169308 conn(0x564eb3f5a7f0 msgr2=0x564eb3f5abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.672+0000 7fc541403740 1 -- 172.21.15.83:0/3242169308 shutdown_connections 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.673+0000 7fc541403740 1 -- 172.21.15.83:0/3242169308 wait complete. 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.673+0000 7fc541403740 1 -- start start 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.673+0000 7fc541403740 1 Processor -- start 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc541403740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 0x564eb3f691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc541403740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 0x564eb3f69700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc541403740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f64e30 0x564eb3f70740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc541403740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3b9b430 con 0x564eb3f64e30 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc541403740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3b57e00 con 0x564eb3f5f2d0 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc541403740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3e4cfd0 con 0x564eb3f5fcc0 2024-03-11T19:41:13.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc541403740 10 client.0 _ll_register_callbacks cb 0x564eb3ea2fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc537fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 0x564eb3f691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc537fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 0x564eb3f691c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:60972/0 (socket says 172.21.15.83:60972) 2024-03-11T19:41:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc537fff640 1 -- 172.21.15.83:0/580957410 learned_addr learned my addr 172.21.15.83:0/580957410 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc53ccec640 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f64e30 0x564eb3f70740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc5377fe640 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 0x564eb3f69700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc537fff640 1 -- 172.21.15.83:0/580957410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 msgr2=0x564eb3f69700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc537fff640 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 0x564eb3f69700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc537fff640 1 -- 172.21.15.83:0/580957410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f64e30 msgr2=0x564eb3f70740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc537fff640 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f64e30 0x564eb3f70740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.674+0000 7fc537fff640 1 -- 172.21.15.83:0/580957410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5380682e0 con 0x564eb3f5f2d0 2024-03-11T19:41:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.675+0000 7fc53ccec640 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f64e30 0x564eb3f70740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:41:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.675+0000 7fc537fff640 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 0x564eb3f691c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c760 tx=0x7fc530098ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50006ca4d7b92f5b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:13.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.675+0000 7fc5177fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:41:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.675+0000 7fc5177fe640 1 -- 172.21.15.83:0/580957410 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5300a2f30 con 0x564eb3f5f2d0 2024-03-11T19:41:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.675+0000 7fc5177fe640 1 -- 172.21.15.83:0/580957410 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc530004030 con 0x564eb3f5f2d0 2024-03-11T19:41:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.675+0000 7fc541403740 1 -- 172.21.15.83:0/580957410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564eb3bc77e0 con 0x564eb3f5f2d0 2024-03-11T19:41:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc5177fe640 1 -- 172.21.15.83:0/580957410 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5300ba0c0 con 0x564eb3f5f2d0 2024-03-11T19:41:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc541403740 10 client.25181 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:13.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc541403740 1 -- 172.21.15.83:0/580957410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564eb3ec6610 con 0x564eb3f5f2d0 2024-03-11T19:41:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc5177fe640 1 -- 172.21.15.83:0/580957410 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564eb3bc77e0 con 0x564eb3f5f2d0 2024-03-11T19:41:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc5177fe640 1 -- 172.21.15.83:0/580957410 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc530099be0 con 0x564eb3f5f2d0 2024-03-11T19:41:13.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc541403740 10 client.25181 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc541403740 10 client.25181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc541403740 20 client.25181 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc541403740 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564eb3f75370 0x564eb3f777d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc541403740 4 client.25181 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc541403740 1 -- 172.21.15.83:0/580957410 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564eb3c04720 con 0x564eb3f75370 2024-03-11T19:41:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.676+0000 7fc5377fe640 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564eb3f75370 0x564eb3f777d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.677+0000 7fc5177fe640 10 client.25181 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:13.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.677+0000 7fc5377fe640 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564eb3f75370 0x564eb3f777d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d64d42457b4224e6 server_cookie=a498797f9a421361 in_seq=0 out_seq=0 2024-03-11T19:41:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.678+0000 7fc5177fe640 1 -- 172.21.15.83:0/580957410 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564eb3c04720 con 0x564eb3f75370 2024-03-11T19:41:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.678+0000 7fc5177fe640 10 client.25181 handle_command_reply: tid=0 2024-03-11T19:41:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc541403740 1 client.25181 shutdown 2024-03-11T19:41:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc541403740 1 -- 172.21.15.83:0/580957410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 msgr2=0x564eb3f691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc541403740 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 0x564eb3f691c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c760 tx=0x7fc530098ca0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc541403740 1 -- 172.21.15.83:0/580957410 shutdown_connections 2024-03-11T19:41:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc541403740 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564eb3f75370 0x564eb3f777d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc5177fe640 0 client.25181 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc541403740 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f64e30 0x564eb3f70740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc541403740 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5fcc0 0x564eb3f69700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc541403740 1 --2- 172.21.15.83:0/580957410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 0x564eb3f691c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc541403740 1 -- 172.21.15.83:0/580957410 >> 172.21.15.83:0/580957410 conn(0x564eb3f5a7f0 msgr2=0x564eb3f59a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc541403740 1 -- 172.21.15.83:0/580957410 shutdown_connections 2024-03-11T19:41:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.679+0000 7fc541403740 1 -- 172.21.15.83:0/580957410 wait complete. 2024-03-11T19:41:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.680+0000 7fc541403740 20 client.25181 trim_cache size 0 max 16384 2024-03-11T19:41:13.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.704+0000 7fc541403740 1 Processor -- start 2024-03-11T19:41:13.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.704+0000 7fc541403740 1 -- start start 2024-03-11T19:41:13.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc541403740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 0x564eb3f68dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc541403740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 0x564eb3f706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc541403740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f64e30 0x564eb3f69310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc541403740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3f600f0 con 0x564eb3f5fcc0 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc541403740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3f64bc0 con 0x564eb3f5f2d0 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc541403740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3f69ef0 con 0x564eb3f64e30 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc5377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 0x564eb3f706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc537fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 0x564eb3f68dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc5377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 0x564eb3f706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39230/0 (socket says 172.21.15.83:39230) 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc5377fe640 1 -- 172.21.15.83:0/448902897 learned_addr learned my addr 172.21.15.83:0/448902897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc53ccec640 1 --2- 172.21.15.83:0/448902897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f64e30 0x564eb3f69310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc537fff640 1 -- 172.21.15.83:0/448902897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f64e30 msgr2=0x564eb3f69310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc537fff640 1 --2- 172.21.15.83:0/448902897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f64e30 0x564eb3f69310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc537fff640 1 -- 172.21.15.83:0/448902897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 msgr2=0x564eb3f706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc537fff640 1 --2- 172.21.15.83:0/448902897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 0x564eb3f706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc537fff640 1 -- 172.21.15.83:0/448902897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eb3e62ac0 con 0x564eb3f5f2d0 2024-03-11T19:41:13.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc5377fe640 1 --2- 172.21.15.83:0/448902897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 0x564eb3f706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:13.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.705+0000 7fc537fff640 1 --2- 172.21.15.83:0/448902897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 0x564eb3f68dd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc53009a720 tx=0x7fc530096b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25f01cb2d6184d4f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.706+0000 7fc51dffb640 1 -- 172.21.15.83:0/448902897 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53009a2d0 con 0x564eb3f5f2d0 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.706+0000 7fc51dffb640 1 -- 172.21.15.83:0/448902897 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5300a2f30 con 0x564eb3f5f2d0 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.706+0000 7fc51dffb640 1 -- 172.21.15.83:0/448902897 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530085180 con 0x564eb3f5f2d0 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.707+0000 7fc541403740 1 -- 172.21.15.83:0/448902897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 msgr2=0x564eb3f68dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.707+0000 7fc541403740 1 --2- 172.21.15.83:0/448902897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 0x564eb3f68dd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc53009a720 tx=0x7fc530096b20 comp rx=0 tx=0).stop 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.707+0000 7fc541403740 1 -- 172.21.15.83:0/448902897 shutdown_connections 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.707+0000 7fc541403740 1 --2- 172.21.15.83:0/448902897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f64e30 0x564eb3f69310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.707+0000 7fc541403740 1 --2- 172.21.15.83:0/448902897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 0x564eb3f706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.707+0000 7fc541403740 1 --2- 172.21.15.83:0/448902897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f5f2d0 0x564eb3f68dd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.707+0000 7fc541403740 1 -- 172.21.15.83:0/448902897 >> 172.21.15.83:0/448902897 conn(0x564eb3f5a7f0 msgr2=0x564eb3f5abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.707+0000 7fc541403740 1 -- 172.21.15.83:0/448902897 shutdown_connections 2024-03-11T19:41:13.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.707+0000 7fc541403740 1 -- 172.21.15.83:0/448902897 wait complete. 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.707+0000 7fc541403740 1 -- start start 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.707+0000 7fc541403740 1 Processor -- start 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.708+0000 7fc541403740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5f2d0 0x564eb3f6eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.708+0000 7fc541403740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 0x564eb3f6f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.708+0000 7fc537fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5f2d0 0x564eb3f6eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.708+0000 7fc537fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5f2d0 0x564eb3f6eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52604/0 (socket says 172.21.15.83:52604) 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.708+0000 7fc537fff640 1 -- 172.21.15.83:0/4237709311 learned_addr learned my addr 172.21.15.83:0/4237709311 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.708+0000 7fc5377fe640 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 0x564eb3f6f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.709+0000 7fc541403740 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f64e30 0x564eb3f6f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.709+0000 7fc541403740 1 -- 172.21.15.83:0/4237709311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3f70ac0 con 0x564eb3f5fcc0 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.709+0000 7fc541403740 1 -- 172.21.15.83:0/4237709311 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3fcb8e0 con 0x564eb3f64e30 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.709+0000 7fc541403740 1 -- 172.21.15.83:0/4237709311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb3fcba20 con 0x564eb3f5f2d0 2024-03-11T19:41:13.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.709+0000 7fc541403740 10 client.0 _ll_register_callbacks cb 0x7fc528005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:13.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.709+0000 7fc53ccec640 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f64e30 0x564eb3f6f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.709+0000 7fc537fff640 1 -- 172.21.15.83:0/4237709311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f64e30 msgr2=0x564eb3f6f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:13.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.709+0000 7fc537fff640 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f64e30 0x564eb3f6f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.709+0000 7fc537fff640 1 -- 172.21.15.83:0/4237709311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 msgr2=0x564eb3f6f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:13.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.709+0000 7fc537fff640 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 0x564eb3f6f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.709+0000 7fc537fff640 1 -- 172.21.15.83:0/4237709311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564eb3e86570 con 0x564eb3f5f2d0 2024-03-11T19:41:13.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.710+0000 7fc537fff640 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5f2d0 0x564eb3f6eee0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc530097160 tx=0x7fc530098ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f28ad595a77108a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:13.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.710+0000 7fc5177fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:41:13.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.710+0000 7fc5177fe640 1 -- 172.21.15.83:0/4237709311 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530085380 con 0x564eb3f5f2d0 2024-03-11T19:41:13.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.710+0000 7fc5177fe640 1 -- 172.21.15.83:0/4237709311 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc5300ba0b0 con 0x564eb3f5f2d0 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.710+0000 7fc541403740 1 -- 172.21.15.83:0/4237709311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc52c02cb80 con 0x564eb3f5f2d0 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.710+0000 7fc5177fe640 1 -- 172.21.15.83:0/4237709311 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc53009a2d0 con 0x564eb3f5f2d0 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.710+0000 7fc541403740 10 client.25294 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.711+0000 7fc541403740 1 -- 172.21.15.83:0/4237709311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564eb3ec6610 con 0x564eb3f5f2d0 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.711+0000 7fc5177fe640 1 -- 172.21.15.83:0/4237709311 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc530089020 con 0x564eb3f5f2d0 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.711+0000 7fc5177fe640 1 -- 172.21.15.83:0/4237709311 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc53009bac0 con 0x564eb3f5f2d0 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.712+0000 7fc541403740 10 client.25294 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.712+0000 7fc541403740 10 client.25294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.712+0000 7fc541403740 20 client.25294 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.712+0000 7fc541403740 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564eb3f75370 0x564eb3fcf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.712+0000 7fc541403740 4 client.25294 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.712+0000 7fc541403740 1 -- 172.21.15.83:0/4237709311 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564eb3c53890 con 0x564eb3f75370 2024-03-11T19:41:13.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.712+0000 7fc5377fe640 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564eb3f75370 0x564eb3fcf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.714+0000 7fc5377fe640 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564eb3f75370 0x564eb3fcf070 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d1c43e7a0ec1b0c server_cookie=a04265d7cd51685e in_seq=0 out_seq=0 2024-03-11T19:41:13.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.714+0000 7fc5177fe640 10 client.25294 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:13.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.714+0000 7fc5177fe640 1 -- 172.21.15.83:0/4237709311 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564eb3c53890 con 0x564eb3f75370 2024-03-11T19:41:13.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.714+0000 7fc5177fe640 10 client.25294 handle_command_reply: tid=0 2024-03-11T19:41:13.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.714+0000 7fc541403740 1 client.25294 shutdown 2024-03-11T19:41:13.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.714+0000 7fc541403740 1 -- 172.21.15.83:0/4237709311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5f2d0 msgr2=0x564eb3f6eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.714+0000 7fc541403740 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5f2d0 0x564eb3f6eee0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc530097160 tx=0x7fc530098ca0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.714+0000 7fc541403740 1 -- 172.21.15.83:0/4237709311 shutdown_connections 2024-03-11T19:41:13.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.714+0000 7fc541403740 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564eb3f75370 0x564eb3fcf070 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.714+0000 7fc541403740 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564eb3f64e30 0x564eb3f6f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.714+0000 7fc541403740 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564eb3f5fcc0 0x564eb3f6f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.715+0000 7fc541403740 1 --2- 172.21.15.83:0/4237709311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564eb3f5f2d0 0x564eb3f6eee0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.715+0000 7fc541403740 1 -- 172.21.15.83:0/4237709311 >> 172.21.15.83:0/4237709311 conn(0x564eb3f5a7f0 msgr2=0x564eb3f68db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:13.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.715+0000 7fc5177fe640 0 client.25294 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:13.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.715+0000 7fc541403740 1 -- 172.21.15.83:0/4237709311 shutdown_connections 2024-03-11T19:41:13.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.715+0000 7fc541403740 1 -- 172.21.15.83:0/4237709311 wait complete. 2024-03-11T19:41:13.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.715+0000 7fc541403740 20 client.25294 trim_cache size 0 max 16384 2024-03-11T19:41:13.720 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:41:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:41:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:41:13.721 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:41:13.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 -- 172.21.15.83:0/2184480036 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5200a1b30 msgr2=0x7fc5200a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5200a1b30 0x7fc5200a3ff0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc528009810 tx=0x7fc528009b70 comp rx=0 tx=0).stop 2024-03-11T19:41:13.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 -- 172.21.15.83:0/2184480036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538100080 msgr2=0x7fc53819d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538100080 0x7fc53819d7e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc530005fd0 tx=0x7fc53000ef90 comp rx=0 tx=0).stop 2024-03-11T19:41:13.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 -- 172.21.15.83:0/2184480036 shutdown_connections 2024-03-11T19:41:13.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5200a1b30 0x7fc5200a3ff0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc538101c30 0x7fc5381a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc538101270 0x7fc53819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 --2- 172.21.15.83:0/2184480036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538100080 0x7fc53819d7e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 -- 172.21.15.83:0/2184480036 >> 172.21.15.83:0/2184480036 conn(0x7fc5380fb5f0 msgr2=0x7fc53810d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:13.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 -- 172.21.15.83:0/2184480036 shutdown_connections 2024-03-11T19:41:13.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.716+0000 7fc53e74f640 1 -- 172.21.15.83:0/2184480036 wait complete. 2024-03-11T19:41:13.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:41:13.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:41:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.881+0000 7f8278664640 1 -- 172.21.15.83:0/229991137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82701077c0 msgr2=0x7f8270107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.881+0000 7f8278664640 1 --2- 172.21.15.83:0/229991137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82701077c0 0x7f8270107be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f826c009900 tx=0x7f826c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:41:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.881+0000 7f8278664640 1 -- 172.21.15.83:0/229991137 shutdown_connections 2024-03-11T19:41:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.881+0000 7f8278664640 1 --2- 172.21.15.83:0/229991137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8270109370 0x7f827010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.881+0000 7f8278664640 1 --2- 172.21.15.83:0/229991137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82701089b0 0x7f8270108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.881+0000 7f8278664640 1 --2- 172.21.15.83:0/229991137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82701077c0 0x7f8270107be0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.881+0000 7f8278664640 1 -- 172.21.15.83:0/229991137 >> 172.21.15.83:0/229991137 conn(0x7f8270076790 msgr2=0x7f8270078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.882+0000 7f8278664640 1 -- 172.21.15.83:0/229991137 shutdown_connections 2024-03-11T19:41:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.882+0000 7f8278664640 1 -- 172.21.15.83:0/229991137 wait complete. 2024-03-11T19:41:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.882+0000 7f8278664640 1 Processor -- start 2024-03-11T19:41:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.882+0000 7f8278664640 1 -- start start 2024-03-11T19:41:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8278664640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82701077c0 0x7f827019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8278664640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82701089b0 0x7f827019de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8278664640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8270109370 0x7f82701a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8278664640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82701a5360 con 0x7f8270109370 2024-03-11T19:41:13.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8278664640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82701a54d0 con 0x7f82701077c0 2024-03-11T19:41:13.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8278664640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82701a5690 con 0x7f82701089b0 2024-03-11T19:41:13.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8276c01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8270109370 0x7f82701a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8276c01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8270109370 0x7f82701a4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39256/0 (socket says 172.21.15.83:39256) 2024-03-11T19:41:13.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8276c01640 1 -- 172.21.15.83:0/900400616 learned_addr learned my addr 172.21.15.83:0/900400616 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:13.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8276400640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82701077c0 0x7f827019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8275bff640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82701089b0 0x7f827019de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8276c01640 1 -- 172.21.15.83:0/900400616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82701089b0 msgr2=0x7f827019de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8276c01640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82701089b0 0x7f827019de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8276c01640 1 -- 172.21.15.83:0/900400616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82701077c0 msgr2=0x7f827019d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:13.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8276c01640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82701077c0 0x7f827019d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:13.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8276c01640 1 -- 172.21.15.83:0/900400616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268009840 con 0x7f8270109370 2024-03-11T19:41:13.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8276400640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82701077c0 0x7f827019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:13.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f8276c01640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8270109370 0x7f82701a4e20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f826800efd0 tx=0x7f826800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:13.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.883+0000 7f82677fe640 1 -- 172.21.15.83:0/900400616 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268009b40 con 0x7f8270109370 2024-03-11T19:41:13.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.884+0000 7f82677fe640 1 -- 172.21.15.83:0/900400616 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8268003b70 con 0x7f8270109370 2024-03-11T19:41:13.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.884+0000 7f82677fe640 1 -- 172.21.15.83:0/900400616 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82680108f0 con 0x7f8270109370 2024-03-11T19:41:13.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.884+0000 7f8278664640 1 -- 172.21.15.83:0/900400616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c009560 con 0x7f8270109370 2024-03-11T19:41:13.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.884+0000 7f8278664640 1 -- 172.21.15.83:0/900400616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82701a5ca0 con 0x7f8270109370 2024-03-11T19:41:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.885+0000 7f8278664640 1 -- 172.21.15.83:0/900400616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827010eb60 con 0x7f8270109370 2024-03-11T19:41:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.887+0000 7f82677fe640 1 -- 172.21.15.83:0/900400616 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8268003ce0 con 0x7f8270109370 2024-03-11T19:41:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.888+0000 7f82677fe640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f825c0a1b80 0x7f825c0a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.888+0000 7f82677fe640 1 -- 172.21.15.83:0/900400616 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8268014070 con 0x7f8270109370 2024-03-11T19:41:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.888+0000 7f8276400640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f825c0a1b80 0x7f825c0a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.889+0000 7f8276400640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f825c0a1b80 0x7f825c0a4040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f826c02c240 tx=0x7f826c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:13.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:13.894+0000 7f82677fe640 1 -- 172.21.15.83:0/900400616 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8268096b50 con 0x7f8270109370 2024-03-11T19:41:14.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.286+0000 7f8278664640 1 -- 172.21.15.83:0/900400616 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f827002ce40 con 0x7f8270109370 2024-03-11T19:41:14.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.288+0000 7f82677fe640 1 -- 172.21.15.83:0/900400616 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11353 (secure 0 0 0) 0x7f826809a210 con 0x7f8270109370 2024-03-11T19:41:14.292 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:41:14.292 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:41:06.999655+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:41:14.292 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.291+0000 7f8278664640 1 -- 172.21.15.83:0/900400616 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f825c0a1b80 msgr2=0x7f825c0a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.291+0000 7f8278664640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f825c0a1b80 0x7f825c0a4040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f826c02c240 tx=0x7f826c005fd0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.291+0000 7f8278664640 1 -- 172.21.15.83:0/900400616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8270109370 msgr2=0x7f82701a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.291+0000 7f8278664640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8270109370 0x7f82701a4e20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f826800efd0 tx=0x7f826800c6a0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.291+0000 7f8278664640 1 -- 172.21.15.83:0/900400616 shutdown_connections 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.292+0000 7f8278664640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f825c0a1b80 0x7f825c0a4040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.292+0000 7f8278664640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8270109370 0x7f82701a4e20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.292+0000 7f8278664640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82701089b0 0x7f827019de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.292+0000 7f8278664640 1 --2- 172.21.15.83:0/900400616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82701077c0 0x7f827019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.292+0000 7f8278664640 1 -- 172.21.15.83:0/900400616 >> 172.21.15.83:0/900400616 conn(0x7f8270076790 msgr2=0x7f827010bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.292+0000 7f8278664640 1 -- 172.21.15.83:0/900400616 shutdown_connections 2024-03-11T19:41:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.292+0000 7f8278664640 1 -- 172.21.15.83:0/900400616 wait complete. 2024-03-11T19:41:14.313 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:41:14.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:41:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.485+0000 7f5c5a239640 1 -- 172.21.15.83:0/2329507332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c541009f0 msgr2=0x7f5c54100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.485+0000 7f5c5a239640 1 --2- 172.21.15.83:0/2329507332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c541009f0 0x7f5c54100e70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009860 tx=0x7f5c4402bb70 comp rx=0 tx=0).stop 2024-03-11T19:41:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.486+0000 7f5c5a239640 1 -- 172.21.15.83:0/2329507332 shutdown_connections 2024-03-11T19:41:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.486+0000 7f5c5a239640 1 --2- 172.21.15.83:0/2329507332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c541013b0 0x7f5c54107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.486+0000 7f5c5a239640 1 --2- 172.21.15.83:0/2329507332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c541009f0 0x7f5c54100e70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.486+0000 7f5c5a239640 1 --2- 172.21.15.83:0/2329507332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c540ff800 0x7f5c540ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.486+0000 7f5c5a239640 1 -- 172.21.15.83:0/2329507332 >> 172.21.15.83:0/2329507332 conn(0x7f5c540fad70 msgr2=0x7f5c540fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.486+0000 7f5c5a239640 1 -- 172.21.15.83:0/2329507332 shutdown_connections 2024-03-11T19:41:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.486+0000 7f5c5a239640 1 -- 172.21.15.83:0/2329507332 wait complete. 2024-03-11T19:41:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.487+0000 7f5c5a239640 1 Processor -- start 2024-03-11T19:41:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.487+0000 7f5c5a239640 1 -- start start 2024-03-11T19:41:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c5a239640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c540ff800 0x7f5c5406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c5a239640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c541009f0 0x7f5c5406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c5a239640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c541013b0 0x7f5c54072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c5a239640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c54072ff0 con 0x7f5c540ff800 2024-03-11T19:41:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c5a239640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c54073160 con 0x7f5c541013b0 2024-03-11T19:41:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c5a239640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541adef0 con 0x7f5c541009f0 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c541009f0 0x7f5c5406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c540ff800 0x7f5c5406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c541013b0 0x7f5c54072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c540ff800 0x7f5c5406d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39268/0 (socket says 172.21.15.83:39268) 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c541009f0 0x7f5c5406da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52648/0 (socket says 172.21.15.83:52648) 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c537fe640 1 -- 172.21.15.83:0/3885923949 learned_addr learned my addr 172.21.15.83:0/3885923949 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c537fe640 1 -- 172.21.15.83:0/3885923949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c541009f0 msgr2=0x7f5c5406da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c537fe640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c541009f0 0x7f5c5406da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c537fe640 1 -- 172.21.15.83:0/3885923949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c541013b0 msgr2=0x7f5c54072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c537fe640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c541013b0 0x7f5c54072ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.488+0000 7f5c537fe640 1 -- 172.21.15.83:0/3885923949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44009490 con 0x7f5c540ff800 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.489+0000 7f5c537fe640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c540ff800 0x7f5c5406d510 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00cf30 tx=0x7f5c4c008760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.489+0000 7f5c50ff9640 1 -- 172.21.15.83:0/3885923949 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c012070 con 0x7f5c540ff800 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.489+0000 7f5c50ff9640 1 -- 172.21.15.83:0/3885923949 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c003e50 con 0x7f5c540ff800 2024-03-11T19:41:14.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.489+0000 7f5c5a239640 1 -- 172.21.15.83:0/3885923949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c541ae150 con 0x7f5c540ff800 2024-03-11T19:41:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.489+0000 7f5c50ff9640 1 -- 172.21.15.83:0/3885923949 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c018680 con 0x7f5c540ff800 2024-03-11T19:41:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.489+0000 7f5c5a239640 1 -- 172.21.15.83:0/3885923949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c541ae630 con 0x7f5c540ff800 2024-03-11T19:41:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.493+0000 7f5c50ff9640 1 -- 172.21.15.83:0/3885923949 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5c4c016070 con 0x7f5c540ff800 2024-03-11T19:41:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.494+0000 7f5c50ff9640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c3c0a1ab0 0x7f5c3c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.494+0000 7f5c50ff9640 1 -- 172.21.15.83:0/3885923949 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5c4c0ccd00 con 0x7f5c540ff800 2024-03-11T19:41:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.494+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c3c0a1ab0 0x7f5c3c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.495+0000 7f5c5ce07740 1 Processor -- start 2024-03-11T19:41:14.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.495+0000 7f5c5ce07740 1 -- start start 2024-03-11T19:41:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.495+0000 7f5c5ce07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 0x559919c44690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.495+0000 7f5c5ce07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44c60 0x559919c49760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.496+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 0x559919c44690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.496+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 0x559919c44690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32782/0 (socket says 172.21.15.83:32782) 2024-03-11T19:41:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.496+0000 7f5c537fe640 1 -- 172.21.15.83:0/1593887993 learned_addr learned my addr 172.21.15.83:0/1593887993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.496+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 0x559919c4c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.496+0000 7f5c5ce07740 1 -- 172.21.15.83:0/1593887993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559919c00960 con 0x559919c44c60 2024-03-11T19:41:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.496+0000 7f5c53fff640 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 0x559919c4c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.496+0000 7f5c5ce07740 1 -- 172.21.15.83:0/1593887993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559919bb4310 con 0x559919c44270 2024-03-11T19:41:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.496+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44c60 0x559919c49760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.496+0000 7f5c5ce07740 1 -- 172.21.15.83:0/1593887993 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559919bb1840 con 0x559919c49dd0 2024-03-11T19:41:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c537fe640 1 -- 172.21.15.83:0/1593887993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 msgr2=0x559919c4c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c537fe640 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 0x559919c4c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c537fe640 1 -- 172.21.15.83:0/1593887993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44c60 msgr2=0x559919c49760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c537fe640 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44c60 0x559919c49760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c537fe640 1 -- 172.21.15.83:0/1593887993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559919b6f060 con 0x559919c44270 2024-03-11T19:41:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c53fff640 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 0x559919c4c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44c60 0x559919c49760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:41:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c537fe640 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 0x559919c44690 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c099980 tx=0x7f5c4c099b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ed33e98424d775 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c3c0a1ab0 0x7f5c3c0a3f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009830 tx=0x7f5c44034bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c39ffb640 1 -- 172.21.15.83:0/1593887993 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0a2420 con 0x559919c44270 2024-03-11T19:41:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c39ffb640 1 -- 172.21.15.83:0/1593887993 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c097f80 con 0x559919c44270 2024-03-11T19:41:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.497+0000 7f5c39ffb640 1 -- 172.21.15.83:0/1593887993 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0a1f90 con 0x559919c44270 2024-03-11T19:41:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.498+0000 7f5c5ce07740 1 -- 172.21.15.83:0/1593887993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 msgr2=0x559919c44690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.498+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 0x559919c44690 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c099980 tx=0x7f5c4c099b80 comp rx=0 tx=0).stop 2024-03-11T19:41:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.499+0000 7f5c5ce07740 1 -- 172.21.15.83:0/1593887993 shutdown_connections 2024-03-11T19:41:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.499+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 0x559919c4c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.499+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44c60 0x559919c49760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.499+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/1593887993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 0x559919c44690 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.499+0000 7f5c5ce07740 1 -- 172.21.15.83:0/1593887993 >> 172.21.15.83:0/1593887993 conn(0x559919c3f7e0 msgr2=0x559919c3fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.499+0000 7f5c5ce07740 1 -- 172.21.15.83:0/1593887993 shutdown_connections 2024-03-11T19:41:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.499+0000 7f5c5ce07740 1 -- 172.21.15.83:0/1593887993 wait complete. 2024-03-11T19:41:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.499+0000 7f5c5ce07740 1 -- start start 2024-03-11T19:41:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c5ce07740 1 Processor -- start 2024-03-11T19:41:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c5ce07740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44270 0x559919c4e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c5ce07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44c60 0x559919c4ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c5ce07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c49dd0 0x559919c55ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44270 0x559919c4e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c49dd0 0x559919c55ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44270 0x559919c4e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52664/0 (socket says 172.21.15.83:52664) 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c537fe640 1 -- 172.21.15.83:0/4112614489 learned_addr learned my addr 172.21.15.83:0/4112614489 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c5ce07740 1 -- 172.21.15.83:0/4112614489 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559919bad020 con 0x559919c49dd0 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c5ce07740 1 -- 172.21.15.83:0/4112614489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559919880430 con 0x559919c44c60 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c5ce07740 1 -- 172.21.15.83:0/4112614489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55991983ce00 con 0x559919c44270 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44c60 0x559919c4ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.500+0000 7f5c5ce07740 10 client.0 _ll_register_callbacks cb 0x559919b54cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.501+0000 7f5c537fe640 1 -- 172.21.15.83:0/4112614489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44c60 msgr2=0x559919c4ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.501+0000 7f5c537fe640 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44c60 0x559919c4ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.501+0000 7f5c537fe640 1 -- 172.21.15.83:0/4112614489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c49dd0 msgr2=0x559919c55ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.501+0000 7f5c537fe640 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c49dd0 0x559919c55ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.501+0000 7f5c537fe640 1 -- 172.21.15.83:0/4112614489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c541ae630 con 0x559919c44270 2024-03-11T19:41:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.501+0000 7f5c537fe640 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44270 0x559919c4e540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0960e0 tx=0x7f5c4c098160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8070cb06d884668 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.501+0000 7f5c32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:41:14.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.501+0000 7f5c32ffd640 1 -- 172.21.15.83:0/4112614489 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0a2420 con 0x559919c44270 2024-03-11T19:41:14.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.501+0000 7f5c32ffd640 1 -- 172.21.15.83:0/4112614489 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5c4c0990d0 con 0x559919c44270 2024-03-11T19:41:14.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.501+0000 7f5c5ce07740 1 -- 172.21.15.83:0/4112614489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599198ac7e0 con 0x559919c44270 2024-03-11T19:41:14.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.502+0000 7f5c32ffd640 1 -- 172.21.15.83:0/4112614489 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c4c099290 con 0x559919c44270 2024-03-11T19:41:14.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.502+0000 7f5c5ce07740 10 client.25312 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:14.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.502+0000 7f5c5ce07740 1 -- 172.21.15.83:0/4112614489 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559919bab6a0 con 0x559919c44270 2024-03-11T19:41:14.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.502+0000 7f5c32ffd640 1 -- 172.21.15.83:0/4112614489 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c0993f0 con 0x559919c44270 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.502+0000 7f5c32ffd640 1 -- 172.21.15.83:0/4112614489 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5c4c098ce0 con 0x559919c44270 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.502+0000 7f5c5ce07740 10 client.25312 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.502+0000 7f5c5ce07740 10 client.25312 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.502+0000 7f5c5ce07740 20 client.25312 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.502+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559919c5a1f0 0x559919c5c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.502+0000 7f5c5ce07740 4 client.25312 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.503+0000 7f5c5ce07740 1 -- 172.21.15.83:0/4112614489 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599198ac7e0 con 0x559919c5a1f0 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.503+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559919c5a1f0 0x559919c5c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.503+0000 7f5c32ffd640 10 client.25312 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.504+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559919c5a1f0 0x559919c5c600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f49a201792fd1a54 server_cookie=64e88258a5eaa6d5 in_seq=0 out_seq=0 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.504+0000 7f5c32ffd640 1 -- 172.21.15.83:0/4112614489 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599198ac7e0 con 0x559919c5a1f0 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.504+0000 7f5c32ffd640 10 client.25312 handle_command_reply: tid=0 2024-03-11T19:41:14.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.505+0000 7f5c5ce07740 1 client.25312 shutdown 2024-03-11T19:41:14.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.505+0000 7f5c5ce07740 1 -- 172.21.15.83:0/4112614489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44270 msgr2=0x559919c4e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.505+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44270 0x559919c4e540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0960e0 tx=0x7f5c4c098160 comp rx=0 tx=0).stop 2024-03-11T19:41:14.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.505+0000 7f5c5ce07740 1 -- 172.21.15.83:0/4112614489 shutdown_connections 2024-03-11T19:41:14.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.505+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559919c5a1f0 0x559919c5c600 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.505+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c49dd0 0x559919c55ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.505+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44c60 0x559919c4ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.505+0000 7f5c32ffd640 0 client.25312 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:41:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.505+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/4112614489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44270 0x559919c4e540 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.505+0000 7f5c5ce07740 1 -- 172.21.15.83:0/4112614489 >> 172.21.15.83:0/4112614489 conn(0x559919c3f7e0 msgr2=0x559919c3eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.505+0000 7f5c5ce07740 1 -- 172.21.15.83:0/4112614489 shutdown_connections 2024-03-11T19:41:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.506+0000 7f5c5ce07740 1 -- 172.21.15.83:0/4112614489 wait complete. 2024-03-11T19:41:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.506+0000 7f5c5ce07740 20 client.25312 trim_cache size 0 max 16384 2024-03-11T19:41:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c5ce07740 1 Processor -- start 2024-03-11T19:41:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c5ce07740 1 -- start start 2024-03-11T19:41:14.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c5ce07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 0x559919c4e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c5ce07740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44c60 0x559919c55a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c5ce07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c49dd0 0x559919c3e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 0x559919c4e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 0x559919c4e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32788/0 (socket says 172.21.15.83:32788) 2024-03-11T19:41:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44c60 0x559919c55a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c5ce07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559919b32130 con 0x559919c49dd0 2024-03-11T19:41:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c5ce07740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559919c56740 con 0x559919c44270 2024-03-11T19:41:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c5ce07740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559919c4c3f0 con 0x559919c44c60 2024-03-11T19:41:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.526+0000 7f5c52ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44c60 0x559919c55a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52676/0 (socket says 172.21.15.83:52676) 2024-03-11T19:41:14.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.527+0000 7f5c52ffd640 1 -- 172.21.15.83:0/3400867003 learned_addr learned my addr 172.21.15.83:0/3400867003 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.527+0000 7f5c53fff640 1 --2- 172.21.15.83:0/3400867003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c49dd0 0x559919c3e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.527+0000 7f5c52ffd640 1 -- 172.21.15.83:0/3400867003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 msgr2=0x559919c4e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.527+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/3400867003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 0x559919c4e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.527+0000 7f5c52ffd640 1 -- 172.21.15.83:0/3400867003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c49dd0 msgr2=0x559919c3e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.527+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/3400867003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c49dd0 0x559919c3e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.527+0000 7f5c52ffd640 1 -- 172.21.15.83:0/3400867003 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559919b47c20 con 0x559919c44c60 2024-03-11T19:41:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.527+0000 7f5c53fff640 1 --2- 172.21.15.83:0/3400867003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c49dd0 0x559919c3e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:41:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.527+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/3400867003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44c60 0x559919c55a80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009860 tx=0x7f5c44047000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed10d43fa37d9e29 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.527+0000 7f5c397fa640 1 -- 172.21.15.83:0/3400867003 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4402c710 con 0x559919c44c60 2024-03-11T19:41:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.528+0000 7f5c397fa640 1 -- 172.21.15.83:0/3400867003 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c440348d0 con 0x559919c44c60 2024-03-11T19:41:14.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.528+0000 7f5c397fa640 1 -- 172.21.15.83:0/3400867003 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44004ac0 con 0x559919c44c60 2024-03-11T19:41:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.529+0000 7f5c5ce07740 1 -- 172.21.15.83:0/3400867003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44c60 msgr2=0x559919c55a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.529+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/3400867003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44c60 0x559919c55a80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009860 tx=0x7f5c44047000 comp rx=0 tx=0).stop 2024-03-11T19:41:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.529+0000 7f5c5ce07740 1 -- 172.21.15.83:0/3400867003 shutdown_connections 2024-03-11T19:41:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.529+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/3400867003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c49dd0 0x559919c3e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.529+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/3400867003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c44c60 0x559919c55a80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.529+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/3400867003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44270 0x559919c4e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.529+0000 7f5c5ce07740 1 -- 172.21.15.83:0/3400867003 >> 172.21.15.83:0/3400867003 conn(0x559919c3f7e0 msgr2=0x559919c4e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.529+0000 7f5c5ce07740 1 -- 172.21.15.83:0/3400867003 shutdown_connections 2024-03-11T19:41:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.529+0000 7f5c5ce07740 1 -- 172.21.15.83:0/3400867003 wait complete. 2024-03-11T19:41:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.530+0000 7f5c5ce07740 1 -- start start 2024-03-11T19:41:14.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.530+0000 7f5c5ce07740 1 Processor -- start 2024-03-11T19:41:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c5ce07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44270 0x559919c546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c5ce07740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44c60 0x559919c54c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c5ce07740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 0x559919cb06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44270 0x559919c546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c5ce07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559919c553f0 con 0x559919c44270 2024-03-11T19:41:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c5ce07740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559919c55560 con 0x559919c44c60 2024-03-11T19:41:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c5ce07740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559919cb0c60 con 0x559919c49dd0 2024-03-11T19:41:14.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44270 0x559919c546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39308/0 (socket says 172.21.15.83:39308) 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c537fe640 1 -- 172.21.15.83:0/499533676 learned_addr learned my addr 172.21.15.83:0/499533676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44c60 0x559919c54c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c5ce07740 10 client.0 _ll_register_callbacks cb 0x7f5c4c00aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c53fff640 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 0x559919cb06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c53fff640 1 -- 172.21.15.83:0/499533676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44c60 msgr2=0x559919c54c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c53fff640 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44c60 0x559919c54c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c53fff640 1 -- 172.21.15.83:0/499533676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44270 msgr2=0x559919c546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c53fff640 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44270 0x559919c546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.531+0000 7f5c53fff640 1 -- 172.21.15.83:0/499533676 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c44047070 con 0x559919c49dd0 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.532+0000 7f5c52ffd640 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44c60 0x559919c54c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.532+0000 7f5c537fe640 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44270 0x559919c546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.532+0000 7f5c53fff640 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 0x559919cb06d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5c480041a0 tx=0x7f5c4800bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f96926b6e7520b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.532+0000 7f5c32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:41:14.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.532+0000 7f5c32ffd640 1 -- 172.21.15.83:0/499533676 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48016040 con 0x559919c49dd0 2024-03-11T19:41:14.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.532+0000 7f5c32ffd640 1 -- 172.21.15.83:0/499533676 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5c48003e50 con 0x559919c49dd0 2024-03-11T19:41:14.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.532+0000 7f5c5ce07740 1 -- 172.21.15.83:0/499533676 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c4c097f80 con 0x559919c49dd0 2024-03-11T19:41:14.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.533+0000 7f5c32ffd640 1 -- 172.21.15.83:0/499533676 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c4801c670 con 0x559919c49dd0 2024-03-11T19:41:14.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.533+0000 7f5c5ce07740 10 client.25318 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:14.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.533+0000 7f5c5ce07740 1 -- 172.21.15.83:0/499533676 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559919b6d8c0 con 0x559919c49dd0 2024-03-11T19:41:14.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.533+0000 7f5c32ffd640 1 -- 172.21.15.83:0/499533676 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c097f80 con 0x559919c49dd0 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.534+0000 7f5c32ffd640 1 -- 172.21.15.83:0/499533676 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5c48026ab0 con 0x559919c49dd0 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.534+0000 7f5c5ce07740 10 client.25318 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.534+0000 7f5c5ce07740 10 client.25318 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.534+0000 7f5c5ce07740 20 client.25318 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.534+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559919cb26d0 0x559919cb4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.534+0000 7f5c5ce07740 4 client.25318 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.534+0000 7f5c5ce07740 1 -- 172.21.15.83:0/499533676 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5599198e9720 con 0x559919cb26d0 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.534+0000 7f5c537fe640 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559919cb26d0 0x559919cb4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.535+0000 7f5c537fe640 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559919cb26d0 0x559919cb4af0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ca3ce31b9dd9fe5 server_cookie=9bf7d190e5e23271 in_seq=0 out_seq=0 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.535+0000 7f5c32ffd640 10 client.25318 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.535+0000 7f5c32ffd640 1 -- 172.21.15.83:0/499533676 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5599198e9720 con 0x559919cb26d0 2024-03-11T19:41:14.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.535+0000 7f5c32ffd640 10 client.25318 handle_command_reply: tid=0 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.535+0000 7f5c5ce07740 1 client.25318 shutdown 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.536+0000 7f5c5ce07740 1 -- 172.21.15.83:0/499533676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 msgr2=0x559919cb06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.536+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 0x559919cb06d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5c480041a0 tx=0x7f5c4800bee0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.536+0000 7f5c5ce07740 1 -- 172.21.15.83:0/499533676 shutdown_connections 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.536+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x559919cb26d0 0x559919cb4af0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.536+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559919c49dd0 0x559919cb06d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.536+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559919c44c60 0x559919c54c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.536+0000 7f5c5ce07740 1 --2- 172.21.15.83:0/499533676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559919c44270 0x559919c546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.536+0000 7f5c5ce07740 1 -- 172.21.15.83:0/499533676 >> 172.21.15.83:0/499533676 conn(0x559919c3f7e0 msgr2=0x559919c4e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.536+0000 7f5c32ffd640 0 client.25318 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.536+0000 7f5c5ce07740 1 -- 172.21.15.83:0/499533676 shutdown_connections 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.536+0000 7f5c5ce07740 1 -- 172.21.15.83:0/499533676 wait complete. 2024-03-11T19:41:14.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.537+0000 7f5c5ce07740 20 client.25318 trim_cache size 0 max 16384 2024-03-11T19:41:14.543 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:41:14.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.539+0000 7f5c5a239640 1 -- 172.21.15.83:0/3885923949 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c3c0a1ab0 msgr2=0x7f5c3c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.539+0000 7f5c5a239640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c3c0a1ab0 0x7f5c3c0a3f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009830 tx=0x7f5c44034bf0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.539+0000 7f5c5a239640 1 -- 172.21.15.83:0/3885923949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c540ff800 msgr2=0x7f5c5406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.539+0000 7f5c5a239640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c540ff800 0x7f5c5406d510 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00cf30 tx=0x7f5c4c008760 comp rx=0 tx=0).stop 2024-03-11T19:41:14.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.539+0000 7f5c5a239640 1 -- 172.21.15.83:0/3885923949 shutdown_connections 2024-03-11T19:41:14.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.539+0000 7f5c5a239640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5c3c0a1ab0 0x7f5c3c0a3f70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.539+0000 7f5c5a239640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c541013b0 0x7f5c54072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.539+0000 7f5c5a239640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c541009f0 0x7f5c5406da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.539+0000 7f5c5a239640 1 --2- 172.21.15.83:0/3885923949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c540ff800 0x7f5c5406d510 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.539+0000 7f5c5a239640 1 -- 172.21.15.83:0/3885923949 >> 172.21.15.83:0/3885923949 conn(0x7f5c540fad70 msgr2=0x7f5c54101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.540+0000 7f5c5a239640 1 -- 172.21.15.83:0/3885923949 shutdown_connections 2024-03-11T19:41:14.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.540+0000 7f5c5a239640 1 -- 172.21.15.83:0/3885923949 wait complete. 2024-03-11T19:41:14.561 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:41:14.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:41:14.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:14 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/900400616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:41:14.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:14 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/900400616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:41:14.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:14 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/900400616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:41:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.723+0000 7f6ce239a640 1 -- 172.21.15.83:0/4266840372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc1000a0 msgr2=0x7f6cdc1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.724+0000 7f6ce239a640 1 --2- 172.21.15.83:0/4266840372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc1000a0 0x7f6cdc1004c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009900 tx=0x7f6ccc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:41:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.724+0000 7f6ce239a640 1 -- 172.21.15.83:0/4266840372 shutdown_connections 2024-03-11T19:41:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.724+0000 7f6ce239a640 1 --2- 172.21.15.83:0/4266840372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cdc101c50 0x7f6cdc1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.724+0000 7f6ce239a640 1 --2- 172.21.15.83:0/4266840372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cdc101290 0x7f6cdc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.724+0000 7f6ce239a640 1 --2- 172.21.15.83:0/4266840372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc1000a0 0x7f6cdc1004c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.724+0000 7f6ce239a640 1 -- 172.21.15.83:0/4266840372 >> 172.21.15.83:0/4266840372 conn(0x7f6cdc0fb630 msgr2=0x7f6cdc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.724+0000 7f6ce239a640 1 -- 172.21.15.83:0/4266840372 shutdown_connections 2024-03-11T19:41:14.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.724+0000 7f6ce239a640 1 -- 172.21.15.83:0/4266840372 wait complete. 2024-03-11T19:41:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.725+0000 7f6ce239a640 1 Processor -- start 2024-03-11T19:41:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.725+0000 7f6ce239a640 1 -- start start 2024-03-11T19:41:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6ce239a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cdc1000a0 0x7f6cdc06a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6ce239a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc101290 0x7f6cdc19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6ce239a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cdc101c50 0x7f6cdc19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6ce239a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc19ed60 con 0x7f6cdc101290 2024-03-11T19:41:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6ce239a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc19eed0 con 0x7f6cdc1000a0 2024-03-11T19:41:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6ce239a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc19f090 con 0x7f6cdc101c50 2024-03-11T19:41:14.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cdc1000a0 0x7f6cdc06a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6ce0937640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cdc101c50 0x7f6cdc19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cdc1000a0 0x7f6cdc06a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32806/0 (socket says 172.21.15.83:32806) 2024-03-11T19:41:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6cdbfff640 1 -- 172.21.15.83:0/4284214538 learned_addr learned my addr 172.21.15.83:0/4284214538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6cdbfff640 1 -- 172.21.15.83:0/4284214538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cdc101c50 msgr2=0x7f6cdc19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6cdbfff640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cdc101c50 0x7f6cdc19e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6cdbfff640 1 -- 172.21.15.83:0/4284214538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc101290 msgr2=0x7f6cdc19bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6cdbfff640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc101290 0x7f6cdc19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.726+0000 7f6cdbfff640 1 -- 172.21.15.83:0/4284214538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc009560 con 0x7f6cdc1000a0 2024-03-11T19:41:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.727+0000 7f6cdbfff640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cdc1000a0 0x7f6cdc06a2c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005ab0 tx=0x7f6ccc005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.727+0000 7f6cd97fa640 1 -- 172.21.15.83:0/4284214538 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0043d0 con 0x7f6cdc1000a0 2024-03-11T19:41:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.727+0000 7f6ce239a640 1 -- 172.21.15.83:0/4284214538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc195d30 con 0x7f6cdc1000a0 2024-03-11T19:41:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.727+0000 7f6cd97fa640 1 -- 172.21.15.83:0/4284214538 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ccc042070 con 0x7f6cdc1000a0 2024-03-11T19:41:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.727+0000 7f6cd97fa640 1 -- 172.21.15.83:0/4284214538 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0053c0 con 0x7f6cdc1000a0 2024-03-11T19:41:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.727+0000 7f6ce239a640 1 -- 172.21.15.83:0/4284214538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc1961f0 con 0x7f6cdc1000a0 2024-03-11T19:41:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce4f75740 1 Processor -- start 2024-03-11T19:41:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6cd97fa640 1 -- 172.21.15.83:0/4284214538 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6ccc005520 con 0x7f6cdc1000a0 2024-03-11T19:41:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce4f75740 1 -- start start 2024-03-11T19:41:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 0x564e820bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bbc60 0x564e820c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820c0dd0 0x564e820c3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e82077960 con 0x564e820bbc60 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e8202b310 con 0x564e820bb270 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e82028840 con 0x564e820c0dd0 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6cdb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820c0dd0 0x564e820c3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bbc60 0x564e820c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce0937640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 0x564e820bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce0937640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 0x564e820bb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32818/0 (socket says 172.21.15.83:32818) 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce0937640 1 -- 172.21.15.83:0/401764032 learned_addr learned my addr 172.21.15.83:0/401764032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6cd97fa640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6cc40a1a30 0x7f6cc40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce0937640 1 -- 172.21.15.83:0/401764032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820c0dd0 msgr2=0x564e820c3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce0937640 1 --2- 172.21.15.83:0/401764032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820c0dd0 0x564e820c3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce0937640 1 -- 172.21.15.83:0/401764032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bbc60 msgr2=0x564e820c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce0937640 1 --2- 172.21.15.83:0/401764032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bbc60 0x564e820c0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.731+0000 7f6ce0937640 1 -- 172.21.15.83:0/401764032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e81fe6060 con 0x564e820bb270 2024-03-11T19:41:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.732+0000 7f6cd97fa640 1 -- 172.21.15.83:0/4284214538 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6ccc044070 con 0x7f6cdc1000a0 2024-03-11T19:41:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.732+0000 7f6ce0937640 1 --2- 172.21.15.83:0/401764032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 0x564e820bb690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009fd0 tx=0x7f6cd40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e21c080fb47e1a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.732+0000 7f6ce0937640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6cc40a1a30 0x7f6cc40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.733+0000 7f6cc27fc640 1 -- 172.21.15.83:0/401764032 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4020030 con 0x564e820bb270 2024-03-11T19:41:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.733+0000 7f6cc27fc640 1 -- 172.21.15.83:0/401764032 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cd4003e70 con 0x564e820bb270 2024-03-11T19:41:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.733+0000 7f6cc27fc640 1 -- 172.21.15.83:0/401764032 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd401b430 con 0x564e820bb270 2024-03-11T19:41:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.734+0000 7f6ce4f75740 1 -- 172.21.15.83:0/401764032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 msgr2=0x564e820bb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.734+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/401764032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 0x564e820bb690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009fd0 tx=0x7f6cd40118d0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.734+0000 7f6ce0937640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6cc40a1a30 0x7f6cc40a3ef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4020000 tx=0x7f6cd401ba20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.734+0000 7f6ce4f75740 1 -- 172.21.15.83:0/401764032 shutdown_connections 2024-03-11T19:41:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.734+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/401764032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820c0dd0 0x564e820c3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.734+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/401764032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bbc60 0x564e820c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.734+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/401764032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 0x564e820bb690 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.734+0000 7f6ce4f75740 1 -- 172.21.15.83:0/401764032 >> 172.21.15.83:0/401764032 conn(0x564e820b67e0 msgr2=0x564e820b6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.734+0000 7f6ce4f75740 1 -- 172.21.15.83:0/401764032 shutdown_connections 2024-03-11T19:41:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.735+0000 7f6ce4f75740 1 -- 172.21.15.83:0/401764032 wait complete. 2024-03-11T19:41:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.735+0000 7f6ce4f75740 1 -- start start 2024-03-11T19:41:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.735+0000 7f6ce4f75740 1 Processor -- start 2024-03-11T19:41:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bb270 0x564e820c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bbc60 0x564e820c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820c0dd0 0x564e820cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e82024020 con 0x564e820bb270 2024-03-11T19:41:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e81cf7430 con 0x564e820bbc60 2024-03-11T19:41:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e81cb3e00 con 0x564e820c0dd0 2024-03-11T19:41:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6ce4f75740 10 client.0 _ll_register_callbacks cb 0x564e81fcbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6cdb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bbc60 0x564e820c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6cdb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bbc60 0x564e820c5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32824/0 (socket says 172.21.15.83:32824) 2024-03-11T19:41:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6cdb7fe640 1 -- 172.21.15.83:0/145404037 learned_addr learned my addr 172.21.15.83:0/145404037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6ce0937640 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820c0dd0 0x564e820cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6cdbfff640 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bb270 0x564e820c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6cdb7fe640 1 -- 172.21.15.83:0/145404037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820c0dd0 msgr2=0x564e820cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6cdb7fe640 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820c0dd0 0x564e820cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6cdb7fe640 1 -- 172.21.15.83:0/145404037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bb270 msgr2=0x564e820c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6cdb7fe640 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bb270 0x564e820c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6cdb7fe640 1 -- 172.21.15.83:0/145404037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd4009c80 con 0x564e820bbc60 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.736+0000 7f6ce0937640 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820c0dd0 0x564e820cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.737+0000 7f6cdb7fe640 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bbc60 0x564e820c5930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0011690 tx=0x7f6cd0011b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=309201fdc13a428b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.737+0000 7f6cbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.737+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/145404037 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd000fac0 con 0x564e820bbc60 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.737+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/145404037 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6cd000fc20 con 0x564e820bbc60 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.737+0000 7f6ce4f75740 1 -- 172.21.15.83:0/145404037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e81d237e0 con 0x564e820bbc60 2024-03-11T19:41:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.738+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/145404037 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cd0011e90 con 0x564e820bbc60 2024-03-11T19:41:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.738+0000 7f6ce4f75740 10 client.25220 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.738+0000 7f6ce4f75740 1 -- 172.21.15.83:0/145404037 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564e820226a0 con 0x564e820bbc60 2024-03-11T19:41:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.738+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/145404037 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564e81d237e0 con 0x564e820bbc60 2024-03-11T19:41:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.738+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/145404037 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6cd0022070 con 0x564e820bbc60 2024-03-11T19:41:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.738+0000 7f6ce4f75740 10 client.25220 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.738+0000 7f6ce4f75740 10 client.25220 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:41:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.738+0000 7f6ce4f75740 20 client.25220 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.738+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564e820d9d80 0x564e820dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.738+0000 7f6ce4f75740 4 client.25220 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.738+0000 7f6ce4f75740 1 -- 172.21.15.83:0/145404037 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e81d60720 con 0x564e820d9d80 2024-03-11T19:41:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.740+0000 7f6cdbfff640 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564e820d9d80 0x564e820dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.741+0000 7f6cbb7fe640 10 client.25220 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:41:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.741+0000 7f6cdbfff640 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564e820d9d80 0x564e820dc1c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27edba86312b28fd server_cookie=f908a5e3e967717d in_seq=0 out_seq=0 2024-03-11T19:41:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/145404037 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e81d60720 con 0x564e820d9d80 2024-03-11T19:41:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6cbb7fe640 10 client.25220 handle_command_reply: tid=0 2024-03-11T19:41:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6ce4f75740 1 client.25220 shutdown 2024-03-11T19:41:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6ce4f75740 1 -- 172.21.15.83:0/145404037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bbc60 msgr2=0x564e820c5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bbc60 0x564e820c5930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0011690 tx=0x7f6cd0011b50 comp rx=0 tx=0).stop 2024-03-11T19:41:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6ce4f75740 1 -- 172.21.15.83:0/145404037 shutdown_connections 2024-03-11T19:41:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564e820d9d80 0x564e820dc1c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820c0dd0 0x564e820cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bbc60 0x564e820c5930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/145404037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bb270 0x564e820c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6ce4f75740 1 -- 172.21.15.83:0/145404037 >> 172.21.15.83:0/145404037 conn(0x564e820b67e0 msgr2=0x564e820b5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.742+0000 7f6cbb7fe640 0 client.25220 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:41:14.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.743+0000 7f6ce4f75740 1 -- 172.21.15.83:0/145404037 shutdown_connections 2024-03-11T19:41:14.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.743+0000 7f6ce4f75740 1 -- 172.21.15.83:0/145404037 wait complete. 2024-03-11T19:41:14.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.743+0000 7f6ce4f75740 20 client.25220 trim_cache size 0 max 16384 2024-03-11T19:41:14.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.761+0000 7f6ce4f75740 1 Processor -- start 2024-03-11T19:41:14.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.761+0000 7f6ce4f75740 1 -- start start 2024-03-11T19:41:14.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.761+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bb270 0x564e820bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.761+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bbc60 0x564e820bd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.761+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820c0dd0 0x564e820cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.761+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e81fa9130 con 0x564e820bbc60 2024-03-11T19:41:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.761+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e81d60720 con 0x564e820c0dd0 2024-03-11T19:41:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.761+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e81d237e0 con 0x564e820bb270 2024-03-11T19:41:14.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.762+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bb270 0x564e820bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.762+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bb270 0x564e820bd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52734/0 (socket says 172.21.15.83:52734) 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.762+0000 7f6cdbfff640 1 -- 172.21.15.83:0/3109163167 learned_addr learned my addr 172.21.15.83:0/3109163167 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.762+0000 7f6ce0937640 1 --2- 172.21.15.83:0/3109163167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820c0dd0 0x564e820cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.762+0000 7f6cdb7fe640 1 --2- 172.21.15.83:0/3109163167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bbc60 0x564e820bd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.762+0000 7f6cdbfff640 1 -- 172.21.15.83:0/3109163167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820c0dd0 msgr2=0x564e820cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.762+0000 7f6cdbfff640 1 --2- 172.21.15.83:0/3109163167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820c0dd0 0x564e820cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.762+0000 7f6cdbfff640 1 -- 172.21.15.83:0/3109163167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bbc60 msgr2=0x564e820bd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.762+0000 7f6cdbfff640 1 --2- 172.21.15.83:0/3109163167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bbc60 0x564e820bd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.762+0000 7f6cdbfff640 1 -- 172.21.15.83:0/3109163167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e81fbec20 con 0x564e820bb270 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.762+0000 7f6cdbfff640 1 --2- 172.21.15.83:0/3109163167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bb270 0x564e820bd1b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc004650 tx=0x7f6ccc0f0d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e78d3a5033cfa61 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.763+0000 7f6cc1ffb640 1 -- 172.21.15.83:0/3109163167 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0baea0 con 0x564e820bb270 2024-03-11T19:41:14.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.763+0000 7f6cc1ffb640 1 -- 172.21.15.83:0/3109163167 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ccc0c0a30 con 0x564e820bb270 2024-03-11T19:41:14.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.763+0000 7f6cc1ffb640 1 -- 172.21.15.83:0/3109163167 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc03ed30 con 0x564e820bb270 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.763+0000 7f6ce4f75740 1 -- 172.21.15.83:0/3109163167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bb270 msgr2=0x564e820bd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.763+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/3109163167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bb270 0x564e820bd1b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc004650 tx=0x7f6ccc0f0d90 comp rx=0 tx=0).stop 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.764+0000 7f6ce4f75740 1 -- 172.21.15.83:0/3109163167 shutdown_connections 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.764+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/3109163167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820c0dd0 0x564e820cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.764+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/3109163167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820bbc60 0x564e820bd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.764+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/3109163167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bb270 0x564e820bd1b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.764+0000 7f6ce4f75740 1 -- 172.21.15.83:0/3109163167 >> 172.21.15.83:0/3109163167 conn(0x564e820b67e0 msgr2=0x564e820e0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.764+0000 7f6ce4f75740 1 -- 172.21.15.83:0/3109163167 shutdown_connections 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.764+0000 7f6ce4f75740 1 -- 172.21.15.83:0/3109163167 wait complete. 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.764+0000 7f6ce4f75740 1 -- start start 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.764+0000 7f6ce4f75740 1 Processor -- start 2024-03-11T19:41:14.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 0x564e820cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bbc60 0x564e820cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6ce4f75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820c0dd0 0x564e820ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e820cadd0 con 0x564e820c0dd0 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e820caf40 con 0x564e820bb270 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6ce4f75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e820cb100 con 0x564e820bbc60 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6ce4f75740 10 client.0 _ll_register_callbacks cb 0x7f6cd4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cdb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bbc60 0x564e820cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 0x564e820cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cdb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bbc60 0x564e820cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52742/0 (socket says 172.21.15.83:52742) 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cdb7fe640 1 -- 172.21.15.83:0/2180108208 learned_addr learned my addr 172.21.15.83:0/2180108208 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6ce0937640 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820c0dd0 0x564e820ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cdb7fe640 1 -- 172.21.15.83:0/2180108208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 msgr2=0x564e820cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cdb7fe640 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 0x564e820cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cdb7fe640 1 -- 172.21.15.83:0/2180108208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820c0dd0 msgr2=0x564e820ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cdb7fe640 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820c0dd0 0x564e820ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cdb7fe640 1 -- 172.21.15.83:0/2180108208 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc1961f0 con 0x564e820bbc60 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cdb7fe640 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bbc60 0x564e820cc720 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0011660 tx=0x7f6cd000f7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2613f84739d2c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/2180108208 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0011e90 con 0x564e820bbc60 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6ce4f75740 1 -- 172.21.15.83:0/2180108208 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cd4003e70 con 0x564e820bbc60 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.765+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/2180108208 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6cd0000e40 con 0x564e820bbc60 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/2180108208 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cd0004040 con 0x564e820bbc60 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6ce4f75740 10 client.25336 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6ce4f75740 1 -- 172.21.15.83:0/2180108208 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564e81fe48c0 con 0x564e820bbc60 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/2180108208 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cd4003e70 con 0x564e820bbc60 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/2180108208 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6cd0022040 con 0x564e820bbc60 2024-03-11T19:41:14.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6ce4f75740 10 client.25336 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6ce4f75740 10 client.25336 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6ce4f75740 20 client.25336 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564e82131740 0x564e82133b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6ce4f75740 4 client.25336 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6ce4f75740 1 -- 172.21.15.83:0/2180108208 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564e81daf890 con 0x564e82131740 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.766+0000 7f6cdbfff640 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564e82131740 0x564e82133b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6cbb7fe640 10 client.25336 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6cdbfff640 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564e82131740 0x564e82133b60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35c7e81dd7912d23 server_cookie=4de3dc029d7436b7 in_seq=0 out_seq=0 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6cbb7fe640 1 -- 172.21.15.83:0/2180108208 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564e81daf890 con 0x564e82131740 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6cbb7fe640 10 client.25336 handle_command_reply: tid=0 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 1 client.25336 shutdown 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 1 -- 172.21.15.83:0/2180108208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bbc60 msgr2=0x564e820cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bbc60 0x564e820cc720 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0011660 tx=0x7f6cd000f7b0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 1 -- 172.21.15.83:0/2180108208 shutdown_connections 2024-03-11T19:41:14.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564e82131740 0x564e82133b60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6cbb7fe640 0 client.25336 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:41:14.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564e820c0dd0 0x564e820ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564e820bbc60 0x564e820cc720 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 1 --2- 172.21.15.83:0/2180108208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e820bb270 0x564e820cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 1 -- 172.21.15.83:0/2180108208 >> 172.21.15.83:0/2180108208 conn(0x564e820b67e0 msgr2=0x564e820bd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 1 -- 172.21.15.83:0/2180108208 shutdown_connections 2024-03-11T19:41:14.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 1 -- 172.21.15.83:0/2180108208 wait complete. 2024-03-11T19:41:14.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.767+0000 7f6ce4f75740 20 client.25336 trim_cache size 0 max 16384 2024-03-11T19:41:14.776 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:41:14.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 -- 172.21.15.83:0/4284214538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6cc40a1a30 msgr2=0x7f6cc40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6cc40a1a30 0x7f6cc40a3ef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4020000 tx=0x7f6cd401ba20 comp rx=0 tx=0).stop 2024-03-11T19:41:14.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 -- 172.21.15.83:0/4284214538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cdc1000a0 msgr2=0x7f6cdc06a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:14.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cdc1000a0 0x7f6cdc06a2c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005ab0 tx=0x7f6ccc005e50 comp rx=0 tx=0).stop 2024-03-11T19:41:14.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 -- 172.21.15.83:0/4284214538 shutdown_connections 2024-03-11T19:41:14.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6cc40a1a30 0x7f6cc40a3ef0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cdc101c50 0x7f6cdc19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc101290 0x7f6cdc19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 --2- 172.21.15.83:0/4284214538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6cdc1000a0 0x7f6cdc06a2c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:14.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 -- 172.21.15.83:0/4284214538 >> 172.21.15.83:0/4284214538 conn(0x7f6cdc0fb630 msgr2=0x7f6cdc1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:14.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 -- 172.21.15.83:0/4284214538 shutdown_connections 2024-03-11T19:41:14.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:14.768+0000 7f6ce239a640 1 -- 172.21.15.83:0/4284214538 wait complete. 2024-03-11T19:41:14.784 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:41:14.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:41:15.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.028+0000 7f0d1a88a640 1 -- 172.21.15.83:0/845478531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d140ff440 msgr2=0x7f0d140ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:15.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.028+0000 7f0d1a88a640 1 --2- 172.21.15.83:0/845478531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d140ff440 0x7f0d140ff860 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009900 tx=0x7f0d0402bc40 comp rx=0 tx=0).stop 2024-03-11T19:41:15.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.035+0000 7f0d1a88a640 1 -- 172.21.15.83:0/845478531 shutdown_connections 2024-03-11T19:41:15.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.035+0000 7f0d1a88a640 1 --2- 172.21.15.83:0/845478531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d14100b60 0x7f0d14105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.035+0000 7f0d1a88a640 1 --2- 172.21.15.83:0/845478531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d141001a0 0x7f0d14100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.035+0000 7f0d1a88a640 1 --2- 172.21.15.83:0/845478531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d140ff440 0x7f0d140ff860 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.035+0000 7f0d1a88a640 1 -- 172.21.15.83:0/845478531 >> 172.21.15.83:0/845478531 conn(0x7f0d140fad70 msgr2=0x7f0d140fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:15.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.044+0000 7f0d1a88a640 1 -- 172.21.15.83:0/845478531 shutdown_connections 2024-03-11T19:41:15.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.047+0000 7f0d1a88a640 1 -- 172.21.15.83:0/845478531 wait complete. 2024-03-11T19:41:15.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.048+0000 7f0d1a88a640 1 Processor -- start 2024-03-11T19:41:15.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.051+0000 7f0d1a88a640 1 -- start start 2024-03-11T19:41:15.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.052+0000 7f0d1a88a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d140ff440 0x7f0d1406fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.052+0000 7f0d1a88a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d141001a0 0x7f0d14070000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.052+0000 7f0d1a88a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d14100b60 0x7f0d141ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.052+0000 7f0d1a88a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14072fb0 con 0x7f0d14100b60 2024-03-11T19:41:15.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.052+0000 7f0d1a88a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14073120 con 0x7f0d140ff440 2024-03-11T19:41:15.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.052+0000 7f0d1a88a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14106820 con 0x7f0d141001a0 2024-03-11T19:41:15.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.054+0000 7f0d13fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d140ff440 0x7f0d1406fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.054+0000 7f0d13fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d140ff440 0x7f0d1406fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32868/0 (socket says 172.21.15.83:32868) 2024-03-11T19:41:15.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.054+0000 7f0d13fff640 1 -- 172.21.15.83:0/2297019524 learned_addr learned my addr 172.21.15.83:0/2297019524 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:15.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.054+0000 7f0d18e27640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d14100b60 0x7f0d141ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.056+0000 7f0d137fe640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d141001a0 0x7f0d14070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.057+0000 7f0d13fff640 1 -- 172.21.15.83:0/2297019524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d141001a0 msgr2=0x7f0d14070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:15.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.057+0000 7f0d13fff640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d141001a0 0x7f0d14070000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.057+0000 7f0d13fff640 1 -- 172.21.15.83:0/2297019524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d14100b60 msgr2=0x7f0d141ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:15.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.057+0000 7f0d13fff640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d14100b60 0x7f0d141ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.057+0000 7f0d13fff640 1 -- 172.21.15.83:0/2297019524 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d04009560 con 0x7f0d140ff440 2024-03-11T19:41:15.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.057+0000 7f0d137fe640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d141001a0 0x7f0d14070000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:41:15.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.059+0000 7f0d13fff640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d140ff440 0x7f0d1406fac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0d04000c00 tx=0x7f0d040042c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:15.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.060+0000 7f0d117fa640 1 -- 172.21.15.83:0/2297019524 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04004df0 con 0x7f0d140ff440 2024-03-11T19:41:15.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.060+0000 7f0d1a88a640 1 -- 172.21.15.83:0/2297019524 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d14106aa0 con 0x7f0d140ff440 2024-03-11T19:41:15.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.061+0000 7f0d1a88a640 1 -- 172.21.15.83:0/2297019524 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d14107000 con 0x7f0d140ff440 2024-03-11T19:41:15.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.061+0000 7f0d117fa640 1 -- 172.21.15.83:0/2297019524 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d04005430 con 0x7f0d140ff440 2024-03-11T19:41:15.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.061+0000 7f0d117fa640 1 -- 172.21.15.83:0/2297019524 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0403e450 con 0x7f0d140ff440 2024-03-11T19:41:15.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.065+0000 7f0d1d457740 1 Processor -- start 2024-03-11T19:41:15.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.065+0000 7f0d1d457740 1 -- start start 2024-03-11T19:41:15.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.065+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504270 0x564aa5504690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.065+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504c60 0x564aa5509760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.065+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5509dd0 0x564aa550c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.065+0000 7f0d1d457740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa54c0960 con 0x564aa5509dd0 2024-03-11T19:41:15.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.065+0000 7f0d1d457740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa5474310 con 0x564aa5504c60 2024-03-11T19:41:15.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.065+0000 7f0d1d457740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa5471840 con 0x564aa5504270 2024-03-11T19:41:15.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.066+0000 7f0d137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5509dd0 0x564aa550c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.066+0000 7f0d137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5509dd0 0x564aa550c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39382/0 (socket says 172.21.15.83:39382) 2024-03-11T19:41:15.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.066+0000 7f0d137fe640 1 -- 172.21.15.83:0/1534381043 learned_addr learned my addr 172.21.15.83:0/1534381043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:15.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.066+0000 7f0d18e27640 1 --2- 172.21.15.83:0/1534381043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504270 0x564aa5504690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.067+0000 7f0d13fff640 1 --2- 172.21.15.83:0/1534381043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504c60 0x564aa5509760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.067+0000 7f0d137fe640 1 -- 172.21.15.83:0/1534381043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504270 msgr2=0x564aa5504690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:15.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.067+0000 7f0d137fe640 1 --2- 172.21.15.83:0/1534381043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504270 0x564aa5504690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.067+0000 7f0d137fe640 1 -- 172.21.15.83:0/1534381043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504c60 msgr2=0x564aa5509760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:15.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.067+0000 7f0d137fe640 1 --2- 172.21.15.83:0/1534381043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504c60 0x564aa5509760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.067+0000 7f0d137fe640 1 -- 172.21.15.83:0/1534381043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564aa542f060 con 0x564aa5509dd0 2024-03-11T19:41:15.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.070+0000 7f0d137fe640 1 --2- 172.21.15.83:0/1534381043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5509dd0 0x564aa550c260 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800cc40 tx=0x7f0d0800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcd90a0626e522b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:15.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.071+0000 7f0d117fa640 1 -- 172.21.15.83:0/2297019524 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0d040035f0 con 0x7f0d140ff440 2024-03-11T19:41:15.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.072+0000 7f0cfa7fc640 1 -- 172.21.15.83:0/1534381043 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08019660 con 0x564aa5509dd0 2024-03-11T19:41:15.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.072+0000 7f0cfa7fc640 1 -- 172.21.15.83:0/1534381043 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d08019ca0 con 0x564aa5509dd0 2024-03-11T19:41:15.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.072+0000 7f0d117fa640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cfc0a1b00 0x7f0cfc0a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.072+0000 7f0d117fa640 1 -- 172.21.15.83:0/2297019524 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0d040eff60 con 0x7f0d140ff440 2024-03-11T19:41:15.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.073+0000 7f0d18e27640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cfc0a1b00 0x7f0cfc0a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.074+0000 7f0d1d457740 1 -- 172.21.15.83:0/1534381043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5509dd0 msgr2=0x564aa550c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.074+0000 7f0d1d457740 1 --2- 172.21.15.83:0/1534381043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5509dd0 0x564aa550c260 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800cc40 tx=0x7f0d0800e720 comp rx=0 tx=0).stop 2024-03-11T19:41:15.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.074+0000 7f0d18e27640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cfc0a1b00 0x7f0cfc0a3fc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x564aa55054c0 tx=0x7f0d0c00a4d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.077+0000 7f0d1d457740 1 -- 172.21.15.83:0/1534381043 shutdown_connections 2024-03-11T19:41:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.077+0000 7f0d1d457740 1 --2- 172.21.15.83:0/1534381043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5509dd0 0x564aa550c260 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.077+0000 7f0d1d457740 1 --2- 172.21.15.83:0/1534381043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504c60 0x564aa5509760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.077+0000 7f0d1d457740 1 --2- 172.21.15.83:0/1534381043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504270 0x564aa5504690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.077+0000 7f0d1d457740 1 -- 172.21.15.83:0/1534381043 >> 172.21.15.83:0/1534381043 conn(0x564aa54ff7e0 msgr2=0x564aa54ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.078+0000 7f0d1d457740 1 -- 172.21.15.83:0/1534381043 shutdown_connections 2024-03-11T19:41:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.078+0000 7f0d1d457740 1 -- 172.21.15.83:0/1534381043 wait complete. 2024-03-11T19:41:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.079+0000 7f0d1d457740 1 -- start start 2024-03-11T19:41:15.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.079+0000 7f0d1d457740 1 Processor -- start 2024-03-11T19:41:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.080+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 0x564aa550e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.080+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504c60 0x564aa550e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.080+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5509dd0 0x564aa55159b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.080+0000 7f0d1d457740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa546d020 con 0x564aa5504270 2024-03-11T19:41:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.080+0000 7f0d1d457740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa5140430 con 0x564aa5509dd0 2024-03-11T19:41:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.080+0000 7f0d1d457740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa50fce00 con 0x564aa5504c60 2024-03-11T19:41:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.080+0000 7f0d1d457740 10 client.0 _ll_register_callbacks cb 0x564aa5414cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.083+0000 7f0d18e27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5509dd0 0x564aa55159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.083+0000 7f0d137fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504c60 0x564aa550e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.083+0000 7f0d137fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504c60 0x564aa550e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52770/0 (socket says 172.21.15.83:52770) 2024-03-11T19:41:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.083+0000 7f0d137fe640 1 -- 172.21.15.83:0/2901633425 learned_addr learned my addr 172.21.15.83:0/2901633425 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:15.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.083+0000 7f0d13fff640 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 0x564aa550e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.085+0000 7f0d13fff640 1 -- 172.21.15.83:0/2901633425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504c60 msgr2=0x564aa550e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.085+0000 7f0d13fff640 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504c60 0x564aa550e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.085+0000 7f0d13fff640 1 -- 172.21.15.83:0/2901633425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5509dd0 msgr2=0x564aa55159b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.085+0000 7f0d13fff640 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5509dd0 0x564aa55159b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.085+0000 7f0d13fff640 1 -- 172.21.15.83:0/2901633425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0800c870 con 0x564aa5504270 2024-03-11T19:41:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.086+0000 7f0d13fff640 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 0x564aa550e430 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0d04006010 tx=0x7f0d040bb020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad65e7954508d56 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:15.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.086+0000 7f0d137fe640 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504c60 0x564aa550e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:15.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.087+0000 7f0cf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:41:15.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.087+0000 7f0cf37fe640 1 -- 172.21.15.83:0/2901633425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040c1030 con 0x564aa5504270 2024-03-11T19:41:15.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.087+0000 7f0cf37fe640 1 -- 172.21.15.83:0/2901633425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0d040c5680 con 0x564aa5504270 2024-03-11T19:41:15.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.089+0000 7f0d1d457740 1 -- 172.21.15.83:0/2901633425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564aa516c7e0 con 0x564aa5504270 2024-03-11T19:41:15.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.091+0000 7f0cf37fe640 1 -- 172.21.15.83:0/2901633425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d040038a0 con 0x564aa5504270 2024-03-11T19:41:15.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.092+0000 7f0d1d457740 10 client.15735 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:15.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.092+0000 7f0d1d457740 1 -- 172.21.15.83:0/2901633425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564aa546b6a0 con 0x564aa5504270 2024-03-11T19:41:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.093+0000 7f0cf37fe640 1 -- 172.21.15.83:0/2901633425 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564aa516c7e0 con 0x564aa5504270 2024-03-11T19:41:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.093+0000 7f0cf37fe640 1 -- 172.21.15.83:0/2901633425 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0d040c0030 con 0x564aa5504270 2024-03-11T19:41:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.094+0000 7f0d1d457740 10 client.15735 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.094+0000 7f0d1d457740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.094+0000 7f0d1d457740 20 client.15735 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.094+0000 7f0d1d457740 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564aa551a340 0x564aa551c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.094+0000 7f0d1d457740 4 client.15735 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.094+0000 7f0d1d457740 1 -- 172.21.15.83:0/2901633425 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564aa51a9720 con 0x564aa551a340 2024-03-11T19:41:15.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.096+0000 7f0d137fe640 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564aa551a340 0x564aa551c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.101+0000 7f0d137fe640 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564aa551a340 0x564aa551c760 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f4e154ea180714 server_cookie=8db814c729da336f in_seq=0 out_seq=0 2024-03-11T19:41:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.101+0000 7f0cf37fe640 10 client.15735 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:15.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.106+0000 7f0cf37fe640 1 -- 172.21.15.83:0/2901633425 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564aa51a9720 con 0x564aa551a340 2024-03-11T19:41:15.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.106+0000 7f0cf37fe640 10 client.15735 handle_command_reply: tid=0 2024-03-11T19:41:15.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.106+0000 7f0d1d457740 1 client.15735 shutdown 2024-03-11T19:41:15.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.108+0000 7f0d1d457740 1 -- 172.21.15.83:0/2901633425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 msgr2=0x564aa550e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:15.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.108+0000 7f0d1d457740 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 0x564aa550e430 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0d04006010 tx=0x7f0d040bb020 comp rx=0 tx=0).stop 2024-03-11T19:41:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.108+0000 7f0d1d457740 1 -- 172.21.15.83:0/2901633425 shutdown_connections 2024-03-11T19:41:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.108+0000 7f0d1d457740 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564aa551a340 0x564aa551c760 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.108+0000 7f0d1d457740 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5509dd0 0x564aa55159b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.108+0000 7f0d1d457740 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5504c60 0x564aa550e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.108+0000 7f0d1d457740 1 --2- 172.21.15.83:0/2901633425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 0x564aa550e430 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.108+0000 7f0d1d457740 1 -- 172.21.15.83:0/2901633425 >> 172.21.15.83:0/2901633425 conn(0x564aa54ff7e0 msgr2=0x564aa54feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.109+0000 7f0cf37fe640 0 client.15735 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.109+0000 7f0d1d457740 1 -- 172.21.15.83:0/2901633425 shutdown_connections 2024-03-11T19:41:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.109+0000 7f0d1d457740 1 -- 172.21.15.83:0/2901633425 wait complete. 2024-03-11T19:41:15.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.112+0000 7f0d1d457740 20 client.15735 trim_cache size 0 max 16384 2024-03-11T19:41:15.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.150+0000 7f0d1d457740 1 Processor -- start 2024-03-11T19:41:15.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.151+0000 7f0d1d457740 1 -- start start 2024-03-11T19:41:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.151+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 0x564aa55210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.151+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504c60 0x564aa5515970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.151+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5509dd0 0x564aa5506020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.151+0000 7f0d1d457740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa53f2130 con 0x564aa5504270 2024-03-11T19:41:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.151+0000 7f0d1d457740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa5505040 con 0x564aa5504c60 2024-03-11T19:41:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.151+0000 7f0d1d457740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa5516ea0 con 0x564aa5509dd0 2024-03-11T19:41:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.152+0000 7f0d13fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 0x564aa55210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.152+0000 7f0d13fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 0x564aa55210c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39398/0 (socket says 172.21.15.83:39398) 2024-03-11T19:41:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.152+0000 7f0d13fff640 1 -- 172.21.15.83:0/2075717962 learned_addr learned my addr 172.21.15.83:0/2075717962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.156+0000 7f0d18e27640 1 --2- 172.21.15.83:0/2075717962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5509dd0 0x564aa5506020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.156+0000 7f0d137fe640 1 --2- 172.21.15.83:0/2075717962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504c60 0x564aa5515970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.157+0000 7f0d13fff640 1 -- 172.21.15.83:0/2075717962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5509dd0 msgr2=0x564aa5506020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:15.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.157+0000 7f0d13fff640 1 --2- 172.21.15.83:0/2075717962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5509dd0 0x564aa5506020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.157+0000 7f0d13fff640 1 -- 172.21.15.83:0/2075717962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504c60 msgr2=0x564aa5515970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:15.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.157+0000 7f0d13fff640 1 --2- 172.21.15.83:0/2075717962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504c60 0x564aa5515970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.157+0000 7f0d13fff640 1 -- 172.21.15.83:0/2075717962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564aa5407c20 con 0x564aa5504270 2024-03-11T19:41:15.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.162+0000 7f0d13fff640 1 --2- 172.21.15.83:0/2075717962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 0x564aa55210c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0d0402c250 tx=0x7f0d040bd060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d901e0feb10306 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.166+0000 7f0cf9ffb640 1 -- 172.21.15.83:0/2075717962 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040c5680 con 0x564aa5504270 2024-03-11T19:41:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.166+0000 7f0cf9ffb640 1 -- 172.21.15.83:0/2075717962 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d040038a0 con 0x564aa5504270 2024-03-11T19:41:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.166+0000 7f0cf9ffb640 1 -- 172.21.15.83:0/2075717962 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040bc330 con 0x564aa5504270 2024-03-11T19:41:15.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.167+0000 7f0d1d457740 1 -- 172.21.15.83:0/2075717962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 msgr2=0x564aa55210c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:15.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.168+0000 7f0d1d457740 1 --2- 172.21.15.83:0/2075717962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 0x564aa55210c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0d0402c250 tx=0x7f0d040bd060 comp rx=0 tx=0).stop 2024-03-11T19:41:15.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.168+0000 7f0d1d457740 1 -- 172.21.15.83:0/2075717962 shutdown_connections 2024-03-11T19:41:15.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.168+0000 7f0d1d457740 1 --2- 172.21.15.83:0/2075717962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5509dd0 0x564aa5506020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.169+0000 7f0d1d457740 1 --2- 172.21.15.83:0/2075717962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504c60 0x564aa5515970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.169+0000 7f0d1d457740 1 --2- 172.21.15.83:0/2075717962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504270 0x564aa55210c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.169+0000 7f0d1d457740 1 -- 172.21.15.83:0/2075717962 >> 172.21.15.83:0/2075717962 conn(0x564aa54ff7e0 msgr2=0x564aa54ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:15.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.170+0000 7f0d1d457740 1 -- 172.21.15.83:0/2075717962 shutdown_connections 2024-03-11T19:41:15.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.171+0000 7f0d1d457740 1 -- 172.21.15.83:0/2075717962 wait complete. 2024-03-11T19:41:15.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.171+0000 7f0d1d457740 1 -- start start 2024-03-11T19:41:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.172+0000 7f0d1d457740 1 Processor -- start 2024-03-11T19:41:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.173+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504270 0x564aa55152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.173+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504c60 0x564aa5515830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.173+0000 7f0d1d457740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5509dd0 0x564aa5513900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.173+0000 7f0d1d457740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa5513ec0 con 0x564aa5504c60 2024-03-11T19:41:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.173+0000 7f0d1d457740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa5514030 con 0x564aa5504270 2024-03-11T19:41:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.173+0000 7f0d1d457740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aa55141f0 con 0x564aa5509dd0 2024-03-11T19:41:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.173+0000 7f0d1d457740 10 client.0 _ll_register_callbacks cb 0x7f0d0800c3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.174+0000 7f0d137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504c60 0x564aa5515830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.174+0000 7f0d137fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504c60 0x564aa5515830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39406/0 (socket says 172.21.15.83:39406) 2024-03-11T19:41:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.174+0000 7f0d137fe640 1 -- 172.21.15.83:0/1899563594 learned_addr learned my addr 172.21.15.83:0/1899563594 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.175+0000 7f0d137fe640 1 -- 172.21.15.83:0/1899563594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5509dd0 msgr2=0x564aa5513900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.175+0000 7f0d137fe640 1 --2- 172.21.15.83:0/1899563594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5509dd0 0x564aa5513900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.175+0000 7f0d137fe640 1 -- 172.21.15.83:0/1899563594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504270 msgr2=0x564aa55152f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.175+0000 7f0d137fe640 1 --2- 172.21.15.83:0/1899563594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504270 0x564aa55152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.175+0000 7f0d137fe640 1 -- 172.21.15.83:0/1899563594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564aa546b6a0 con 0x564aa5504c60 2024-03-11T19:41:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.176+0000 7f0d137fe640 1 --2- 172.21.15.83:0/1899563594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504c60 0x564aa5515830 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x564aa551d070 tx=0x7f0d0800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511f91201a64ad0f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.177+0000 7f0cf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:41:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.178+0000 7f0cf37fe640 1 -- 172.21.15.83:0/1899563594 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0801f040 con 0x564aa5504c60 2024-03-11T19:41:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.178+0000 7f0d1d457740 1 -- 172.21.15.83:0/1899563594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d08019ca0 con 0x564aa5504c60 2024-03-11T19:41:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.178+0000 7f0cf37fe640 1 -- 172.21.15.83:0/1899563594 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0d08004240 con 0x564aa5504c60 2024-03-11T19:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.181+0000 7f0cf37fe640 1 -- 172.21.15.83:0/1899563594 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d0800c6f0 con 0x564aa5504c60 2024-03-11T19:41:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.181+0000 7f0d1d457740 10 client.15741 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.181+0000 7f0d1d457740 1 -- 172.21.15.83:0/1899563594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564aa542d8c0 con 0x564aa5504c60 2024-03-11T19:41:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.187+0000 7f0cf37fe640 1 -- 172.21.15.83:0/1899563594 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d08019ca0 con 0x564aa5504c60 2024-03-11T19:41:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.187+0000 7f0cf37fe640 1 -- 172.21.15.83:0/1899563594 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0d0803a070 con 0x564aa5504c60 2024-03-11T19:41:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.187+0000 7f0d1d457740 10 client.15741 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:15.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.187+0000 7f0d1d457740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.187+0000 7f0d1d457740 20 client.15741 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.187+0000 7f0d1d457740 1 --2- 172.21.15.83:0/1899563594 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564aa5571c90 0x564aa55740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.187+0000 7f0d1d457740 4 client.15741 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:41:15.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.187+0000 7f0d1d457740 1 -- 172.21.15.83:0/1899563594 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564aa51f8890 con 0x564aa5571c90 2024-03-11T19:41:15.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.192+0000 7f0d13fff640 1 --2- 172.21.15.83:0/1899563594 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564aa5571c90 0x564aa55740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:15.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.200+0000 7f0d13fff640 1 --2- 172.21.15.83:0/1899563594 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564aa5571c90 0x564aa55740b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7da6673f78f11ffc server_cookie=467b0c4156d81cae in_seq=0 out_seq=0 2024-03-11T19:41:15.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.200+0000 7f0cf37fe640 10 client.15741 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:15.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.202+0000 7f0cf37fe640 1 -- 172.21.15.83:0/1899563594 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564aa51f8890 con 0x564aa5571c90 2024-03-11T19:41:15.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.202+0000 7f0cf37fe640 10 client.15741 handle_command_reply: tid=0 2024-03-11T19:41:15.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.202+0000 7f0d1d457740 1 client.15741 shutdown 2024-03-11T19:41:15.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.207+0000 7f0d1d457740 1 -- 172.21.15.83:0/1899563594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504c60 msgr2=0x564aa5515830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:15.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.207+0000 7f0d1d457740 1 --2- 172.21.15.83:0/1899563594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504c60 0x564aa5515830 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x564aa551d070 tx=0x7f0d0800ed50 comp rx=0 tx=0).stop 2024-03-11T19:41:15.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.210+0000 7f0d1d457740 1 -- 172.21.15.83:0/1899563594 shutdown_connections 2024-03-11T19:41:15.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.210+0000 7f0d1d457740 1 --2- 172.21.15.83:0/1899563594 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564aa5571c90 0x564aa55740b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.210+0000 7f0d1d457740 1 --2- 172.21.15.83:0/1899563594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564aa5509dd0 0x564aa5513900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.210+0000 7f0d1d457740 1 --2- 172.21.15.83:0/1899563594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aa5504c60 0x564aa5515830 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.210+0000 7f0d1d457740 1 --2- 172.21.15.83:0/1899563594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564aa5504270 0x564aa55152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.210+0000 7f0d1d457740 1 -- 172.21.15.83:0/1899563594 >> 172.21.15.83:0/1899563594 conn(0x564aa54ff7e0 msgr2=0x564aa55210c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:15.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.210+0000 7f0cf37fe640 0 client.15741 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:15.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.213+0000 7f0d1d457740 1 -- 172.21.15.83:0/1899563594 shutdown_connections 2024-03-11T19:41:15.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.215+0000 7f0d1d457740 1 -- 172.21.15.83:0/1899563594 wait complete. 2024-03-11T19:41:15.222 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:41:15.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.217+0000 7f0d1d457740 20 client.15741 trim_cache size 0 max 16384 2024-03-11T19:41:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.220+0000 7f0d1a88a640 1 -- 172.21.15.83:0/2297019524 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cfc0a1b00 msgr2=0x7f0cfc0a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.220+0000 7f0d1a88a640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cfc0a1b00 0x7f0cfc0a3fc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x564aa55054c0 tx=0x7f0d0c00a4d0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.220+0000 7f0d1a88a640 1 -- 172.21.15.83:0/2297019524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d140ff440 msgr2=0x7f0d1406fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.220+0000 7f0d1a88a640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d140ff440 0x7f0d1406fac0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0d04000c00 tx=0x7f0d040042c0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.220+0000 7f0d1a88a640 1 -- 172.21.15.83:0/2297019524 shutdown_connections 2024-03-11T19:41:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.220+0000 7f0d1a88a640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cfc0a1b00 0x7f0cfc0a3fc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.220+0000 7f0d1a88a640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d14100b60 0x7f0d141ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.220+0000 7f0d1a88a640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d141001a0 0x7f0d14070000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.220+0000 7f0d1a88a640 1 --2- 172.21.15.83:0/2297019524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d140ff440 0x7f0d1406fac0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.220+0000 7f0d1a88a640 1 -- 172.21.15.83:0/2297019524 >> 172.21.15.83:0/2297019524 conn(0x7f0d140fad70 msgr2=0x7f0d141013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:15.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.222+0000 7f0d1a88a640 1 -- 172.21.15.83:0/2297019524 shutdown_connections 2024-03-11T19:41:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:15.222+0000 7f0d1a88a640 1 -- 172.21.15.83:0/2297019524 wait complete. 2024-03-11T19:41:15.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:15 smithi083 ceph-mon[31864]: pgmap v410: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 55 MiB/s wr, 962 op/s 2024-03-11T19:41:15.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:15 smithi137 ceph-mon[36220]: pgmap v410: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 55 MiB/s wr, 962 op/s 2024-03-11T19:41:15.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:15 smithi204 ceph-mon[36739]: pgmap v410: 97 pgs: 97 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 55 MiB/s wr, 962 op/s 2024-03-11T19:41:16.253 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:41:16.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:41:16.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:41:16.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.417+0000 7f68f9c64640 1 -- 172.21.15.83:0/4186897798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f4104570 msgr2=0x7f68f4106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.417+0000 7f68f9c64640 1 --2- 172.21.15.83:0/4186897798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f4104570 0x7f68f4106960 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009900 tx=0x7f68e802bc30 comp rx=0 tx=0).stop 2024-03-11T19:41:16.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.417+0000 7f68f9c64640 1 -- 172.21.15.83:0/4186897798 shutdown_connections 2024-03-11T19:41:16.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.417+0000 7f68f9c64640 1 --2- 172.21.15.83:0/4186897798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68f4106ea0 0x7f68f41092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.417+0000 7f68f9c64640 1 --2- 172.21.15.83:0/4186897798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f4104570 0x7f68f4106960 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.417+0000 7f68f9c64640 1 --2- 172.21.15.83:0/4186897798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68f4101c40 0x7f68f4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.417+0000 7f68f9c64640 1 -- 172.21.15.83:0/4186897798 >> 172.21.15.83:0/4186897798 conn(0x7f68f40fb630 msgr2=0x7f68f40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.418+0000 7f68f9c64640 1 -- 172.21.15.83:0/4186897798 shutdown_connections 2024-03-11T19:41:16.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.418+0000 7f68f9c64640 1 -- 172.21.15.83:0/4186897798 wait complete. 2024-03-11T19:41:16.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.418+0000 7f68f9c64640 1 Processor -- start 2024-03-11T19:41:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.418+0000 7f68f9c64640 1 -- start start 2024-03-11T19:41:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f9c64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68f4101c40 0x7f68f41972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f9c64640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68f4104570 0x7f68f4197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f9c64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f4106ea0 0x7f68f419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f9c64640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f419ede0 con 0x7f68f4106ea0 2024-03-11T19:41:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f9c64640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f419ef50 con 0x7f68f4104570 2024-03-11T19:41:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f9c64640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f419f110 con 0x7f68f4101c40 2024-03-11T19:41:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68f4101c40 0x7f68f41972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68f4101c40 0x7f68f41972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52788/0 (socket says 172.21.15.83:52788) 2024-03-11T19:41:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f37fe640 1 -- 172.21.15.83:0/659524267 learned_addr learned my addr 172.21.15.83:0/659524267 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f3fff640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f4106ea0 0x7f68f419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f37fe640 1 -- 172.21.15.83:0/659524267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68f4104570 msgr2=0x7f68f4197830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-11T19:41:16.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f37fe640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68f4104570 0x7f68f4197830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f37fe640 1 -- 172.21.15.83:0/659524267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f4106ea0 msgr2=0x7f68f419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f37fe640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f4106ea0 0x7f68f419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.419+0000 7f68f37fe640 1 -- 172.21.15.83:0/659524267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8009560 con 0x7f68f4101c40 2024-03-11T19:41:16.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.420+0000 7f68f37fe640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68f4101c40 0x7f68f41972f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f68e400e9f0 tx=0x7f68e400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.420+0000 7f68f0ff9640 1 -- 172.21.15.83:0/659524267 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4009720 con 0x7f68f4101c40 2024-03-11T19:41:16.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.420+0000 7f68f0ff9640 1 -- 172.21.15.83:0/659524267 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68e400cd80 con 0x7f68f4101c40 2024-03-11T19:41:16.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.420+0000 7f68f0ff9640 1 -- 172.21.15.83:0/659524267 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4010690 con 0x7f68f4101c40 2024-03-11T19:41:16.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.421+0000 7f68f9c64640 1 -- 172.21.15.83:0/659524267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f40ffa90 con 0x7f68f4101c40 2024-03-11T19:41:16.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.421+0000 7f68f9c64640 1 -- 172.21.15.83:0/659524267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f40fff50 con 0x7f68f4101c40 2024-03-11T19:41:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.425+0000 7f68fc83c740 1 Processor -- start 2024-03-11T19:41:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.425+0000 7f68fc83c740 1 -- start start 2024-03-11T19:41:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.425+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 0x55f39a9198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.425+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91a690 0x55f39a91ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.425+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 0x55f39a9218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.425+0000 7f68fc83c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a888f90 con 0x55f39a91b050 2024-03-11T19:41:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.425+0000 7f68fc83c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a8864c0 con 0x55f39a9194c0 2024-03-11T19:41:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.425+0000 7f68fc83c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a881ca0 con 0x55f39a91a690 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.425+0000 7f68f0ff9640 1 -- 172.21.15.83:0/659524267 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f68e40046c0 con 0x7f68f4101c40 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 0x55f39a9198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 0x55f39a9218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 0x55f39a9218e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39432/0 (socket says 172.21.15.83:39432) 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f2ffd640 1 -- 172.21.15.83:0/339430368 learned_addr learned my addr 172.21.15.83:0/339430368 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f37fe640 1 --2- 172.21.15.83:0/339430368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91a690 0x55f39a91ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f37fe640 1 -- 172.21.15.83:0/339430368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 msgr2=0x55f39a9198e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f37fe640 1 --2- 172.21.15.83:0/339430368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 0x55f39a9198e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f37fe640 1 -- 172.21.15.83:0/339430368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 msgr2=0x55f39a9218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f37fe640 1 --2- 172.21.15.83:0/339430368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 0x55f39a9218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f37fe640 1 -- 172.21.15.83:0/339430368 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f39a8440b0 con 0x55f39a91a690 2024-03-11T19:41:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f2ffd640 1 --2- 172.21.15.83:0/339430368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 0x55f39a9218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f0ff9640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f68dc0a1ab0 0x7f68dc0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.426+0000 7f68f37fe640 1 --2- 172.21.15.83:0/339430368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91a690 0x55f39a91ab10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f68e40cacb0 tx=0x7f68e40d54b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5e4cbb62863cc60 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.427+0000 7f68f0ff9640 1 -- 172.21.15.83:0/659524267 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f68e4014070 con 0x7f68f4101c40 2024-03-11T19:41:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.427+0000 7f68f3fff640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f68dc0a1ab0 0x7f68dc0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.427+0000 7f68d9ffb640 1 -- 172.21.15.83:0/339430368 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e40963f0 con 0x55f39a91a690 2024-03-11T19:41:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.427+0000 7f68d9ffb640 1 -- 172.21.15.83:0/339430368 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68e4099f20 con 0x55f39a91a690 2024-03-11T19:41:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.427+0000 7f68d9ffb640 1 -- 172.21.15.83:0/339430368 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e409a850 con 0x55f39a91a690 2024-03-11T19:41:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.428+0000 7f68fc83c740 1 -- 172.21.15.83:0/339430368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91a690 msgr2=0x55f39a91ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.428+0000 7f68fc83c740 1 --2- 172.21.15.83:0/339430368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91a690 0x55f39a91ab10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f68e40cacb0 tx=0x7f68e40d54b0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.428+0000 7f68f3fff640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f68dc0a1ab0 0x7f68dc0a3f70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55f39a91a590 tx=0x7f68ec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.428+0000 7f68fc83c740 1 -- 172.21.15.83:0/339430368 shutdown_connections 2024-03-11T19:41:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.428+0000 7f68fc83c740 1 --2- 172.21.15.83:0/339430368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 0x55f39a9218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.428+0000 7f68fc83c740 1 --2- 172.21.15.83:0/339430368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91a690 0x55f39a91ab10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.428+0000 7f68fc83c740 1 --2- 172.21.15.83:0/339430368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 0x55f39a9198e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.428+0000 7f68fc83c740 1 -- 172.21.15.83:0/339430368 >> 172.21.15.83:0/339430368 conn(0x55f39a9146a0 msgr2=0x55f39a914aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.428+0000 7f68fc83c740 1 -- 172.21.15.83:0/339430368 shutdown_connections 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.428+0000 7f68fc83c740 1 -- 172.21.15.83:0/339430368 wait complete. 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.429+0000 7f68fc83c740 1 -- start start 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.429+0000 7f68fc83c740 1 Processor -- start 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.429+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 0x55f39a934710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.429+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91a690 0x55f39a934c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.429+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91b050 0x55f39a93bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.429+0000 7f68fc83c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a826f40 con 0x55f39a91a690 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.429+0000 7f68fc83c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a5554b0 con 0x55f39a9194c0 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.429+0000 7f68fc83c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a807670 con 0x55f39a91b050 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.429+0000 7f68fc83c740 10 client.0 _ll_register_callbacks cb 0x55f39a874f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.430+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 0x55f39a934710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.430+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 0x55f39a934710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32930/0 (socket says 172.21.15.83:32930) 2024-03-11T19:41:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.430+0000 7f68f37fe640 1 -- 172.21.15.83:0/1132114477 learned_addr learned my addr 172.21.15.83:0/1132114477 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:16.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.430+0000 7f68f2ffd640 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91a690 0x55f39a934c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.430+0000 7f68f3fff640 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91b050 0x55f39a93bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.430+0000 7f68f37fe640 1 -- 172.21.15.83:0/1132114477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91b050 msgr2=0x55f39a93bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.430+0000 7f68f37fe640 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91b050 0x55f39a93bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.430+0000 7f68f37fe640 1 -- 172.21.15.83:0/1132114477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91a690 msgr2=0x55f39a934c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.430+0000 7f68f37fe640 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91a690 0x55f39a934c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.430+0000 7f68f37fe640 1 -- 172.21.15.83:0/1132114477 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f40fff50 con 0x55f39a9194c0 2024-03-11T19:41:16.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.431+0000 7f68d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:41:16.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.431+0000 7f68f37fe640 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 0x55f39a934710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f68e40a23c0 tx=0x7f68e4005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3908def406a80c55 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.432+0000 7f68d2ffd640 1 -- 172.21.15.83:0/1132114477 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e4099f20 con 0x55f39a9194c0 2024-03-11T19:41:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.432+0000 7f68d2ffd640 1 -- 172.21.15.83:0/1132114477 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f68e409a850 con 0x55f39a9194c0 2024-03-11T19:41:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.432+0000 7f68fc83c740 1 -- 172.21.15.83:0/1132114477 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f39a85bf40 con 0x55f39a9194c0 2024-03-11T19:41:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.432+0000 7f68d2ffd640 1 -- 172.21.15.83:0/1132114477 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68e409e0d0 con 0x55f39a9194c0 2024-03-11T19:41:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.432+0000 7f68fc83c740 10 client.25247 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.432+0000 7f68fc83c740 1 -- 172.21.15.83:0/1132114477 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f39a880320 con 0x55f39a9194c0 2024-03-11T19:41:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.433+0000 7f68d2ffd640 1 -- 172.21.15.83:0/1132114477 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68e409e0d0 con 0x55f39a9194c0 2024-03-11T19:41:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.433+0000 7f68d2ffd640 1 -- 172.21.15.83:0/1132114477 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f68e401f060 con 0x55f39a9194c0 2024-03-11T19:41:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.433+0000 7f68fc83c740 10 client.25247 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.433+0000 7f68fc83c740 10 client.25247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.433+0000 7f68fc83c740 20 client.25247 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.433+0000 7f68fc83c740 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f39a927020 0x55f39a929440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.433+0000 7f68fc83c740 4 client.25247 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.433+0000 7f68fc83c740 1 -- 172.21.15.83:0/1132114477 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f39a878310 con 0x55f39a927020 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.433+0000 7f68f2ffd640 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f39a927020 0x55f39a929440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.434+0000 7f68d2ffd640 10 client.25247 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.434+0000 7f68f2ffd640 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f39a927020 0x55f39a929440 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52aacb074b44e210 server_cookie=4a9fddad60a72fcc in_seq=0 out_seq=0 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.435+0000 7f68d2ffd640 1 -- 172.21.15.83:0/1132114477 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f39a878310 con 0x55f39a927020 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.435+0000 7f68d2ffd640 10 client.25247 handle_command_reply: tid=0 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.435+0000 7f68fc83c740 1 client.25247 shutdown 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.435+0000 7f68fc83c740 1 -- 172.21.15.83:0/1132114477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 msgr2=0x55f39a934710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.435+0000 7f68fc83c740 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 0x55f39a934710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f68e40a23c0 tx=0x7f68e4005720 comp rx=0 tx=0).stop 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.436+0000 7f68fc83c740 1 -- 172.21.15.83:0/1132114477 shutdown_connections 2024-03-11T19:41:16.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.436+0000 7f68fc83c740 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f39a927020 0x55f39a929440 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.436+0000 7f68fc83c740 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91b050 0x55f39a93bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.436+0000 7f68fc83c740 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91a690 0x55f39a934c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.436+0000 7f68fc83c740 1 --2- 172.21.15.83:0/1132114477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a9194c0 0x55f39a934710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.436+0000 7f68fc83c740 1 -- 172.21.15.83:0/1132114477 >> 172.21.15.83:0/1132114477 conn(0x55f39a9146a0 msgr2=0x55f39a9138e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.436+0000 7f68d2ffd640 0 client.25247 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:16.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.436+0000 7f68fc83c740 1 -- 172.21.15.83:0/1132114477 shutdown_connections 2024-03-11T19:41:16.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.436+0000 7f68fc83c740 1 -- 172.21.15.83:0/1132114477 wait complete. 2024-03-11T19:41:16.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.436+0000 7f68fc83c740 20 client.25247 trim_cache size 0 max 16384 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68fc83c740 1 Processor -- start 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68fc83c740 1 -- start start 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a9194c0 0x55f39a91e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a93bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 0x55f39a91e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68fc83c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a909350 con 0x55f39a91b050 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68fc83c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a923d00 con 0x55f39a91a690 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68fc83c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a910110 con 0x55f39a9194c0 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68f2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a93bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a9194c0 0x55f39a91e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a9194c0 0x55f39a91e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52798/0 (socket says 172.21.15.83:52798) 2024-03-11T19:41:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68f37fe640 1 -- 172.21.15.83:0/3959075067 learned_addr learned my addr 172.21.15.83:0/3959075067 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68f2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a93bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32932/0 (socket says 172.21.15.83:32932) 2024-03-11T19:41:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.459+0000 7f68f3fff640 1 --2- 172.21.15.83:0/3959075067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 0x55f39a91e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.460+0000 7f68f37fe640 1 -- 172.21.15.83:0/3959075067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 msgr2=0x55f39a93bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.460+0000 7f68f37fe640 1 --2- 172.21.15.83:0/3959075067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a93bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.460+0000 7f68f37fe640 1 -- 172.21.15.83:0/3959075067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 msgr2=0x55f39a91e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.460+0000 7f68f37fe640 1 --2- 172.21.15.83:0/3959075067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 0x55f39a91e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.460+0000 7f68f37fe640 1 -- 172.21.15.83:0/3959075067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f39a81cc00 con 0x55f39a9194c0 2024-03-11T19:41:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.460+0000 7f68f2ffd640 1 --2- 172.21.15.83:0/3959075067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a93bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.460+0000 7f68f37fe640 1 --2- 172.21.15.83:0/3959075067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a9194c0 0x55f39a91e260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f68e409a090 tx=0x7f68e40963f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef05d00d11289212 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.460+0000 7f68d97fa640 1 -- 172.21.15.83:0/3959075067 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e409e0d0 con 0x55f39a9194c0 2024-03-11T19:41:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.460+0000 7f68d97fa640 1 -- 172.21.15.83:0/3959075067 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f39a85bf40 con 0x55f39a9194c0 2024-03-11T19:41:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.460+0000 7f68d97fa640 1 -- 172.21.15.83:0/3959075067 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e40b2e80 con 0x55f39a9194c0 2024-03-11T19:41:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.462+0000 7f68fc83c740 1 -- 172.21.15.83:0/3959075067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a9194c0 msgr2=0x55f39a91e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.462+0000 7f68fc83c740 1 --2- 172.21.15.83:0/3959075067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a9194c0 0x55f39a91e260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f68e409a090 tx=0x7f68e40963f0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.462+0000 7f68fc83c740 1 -- 172.21.15.83:0/3959075067 shutdown_connections 2024-03-11T19:41:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.462+0000 7f68fc83c740 1 --2- 172.21.15.83:0/3959075067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a91b050 0x55f39a91e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.462+0000 7f68fc83c740 1 --2- 172.21.15.83:0/3959075067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a93bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.462+0000 7f68fc83c740 1 --2- 172.21.15.83:0/3959075067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a9194c0 0x55f39a91e260 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f68e409a090 tx=0x7f68e40963f0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.462+0000 7f68fc83c740 1 -- 172.21.15.83:0/3959075067 >> 172.21.15.83:0/3959075067 conn(0x55f39a9146a0 msgr2=0x55f39a92dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.462+0000 7f68fc83c740 1 -- 172.21.15.83:0/3959075067 shutdown_connections 2024-03-11T19:41:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.462+0000 7f68fc83c740 1 -- 172.21.15.83:0/3959075067 wait complete. 2024-03-11T19:41:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.463+0000 7f68fc83c740 1 -- start start 2024-03-11T19:41:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.463+0000 7f68fc83c740 1 Processor -- start 2024-03-11T19:41:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.463+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a937740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.463+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91b050 0x55f39a937c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68fc83c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a93aca0 0x55f39a93b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68fc83c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a93b660 con 0x55f39a93aca0 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68fc83c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a93b7d0 con 0x55f39a91a690 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68fc83c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39a93b990 con 0x55f39a91b050 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a93aca0 0x55f39a93b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a937740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68f3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a93aca0 0x55f39a93b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39458/0 (socket says 172.21.15.83:39458) 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68f37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a937740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32938/0 (socket says 172.21.15.83:32938) 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68f3fff640 1 -- 172.21.15.83:0/909712961 learned_addr learned my addr 172.21.15.83:0/909712961 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68fc83c740 10 client.0 _ll_register_callbacks cb 0x7f68e40d0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68f3fff640 1 -- 172.21.15.83:0/909712961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91b050 msgr2=0x55f39a937c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68f3fff640 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91b050 0x55f39a937c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68f3fff640 1 -- 172.21.15.83:0/909712961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 msgr2=0x55f39a937740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68f3fff640 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a937740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68f3fff640 1 -- 172.21.15.83:0/909712961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e8030070 con 0x55f39a93aca0 2024-03-11T19:41:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.464+0000 7f68f37fe640 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a937740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-11T19:41:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.465+0000 7f68f3fff640 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a93aca0 0x55f39a93b120 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f68ec019540 tx=0x7f68ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d11fdd3fc295f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.465+0000 7f68d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:41:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.465+0000 7f68d2ffd640 1 -- 172.21.15.83:0/909712961 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec01e070 con 0x55f39a93aca0 2024-03-11T19:41:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.465+0000 7f68d2ffd640 1 -- 172.21.15.83:0/909712961 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f68ec019570 con 0x55f39a93aca0 2024-03-11T19:41:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.465+0000 7f68fc83c740 1 -- 172.21.15.83:0/909712961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f39a8a7900 con 0x55f39a93aca0 2024-03-11T19:41:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.465+0000 7f68d2ffd640 1 -- 172.21.15.83:0/909712961 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68ec024900 con 0x55f39a93aca0 2024-03-11T19:41:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.466+0000 7f68fc83c740 10 client.15759 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.466+0000 7f68fc83c740 1 -- 172.21.15.83:0/909712961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f39a880320 con 0x55f39a93aca0 2024-03-11T19:41:16.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.466+0000 7f68d2ffd640 1 -- 172.21.15.83:0/909712961 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68ec036400 con 0x55f39a93aca0 2024-03-11T19:41:16.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.466+0000 7f68d2ffd640 1 -- 172.21.15.83:0/909712961 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f68ec03e3f0 con 0x55f39a93aca0 2024-03-11T19:41:16.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.466+0000 7f68fc83c740 10 client.15759 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:16.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.466+0000 7f68fc83c740 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:16.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.466+0000 7f68fc83c740 20 client.15759 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:16.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.466+0000 7f68fc83c740 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f39a9a9750 0x55f39a9abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.466+0000 7f68fc83c740 4 client.15759 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:41:16.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.466+0000 7f68fc83c740 1 -- 172.21.15.83:0/909712961 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f39a8a7900 con 0x55f39a9a9750 2024-03-11T19:41:16.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.474+0000 7f68f37fe640 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f39a9a9750 0x55f39a9abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.475+0000 7f68f37fe640 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f39a9a9750 0x55f39a9abb70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258ea6d15a99d53c server_cookie=2b4c4f5a90aaef5f in_seq=0 out_seq=0 2024-03-11T19:41:16.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.475+0000 7f68d2ffd640 10 client.15759 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:16.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.477+0000 7f68d2ffd640 1 -- 172.21.15.83:0/909712961 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f39a8a7900 con 0x55f39a9a9750 2024-03-11T19:41:16.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.477+0000 7f68d2ffd640 10 client.15759 handle_command_reply: tid=0 2024-03-11T19:41:16.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.477+0000 7f68fc83c740 1 client.15759 shutdown 2024-03-11T19:41:16.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.480+0000 7f68fc83c740 1 -- 172.21.15.83:0/909712961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a93aca0 msgr2=0x55f39a93b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.480+0000 7f68fc83c740 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a93aca0 0x55f39a93b120 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f68ec019540 tx=0x7f68ec004ab0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.480+0000 7f68fc83c740 1 -- 172.21.15.83:0/909712961 shutdown_connections 2024-03-11T19:41:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.480+0000 7f68fc83c740 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f39a9a9750 0x55f39a9abb70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.480+0000 7f68fc83c740 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f39a93aca0 0x55f39a93b120 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.480+0000 7f68d2ffd640 0 client.15759 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.480+0000 7f68fc83c740 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f39a91b050 0x55f39a937c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.480+0000 7f68fc83c740 1 --2- 172.21.15.83:0/909712961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f39a91a690 0x55f39a937740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.480+0000 7f68fc83c740 1 -- 172.21.15.83:0/909712961 >> 172.21.15.83:0/909712961 conn(0x55f39a9146a0 msgr2=0x55f39a91e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.480+0000 7f68fc83c740 1 -- 172.21.15.83:0/909712961 shutdown_connections 2024-03-11T19:41:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.481+0000 7f68fc83c740 1 -- 172.21.15.83:0/909712961 wait complete. 2024-03-11T19:41:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.481+0000 7f68fc83c740 20 client.15759 trim_cache size 0 max 16384 2024-03-11T19:41:16.486 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:41:16.486 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:41:16.486 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "045be88a-fcf8-4926-800a-f5ad9a35c4a8", 2024-03-11T19:41:16.486 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:41:16.486 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:41:16.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.482+0000 7f68f9c64640 1 -- 172.21.15.83:0/659524267 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f68dc0a1ab0 msgr2=0x7f68dc0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.482+0000 7f68f9c64640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f68dc0a1ab0 0x7f68dc0a3f70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55f39a91a590 tx=0x7f68ec009500 comp rx=0 tx=0).stop 2024-03-11T19:41:16.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.482+0000 7f68f9c64640 1 -- 172.21.15.83:0/659524267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68f4101c40 msgr2=0x7f68f41972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.482+0000 7f68f9c64640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68f4101c40 0x7f68f41972f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f68e400e9f0 tx=0x7f68e400edf0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.482+0000 7f68f9c64640 1 -- 172.21.15.83:0/659524267 shutdown_connections 2024-03-11T19:41:16.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.482+0000 7f68f9c64640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f68dc0a1ab0 0x7f68dc0a3f70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.482+0000 7f68f9c64640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68f4106ea0 0x7f68f419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.482+0000 7f68f9c64640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68f4104570 0x7f68f4197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.482+0000 7f68f9c64640 1 --2- 172.21.15.83:0/659524267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68f4101c40 0x7f68f41972f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.482+0000 7f68f9c64640 1 -- 172.21.15.83:0/659524267 >> 172.21.15.83:0/659524267 conn(0x7f68f40fb630 msgr2=0x7f68f41026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.483+0000 7f68f9c64640 1 -- 172.21.15.83:0/659524267 shutdown_connections 2024-03-11T19:41:16.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.483+0000 7f68f9c64640 1 -- 172.21.15.83:0/659524267 wait complete. 2024-03-11T19:41:16.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:41:16.506 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:41:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.657+0000 7f6addd7a640 1 -- 172.21.15.83:0/2920649066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ad8106f30 msgr2=0x7f6ad8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.657+0000 7f6addd7a640 1 --2- 172.21.15.83:0/2920649066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ad8106f30 0x7f6ad8109320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009ca0 tx=0x7f6ad002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.657+0000 7f6addd7a640 1 -- 172.21.15.83:0/2920649066 shutdown_connections 2024-03-11T19:41:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.657+0000 7f6addd7a640 1 --2- 172.21.15.83:0/2920649066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ad8106f30 0x7f6ad8109320 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.657+0000 7f6addd7a640 1 --2- 172.21.15.83:0/2920649066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad8104600 0x7f6ad81069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.657+0000 7f6addd7a640 1 --2- 172.21.15.83:0/2920649066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad8101cd0 0x7f6ad81040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.657+0000 7f6addd7a640 1 -- 172.21.15.83:0/2920649066 >> 172.21.15.83:0/2920649066 conn(0x7f6ad80fb620 msgr2=0x7f6ad80fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.657+0000 7f6addd7a640 1 -- 172.21.15.83:0/2920649066 shutdown_connections 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.657+0000 7f6addd7a640 1 -- 172.21.15.83:0/2920649066 wait complete. 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.657+0000 7f6addd7a640 1 Processor -- start 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.657+0000 7f6addd7a640 1 -- start start 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6addd7a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ad8101cd0 0x7f6ad8110290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6addd7a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad8104600 0x7f6ad81107d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6addd7a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad8106f30 0x7f6ad8111810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6addd7a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad8111d50 con 0x7f6ad8106f30 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6addd7a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad8111ec0 con 0x7f6ad8101cd0 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6addd7a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad8112080 con 0x7f6ad8104600 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad8106f30 0x7f6ad8111810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad8106f30 0x7f6ad8111810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39470/0 (socket says 172.21.15.83:39470) 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad7fff640 1 -- 172.21.15.83:0/1097646711 learned_addr learned my addr 172.21.15.83:0/1097646711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6acffff640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad8104600 0x7f6ad81107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad77fe640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ad8101cd0 0x7f6ad8110290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad7fff640 1 -- 172.21.15.83:0/1097646711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad8104600 msgr2=0x7f6ad81107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad7fff640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad8104600 0x7f6ad81107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad7fff640 1 -- 172.21.15.83:0/1097646711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ad8101cd0 msgr2=0x7f6ad8110290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad7fff640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ad8101cd0 0x7f6ad8110290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad7fff640 1 -- 172.21.15.83:0/1097646711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad0009900 con 0x7f6ad8106f30 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6acffff640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad8104600 0x7f6ad81107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad7fff640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad8106f30 0x7f6ad8111810 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009bf0 tx=0x7f6ad0034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad57fa640 1 -- 172.21.15.83:0/1097646711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00323b0 con 0x7f6ad8106f30 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6addd7a640 1 -- 172.21.15.83:0/1097646711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad8112300 con 0x7f6ad8106f30 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad57fa640 1 -- 172.21.15.83:0/1097646711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ad0003e50 con 0x7f6ad8106f30 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.658+0000 7f6ad57fa640 1 -- 172.21.15.83:0/1097646711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0040450 con 0x7f6ad8106f30 2024-03-11T19:41:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.659+0000 7f6addd7a640 1 -- 172.21.15.83:0/1097646711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad81127b0 con 0x7f6ad8106f30 2024-03-11T19:41:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.662+0000 7f6ad57fa640 1 -- 172.21.15.83:0/1097646711 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6ad000a040 con 0x7f6ad8106f30 2024-03-11T19:41:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ae0a2e740 1 Processor -- start 2024-03-11T19:41:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ae0a2e740 1 -- start start 2024-03-11T19:41:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ad57fa640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ac40a1a70 0x7f6ac40a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62dbcc0 0x5599d62e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62e0e30 0x5599d62e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ae0a2e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d624b280 con 0x5599d62e0e30 2024-03-11T19:41:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ae0a2e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d62487b0 con 0x5599d62dbcc0 2024-03-11T19:41:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ae0a2e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d6243f90 con 0x5599d62db2d0 2024-03-11T19:41:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ad57fa640 1 -- 172.21.15.83:0/1097646711 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6ad00f0080 con 0x7f6ad8106f30 2024-03-11T19:41:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ad77fe640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ac40a1a70 0x7f6ac40a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ad7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62e0e30 0x5599d62e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ad7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62e0e30 0x5599d62e32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39482/0 (socket says 172.21.15.83:39482) 2024-03-11T19:41:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6acffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62dbcc0 0x5599d62e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6ad7fff640 1 -- 172.21.15.83:0/1204789159 learned_addr learned my addr 172.21.15.83:0/1204789159 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.663+0000 7f6acffff640 1 --2- 172.21.15.83:0/1204789159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.664+0000 7f6ad7fff640 1 -- 172.21.15.83:0/1204789159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 msgr2=0x5599d62db6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.664+0000 7f6ad7fff640 1 --2- 172.21.15.83:0/1204789159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62db6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.664+0000 7f6ad7fff640 1 -- 172.21.15.83:0/1204789159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62dbcc0 msgr2=0x5599d62e07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.664+0000 7f6ad7fff640 1 --2- 172.21.15.83:0/1204789159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62dbcc0 0x5599d62e07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.664+0000 7f6ad7fff640 1 -- 172.21.15.83:0/1204789159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599d6202570 con 0x5599d62e0e30 2024-03-11T19:41:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.664+0000 7f6acffff640 1 --2- 172.21.15.83:0/1204789159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62dbcc0 0x5599d62e07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.664+0000 7f6ad77fe640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ac40a1a70 0x7f6ac40a3f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009bd0 tx=0x7f6ac8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.664+0000 7f6ad7fff640 1 --2- 172.21.15.83:0/1204789159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62e0e30 0x5599d62e32c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0005e30 tx=0x7f6ad0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfe735740963fef server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.664+0000 7f6acdffb640 1 -- 172.21.15.83:0/1204789159 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00d5fb0 con 0x5599d62e0e30 2024-03-11T19:41:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.664+0000 7f6acdffb640 1 -- 172.21.15.83:0/1204789159 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ad00d5dd0 con 0x5599d62e0e30 2024-03-11T19:41:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.664+0000 7f6acdffb640 1 -- 172.21.15.83:0/1204789159 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00b9710 con 0x5599d62e0e30 2024-03-11T19:41:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.665+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/1204789159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62e0e30 msgr2=0x5599d62e32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.665+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/1204789159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62e0e30 0x5599d62e32c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0005e30 tx=0x7f6ad0004470 comp rx=0 tx=0).stop 2024-03-11T19:41:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.665+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/1204789159 shutdown_connections 2024-03-11T19:41:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.665+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/1204789159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62e0e30 0x5599d62e32c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.665+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/1204789159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62dbcc0 0x5599d62e07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.665+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/1204789159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62db6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.665+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/1204789159 >> 172.21.15.83:0/1204789159 conn(0x5599d62d67f0 msgr2=0x5599d62d6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.665+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/1204789159 shutdown_connections 2024-03-11T19:41:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.665+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/1204789159 wait complete. 2024-03-11T19:41:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.665+0000 7f6ae0a2e740 1 -- start start 2024-03-11T19:41:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.666+0000 7f6ae0a2e740 1 Processor -- start 2024-03-11T19:41:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.666+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62db2d0 0x5599d62e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.666+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 0x5599d62e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.666+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62e0e30 0x5599d62ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.666+0000 7f6ae0a2e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d5f17430 con 0x5599d62dbcc0 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.666+0000 7f6ae0a2e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d5ed3e00 con 0x5599d62db2d0 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.666+0000 7f6ae0a2e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d61c8fd0 con 0x5599d62e0e30 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.666+0000 7f6ae0a2e740 10 client.0 _ll_register_callbacks cb 0x5599d621efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6acffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 0x5599d62e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6acffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 0x5599d62e55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39494/0 (socket says 172.21.15.83:39494) 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6acffff640 1 -- 172.21.15.83:0/182114361 learned_addr learned my addr 172.21.15.83:0/182114361 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6ad7fff640 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62e0e30 0x5599d62ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6ad7fff640 1 -- 172.21.15.83:0/182114361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62db2d0 msgr2=0x5599d62e5090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6ad7fff640 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62db2d0 0x5599d62e5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6ad7fff640 1 -- 172.21.15.83:0/182114361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 msgr2=0x5599d62e55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6ad7fff640 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 0x5599d62e55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6ad7fff640 1 -- 172.21.15.83:0/182114361 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad81127b0 con 0x5599d62e0e30 2024-03-11T19:41:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6ad7fff640 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62e0e30 0x5599d62ec610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00bd260 tx=0x7f6ad00bb5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce49291a9131d53f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6ab6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:41:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.667+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/182114361 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00d9070 con 0x5599d62e0e30 2024-03-11T19:41:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/182114361 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6ad00bd7a0 con 0x5599d62e0e30 2024-03-11T19:41:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/182114361 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599d5f437e0 con 0x5599d62e0e30 2024-03-11T19:41:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/182114361 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ad00d1300 con 0x5599d62e0e30 2024-03-11T19:41:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ae0a2e740 10 client.25375 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/182114361 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5599d6242610 con 0x5599d62e0e30 2024-03-11T19:41:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/182114361 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ad00bb140 con 0x5599d62e0e30 2024-03-11T19:41:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/182114361 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6ad00c5440 con 0x5599d62e0e30 2024-03-11T19:41:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ae0a2e740 10 client.25375 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ae0a2e740 10 client.25375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ae0a2e740 20 client.25375 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d6301e30 0x5599d63042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ae0a2e740 4 client.25375 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/182114361 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599d5f437e0 con 0x5599d6301e30 2024-03-11T19:41:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.668+0000 7f6ad77fe640 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d6301e30 0x5599d63042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.669+0000 7f6ad77fe640 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d6301e30 0x5599d63042f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=429ac765f4c7dec1 server_cookie=a4209ba0a112d370 in_seq=0 out_seq=0 2024-03-11T19:41:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.669+0000 7f6ab6ffd640 10 client.25375 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/182114361 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599d5f437e0 con 0x5599d6301e30 2024-03-11T19:41:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ab6ffd640 10 client.25375 handle_command_reply: tid=0 2024-03-11T19:41:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 1 client.25375 shutdown 2024-03-11T19:41:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/182114361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62e0e30 msgr2=0x5599d62ec610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62e0e30 0x5599d62ec610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00bd260 tx=0x7f6ad00bb5b0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/182114361 shutdown_connections 2024-03-11T19:41:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d6301e30 0x5599d63042f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62e0e30 0x5599d62ec610 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ab6ffd640 0 client.25375 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 0x5599d62e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/182114361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62db2d0 0x5599d62e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/182114361 >> 172.21.15.83:0/182114361 conn(0x5599d62d67f0 msgr2=0x5599d62d5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/182114361 shutdown_connections 2024-03-11T19:41:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/182114361 wait complete. 2024-03-11T19:41:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.671+0000 7f6ae0a2e740 20 client.25375 trim_cache size 0 max 16384 2024-03-11T19:41:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.698+0000 7f6ae0a2e740 1 Processor -- start 2024-03-11T19:41:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.698+0000 7f6ae0a2e740 1 -- start start 2024-03-11T19:41:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.698+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.698+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 0x5599d62ec5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.698+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62e0e30 0x5599d62e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.698+0000 7f6ae0a2e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d62e4d20 con 0x5599d62dbcc0 2024-03-11T19:41:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.698+0000 7f6ae0a2e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d5f437e0 con 0x5599d62e0e30 2024-03-11T19:41:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.698+0000 7f6ae0a2e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d62d22b0 con 0x5599d62db2d0 2024-03-11T19:41:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.699+0000 7f6acffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 0x5599d62ec5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.699+0000 7f6acffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 0x5599d62ec5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39506/0 (socket says 172.21.15.83:39506) 2024-03-11T19:41:16.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.699+0000 7f6acffff640 1 -- 172.21.15.83:0/1902918905 learned_addr learned my addr 172.21.15.83:0/1902918905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.699+0000 7f6acffff640 1 -- 172.21.15.83:0/1902918905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 msgr2=0x5599d62e5090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:41:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.699+0000 7f6acffff640 1 --2- 172.21.15.83:0/1902918905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62e5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.699+0000 7f6acffff640 1 -- 172.21.15.83:0/1902918905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62e0e30 msgr2=0x5599d62e55d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:41:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.699+0000 7f6acffff640 1 --2- 172.21.15.83:0/1902918905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62e0e30 0x5599d62e55d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.699+0000 7f6acffff640 1 -- 172.21.15.83:0/1902918905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599d61deac0 con 0x5599d62dbcc0 2024-03-11T19:41:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.700+0000 7f6acffff640 1 --2- 172.21.15.83:0/1902918905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 0x5599d62ec5d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0004390 tx=0x7f6ac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78d9a2664d40248 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.702+0000 7f6acd7fa640 1 -- 172.21.15.83:0/1902918905 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac00116a0 con 0x5599d62dbcc0 2024-03-11T19:41:16.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.702+0000 7f6acd7fa640 1 -- 172.21.15.83:0/1902918905 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ac0017040 con 0x5599d62dbcc0 2024-03-11T19:41:16.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.702+0000 7f6acd7fa640 1 -- 172.21.15.83:0/1902918905 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0020ba0 con 0x5599d62dbcc0 2024-03-11T19:41:16.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.703+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/1902918905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 msgr2=0x5599d62ec5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.703+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/1902918905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 0x5599d62ec5d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0004390 tx=0x7f6ac00046c0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.703+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/1902918905 shutdown_connections 2024-03-11T19:41:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.704+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/1902918905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62e0e30 0x5599d62e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.704+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/1902918905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62dbcc0 0x5599d62ec5d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.704+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/1902918905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.704+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/1902918905 >> 172.21.15.83:0/1902918905 conn(0x5599d62d67f0 msgr2=0x5599d6308c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.706+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/1902918905 shutdown_connections 2024-03-11T19:41:16.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.706+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/1902918905 wait complete. 2024-03-11T19:41:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.706+0000 7f6ae0a2e740 1 -- start start 2024-03-11T19:41:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.707+0000 7f6ae0a2e740 1 Processor -- start 2024-03-11T19:41:16.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.707+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62ec2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.707+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62dbcc0 0x5599d62ea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.708+0000 7f6ae0a2e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62e0e30 0x5599d62eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.708+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62ec2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.708+0000 7f6ad77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62ec2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52880/0 (socket says 172.21.15.83:52880) 2024-03-11T19:41:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.708+0000 7f6ad77fe640 1 -- 172.21.15.83:0/4272343549 learned_addr learned my addr 172.21.15.83:0/4272343549 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.708+0000 7f6acffff640 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62dbcc0 0x5599d62ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.708+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/4272343549 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d62eb3e0 con 0x5599d62e0e30 2024-03-11T19:41:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.708+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/4272343549 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d62eb550 con 0x5599d62dbcc0 2024-03-11T19:41:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.708+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/4272343549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d62eb710 con 0x5599d62db2d0 2024-03-11T19:41:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.708+0000 7f6ae0a2e740 10 client.0 _ll_register_callbacks cb 0x7f6ad0032bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.709+0000 7f6ad77fe640 1 -- 172.21.15.83:0/4272343549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62dbcc0 msgr2=0x5599d62ea960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.709+0000 7f6ad77fe640 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62dbcc0 0x5599d62ea960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.709+0000 7f6ad77fe640 1 -- 172.21.15.83:0/4272343549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62e0e30 msgr2=0x5599d62eaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.709+0000 7f6ad77fe640 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62e0e30 0x5599d62eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.709+0000 7f6ad77fe640 1 -- 172.21.15.83:0/4272343549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac0004150 con 0x5599d62db2d0 2024-03-11T19:41:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.710+0000 7f6ad77fe640 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62ec2e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5599d6304c00 tx=0x7f6ac8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cbad60e498c5f59 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.710+0000 7f6ab6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:41:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.710+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/4272343549 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8024070 con 0x5599d62db2d0 2024-03-11T19:41:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.710+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/4272343549 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6ac801fe10 con 0x5599d62db2d0 2024-03-11T19:41:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.710+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/4272343549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ad00d5dd0 con 0x5599d62db2d0 2024-03-11T19:41:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.710+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/4272343549 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ac8033030 con 0x5599d62db2d0 2024-03-11T19:41:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.711+0000 7f6ae0a2e740 10 client.25378 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.711+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/4272343549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5599d62046a0 con 0x5599d62db2d0 2024-03-11T19:41:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.711+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/4272343549 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ad00d5dd0 con 0x5599d62db2d0 2024-03-11T19:41:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.711+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/4272343549 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6ac801a040 con 0x5599d62db2d0 2024-03-11T19:41:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.711+0000 7f6ae0a2e740 10 client.25378 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.711+0000 7f6ae0a2e740 10 client.25378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.711+0000 7f6ae0a2e740 20 client.25378 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.711+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d6301e30 0x5599d635bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.711+0000 7f6ae0a2e740 4 client.25378 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.711+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/4272343549 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599d5f80720 con 0x5599d6301e30 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.711+0000 7f6acffff640 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d6301e30 0x5599d635bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.712+0000 7f6acffff640 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d6301e30 0x5599d635bcb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=777fca78ef9083b server_cookie=b6d1ed67c52da0c8 in_seq=0 out_seq=0 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.712+0000 7f6ab6ffd640 10 client.25378 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.712+0000 7f6ab6ffd640 1 -- 172.21.15.83:0/4272343549 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5599d5f80720 con 0x5599d6301e30 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.712+0000 7f6ab6ffd640 10 client.25378 handle_command_reply: tid=0 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.712+0000 7f6ae0a2e740 1 client.25378 shutdown 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.713+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/4272343549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 msgr2=0x5599d62ec2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.713+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62ec2e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5599d6304c00 tx=0x7f6ac8004730 comp rx=0 tx=0).stop 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.713+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/4272343549 shutdown_connections 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.713+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5599d6301e30 0x5599d635bcb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.713+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599d62e0e30 0x5599d62eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.713+0000 7f6ab6ffd640 0 client.25378 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.713+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599d62dbcc0 0x5599d62ea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.713+0000 7f6ae0a2e740 1 --2- 172.21.15.83:0/4272343549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599d62db2d0 0x5599d62ec2e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.713+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/4272343549 >> 172.21.15.83:0/4272343549 conn(0x5599d62d67f0 msgr2=0x5599d62e5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.714+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/4272343549 shutdown_connections 2024-03-11T19:41:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.714+0000 7f6ae0a2e740 1 -- 172.21.15.83:0/4272343549 wait complete. 2024-03-11T19:41:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.714+0000 7f6ae0a2e740 20 client.25378 trim_cache size 0 max 16384 2024-03-11T19:41:16.720 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:41:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:41:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:41:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "045be88a-fcf8-4926-800a-f5ad9a35c4a8": { 2024-03-11T19:41:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:41:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "045be88a-fcf8-4926-800a-f5ad9a35c4a8", 2024-03-11T19:41:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:41:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:41:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:41:16.720 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:41:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.717+0000 7f6addd7a640 1 -- 172.21.15.83:0/1097646711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ac40a1a70 msgr2=0x7f6ac40a3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.717+0000 7f6addd7a640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ac40a1a70 0x7f6ac40a3f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009bd0 tx=0x7f6ac8005f90 comp rx=0 tx=0).stop 2024-03-11T19:41:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.717+0000 7f6addd7a640 1 -- 172.21.15.83:0/1097646711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad8106f30 msgr2=0x7f6ad8111810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.717+0000 7f6addd7a640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad8106f30 0x7f6ad8111810 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009bf0 tx=0x7f6ad0034cc0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.717+0000 7f6addd7a640 1 -- 172.21.15.83:0/1097646711 shutdown_connections 2024-03-11T19:41:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.717+0000 7f6addd7a640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ac40a1a70 0x7f6ac40a3f30 secure :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009bd0 tx=0x7f6ac8005f90 comp rx=0 tx=0).stop 2024-03-11T19:41:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.717+0000 7f6addd7a640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ad8106f30 0x7f6ad8111810 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.717+0000 7f6addd7a640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ad8104600 0x7f6ad81107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.717+0000 7f6addd7a640 1 --2- 172.21.15.83:0/1097646711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ad8101cd0 0x7f6ad8110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.717+0000 7f6addd7a640 1 -- 172.21.15.83:0/1097646711 >> 172.21.15.83:0/1097646711 conn(0x7f6ad80fb620 msgr2=0x7f6ad8102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.718+0000 7f6addd7a640 1 -- 172.21.15.83:0/1097646711 shutdown_connections 2024-03-11T19:41:16.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:16.718+0000 7f6addd7a640 1 -- 172.21.15.83:0/1097646711 wait complete. 2024-03-11T19:41:16.738 INFO:tasks.cephfs.filesystem:scrub status for tag:045be88a-fcf8-4926-800a-f5ad9a35c4a8 - {'path': '/', 'tag': '045be88a-fcf8-4926-800a-f5ad9a35c4a8', 'options': 'recursive,force'} 2024-03-11T19:41:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:16 smithi083 ceph-mon[31864]: pgmap v411: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 63 MiB/s wr, 1.11k op/s 2024-03-11T19:41:17.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:16 smithi137 ceph-mon[36220]: pgmap v411: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 63 MiB/s wr, 1.11k op/s 2024-03-11T19:41:17.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:16 smithi204 ceph-mon[36739]: pgmap v411: 97 pgs: 97 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 63 MiB/s wr, 1.11k op/s 2024-03-11T19:41:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 120.0 s, 3799.4 tps, lat 25.925 ms stddev 44.882 2024-03-11T19:41:18.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:17 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:41:18.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:17 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:17 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:41:18.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:17 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:41:18.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:17 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:17 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:41:18.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:17 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:41:18.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:17 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:17 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:17 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:17 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:41:18.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:17 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:17 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:41:18.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:17 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:41:18.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:17 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:17 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:41:18.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:17 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:41:18.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:17 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:17 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:17 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:17 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:41:18.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:17 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:17 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:41:18.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:17 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:41:18.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:17 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:17 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:41:18.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:17 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:41:18.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:17 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:17 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:18.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:17 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:18 smithi083 ceph-mon[31864]: pgmap v412: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 74 MiB/s wr, 1.55k op/s 2024-03-11T19:41:19.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:18 smithi137 ceph-mon[36220]: pgmap v412: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 74 MiB/s wr, 1.55k op/s 2024-03-11T19:41:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:18 smithi204 ceph-mon[36739]: pgmap v412: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 74 MiB/s wr, 1.55k op/s 2024-03-11T19:41:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:20 smithi083 ceph-mon[31864]: pgmap v413: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 69 MiB/s wr, 1.62k op/s 2024-03-11T19:41:21.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:20 smithi137 ceph-mon[36220]: pgmap v413: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 69 MiB/s wr, 1.62k op/s 2024-03-11T19:41:21.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:20 smithi204 ceph-mon[36739]: pgmap v413: 97 pgs: 97 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 69 MiB/s wr, 1.62k op/s 2024-03-11T19:41:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 125.0 s, 4372.2 tps, lat 23.077 ms stddev 26.725 2024-03-11T19:41:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:22 smithi083 ceph-mon[31864]: pgmap v414: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 67 MiB/s wr, 1.72k op/s 2024-03-11T19:41:23.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:22 smithi137 ceph-mon[36220]: pgmap v414: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 67 MiB/s wr, 1.72k op/s 2024-03-11T19:41:23.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:22 smithi204 ceph-mon[36739]: pgmap v414: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 67 MiB/s wr, 1.72k op/s 2024-03-11T19:41:24.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:24 smithi137 ceph-mon[36220]: pgmap v415: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 65 MiB/s wr, 1.80k op/s 2024-03-11T19:41:24.958 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:24 smithi204 ceph-mon[36739]: pgmap v415: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 65 MiB/s wr, 1.80k op/s 2024-03-11T19:41:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:24 smithi083 ceph-mon[31864]: pgmap v415: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 65 MiB/s wr, 1.80k op/s 2024-03-11T19:41:26.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:25 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:41:26.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:25 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:26.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:25 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:26.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:25 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:41:26.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:25 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:26.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:25 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:26.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:25 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:41:26.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:25 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:26.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:25 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:26.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:25 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:41:26.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:25 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:26.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:25 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:26.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:25 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:41:26.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:25 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:26.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:25 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:26.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:25 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:41:26.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:25 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:26.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:25 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:26 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:41:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:26 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:41:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:26 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:41:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:26 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:41:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:26 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:27.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:26 smithi083 ceph-mon[31864]: pgmap v416: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 58 MiB/s wr, 1.74k op/s 2024-03-11T19:41:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:26 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:41:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:26 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:41:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:26 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:41:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:26 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:41:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:26 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:26 smithi137 ceph-mon[36220]: pgmap v416: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 58 MiB/s wr, 1.74k op/s 2024-03-11T19:41:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:26 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:41:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:26 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:41:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:26 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:41:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:26 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:41:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:26 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:41:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:26 smithi204 ceph-mon[36739]: pgmap v416: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 58 MiB/s wr, 1.74k op/s 2024-03-11T19:41:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 130.0 s, 4423.4 tps, lat 22.725 ms stddev 26.118 2024-03-11T19:41:29.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:28 smithi083 ceph-mon[31864]: pgmap v417: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 44 MiB/s wr, 1.62k op/s 2024-03-11T19:41:29.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:28 smithi137 ceph-mon[36220]: pgmap v417: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 44 MiB/s wr, 1.62k op/s 2024-03-11T19:41:29.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:28 smithi204 ceph-mon[36739]: pgmap v417: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 44 MiB/s wr, 1.62k op/s 2024-03-11T19:41:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:30 smithi083 ceph-mon[31864]: pgmap v418: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 21 MiB/s wr, 1.08k op/s 2024-03-11T19:41:31.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:30 smithi137 ceph-mon[36220]: pgmap v418: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 21 MiB/s wr, 1.08k op/s 2024-03-11T19:41:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:30 smithi204 ceph-mon[36739]: pgmap v418: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 21 MiB/s wr, 1.08k op/s 2024-03-11T19:41:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 135.0 s, 4179.8 tps, lat 23.936 ms stddev 25.823 2024-03-11T19:41:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:32 smithi083 ceph-mon[31864]: pgmap v419: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 937 op/s 2024-03-11T19:41:33.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:32 smithi204 ceph-mon[36739]: pgmap v419: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 937 op/s 2024-03-11T19:41:33.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:32 smithi137 ceph-mon[36220]: pgmap v419: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 937 op/s 2024-03-11T19:41:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:34 smithi083 ceph-mon[31864]: pgmap v420: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 873 op/s 2024-03-11T19:41:35.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:34 smithi137 ceph-mon[36220]: pgmap v420: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 873 op/s 2024-03-11T19:41:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:34 smithi204 ceph-mon[36739]: pgmap v420: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 873 op/s 2024-03-11T19:41:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:36 smithi083 ceph-mon[31864]: pgmap v421: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 843 op/s 2024-03-11T19:41:37.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:36 smithi137 ceph-mon[36220]: pgmap v421: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 843 op/s 2024-03-11T19:41:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:36 smithi204 ceph-mon[36739]: pgmap v421: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 843 op/s 2024-03-11T19:41:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 140.0 s, 4922.2 tps, lat 20.252 ms stddev 22.337 2024-03-11T19:41:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:38 smithi083 ceph-mon[31864]: pgmap v422: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 885 op/s 2024-03-11T19:41:39.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:38 smithi137 ceph-mon[36220]: pgmap v422: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 885 op/s 2024-03-11T19:41:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:38 smithi204 ceph-mon[36739]: pgmap v422: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 885 op/s 2024-03-11T19:41:41.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:40 smithi083 ceph-mon[31864]: pgmap v423: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 809 op/s 2024-03-11T19:41:41.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:40 smithi137 ceph-mon[36220]: pgmap v423: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 809 op/s 2024-03-11T19:41:41.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:40 smithi204 ceph-mon[36739]: pgmap v423: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 809 op/s 2024-03-11T19:41:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 145.0 s, 5052.8 tps, lat 19.891 ms stddev 21.480 2024-03-11T19:41:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:42 smithi083 ceph-mon[31864]: pgmap v424: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 808 op/s 2024-03-11T19:41:43.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:42 smithi137 ceph-mon[36220]: pgmap v424: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 808 op/s 2024-03-11T19:41:43.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:42 smithi204 ceph-mon[36739]: pgmap v424: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 808 op/s 2024-03-11T19:41:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:44 smithi083 ceph-mon[31864]: pgmap v425: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-03-11T19:41:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:44 smithi137 ceph-mon[36220]: pgmap v425: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-03-11T19:41:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:44 smithi204 ceph-mon[36739]: pgmap v425: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-03-11T19:41:46.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:41:46.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:41:46.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.888+0000 7f23491c7640 1 -- 172.21.15.83:0/327987530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2344101c60 msgr2=0x7f2344104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.888+0000 7f23491c7640 1 --2- 172.21.15.83:0/327987530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2344101c60 0x7f2344104050 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2334013150 tx=0x7f2334033fa0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.888+0000 7f23491c7640 1 -- 172.21.15.83:0/327987530 shutdown_connections 2024-03-11T19:41:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.888+0000 7f23491c7640 1 --2- 172.21.15.83:0/327987530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2344106ec0 0x7f23441092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.888+0000 7f23491c7640 1 --2- 172.21.15.83:0/327987530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2344104590 0x7f2344106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.888+0000 7f23491c7640 1 --2- 172.21.15.83:0/327987530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2344101c60 0x7f2344104050 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.888+0000 7f23491c7640 1 -- 172.21.15.83:0/327987530 >> 172.21.15.83:0/327987530 conn(0x7f23440fb5f0 msgr2=0x7f23440fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.888+0000 7f23491c7640 1 -- 172.21.15.83:0/327987530 shutdown_connections 2024-03-11T19:41:46.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.888+0000 7f23491c7640 1 -- 172.21.15.83:0/327987530 wait complete. 2024-03-11T19:41:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f23491c7640 1 Processor -- start 2024-03-11T19:41:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f23491c7640 1 -- start start 2024-03-11T19:41:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f23491c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2344101c60 0x7f23441950a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f23491c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2344104590 0x7f23441955e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f23491c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2344106ec0 0x7f234419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f23491c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234419cb90 con 0x7f2344104590 2024-03-11T19:41:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f23491c7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234419cd00 con 0x7f2344101c60 2024-03-11T19:41:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f23491c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234419cec0 con 0x7f2344106ec0 2024-03-11T19:41:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f234259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2344104590 0x7f23441955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f2342d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2344101c60 0x7f23441950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f234259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2344104590 0x7f23441955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49060/0 (socket says 172.21.15.83:49060) 2024-03-11T19:41:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f2342d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2344101c60 0x7f23441950a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32912/0 (socket says 172.21.15.83:32912) 2024-03-11T19:41:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.889+0000 7f234259c640 1 -- 172.21.15.83:0/2034790781 learned_addr learned my addr 172.21.15.83:0/2034790781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.890+0000 7f234359e640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2344106ec0 0x7f234419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.890+0000 7f2342d9d640 1 -- 172.21.15.83:0/2034790781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2344106ec0 msgr2=0x7f234419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.890+0000 7f2342d9d640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2344106ec0 0x7f234419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.890+0000 7f2342d9d640 1 -- 172.21.15.83:0/2034790781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2344104590 msgr2=0x7f23441955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.890+0000 7f2342d9d640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2344104590 0x7f23441955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.890+0000 7f2342d9d640 1 -- 172.21.15.83:0/2034790781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2334011d70 con 0x7f2344101c60 2024-03-11T19:41:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.890+0000 7f2342d9d640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2344101c60 0x7f23441950a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f23340057c0 tx=0x7f23340042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.891+0000 7f232bfff640 1 -- 172.21.15.83:0/2034790781 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334034cb0 con 0x7f2344101c60 2024-03-11T19:41:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.891+0000 7f232bfff640 1 -- 172.21.15.83:0/2034790781 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23340044e0 con 0x7f2344101c60 2024-03-11T19:41:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.891+0000 7f232bfff640 1 -- 172.21.15.83:0/2034790781 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334003e10 con 0x7f2344101c60 2024-03-11T19:41:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.891+0000 7f23491c7640 1 -- 172.21.15.83:0/2034790781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234419d140 con 0x7f2344101c60 2024-03-11T19:41:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.891+0000 7f23491c7640 1 -- 172.21.15.83:0/2034790781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234419d530 con 0x7f2344101c60 2024-03-11T19:41:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.894+0000 7f232bfff640 1 -- 172.21.15.83:0/2034790781 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f233404d410 con 0x7f2344101c60 2024-03-11T19:41:46.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.894+0000 7f232bfff640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f232c0a1c00 0x7f232c0a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.894+0000 7f232bfff640 1 -- 172.21.15.83:0/2034790781 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f23340f7010 con 0x7f2344101c60 2024-03-11T19:41:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.895+0000 7f234259c640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f232c0a1c00 0x7f232c0a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.895+0000 7f234259c640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f232c0a1c00 0x7f232c0a40c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f23441965b0 tx=0x7f233800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.895+0000 7f234bd99740 1 Processor -- start 2024-03-11T19:41:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.895+0000 7f234bd99740 1 -- start start 2024-03-11T19:41:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f234bd99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dc2d0 0x55b3051dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f234bd99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dccc0 0x55b3051e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f234bd99740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051e1e30 0x55b3051e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f234bd99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30514c280 con 0x55b3051dccc0 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f234bd99740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3051497b0 con 0x55b3051e1e30 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f234359e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051e1e30 0x55b3051e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f2342d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dc2d0 0x55b3051dc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f234359e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051e1e30 0x55b3051e42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32926/0 (socket says 172.21.15.83:32926) 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f234359e640 1 -- 172.21.15.83:0/2543126056 learned_addr learned my addr 172.21.15.83:0/2543126056 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f234bd99740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b305144f90 con 0x55b3051dc2d0 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f234259c640 1 --2- 172.21.15.83:0/2543126056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dccc0 0x55b3051e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f2342d9d640 1 -- 172.21.15.83:0/2543126056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051e1e30 msgr2=0x55b3051e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f2342d9d640 1 --2- 172.21.15.83:0/2543126056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051e1e30 0x55b3051e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f2342d9d640 1 -- 172.21.15.83:0/2543126056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dccc0 msgr2=0x55b3051e17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:46.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.896+0000 7f2342d9d640 1 --2- 172.21.15.83:0/2543126056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dccc0 0x55b3051e17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.897+0000 7f2342d9d640 1 -- 172.21.15.83:0/2543126056 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b305103570 con 0x55b3051dc2d0 2024-03-11T19:41:46.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.897+0000 7f2342d9d640 1 --2- 172.21.15.83:0/2543126056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dc2d0 0x55b3051dc6f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f23340f4b20 tx=0x7f2334011a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f0c3236b73b1796 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:46.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.897+0000 7f23297fa640 1 -- 172.21.15.83:0/2543126056 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23340c3f90 con 0x55b3051dc2d0 2024-03-11T19:41:46.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.897+0000 7f23297fa640 1 -- 172.21.15.83:0/2543126056 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23340c80e0 con 0x55b3051dc2d0 2024-03-11T19:41:46.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.897+0000 7f23297fa640 1 -- 172.21.15.83:0/2543126056 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23340cc500 con 0x55b3051dc2d0 2024-03-11T19:41:46.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.898+0000 7f234bd99740 1 -- 172.21.15.83:0/2543126056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dc2d0 msgr2=0x55b3051dc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.898+0000 7f234bd99740 1 --2- 172.21.15.83:0/2543126056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dc2d0 0x55b3051dc6f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f23340f4b20 tx=0x7f2334011a70 comp rx=0 tx=0).stop 2024-03-11T19:41:46.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.899+0000 7f234bd99740 1 -- 172.21.15.83:0/2543126056 shutdown_connections 2024-03-11T19:41:46.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.899+0000 7f234bd99740 1 --2- 172.21.15.83:0/2543126056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051e1e30 0x55b3051e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.899+0000 7f234bd99740 1 --2- 172.21.15.83:0/2543126056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dccc0 0x55b3051e17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.899+0000 7f234bd99740 1 --2- 172.21.15.83:0/2543126056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dc2d0 0x55b3051dc6f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.899+0000 7f234bd99740 1 -- 172.21.15.83:0/2543126056 >> 172.21.15.83:0/2543126056 conn(0x55b3051d77f0 msgr2=0x55b3051d7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:46.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.899+0000 7f234bd99740 1 -- 172.21.15.83:0/2543126056 shutdown_connections 2024-03-11T19:41:46.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.899+0000 7f234bd99740 1 -- 172.21.15.83:0/2543126056 wait complete. 2024-03-11T19:41:46.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.899+0000 7f234bd99740 1 -- start start 2024-03-11T19:41:46.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.899+0000 7f234bd99740 1 Processor -- start 2024-03-11T19:41:46.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f234bd99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f234bd99740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 0x55b3051e6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f234bd99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051e1e30 0x55b3051ed4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f234259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 0x55b3051e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f2342d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f2342d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051e5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49080/0 (socket says 172.21.15.83:49080) 2024-03-11T19:41:46.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f234259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 0x55b3051e6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32940/0 (socket says 172.21.15.83:32940) 2024-03-11T19:41:46.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f2342d9d640 1 -- 172.21.15.83:0/795451347 learned_addr learned my addr 172.21.15.83:0/795451347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:46.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f234359e640 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051e1e30 0x55b3051ed4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f234bd99740 1 -- 172.21.15.83:0/795451347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b304e18430 con 0x55b3051dc2d0 2024-03-11T19:41:46.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f234bd99740 1 -- 172.21.15.83:0/795451347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b304dd4e00 con 0x55b3051dccc0 2024-03-11T19:41:46.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.900+0000 7f234bd99740 1 -- 172.21.15.83:0/795451347 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3050c9fd0 con 0x55b3051e1e30 2024-03-11T19:41:46.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.901+0000 7f234259c640 1 -- 172.21.15.83:0/795451347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051e1e30 msgr2=0x55b3051ed4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:46.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.901+0000 7f234259c640 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051e1e30 0x55b3051ed4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.901+0000 7f234259c640 1 -- 172.21.15.83:0/795451347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 msgr2=0x55b3051e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.901+0000 7f234259c640 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.901+0000 7f234259c640 1 -- 172.21.15.83:0/795451347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234419d530 con 0x55b3051dccc0 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.901+0000 7f234259c640 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 0x55b3051e6480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55b3051dd730 tx=0x7f2338004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc9a80e514a1817e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.901+0000 7f234bd99740 10 client.0 _ll_register_callbacks cb 0x55b30511ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.901+0000 7f234bd99740 1 -- 172.21.15.83:0/795451347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b304e447e0 con 0x55b3051dccc0 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.902+0000 7f2326ff5640 10 client.25274 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.902+0000 7f2326ff5640 1 -- 172.21.15.83:0/795451347 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233802f710 con 0x55b3051dccc0 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.902+0000 7f2326ff5640 1 -- 172.21.15.83:0/795451347 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2338017070 con 0x55b3051dccc0 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.902+0000 7f2326ff5640 1 -- 172.21.15.83:0/795451347 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f233802f870 con 0x55b3051dccc0 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.902+0000 7f234bd99740 10 client.25274 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.902+0000 7f234bd99740 1 -- 172.21.15.83:0/795451347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55b305143610 con 0x55b3051dccc0 2024-03-11T19:41:46.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.902+0000 7f2326ff5640 1 -- 172.21.15.83:0/795451347 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f233802a040 con 0x55b3051dccc0 2024-03-11T19:41:46.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.903+0000 7f234bd99740 10 client.25274 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:46.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.903+0000 7f234bd99740 10 client.25274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:46.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.903+0000 7f234bd99740 20 client.25274 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:46.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.903+0000 7f234bd99740 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b3051f1f80 0x55b3051f4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.903+0000 7f234bd99740 4 client.25274 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:46.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.903+0000 7f234bd99740 1 -- 172.21.15.83:0/795451347 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b304e447e0 con 0x55b3051f1f80 2024-03-11T19:41:46.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.903+0000 7f2342d9d640 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b3051f1f80 0x55b3051f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.903+0000 7f2342d9d640 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b3051f1f80 0x55b3051f4440 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb17293eeecff64f server_cookie=a6880274bc950b2f in_seq=0 out_seq=0 2024-03-11T19:41:46.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.904+0000 7f2326ff5640 10 client.25274 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.904+0000 7f2326ff5640 1 -- 172.21.15.83:0/795451347 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b304e447e0 con 0x55b3051f1f80 2024-03-11T19:41:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.904+0000 7f2326ff5640 10 client.25274 handle_command_reply: tid=0 2024-03-11T19:41:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.904+0000 7f234bd99740 1 client.25274 shutdown 2024-03-11T19:41:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.904+0000 7f234bd99740 1 -- 172.21.15.83:0/795451347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 msgr2=0x55b3051e6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.904+0000 7f234bd99740 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 0x55b3051e6480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55b3051dd730 tx=0x7f2338004730 comp rx=0 tx=0).stop 2024-03-11T19:41:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.904+0000 7f234bd99740 1 -- 172.21.15.83:0/795451347 shutdown_connections 2024-03-11T19:41:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.904+0000 7f234bd99740 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b3051f1f80 0x55b3051f4440 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.904+0000 7f234bd99740 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051e1e30 0x55b3051ed4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.905+0000 7f234bd99740 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 0x55b3051e6480 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.905+0000 7f234bd99740 1 --2- 172.21.15.83:0/795451347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.905+0000 7f234bd99740 1 -- 172.21.15.83:0/795451347 >> 172.21.15.83:0/795451347 conn(0x55b3051d77f0 msgr2=0x55b3051d6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.905+0000 7f2326ff5640 0 client.25274 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.906+0000 7f234bd99740 1 -- 172.21.15.83:0/795451347 shutdown_connections 2024-03-11T19:41:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.906+0000 7f234bd99740 1 -- 172.21.15.83:0/795451347 wait complete. 2024-03-11T19:41:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.906+0000 7f234bd99740 20 client.25274 trim_cache size 0 max 16384 2024-03-11T19:41:46.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f234bd99740 1 Processor -- start 2024-03-11T19:41:46.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f234bd99740 1 -- start start 2024-03-11T19:41:46.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f234bd99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051e5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f234bd99740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 0x55b3051ed480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f234bd99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051e1e30 0x55b3051e5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f234bd99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3051f04b0 con 0x55b3051dc2d0 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f234bd99740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3051dd0a0 con 0x55b3051dccc0 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f234bd99740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3051ee5d0 con 0x55b3051e1e30 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f2342d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051e5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f234259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 0x55b3051ed480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f2342d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051e5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49086/0 (socket says 172.21.15.83:49086) 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f2342d9d640 1 -- 172.21.15.83:0/3783723751 learned_addr learned my addr 172.21.15.83:0/3783723751 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.931+0000 7f234359e640 1 --2- 172.21.15.83:0/3783723751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051e1e30 0x55b3051e5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.932+0000 7f234259c640 1 -- 172.21.15.83:0/3783723751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051e1e30 msgr2=0x55b3051e5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.932+0000 7f234259c640 1 --2- 172.21.15.83:0/3783723751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051e1e30 0x55b3051e5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.932+0000 7f234259c640 1 -- 172.21.15.83:0/3783723751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 msgr2=0x55b3051e5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.932+0000 7f234259c640 1 --2- 172.21.15.83:0/3783723751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051e5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.932+0000 7f234259c640 1 -- 172.21.15.83:0/3783723751 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3050dfac0 con 0x55b3051dccc0 2024-03-11T19:41:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.932+0000 7f2342d9d640 1 --2- 172.21.15.83:0/3783723751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:41:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.932+0000 7f234259c640 1 --2- 172.21.15.83:0/3783723751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 0x55b3051ed480 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2338010c30 tx=0x7f2338032070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=634073382b98a43d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.932+0000 7f2328ff9640 1 -- 172.21.15.83:0/3783723751 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233802fd70 con 0x55b3051dccc0 2024-03-11T19:41:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.932+0000 7f2328ff9640 1 -- 172.21.15.83:0/3783723751 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2338034070 con 0x55b3051dccc0 2024-03-11T19:41:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.933+0000 7f2328ff9640 1 -- 172.21.15.83:0/3783723751 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338004990 con 0x55b3051dccc0 2024-03-11T19:41:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.933+0000 7f234bd99740 1 -- 172.21.15.83:0/3783723751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 msgr2=0x55b3051ed480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.933+0000 7f234bd99740 1 --2- 172.21.15.83:0/3783723751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 0x55b3051ed480 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2338010c30 tx=0x7f2338032070 comp rx=0 tx=0).stop 2024-03-11T19:41:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.933+0000 7f234bd99740 1 -- 172.21.15.83:0/3783723751 shutdown_connections 2024-03-11T19:41:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.933+0000 7f234bd99740 1 --2- 172.21.15.83:0/3783723751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051e1e30 0x55b3051e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.933+0000 7f234bd99740 1 --2- 172.21.15.83:0/3783723751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051dccc0 0x55b3051ed480 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.933+0000 7f234bd99740 1 --2- 172.21.15.83:0/3783723751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.933+0000 7f234bd99740 1 -- 172.21.15.83:0/3783723751 >> 172.21.15.83:0/3783723751 conn(0x55b3051d77f0 msgr2=0x55b3051dde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:46.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.933+0000 7f234bd99740 1 -- 172.21.15.83:0/3783723751 shutdown_connections 2024-03-11T19:41:46.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.933+0000 7f234bd99740 1 -- 172.21.15.83:0/3783723751 wait complete. 2024-03-11T19:41:46.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.934+0000 7f234bd99740 1 -- start start 2024-03-11T19:41:46.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.934+0000 7f234bd99740 1 Processor -- start 2024-03-11T19:41:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.935+0000 7f234bd99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.935+0000 7f234bd99740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dccc0 0x55b3051ed390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234bd99740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051e1e30 0x55b3051eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234bd99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3051ebbb0 con 0x55b3051dc2d0 2024-03-11T19:41:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234bd99740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3051ebd20 con 0x55b3051e1e30 2024-03-11T19:41:46.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234bd99740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3051ebee0 con 0x55b3051dccc0 2024-03-11T19:41:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dccc0 0x55b3051ed390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dccc0 0x55b3051ed390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48220/0 (socket says 172.21.15.83:48220) 2024-03-11T19:41:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234259c640 1 -- 172.21.15.83:0/1368537318 learned_addr learned my addr 172.21.15.83:0/1368537318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234bd99740 10 client.0 _ll_register_callbacks cb 0x7f233800bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f2342d9d640 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234359e640 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051e1e30 0x55b3051eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234259c640 1 -- 172.21.15.83:0/1368537318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051e1e30 msgr2=0x55b3051eb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234259c640 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051e1e30 0x55b3051eb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234259c640 1 -- 172.21.15.83:0/1368537318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 msgr2=0x55b3051ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234259c640 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051ece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f234259c640 1 -- 172.21.15.83:0/1368537318 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b305143610 con 0x55b3051dccc0 2024-03-11T19:41:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.936+0000 7f2342d9d640 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.937+0000 7f234259c640 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dccc0 0x55b3051ed390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2338010d50 tx=0x7f233802f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b94409255e5ffca server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.937+0000 7f2326ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:41:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.937+0000 7f2326ff5640 1 -- 172.21.15.83:0/1368537318 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338053070 con 0x55b3051dccc0 2024-03-11T19:41:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.937+0000 7f234bd99740 1 -- 172.21.15.83:0/1368537318 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23340c80e0 con 0x55b3051dccc0 2024-03-11T19:41:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.937+0000 7f2326ff5640 1 -- 172.21.15.83:0/1368537318 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2338017040 con 0x55b3051dccc0 2024-03-11T19:41:46.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f2326ff5640 1 -- 172.21.15.83:0/1368537318 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f233802fa20 con 0x55b3051dccc0 2024-03-11T19:41:46.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f234bd99740 10 client.25396 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f234bd99740 1 -- 172.21.15.83:0/1368537318 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b3051056a0 con 0x55b3051dccc0 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f2326ff5640 1 -- 172.21.15.83:0/1368537318 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f233805e020 con 0x55b3051dccc0 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f2326ff5640 1 -- 172.21.15.83:0/1368537318 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f233802a070 con 0x55b3051dccc0 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f234bd99740 10 client.25396 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f234bd99740 10 client.25396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f234bd99740 20 client.25396 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f234bd99740 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b3052497b0 0x55b30524bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f234bd99740 4 client.25396 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f234bd99740 1 -- 172.21.15.83:0/1368537318 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b304e81720 con 0x55b3052497b0 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.938+0000 7f2342d9d640 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b3052497b0 0x55b30524bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.939+0000 7f2326ff5640 10 client.25396 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:46.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.939+0000 7f2342d9d640 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b3052497b0 0x55b30524bbd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97c46037dc02df68 server_cookie=3ceb8752a25558c8 in_seq=0 out_seq=0 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.940+0000 7f2326ff5640 1 -- 172.21.15.83:0/1368537318 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b304e81720 con 0x55b3052497b0 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.940+0000 7f2326ff5640 10 client.25396 handle_command_reply: tid=0 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.940+0000 7f234bd99740 1 client.25396 shutdown 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.940+0000 7f234bd99740 1 -- 172.21.15.83:0/1368537318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dccc0 msgr2=0x55b3051ed390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.940+0000 7f234bd99740 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dccc0 0x55b3051ed390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2338010d50 tx=0x7f233802f510 comp rx=0 tx=0).stop 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.941+0000 7f234bd99740 1 -- 172.21.15.83:0/1368537318 shutdown_connections 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.941+0000 7f234bd99740 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b3052497b0 0x55b30524bbd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.941+0000 7f234bd99740 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b3051e1e30 0x55b3051eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.941+0000 7f234bd99740 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b3051dccc0 0x55b3051ed390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.941+0000 7f234bd99740 1 --2- 172.21.15.83:0/1368537318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b3051dc2d0 0x55b3051ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.941+0000 7f234bd99740 1 -- 172.21.15.83:0/1368537318 >> 172.21.15.83:0/1368537318 conn(0x55b3051d77f0 msgr2=0x55b3051daaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.941+0000 7f2326ff5640 0 client.25396 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.941+0000 7f234bd99740 1 -- 172.21.15.83:0/1368537318 shutdown_connections 2024-03-11T19:41:46.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.941+0000 7f234bd99740 1 -- 172.21.15.83:0/1368537318 wait complete. 2024-03-11T19:41:46.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.941+0000 7f234bd99740 20 client.25396 trim_cache size 0 max 16384 2024-03-11T19:41:46.948 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:41:46.948 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:41:46.948 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:41:46.948 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:41:46.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 -- 172.21.15.83:0/2034790781 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f232c0a1c00 msgr2=0x7f232c0a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f232c0a1c00 0x7f232c0a40c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f23441965b0 tx=0x7f233800b410 comp rx=0 tx=0).stop 2024-03-11T19:41:46.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 -- 172.21.15.83:0/2034790781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2344101c60 msgr2=0x7f23441950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:46.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2344101c60 0x7f23441950a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f23340057c0 tx=0x7f23340042f0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 -- 172.21.15.83:0/2034790781 shutdown_connections 2024-03-11T19:41:46.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f232c0a1c00 0x7f232c0a40c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2344106ec0 0x7f234419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2344104590 0x7f23441955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 --2- 172.21.15.83:0/2034790781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2344101c60 0x7f23441950a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:46.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 -- 172.21.15.83:0/2034790781 >> 172.21.15.83:0/2034790781 conn(0x7f23440fb5f0 msgr2=0x7f23441025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:46.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 -- 172.21.15.83:0/2034790781 shutdown_connections 2024-03-11T19:41:46.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:46.942+0000 7f23491c7640 1 -- 172.21.15.83:0/2034790781 wait complete. 2024-03-11T19:41:46.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:46 smithi083 ceph-mon[31864]: pgmap v426: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 MiB/s wr, 990 op/s 2024-03-11T19:41:46.963 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:41:46.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:41:47.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.115+0000 7fb342e4d640 1 -- 172.21.15.83:0/464503634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb33c109bd0 msgr2=0x7fb33c110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.115+0000 7fb342e4d640 1 --2- 172.21.15.83:0/464503634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb33c109bd0 0x7fb33c110480 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb334009ca0 tx=0x7fb33402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.115+0000 7fb342e4d640 1 -- 172.21.15.83:0/464503634 shutdown_connections 2024-03-11T19:41:47.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.115+0000 7fb342e4d640 1 --2- 172.21.15.83:0/464503634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb33c109bd0 0x7fb33c110480 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.115+0000 7fb342e4d640 1 --2- 172.21.15.83:0/464503634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb33c109230 0x7fb33c109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.115+0000 7fb342e4d640 1 --2- 172.21.15.83:0/464503634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb33c108520 0x7fb33c108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.115+0000 7fb342e4d640 1 -- 172.21.15.83:0/464503634 >> 172.21.15.83:0/464503634 conn(0x7fb33c06dff0 msgr2=0x7fb33c070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.115+0000 7fb342e4d640 1 -- 172.21.15.83:0/464503634 shutdown_connections 2024-03-11T19:41:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.115+0000 7fb342e4d640 1 -- 172.21.15.83:0/464503634 wait complete. 2024-03-11T19:41:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.115+0000 7fb342e4d640 1 Processor -- start 2024-03-11T19:41:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb342e4d640 1 -- start start 2024-03-11T19:41:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb342e4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb33c108520 0x7fb33c1a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb342e4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb33c109230 0x7fb33c1a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb342e4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb33c109bd0 0x7fb33c1ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb342e4d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1adad0 con 0x7fb33c109230 2024-03-11T19:41:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb342e4d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1adc40 con 0x7fb33c109bd0 2024-03-11T19:41:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb342e4d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1ade00 con 0x7fb33c108520 2024-03-11T19:41:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb3413ea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb33c109bd0 0x7fb33c1ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb3413ea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb33c109bd0 0x7fb33c1ad590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32968/0 (socket says 172.21.15.83:32968) 2024-03-11T19:41:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb3413ea640 1 -- 172.21.15.83:0/3260144639 learned_addr learned my addr 172.21.15.83:0/3260144639 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb33bfff640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb33c109230 0x7fb33c1a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.116+0000 7fb340be9640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb33c108520 0x7fb33c1a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.117+0000 7fb33bfff640 1 -- 172.21.15.83:0/3260144639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb33c108520 msgr2=0x7fb33c1a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.117+0000 7fb33bfff640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb33c108520 0x7fb33c1a5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.117+0000 7fb33bfff640 1 -- 172.21.15.83:0/3260144639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb33c109bd0 msgr2=0x7fb33c1ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.117+0000 7fb33bfff640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb33c109bd0 0x7fb33c1ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.117+0000 7fb33bfff640 1 -- 172.21.15.83:0/3260144639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb334009900 con 0x7fb33c109230 2024-03-11T19:41:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.117+0000 7fb33bfff640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb33c109230 0x7fb33c1a6520 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb32800d6d0 tx=0x7fb32800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.118+0000 7fb339ffb640 1 -- 172.21.15.83:0/3260144639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328005220 con 0x7fb33c109230 2024-03-11T19:41:47.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.118+0000 7fb339ffb640 1 -- 172.21.15.83:0/3260144639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb328009d00 con 0x7fb33c109230 2024-03-11T19:41:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.118+0000 7fb342e4d640 1 -- 172.21.15.83:0/3260144639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c1ae0e0 con 0x7fb33c109230 2024-03-11T19:41:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.119+0000 7fb342e4d640 1 -- 172.21.15.83:0/3260144639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c1ae530 con 0x7fb33c109230 2024-03-11T19:41:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.119+0000 7fb339ffb640 1 -- 172.21.15.83:0/3260144639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328004dd0 con 0x7fb33c109230 2024-03-11T19:41:47.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.120+0000 7fb342e4d640 1 -- 172.21.15.83:0/3260144639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c117570 con 0x7fb33c109230 2024-03-11T19:41:47.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.122+0000 7fb339ffb640 1 -- 172.21.15.83:0/3260144639 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fb328009820 con 0x7fb33c109230 2024-03-11T19:41:47.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.123+0000 7fb339ffb640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb32c0a1be0 0x7fb32c0a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.123+0000 7fb339ffb640 1 -- 172.21.15.83:0/3260144639 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fb3280ccda0 con 0x7fb33c109230 2024-03-11T19:41:47.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.124+0000 7fb340be9640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb32c0a1be0 0x7fb32c0a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.125+0000 7fb340be9640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb32c0a1be0 0x7fb32c0a40a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb330009bd0 tx=0x7fb330005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.130+0000 7fb339ffb640 1 -- 172.21.15.83:0/3260144639 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb328096070 con 0x7fb33c109230 2024-03-11T19:41:47.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:46 smithi137 ceph-mon[36220]: pgmap v426: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 MiB/s wr, 990 op/s 2024-03-11T19:41:47.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:46 smithi204 ceph-mon[36739]: pgmap v426: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 13 MiB/s wr, 990 op/s 2024-03-11T19:41:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 150.0 s, 6150.4 tps, lat 16.196 ms stddev 17.870 2024-03-11T19:41:47.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.520+0000 7fb342e4d640 1 -- 172.21.15.83:0/3260144639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb33c1ae9e0 con 0x7fb33c109230 2024-03-11T19:41:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.522+0000 7fb339ffb640 1 -- 172.21.15.83:0/3260144639 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11353 (secure 0 0 0) 0x7fb328017090 con 0x7fb33c109230 2024-03-11T19:41:47.526 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:41:47.526 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:41:06.999655+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:41:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2024-03-11T19:41:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.525+0000 7fb342e4d640 1 -- 172.21.15.83:0/3260144639 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb32c0a1be0 msgr2=0x7fb32c0a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.525+0000 7fb342e4d640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb32c0a1be0 0x7fb32c0a40a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb330009bd0 tx=0x7fb330005f90 comp rx=0 tx=0).stop 2024-03-11T19:41:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.525+0000 7fb342e4d640 1 -- 172.21.15.83:0/3260144639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb33c109230 msgr2=0x7fb33c1a6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.525+0000 7fb342e4d640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb33c109230 0x7fb33c1a6520 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb32800d6d0 tx=0x7fb32800db90 comp rx=0 tx=0).stop 2024-03-11T19:41:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.526+0000 7fb342e4d640 1 -- 172.21.15.83:0/3260144639 shutdown_connections 2024-03-11T19:41:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.526+0000 7fb342e4d640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb32c0a1be0 0x7fb32c0a40a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.526+0000 7fb342e4d640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb33c109bd0 0x7fb33c1ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.526+0000 7fb342e4d640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb33c109230 0x7fb33c1a6520 secure :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb32800d6d0 tx=0x7fb32800db90 comp rx=0 tx=0).stop 2024-03-11T19:41:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.526+0000 7fb342e4d640 1 --2- 172.21.15.83:0/3260144639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb33c108520 0x7fb33c1a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.526+0000 7fb342e4d640 1 -- 172.21.15.83:0/3260144639 >> 172.21.15.83:0/3260144639 conn(0x7fb33c06dff0 msgr2=0x7fb33c1135f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.527+0000 7fb342e4d640 1 -- 172.21.15.83:0/3260144639 shutdown_connections 2024-03-11T19:41:47.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.527+0000 7fb342e4d640 1 -- 172.21.15.83:0/3260144639 wait complete. 2024-03-11T19:41:47.546 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:41:47.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:41:47.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.709+0000 7f375799a640 1 -- 172.21.15.83:0/176687748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3750102380 msgr2=0x7f3750102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.709+0000 7f375799a640 1 --2- 172.21.15.83:0/176687748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3750102380 0x7f3750102800 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3740009900 tx=0x7f374002bc30 comp rx=0 tx=0).stop 2024-03-11T19:41:47.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.709+0000 7f375799a640 1 -- 172.21.15.83:0/176687748 shutdown_connections 2024-03-11T19:41:47.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.709+0000 7f375799a640 1 --2- 172.21.15.83:0/176687748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3750102d40 0x7f37501095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.709+0000 7f375799a640 1 --2- 172.21.15.83:0/176687748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3750102380 0x7f3750102800 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.709+0000 7f375799a640 1 --2- 172.21.15.83:0/176687748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3750101190 0x7f37501015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.709+0000 7f375799a640 1 -- 172.21.15.83:0/176687748 >> 172.21.15.83:0/176687748 conn(0x7f37500fc6e0 msgr2=0x7f37500feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.709+0000 7f375799a640 1 -- 172.21.15.83:0/176687748 shutdown_connections 2024-03-11T19:41:47.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.709+0000 7f375799a640 1 -- 172.21.15.83:0/176687748 wait complete. 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.710+0000 7f375799a640 1 Processor -- start 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.710+0000 7f375799a640 1 -- start start 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.710+0000 7f375799a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3750101190 0x7f3750199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.710+0000 7f375799a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3750102380 0x7f3750199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.710+0000 7f375799a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3750102d40 0x7f37501a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.710+0000 7f375799a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37501a0f40 con 0x7f3750102d40 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.710+0000 7f3755f37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3750102d40 0x7f37501a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f3755f37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3750102d40 0x7f37501a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49112/0 (socket says 172.21.15.83:49112) 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f3755f37640 1 -- 172.21.15.83:0/1692723529 learned_addr learned my addr 172.21.15.83:0/1692723529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f375799a640 1 -- 172.21.15.83:0/1692723529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37501a10b0 con 0x7f3750101190 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f375799a640 1 -- 172.21.15.83:0/1692723529 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37501a1270 con 0x7f3750102380 2024-03-11T19:41:47.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f3755736640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3750101190 0x7f3750199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f3755f37640 1 -- 172.21.15.83:0/1692723529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3750102380 msgr2=0x7f3750199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f3755f37640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3750102380 0x7f3750199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f3755f37640 1 -- 172.21.15.83:0/1692723529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3750101190 msgr2=0x7f3750199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f3755f37640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3750101190 0x7f3750199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f3755f37640 1 -- 172.21.15.83:0/1692723529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3740009560 con 0x7f3750102d40 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f3755736640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3750101190 0x7f3750199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.711+0000 7f3755f37640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3750102d40 0x7f37501a0a00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f374800eb10 tx=0x7f374800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.712+0000 7f37467fc640 1 -- 172.21.15.83:0/1692723529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748004b20 con 0x7f3750102d40 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.712+0000 7f375799a640 1 -- 172.21.15.83:0/1692723529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37501a1550 con 0x7f3750102d40 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.712+0000 7f37467fc640 1 -- 172.21.15.83:0/1692723529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3748004c80 con 0x7f3750102d40 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.712+0000 7f37467fc640 1 -- 172.21.15.83:0/1692723529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748010670 con 0x7f3750102d40 2024-03-11T19:41:47.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.712+0000 7f375799a640 1 -- 172.21.15.83:0/1692723529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37501a1ab0 con 0x7f3750102d40 2024-03-11T19:41:47.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.715+0000 7f37467fc640 1 -- 172.21.15.83:0/1692723529 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3748005020 con 0x7f3750102d40 2024-03-11T19:41:47.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.715+0000 7f375a573740 1 Processor -- start 2024-03-11T19:41:47.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.715+0000 7f375a573740 1 -- start start 2024-03-11T19:41:47.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.715+0000 7f375a573740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa6f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.715+0000 7f375a573740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 0x55c43aa74760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.715+0000 7f375a573740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa74dd0 0x55c43aa77260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.715+0000 7f375a573740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43aa2b960 con 0x55c43aa6fc60 2024-03-11T19:41:47.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.715+0000 7f375a573740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43a9df310 con 0x55c43aa74dd0 2024-03-11T19:41:47.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.715+0000 7f375a573740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43a9dc840 con 0x55c43aa6f270 2024-03-11T19:41:47.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f3754f35640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa6f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f3754f35640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa6f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48248/0 (socket says 172.21.15.83:48248) 2024-03-11T19:41:47.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f3754f35640 1 -- 172.21.15.83:0/886535670 learned_addr learned my addr 172.21.15.83:0/886535670 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:47.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f3754f35640 1 --2- 172.21.15.83:0/886535670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa74dd0 0x55c43aa77260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f37467fc640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f373c0a1a80 0x7f373c0a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f3755736640 1 --2- 172.21.15.83:0/886535670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 0x55c43aa74760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f37467fc640 1 -- 172.21.15.83:0/1692723529 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3748014070 con 0x7f3750102d40 2024-03-11T19:41:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f3754f35640 1 -- 172.21.15.83:0/886535670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa74dd0 msgr2=0x55c43aa77260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f3754f35640 1 --2- 172.21.15.83:0/886535670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa74dd0 0x55c43aa77260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f3754f35640 1 -- 172.21.15.83:0/886535670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 msgr2=0x55c43aa74760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f3754f35640 1 --2- 172.21.15.83:0/886535670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 0x55c43aa74760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f3754f35640 1 -- 172.21.15.83:0/886535670 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c43a99a060 con 0x55c43aa6f270 2024-03-11T19:41:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.716+0000 7f3755736640 1 --2- 172.21.15.83:0/886535670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 0x55c43aa74760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:41:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.717+0000 7f3755f37640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f373c0a1a80 0x7f373c0a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.717+0000 7f3754f35640 1 --2- 172.21.15.83:0/886535670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa6f690 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3740036040 tx=0x7f374002c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeb6c18385de0182 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.717+0000 7f373b7fe640 1 -- 172.21.15.83:0/886535670 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37400048f0 con 0x55c43aa6f270 2024-03-11T19:41:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.717+0000 7f373b7fe640 1 -- 172.21.15.83:0/886535670 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3740004a50 con 0x55c43aa6f270 2024-03-11T19:41:47.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.717+0000 7f373b7fe640 1 -- 172.21.15.83:0/886535670 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740004df0 con 0x55c43aa6f270 2024-03-11T19:41:47.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.718+0000 7f3755f37640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f373c0a1a80 0x7f373c0a3f40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f37480049d0 tx=0x7f3748000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.718+0000 7f375a573740 1 -- 172.21.15.83:0/886535670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 msgr2=0x55c43aa6f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.718+0000 7f375a573740 1 --2- 172.21.15.83:0/886535670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa6f690 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3740036040 tx=0x7f374002c7e0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.718+0000 7f375a573740 1 -- 172.21.15.83:0/886535670 shutdown_connections 2024-03-11T19:41:47.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.718+0000 7f375a573740 1 --2- 172.21.15.83:0/886535670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa74dd0 0x55c43aa77260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.718+0000 7f375a573740 1 --2- 172.21.15.83:0/886535670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 0x55c43aa74760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.718+0000 7f375a573740 1 --2- 172.21.15.83:0/886535670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa6f690 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.718+0000 7f375a573740 1 -- 172.21.15.83:0/886535670 >> 172.21.15.83:0/886535670 conn(0x55c43aa6a7e0 msgr2=0x55c43aa6abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.718+0000 7f375a573740 1 -- 172.21.15.83:0/886535670 shutdown_connections 2024-03-11T19:41:47.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.719+0000 7f375a573740 1 -- 172.21.15.83:0/886535670 wait complete. 2024-03-11T19:41:47.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.719+0000 7f375a573740 1 -- start start 2024-03-11T19:41:47.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.720+0000 7f375a573740 1 Processor -- start 2024-03-11T19:41:47.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.720+0000 7f375a573740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa794b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.720+0000 7f375a573740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6fc60 0x55c43aa799f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.720+0000 7f375a573740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa74dd0 0x55c43aa80a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.720+0000 7f375a573740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43a9d8020 con 0x55c43aa74dd0 2024-03-11T19:41:47.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.720+0000 7f375a573740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43a6ab430 con 0x55c43aa6fc60 2024-03-11T19:41:47.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.720+0000 7f375a573740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43a667e00 con 0x55c43aa6f270 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.720+0000 7f375a573740 10 client.0 _ll_register_callbacks cb 0x55c43a97fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.720+0000 7f3754f35640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa794b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.720+0000 7f3754f35640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa794b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48254/0 (socket says 172.21.15.83:48254) 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.720+0000 7f3754f35640 1 -- 172.21.15.83:0/3747080792 learned_addr learned my addr 172.21.15.83:0/3747080792 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3754f35640 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa74dd0 0x55c43aa80a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3755736640 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6fc60 0x55c43aa799f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3754f35640 1 -- 172.21.15.83:0/3747080792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6fc60 msgr2=0x55c43aa799f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3754f35640 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6fc60 0x55c43aa799f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3754f35640 1 -- 172.21.15.83:0/3747080792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa74dd0 msgr2=0x55c43aa80a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3754f35640 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa74dd0 0x55c43aa80a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3754f35640 1 -- 172.21.15.83:0/3747080792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3750109ed0 con 0x55c43aa6f270 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3754f35640 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa794b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3740004a20 tx=0x7f374002c150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64901ef8d183ccba server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3738ff9640 1 -- 172.21.15.83:0/3747080792 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740004450 con 0x55c43aa6f270 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f375a573740 1 -- 172.21.15.83:0/3747080792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c43a6d77e0 con 0x55c43aa6f270 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3738ff9640 1 -- 172.21.15.83:0/3747080792 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3740003820 con 0x55c43aa6f270 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f3738ff9640 1 -- 172.21.15.83:0/3747080792 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3740004f90 con 0x55c43aa6f270 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f375a573740 10 client.25411 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.721+0000 7f375a573740 1 -- 172.21.15.83:0/3747080792 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c43a9d66a0 con 0x55c43aa6f270 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.722+0000 7f3738ff9640 1 -- 172.21.15.83:0/3747080792 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3740061020 con 0x55c43aa6f270 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.722+0000 7f3738ff9640 1 -- 172.21.15.83:0/3747080792 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3740034650 con 0x55c43aa6f270 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.722+0000 7f375a573740 10 client.25411 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.722+0000 7f375a573740 10 client.25411 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.722+0000 7f375a573740 20 client.25411 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.722+0000 7f375a573740 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c43aa85340 0x55c43aa87730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.722+0000 7f375a573740 4 client.25411 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.722+0000 7f375a573740 1 -- 172.21.15.83:0/3747080792 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c43a714720 con 0x55c43aa85340 2024-03-11T19:41:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.722+0000 7f3755736640 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c43aa85340 0x55c43aa87730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.723+0000 7f3755736640 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c43aa85340 0x55c43aa87730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b24ffd2f79b14253 server_cookie=2ce198fa6a7f193f in_seq=0 out_seq=0 2024-03-11T19:41:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.723+0000 7f3738ff9640 10 client.25411 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:41:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.723+0000 7f3738ff9640 1 -- 172.21.15.83:0/3747080792 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c43a714720 con 0x55c43aa85340 2024-03-11T19:41:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.723+0000 7f3738ff9640 10 client.25411 handle_command_reply: tid=0 2024-03-11T19:41:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.723+0000 7f375a573740 1 client.25411 shutdown 2024-03-11T19:41:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f375a573740 1 -- 172.21.15.83:0/3747080792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 msgr2=0x55c43aa794b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f375a573740 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa794b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3740004a20 tx=0x7f374002c150 comp rx=0 tx=0).stop 2024-03-11T19:41:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f375a573740 1 -- 172.21.15.83:0/3747080792 shutdown_connections 2024-03-11T19:41:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f375a573740 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c43aa85340 0x55c43aa87730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f375a573740 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa74dd0 0x55c43aa80a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f375a573740 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6fc60 0x55c43aa799f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f375a573740 1 --2- 172.21.15.83:0/3747080792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa6f270 0x55c43aa794b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f375a573740 1 -- 172.21.15.83:0/3747080792 >> 172.21.15.83:0/3747080792 conn(0x55c43aa6a7e0 msgr2=0x55c43aa69b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f3738ff9640 0 client.25411 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:41:47.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f375a573740 1 -- 172.21.15.83:0/3747080792 shutdown_connections 2024-03-11T19:41:47.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f375a573740 1 -- 172.21.15.83:0/3747080792 wait complete. 2024-03-11T19:41:47.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.724+0000 7f375a573740 20 client.25411 trim_cache size 0 max 16384 2024-03-11T19:41:47.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f375a573740 1 Processor -- start 2024-03-11T19:41:47.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f375a573740 1 -- start start 2024-03-11T19:41:47.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f375a573740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6f270 0x55c43aa77f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f375a573740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6fc60 0x55c43aa78480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f375a573740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa74dd0 0x55c43aa809c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f3754f35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6f270 0x55c43aa77f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f375a573740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43a95d130 con 0x55c43aa6f270 2024-03-11T19:41:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f375a573740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43aa5ca40 con 0x55c43aa6fc60 2024-03-11T19:41:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f3754f35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6f270 0x55c43aa77f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49134/0 (socket says 172.21.15.83:49134) 2024-03-11T19:41:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f3754f35640 1 -- 172.21.15.83:0/2378464836 learned_addr learned my addr 172.21.15.83:0/2378464836 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:47.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f375a573740 1 -- 172.21.15.83:0/2378464836 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43aa7a6a0 con 0x55c43aa74dd0 2024-03-11T19:41:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.742+0000 7f3755736640 1 --2- 172.21.15.83:0/2378464836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6fc60 0x55c43aa78480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.743+0000 7f3754f35640 1 --2- 172.21.15.83:0/2378464836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa74dd0 0x55c43aa809c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.743+0000 7f3755736640 1 -- 172.21.15.83:0/2378464836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa74dd0 msgr2=0x55c43aa809c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.743+0000 7f3755736640 1 --2- 172.21.15.83:0/2378464836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa74dd0 0x55c43aa809c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.743+0000 7f3755736640 1 -- 172.21.15.83:0/2378464836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6f270 msgr2=0x55c43aa77f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.743+0000 7f3755736640 1 --2- 172.21.15.83:0/2378464836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6f270 0x55c43aa77f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.743+0000 7f3755736640 1 -- 172.21.15.83:0/2378464836 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c43a972c20 con 0x55c43aa6fc60 2024-03-11T19:41:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.743+0000 7f3754f35640 1 --2- 172.21.15.83:0/2378464836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa74dd0 0x55c43aa809c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:41:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.743+0000 7f3755736640 1 --2- 172.21.15.83:0/2378464836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6fc60 0x55c43aa78480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f374c005940 tx=0x7f374c0054d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6257beef541a9c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.743+0000 7f373affd640 1 -- 172.21.15.83:0/2378464836 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c017040 con 0x55c43aa6fc60 2024-03-11T19:41:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.744+0000 7f373affd640 1 -- 172.21.15.83:0/2378464836 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f374c019040 con 0x55c43aa6fc60 2024-03-11T19:41:47.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.744+0000 7f373affd640 1 -- 172.21.15.83:0/2378464836 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c012870 con 0x55c43aa6fc60 2024-03-11T19:41:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.744+0000 7f375a573740 1 -- 172.21.15.83:0/2378464836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6fc60 msgr2=0x55c43aa78480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.745+0000 7f375a573740 1 --2- 172.21.15.83:0/2378464836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6fc60 0x55c43aa78480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f374c005940 tx=0x7f374c0054d0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.745+0000 7f375a573740 1 -- 172.21.15.83:0/2378464836 shutdown_connections 2024-03-11T19:41:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.745+0000 7f375a573740 1 --2- 172.21.15.83:0/2378464836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa74dd0 0x55c43aa809c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.745+0000 7f375a573740 1 --2- 172.21.15.83:0/2378464836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6fc60 0x55c43aa78480 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.745+0000 7f375a573740 1 --2- 172.21.15.83:0/2378464836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6f270 0x55c43aa77f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.745+0000 7f375a573740 1 -- 172.21.15.83:0/2378464836 >> 172.21.15.83:0/2378464836 conn(0x55c43aa6a7e0 msgr2=0x55c43aa711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.745+0000 7f375a573740 1 -- 172.21.15.83:0/2378464836 shutdown_connections 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.745+0000 7f375a573740 1 -- 172.21.15.83:0/2378464836 wait complete. 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.745+0000 7f375a573740 1 -- start start 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.745+0000 7f375a573740 1 Processor -- start 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f375a573740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6f270 0x55c43aa80310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f375a573740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 0x55c43aa80850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f375a573740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa74dd0 0x55c43aa7e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f375a573740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43aa7ef30 con 0x55c43aa6fc60 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f375a573740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43aa7f0a0 con 0x55c43aa6f270 2024-03-11T19:41:47.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f375a573740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43aa7f260 con 0x55c43aa74dd0 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f375a573740 10 client.0 _ll_register_callbacks cb 0x7f37480093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f3755736640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 0x55c43aa80850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f3755736640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 0x55c43aa80850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49146/0 (socket says 172.21.15.83:49146) 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f3755736640 1 -- 172.21.15.83:0/2276243271 learned_addr learned my addr 172.21.15.83:0/2276243271 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f3754f35640 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa74dd0 0x55c43aa7e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.746+0000 7f3754f35640 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6f270 0x55c43aa80310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.747+0000 7f3755736640 1 -- 172.21.15.83:0/2276243271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa74dd0 msgr2=0x55c43aa7e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.747+0000 7f3755736640 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa74dd0 0x55c43aa7e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.747+0000 7f3755736640 1 -- 172.21.15.83:0/2276243271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6f270 msgr2=0x55c43aa80310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.747+0000 7f3755736640 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6f270 0x55c43aa80310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.747+0000 7f3755736640 1 -- 172.21.15.83:0/2276243271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374c00bdc0 con 0x55c43aa6fc60 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.747+0000 7f3755736640 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 0x55c43aa80850 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f374c004770 tx=0x7f374c004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1cbbac28584860 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.747+0000 7f3738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:41:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.747+0000 7f3738ff9640 1 -- 172.21.15.83:0/2276243271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c017040 con 0x55c43aa6fc60 2024-03-11T19:41:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.747+0000 7f375a573740 1 -- 172.21.15.83:0/2276243271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3740004a50 con 0x55c43aa6fc60 2024-03-11T19:41:47.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.747+0000 7f3738ff9640 1 -- 172.21.15.83:0/2276243271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f374c01e670 con 0x55c43aa6fc60 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.748+0000 7f3738ff9640 1 -- 172.21.15.83:0/2276243271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f374c036030 con 0x55c43aa6fc60 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.748+0000 7f375a573740 10 client.15810 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.748+0000 7f375a573740 1 -- 172.21.15.83:0/2276243271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c43a9988c0 con 0x55c43aa6fc60 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.748+0000 7f3738ff9640 1 -- 172.21.15.83:0/2276243271 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3740004a50 con 0x55c43aa6fc60 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.748+0000 7f3738ff9640 1 -- 172.21.15.83:0/2276243271 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f374c02e020 con 0x55c43aa6fc60 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.748+0000 7f375a573740 10 client.15810 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.748+0000 7f375a573740 10 client.15810 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.748+0000 7f375a573740 20 client.15810 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.749+0000 7f375a573740 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c43aadcaf0 0x55c43aadef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.749+0000 7f375a573740 4 client.15810 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.749+0000 7f375a573740 1 -- 172.21.15.83:0/2276243271 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c43a763890 con 0x55c43aadcaf0 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.750+0000 7f3754f35640 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c43aadcaf0 0x55c43aadef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.751+0000 7f3754f35640 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c43aadcaf0 0x55c43aadef10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63be1032173bf302 server_cookie=fcf84029a89bdfc3 in_seq=0 out_seq=0 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.751+0000 7f3738ff9640 10 client.15810 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.751+0000 7f3738ff9640 1 -- 172.21.15.83:0/2276243271 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c43a763890 con 0x55c43aadcaf0 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.751+0000 7f3738ff9640 10 client.15810 handle_command_reply: tid=0 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.751+0000 7f375a573740 1 client.15810 shutdown 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.751+0000 7f375a573740 1 -- 172.21.15.83:0/2276243271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 msgr2=0x55c43aa80850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.751+0000 7f375a573740 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 0x55c43aa80850 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f374c004770 tx=0x7f374c004ee0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.752+0000 7f375a573740 1 -- 172.21.15.83:0/2276243271 shutdown_connections 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.752+0000 7f375a573740 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c43aadcaf0 0x55c43aadef10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.752+0000 7f3738ff9640 0 client.15810 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.752+0000 7f375a573740 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c43aa74dd0 0x55c43aa7e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.752+0000 7f375a573740 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c43aa6fc60 0x55c43aa80850 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.752+0000 7f375a573740 1 --2- 172.21.15.83:0/2276243271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43aa6f270 0x55c43aa80310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.752+0000 7f375a573740 1 -- 172.21.15.83:0/2276243271 >> 172.21.15.83:0/2276243271 conn(0x55c43aa6a7e0 msgr2=0x55c43aa711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.753+0000 7f375a573740 1 -- 172.21.15.83:0/2276243271 shutdown_connections 2024-03-11T19:41:47.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.753+0000 7f375a573740 1 -- 172.21.15.83:0/2276243271 wait complete. 2024-03-11T19:41:47.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.753+0000 7f375a573740 20 client.15810 trim_cache size 0 max 16384 2024-03-11T19:41:47.759 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:41:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.754+0000 7f375799a640 1 -- 172.21.15.83:0/1692723529 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f373c0a1a80 msgr2=0x7f373c0a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.754+0000 7f375799a640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f373c0a1a80 0x7f373c0a3f40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f37480049d0 tx=0x7f3748000f40 comp rx=0 tx=0).stop 2024-03-11T19:41:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.754+0000 7f375799a640 1 -- 172.21.15.83:0/1692723529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3750102d40 msgr2=0x7f37501a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.754+0000 7f375799a640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3750102d40 0x7f37501a0a00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f374800eb10 tx=0x7f374800c2d0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.754+0000 7f375799a640 1 -- 172.21.15.83:0/1692723529 shutdown_connections 2024-03-11T19:41:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.754+0000 7f375799a640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f373c0a1a80 0x7f373c0a3f40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.754+0000 7f375799a640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3750102d40 0x7f37501a0a00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.754+0000 7f375799a640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3750102380 0x7f3750199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.754+0000 7f375799a640 1 --2- 172.21.15.83:0/1692723529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3750101190 0x7f3750199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.754+0000 7f375799a640 1 -- 172.21.15.83:0/1692723529 >> 172.21.15.83:0/1692723529 conn(0x7f37500fc6e0 msgr2=0x7f37500733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.756+0000 7f375799a640 1 -- 172.21.15.83:0/1692723529 shutdown_connections 2024-03-11T19:41:47.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.756+0000 7f375799a640 1 -- 172.21.15.83:0/1692723529 wait complete. 2024-03-11T19:41:47.778 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:41:47.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:41:47.814 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:47 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3260144639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:41:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.940+0000 7f8e541b5640 1 -- 172.21.15.83:0/2879421345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e4c104f70 msgr2=0x7f8e4c10b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.940+0000 7f8e541b5640 1 --2- 172.21.15.83:0/2879421345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e4c104f70 0x7f8e4c10b800 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009ca0 tx=0x7f8e4402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.940+0000 7f8e541b5640 1 -- 172.21.15.83:0/2879421345 shutdown_connections 2024-03-11T19:41:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.940+0000 7f8e541b5640 1 --2- 172.21.15.83:0/2879421345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e4c104f70 0x7f8e4c10b800 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.940+0000 7f8e541b5640 1 --2- 172.21.15.83:0/2879421345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e4c1045b0 0x7f8e4c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.940+0000 7f8e541b5640 1 --2- 172.21.15.83:0/2879421345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e4c1033c0 0x7f8e4c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.940+0000 7f8e541b5640 1 -- 172.21.15.83:0/2879421345 >> 172.21.15.83:0/2879421345 conn(0x7f8e4c0fe930 msgr2=0x7f8e4c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.941+0000 7f8e541b5640 1 -- 172.21.15.83:0/2879421345 shutdown_connections 2024-03-11T19:41:47.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.942+0000 7f8e541b5640 1 -- 172.21.15.83:0/2879421345 wait complete. 2024-03-11T19:41:47.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.943+0000 7f8e541b5640 1 Processor -- start 2024-03-11T19:41:47.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.943+0000 7f8e541b5640 1 -- start start 2024-03-11T19:41:47.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.944+0000 7f8e541b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e4c1033c0 0x7f8e4c1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.944+0000 7f8e541b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e4c1045b0 0x7f8e4c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.944+0000 7f8e541b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e4c104f70 0x7f8e4c1a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.944+0000 7f8e541b5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1a0eb0 con 0x7f8e4c1033c0 2024-03-11T19:41:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.944+0000 7f8e541b5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1a1020 con 0x7f8e4c104f70 2024-03-11T19:41:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.944+0000 7f8e541b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1a11e0 con 0x7f8e4c1045b0 2024-03-11T19:41:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.944+0000 7f8e52752640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e4c104f70 0x7f8e4c1a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.944+0000 7f8e52752640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e4c104f70 0x7f8e4c1a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:33030/0 (socket says 172.21.15.83:33030) 2024-03-11T19:41:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.944+0000 7f8e52752640 1 -- 172.21.15.83:0/1744449041 learned_addr learned my addr 172.21.15.83:0/1744449041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:47.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.944+0000 7f8e51750640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e4c1045b0 0x7f8e4c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.945+0000 7f8e52752640 1 -- 172.21.15.83:0/1744449041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e4c1045b0 msgr2=0x7f8e4c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.945+0000 7f8e52752640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e4c1045b0 0x7f8e4c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.945+0000 7f8e52752640 1 -- 172.21.15.83:0/1744449041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e4c1033c0 msgr2=0x7f8e4c1993c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:41:47.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.945+0000 7f8e52752640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e4c1033c0 0x7f8e4c1993c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.945+0000 7f8e52752640 1 -- 172.21.15.83:0/1744449041 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e44009900 con 0x7f8e4c104f70 2024-03-11T19:41:47.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.945+0000 7f8e52752640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e4c104f70 0x7f8e4c1a0970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8e4402c630 tx=0x7f8e44032f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.945+0000 7f8e42ffd640 1 -- 172.21.15.83:0/1744449041 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44034e20 con 0x7f8e4c104f70 2024-03-11T19:41:47.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.946+0000 7f8e42ffd640 1 -- 172.21.15.83:0/1744449041 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e44003e50 con 0x7f8e4c104f70 2024-03-11T19:41:47.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.946+0000 7f8e42ffd640 1 -- 172.21.15.83:0/1744449041 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e440048a0 con 0x7f8e4c104f70 2024-03-11T19:41:47.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.946+0000 7f8e541b5640 1 -- 172.21.15.83:0/1744449041 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c1a1460 con 0x7f8e4c104f70 2024-03-11T19:41:47.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.946+0000 7f8e541b5640 1 -- 172.21.15.83:0/1744449041 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c1a1960 con 0x7f8e4c104f70 2024-03-11T19:41:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.950+0000 7f8e42ffd640 1 -- 172.21.15.83:0/1744449041 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8e4400a040 con 0x7f8e4c104f70 2024-03-11T19:41:47.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.950+0000 7f8e56d81740 1 Processor -- start 2024-03-11T19:41:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.951+0000 7f8e56d81740 1 -- start start 2024-03-11T19:41:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.951+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aeba9ea80 0x561aeba9eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.951+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.951+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aebaa45e0 0x561aebaa6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.951+0000 7f8e56d81740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeba52960 con 0x561aeba9ea80 2024-03-11T19:41:47.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.951+0000 7f8e56d81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeba06310 con 0x561aebaa45e0 2024-03-11T19:41:47.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.951+0000 7f8e56d81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeba03840 con 0x561aeba9f470 2024-03-11T19:41:47.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51f51640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51750640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aeba9ea80 0x561aeba9eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51f51640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48290/0 (socket says 172.21.15.83:48290) 2024-03-11T19:41:47.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51f51640 1 -- 172.21.15.83:0/1559385789 learned_addr learned my addr 172.21.15.83:0/1559385789 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:47.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51750640 1 --2- 172.21.15.83:0/1559385789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aebaa45e0 0x561aebaa6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51f51640 1 -- 172.21.15.83:0/1559385789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aebaa45e0 msgr2=0x561aebaa6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51f51640 1 --2- 172.21.15.83:0/1559385789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aebaa45e0 0x561aebaa6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51f51640 1 -- 172.21.15.83:0/1559385789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aeba9ea80 msgr2=0x561aeba9eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51f51640 1 --2- 172.21.15.83:0/1559385789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aeba9ea80 0x561aeba9eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51f51640 1 -- 172.21.15.83:0/1559385789 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aeb9c1060 con 0x561aeba9f470 2024-03-11T19:41:47.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e42ffd640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8e380a1b30 0x7f8e380a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51f51640 1 --2- 172.21.15.83:0/1559385789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa3f70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009b80 tx=0x7f8e4800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58399eb47cdf9ec5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e51750640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8e380a1b30 0x7f8e380a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.952+0000 7f8e42ffd640 1 -- 172.21.15.83:0/1744449041 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8e440eff80 con 0x7f8e4c104f70 2024-03-11T19:41:47.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.953+0000 7f8e37fff640 1 -- 172.21.15.83:0/1559385789 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48009700 con 0x561aeba9f470 2024-03-11T19:41:47.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.953+0000 7f8e37fff640 1 -- 172.21.15.83:0/1559385789 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e48009e70 con 0x561aeba9f470 2024-03-11T19:41:47.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.953+0000 7f8e37fff640 1 -- 172.21.15.83:0/1559385789 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48010690 con 0x561aeba9f470 2024-03-11T19:41:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.954+0000 7f8e56d81740 1 -- 172.21.15.83:0/1559385789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 msgr2=0x561aebaa3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.954+0000 7f8e56d81740 1 --2- 172.21.15.83:0/1559385789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa3f70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009b80 tx=0x7f8e4800e720 comp rx=0 tx=0).stop 2024-03-11T19:41:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.954+0000 7f8e51750640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8e380a1b30 0x7f8e380a3ff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x561aeba9fcd0 tx=0x7f8e3c0095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.954+0000 7f8e56d81740 1 -- 172.21.15.83:0/1559385789 shutdown_connections 2024-03-11T19:41:47.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.954+0000 7f8e56d81740 1 --2- 172.21.15.83:0/1559385789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aebaa45e0 0x561aebaa6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.954+0000 7f8e56d81740 1 --2- 172.21.15.83:0/1559385789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa3f70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.954+0000 7f8e56d81740 1 --2- 172.21.15.83:0/1559385789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aeba9ea80 0x561aeba9eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.954+0000 7f8e56d81740 1 -- 172.21.15.83:0/1559385789 >> 172.21.15.83:0/1559385789 conn(0x561aeba99ff0 msgr2=0x561aeba9a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.955+0000 7f8e56d81740 1 -- 172.21.15.83:0/1559385789 shutdown_connections 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.955+0000 7f8e56d81740 1 -- 172.21.15.83:0/1559385789 wait complete. 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.955+0000 7f8e56d81740 1 -- start start 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.955+0000 7f8e56d81740 1 Processor -- start 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9ea80 0x561aebaa8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aebaa45e0 0x561aebab0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e51750640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e51750640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48298/0 (socket says 172.21.15.83:48298) 2024-03-11T19:41:47.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e51750640 1 -- 172.21.15.83:0/1601323774 learned_addr learned my addr 172.21.15.83:0/1601323774 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:47.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e56d81740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb9ff020 con 0x561aebaa45e0 2024-03-11T19:41:47.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e56d81740 1 -- 172.21.15.83:0/1601323774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb6d2430 con 0x561aeba9ea80 2024-03-11T19:41:47.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e56d81740 1 -- 172.21.15.83:0/1601323774 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb68ee00 con 0x561aeba9f470 2024-03-11T19:41:47.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e51f51640 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9ea80 0x561aebaa8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e56d81740 10 client.0 _ll_register_callbacks cb 0x561aeb9a6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e51750640 1 -- 172.21.15.83:0/1601323774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9ea80 msgr2=0x561aebaa8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e51750640 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9ea80 0x561aebaa8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e51750640 1 -- 172.21.15.83:0/1601323774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aebaa45e0 msgr2=0x561aebab0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e51750640 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aebaa45e0 0x561aebab0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.956+0000 7f8e51750640 1 -- 172.21.15.83:0/1601323774 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e48009860 con 0x561aeba9f470 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.957+0000 7f8e51750640 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa9210 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009c60 tx=0x7f8e3c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ceedd1b5832c4d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.957+0000 7f8e56d81740 1 -- 172.21.15.83:0/1601323774 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561aeb6fe7e0 con 0x561aeba9f470 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.957+0000 7f8e357fa640 10 client.25426 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.957+0000 7f8e357fa640 1 -- 172.21.15.83:0/1601323774 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c021b60 con 0x561aeba9f470 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.957+0000 7f8e357fa640 1 -- 172.21.15.83:0/1601323774 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8e3c021cc0 con 0x561aeba9f470 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.957+0000 7f8e357fa640 1 -- 172.21.15.83:0/1601323774 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e3c021ea0 con 0x561aeba9f470 2024-03-11T19:41:47.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.957+0000 7f8e56d81740 10 client.25426 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.957+0000 7f8e56d81740 1 -- 172.21.15.83:0/1601323774 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x561aeb9fd6a0 con 0x561aeba9f470 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.957+0000 7f8e357fa640 1 -- 172.21.15.83:0/1601323774 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8e3c034470 con 0x561aeba9f470 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.958+0000 7f8e56d81740 10 client.25426 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.958+0000 7f8e56d81740 10 client.25426 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.958+0000 7f8e56d81740 20 client.25426 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.958+0000 7f8e56d81740 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x561aebab4ed0 0x561aebab7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.958+0000 7f8e56d81740 4 client.25426 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.958+0000 7f8e56d81740 1 -- 172.21.15.83:0/1601323774 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561aeb6fe7e0 con 0x561aebab4ed0 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.958+0000 7f8e51f51640 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x561aebab4ed0 0x561aebab7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.959+0000 7f8e51f51640 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x561aebab4ed0 0x561aebab7350 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13e4f82e6b2fdf00 server_cookie=d00aafc09b09b508 in_seq=0 out_seq=0 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.959+0000 7f8e357fa640 10 client.25426 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.959+0000 7f8e357fa640 1 -- 172.21.15.83:0/1601323774 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561aeb6fe7e0 con 0x561aebab4ed0 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.959+0000 7f8e357fa640 10 client.25426 handle_command_reply: tid=0 2024-03-11T19:41:47.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.959+0000 7f8e56d81740 1 client.25426 shutdown 2024-03-11T19:41:47.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.960+0000 7f8e56d81740 1 -- 172.21.15.83:0/1601323774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 msgr2=0x561aebaa9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.960+0000 7f8e56d81740 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa9210 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009c60 tx=0x7f8e3c004730 comp rx=0 tx=0).stop 2024-03-11T19:41:47.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.960+0000 7f8e56d81740 1 -- 172.21.15.83:0/1601323774 shutdown_connections 2024-03-11T19:41:47.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.960+0000 7f8e56d81740 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x561aebab4ed0 0x561aebab7350 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.960+0000 7f8e56d81740 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aebaa45e0 0x561aebab0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.960+0000 7f8e357fa640 0 client.25426 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:41:47.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.960+0000 7f8e56d81740 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9f470 0x561aebaa9210 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.960+0000 7f8e56d81740 1 --2- 172.21.15.83:0/1601323774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9ea80 0x561aebaa8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.960+0000 7f8e56d81740 1 -- 172.21.15.83:0/1601323774 >> 172.21.15.83:0/1601323774 conn(0x561aeba99ff0 msgr2=0x561aeba9b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.960+0000 7f8e56d81740 1 -- 172.21.15.83:0/1601323774 shutdown_connections 2024-03-11T19:41:47.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.960+0000 7f8e56d81740 1 -- 172.21.15.83:0/1601323774 wait complete. 2024-03-11T19:41:47.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.961+0000 7f8e56d81740 20 client.25426 trim_cache size 0 max 16384 2024-03-11T19:41:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.980+0000 7f8e56d81740 1 Processor -- start 2024-03-11T19:41:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.980+0000 7f8e56d81740 1 -- start start 2024-03-11T19:41:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.980+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9ea80 0x561aeba9a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.980+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9f470 0x561aebab0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.980+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aebaa45e0 0x561aebaa0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.980+0000 7f8e56d81740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb984130 con 0x561aebaa45e0 2024-03-11T19:41:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.980+0000 7f8e56d81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aebaa8b10 con 0x561aeba9f470 2024-03-11T19:41:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.980+0000 7f8e56d81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aeb6fe7e0 con 0x561aeba9ea80 2024-03-11T19:41:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.981+0000 7f8e52752640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aebaa45e0 0x561aebaa0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.981+0000 7f8e52752640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aebaa45e0 0x561aebaa0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49188/0 (socket says 172.21.15.83:49188) 2024-03-11T19:41:47.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.981+0000 7f8e52752640 1 -- 172.21.15.83:0/936799483 learned_addr learned my addr 172.21.15.83:0/936799483 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:47.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.981+0000 7f8e51750640 1 --2- 172.21.15.83:0/936799483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9f470 0x561aebab0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.981+0000 7f8e51f51640 1 --2- 172.21.15.83:0/936799483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9ea80 0x561aeba9a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.981+0000 7f8e52752640 1 -- 172.21.15.83:0/936799483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9ea80 msgr2=0x561aeba9a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.981+0000 7f8e52752640 1 --2- 172.21.15.83:0/936799483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9ea80 0x561aeba9a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.981+0000 7f8e52752640 1 -- 172.21.15.83:0/936799483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9f470 msgr2=0x561aebab0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.981+0000 7f8e52752640 1 --2- 172.21.15.83:0/936799483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9f470 0x561aebab0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.981+0000 7f8e52752640 1 -- 172.21.15.83:0/936799483 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aeb999c20 con 0x561aebaa45e0 2024-03-11T19:41:47.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.981+0000 7f8e51f51640 1 --2- 172.21.15.83:0/936799483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9ea80 0x561aeba9a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:47.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.982+0000 7f8e52752640 1 --2- 172.21.15.83:0/936799483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aebaa45e0 0x561aebaa0990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8e440bce90 tx=0x7f8e440bc860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e8812d915253f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.982+0000 7f8e377fe640 1 -- 172.21.15.83:0/936799483 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e440bcf00 con 0x561aebaa45e0 2024-03-11T19:41:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.982+0000 7f8e377fe640 1 -- 172.21.15.83:0/936799483 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e440c1050 con 0x561aebaa45e0 2024-03-11T19:41:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.982+0000 7f8e377fe640 1 -- 172.21.15.83:0/936799483 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e440bd160 con 0x561aebaa45e0 2024-03-11T19:41:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.983+0000 7f8e56d81740 1 -- 172.21.15.83:0/936799483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aebaa45e0 msgr2=0x561aebaa0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.983+0000 7f8e56d81740 1 --2- 172.21.15.83:0/936799483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aebaa45e0 0x561aebaa0990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8e440bce90 tx=0x7f8e440bc860 comp rx=0 tx=0).stop 2024-03-11T19:41:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.984+0000 7f8e56d81740 1 -- 172.21.15.83:0/936799483 shutdown_connections 2024-03-11T19:41:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.984+0000 7f8e56d81740 1 --2- 172.21.15.83:0/936799483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aebaa45e0 0x561aebaa0990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.984+0000 7f8e56d81740 1 --2- 172.21.15.83:0/936799483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9f470 0x561aebab0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.984+0000 7f8e56d81740 1 --2- 172.21.15.83:0/936799483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aeba9ea80 0x561aeba9a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.984+0000 7f8e56d81740 1 -- 172.21.15.83:0/936799483 >> 172.21.15.83:0/936799483 conn(0x561aeba99ff0 msgr2=0x561aeba8ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.984+0000 7f8e56d81740 1 -- 172.21.15.83:0/936799483 shutdown_connections 2024-03-11T19:41:47.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.985+0000 7f8e56d81740 1 -- 172.21.15.83:0/936799483 wait complete. 2024-03-11T19:41:47.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.985+0000 7f8e56d81740 1 -- start start 2024-03-11T19:41:47.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.985+0000 7f8e56d81740 1 Processor -- start 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.986+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9ea80 0x561aebab0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.986+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aeba9f470 0x561aebaae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.986+0000 7f8e56d81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aebaa45e0 0x561aebaaee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.986+0000 7f8e56d81740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aebaaf560 con 0x561aeba9f470 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.986+0000 7f8e56d81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aebaaf6d0 con 0x561aeba9ea80 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.986+0000 7f8e56d81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aebb0b450 con 0x561aebaa45e0 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.986+0000 7f8e56d81740 10 client.0 _ll_register_callbacks cb 0x7f8e48009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.986+0000 7f8e51f51640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9ea80 0x561aebab0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.986+0000 7f8e52752640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aebaa45e0 0x561aebaaee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.986+0000 7f8e52752640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aebaa45e0 0x561aebaaee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48326/0 (socket says 172.21.15.83:48326) 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.986+0000 7f8e52752640 1 -- 172.21.15.83:0/2971919067 learned_addr learned my addr 172.21.15.83:0/2971919067 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.987+0000 7f8e51750640 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aeba9f470 0x561aebaae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.987+0000 7f8e52752640 1 -- 172.21.15.83:0/2971919067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9ea80 msgr2=0x561aebab0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.987+0000 7f8e52752640 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9ea80 0x561aebab0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.987+0000 7f8e52752640 1 -- 172.21.15.83:0/2971919067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aeba9f470 msgr2=0x561aebaae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.987+0000 7f8e52752640 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aeba9f470 0x561aebaae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.987+0000 7f8e52752640 1 -- 172.21.15.83:0/2971919067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4801c040 con 0x561aebaa45e0 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.987+0000 7f8e52752640 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aebaa45e0 0x561aebaaee60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e44005e30 tx=0x7f8e4402c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e895f303f948e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.987+0000 7f8e357fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.987+0000 7f8e357fa640 1 -- 172.21.15.83:0/2971919067 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e440bcf00 con 0x561aebaa45e0 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.987+0000 7f8e357fa640 1 -- 172.21.15.83:0/2971919067 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8e440bafa0 con 0x561aebaa45e0 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.987+0000 7f8e56d81740 1 -- 172.21.15.83:0/2971919067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e48009e70 con 0x561aebaa45e0 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.988+0000 7f8e357fa640 1 -- 172.21.15.83:0/2971919067 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e440bd370 con 0x561aebaa45e0 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.988+0000 7f8e56d81740 10 client.25432 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.988+0000 7f8e56d81740 1 -- 172.21.15.83:0/2971919067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f8e4c1a1960 con 0x561aebaa45e0 2024-03-11T19:41:47.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.988+0000 7f8e357fa640 1 -- 172.21.15.83:0/2971919067 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e440db2c0 con 0x561aebaa45e0 2024-03-11T19:41:47.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.988+0000 7f8e357fa640 1 -- 172.21.15.83:0/2971919067 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8e44004bd0 con 0x561aebaa45e0 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.988+0000 7f8e56d81740 10 client.25432 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.988+0000 7f8e56d81740 10 client.25432 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.988+0000 7f8e56d81740 20 client.25432 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.988+0000 7f8e56d81740 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x561aebab4ed0 0x561aebb0e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.988+0000 7f8e56d81740 4 client.25432 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.988+0000 7f8e56d81740 1 -- 172.21.15.83:0/2971919067 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f8e48009e70 con 0x561aebab4ed0 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.989+0000 7f8e51f51640 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x561aebab4ed0 0x561aebb0e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.990+0000 7f8e357fa640 10 client.25432 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.990+0000 7f8e51f51640 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x561aebab4ed0 0x561aebb0e9d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa9e380f486e9d89 server_cookie=da88bc6e23f67418 in_seq=0 out_seq=0 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.990+0000 7f8e357fa640 1 -- 172.21.15.83:0/2971919067 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f8e48009e70 con 0x561aebab4ed0 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.990+0000 7f8e357fa640 10 client.25432 handle_command_reply: tid=0 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.990+0000 7f8e56d81740 1 client.25432 shutdown 2024-03-11T19:41:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.990+0000 7f8e56d81740 1 -- 172.21.15.83:0/2971919067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aebaa45e0 msgr2=0x561aebaaee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.990+0000 7f8e56d81740 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aebaa45e0 0x561aebaaee60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e44005e30 tx=0x7f8e4402c930 comp rx=0 tx=0).stop 2024-03-11T19:41:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.991+0000 7f8e56d81740 1 -- 172.21.15.83:0/2971919067 shutdown_connections 2024-03-11T19:41:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.991+0000 7f8e56d81740 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x561aebab4ed0 0x561aebb0e9d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.991+0000 7f8e357fa640 0 client.25432 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:41:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.991+0000 7f8e56d81740 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561aebaa45e0 0x561aebaaee60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.991+0000 7f8e56d81740 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561aeba9f470 0x561aebaae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.991+0000 7f8e56d81740 1 --2- 172.21.15.83:0/2971919067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561aeba9ea80 0x561aebab0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.991+0000 7f8e56d81740 1 -- 172.21.15.83:0/2971919067 >> 172.21.15.83:0/2971919067 conn(0x561aeba99ff0 msgr2=0x561aeba9a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.991+0000 7f8e56d81740 1 -- 172.21.15.83:0/2971919067 shutdown_connections 2024-03-11T19:41:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.991+0000 7f8e56d81740 1 -- 172.21.15.83:0/2971919067 wait complete. 2024-03-11T19:41:47.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.991+0000 7f8e56d81740 20 client.25432 trim_cache size 0 max 16384 2024-03-11T19:41:47.998 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 -- 172.21.15.83:0/1744449041 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8e380a1b30 msgr2=0x7f8e380a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8e380a1b30 0x7f8e380a3ff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x561aeba9fcd0 tx=0x7f8e3c0095e0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 -- 172.21.15.83:0/1744449041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e4c104f70 msgr2=0x7f8e4c1a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e4c104f70 0x7f8e4c1a0970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8e4402c630 tx=0x7f8e44032f80 comp rx=0 tx=0).stop 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 -- 172.21.15.83:0/1744449041 shutdown_connections 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8e380a1b30 0x7f8e380a3ff0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e4c104f70 0x7f8e4c1a0970 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e4c1045b0 0x7f8e4c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 --2- 172.21.15.83:0/1744449041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8e4c1033c0 0x7f8e4c1993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 -- 172.21.15.83:0/1744449041 >> 172.21.15.83:0/1744449041 conn(0x7f8e4c0fe930 msgr2=0x7f8e4c1078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 -- 172.21.15.83:0/1744449041 shutdown_connections 2024-03-11T19:41:47.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:47.993+0000 7f8e541b5640 1 -- 172.21.15.83:0/1744449041 wait complete. 2024-03-11T19:41:48.013 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:41:48.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:41:48.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:47 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3260144639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:41:48.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:47 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3260144639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:41:48.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.223+0000 7f56c5a4e640 1 -- 172.21.15.83:0/4031998399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101280 msgr2=0x7f56c0101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:48.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.223+0000 7f56c5a4e640 1 --2- 172.21.15.83:0/4031998399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101280 0x7f56c0101700 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009860 tx=0x7f56b402bb70 comp rx=0 tx=0).stop 2024-03-11T19:41:48.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.223+0000 7f56c5a4e640 1 -- 172.21.15.83:0/4031998399 shutdown_connections 2024-03-11T19:41:48.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.223+0000 7f56c5a4e640 1 --2- 172.21.15.83:0/4031998399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0101c40 0x7f56c01084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.223+0000 7f56c5a4e640 1 --2- 172.21.15.83:0/4031998399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101280 0x7f56c0101700 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.223+0000 7f56c5a4e640 1 --2- 172.21.15.83:0/4031998399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0100090 0x7f56c01004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.223+0000 7f56c5a4e640 1 -- 172.21.15.83:0/4031998399 >> 172.21.15.83:0/4031998399 conn(0x7f56c00fb620 msgr2=0x7f56c00fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:48.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.223+0000 7f56c5a4e640 1 -- 172.21.15.83:0/4031998399 shutdown_connections 2024-03-11T19:41:48.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.223+0000 7f56c5a4e640 1 -- 172.21.15.83:0/4031998399 wait complete. 2024-03-11T19:41:48.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56c5a4e640 1 Processor -- start 2024-03-11T19:41:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56c5a4e640 1 -- start start 2024-03-11T19:41:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56c5a4e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100090 0x7f56c006a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56c5a4e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101280 0x7f56c0199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56c5a4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101c40 0x7f56c019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56c5a4e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019aba0 con 0x7f56c0101c40 2024-03-11T19:41:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56c5a4e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019ad10 con 0x7f56c0100090 2024-03-11T19:41:48.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56c5a4e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019aed0 con 0x7f56c0101280 2024-03-11T19:41:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56beffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100090 0x7f56c006a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56beffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100090 0x7f56c006a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:33092/0 (socket says 172.21.15.83:33092) 2024-03-11T19:41:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56be7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101280 0x7f56c0199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.224+0000 7f56beffd640 1 -- 172.21.15.83:0/2837707572 learned_addr learned my addr 172.21.15.83:0/2837707572 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.225+0000 7f56beffd640 1 -- 172.21.15.83:0/2837707572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101280 msgr2=0x7f56c0199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.225+0000 7f56beffd640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101280 0x7f56c0199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.225+0000 7f56beffd640 1 -- 172.21.15.83:0/2837707572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101c40 msgr2=0x7f56c019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.225+0000 7f56beffd640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101c40 0x7f56c019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.225+0000 7f56beffd640 1 -- 172.21.15.83:0/2837707572 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4009490 con 0x7f56c0100090 2024-03-11T19:41:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.225+0000 7f56be7fc640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101280 0x7f56c0199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:41:48.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.225+0000 7f56beffd640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100090 0x7f56c006a280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56b000e9f0 tx=0x7f56b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:48.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.225+0000 7f56c4a4c640 1 -- 172.21.15.83:0/2837707572 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b000ccc0 con 0x7f56c0100090 2024-03-11T19:41:48.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.226+0000 7f56c5a4e640 1 -- 172.21.15.83:0/2837707572 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c019b1b0 con 0x7f56c0100090 2024-03-11T19:41:48.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.226+0000 7f56c4a4c640 1 -- 172.21.15.83:0/2837707572 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56b0004510 con 0x7f56c0100090 2024-03-11T19:41:48.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.226+0000 7f56c4a4c640 1 -- 172.21.15.83:0/2837707572 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0010430 con 0x7f56c0100090 2024-03-11T19:41:48.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.226+0000 7f56c5a4e640 1 -- 172.21.15.83:0/2837707572 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c019b690 con 0x7f56c0100090 2024-03-11T19:41:48.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.229+0000 7f56c4a4c640 1 -- 172.21.15.83:0/2837707572 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f56b0004030 con 0x7f56c0100090 2024-03-11T19:41:48.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.229+0000 7f56c8624740 1 Processor -- start 2024-03-11T19:41:48.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.229+0000 7f56c8624740 1 -- start start 2024-03-11T19:41:48.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.230+0000 7f56c8624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d30a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.230+0000 7f56c8624740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 0x55df8d30f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.230+0000 7f56c8624740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30fdd0 0x55df8d312260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.230+0000 7f56c8624740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8d2c6960 con 0x55df8d30ac60 2024-03-11T19:41:48.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.230+0000 7f56c8624740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8d27a310 con 0x55df8d30a270 2024-03-11T19:41:48.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.230+0000 7f56c8624740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8d277840 con 0x55df8d30fdd0 2024-03-11T19:41:48.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.230+0000 7f56beffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 0x55df8d30f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.230+0000 7f56beffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 0x55df8d30f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49226/0 (socket says 172.21.15.83:49226) 2024-03-11T19:41:48.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.230+0000 7f56beffd640 1 -- 172.21.15.83:0/157500086 learned_addr learned my addr 172.21.15.83:0/157500086 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:48.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.230+0000 7f56bf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d30a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.231+0000 7f56beffd640 1 -- 172.21.15.83:0/157500086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30fdd0 msgr2=0x55df8d312260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:41:48.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.231+0000 7f56beffd640 1 --2- 172.21.15.83:0/157500086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30fdd0 0x55df8d312260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.231+0000 7f56beffd640 1 -- 172.21.15.83:0/157500086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 msgr2=0x55df8d30a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:48.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.231+0000 7f56beffd640 1 --2- 172.21.15.83:0/157500086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d30a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.231+0000 7f56beffd640 1 -- 172.21.15.83:0/157500086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df8d235060 con 0x55df8d30ac60 2024-03-11T19:41:48.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.231+0000 7f56bf7fe640 1 --2- 172.21.15.83:0/157500086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d30a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:48.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.231+0000 7f56c4a4c640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56a80a1b80 0x7f56a80a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.231+0000 7f56beffd640 1 --2- 172.21.15.83:0/157500086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 0x55df8d30f760 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f56b00d3a90 tx=0x7f56b00d3f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822b12422acd5e24 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:48.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.231+0000 7f56c4a4c640 1 -- 172.21.15.83:0/2837707572 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f56b0014070 con 0x7f56c0100090 2024-03-11T19:41:48.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.232+0000 7f56bf7fe640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56a80a1b80 0x7f56a80a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.232+0000 7f56a5ffb640 1 -- 172.21.15.83:0/157500086 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0066110 con 0x55df8d30ac60 2024-03-11T19:41:48.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.232+0000 7f56a5ffb640 1 -- 172.21.15.83:0/157500086 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56b0096b50 con 0x55df8d30ac60 2024-03-11T19:41:48.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.232+0000 7f56a5ffb640 1 -- 172.21.15.83:0/157500086 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b009afb0 con 0x55df8d30ac60 2024-03-11T19:41:48.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.233+0000 7f56c8624740 1 -- 172.21.15.83:0/157500086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 msgr2=0x55df8d30f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:48.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.233+0000 7f56c8624740 1 --2- 172.21.15.83:0/157500086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 0x55df8d30f760 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f56b00d3a90 tx=0x7f56b00d3f50 comp rx=0 tx=0).stop 2024-03-11T19:41:48.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.233+0000 7f56bf7fe640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56a80a1b80 0x7f56a80a4040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009270 tx=0x7f56b80095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:48.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.235+0000 7f56c8624740 1 -- 172.21.15.83:0/157500086 shutdown_connections 2024-03-11T19:41:48.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.235+0000 7f56c8624740 1 --2- 172.21.15.83:0/157500086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30fdd0 0x55df8d312260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.235+0000 7f56c8624740 1 --2- 172.21.15.83:0/157500086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 0x55df8d30f760 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.235+0000 7f56c8624740 1 --2- 172.21.15.83:0/157500086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d30a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.235+0000 7f56c8624740 1 -- 172.21.15.83:0/157500086 >> 172.21.15.83:0/157500086 conn(0x55df8d3057e0 msgr2=0x55df8d305be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:48.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.235+0000 7f56c8624740 1 -- 172.21.15.83:0/157500086 shutdown_connections 2024-03-11T19:41:48.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.235+0000 7f56c8624740 1 -- 172.21.15.83:0/157500086 wait complete. 2024-03-11T19:41:48.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.235+0000 7f56c8624740 1 -- start start 2024-03-11T19:41:48.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.235+0000 7f56c8624740 1 Processor -- start 2024-03-11T19:41:48.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.236+0000 7f56c8624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d314480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.236+0000 7f56c8624740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30ac60 0x55df8d3149c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.236+0000 7f56c8624740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30fdd0 0x55df8d31ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.236+0000 7f56c8624740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8d273020 con 0x55df8d30fdd0 2024-03-11T19:41:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.236+0000 7f56be7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30ac60 0x55df8d3149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.236+0000 7f56beffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d314480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.236+0000 7f56beffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d314480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:33102/0 (socket says 172.21.15.83:33102) 2024-03-11T19:41:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.236+0000 7f56be7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30ac60 0x55df8d3149c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48370/0 (socket says 172.21.15.83:48370) 2024-03-11T19:41:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.236+0000 7f56beffd640 1 -- 172.21.15.83:0/3638145492 learned_addr learned my addr 172.21.15.83:0/3638145492 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.236+0000 7f56c8624740 1 -- 172.21.15.83:0/3638145492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8cf46430 con 0x55df8d30a270 2024-03-11T19:41:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f56bf7fe640 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30fdd0 0x55df8d31ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f56c8624740 1 -- 172.21.15.83:0/3638145492 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8cf02e00 con 0x55df8d30ac60 2024-03-11T19:41:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f56c8624740 10 client.0 _ll_register_callbacks cb 0x55df8d21acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f56beffd640 1 -- 172.21.15.83:0/3638145492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30ac60 msgr2=0x55df8d3149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:48.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f56beffd640 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30ac60 0x55df8d3149c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f56beffd640 1 -- 172.21.15.83:0/3638145492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30fdd0 msgr2=0x55df8d31ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:48.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f56beffd640 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30fdd0 0x55df8d31ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f56beffd640 1 -- 172.21.15.83:0/3638145492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c019b690 con 0x55df8d30a270 2024-03-11T19:41:48.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f56c8624740 1 -- 172.21.15.83:0/3638145492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df8cf727e0 con 0x55df8d30a270 2024-03-11T19:41:48.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f56be7fc640 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30ac60 0x55df8d3149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:41:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f56beffd640 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d314480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56b0097280 tx=0x7f56b00cd980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee6b739be77be1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.237+0000 7f569effd640 10 client.25334 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:41:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.238+0000 7f569effd640 1 -- 172.21.15.83:0/3638145492 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0066110 con 0x55df8d30a270 2024-03-11T19:41:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.238+0000 7f569effd640 1 -- 172.21.15.83:0/3638145492 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f56b000ce20 con 0x55df8d30a270 2024-03-11T19:41:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.238+0000 7f569effd640 1 -- 172.21.15.83:0/3638145492 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56b009a680 con 0x55df8d30a270 2024-03-11T19:41:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.238+0000 7f56c8624740 10 client.25334 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.238+0000 7f56c8624740 1 -- 172.21.15.83:0/3638145492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55df8d2716a0 con 0x55df8d30a270 2024-03-11T19:41:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.239+0000 7f569effd640 1 -- 172.21.15.83:0/3638145492 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f56b001f060 con 0x55df8d30a270 2024-03-11T19:41:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.239+0000 7f56c8624740 10 client.25334 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.239+0000 7f56c8624740 10 client.25334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.239+0000 7f56c8624740 20 client.25334 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.239+0000 7f56c8624740 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55df8d320070 0x55df8d3224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.239+0000 7f56c8624740 4 client.25334 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.239+0000 7f56c8624740 1 -- 172.21.15.83:0/3638145492 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df8cfaf720 con 0x55df8d320070 2024-03-11T19:41:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.240+0000 7f56be7fc640 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55df8d320070 0x55df8d3224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.241+0000 7f56be7fc640 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55df8d320070 0x55df8d3224a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f7e11c1fdaf7a server_cookie=8da636ae00093523 in_seq=0 out_seq=0 2024-03-11T19:41:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.241+0000 7f569effd640 10 client.25334 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.241+0000 7f569effd640 1 -- 172.21.15.83:0/3638145492 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df8cfaf720 con 0x55df8d320070 2024-03-11T19:41:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.241+0000 7f569effd640 10 client.25334 handle_command_reply: tid=0 2024-03-11T19:41:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 1 client.25334 shutdown 2024-03-11T19:41:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 1 -- 172.21.15.83:0/3638145492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 msgr2=0x55df8d314480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d314480 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56b0097280 tx=0x7f56b00cd980 comp rx=0 tx=0).stop 2024-03-11T19:41:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 1 -- 172.21.15.83:0/3638145492 shutdown_connections 2024-03-11T19:41:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55df8d320070 0x55df8d3224a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30fdd0 0x55df8d31ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f569effd640 0 client.25334 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30ac60 0x55df8d3149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 1 --2- 172.21.15.83:0/3638145492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d314480 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 1 -- 172.21.15.83:0/3638145492 >> 172.21.15.83:0/3638145492 conn(0x55df8d3057e0 msgr2=0x55df8d304b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 1 -- 172.21.15.83:0/3638145492 shutdown_connections 2024-03-11T19:41:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 1 -- 172.21.15.83:0/3638145492 wait complete. 2024-03-11T19:41:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.242+0000 7f56c8624740 20 client.25334 trim_cache size 0 max 16384 2024-03-11T19:41:48.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.262+0000 7f56c8624740 1 Processor -- start 2024-03-11T19:41:48.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.262+0000 7f56c8624740 1 -- start start 2024-03-11T19:41:48.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.262+0000 7f56c8624740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30a270 0x55df8d30c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.262+0000 7f56c8624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30ac60 0x55df8d31b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.262+0000 7f56c8624740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30fdd0 0x55df8d30c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.262+0000 7f56c8624740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8d1f8130 con 0x55df8d30fdd0 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.262+0000 7f56c8624740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8d3154f0 con 0x55df8d30ac60 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.262+0000 7f56c8624740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8d30b350 con 0x55df8d30a270 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.262+0000 7f56beffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30a270 0x55df8d30c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.262+0000 7f56beffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30a270 0x55df8d30c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48384/0 (socket says 172.21.15.83:48384) 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.262+0000 7f56beffd640 1 -- 172.21.15.83:0/463722436 learned_addr learned my addr 172.21.15.83:0/463722436 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.263+0000 7f56bf7fe640 1 --2- 172.21.15.83:0/463722436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30fdd0 0x55df8d30c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.263+0000 7f56be7fc640 1 --2- 172.21.15.83:0/463722436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30ac60 0x55df8d31b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.263+0000 7f56beffd640 1 -- 172.21.15.83:0/463722436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30ac60 msgr2=0x55df8d31b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.263+0000 7f56beffd640 1 --2- 172.21.15.83:0/463722436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30ac60 0x55df8d31b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.263+0000 7f56beffd640 1 -- 172.21.15.83:0/463722436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30fdd0 msgr2=0x55df8d30c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.263+0000 7f56beffd640 1 --2- 172.21.15.83:0/463722436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30fdd0 0x55df8d30c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.263+0000 7f56beffd640 1 -- 172.21.15.83:0/463722436 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df8d20dc20 con 0x55df8d30a270 2024-03-11T19:41:48.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.263+0000 7f56bf7fe640 1 --2- 172.21.15.83:0/463722436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30fdd0 0x55df8d30c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:41:48.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.263+0000 7f56be7fc640 1 --2- 172.21.15.83:0/463722436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30ac60 0x55df8d31b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:41:48.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.263+0000 7f56beffd640 1 --2- 172.21.15.83:0/463722436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30a270 0x55df8d30c1b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f56b0096d10 tx=0x7f56b00cd000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1f9bc81bdaacdec server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:48.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.263+0000 7f56a57fa640 1 -- 172.21.15.83:0/463722436 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b009a680 con 0x55df8d30a270 2024-03-11T19:41:48.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.264+0000 7f56a57fa640 1 -- 172.21.15.83:0/463722436 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56b00b7020 con 0x55df8d30a270 2024-03-11T19:41:48.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.264+0000 7f56a57fa640 1 -- 172.21.15.83:0/463722436 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b00a3050 con 0x55df8d30a270 2024-03-11T19:41:48.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.264+0000 7f56c8624740 1 -- 172.21.15.83:0/463722436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30a270 msgr2=0x55df8d30c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.264+0000 7f56c8624740 1 --2- 172.21.15.83:0/463722436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30a270 0x55df8d30c1b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f56b0096d10 tx=0x7f56b00cd000 comp rx=0 tx=0).stop 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.265+0000 7f56c8624740 1 -- 172.21.15.83:0/463722436 shutdown_connections 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.265+0000 7f56c8624740 1 --2- 172.21.15.83:0/463722436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30fdd0 0x55df8d30c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.265+0000 7f56c8624740 1 --2- 172.21.15.83:0/463722436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30ac60 0x55df8d31b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.265+0000 7f56c8624740 1 --2- 172.21.15.83:0/463722436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30a270 0x55df8d30c1b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.265+0000 7f56c8624740 1 -- 172.21.15.83:0/463722436 >> 172.21.15.83:0/463722436 conn(0x55df8d3057e0 msgr2=0x55df8d300a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.265+0000 7f56c8624740 1 -- 172.21.15.83:0/463722436 shutdown_connections 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.265+0000 7f56c8624740 1 -- 172.21.15.83:0/463722436 wait complete. 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.265+0000 7f56c8624740 1 -- start start 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56c8624740 1 Processor -- start 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56c8624740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d31b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56c8624740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 0x55df8d319a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56c8624740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30fdd0 0x55df8d319fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56c8624740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8d31bc60 con 0x55df8d30ac60 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56c8624740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8d31a500 con 0x55df8d30a270 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56c8624740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df8d31a6c0 con 0x55df8d30fdd0 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56c8624740 10 client.0 _ll_register_callbacks cb 0x7f56b00d3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56be7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 0x55df8d319a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56beffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d31b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56be7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 0x55df8d319a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49252/0 (socket says 172.21.15.83:49252) 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56be7fc640 1 -- 172.21.15.83:0/4047618133 learned_addr learned my addr 172.21.15.83:0/4047618133 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.266+0000 7f56bf7fe640 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30fdd0 0x55df8d319fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.267+0000 7f56beffd640 1 -- 172.21.15.83:0/4047618133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30fdd0 msgr2=0x55df8d319fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.267+0000 7f56beffd640 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30fdd0 0x55df8d319fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.267+0000 7f56beffd640 1 -- 172.21.15.83:0/4047618133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 msgr2=0x55df8d319a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:48.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.267+0000 7f56beffd640 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 0x55df8d319a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.267+0000 7f56beffd640 1 -- 172.21.15.83:0/4047618133 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df8d2716a0 con 0x55df8d30a270 2024-03-11T19:41:48.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.267+0000 7f56beffd640 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d31b470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f56b0097230 tx=0x7f56b0098dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6be41848e85e1704 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:48.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.267+0000 7f569effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:41:48.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.267+0000 7f569effd640 1 -- 172.21.15.83:0/4047618133 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b009bde0 con 0x55df8d30a270 2024-03-11T19:41:48.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.267+0000 7f569effd640 1 -- 172.21.15.83:0/4047618133 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f56b00b8060 con 0x55df8d30a270 2024-03-11T19:41:48.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.267+0000 7f56c8624740 1 -- 172.21.15.83:0/4047618133 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56b0096b50 con 0x55df8d30a270 2024-03-11T19:41:48.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.268+0000 7f569effd640 1 -- 172.21.15.83:0/4047618133 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56b009ea70 con 0x55df8d30a270 2024-03-11T19:41:48.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.268+0000 7f56c8624740 10 client.25337 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:48.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.268+0000 7f56c8624740 1 -- 172.21.15.83:0/4047618133 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55df8d2338c0 con 0x55df8d30a270 2024-03-11T19:41:48.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.268+0000 7f569effd640 1 -- 172.21.15.83:0/4047618133 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55df8cf727e0 con 0x55df8d30a270 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.269+0000 7f569effd640 1 -- 172.21.15.83:0/4047618133 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f56b00a2bc0 con 0x55df8d30a270 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.269+0000 7f56c8624740 10 client.25337 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.269+0000 7f56c8624740 10 client.25337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.269+0000 7f56c8624740 20 client.25337 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.269+0000 7f56c8624740 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55df8d320070 0x55df8d379570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.269+0000 7f56c8624740 4 client.25337 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.269+0000 7f56c8624740 1 -- 172.21.15.83:0/4047618133 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f56b0096b50 con 0x55df8d320070 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.269+0000 7f56be7fc640 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55df8d320070 0x55df8d379570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.269+0000 7f569effd640 10 client.25337 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.270+0000 7f56be7fc640 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55df8d320070 0x55df8d379570 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ba5a42160e04cd9 server_cookie=928e0a608c60c344 in_seq=0 out_seq=0 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.271+0000 7f569effd640 1 -- 172.21.15.83:0/4047618133 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f56b0096b50 con 0x55df8d320070 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.271+0000 7f569effd640 10 client.25337 handle_command_reply: tid=0 2024-03-11T19:41:48.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.271+0000 7f56c8624740 1 client.25337 shutdown 2024-03-11T19:41:48.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f56c8624740 1 -- 172.21.15.83:0/4047618133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 msgr2=0x55df8d31b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:48.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f56c8624740 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d31b470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f56b0097230 tx=0x7f56b0098dc0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f56c8624740 1 -- 172.21.15.83:0/4047618133 shutdown_connections 2024-03-11T19:41:48.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f56c8624740 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55df8d320070 0x55df8d379570 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f56c8624740 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df8d30fdd0 0x55df8d319fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f56c8624740 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df8d30ac60 0x55df8d319a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f56c8624740 1 --2- 172.21.15.83:0/4047618133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df8d30a270 0x55df8d31b470 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f569effd640 0 client.25337 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:48.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f56c8624740 1 -- 172.21.15.83:0/4047618133 >> 172.21.15.83:0/4047618133 conn(0x55df8d3057e0 msgr2=0x55df8d30c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:48.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f56c8624740 1 -- 172.21.15.83:0/4047618133 shutdown_connections 2024-03-11T19:41:48.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f56c8624740 1 -- 172.21.15.83:0/4047618133 wait complete. 2024-03-11T19:41:48.279 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.272+0000 7f56c8624740 20 client.25337 trim_cache size 0 max 16384 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.274+0000 7f56c5a4e640 1 -- 172.21.15.83:0/2837707572 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56a80a1b80 msgr2=0x7f56a80a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.274+0000 7f56c5a4e640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56a80a1b80 0x7f56a80a4040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009270 tx=0x7f56b80095e0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.274+0000 7f56c5a4e640 1 -- 172.21.15.83:0/2837707572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100090 msgr2=0x7f56c006a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.274+0000 7f56c5a4e640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100090 0x7f56c006a280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f56b000e9f0 tx=0x7f56b000eeb0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.274+0000 7f56c5a4e640 1 -- 172.21.15.83:0/2837707572 shutdown_connections 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.274+0000 7f56c5a4e640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56a80a1b80 0x7f56a80a4040 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.274+0000 7f56c5a4e640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101c40 0x7f56c019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.274+0000 7f56c5a4e640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101280 0x7f56c0199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.274+0000 7f56c5a4e640 1 --2- 172.21.15.83:0/2837707572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100090 0x7f56c006a280 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.274+0000 7f56c5a4e640 1 -- 172.21.15.83:0/2837707572 >> 172.21.15.83:0/2837707572 conn(0x7f56c00fb620 msgr2=0x7f56c01045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:48.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.275+0000 7f56c5a4e640 1 -- 172.21.15.83:0/2837707572 shutdown_connections 2024-03-11T19:41:48.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:48.275+0000 7f56c5a4e640 1 -- 172.21.15.83:0/2837707572 wait complete. 2024-03-11T19:41:49.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:48 smithi083 ceph-mon[31864]: pgmap v427: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-03-11T19:41:49.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:48 smithi137 ceph-mon[36220]: pgmap v427: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-03-11T19:41:49.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:48 smithi204 ceph-mon[36739]: pgmap v427: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-03-11T19:41:49.300 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:41:49.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:41:49.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:41:49.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.466+0000 7f760faf1640 1 -- 172.21.15.83:0/1088923455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76081011a0 msgr2=0x7f76081015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.466+0000 7f760faf1640 1 --2- 172.21.15.83:0/1088923455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76081011a0 0x7f76081015c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7604009860 tx=0x7f760402bb90 comp rx=0 tx=0).stop 2024-03-11T19:41:49.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.466+0000 7f760faf1640 1 -- 172.21.15.83:0/1088923455 shutdown_connections 2024-03-11T19:41:49.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.466+0000 7f760faf1640 1 --2- 172.21.15.83:0/1088923455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7608102d50 0x7f76081095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.466+0000 7f760faf1640 1 --2- 172.21.15.83:0/1088923455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7608102390 0x7f7608102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.466+0000 7f760faf1640 1 --2- 172.21.15.83:0/1088923455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76081011a0 0x7f76081015c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.466+0000 7f760faf1640 1 -- 172.21.15.83:0/1088923455 >> 172.21.15.83:0/1088923455 conn(0x7f76080fc710 msgr2=0x7f76080feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.466+0000 7f760faf1640 1 -- 172.21.15.83:0/1088923455 shutdown_connections 2024-03-11T19:41:49.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.466+0000 7f760faf1640 1 -- 172.21.15.83:0/1088923455 wait complete. 2024-03-11T19:41:49.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.467+0000 7f760faf1640 1 Processor -- start 2024-03-11T19:41:49.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.467+0000 7f760faf1640 1 -- start start 2024-03-11T19:41:49.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.467+0000 7f760faf1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76081011a0 0x7f7608195050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.467+0000 7f760faf1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7608102390 0x7f7608195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.468+0000 7f760d88d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76081011a0 0x7f7608195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.468+0000 7f760d88d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76081011a0 0x7f7608195050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49276/0 (socket says 172.21.15.83:49276) 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.468+0000 7f760faf1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7608102d50 0x7f760819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.468+0000 7f760faf1640 1 -- 172.21.15.83:0/3328155400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760819cb40 con 0x7f76081011a0 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.468+0000 7f760faf1640 1 -- 172.21.15.83:0/3328155400 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760819ccb0 con 0x7f7608102390 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.468+0000 7f760faf1640 1 -- 172.21.15.83:0/3328155400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760819ce70 con 0x7f7608102d50 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.468+0000 7f760d08c640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7608102390 0x7f7608195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.468+0000 7f760e08e640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7608102d50 0x7f760819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.468+0000 7f760d88d640 1 -- 172.21.15.83:0/3328155400 learned_addr learned my addr 172.21.15.83:0/3328155400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.469+0000 7f760d08c640 1 -- 172.21.15.83:0/3328155400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7608102d50 msgr2=0x7f760819c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.469+0000 7f760d08c640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7608102d50 0x7f760819c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.469+0000 7f760d08c640 1 -- 172.21.15.83:0/3328155400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76081011a0 msgr2=0x7f7608195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.469+0000 7f760d08c640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76081011a0 0x7f7608195050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.469+0000 7f760d08c640 1 -- 172.21.15.83:0/3328155400 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7604009490 con 0x7f7608102390 2024-03-11T19:41:49.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.469+0000 7f760e08e640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7608102d50 0x7f760819c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:49.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.469+0000 7f760d08c640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7608102390 0x7f7608195590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f75f800d6d0 tx=0x7f75f800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.469+0000 7f75feffd640 1 -- 172.21.15.83:0/3328155400 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8009a20 con 0x7f7608102390 2024-03-11T19:41:49.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.469+0000 7f75feffd640 1 -- 172.21.15.83:0/3328155400 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75f8009b80 con 0x7f7608102390 2024-03-11T19:41:49.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.469+0000 7f75feffd640 1 -- 172.21.15.83:0/3328155400 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f800f620 con 0x7f7608102390 2024-03-11T19:41:49.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.470+0000 7f760faf1640 1 -- 172.21.15.83:0/3328155400 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760819d150 con 0x7f7608102390 2024-03-11T19:41:49.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.470+0000 7f760faf1640 1 -- 172.21.15.83:0/3328155400 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760819d610 con 0x7f7608102390 2024-03-11T19:41:49.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.473+0000 7f761279e740 1 Processor -- start 2024-03-11T19:41:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.474+0000 7f761279e740 1 -- start start 2024-03-11T19:41:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.474+0000 7f761279e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c4b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.474+0000 7f761279e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c4cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.474+0000 7f761279e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.474+0000 7f761279e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e70bbaf90 con 0x563e70c4b4c0 2024-03-11T19:41:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.474+0000 7f761279e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e70bb84c0 con 0x563e70c4d050 2024-03-11T19:41:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.474+0000 7f761279e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e70bb3ca0 con 0x563e70c4c690 2024-03-11T19:41:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.474+0000 7f760d88d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c4cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.475+0000 7f760e08e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c4b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.475+0000 7f760d88d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c4cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48412/0 (socket says 172.21.15.83:48412) 2024-03-11T19:41:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.475+0000 7f760e08e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c4b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49284/0 (socket says 172.21.15.83:49284) 2024-03-11T19:41:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.475+0000 7f760d88d640 1 -- 172.21.15.83:0/1997218723 learned_addr learned my addr 172.21.15.83:0/1997218723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:49.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.475+0000 7f760d88d640 1 -- 172.21.15.83:0/1997218723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 msgr2=0x563e70c538e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:41:49.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.475+0000 7f760d88d640 1 --2- 172.21.15.83:0/1997218723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c538e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.475+0000 7f760d88d640 1 -- 172.21.15.83:0/1997218723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 msgr2=0x563e70c4b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.475+0000 7f760d88d640 1 --2- 172.21.15.83:0/1997218723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c4b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.475+0000 7f760d88d640 1 -- 172.21.15.83:0/1997218723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e70b760b0 con 0x563e70c4c690 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.475+0000 7f760e08e640 1 --2- 172.21.15.83:0/1997218723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c4b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.475+0000 7f760d88d640 1 --2- 172.21.15.83:0/1997218723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c4cb10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f760402c0d0 tx=0x7f76040323d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35ad121d944d0f62 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.476+0000 7f75feffd640 1 -- 172.21.15.83:0/3328155400 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f75f8009cf0 con 0x7f7608102390 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.476+0000 7f75f3fff640 1 -- 172.21.15.83:0/1997218723 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604034440 con 0x563e70c4c690 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.476+0000 7f75f3fff640 1 -- 172.21.15.83:0/1997218723 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7604004510 con 0x563e70c4c690 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.476+0000 7f75f3fff640 1 -- 172.21.15.83:0/1997218723 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760402c530 con 0x563e70c4c690 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.476+0000 7f75feffd640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75f40a1c00 0x7f75f40a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.477+0000 7f761279e740 1 -- 172.21.15.83:0/1997218723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 msgr2=0x563e70c4cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.477+0000 7f761279e740 1 --2- 172.21.15.83:0/1997218723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c4cb10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f760402c0d0 tx=0x7f76040323d0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.477+0000 7f760e08e640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75f40a1c00 0x7f75f40a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.477+0000 7f761279e740 1 -- 172.21.15.83:0/1997218723 shutdown_connections 2024-03-11T19:41:49.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.477+0000 7f75feffd640 1 -- 172.21.15.83:0/3328155400 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f75f80cd0e0 con 0x7f7608102390 2024-03-11T19:41:49.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.477+0000 7f761279e740 1 --2- 172.21.15.83:0/1997218723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.477+0000 7f761279e740 1 --2- 172.21.15.83:0/1997218723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c4cb10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.477+0000 7f761279e740 1 --2- 172.21.15.83:0/1997218723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c4b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.477+0000 7f761279e740 1 -- 172.21.15.83:0/1997218723 >> 172.21.15.83:0/1997218723 conn(0x563e70c466a0 msgr2=0x563e70c46aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.478+0000 7f760e08e640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75f40a1c00 0x7f75f40a40c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7600003c20 tx=0x7f760000f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.478+0000 7f761279e740 1 -- 172.21.15.83:0/1997218723 shutdown_connections 2024-03-11T19:41:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.478+0000 7f761279e740 1 -- 172.21.15.83:0/1997218723 wait complete. 2024-03-11T19:41:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.478+0000 7f761279e740 1 -- start start 2024-03-11T19:41:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.479+0000 7f761279e740 1 Processor -- start 2024-03-11T19:41:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.479+0000 7f761279e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c66700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.479+0000 7f761279e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c66c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.479+0000 7f761279e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c6dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.479+0000 7f761279e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e70b58f40 con 0x563e70c4b4c0 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.479+0000 7f761279e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e708874b0 con 0x563e70c4d050 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.479+0000 7f761279e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e70b39670 con 0x563e70c4c690 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.479+0000 7f761279e740 10 client.0 _ll_register_callbacks cb 0x563e70ba6f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760e08e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c6dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760e08e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c6dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:33144/0 (socket says 172.21.15.83:33144) 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760e08e640 1 -- 172.21.15.83:0/2523630830 learned_addr learned my addr 172.21.15.83:0/2523630830 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760d88d640 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c66700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760d08c640 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c66c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760d88d640 1 -- 172.21.15.83:0/2523630830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 msgr2=0x563e70c66c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760d88d640 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c66c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760d88d640 1 -- 172.21.15.83:0/2523630830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 msgr2=0x563e70c6dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760d88d640 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c6dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760d88d640 1 -- 172.21.15.83:0/2523630830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7608109ee0 con 0x563e70c4b4c0 2024-03-11T19:41:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760e08e640 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c6dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:41:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.480+0000 7f760d88d640 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c66700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f760402c1a0 tx=0x7f7604034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1bdc9f3ff6d9f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.481+0000 7f75f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:41:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.481+0000 7f75f17fa640 1 -- 172.21.15.83:0/2523630830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604004450 con 0x563e70c4b4c0 2024-03-11T19:41:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.481+0000 7f75f17fa640 1 -- 172.21.15.83:0/2523630830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7604035e30 con 0x563e70c4b4c0 2024-03-11T19:41:49.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.481+0000 7f761279e740 1 -- 172.21.15.83:0/2523630830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e70b8df40 con 0x563e70c4b4c0 2024-03-11T19:41:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.481+0000 7f75f17fa640 1 -- 172.21.15.83:0/2523630830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f760404a070 con 0x563e70c4b4c0 2024-03-11T19:41:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.481+0000 7f761279e740 10 client.15849 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.481+0000 7f761279e740 1 -- 172.21.15.83:0/2523630830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563e70bb2320 con 0x563e70c4b4c0 2024-03-11T19:41:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.483+0000 7f75f17fa640 1 -- 172.21.15.83:0/2523630830 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563e70b8df40 con 0x563e70c4b4c0 2024-03-11T19:41:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.484+0000 7f75f17fa640 1 -- 172.21.15.83:0/2523630830 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7604003bd0 con 0x563e70c4b4c0 2024-03-11T19:41:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.484+0000 7f761279e740 10 client.15849 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.484+0000 7f761279e740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.484+0000 7f761279e740 20 client.15849 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.484+0000 7f761279e740 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563e70c72670 0x563e70c74a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.484+0000 7f761279e740 4 client.15849 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.484+0000 7f761279e740 1 -- 172.21.15.83:0/2523630830 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e70baa310 con 0x563e70c72670 2024-03-11T19:41:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.484+0000 7f760d08c640 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563e70c72670 0x563e70c74a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.485+0000 7f760d08c640 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563e70c72670 0x563e70c74a90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16fa72eccd34d8bb server_cookie=c59ad383d900c9b1 in_seq=0 out_seq=0 2024-03-11T19:41:49.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.485+0000 7f75f17fa640 10 client.15849 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.486+0000 7f75f17fa640 1 -- 172.21.15.83:0/2523630830 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563e70baa310 con 0x563e70c72670 2024-03-11T19:41:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.486+0000 7f75f17fa640 10 client.15849 handle_command_reply: tid=0 2024-03-11T19:41:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.487+0000 7f761279e740 1 client.15849 shutdown 2024-03-11T19:41:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.487+0000 7f761279e740 1 -- 172.21.15.83:0/2523630830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 msgr2=0x563e70c66700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.487+0000 7f761279e740 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c66700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f760402c1a0 tx=0x7f7604034670 comp rx=0 tx=0).stop 2024-03-11T19:41:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.487+0000 7f761279e740 1 -- 172.21.15.83:0/2523630830 shutdown_connections 2024-03-11T19:41:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.487+0000 7f761279e740 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563e70c72670 0x563e70c74a90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.487+0000 7f761279e740 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c6dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.487+0000 7f761279e740 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c66c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.487+0000 7f761279e740 1 --2- 172.21.15.83:0/2523630830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c66700 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.487+0000 7f761279e740 1 -- 172.21.15.83:0/2523630830 >> 172.21.15.83:0/2523630830 conn(0x563e70c466a0 msgr2=0x563e70c458e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.487+0000 7f75f17fa640 0 client.15849 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.488+0000 7f761279e740 1 -- 172.21.15.83:0/2523630830 shutdown_connections 2024-03-11T19:41:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.488+0000 7f761279e740 1 -- 172.21.15.83:0/2523630830 wait complete. 2024-03-11T19:41:49.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.488+0000 7f761279e740 20 client.15849 trim_cache size 0 max 16384 2024-03-11T19:41:49.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.514+0000 7f761279e740 1 Processor -- start 2024-03-11T19:41:49.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.514+0000 7f761279e740 1 -- start start 2024-03-11T19:41:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.515+0000 7f761279e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c65d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.515+0000 7f761279e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c6dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.515+0000 7f761279e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c66260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.515+0000 7f761279e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e70c4c190 con 0x563e70c4b4c0 2024-03-11T19:41:49.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.515+0000 7f761279e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e70c79a80 con 0x563e70c4d050 2024-03-11T19:41:49.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.515+0000 7f761279e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e70baa310 con 0x563e70c4c690 2024-03-11T19:41:49.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.516+0000 7f760e08e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c66260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.516+0000 7f760d08c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c6dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.516+0000 7f760e08e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c66260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:57324/0 (socket says 172.21.15.83:57324) 2024-03-11T19:41:49.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.516+0000 7f760e08e640 1 -- 172.21.15.83:0/149012376 learned_addr learned my addr 172.21.15.83:0/149012376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.516+0000 7f760d88d640 1 --2- 172.21.15.83:0/149012376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c65d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.516+0000 7f760d08c640 1 -- 172.21.15.83:0/149012376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 msgr2=0x563e70c66260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.516+0000 7f760d08c640 1 --2- 172.21.15.83:0/149012376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c66260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.516+0000 7f760d08c640 1 -- 172.21.15.83:0/149012376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 msgr2=0x563e70c65d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.516+0000 7f760d08c640 1 --2- 172.21.15.83:0/149012376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c65d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.516+0000 7f760d08c640 1 -- 172.21.15.83:0/149012376 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e70b4ec00 con 0x563e70c4c690 2024-03-11T19:41:49.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.517+0000 7f760d08c640 1 --2- 172.21.15.83:0/149012376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c6dc40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009fd0 tx=0x7f75f80caac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a210cb0970db1a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.518+0000 7f75f37fe640 1 -- 172.21.15.83:0/149012376 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8098410 con 0x563e70c4c690 2024-03-11T19:41:49.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.518+0000 7f75f37fe640 1 -- 172.21.15.83:0/149012376 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75f80cad00 con 0x563e70c4c690 2024-03-11T19:41:49.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.518+0000 7f75f37fe640 1 -- 172.21.15.83:0/149012376 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8098fd0 con 0x563e70c4c690 2024-03-11T19:41:49.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.519+0000 7f761279e740 1 -- 172.21.15.83:0/149012376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 msgr2=0x563e70c6dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.519+0000 7f761279e740 1 --2- 172.21.15.83:0/149012376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c6dc40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009fd0 tx=0x7f75f80caac0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.519+0000 7f761279e740 1 -- 172.21.15.83:0/149012376 shutdown_connections 2024-03-11T19:41:49.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.519+0000 7f761279e740 1 --2- 172.21.15.83:0/149012376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4d050 0x563e70c66260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.519+0000 7f761279e740 1 --2- 172.21.15.83:0/149012376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4c690 0x563e70c6dc40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.519+0000 7f761279e740 1 --2- 172.21.15.83:0/149012376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4b4c0 0x563e70c65d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.519+0000 7f761279e740 1 -- 172.21.15.83:0/149012376 >> 172.21.15.83:0/149012376 conn(0x563e70c466a0 msgr2=0x563e70c50250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.519+0000 7f761279e740 1 -- 172.21.15.83:0/149012376 shutdown_connections 2024-03-11T19:41:49.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.520+0000 7f761279e740 1 -- 172.21.15.83:0/149012376 wait complete. 2024-03-11T19:41:49.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.520+0000 7f761279e740 1 -- start start 2024-03-11T19:41:49.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.520+0000 7f761279e740 1 Processor -- start 2024-03-11T19:41:49.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.520+0000 7f761279e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4b4c0 0x563e70c69e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.521+0000 7f761279e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4c690 0x563e70c6a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.521+0000 7f760d88d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4b4c0 0x563e70c69e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.521+0000 7f760d88d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4b4c0 0x563e70c69e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:57340/0 (socket says 172.21.15.83:57340) 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.521+0000 7f760d88d640 1 -- 172.21.15.83:0/1814399280 learned_addr learned my addr 172.21.15.83:0/1814399280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.521+0000 7f760d08c640 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4c690 0x563e70c6a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.521+0000 7f761279e740 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4d050 0x563e70c6d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.521+0000 7f761279e740 1 -- 172.21.15.83:0/1814399280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e70c6d8d0 con 0x563e70c4c690 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.521+0000 7f761279e740 1 -- 172.21.15.83:0/1814399280 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e70c6da40 con 0x563e70c4b4c0 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.521+0000 7f761279e740 1 -- 172.21.15.83:0/1814399280 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e70c6dc00 con 0x563e70c4d050 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.521+0000 7f760e08e640 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4d050 0x563e70c6d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.522+0000 7f761279e740 10 client.0 _ll_register_callbacks cb 0x7f760000d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:49.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.522+0000 7f760e08e640 1 -- 172.21.15.83:0/1814399280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4b4c0 msgr2=0x563e70c69e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.522+0000 7f760e08e640 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4b4c0 0x563e70c69e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.522+0000 7f760e08e640 1 -- 172.21.15.83:0/1814399280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4c690 msgr2=0x563e70c6a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.522+0000 7f760e08e640 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4c690 0x563e70c6a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.522+0000 7f760e08e640 1 -- 172.21.15.83:0/1814399280 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760819d610 con 0x563e70c4d050 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.522+0000 7f760e08e640 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4d050 0x563e70c6d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7600005fd0 tx=0x7f7600004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be1f98249fffb9e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.522+0000 7f75f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.522+0000 7f75f17fa640 1 -- 172.21.15.83:0/1814399280 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600012040 con 0x563e70c4d050 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.522+0000 7f761279e740 1 -- 172.21.15.83:0/1814399280 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e70bd9900 con 0x563e70c4d050 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.522+0000 7f75f17fa640 1 -- 172.21.15.83:0/1814399280 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7600018d10 con 0x563e70c4d050 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.523+0000 7f75f17fa640 1 -- 172.21.15.83:0/1814399280 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7600036030 con 0x563e70c4d050 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.523+0000 7f761279e740 10 client.25465 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.523+0000 7f761279e740 1 -- 172.21.15.83:0/1814399280 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563e70b74910 con 0x563e70c4d050 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.523+0000 7f75f17fa640 1 -- 172.21.15.83:0/1814399280 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563e70bd9900 con 0x563e70c4d050 2024-03-11T19:41:49.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.523+0000 7f75f17fa640 1 -- 172.21.15.83:0/1814399280 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f760002e020 con 0x563e70c4d050 2024-03-11T19:41:49.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.524+0000 7f761279e740 10 client.25465 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:49.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.524+0000 7f761279e740 10 client.25465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:49.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.524+0000 7f761279e740 20 client.25465 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:49.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.524+0000 7f761279e740 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563e70cca090 0x563e70ccc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.524+0000 7f761279e740 4 client.25465 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:41:49.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.524+0000 7f761279e740 1 -- 172.21.15.83:0/1814399280 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563e708b3860 con 0x563e70cca090 2024-03-11T19:41:49.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.524+0000 7f760d88d640 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563e70cca090 0x563e70ccc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.525+0000 7f760d88d640 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563e70cca090 0x563e70ccc4b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c810c10a67b4357 server_cookie=4240318affe9bdbc in_seq=0 out_seq=0 2024-03-11T19:41:49.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.525+0000 7f75f17fa640 10 client.25465 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.526+0000 7f75f17fa640 1 -- 172.21.15.83:0/1814399280 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563e708b3860 con 0x563e70cca090 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.526+0000 7f75f17fa640 10 client.25465 handle_command_reply: tid=0 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.526+0000 7f761279e740 1 client.25465 shutdown 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.526+0000 7f761279e740 1 -- 172.21.15.83:0/1814399280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4d050 msgr2=0x563e70c6d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.526+0000 7f761279e740 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4d050 0x563e70c6d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7600005fd0 tx=0x7f7600004ab0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.527+0000 7f761279e740 1 -- 172.21.15.83:0/1814399280 shutdown_connections 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.527+0000 7f761279e740 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563e70cca090 0x563e70ccc4b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.527+0000 7f761279e740 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e70c4d050 0x563e70c6d390 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.527+0000 7f761279e740 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e70c4c690 0x563e70c6a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.527+0000 7f75f17fa640 0 client.25465 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.527+0000 7f761279e740 1 --2- 172.21.15.83:0/1814399280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563e70c4b4c0 0x563e70c69e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.527+0000 7f761279e740 1 -- 172.21.15.83:0/1814399280 >> 172.21.15.83:0/1814399280 conn(0x563e70c466a0 msgr2=0x563e70c4a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.527+0000 7f761279e740 1 -- 172.21.15.83:0/1814399280 shutdown_connections 2024-03-11T19:41:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.527+0000 7f761279e740 1 -- 172.21.15.83:0/1814399280 wait complete. 2024-03-11T19:41:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.527+0000 7f761279e740 20 client.25465 trim_cache size 0 max 16384 2024-03-11T19:41:49.533 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:41:49.533 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:41:49.533 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "df38e5eb-b083-414b-bed1-ac4812650ab8", 2024-03-11T19:41:49.533 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:41:49.533 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:41:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.528+0000 7f760faf1640 1 -- 172.21.15.83:0/3328155400 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75f40a1c00 msgr2=0x7f75f40a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.528+0000 7f760faf1640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75f40a1c00 0x7f75f40a40c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7600003c20 tx=0x7f760000f040 comp rx=0 tx=0).stop 2024-03-11T19:41:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.528+0000 7f760faf1640 1 -- 172.21.15.83:0/3328155400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7608102390 msgr2=0x7f7608195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.528+0000 7f760faf1640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7608102390 0x7f7608195590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f75f800d6d0 tx=0x7f75f800db90 comp rx=0 tx=0).stop 2024-03-11T19:41:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.529+0000 7f760faf1640 1 -- 172.21.15.83:0/3328155400 shutdown_connections 2024-03-11T19:41:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.529+0000 7f760faf1640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75f40a1c00 0x7f75f40a40c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.529+0000 7f760faf1640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7608102d50 0x7f760819c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.529+0000 7f760faf1640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7608102390 0x7f7608195590 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.529+0000 7f760faf1640 1 --2- 172.21.15.83:0/3328155400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76081011a0 0x7f7608195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.529+0000 7f760faf1640 1 -- 172.21.15.83:0/3328155400 >> 172.21.15.83:0/3328155400 conn(0x7f76080fc710 msgr2=0x7f7608105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.529+0000 7f760faf1640 1 -- 172.21.15.83:0/3328155400 shutdown_connections 2024-03-11T19:41:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.529+0000 7f760faf1640 1 -- 172.21.15.83:0/3328155400 wait complete. 2024-03-11T19:41:49.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:41:49.552 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:41:49.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.716+0000 7fd83adc7640 1 -- 172.21.15.83:0/4176824566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd834102fb0 msgr2=0x7fd834103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.716+0000 7fd83adc7640 1 --2- 172.21.15.83:0/4176824566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd834102fb0 0x7fd834103430 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd820009900 tx=0x7fd82002bc40 comp rx=0 tx=0).stop 2024-03-11T19:41:49.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.718+0000 7fd83adc7640 1 -- 172.21.15.83:0/4176824566 shutdown_connections 2024-03-11T19:41:49.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.718+0000 7fd83adc7640 1 --2- 172.21.15.83:0/4176824566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834103970 0x7fd83410b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.718+0000 7fd83adc7640 1 --2- 172.21.15.83:0/4176824566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd834102fb0 0x7fd834103430 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.718+0000 7fd83adc7640 1 --2- 172.21.15.83:0/4176824566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd834104a80 0x7fd834104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.718+0000 7fd83adc7640 1 -- 172.21.15.83:0/4176824566 >> 172.21.15.83:0/4176824566 conn(0x7fd8340fe930 msgr2=0x7fd834100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.719+0000 7fd83adc7640 1 -- 172.21.15.83:0/4176824566 shutdown_connections 2024-03-11T19:41:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.720+0000 7fd83adc7640 1 -- 172.21.15.83:0/4176824566 wait complete. 2024-03-11T19:41:49.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.721+0000 7fd83adc7640 1 Processor -- start 2024-03-11T19:41:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.721+0000 7fd83adc7640 1 -- start start 2024-03-11T19:41:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.722+0000 7fd83adc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834102fb0 0x7fd834078f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.722+0000 7fd83adc7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd834103970 0x7fd834079470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.722+0000 7fd83adc7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd834104a80 0x7fd834075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.722+0000 7fd83adc7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd834076230 con 0x7fd834104a80 2024-03-11T19:41:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.722+0000 7fd83adc7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8340763a0 con 0x7fd834102fb0 2024-03-11T19:41:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.722+0000 7fd83adc7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd834076560 con 0x7fd834103970 2024-03-11T19:41:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.722+0000 7fd839364640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd834104a80 0x7fd834075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.722+0000 7fd839364640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd834104a80 0x7fd834075aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44660/0 (socket says 172.21.15.83:44660) 2024-03-11T19:41:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.722+0000 7fd839364640 1 -- 172.21.15.83:0/1980277445 learned_addr learned my addr 172.21.15.83:0/1980277445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.722+0000 7fd838b63640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834102fb0 0x7fd834078f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.722+0000 7fd833fff640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd834103970 0x7fd834079470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.723+0000 7fd839364640 1 -- 172.21.15.83:0/1980277445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd834103970 msgr2=0x7fd834079470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.723+0000 7fd839364640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd834103970 0x7fd834079470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.723+0000 7fd839364640 1 -- 172.21.15.83:0/1980277445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834102fb0 msgr2=0x7fd834078f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.723+0000 7fd839364640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834102fb0 0x7fd834078f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.723+0000 7fd839364640 1 -- 172.21.15.83:0/1980277445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd820009560 con 0x7fd834104a80 2024-03-11T19:41:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.724+0000 7fd839364640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd834104a80 0x7fd834075aa0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00eb10 tx=0x7fd82c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.724+0000 7fd831ffb640 1 -- 172.21.15.83:0/1980277445 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c004b20 con 0x7fd834104a80 2024-03-11T19:41:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.724+0000 7fd831ffb640 1 -- 172.21.15.83:0/1980277445 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd82c004c80 con 0x7fd834104a80 2024-03-11T19:41:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.724+0000 7fd831ffb640 1 -- 172.21.15.83:0/1980277445 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c010650 con 0x7fd834104a80 2024-03-11T19:41:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.724+0000 7fd83adc7640 1 -- 172.21.15.83:0/1980277445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd834199170 con 0x7fd834104a80 2024-03-11T19:41:49.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.725+0000 7fd83adc7640 1 -- 172.21.15.83:0/1980277445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd834199630 con 0x7fd834104a80 2024-03-11T19:41:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.728+0000 7fd831ffb640 1 -- 172.21.15.83:0/1980277445 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd82c005020 con 0x7fd834104a80 2024-03-11T19:41:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.729+0000 7fd83d997740 1 Processor -- start 2024-03-11T19:41:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.729+0000 7fd83d997740 1 -- start start 2024-03-11T19:41:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.729+0000 7fd831ffb640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd8240a1b30 0x7fd8240a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.729+0000 7fd83d997740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 0x55c122db56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd83d997740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122db5cc0 0x55c122dba7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd83d997740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122dbae30 0x55c122dbd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd83d997740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c122d25280 con 0x55c122db52d0 2024-03-11T19:41:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd83d997740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c122d227b0 con 0x55c122db5cc0 2024-03-11T19:41:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd83d997740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c122d1df90 con 0x55c122dbae30 2024-03-11T19:41:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd838b63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 0x55c122db56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd838b63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 0x55c122db56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44676/0 (socket says 172.21.15.83:44676) 2024-03-11T19:41:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd838b63640 1 -- 172.21.15.83:0/2429174694 learned_addr learned my addr 172.21.15.83:0/2429174694 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd831ffb640 1 -- 172.21.15.83:0/1980277445 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd82c014070 con 0x7fd834104a80 2024-03-11T19:41:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd833fff640 1 --2- 172.21.15.83:0/2429174694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122db5cc0 0x55c122dba7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd839364640 1 --2- 172.21.15.83:0/2429174694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122dbae30 0x55c122dbd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd833fff640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd8240a1b30 0x7fd8240a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd833fff640 1 -- 172.21.15.83:0/2429174694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122dbae30 msgr2=0x55c122dbd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd833fff640 1 --2- 172.21.15.83:0/2429174694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122dbae30 0x55c122dbd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd833fff640 1 -- 172.21.15.83:0/2429174694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 msgr2=0x55c122db56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd833fff640 1 --2- 172.21.15.83:0/2429174694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 0x55c122db56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.730+0000 7fd833fff640 1 -- 172.21.15.83:0/2429174694 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c122cdc570 con 0x55c122db5cc0 2024-03-11T19:41:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.731+0000 7fd833fff640 1 --2- 172.21.15.83:0/2429174694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122db5cc0 0x55c122dba7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd820009230 tx=0x7fd820004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150774b5e4e135d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.731+0000 7fd81effd640 1 -- 172.21.15.83:0/2429174694 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820004040 con 0x55c122db5cc0 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.731+0000 7fd81effd640 1 -- 172.21.15.83:0/2429174694 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd820004df0 con 0x55c122db5cc0 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.731+0000 7fd81effd640 1 -- 172.21.15.83:0/2429174694 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820003850 con 0x55c122db5cc0 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.732+0000 7fd83d997740 1 -- 172.21.15.83:0/2429174694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122db5cc0 msgr2=0x55c122dba7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.732+0000 7fd83d997740 1 --2- 172.21.15.83:0/2429174694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122db5cc0 0x55c122dba7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd820009230 tx=0x7fd820004740 comp rx=0 tx=0).stop 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.732+0000 7fd833fff640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd8240a1b30 0x7fd8240a3ff0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd820004c50 tx=0x7fd8200039b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.732+0000 7fd83d997740 1 -- 172.21.15.83:0/2429174694 shutdown_connections 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.732+0000 7fd83d997740 1 --2- 172.21.15.83:0/2429174694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122dbae30 0x55c122dbd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.732+0000 7fd83d997740 1 --2- 172.21.15.83:0/2429174694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122db5cc0 0x55c122dba7c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.732+0000 7fd83d997740 1 --2- 172.21.15.83:0/2429174694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 0x55c122db56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.732+0000 7fd83d997740 1 -- 172.21.15.83:0/2429174694 >> 172.21.15.83:0/2429174694 conn(0x55c122db07f0 msgr2=0x55c122db0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.732+0000 7fd83d997740 1 -- 172.21.15.83:0/2429174694 shutdown_connections 2024-03-11T19:41:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.732+0000 7fd83d997740 1 -- 172.21.15.83:0/2429174694 wait complete. 2024-03-11T19:41:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.732+0000 7fd83d997740 1 -- start start 2024-03-11T19:41:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.733+0000 7fd83d997740 1 Processor -- start 2024-03-11T19:41:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.734+0000 7fd83d997740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 0x55c122dd0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.734+0000 7fd83d997740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db5cc0 0x55c122dd0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.734+0000 7fd83d997740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 0x55c122dd7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.734+0000 7fd83d997740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1229f1430 con 0x55c122db52d0 2024-03-11T19:41:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.734+0000 7fd833fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db5cc0 0x55c122dd0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd838b63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 0x55c122dd0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd83d997740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1229ade00 con 0x55c122dbae30 2024-03-11T19:41:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd83d997740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c122ca2fd0 con 0x55c122db5cc0 2024-03-11T19:41:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd839364640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 0x55c122dd7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd833fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db5cc0 0x55c122dd0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47276/0 (socket says 172.21.15.83:47276) 2024-03-11T19:41:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd833fff640 1 -- 172.21.15.83:0/1055493660 learned_addr learned my addr 172.21.15.83:0/1055493660 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd838b63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 0x55c122dd0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44690/0 (socket says 172.21.15.83:44690) 2024-03-11T19:41:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd83d997740 10 client.0 _ll_register_callbacks cb 0x55c122cf8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:49.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd839364640 1 -- 172.21.15.83:0/1055493660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db5cc0 msgr2=0x55c122dd0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd839364640 1 --2- 172.21.15.83:0/1055493660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db5cc0 0x55c122dd0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd839364640 1 -- 172.21.15.83:0/1055493660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 msgr2=0x55c122dd0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd839364640 1 --2- 172.21.15.83:0/1055493660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 0x55c122dd0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd839364640 1 -- 172.21.15.83:0/1055493660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83410c1b0 con 0x55c122dbae30 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd839364640 1 --2- 172.21.15.83:0/1055493660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 0x55c122dd7890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd82c096ed0 tx=0x7fd82c09ab30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1f40859f10ab04 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd813fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd813fff640 1 -- 172.21.15.83:0/1055493660 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c098d70 con 0x55c122dbae30 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd813fff640 1 -- 172.21.15.83:0/1055493660 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd82c016df0 con 0x55c122dbae30 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.735+0000 7fd83d997740 1 -- 172.21.15.83:0/1055493660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c122a1d7e0 con 0x55c122dbae30 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.736+0000 7fd813fff640 1 -- 172.21.15.83:0/1055493660 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd82c0cb710 con 0x55c122dbae30 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.736+0000 7fd83d997740 10 client.25364 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.736+0000 7fd83d997740 1 -- 172.21.15.83:0/1055493660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c122d1c610 con 0x55c122dbae30 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.736+0000 7fd813fff640 1 -- 172.21.15.83:0/1055493660 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd82c07d020 con 0x55c122dbae30 2024-03-11T19:41:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.736+0000 7fd813fff640 1 -- 172.21.15.83:0/1055493660 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd82c01f060 con 0x55c122dbae30 2024-03-11T19:41:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.736+0000 7fd83d997740 10 client.25364 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.736+0000 7fd83d997740 10 client.25364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.736+0000 7fd83d997740 20 client.25364 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.736+0000 7fd83d997740 1 --2- 172.21.15.83:0/1055493660 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c122ddc580 0x55c122dde9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.736+0000 7fd83d997740 4 client.25364 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:41:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.736+0000 7fd83d997740 1 -- 172.21.15.83:0/1055493660 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c122a5a720 con 0x55c122ddc580 2024-03-11T19:41:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.737+0000 7fd838b63640 1 --2- 172.21.15.83:0/1055493660 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c122ddc580 0x55c122dde9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.738+0000 7fd838b63640 1 --2- 172.21.15.83:0/1055493660 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c122ddc580 0x55c122dde9a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa79283dc69fd654 server_cookie=c78ef2e655a145ab in_seq=0 out_seq=0 2024-03-11T19:41:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.738+0000 7fd813fff640 10 client.25364 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.739+0000 7fd813fff640 1 -- 172.21.15.83:0/1055493660 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c122a5a720 con 0x55c122ddc580 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.739+0000 7fd813fff640 10 client.25364 handle_command_reply: tid=0 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 1 client.25364 shutdown 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 1 -- 172.21.15.83:0/1055493660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 msgr2=0x55c122dd7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 1 --2- 172.21.15.83:0/1055493660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 0x55c122dd7890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd82c096ed0 tx=0x7fd82c09ab30 comp rx=0 tx=0).stop 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 1 -- 172.21.15.83:0/1055493660 shutdown_connections 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 1 --2- 172.21.15.83:0/1055493660 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c122ddc580 0x55c122dde9a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 1 --2- 172.21.15.83:0/1055493660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 0x55c122dd7890 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd813fff640 0 client.25364 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 1 --2- 172.21.15.83:0/1055493660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db5cc0 0x55c122dd0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 1 --2- 172.21.15.83:0/1055493660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db52d0 0x55c122dd0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 1 -- 172.21.15.83:0/1055493660 >> 172.21.15.83:0/1055493660 conn(0x55c122db07f0 msgr2=0x55c122dafa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 1 -- 172.21.15.83:0/1055493660 shutdown_connections 2024-03-11T19:41:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 1 -- 172.21.15.83:0/1055493660 wait complete. 2024-03-11T19:41:49.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.740+0000 7fd83d997740 20 client.25364 trim_cache size 0 max 16384 2024-03-11T19:41:49.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.769+0000 7fd83d997740 1 Processor -- start 2024-03-11T19:41:49.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.770+0000 7fd83d997740 1 -- start start 2024-03-11T19:41:49.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.770+0000 7fd83d997740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db52d0 0x55c122db6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.770+0000 7fd83d997740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db5cc0 0x55c122db7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.770+0000 7fd83d997740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 0x55c122dd7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.770+0000 7fd83d997740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c122a5a720 con 0x55c122db5cc0 2024-03-11T19:41:49.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.771+0000 7fd83d997740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c122db63b0 con 0x55c122dbae30 2024-03-11T19:41:49.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.771+0000 7fd83d997740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c122dd1490 con 0x55c122db52d0 2024-03-11T19:41:49.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.771+0000 7fd839364640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 0x55c122dd7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.771+0000 7fd839364640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 0x55c122dd7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:57386/0 (socket says 172.21.15.83:57386) 2024-03-11T19:41:49.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.771+0000 7fd839364640 1 -- 172.21.15.83:0/989217609 learned_addr learned my addr 172.21.15.83:0/989217609 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:49.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.771+0000 7fd839364640 1 -- 172.21.15.83:0/989217609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db52d0 msgr2=0x55c122db6fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:41:49.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.771+0000 7fd839364640 1 --2- 172.21.15.83:0/989217609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db52d0 0x55c122db6fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.771+0000 7fd839364640 1 -- 172.21.15.83:0/989217609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db5cc0 msgr2=0x55c122db7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.771+0000 7fd839364640 1 --2- 172.21.15.83:0/989217609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db5cc0 0x55c122db7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.771+0000 7fd839364640 1 -- 172.21.15.83:0/989217609 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c122cb8ac0 con 0x55c122dbae30 2024-03-11T19:41:49.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.772+0000 7fd839364640 1 --2- 172.21.15.83:0/989217609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 0x55c122dd7850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0b8040 tx=0x7fd82c09a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd56a654b2580ba server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.773+0000 7fd81e7fc640 1 -- 172.21.15.83:0/989217609 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c09b170 con 0x55c122dbae30 2024-03-11T19:41:49.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.773+0000 7fd81e7fc640 1 -- 172.21.15.83:0/989217609 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd82c07d070 con 0x55c122dbae30 2024-03-11T19:41:49.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.773+0000 7fd81e7fc640 1 -- 172.21.15.83:0/989217609 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0967c0 con 0x55c122dbae30 2024-03-11T19:41:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.774+0000 7fd83d997740 1 -- 172.21.15.83:0/989217609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 msgr2=0x55c122dd7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.774+0000 7fd83d997740 1 --2- 172.21.15.83:0/989217609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 0x55c122dd7850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0b8040 tx=0x7fd82c09a5b0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.774+0000 7fd83d997740 1 -- 172.21.15.83:0/989217609 shutdown_connections 2024-03-11T19:41:49.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.774+0000 7fd83d997740 1 --2- 172.21.15.83:0/989217609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122dbae30 0x55c122dd7850 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.774+0000 7fd83d997740 1 --2- 172.21.15.83:0/989217609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122db5cc0 0x55c122db7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.774+0000 7fd83d997740 1 --2- 172.21.15.83:0/989217609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db52d0 0x55c122db6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.774+0000 7fd83d997740 1 -- 172.21.15.83:0/989217609 >> 172.21.15.83:0/989217609 conn(0x55c122db07f0 msgr2=0x55c122de3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.775+0000 7fd83d997740 1 -- 172.21.15.83:0/989217609 shutdown_connections 2024-03-11T19:41:49.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.775+0000 7fd83d997740 1 -- 172.21.15.83:0/989217609 wait complete. 2024-03-11T19:41:49.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.775+0000 7fd83d997740 1 -- start start 2024-03-11T19:41:49.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.775+0000 7fd83d997740 1 Processor -- start 2024-03-11T19:41:49.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd83d997740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db52d0 0x55c122dd5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd83d997740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122db5cc0 0x55c122dd64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd83d997740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122dbae30 0x55c122dd69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd83d997740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c122dd7bb0 con 0x55c122dbae30 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd83d997740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c122e32ab0 con 0x55c122db5cc0 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd83d997740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c122e32c20 con 0x55c122db52d0 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd83d997740 10 client.0 _ll_register_callbacks cb 0x7fd82c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd838b63640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db52d0 0x55c122dd5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd838b63640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db52d0 0x55c122dd5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47284/0 (socket says 172.21.15.83:47284) 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd838b63640 1 -- 172.21.15.83:0/4085865611 learned_addr learned my addr 172.21.15.83:0/4085865611 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd839364640 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122dbae30 0x55c122dd69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.776+0000 7fd833fff640 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122db5cc0 0x55c122dd64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd839364640 1 -- 172.21.15.83:0/4085865611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db52d0 msgr2=0x55c122dd5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd839364640 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db52d0 0x55c122dd5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd839364640 1 -- 172.21.15.83:0/4085865611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122db5cc0 msgr2=0x55c122dd64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd839364640 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122db5cc0 0x55c122dd64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd839364640 1 -- 172.21.15.83:0/4085865611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c122d1c610 con 0x55c122dbae30 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd839364640 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122dbae30 0x55c122dd69e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd82c096ed0 tx=0x7fd82c09ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0e123923657413 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd813fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd813fff640 1 -- 172.21.15.83:0/4085865611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0a2ed0 con 0x55c122dbae30 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd83d997740 1 -- 172.21.15.83:0/4085865611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd820004df0 con 0x55c122dbae30 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd813fff640 1 -- 172.21.15.83:0/4085865611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd82c07e080 con 0x55c122dbae30 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd813fff640 1 -- 172.21.15.83:0/4085865611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd82c09b170 con 0x55c122dbae30 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd83d997740 10 client.15870 fetch_fsmap learned FSMap version 25 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd83d997740 1 -- 172.21.15.83:0/4085865611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c122cde6a0 con 0x55c122dbae30 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.777+0000 7fd813fff640 1 -- 172.21.15.83:0/4085865611 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd820004df0 con 0x55c122dbae30 2024-03-11T19:41:49.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.778+0000 7fd813fff640 1 -- 172.21.15.83:0/4085865611 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd82c0ac070 con 0x55c122dbae30 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.778+0000 7fd83d997740 10 client.15870 fetch_fsmap finished waiting for FSMap version 25 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.778+0000 7fd83d997740 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.778+0000 7fd83d997740 20 client.15870 populate_metadata read hostname 'smithi083' 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.778+0000 7fd83d997740 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c122e34110 0x55c122e36500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.778+0000 7fd83d997740 4 client.15870 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.778+0000 7fd83d997740 1 -- 172.21.15.83:0/4085865611 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c122aa9890 con 0x55c122e34110 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.778+0000 7fd838b63640 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c122e34110 0x55c122e36500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd838b63640 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c122e34110 0x55c122e36500 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bcbdc045a865227 server_cookie=90285f3954685289 in_seq=0 out_seq=0 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd813fff640 10 client.15870 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd813fff640 1 -- 172.21.15.83:0/4085865611 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c122aa9890 con 0x55c122e34110 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd813fff640 10 client.15870 handle_command_reply: tid=0 2024-03-11T19:41:49.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 1 client.15870 shutdown 2024-03-11T19:41:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 1 -- 172.21.15.83:0/4085865611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122dbae30 msgr2=0x55c122dd69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122dbae30 0x55c122dd69e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd82c096ed0 tx=0x7fd82c09ab30 comp rx=0 tx=0).stop 2024-03-11T19:41:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 1 -- 172.21.15.83:0/4085865611 shutdown_connections 2024-03-11T19:41:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c122e34110 0x55c122e36500 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c122dbae30 0x55c122dd69e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c122db5cc0 0x55c122dd64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 1 --2- 172.21.15.83:0/4085865611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c122db52d0 0x55c122dd5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 1 -- 172.21.15.83:0/4085865611 >> 172.21.15.83:0/4085865611 conn(0x55c122db07f0 msgr2=0x55c122db6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd813fff640 0 client.15870 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:41:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 1 -- 172.21.15.83:0/4085865611 shutdown_connections 2024-03-11T19:41:49.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 1 -- 172.21.15.83:0/4085865611 wait complete. 2024-03-11T19:41:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.779+0000 7fd83d997740 20 client.15870 trim_cache size 0 max 16384 2024-03-11T19:41:49.788 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:41:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:41:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:41:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "df38e5eb-b083-414b-bed1-ac4812650ab8": { 2024-03-11T19:41:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:41:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "df38e5eb-b083-414b-bed1-ac4812650ab8", 2024-03-11T19:41:49.788 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:41:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:41:49.789 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:41:49.789 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:41:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 -- 172.21.15.83:0/1980277445 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd8240a1b30 msgr2=0x7fd8240a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd8240a1b30 0x7fd8240a3ff0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd820004c50 tx=0x7fd8200039b0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 -- 172.21.15.83:0/1980277445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd834104a80 msgr2=0x7fd834075aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:41:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd834104a80 0x7fd834075aa0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00eb10 tx=0x7fd82c00c2d0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 -- 172.21.15.83:0/1980277445 shutdown_connections 2024-03-11T19:41:49.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd8240a1b30 0x7fd8240a3ff0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd834104a80 0x7fd834075aa0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd834103970 0x7fd834079470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 --2- 172.21.15.83:0/1980277445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834102fb0 0x7fd834078f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:41:49.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 -- 172.21.15.83:0/1980277445 >> 172.21.15.83:0/1980277445 conn(0x7fd8340fe930 msgr2=0x7fd834107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:41:49.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 -- 172.21.15.83:0/1980277445 shutdown_connections 2024-03-11T19:41:49.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:41:49.780+0000 7fd83adc7640 1 -- 172.21.15.83:0/1980277445 wait complete. 2024-03-11T19:41:49.801 INFO:tasks.cephfs.filesystem:scrub status for tag:df38e5eb-b083-414b-bed1-ac4812650ab8 - {'path': '/', 'tag': 'df38e5eb-b083-414b-bed1-ac4812650ab8', 'options': 'recursive,force'} 2024-03-11T19:41:50.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:50 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:41:50.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:50 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:50.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:50 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:41:50.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:50 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:50.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:50 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:41:50.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:50 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:51 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:41:51.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:51 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:51 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:41:51.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:51 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:41:51.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:51 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:51 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:41:51.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:51 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:51 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:51 smithi137 ceph-mon[36220]: pgmap v428: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 894 op/s 2024-03-11T19:41:51.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:51 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:41:51.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:51 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:51 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:41:51.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:51 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:41:51.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:51 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:51 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:41:51.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:51 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:51 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:51 smithi204 ceph-mon[36739]: pgmap v428: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 894 op/s 2024-03-11T19:41:51.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:51 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:41:51.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:51 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:51 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:41:51.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:51 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:41:51.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:51 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:51 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:41:51.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:51 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:51 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:41:51.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:51 smithi083 ceph-mon[31864]: pgmap v428: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 894 op/s 2024-03-11T19:41:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 155.0 s, 4146.2 tps, lat 24.119 ms stddev 26.120 2024-03-11T19:41:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:52 smithi083 ceph-mon[31864]: pgmap v429: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 868 op/s 2024-03-11T19:41:53.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:52 smithi137 ceph-mon[36220]: pgmap v429: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 868 op/s 2024-03-11T19:41:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:52 smithi204 ceph-mon[36739]: pgmap v429: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 868 op/s 2024-03-11T19:41:54.958 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:54 smithi204 ceph-mon[36739]: pgmap v430: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 951 op/s 2024-03-11T19:41:55.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:54 smithi083 ceph-mon[31864]: pgmap v430: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 951 op/s 2024-03-11T19:41:55.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:54 smithi137 ceph-mon[36220]: pgmap v430: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 951 op/s 2024-03-11T19:41:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:55 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:41:56.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:55 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:41:56.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:55 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:41:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:56 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:41:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:56 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:41:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:56 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:41:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:56 smithi083 ceph-mon[31864]: pgmap v431: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 918 op/s 2024-03-11T19:41:57.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:56 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:41:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:56 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:41:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:56 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:41:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:56 smithi137 ceph-mon[36220]: pgmap v431: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 918 op/s 2024-03-11T19:41:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:56 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:41:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:56 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:41:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:56 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:41:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:56 smithi204 ceph-mon[36739]: pgmap v431: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 918 op/s 2024-03-11T19:41:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 160.0 s, 4901.0 tps, lat 20.447 ms stddev 20.845 2024-03-11T19:41:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:41:58 smithi083 ceph-mon[31864]: pgmap v432: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 934 op/s 2024-03-11T19:41:59.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:41:58 smithi137 ceph-mon[36220]: pgmap v432: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 934 op/s 2024-03-11T19:41:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:41:58 smithi204 ceph-mon[36739]: pgmap v432: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 934 op/s 2024-03-11T19:42:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:01 smithi204 ceph-mon[36739]: pgmap v433: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 925 op/s 2024-03-11T19:42:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:01 smithi083 ceph-mon[31864]: pgmap v433: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 925 op/s 2024-03-11T19:42:02.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:01 smithi137 ceph-mon[36220]: pgmap v433: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 925 op/s 2024-03-11T19:42:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 165.0 s, 4541.2 tps, lat 22.024 ms stddev 23.741 2024-03-11T19:42:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:02 smithi083 ceph-mon[31864]: pgmap v434: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 864 op/s 2024-03-11T19:42:03.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:02 smithi137 ceph-mon[36220]: pgmap v434: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 864 op/s 2024-03-11T19:42:03.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:02 smithi204 ceph-mon[36739]: pgmap v434: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 864 op/s 2024-03-11T19:42:05.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:05 smithi204 ceph-mon[36739]: pgmap v435: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 948 op/s 2024-03-11T19:42:06.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:05 smithi083 ceph-mon[31864]: pgmap v435: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 948 op/s 2024-03-11T19:42:06.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:05 smithi137 ceph-mon[36220]: pgmap v435: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 948 op/s 2024-03-11T19:42:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:06 smithi083 ceph-mon[31864]: pgmap v436: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 946 op/s 2024-03-11T19:42:07.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:06 smithi137 ceph-mon[36220]: pgmap v436: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 946 op/s 2024-03-11T19:42:07.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:06 smithi204 ceph-mon[36739]: pgmap v436: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 946 op/s 2024-03-11T19:42:07.504 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 170.0 s, 4599.4 tps, lat 21.665 ms stddev 23.841 2024-03-11T19:42:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:09 smithi204 ceph-mon[36739]: pgmap v437: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 966 op/s 2024-03-11T19:42:10.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:09 smithi083 ceph-mon[31864]: pgmap v437: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 966 op/s 2024-03-11T19:42:10.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:09 smithi137 ceph-mon[36220]: pgmap v437: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 966 op/s 2024-03-11T19:42:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:10 smithi083 ceph-mon[31864]: pgmap v438: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.6 MiB/s wr, 800 op/s 2024-03-11T19:42:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:10 smithi137 ceph-mon[36220]: pgmap v438: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.6 MiB/s wr, 800 op/s 2024-03-11T19:42:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:10 smithi204 ceph-mon[36739]: pgmap v438: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.6 MiB/s wr, 800 op/s 2024-03-11T19:42:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 175.0 s, 5204.6 tps, lat 19.184 ms stddev 22.537 2024-03-11T19:42:13.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:13 smithi204 ceph-mon[36739]: pgmap v439: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.5 MiB/s wr, 876 op/s 2024-03-11T19:42:14.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:13 smithi083 ceph-mon[31864]: pgmap v439: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.5 MiB/s wr, 876 op/s 2024-03-11T19:42:14.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:13 smithi137 ceph-mon[36220]: pgmap v439: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 9.5 MiB/s wr, 876 op/s 2024-03-11T19:42:15.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:14 smithi083 ceph-mon[31864]: pgmap v440: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 910 op/s 2024-03-11T19:42:15.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:14 smithi137 ceph-mon[36220]: pgmap v440: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 910 op/s 2024-03-11T19:42:15.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:14 smithi204 ceph-mon[36739]: pgmap v440: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 910 op/s 2024-03-11T19:42:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 180.0 s, 4143.2 tps, lat 24.259 ms stddev 28.097 2024-03-11T19:42:17.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:17 smithi204 ceph-mon[36739]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.3 MiB/s wr, 842 op/s 2024-03-11T19:42:18.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:17 smithi083 ceph-mon[31864]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.3 MiB/s wr, 842 op/s 2024-03-11T19:42:18.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:17 smithi137 ceph-mon[36220]: pgmap v441: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.3 MiB/s wr, 842 op/s 2024-03-11T19:42:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:18 smithi083 ceph-mon[31864]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.5 MiB/s wr, 847 op/s 2024-03-11T19:42:19.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:18 smithi137 ceph-mon[36220]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.5 MiB/s wr, 847 op/s 2024-03-11T19:42:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:18 smithi204 ceph-mon[36739]: pgmap v442: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.5 MiB/s wr, 847 op/s 2024-03-11T19:42:19.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:42:19.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:42:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.943+0000 7efc13b56640 1 -- 172.21.15.83:0/3092735743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc0c1000c0 msgr2=0x7efc0c1004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.943+0000 7efc13b56640 1 --2- 172.21.15.83:0/3092735743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc0c1000c0 0x7efc0c1004e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efc08009900 tx=0x7efc0802bc50 comp rx=0 tx=0).stop 2024-03-11T19:42:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.943+0000 7efc13b56640 1 -- 172.21.15.83:0/3092735743 shutdown_connections 2024-03-11T19:42:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.943+0000 7efc13b56640 1 --2- 172.21.15.83:0/3092735743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c101c70 0x7efc0c108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.943+0000 7efc13b56640 1 --2- 172.21.15.83:0/3092735743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c1012b0 0x7efc0c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.943+0000 7efc13b56640 1 --2- 172.21.15.83:0/3092735743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc0c1000c0 0x7efc0c1004e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.943+0000 7efc13b56640 1 -- 172.21.15.83:0/3092735743 >> 172.21.15.83:0/3092735743 conn(0x7efc0c0fb650 msgr2=0x7efc0c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:19.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.944+0000 7efc13b56640 1 -- 172.21.15.83:0/3092735743 shutdown_connections 2024-03-11T19:42:19.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.944+0000 7efc13b56640 1 -- 172.21.15.83:0/3092735743 wait complete. 2024-03-11T19:42:19.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.944+0000 7efc13b56640 1 Processor -- start 2024-03-11T19:42:19.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.944+0000 7efc13b56640 1 -- start start 2024-03-11T19:42:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.944+0000 7efc13b56640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c1000c0 0x7efc0c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.944+0000 7efc13b56640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c1012b0 0x7efc0c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.944+0000 7efc13b56640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc0c101c70 0x7efc0c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.944+0000 7efc13b56640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c19ee70 con 0x7efc0c1012b0 2024-03-11T19:42:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.944+0000 7efc13b56640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c19efe0 con 0x7efc0c1000c0 2024-03-11T19:42:19.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.944+0000 7efc13b56640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c19f1a0 con 0x7efc0c101c70 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.945+0000 7efc118f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c1000c0 0x7efc0c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.945+0000 7efc118f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c1000c0 0x7efc0c197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40402/0 (socket says 172.21.15.83:40402) 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.945+0000 7efc118f2640 1 -- 172.21.15.83:0/2715220836 learned_addr learned my addr 172.21.15.83:0/2715220836 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.945+0000 7efc120f3640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc0c101c70 0x7efc0c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.945+0000 7efc120f3640 1 -- 172.21.15.83:0/2715220836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c1000c0 msgr2=0x7efc0c197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.945+0000 7efc120f3640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c1000c0 0x7efc0c197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.945+0000 7efc120f3640 1 -- 172.21.15.83:0/2715220836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c1012b0 msgr2=0x7efc0c1978c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.945+0000 7efc120f3640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c1012b0 0x7efc0c1978c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.945+0000 7efc120f3640 1 -- 172.21.15.83:0/2715220836 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc08009560 con 0x7efc0c101c70 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.946+0000 7efc120f3640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc0c101c70 0x7efc0c19e930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc0400be20 tx=0x7efc0400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.946+0000 7efc02ffd640 1 -- 172.21.15.83:0/2715220836 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0400ce70 con 0x7efc0c101c70 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.946+0000 7efc02ffd640 1 -- 172.21.15.83:0/2715220836 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc04014920 con 0x7efc0c101c70 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.946+0000 7efc02ffd640 1 -- 172.21.15.83:0/2715220836 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04012a30 con 0x7efc0c101c70 2024-03-11T19:42:19.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.946+0000 7efc13b56640 1 -- 172.21.15.83:0/2715220836 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0c19f480 con 0x7efc0c101c70 2024-03-11T19:42:19.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.946+0000 7efc13b56640 1 -- 172.21.15.83:0/2715220836 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0c19f9c0 con 0x7efc0c101c70 2024-03-11T19:42:19.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.948+0000 7efc02ffd640 1 -- 172.21.15.83:0/2715220836 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7efc04014a90 con 0x7efc0c101c70 2024-03-11T19:42:19.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.949+0000 7efc02ffd640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efbf80a1ab0 0x7efbf80a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.949+0000 7efc118f2640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efbf80a1ab0 0x7efbf80a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.949+0000 7efc02ffd640 1 -- 172.21.15.83:0/2715220836 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7efc040cceb0 con 0x7efc0c101c70 2024-03-11T19:42:19.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.950+0000 7efc16806740 1 Processor -- start 2024-03-11T19:42:19.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.951+0000 7efc16806740 1 -- start start 2024-03-11T19:42:19.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.951+0000 7efc16806740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 0x55cd3d9644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.951+0000 7efc16806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d969050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc16806740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d9696c0 0x55cd3d96bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc16806740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d8cb280 con 0x55cd3d964a70 2024-03-11T19:42:19.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc16806740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d8c87b0 con 0x55cd3d9696c0 2024-03-11T19:42:19.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc16806740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d8c3f90 con 0x55cd3d964080 2024-03-11T19:42:19.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc118f2640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efbf80a1ab0 0x7efbf80a3f70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc08006010 tx=0x7efc0802c760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:19.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc110f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d969050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc110f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d969050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54470/0 (socket says 172.21.15.83:54470) 2024-03-11T19:42:19.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc110f1640 1 -- 172.21.15.83:0/3327590539 learned_addr learned my addr 172.21.15.83:0/3327590539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:19.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc118f2640 1 --2- 172.21.15.83:0/3327590539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 0x55cd3d9644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc120f3640 1 --2- 172.21.15.83:0/3327590539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d9696c0 0x55cd3d96bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc118f2640 1 -- 172.21.15.83:0/3327590539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d9696c0 msgr2=0x55cd3d96bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc118f2640 1 --2- 172.21.15.83:0/3327590539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d9696c0 0x55cd3d96bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc118f2640 1 -- 172.21.15.83:0/3327590539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 msgr2=0x55cd3d969050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc118f2640 1 --2- 172.21.15.83:0/3327590539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d969050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc118f2640 1 -- 172.21.15.83:0/3327590539 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd3d882570 con 0x55cd3d964080 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efc118f2640 1 --2- 172.21.15.83:0/3327590539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 0x55cd3d9644a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc08000c00 tx=0x7efc0803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f210beee0a8ec53b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efbf7fff640 1 -- 172.21.15.83:0/3327590539 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08004880 con 0x55cd3d964080 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efbf7fff640 1 -- 172.21.15.83:0/3327590539 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc080049e0 con 0x55cd3d964080 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.952+0000 7efbf7fff640 1 -- 172.21.15.83:0/3327590539 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc080035f0 con 0x55cd3d964080 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.953+0000 7efc16806740 1 -- 172.21.15.83:0/3327590539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 msgr2=0x55cd3d9644a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.953+0000 7efc16806740 1 --2- 172.21.15.83:0/3327590539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 0x55cd3d9644a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc08000c00 tx=0x7efc0803f000 comp rx=0 tx=0).stop 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.953+0000 7efc16806740 1 -- 172.21.15.83:0/3327590539 shutdown_connections 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.953+0000 7efc16806740 1 --2- 172.21.15.83:0/3327590539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d9696c0 0x55cd3d96bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.953+0000 7efc16806740 1 --2- 172.21.15.83:0/3327590539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d969050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.953+0000 7efc16806740 1 --2- 172.21.15.83:0/3327590539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 0x55cd3d9644a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.953+0000 7efc16806740 1 -- 172.21.15.83:0/3327590539 >> 172.21.15.83:0/3327590539 conn(0x55cd3d956770 msgr2=0x55cd3d956b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:19.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.953+0000 7efc16806740 1 -- 172.21.15.83:0/3327590539 shutdown_connections 2024-03-11T19:42:19.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.953+0000 7efc16806740 1 -- 172.21.15.83:0/3327590539 wait complete. 2024-03-11T19:42:19.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.954+0000 7efc16806740 1 -- start start 2024-03-11T19:42:19.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.954+0000 7efc16806740 1 Processor -- start 2024-03-11T19:42:19.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.954+0000 7efc16806740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 0x55cd3d96d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.954+0000 7efc16806740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964a70 0x55cd3d96de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.954+0000 7efc16806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d9696c0 0x55cd3d974ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.954+0000 7efc16806740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d597430 con 0x55cd3d9696c0 2024-03-11T19:42:19.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.954+0000 7efc16806740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d553e00 con 0x55cd3d964a70 2024-03-11T19:42:19.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.954+0000 7efc118f2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 0x55cd3d96d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc110f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964a70 0x55cd3d96de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc118f2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 0x55cd3d96d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37382/0 (socket says 172.21.15.83:37382) 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc118f2640 1 -- 172.21.15.83:0/1810390214 learned_addr learned my addr 172.21.15.83:0/1810390214 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc120f3640 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d9696c0 0x55cd3d974ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc16806740 1 -- 172.21.15.83:0/1810390214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d848fd0 con 0x55cd3d964080 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc16806740 10 client.0 _ll_register_callbacks cb 0x55cd3d89efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc110f1640 1 -- 172.21.15.83:0/1810390214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 msgr2=0x55cd3d96d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc110f1640 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 0x55cd3d96d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc110f1640 1 -- 172.21.15.83:0/1810390214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d9696c0 msgr2=0x55cd3d974ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc110f1640 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d9696c0 0x55cd3d974ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc110f1640 1 -- 172.21.15.83:0/1810390214 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0c108e00 con 0x55cd3d964a70 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc110f1640 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964a70 0x55cd3d96de90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55cd3d964f60 tx=0x7efbfc003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f12d1dcbcd6396a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efbf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efbf57fa640 1 -- 172.21.15.83:0/1810390214 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc00bd80 con 0x55cd3d964a70 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efc16806740 1 -- 172.21.15.83:0/1810390214 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd3d5c37e0 con 0x55cd3d964a70 2024-03-11T19:42:19.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.955+0000 7efbf57fa640 1 -- 172.21.15.83:0/1810390214 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7efbfc005020 con 0x55cd3d964a70 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efbf57fa640 1 -- 172.21.15.83:0/1810390214 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbfc01f030 con 0x55cd3d964a70 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efc16806740 10 client.25379 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efc16806740 1 -- 172.21.15.83:0/1810390214 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cd3d8c2610 con 0x55cd3d964a70 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efbf57fa640 1 -- 172.21.15.83:0/1810390214 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cd3d5c37e0 con 0x55cd3d964a70 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efbf57fa640 1 -- 172.21.15.83:0/1810390214 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efbfc011440 con 0x55cd3d964a70 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efc16806740 10 client.25379 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efc16806740 10 client.25379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efc16806740 20 client.25379 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efc16806740 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd3d979c00 0x55cd3d97c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efc16806740 4 client.25379 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efc16806740 1 -- 172.21.15.83:0/1810390214 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd3d600720 con 0x55cd3d979c00 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.956+0000 7efc118f2640 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd3d979c00 0x55cd3d97c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.957+0000 7efc118f2640 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd3d979c00 0x55cd3d97c020 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0662983c4c94e4a server_cookie=b3bdda811bb3f1d9 in_seq=0 out_seq=0 2024-03-11T19:42:19.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.957+0000 7efbf57fa640 10 client.25379 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.957+0000 7efbf57fa640 1 -- 172.21.15.83:0/1810390214 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd3d600720 con 0x55cd3d979c00 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.957+0000 7efbf57fa640 10 client.25379 handle_command_reply: tid=0 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.957+0000 7efc16806740 1 client.25379 shutdown 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efc16806740 1 -- 172.21.15.83:0/1810390214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964a70 msgr2=0x55cd3d96de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efc16806740 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964a70 0x55cd3d96de90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55cd3d964f60 tx=0x7efbfc003c40 comp rx=0 tx=0).stop 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efc16806740 1 -- 172.21.15.83:0/1810390214 shutdown_connections 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efc16806740 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd3d979c00 0x55cd3d97c020 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efc16806740 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d9696c0 0x55cd3d974ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efbf57fa640 0 client.25379 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efc16806740 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964a70 0x55cd3d96de90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efc16806740 1 --2- 172.21.15.83:0/1810390214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d964080 0x55cd3d96d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efc16806740 1 -- 172.21.15.83:0/1810390214 >> 172.21.15.83:0/1810390214 conn(0x55cd3d956770 msgr2=0x55cd3d963710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:19.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efc16806740 1 -- 172.21.15.83:0/1810390214 shutdown_connections 2024-03-11T19:42:19.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efc16806740 1 -- 172.21.15.83:0/1810390214 wait complete. 2024-03-11T19:42:19.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.958+0000 7efc16806740 20 client.25379 trim_cache size 0 max 16384 2024-03-11T19:42:19.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.978+0000 7efc16806740 1 Processor -- start 2024-03-11T19:42:19.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc16806740 1 -- start start 2024-03-11T19:42:19.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc16806740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 0x55cd3d96d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc16806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d96de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc16806740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 0x55cd3d974e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc16806740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d975d80 con 0x55cd3d964a70 2024-03-11T19:42:19.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc16806740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d9780c0 con 0x55cd3d964080 2024-03-11T19:42:19.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc16806740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d959f50 con 0x55cd3d9696c0 2024-03-11T19:42:19.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc118f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 0x55cd3d96d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc120f3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 0x55cd3d974e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc118f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 0x55cd3d96d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40442/0 (socket says 172.21.15.83:40442) 2024-03-11T19:42:19.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc118f2640 1 -- 172.21.15.83:0/890289222 learned_addr learned my addr 172.21.15.83:0/890289222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.979+0000 7efc110f1640 1 --2- 172.21.15.83:0/890289222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d96de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.980+0000 7efc118f2640 1 -- 172.21.15.83:0/890289222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 msgr2=0x55cd3d974e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.980+0000 7efc118f2640 1 --2- 172.21.15.83:0/890289222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 0x55cd3d974e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.980+0000 7efc118f2640 1 -- 172.21.15.83:0/890289222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 msgr2=0x55cd3d96de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.980+0000 7efc118f2640 1 --2- 172.21.15.83:0/890289222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d96de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.980+0000 7efc118f2640 1 -- 172.21.15.83:0/890289222 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd3d85eac0 con 0x55cd3d964080 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.980+0000 7efc120f3640 1 --2- 172.21.15.83:0/890289222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 0x55cd3d974e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.980+0000 7efc110f1640 1 --2- 172.21.15.83:0/890289222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d96de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.980+0000 7efc118f2640 1 --2- 172.21.15.83:0/890289222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 0x55cd3d96d950 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efc08000c00 tx=0x7efc08004c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe0da0d669a492 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.980+0000 7efbf77fe640 1 -- 172.21.15.83:0/890289222 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08004880 con 0x55cd3d964080 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.980+0000 7efbf77fe640 1 -- 172.21.15.83:0/890289222 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc08004df0 con 0x55cd3d964080 2024-03-11T19:42:19.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.980+0000 7efbf77fe640 1 -- 172.21.15.83:0/890289222 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc080055b0 con 0x55cd3d964080 2024-03-11T19:42:19.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.981+0000 7efc16806740 1 -- 172.21.15.83:0/890289222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 msgr2=0x55cd3d96d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:19.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.981+0000 7efc16806740 1 --2- 172.21.15.83:0/890289222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 0x55cd3d96d950 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efc08000c00 tx=0x7efc08004c10 comp rx=0 tx=0).stop 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.981+0000 7efc16806740 1 -- 172.21.15.83:0/890289222 shutdown_connections 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.981+0000 7efc16806740 1 --2- 172.21.15.83:0/890289222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 0x55cd3d974e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.981+0000 7efc16806740 1 --2- 172.21.15.83:0/890289222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d96de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.981+0000 7efc16806740 1 --2- 172.21.15.83:0/890289222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 0x55cd3d96d950 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.981+0000 7efc16806740 1 -- 172.21.15.83:0/890289222 >> 172.21.15.83:0/890289222 conn(0x55cd3d956770 msgr2=0x55cd3d9809c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.981+0000 7efc16806740 1 -- 172.21.15.83:0/890289222 shutdown_connections 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.981+0000 7efc16806740 1 -- 172.21.15.83:0/890289222 wait complete. 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.981+0000 7efc16806740 1 -- start start 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.981+0000 7efc16806740 1 Processor -- start 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc16806740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 0x55cd3d974a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc16806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d973080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc16806740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 0x55cd3d9735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc16806740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d9751f0 con 0x55cd3d964a70 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc16806740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d973b00 con 0x55cd3d964080 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc16806740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd3d973cc0 con 0x55cd3d9696c0 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc16806740 10 client.0 _ll_register_callbacks cb 0x7efc08005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc110f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d973080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc118f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 0x55cd3d974a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc110f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d973080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54482/0 (socket says 172.21.15.83:54482) 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc110f1640 1 -- 172.21.15.83:0/4037317033 learned_addr learned my addr 172.21.15.83:0/4037317033 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc120f3640 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 0x55cd3d9735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc110f1640 1 -- 172.21.15.83:0/4037317033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 msgr2=0x55cd3d9735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc110f1640 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 0x55cd3d9735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc110f1640 1 -- 172.21.15.83:0/4037317033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 msgr2=0x55cd3d974a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc110f1640 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 0x55cd3d974a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc110f1640 1 -- 172.21.15.83:0/4037317033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd3d882570 con 0x55cd3d964a70 2024-03-11T19:42:19.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc120f3640 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 0x55cd3d9735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:19.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.982+0000 7efc118f2640 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 0x55cd3d974a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:19.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.983+0000 7efc110f1640 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d973080 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55cd3d96ee30 tx=0x7efbfc0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef779b256d35cd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:19.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.983+0000 7efbf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:42:19.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.983+0000 7efbf57fa640 1 -- 172.21.15.83:0/4037317033 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc01f040 con 0x55cd3d964a70 2024-03-11T19:42:19.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.983+0000 7efbf57fa640 1 -- 172.21.15.83:0/4037317033 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7efbfc020070 con 0x55cd3d964a70 2024-03-11T19:42:19.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.983+0000 7efc16806740 1 -- 172.21.15.83:0/4037317033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc080049e0 con 0x55cd3d964a70 2024-03-11T19:42:19.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.984+0000 7efbf57fa640 1 -- 172.21.15.83:0/4037317033 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbfc004370 con 0x55cd3d964a70 2024-03-11T19:42:19.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.985+0000 7efc16806740 10 client.15885 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:19.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.985+0000 7efc16806740 1 -- 172.21.15.83:0/4037317033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cd3d8846a0 con 0x55cd3d964a70 2024-03-11T19:42:19.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.985+0000 7efbf57fa640 1 -- 172.21.15.83:0/4037317033 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc080049e0 con 0x55cd3d964a70 2024-03-11T19:42:19.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.985+0000 7efbf57fa640 1 -- 172.21.15.83:0/4037317033 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efbfc004730 con 0x55cd3d964a70 2024-03-11T19:42:19.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.985+0000 7efc16806740 10 client.15885 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:19.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.985+0000 7efc16806740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:19.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.985+0000 7efc16806740 20 client.15885 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:19.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.985+0000 7efc16806740 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd3d979c00 0x55cd3d9d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:19.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.985+0000 7efc16806740 4 client.15885 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:42:19.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.985+0000 7efc16806740 1 -- 172.21.15.83:0/4037317033 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd3d64f890 con 0x55cd3d979c00 2024-03-11T19:42:19.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.985+0000 7efc118f2640 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd3d979c00 0x55cd3d9d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:19.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efbf57fa640 10 client.15885 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:19.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc118f2640 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd3d979c00 0x55cd3d9d3860 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d278daf8af01260 server_cookie=7a2e6c7912f15a7 in_seq=0 out_seq=0 2024-03-11T19:42:19.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efbf57fa640 1 -- 172.21.15.83:0/4037317033 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cd3d64f890 con 0x55cd3d979c00 2024-03-11T19:42:19.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efbf57fa640 10 client.15885 handle_command_reply: tid=0 2024-03-11T19:42:19.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 1 client.15885 shutdown 2024-03-11T19:42:19.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 1 -- 172.21.15.83:0/4037317033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 msgr2=0x55cd3d973080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:19.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d973080 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55cd3d96ee30 tx=0x7efbfc0056a0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 1 -- 172.21.15.83:0/4037317033 shutdown_connections 2024-03-11T19:42:19.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cd3d979c00 0x55cd3d9d3860 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd3d9696c0 0x55cd3d9735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd3d964a70 0x55cd3d973080 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efbf57fa640 0 client.15885 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:19.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 1 --2- 172.21.15.83:0/4037317033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cd3d964080 0x55cd3d974a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 1 -- 172.21.15.83:0/4037317033 >> 172.21.15.83:0/4037317033 conn(0x55cd3d956770 msgr2=0x55cd3d9809c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:19.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 1 -- 172.21.15.83:0/4037317033 shutdown_connections 2024-03-11T19:42:19.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 1 -- 172.21.15.83:0/4037317033 wait complete. 2024-03-11T19:42:19.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.986+0000 7efc16806740 20 client.15885 trim_cache size 0 max 16384 2024-03-11T19:42:19.997 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:42:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:42:19.997 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:42:19.997 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:42:19.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.987+0000 7efc13b56640 1 -- 172.21.15.83:0/2715220836 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efbf80a1ab0 msgr2=0x7efbf80a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:19.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.988+0000 7efc13b56640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efbf80a1ab0 0x7efbf80a3f70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efc08006010 tx=0x7efc0802c760 comp rx=0 tx=0).stop 2024-03-11T19:42:19.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.988+0000 7efc13b56640 1 -- 172.21.15.83:0/2715220836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc0c101c70 msgr2=0x7efc0c19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:19.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.988+0000 7efc13b56640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc0c101c70 0x7efc0c19e930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc0400be20 tx=0x7efc0400bf10 comp rx=0 tx=0).stop 2024-03-11T19:42:19.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.988+0000 7efc13b56640 1 -- 172.21.15.83:0/2715220836 shutdown_connections 2024-03-11T19:42:19.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.988+0000 7efc13b56640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efbf80a1ab0 0x7efbf80a3f70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.988+0000 7efc13b56640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc0c101c70 0x7efc0c19e930 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.988+0000 7efc13b56640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c1012b0 0x7efc0c1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.988+0000 7efc13b56640 1 --2- 172.21.15.83:0/2715220836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc0c1000c0 0x7efc0c197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:19.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.988+0000 7efc13b56640 1 -- 172.21.15.83:0/2715220836 >> 172.21.15.83:0/2715220836 conn(0x7efc0c0fb650 msgr2=0x7efc0c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:19.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.988+0000 7efc13b56640 1 -- 172.21.15.83:0/2715220836 shutdown_connections 2024-03-11T19:42:19.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:19.988+0000 7efc13b56640 1 -- 172.21.15.83:0/2715220836 wait complete. 2024-03-11T19:42:20.006 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:42:20.006 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:42:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.139+0000 7f878f676640 1 -- 172.21.15.83:0/2605705616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87881045b0 msgr2=0x7f8788104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.139+0000 7f878f676640 1 --2- 172.21.15.83:0/2605705616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87881045b0 0x7f8788104a30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8778009900 tx=0x7f877802bc30 comp rx=0 tx=0).stop 2024-03-11T19:42:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.139+0000 7f878f676640 1 -- 172.21.15.83:0/2605705616 shutdown_connections 2024-03-11T19:42:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.139+0000 7f878f676640 1 --2- 172.21.15.83:0/2605705616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8788104f70 0x7f878810b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.139+0000 7f878f676640 1 --2- 172.21.15.83:0/2605705616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87881045b0 0x7f8788104a30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.139+0000 7f878f676640 1 --2- 172.21.15.83:0/2605705616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87881033c0 0x7f87881037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.139+0000 7f878f676640 1 -- 172.21.15.83:0/2605705616 >> 172.21.15.83:0/2605705616 conn(0x7f87880fe930 msgr2=0x7f8788100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.139+0000 7f878f676640 1 -- 172.21.15.83:0/2605705616 shutdown_connections 2024-03-11T19:42:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.139+0000 7f878f676640 1 -- 172.21.15.83:0/2605705616 wait complete. 2024-03-11T19:42:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878f676640 1 Processor -- start 2024-03-11T19:42:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878f676640 1 -- start start 2024-03-11T19:42:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878f676640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87881033c0 0x7f878819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878f676640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87881045b0 0x7f878819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878f676640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8788104f70 0x7f87881a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878f676640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87881a5350 con 0x7f8788104f70 2024-03-11T19:42:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878f676640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878806abd0 con 0x7f87881045b0 2024-03-11T19:42:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878f676640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878806ad90 con 0x7f87881033c0 2024-03-11T19:42:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878dc13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8788104f70 0x7f87881a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878dc13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8788104f70 0x7f87881a4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54498/0 (socket says 172.21.15.83:54498) 2024-03-11T19:42:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878dc13640 1 -- 172.21.15.83:0/405890939 learned_addr learned my addr 172.21.15.83:0/405890939 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878cc11640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87881045b0 0x7f878819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.140+0000 7f878d412640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87881033c0 0x7f878819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.141+0000 7f878dc13640 1 -- 172.21.15.83:0/405890939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87881033c0 msgr2=0x7f878819d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.141+0000 7f878dc13640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87881033c0 0x7f878819d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.141+0000 7f878dc13640 1 -- 172.21.15.83:0/405890939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87881045b0 msgr2=0x7f878819dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.141+0000 7f878dc13640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87881045b0 0x7f878819dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.141+0000 7f878dc13640 1 -- 172.21.15.83:0/405890939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8778009560 con 0x7f8788104f70 2024-03-11T19:42:20.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.141+0000 7f878dc13640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8788104f70 0x7f87881a4e10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f878400eb10 tx=0x7f878400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.141+0000 7f877e7fc640 1 -- 172.21.15.83:0/405890939 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784004b20 con 0x7f8788104f70 2024-03-11T19:42:20.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.141+0000 7f877e7fc640 1 -- 172.21.15.83:0/405890939 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8784004c80 con 0x7f8788104f70 2024-03-11T19:42:20.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.141+0000 7f878f676640 1 -- 172.21.15.83:0/405890939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878806b070 con 0x7f8788104f70 2024-03-11T19:42:20.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.142+0000 7f877e7fc640 1 -- 172.21.15.83:0/405890939 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784010760 con 0x7f8788104f70 2024-03-11T19:42:20.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.142+0000 7f878f676640 1 -- 172.21.15.83:0/405890939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878806b550 con 0x7f8788104f70 2024-03-11T19:42:20.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.143+0000 7f878f676640 1 -- 172.21.15.83:0/405890939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8788067f70 con 0x7f8788104f70 2024-03-11T19:42:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.144+0000 7f877e7fc640 1 -- 172.21.15.83:0/405890939 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f87840108c0 con 0x7f8788104f70 2024-03-11T19:42:20.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.145+0000 7f877e7fc640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87740a1b30 0x7f87740a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.145+0000 7f877e7fc640 1 -- 172.21.15.83:0/405890939 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8784014070 con 0x7f8788104f70 2024-03-11T19:42:20.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.145+0000 7f878d412640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87740a1b30 0x7f87740a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.146+0000 7f878d412640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87740a1b30 0x7f87740a3ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8788104410 tx=0x7f8780005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.149+0000 7f877e7fc640 1 -- 172.21.15.83:0/405890939 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8784096d90 con 0x7f8788104f70 2024-03-11T19:42:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.469+0000 7f878f676640 1 -- 172.21.15.83:0/405890939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f878810edd0 con 0x7f8788104f70 2024-03-11T19:42:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.470+0000 7f877e7fc640 1 -- 172.21.15.83:0/405890939 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11351 (secure 0 0 0) 0x7f878409a450 con 0x7f8788104f70 2024-03-11T19:42:20.475 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:42:20.475 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:41:55.449550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:42:20.475 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-03-11T19:42:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.472+0000 7f878f676640 1 -- 172.21.15.83:0/405890939 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87740a1b30 msgr2=0x7f87740a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.472+0000 7f878f676640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87740a1b30 0x7f87740a3ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8788104410 tx=0x7f8780005f90 comp rx=0 tx=0).stop 2024-03-11T19:42:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.472+0000 7f878f676640 1 -- 172.21.15.83:0/405890939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8788104f70 msgr2=0x7f87881a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.472+0000 7f878f676640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8788104f70 0x7f87881a4e10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f878400eb10 tx=0x7f878400c2d0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.472+0000 7f878f676640 1 -- 172.21.15.83:0/405890939 shutdown_connections 2024-03-11T19:42:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.472+0000 7f878f676640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87740a1b30 0x7f87740a3ff0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.472+0000 7f878f676640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8788104f70 0x7f87881a4e10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.472+0000 7f878f676640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87881045b0 0x7f878819dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.472+0000 7f878f676640 1 --2- 172.21.15.83:0/405890939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87881033c0 0x7f878819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.472+0000 7f878f676640 1 -- 172.21.15.83:0/405890939 >> 172.21.15.83:0/405890939 conn(0x7f87880fe930 msgr2=0x7f87880775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.472+0000 7f878f676640 1 -- 172.21.15.83:0/405890939 shutdown_connections 2024-03-11T19:42:20.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.473+0000 7f878f676640 1 -- 172.21.15.83:0/405890939 wait complete. 2024-03-11T19:42:20.491 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:42:20.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:42:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.633+0000 7f147ea57640 1 -- 172.21.15.83:0/638785735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147810e540 msgr2=0x7f14781109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.633+0000 7f147ea57640 1 --2- 172.21.15.83:0/638785735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147810e540 0x7f14781109d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1470009ca0 tx=0x7f147002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.633+0000 7f147ea57640 1 -- 172.21.15.83:0/638785735 shutdown_connections 2024-03-11T19:42:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.633+0000 7f147ea57640 1 --2- 172.21.15.83:0/638785735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f147810e540 0x7f14781109d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.633+0000 7f147ea57640 1 --2- 172.21.15.83:0/638785735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f147810bbc0 0x7f147810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.633+0000 7f147ea57640 1 --2- 172.21.15.83:0/638785735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1478109290 0x7f147810b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.633+0000 7f147ea57640 1 -- 172.21.15.83:0/638785735 >> 172.21.15.83:0/638785735 conn(0x7f147806dfe0 msgr2=0x7f1478070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.633+0000 7f147ea57640 1 -- 172.21.15.83:0/638785735 shutdown_connections 2024-03-11T19:42:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.633+0000 7f147ea57640 1 -- 172.21.15.83:0/638785735 wait complete. 2024-03-11T19:42:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.633+0000 7f147ea57640 1 Processor -- start 2024-03-11T19:42:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.634+0000 7f147ea57640 1 -- start start 2024-03-11T19:42:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.634+0000 7f147ea57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1478109290 0x7f14781a8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.634+0000 7f147ea57640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f147810bbc0 0x7f14781aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.634+0000 7f147ea57640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f147810e540 0x7f14781ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.634+0000 7f147ea57640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14781adb40 con 0x7f1478109290 2024-03-11T19:42:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.634+0000 7f147ea57640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14781adcb0 con 0x7f147810e540 2024-03-11T19:42:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.634+0000 7f147ea57640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14781ade70 con 0x7f147810bbc0 2024-03-11T19:42:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.634+0000 7f1477fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1478109290 0x7f14781a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.634+0000 7f1477fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1478109290 0x7f14781a8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54512/0 (socket says 172.21.15.83:54512) 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.634+0000 7f1477fff640 1 -- 172.21.15.83:0/3285202465 learned_addr learned my addr 172.21.15.83:0/3285202465 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f147cff4640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f147810e540 0x7f14781ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f1477fff640 1 -- 172.21.15.83:0/3285202465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f147810bbc0 msgr2=0x7f14781aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f1477fff640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f147810bbc0 0x7f14781aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f1477fff640 1 -- 172.21.15.83:0/3285202465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f147810e540 msgr2=0x7f14781ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f1477fff640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f147810e540 0x7f14781ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f1477fff640 1 -- 172.21.15.83:0/3285202465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1470009900 con 0x7f1478109290 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f1477fff640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1478109290 0x7f14781a8910 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f146800e9f0 tx=0x7f146800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f14757fa640 1 -- 172.21.15.83:0/3285202465 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468009700 con 0x7f1478109290 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f147ea57640 1 -- 172.21.15.83:0/3285202465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14781ae150 con 0x7f1478109290 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f147ea57640 1 -- 172.21.15.83:0/3285202465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14781ae6b0 con 0x7f1478109290 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f14757fa640 1 -- 172.21.15.83:0/3285202465 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f146800cd80 con 0x7f1478109290 2024-03-11T19:42:20.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.635+0000 7f14757fa640 1 -- 172.21.15.83:0/3285202465 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468018920 con 0x7f1478109290 2024-03-11T19:42:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.638+0000 7f1481620740 1 Processor -- start 2024-03-11T19:42:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.638+0000 7f14757fa640 1 -- 172.21.15.83:0/3285202465 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1468018a80 con 0x7f1478109290 2024-03-11T19:42:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f1481620740 1 -- start start 2024-03-11T19:42:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f1481620740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab032690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f1481620740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 0x5576ab037760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f1481620740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab037dd0 0x5576ab03a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f1481620740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576aafee960 con 0x5576ab032c60 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f1481620740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576aafa2310 con 0x5576ab032270 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f1481620740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576aaf9f840 con 0x5576ab037dd0 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f1477fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 0x5576ab037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f147cff4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab032690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f1477fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 0x5576ab037760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54526/0 (socket says 172.21.15.83:54526) 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f147cff4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab032690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40486/0 (socket says 172.21.15.83:40486) 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f1477fff640 1 -- 172.21.15.83:0/1006881737 learned_addr learned my addr 172.21.15.83:0/1006881737 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f14777fe640 1 --2- 172.21.15.83:0/1006881737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab037dd0 0x5576ab03a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f147cff4640 1 -- 172.21.15.83:0/1006881737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab037dd0 msgr2=0x5576ab03a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f147cff4640 1 --2- 172.21.15.83:0/1006881737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab037dd0 0x5576ab03a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f147cff4640 1 -- 172.21.15.83:0/1006881737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 msgr2=0x5576ab037760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f147cff4640 1 --2- 172.21.15.83:0/1006881737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 0x5576ab037760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.639+0000 7f147cff4640 1 -- 172.21.15.83:0/1006881737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576aaf5d060 con 0x5576ab032270 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.640+0000 7f147cff4640 1 --2- 172.21.15.83:0/1006881737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab032690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f147002c630 tx=0x7f14700093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7841e4f67b38165a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.640+0000 7f14757fa640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14600a1b30 0x7f14600a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.640+0000 7f145e7fc640 1 -- 172.21.15.83:0/1006881737 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14700323c0 con 0x5576ab032270 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.640+0000 7f145e7fc640 1 -- 172.21.15.83:0/1006881737 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14700327b0 con 0x5576ab032270 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.640+0000 7f145e7fc640 1 -- 172.21.15.83:0/1006881737 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470003c60 con 0x5576ab032270 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.640+0000 7f1477fff640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14600a1b30 0x7f14600a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.640+0000 7f14757fa640 1 -- 172.21.15.83:0/3285202465 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1468014070 con 0x7f1478109290 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1481620740 1 -- 172.21.15.83:0/1006881737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 msgr2=0x5576ab032690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1477fff640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14600a1b30 0x7f14600a3ff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1468021cc0 tx=0x7f1468083f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1481620740 1 --2- 172.21.15.83:0/1006881737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab032690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f147002c630 tx=0x7f14700093e0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1481620740 1 -- 172.21.15.83:0/1006881737 shutdown_connections 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1481620740 1 --2- 172.21.15.83:0/1006881737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab037dd0 0x5576ab03a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1481620740 1 --2- 172.21.15.83:0/1006881737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 0x5576ab037760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1481620740 1 --2- 172.21.15.83:0/1006881737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab032690 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1481620740 1 -- 172.21.15.83:0/1006881737 >> 172.21.15.83:0/1006881737 conn(0x5576ab02d7e0 msgr2=0x5576ab02dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1481620740 1 -- 172.21.15.83:0/1006881737 shutdown_connections 2024-03-11T19:42:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1481620740 1 -- 172.21.15.83:0/1006881737 wait complete. 2024-03-11T19:42:20.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1481620740 1 -- start start 2024-03-11T19:42:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.641+0000 7f1481620740 1 Processor -- start 2024-03-11T19:42:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f1481620740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab04d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f1481620740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 0x5576ab04d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f1481620740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab037dd0 0x5576ab0549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f1481620740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576aaf9b020 con 0x5576ab032c60 2024-03-11T19:42:20.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f1481620740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576aac6e430 con 0x5576ab032270 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f1481620740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576aac2ae00 con 0x5576ab037dd0 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f1481620740 10 client.0 _ll_register_callbacks cb 0x5576aaf42cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f14777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 0x5576ab04d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f14777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 0x5576ab04d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54532/0 (socket says 172.21.15.83:54532) 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f14777fe640 1 -- 172.21.15.83:0/3837486859 learned_addr learned my addr 172.21.15.83:0/3837486859 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f147cff4640 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab04d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f147cff4640 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab037dd0 0x5576ab0549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f14777fe640 1 -- 172.21.15.83:0/3837486859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab037dd0 msgr2=0x5576ab0549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f14777fe640 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab037dd0 0x5576ab0549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f14777fe640 1 -- 172.21.15.83:0/3837486859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 msgr2=0x5576ab04d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f14777fe640 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab04d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.642+0000 7f14777fe640 1 -- 172.21.15.83:0/3837486859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14781115b0 con 0x5576ab032c60 2024-03-11T19:42:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.643+0000 7f14777fe640 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 0x5576ab04d970 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f146c00b930 tx=0x7f146c00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7520ba521acf9217 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.643+0000 7f14577fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.643+0000 7f14577fe640 1 -- 172.21.15.83:0/3837486859 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c005020 con 0x5576ab032c60 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.643+0000 7f14577fe640 1 -- 172.21.15.83:0/3837486859 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f146c005180 con 0x5576ab032c60 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.643+0000 7f1481620740 1 -- 172.21.15.83:0/3837486859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576aac9a7e0 con 0x5576ab032c60 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.643+0000 7f14577fe640 1 -- 172.21.15.83:0/3837486859 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f146c01f030 con 0x5576ab032c60 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.644+0000 7f1481620740 10 client.15903 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.644+0000 7f1481620740 1 -- 172.21.15.83:0/3837486859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5576aaf996a0 con 0x5576ab032c60 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.644+0000 7f14577fe640 1 -- 172.21.15.83:0/3837486859 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5576aac9a7e0 con 0x5576ab032c60 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.645+0000 7f14577fe640 1 -- 172.21.15.83:0/3837486859 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f146c0114d0 con 0x5576ab032c60 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.645+0000 7f1481620740 10 client.15903 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.645+0000 7f1481620740 10 client.15903 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.645+0000 7f1481620740 20 client.15903 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.645+0000 7f1481620740 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5576ab0596e0 0x5576ab05bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.645+0000 7f1481620740 4 client.15903 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.645+0000 7f1481620740 1 -- 172.21.15.83:0/3837486859 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576aacd7720 con 0x5576ab0596e0 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.645+0000 7f147cff4640 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5576ab0596e0 0x5576ab05bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.645+0000 7f147cff4640 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5576ab0596e0 0x5576ab05bb00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84a1c73d1fba2239 server_cookie=aaa802e9ce11103c in_seq=0 out_seq=0 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.645+0000 7f14577fe640 10 client.15903 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f14577fe640 1 -- 172.21.15.83:0/3837486859 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5576aacd7720 con 0x5576ab0596e0 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f14577fe640 10 client.15903 handle_command_reply: tid=0 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 1 client.15903 shutdown 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 1 -- 172.21.15.83:0/3837486859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 msgr2=0x5576ab04d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 0x5576ab04d970 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f146c00b930 tx=0x7f146c00bdf0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 1 -- 172.21.15.83:0/3837486859 shutdown_connections 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5576ab0596e0 0x5576ab05bb00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab037dd0 0x5576ab0549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab032c60 0x5576ab04d970 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 1 --2- 172.21.15.83:0/3837486859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab04d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 1 -- 172.21.15.83:0/3837486859 >> 172.21.15.83:0/3837486859 conn(0x5576ab02d7e0 msgr2=0x5576ab02cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f14577fe640 0 client.15903 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:42:20.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 1 -- 172.21.15.83:0/3837486859 shutdown_connections 2024-03-11T19:42:20.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 1 -- 172.21.15.83:0/3837486859 wait complete. 2024-03-11T19:42:20.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.646+0000 7f1481620740 20 client.15903 trim_cache size 0 max 16384 2024-03-11T19:42:20.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.665+0000 7f1481620740 1 Processor -- start 2024-03-11T19:42:20.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.665+0000 7f1481620740 1 -- start start 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f1481620740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab04d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f1481620740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab032c60 0x5576ab04d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f1481620740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 0x5576ab054970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f1481620740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576aaf20130 con 0x5576ab037dd0 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f1481620740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ab03a720 con 0x5576ab032270 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f1481620740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ab04e5b0 con 0x5576ab032c60 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f147cff4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 0x5576ab054970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f147cff4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 0x5576ab054970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54534/0 (socket says 172.21.15.83:54534) 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f147cff4640 1 -- 172.21.15.83:0/3058480343 learned_addr learned my addr 172.21.15.83:0/3058480343 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f14777fe640 1 --2- 172.21.15.83:0/3058480343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab032c60 0x5576ab04d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f147cff4640 1 --2- 172.21.15.83:0/3058480343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab04d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f147cff4640 1 -- 172.21.15.83:0/3058480343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab032c60 msgr2=0x5576ab04d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f147cff4640 1 --2- 172.21.15.83:0/3058480343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab032c60 0x5576ab04d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f147cff4640 1 -- 172.21.15.83:0/3058480343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 msgr2=0x5576ab04d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f147cff4640 1 --2- 172.21.15.83:0/3058480343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab04d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f147cff4640 1 -- 172.21.15.83:0/3058480343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576aaf35c20 con 0x5576ab037dd0 2024-03-11T19:42:20.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f147cff4640 1 --2- 172.21.15.83:0/3058480343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 0x5576ab054970 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f14700098d0 tx=0x7f147004d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113e2b59cc27e95b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.666+0000 7f145dffb640 1 -- 172.21.15.83:0/3058480343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470034440 con 0x5576ab037dd0 2024-03-11T19:42:20.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.667+0000 7f145dffb640 1 -- 172.21.15.83:0/3058480343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1470003e50 con 0x5576ab037dd0 2024-03-11T19:42:20.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.667+0000 7f145dffb640 1 -- 172.21.15.83:0/3058480343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14700567c0 con 0x5576ab037dd0 2024-03-11T19:42:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.667+0000 7f1481620740 1 -- 172.21.15.83:0/3058480343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 msgr2=0x5576ab054970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.667+0000 7f1481620740 1 --2- 172.21.15.83:0/3058480343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 0x5576ab054970 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f14700098d0 tx=0x7f147004d040 comp rx=0 tx=0).stop 2024-03-11T19:42:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.667+0000 7f1481620740 1 -- 172.21.15.83:0/3058480343 shutdown_connections 2024-03-11T19:42:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.667+0000 7f1481620740 1 --2- 172.21.15.83:0/3058480343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 0x5576ab054970 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.667+0000 7f1481620740 1 --2- 172.21.15.83:0/3058480343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab032c60 0x5576ab04d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.667+0000 7f1481620740 1 --2- 172.21.15.83:0/3058480343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab04d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.667+0000 7f1481620740 1 -- 172.21.15.83:0/3058480343 >> 172.21.15.83:0/3058480343 conn(0x5576ab02d7e0 msgr2=0x5576ab060460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.667+0000 7f1481620740 1 -- 172.21.15.83:0/3058480343 shutdown_connections 2024-03-11T19:42:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.667+0000 7f1481620740 1 -- 172.21.15.83:0/3058480343 wait complete. 2024-03-11T19:42:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f1481620740 1 -- start start 2024-03-11T19:42:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f1481620740 1 Processor -- start 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f1481620740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab0545a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f1481620740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab032c60 0x5576ab052bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f1481620740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 0x5576ab0530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f1481620740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ab053630 con 0x5576ab037dd0 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f1481620740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ab0537a0 con 0x5576ab032270 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f1481620740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576ab053960 con 0x5576ab032c60 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f1481620740 10 client.0 _ll_register_callbacks cb 0x7f1470037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f147cff4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 0x5576ab0530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f147cff4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 0x5576ab0530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54536/0 (socket says 172.21.15.83:54536) 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f147cff4640 1 -- 172.21.15.83:0/2489135714 learned_addr learned my addr 172.21.15.83:0/2489135714 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.668+0000 7f147cff4640 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab0545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.669+0000 7f14777fe640 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab032c60 0x5576ab052bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.669+0000 7f14777fe640 1 -- 172.21.15.83:0/2489135714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 msgr2=0x5576ab0545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.669+0000 7f14777fe640 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab0545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.669+0000 7f14777fe640 1 -- 172.21.15.83:0/2489135714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 msgr2=0x5576ab0530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.669+0000 7f14777fe640 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 0x5576ab0530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.669+0000 7f14777fe640 1 -- 172.21.15.83:0/2489135714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576aaf5d060 con 0x5576ab032c60 2024-03-11T19:42:20.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.670+0000 7f14777fe640 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab032c60 0x5576ab052bb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f146c005d00 tx=0x7f146c0043a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56bf02bfd1a06b14 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.670+0000 7f14577fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:42:20.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.670+0000 7f14577fe640 1 -- 172.21.15.83:0/2489135714 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c01f040 con 0x5576ab032c60 2024-03-11T19:42:20.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.670+0000 7f14577fe640 1 -- 172.21.15.83:0/2489135714 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f146c020070 con 0x5576ab032c60 2024-03-11T19:42:20.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f1481620740 1 -- 172.21.15.83:0/2489135714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14700327b0 con 0x5576ab032c60 2024-03-11T19:42:20.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f14577fe640 1 -- 172.21.15.83:0/2489135714 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f146c004590 con 0x5576ab032c60 2024-03-11T19:42:20.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f1481620740 10 client.25516 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:20.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f1481620740 1 -- 172.21.15.83:0/2489135714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5576aaf5b8c0 con 0x5576ab032c60 2024-03-11T19:42:20.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f14577fe640 1 -- 172.21.15.83:0/2489135714 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f146c03a030 con 0x5576ab032c60 2024-03-11T19:42:20.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f14577fe640 1 -- 172.21.15.83:0/2489135714 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f146c033020 con 0x5576ab032c60 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f1481620740 10 client.25516 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f1481620740 10 client.25516 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f1481620740 20 client.25516 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f1481620740 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5576ab0b12f0 0x5576ab0b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f1481620740 4 client.25516 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.671+0000 7f1481620740 1 -- 172.21.15.83:0/2489135714 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5576aad26890 con 0x5576ab0b12f0 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f147cff4640 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5576ab0b12f0 0x5576ab0b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f14577fe640 10 client.25516 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f147cff4640 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5576ab0b12f0 0x5576ab0b3710 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5de51fd7638b373 server_cookie=b31b7226515b778a in_seq=0 out_seq=0 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f14577fe640 1 -- 172.21.15.83:0/2489135714 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5576aad26890 con 0x5576ab0b12f0 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f14577fe640 10 client.25516 handle_command_reply: tid=0 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f1481620740 1 client.25516 shutdown 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f1481620740 1 -- 172.21.15.83:0/2489135714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab032c60 msgr2=0x5576ab052bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f1481620740 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab032c60 0x5576ab052bb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f146c005d00 tx=0x7f146c0043a0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f1481620740 1 -- 172.21.15.83:0/2489135714 shutdown_connections 2024-03-11T19:42:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f1481620740 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5576ab0b12f0 0x5576ab0b3710 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f1481620740 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576ab037dd0 0x5576ab0530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f14577fe640 0 client.25516 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:42:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f1481620740 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576ab032c60 0x5576ab052bb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f1481620740 1 --2- 172.21.15.83:0/2489135714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5576ab032270 0x5576ab0545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f1481620740 1 -- 172.21.15.83:0/2489135714 >> 172.21.15.83:0/2489135714 conn(0x5576ab02d7e0 msgr2=0x5576ab034120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f1481620740 1 -- 172.21.15.83:0/2489135714 shutdown_connections 2024-03-11T19:42:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.672+0000 7f1481620740 1 -- 172.21.15.83:0/2489135714 wait complete. 2024-03-11T19:42:20.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.673+0000 7f1481620740 20 client.25516 trim_cache size 0 max 16384 2024-03-11T19:42:20.681 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:42:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 -- 172.21.15.83:0/3285202465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14600a1b30 msgr2=0x7f14600a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14600a1b30 0x7f14600a3ff0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1468021cc0 tx=0x7f1468083f90 comp rx=0 tx=0).stop 2024-03-11T19:42:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 -- 172.21.15.83:0/3285202465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1478109290 msgr2=0x7f14781a8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1478109290 0x7f14781a8910 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f146800e9f0 tx=0x7f146800eeb0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 -- 172.21.15.83:0/3285202465 shutdown_connections 2024-03-11T19:42:20.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f14600a1b30 0x7f14600a3ff0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f147810e540 0x7f14781ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f147810bbc0 0x7f14781aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 --2- 172.21.15.83:0/3285202465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1478109290 0x7f14781a8910 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 -- 172.21.15.83:0/3285202465 >> 172.21.15.83:0/3285202465 conn(0x7f147806dfe0 msgr2=0x7f1478109f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 -- 172.21.15.83:0/3285202465 shutdown_connections 2024-03-11T19:42:20.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.674+0000 7f147ea57640 1 -- 172.21.15.83:0/3285202465 wait complete. 2024-03-11T19:42:20.691 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:42:20.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:42:20.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:20 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/405890939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:42:20.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.837+0000 7fa93a1c9640 1 -- 172.21.15.83:0/819364353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa934102dd0 msgr2=0x7fa934109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.837+0000 7fa93a1c9640 1 --2- 172.21.15.83:0/819364353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa934102dd0 0x7fa934109660 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009ca0 tx=0x7fa92c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.837+0000 7fa93a1c9640 1 -- 172.21.15.83:0/819364353 shutdown_connections 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.837+0000 7fa93a1c9640 1 --2- 172.21.15.83:0/819364353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa934102dd0 0x7fa934109660 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.837+0000 7fa93a1c9640 1 --2- 172.21.15.83:0/819364353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934102410 0x7fa934102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.837+0000 7fa93a1c9640 1 --2- 172.21.15.83:0/819364353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9340ff440 0x7fa9340ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.837+0000 7fa93a1c9640 1 -- 172.21.15.83:0/819364353 >> 172.21.15.83:0/819364353 conn(0x7fa9340fad70 msgr2=0x7fa9340fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.837+0000 7fa93a1c9640 1 -- 172.21.15.83:0/819364353 shutdown_connections 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.837+0000 7fa93a1c9640 1 -- 172.21.15.83:0/819364353 wait complete. 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa93a1c9640 1 Processor -- start 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa93a1c9640 1 -- start start 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa93a1c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9340ff440 0x7fa93419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa93a1c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934102410 0x7fa93419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa93a1c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934102dd0 0x7fa9341a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa93a1c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9341a5340 con 0x7fa9340ff440 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa93a1c9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9341a54b0 con 0x7fa934102dd0 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa93a1c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9341a5670 con 0x7fa934102410 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9340ff440 0x7fa93419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9340ff440 0x7fa93419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54550/0 (socket says 172.21.15.83:54550) 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa9337fe640 1 -- 172.21.15.83:0/832443528 learned_addr learned my addr 172.21.15.83:0/832443528 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa932ffd640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934102410 0x7fa93419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.838+0000 7fa933fff640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934102dd0 0x7fa9341a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa9337fe640 1 -- 172.21.15.83:0/832443528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934102410 msgr2=0x7fa93419e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa9337fe640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934102410 0x7fa93419e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa9337fe640 1 -- 172.21.15.83:0/832443528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934102dd0 msgr2=0x7fa9341a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa9337fe640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934102dd0 0x7fa9341a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa9337fe640 1 -- 172.21.15.83:0/832443528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c009900 con 0x7fa9340ff440 2024-03-11T19:42:20.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa932ffd640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934102410 0x7fa93419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:20.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa9337fe640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9340ff440 0x7fa93419dad0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa92400e9c0 tx=0x7fa92400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa930ff9640 1 -- 172.21.15.83:0/832443528 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240096e0 con 0x7fa9340ff440 2024-03-11T19:42:20.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa93a1c9640 1 -- 172.21.15.83:0/832443528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9341a5950 con 0x7fa9340ff440 2024-03-11T19:42:20.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa930ff9640 1 -- 172.21.15.83:0/832443528 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa92400cd80 con 0x7fa9340ff440 2024-03-11T19:42:20.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa930ff9640 1 -- 172.21.15.83:0/832443528 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924010430 con 0x7fa9340ff440 2024-03-11T19:42:20.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.839+0000 7fa93a1c9640 1 -- 172.21.15.83:0/832443528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9341a5e60 con 0x7fa9340ff440 2024-03-11T19:42:20.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa93cda0740 1 Processor -- start 2024-03-11T19:42:20.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa93cda0740 1 -- start start 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa930ff9640 1 -- 172.21.15.83:0/832443528 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa924003680 con 0x7fa9340ff440 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27270 0x55a521e27690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 0x55a521e2c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e2f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa93cda0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521de3960 con 0x55a521e2cdd0 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa93cda0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521d97310 con 0x55a521e27c60 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa93cda0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521d94840 con 0x55a521e27270 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa932ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e2f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa932ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e2f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54554/0 (socket says 172.21.15.83:54554) 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa932ffd640 1 -- 172.21.15.83:0/2751468203 learned_addr learned my addr 172.21.15.83:0/2751468203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa9337fe640 1 --2- 172.21.15.83:0/2751468203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 0x55a521e2c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.843+0000 7fa933fff640 1 --2- 172.21.15.83:0/2751468203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27270 0x55a521e27690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa932ffd640 1 -- 172.21.15.83:0/2751468203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27270 msgr2=0x55a521e27690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa932ffd640 1 --2- 172.21.15.83:0/2751468203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27270 0x55a521e27690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa932ffd640 1 -- 172.21.15.83:0/2751468203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 msgr2=0x55a521e2c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa932ffd640 1 --2- 172.21.15.83:0/2751468203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 0x55a521e2c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa930ff9640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1a30 0x7fa91c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa932ffd640 1 -- 172.21.15.83:0/2751468203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a521d52060 con 0x55a521e2cdd0 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa933fff640 1 --2- 172.21.15.83:0/2751468203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27270 0x55a521e27690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa930ff9640 1 -- 172.21.15.83:0/832443528 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa924014070 con 0x7fa9340ff440 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa933fff640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1a30 0x7fa91c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa932ffd640 1 --2- 172.21.15.83:0/2751468203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e2f260 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa928010480 tx=0x7fa92800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4516c9442195471f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa919ffb640 1 -- 172.21.15.83:0/2751468203 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928011760 con 0x55a521e2cdd0 2024-03-11T19:42:20.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa919ffb640 1 -- 172.21.15.83:0/2751468203 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa928015070 con 0x55a521e2cdd0 2024-03-11T19:42:20.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa933fff640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1a30 0x7fa91c0a3ef0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa92c02c630 tx=0x7fa92c0326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.844+0000 7fa919ffb640 1 -- 172.21.15.83:0/2751468203 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92801a380 con 0x55a521e2cdd0 2024-03-11T19:42:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.845+0000 7fa93cda0740 1 -- 172.21.15.83:0/2751468203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 msgr2=0x55a521e2f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.845+0000 7fa93cda0740 1 --2- 172.21.15.83:0/2751468203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e2f260 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa928010480 tx=0x7fa92800bf90 comp rx=0 tx=0).stop 2024-03-11T19:42:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.846+0000 7fa93cda0740 1 -- 172.21.15.83:0/2751468203 shutdown_connections 2024-03-11T19:42:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.846+0000 7fa93cda0740 1 --2- 172.21.15.83:0/2751468203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e2f260 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.846+0000 7fa93cda0740 1 --2- 172.21.15.83:0/2751468203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 0x55a521e2c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.846+0000 7fa93cda0740 1 --2- 172.21.15.83:0/2751468203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27270 0x55a521e27690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.846+0000 7fa93cda0740 1 -- 172.21.15.83:0/2751468203 >> 172.21.15.83:0/2751468203 conn(0x55a521e227e0 msgr2=0x55a521e22be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.846+0000 7fa93cda0740 1 -- 172.21.15.83:0/2751468203 shutdown_connections 2024-03-11T19:42:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.846+0000 7fa93cda0740 1 -- 172.21.15.83:0/2751468203 wait complete. 2024-03-11T19:42:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.846+0000 7fa93cda0740 1 -- start start 2024-03-11T19:42:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa93cda0740 1 Processor -- start 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 0x55a521e314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 0x55a521e31a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e38a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa93cda0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521d90020 con 0x55a521e2cdd0 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa93cda0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521a63430 con 0x55a521e27270 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa93cda0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521a1fe00 con 0x55a521e27c60 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa93cda0740 10 client.0 _ll_register_callbacks cb 0x55a521d37cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa933fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e38a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa932ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 0x55a521e31a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 0x55a521e314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa933fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e38a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54564/0 (socket says 172.21.15.83:54564) 2024-03-11T19:42:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa933fff640 1 -- 172.21.15.83:0/3458232414 learned_addr learned my addr 172.21.15.83:0/3458232414 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa9337fe640 1 -- 172.21.15.83:0/3458232414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 msgr2=0x55a521e31a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa9337fe640 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 0x55a521e31a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa9337fe640 1 -- 172.21.15.83:0/3458232414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 msgr2=0x55a521e38a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa9337fe640 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e38a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa9337fe640 1 -- 172.21.15.83:0/3458232414 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92800bc70 con 0x55a521e27270 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.847+0000 7fa933fff640 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e38a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa932ffd640 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 0x55a521e31a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa9337fe640 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 0x55a521e314e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55a521e286d0 tx=0x7fa9240d0af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58ac0cac2c3950eb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa912ffd640 1 -- 172.21.15.83:0/3458232414 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240ba2c0 con 0x55a521e27270 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa912ffd640 1 -- 172.21.15.83:0/3458232414 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa92409ae00 con 0x55a521e27270 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa93cda0740 1 -- 172.21.15.83:0/3458232414 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a521a8f7e0 con 0x55a521e27270 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa912ffd640 1 -- 172.21.15.83:0/3458232414 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa92409a240 con 0x55a521e27270 2024-03-11T19:42:20.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa93cda0740 10 client.25418 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa93cda0740 1 -- 172.21.15.83:0/3458232414 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a521d8e6a0 con 0x55a521e27270 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa912ffd640 1 -- 172.21.15.83:0/3458232414 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a521a8f7e0 con 0x55a521e27270 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa912ffd640 1 -- 172.21.15.83:0/3458232414 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa92401f060 con 0x55a521e27270 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa93cda0740 10 client.25418 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa93cda0740 10 client.25418 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa93cda0740 20 client.25418 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.848+0000 7fa93cda0740 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a521e3d670 0x55a521e3fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.849+0000 7fa93cda0740 4 client.25418 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.849+0000 7fa93cda0740 1 -- 172.21.15.83:0/3458232414 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a521acc720 con 0x55a521e3d670 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.849+0000 7fa932ffd640 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a521e3d670 0x55a521e3fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.849+0000 7fa932ffd640 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a521e3d670 0x55a521e3fab0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ff789dac91b4986 server_cookie=4835fbd47f79d998 in_seq=0 out_seq=0 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.849+0000 7fa912ffd640 10 client.25418 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:42:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.850+0000 7fa912ffd640 1 -- 172.21.15.83:0/3458232414 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a521acc720 con 0x55a521e3d670 2024-03-11T19:42:20.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.850+0000 7fa912ffd640 10 client.25418 handle_command_reply: tid=0 2024-03-11T19:42:20.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.851+0000 7fa93cda0740 1 client.25418 shutdown 2024-03-11T19:42:20.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.851+0000 7fa93cda0740 1 -- 172.21.15.83:0/3458232414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 msgr2=0x55a521e314e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.851+0000 7fa93cda0740 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 0x55a521e314e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55a521e286d0 tx=0x7fa9240d0af0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.851+0000 7fa93cda0740 1 -- 172.21.15.83:0/3458232414 shutdown_connections 2024-03-11T19:42:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.851+0000 7fa93cda0740 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a521e3d670 0x55a521e3fab0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.851+0000 7fa93cda0740 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e38a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.851+0000 7fa93cda0740 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 0x55a521e31a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.851+0000 7fa93cda0740 1 --2- 172.21.15.83:0/3458232414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 0x55a521e314e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.851+0000 7fa912ffd640 0 client.25418 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:42:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.851+0000 7fa93cda0740 1 -- 172.21.15.83:0/3458232414 >> 172.21.15.83:0/3458232414 conn(0x55a521e227e0 msgr2=0x55a521e21b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.851+0000 7fa93cda0740 1 -- 172.21.15.83:0/3458232414 shutdown_connections 2024-03-11T19:42:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.852+0000 7fa93cda0740 1 -- 172.21.15.83:0/3458232414 wait complete. 2024-03-11T19:42:20.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.852+0000 7fa93cda0740 20 client.25418 trim_cache size 0 max 16384 2024-03-11T19:42:20.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa93cda0740 1 Processor -- start 2024-03-11T19:42:20.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa93cda0740 1 -- start start 2024-03-11T19:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27270 0x55a521e2f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 0x55a521e38a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e1da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa93cda0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521d15130 con 0x55a521e2cdd0 2024-03-11T19:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa93cda0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521acc720 con 0x55a521e27c60 2024-03-11T19:42:20.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa93cda0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521a8f7e0 con 0x55a521e27270 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa933fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e1da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa932ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 0x55a521e38a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa932ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 0x55a521e38a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40564/0 (socket says 172.21.15.83:40564) 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa933fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e1da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54572/0 (socket says 172.21.15.83:54572) 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa932ffd640 1 -- 172.21.15.83:0/1824769076 learned_addr learned my addr 172.21.15.83:0/1824769076 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.868+0000 7fa9337fe640 1 --2- 172.21.15.83:0/1824769076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27270 0x55a521e2f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.869+0000 7fa932ffd640 1 -- 172.21.15.83:0/1824769076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27270 msgr2=0x55a521e2f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.869+0000 7fa932ffd640 1 --2- 172.21.15.83:0/1824769076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27270 0x55a521e2f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.869+0000 7fa932ffd640 1 -- 172.21.15.83:0/1824769076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 msgr2=0x55a521e1da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.869+0000 7fa932ffd640 1 --2- 172.21.15.83:0/1824769076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e1da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.869+0000 7fa932ffd640 1 -- 172.21.15.83:0/1824769076 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a521d2ac20 con 0x55a521e27c60 2024-03-11T19:42:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.869+0000 7fa932ffd640 1 --2- 172.21.15.83:0/1824769076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 0x55a521e38a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa92801e010 tx=0x7fa92801c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75880aa696c8086e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.869+0000 7fa9197fa640 1 -- 172.21.15.83:0/1824769076 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928021050 con 0x55a521e27c60 2024-03-11T19:42:20.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.869+0000 7fa9197fa640 1 -- 172.21.15.83:0/1824769076 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa92802d070 con 0x55a521e27c60 2024-03-11T19:42:20.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.869+0000 7fa9197fa640 1 -- 172.21.15.83:0/1824769076 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928004660 con 0x55a521e27c60 2024-03-11T19:42:20.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.870+0000 7fa93cda0740 1 -- 172.21.15.83:0/1824769076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 msgr2=0x55a521e38a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.870+0000 7fa93cda0740 1 --2- 172.21.15.83:0/1824769076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 0x55a521e38a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa92801e010 tx=0x7fa92801c7b0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.870+0000 7fa93cda0740 1 -- 172.21.15.83:0/1824769076 shutdown_connections 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.870+0000 7fa93cda0740 1 --2- 172.21.15.83:0/1824769076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e1da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.870+0000 7fa93cda0740 1 --2- 172.21.15.83:0/1824769076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27c60 0x55a521e38a20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.870+0000 7fa93cda0740 1 --2- 172.21.15.83:0/1824769076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27270 0x55a521e2f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.870+0000 7fa93cda0740 1 -- 172.21.15.83:0/1824769076 >> 172.21.15.83:0/1824769076 conn(0x55a521e227e0 msgr2=0x55a521e2ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.870+0000 7fa93cda0740 1 -- 172.21.15.83:0/1824769076 shutdown_connections 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.871+0000 7fa93cda0740 1 -- 172.21.15.83:0/1824769076 wait complete. 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.871+0000 7fa93cda0740 1 -- start start 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.871+0000 7fa93cda0740 1 Processor -- start 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.871+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 0x55a521e36960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.871+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 0x55a521e36ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.871+0000 7fa93cda0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e37eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.871+0000 7fa93cda0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521e383f0 con 0x55a521e2cdd0 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.871+0000 7fa93cda0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521e38560 con 0x55a521e27270 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.871+0000 7fa93cda0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a521e38720 con 0x55a521e27c60 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa93cda0740 10 client.0 _ll_register_callbacks cb 0x7fa92800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa933fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e37eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa932ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 0x55a521e36ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 0x55a521e36960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 0x55a521e36960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40576/0 (socket says 172.21.15.83:40576) 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa932ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 0x55a521e36ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37528/0 (socket says 172.21.15.83:37528) 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa9337fe640 1 -- 172.21.15.83:0/672754706 learned_addr learned my addr 172.21.15.83:0/672754706 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa932ffd640 1 -- 172.21.15.83:0/672754706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 msgr2=0x55a521e36960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa932ffd640 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 0x55a521e36960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa932ffd640 1 -- 172.21.15.83:0/672754706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 msgr2=0x55a521e37eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa932ffd640 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e37eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa932ffd640 1 -- 172.21.15.83:0/672754706 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a521d52060 con 0x55a521e27c60 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa9337fe640 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 0x55a521e36960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa933fff640 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e37eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa932ffd640 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 0x55a521e36ea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa92801c820 tx=0x7fa928005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810e4b9c9ed540a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa912ffd640 1 -- 172.21.15.83:0/672754706 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928021040 con 0x55a521e27c60 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa912ffd640 1 -- 172.21.15.83:0/672754706 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa928017070 con 0x55a521e27c60 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.872+0000 7fa93cda0740 1 -- 172.21.15.83:0/672754706 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a521e2cb80 con 0x55a521e27c60 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa912ffd640 1 -- 172.21.15.83:0/672754706 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa928005180 con 0x55a521e27c60 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa93cda0740 10 client.25534 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa93cda0740 1 -- 172.21.15.83:0/672754706 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a521d508c0 con 0x55a521e27c60 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa912ffd640 1 -- 172.21.15.83:0/672754706 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa928044020 con 0x55a521e27c60 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa912ffd640 1 -- 172.21.15.83:0/672754706 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa928004930 con 0x55a521e27c60 2024-03-11T19:42:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa93cda0740 10 client.25534 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa93cda0740 10 client.25534 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa93cda0740 20 client.25534 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa93cda0740 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a521e94a10 0x55a521e96e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa93cda0740 4 client.25534 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa93cda0740 1 -- 172.21.15.83:0/672754706 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fa928015070 con 0x55a521e94a10 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.873+0000 7fa9337fe640 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a521e94a10 0x55a521e96e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa912ffd640 10 client.25534 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa9337fe640 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a521e94a10 0x55a521e96e30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed2fa7b7729d26f9 server_cookie=e7dc325db370277e in_seq=0 out_seq=0 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa912ffd640 1 -- 172.21.15.83:0/672754706 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fa928015070 con 0x55a521e94a10 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa912ffd640 10 client.25534 handle_command_reply: tid=0 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa93cda0740 1 client.25534 shutdown 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa93cda0740 1 -- 172.21.15.83:0/672754706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 msgr2=0x55a521e36ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa93cda0740 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 0x55a521e36ea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa92801c820 tx=0x7fa928005bd0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa93cda0740 1 -- 172.21.15.83:0/672754706 shutdown_connections 2024-03-11T19:42:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa93cda0740 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a521e94a10 0x55a521e96e30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa93cda0740 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a521e2cdd0 0x55a521e37eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa93cda0740 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a521e27c60 0x55a521e36ea0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa93cda0740 1 --2- 172.21.15.83:0/672754706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a521e27270 0x55a521e36960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa93cda0740 1 -- 172.21.15.83:0/672754706 >> 172.21.15.83:0/672754706 conn(0x55a521e227e0 msgr2=0x55a521e1da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.874+0000 7fa912ffd640 0 client.25534 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:42:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.875+0000 7fa93cda0740 1 -- 172.21.15.83:0/672754706 shutdown_connections 2024-03-11T19:42:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.875+0000 7fa93cda0740 1 -- 172.21.15.83:0/672754706 wait complete. 2024-03-11T19:42:20.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.875+0000 7fa93cda0740 20 client.25534 trim_cache size 0 max 16384 2024-03-11T19:42:20.885 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:42:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 -- 172.21.15.83:0/832443528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1a30 msgr2=0x7fa91c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1a30 0x7fa91c0a3ef0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa92c02c630 tx=0x7fa92c0326b0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 -- 172.21.15.83:0/832443528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9340ff440 msgr2=0x7fa93419dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9340ff440 0x7fa93419dad0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa92400e9c0 tx=0x7fa92400ee80 comp rx=0 tx=0).stop 2024-03-11T19:42:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 -- 172.21.15.83:0/832443528 shutdown_connections 2024-03-11T19:42:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa91c0a1a30 0x7fa91c0a3ef0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa934102dd0 0x7fa9341a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa934102410 0x7fa93419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 --2- 172.21.15.83:0/832443528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9340ff440 0x7fa93419dad0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 -- 172.21.15.83:0/832443528 >> 172.21.15.83:0/832443528 conn(0x7fa9340fad70 msgr2=0x7fa9340fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 -- 172.21.15.83:0/832443528 shutdown_connections 2024-03-11T19:42:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:20.877+0000 7fa93a1c9640 1 -- 172.21.15.83:0/832443528 wait complete. 2024-03-11T19:42:20.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:20 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/405890939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:42:20.895 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:42:20.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:42:20.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:20 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/405890939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:42:21.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.042+0000 7f006d3bc640 1 -- 172.21.15.83:0/334369030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00680ff440 msgr2=0x7f00680ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:21.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.042+0000 7f006d3bc640 1 --2- 172.21.15.83:0/334369030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00680ff440 0x7f00680ff860 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0060009ca0 tx=0x7f006002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.043+0000 7f006d3bc640 1 -- 172.21.15.83:0/334369030 shutdown_connections 2024-03-11T19:42:21.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.043+0000 7f006d3bc640 1 --2- 172.21.15.83:0/334369030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0068100b60 0x7f0068105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.043+0000 7f006d3bc640 1 --2- 172.21.15.83:0/334369030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00681001a0 0x7f0068100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.043+0000 7f006d3bc640 1 --2- 172.21.15.83:0/334369030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00680ff440 0x7f00680ff860 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.043+0000 7f006d3bc640 1 -- 172.21.15.83:0/334369030 >> 172.21.15.83:0/334369030 conn(0x7f00680fad70 msgr2=0x7f00680fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:21.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.043+0000 7f006d3bc640 1 -- 172.21.15.83:0/334369030 shutdown_connections 2024-03-11T19:42:21.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.043+0000 7f006d3bc640 1 -- 172.21.15.83:0/334369030 wait complete. 2024-03-11T19:42:21.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.043+0000 7f006d3bc640 1 Processor -- start 2024-03-11T19:42:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f006d3bc640 1 -- start start 2024-03-11T19:42:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f006d3bc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00680ff440 0x7f006819d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f006d3bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00681001a0 0x7f006819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f006d3bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0068100b60 0x7f00681a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f006d3bc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681a5340 con 0x7f00681001a0 2024-03-11T19:42:21.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f006d3bc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681a54b0 con 0x7f0068100b60 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f006d3bc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681a5670 con 0x7f00680ff440 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00680ff440 0x7f006819d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f00667fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00681001a0 0x7f006819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00680ff440 0x7f006819d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37544/0 (socket says 172.21.15.83:37544) 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f0066ffd640 1 -- 172.21.15.83:0/1814071431 learned_addr learned my addr 172.21.15.83:0/1814071431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f00677fe640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0068100b60 0x7f00681a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f0066ffd640 1 -- 172.21.15.83:0/1814071431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0068100b60 msgr2=0x7f00681a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f0066ffd640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0068100b60 0x7f00681a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f0066ffd640 1 -- 172.21.15.83:0/1814071431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00681001a0 msgr2=0x7f006819dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f0066ffd640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00681001a0 0x7f006819dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.044+0000 7f0066ffd640 1 -- 172.21.15.83:0/1814071431 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060009900 con 0x7f00680ff440 2024-03-11T19:42:21.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.045+0000 7f0066ffd640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00680ff440 0x7f006819d8a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0060009dc0 tx=0x7f0060034b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.045+0000 7f005ffff640 1 -- 172.21.15.83:0/1814071431 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060034dc0 con 0x7f00680ff440 2024-03-11T19:42:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.045+0000 7f005ffff640 1 -- 172.21.15.83:0/1814071431 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0060003e50 con 0x7f00680ff440 2024-03-11T19:42:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.045+0000 7f006d3bc640 1 -- 172.21.15.83:0/1814071431 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00681a58f0 con 0x7f00680ff440 2024-03-11T19:42:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.045+0000 7f005ffff640 1 -- 172.21.15.83:0/1814071431 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600048a0 con 0x7f00680ff440 2024-03-11T19:42:21.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.045+0000 7f006d3bc640 1 -- 172.21.15.83:0/1814071431 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00681a5ce0 con 0x7f00680ff440 2024-03-11T19:42:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.048+0000 7f005ffff640 1 -- 172.21.15.83:0/1814071431 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f006000a040 con 0x7f00680ff440 2024-03-11T19:42:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.048+0000 7f005ffff640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00540a1ab0 0x7f00540a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.049+0000 7f005ffff640 1 -- 172.21.15.83:0/1814071431 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f00600f0230 con 0x7f00680ff440 2024-03-11T19:42:21.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.049+0000 7f00667fc640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00540a1ab0 0x7f00540a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.050+0000 7f00667fc640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00540a1ab0 0x7f00540a3f70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f006819ed60 tx=0x7f0050005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:21.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.050+0000 7f006ff93740 1 Processor -- start 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f006ff93740 1 -- start start 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f006ff93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231af270 0x55a2231af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f006ff93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 0x55a2231b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f006ff93740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231b4dd0 0x55a2231b7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f006ff93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22316b960 con 0x55a2231af270 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f006ff93740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22311f310 con 0x55a2231b4dd0 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f006ff93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22311c840 con 0x55a2231afc60 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231af270 0x55a2231af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231af270 0x55a2231af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54592/0 (socket says 172.21.15.83:54592) 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f0066ffd640 1 -- 172.21.15.83:0/2266816156 learned_addr learned my addr 172.21.15.83:0/2266816156 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f00677fe640 1 --2- 172.21.15.83:0/2266816156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231b4dd0 0x55a2231b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f00667fc640 1 --2- 172.21.15.83:0/2266816156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 0x55a2231b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f0066ffd640 1 -- 172.21.15.83:0/2266816156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 msgr2=0x55a2231b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f0066ffd640 1 --2- 172.21.15.83:0/2266816156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 0x55a2231b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f0066ffd640 1 -- 172.21.15.83:0/2266816156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231b4dd0 msgr2=0x55a2231b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f0066ffd640 1 --2- 172.21.15.83:0/2266816156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231b4dd0 0x55a2231b7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f0066ffd640 1 -- 172.21.15.83:0/2266816156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2230da060 con 0x55a2231af270 2024-03-11T19:42:21.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.051+0000 7f00667fc640 1 --2- 172.21.15.83:0/2266816156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 0x55a2231b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:42:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.052+0000 7f0066ffd640 1 --2- 172.21.15.83:0/2266816156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231af270 0x55a2231af690 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f00600bceb0 tx=0x7f0060000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66b33f996987aed server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.052+0000 7f005d7fa640 1 -- 172.21.15.83:0/2266816156 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600c1120 con 0x55a2231af270 2024-03-11T19:42:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.052+0000 7f005d7fa640 1 -- 172.21.15.83:0/2266816156 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00600c1300 con 0x55a2231af270 2024-03-11T19:42:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.052+0000 7f005d7fa640 1 -- 172.21.15.83:0/2266816156 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060004260 con 0x55a2231af270 2024-03-11T19:42:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.053+0000 7f006ff93740 1 -- 172.21.15.83:0/2266816156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231af270 msgr2=0x55a2231af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.053+0000 7f006ff93740 1 --2- 172.21.15.83:0/2266816156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231af270 0x55a2231af690 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f00600bceb0 tx=0x7f0060000f40 comp rx=0 tx=0).stop 2024-03-11T19:42:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.053+0000 7f006ff93740 1 -- 172.21.15.83:0/2266816156 shutdown_connections 2024-03-11T19:42:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.053+0000 7f006ff93740 1 --2- 172.21.15.83:0/2266816156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231b4dd0 0x55a2231b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.053+0000 7f006ff93740 1 --2- 172.21.15.83:0/2266816156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 0x55a2231b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.053+0000 7f006ff93740 1 --2- 172.21.15.83:0/2266816156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231af270 0x55a2231af690 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.053+0000 7f006ff93740 1 -- 172.21.15.83:0/2266816156 >> 172.21.15.83:0/2266816156 conn(0x55a2231aa7e0 msgr2=0x55a2231aabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.053+0000 7f006ff93740 1 -- 172.21.15.83:0/2266816156 shutdown_connections 2024-03-11T19:42:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.053+0000 7f006ff93740 1 -- 172.21.15.83:0/2266816156 wait complete. 2024-03-11T19:42:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f006ff93740 1 -- start start 2024-03-11T19:42:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f006ff93740 1 Processor -- start 2024-03-11T19:42:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f006ff93740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231af270 0x55a2231b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f006ff93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 0x55a2231b97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f006ff93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231b4dd0 0x55a2231c0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f006ff93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a223118020 con 0x55a2231afc60 2024-03-11T19:42:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f006ff93740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a222deb430 con 0x55a2231af270 2024-03-11T19:42:21.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f006ff93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a222da7e00 con 0x55a2231b4dd0 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f006ff93740 10 client.0 _ll_register_callbacks cb 0x55a2230bfcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f00667fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 0x55a2231b97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f00667fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 0x55a2231b97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54600/0 (socket says 172.21.15.83:54600) 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f00667fc640 1 -- 172.21.15.83:0/3006640446 learned_addr learned my addr 172.21.15.83:0/3006640446 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231af270 0x55a2231b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f00677fe640 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231b4dd0 0x55a2231c0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f00667fc640 1 -- 172.21.15.83:0/3006640446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231b4dd0 msgr2=0x55a2231c0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f00667fc640 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231b4dd0 0x55a2231c0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f00667fc640 1 -- 172.21.15.83:0/3006640446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231af270 msgr2=0x55a2231b92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f00667fc640 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231af270 0x55a2231b92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f00667fc640 1 -- 172.21.15.83:0/3006640446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00681a5ce0 con 0x55a2231afc60 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f0066ffd640 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231af270 0x55a2231b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f00667fc640 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 0x55a2231b97e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0050012780 tx=0x7f0050004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c030687e53942c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:21.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.054+0000 7f004e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f004e7fc640 1 -- 172.21.15.83:0/3006640446 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005001f940 con 0x55a2231afc60 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f004e7fc640 1 -- 172.21.15.83:0/3006640446 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0050018070 con 0x55a2231afc60 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f006ff93740 1 -- 172.21.15.83:0/3006640446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a222e177e0 con 0x55a2231afc60 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f004e7fc640 1 -- 172.21.15.83:0/3006640446 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f005001faa0 con 0x55a2231afc60 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f006ff93740 10 client.15939 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f006ff93740 1 -- 172.21.15.83:0/3006640446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a2231166a0 con 0x55a2231afc60 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f004e7fc640 1 -- 172.21.15.83:0/3006640446 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a222e177e0 con 0x55a2231afc60 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f004e7fc640 1 -- 172.21.15.83:0/3006640446 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0050029d10 con 0x55a2231afc60 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f006ff93740 10 client.15939 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f006ff93740 10 client.15939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f006ff93740 20 client.15939 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f006ff93740 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a2231c5540 0x55a2231c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f006ff93740 4 client.15939 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:21.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f006ff93740 1 -- 172.21.15.83:0/3006640446 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a222e54720 con 0x55a2231c5540 2024-03-11T19:42:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.055+0000 7f0066ffd640 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a2231c5540 0x55a2231c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.056+0000 7f0066ffd640 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a2231c5540 0x55a2231c7960 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7e8b5369e030545 server_cookie=2ae121d299632da1 in_seq=0 out_seq=0 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.056+0000 7f004e7fc640 10 client.15939 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.057+0000 7f004e7fc640 1 -- 172.21.15.83:0/3006640446 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a222e54720 con 0x55a2231c5540 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.057+0000 7f004e7fc640 10 client.15939 handle_command_reply: tid=0 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.057+0000 7f006ff93740 1 client.15939 shutdown 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.057+0000 7f006ff93740 1 -- 172.21.15.83:0/3006640446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 msgr2=0x55a2231b97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.057+0000 7f006ff93740 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 0x55a2231b97e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0050012780 tx=0x7f0050004730 comp rx=0 tx=0).stop 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.057+0000 7f006ff93740 1 -- 172.21.15.83:0/3006640446 shutdown_connections 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.057+0000 7f006ff93740 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a2231c5540 0x55a2231c7960 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.057+0000 7f006ff93740 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231b4dd0 0x55a2231c0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.057+0000 7f006ff93740 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 0x55a2231b97e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.057+0000 7f006ff93740 1 --2- 172.21.15.83:0/3006640446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231af270 0x55a2231b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.057+0000 7f006ff93740 1 -- 172.21.15.83:0/3006640446 >> 172.21.15.83:0/3006640446 conn(0x55a2231aa7e0 msgr2=0x55a2231a9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:21.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.058+0000 7f004e7fc640 0 client.15939 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.058+0000 7f006ff93740 1 -- 172.21.15.83:0/3006640446 shutdown_connections 2024-03-11T19:42:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.058+0000 7f006ff93740 1 -- 172.21.15.83:0/3006640446 wait complete. 2024-03-11T19:42:21.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.058+0000 7f006ff93740 20 client.15939 trim_cache size 0 max 16384 2024-03-11T19:42:21.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.073+0000 7f006ff93740 1 Processor -- start 2024-03-11T19:42:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.073+0000 7f006ff93740 1 -- start start 2024-03-11T19:42:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.074+0000 7f006ff93740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231af270 0x55a2231b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.074+0000 7f006ff93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 0x55a2231b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.074+0000 7f006ff93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231b4dd0 0x55a2231c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.074+0000 7f006ff93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22309d130 con 0x55a2231b4dd0 2024-03-11T19:42:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.074+0000 7f006ff93740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a222e54750 con 0x55a2231af270 2024-03-11T19:42:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.074+0000 7f006ff93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2231b0350 con 0x55a2231afc60 2024-03-11T19:42:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.074+0000 7f00667fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 0x55a2231b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231af270 0x55a2231b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f00667fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 0x55a2231b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37570/0 (socket says 172.21.15.83:37570) 2024-03-11T19:42:21.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f00667fc640 1 -- 172.21.15.83:0/1364105557 learned_addr learned my addr 172.21.15.83:0/1364105557 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f00677fe640 1 --2- 172.21.15.83:0/1364105557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231b4dd0 0x55a2231c07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f00667fc640 1 -- 172.21.15.83:0/1364105557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231af270 msgr2=0x55a2231b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f00667fc640 1 --2- 172.21.15.83:0/1364105557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231af270 0x55a2231b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f00667fc640 1 -- 172.21.15.83:0/1364105557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231b4dd0 msgr2=0x55a2231c07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f00667fc640 1 --2- 172.21.15.83:0/1364105557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231b4dd0 0x55a2231c07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f00667fc640 1 -- 172.21.15.83:0/1364105557 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2230b2c20 con 0x55a2231afc60 2024-03-11T19:42:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f00667fc640 1 --2- 172.21.15.83:0/1364105557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 0x55a2231b16f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0050020450 tx=0x7f0050000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de555ab6f21b5272 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f005cff9640 1 -- 172.21.15.83:0/1364105557 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050009e80 con 0x55a2231afc60 2024-03-11T19:42:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f005cff9640 1 -- 172.21.15.83:0/1364105557 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0050024070 con 0x55a2231afc60 2024-03-11T19:42:21.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.075+0000 7f005cff9640 1 -- 172.21.15.83:0/1364105557 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0050020df0 con 0x55a2231afc60 2024-03-11T19:42:21.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.076+0000 7f006ff93740 1 -- 172.21.15.83:0/1364105557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 msgr2=0x55a2231b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.076+0000 7f006ff93740 1 --2- 172.21.15.83:0/1364105557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 0x55a2231b16f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0050020450 tx=0x7f0050000bc0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.076+0000 7f006ff93740 1 -- 172.21.15.83:0/1364105557 shutdown_connections 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.076+0000 7f006ff93740 1 --2- 172.21.15.83:0/1364105557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231b4dd0 0x55a2231c07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.076+0000 7f006ff93740 1 --2- 172.21.15.83:0/1364105557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231afc60 0x55a2231b16f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.076+0000 7f006ff93740 1 --2- 172.21.15.83:0/1364105557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231af270 0x55a2231b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.076+0000 7f006ff93740 1 -- 172.21.15.83:0/1364105557 >> 172.21.15.83:0/1364105557 conn(0x55a2231aa7e0 msgr2=0x55a2231b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.076+0000 7f006ff93740 1 -- 172.21.15.83:0/1364105557 shutdown_connections 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.076+0000 7f006ff93740 1 -- 172.21.15.83:0/1364105557 wait complete. 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.077+0000 7f006ff93740 1 -- start start 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.077+0000 7f006ff93740 1 Processor -- start 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.077+0000 7f006ff93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231af270 0x55a2231bf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.077+0000 7f006ff93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 0x55a2231bf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.077+0000 7f006ff93740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231b4dd0 0x55a2231bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.077+0000 7f006ff93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22322cc20 con 0x55a2231afc60 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.077+0000 7f006ff93740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22322cd90 con 0x55a2231b4dd0 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.077+0000 7f006ff93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22322cf50 con 0x55a2231af270 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.077+0000 7f006ff93740 10 client.0 _ll_register_callbacks cb 0x7f0060032bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231af270 0x55a2231bf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231af270 0x55a2231bf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37586/0 (socket says 172.21.15.83:37586) 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f0066ffd640 1 -- 172.21.15.83:0/224567445 learned_addr learned my addr 172.21.15.83:0/224567445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f00677fe640 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231b4dd0 0x55a2231bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f00667fc640 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 0x55a2231bf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f0066ffd640 1 -- 172.21.15.83:0/224567445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231b4dd0 msgr2=0x55a2231bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:21.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f0066ffd640 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231b4dd0 0x55a2231bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f0066ffd640 1 -- 172.21.15.83:0/224567445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 msgr2=0x55a2231bf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f0066ffd640 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 0x55a2231bf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f0066ffd640 1 -- 172.21.15.83:0/224567445 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2231166a0 con 0x55a2231af270 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f00667fc640 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 0x55a2231bf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f0066ffd640 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231af270 0x55a2231bf400 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00600bcd10 tx=0x7f00600bda10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac47e9f19982f237 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f004e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f004e7fc640 1 -- 172.21.15.83:0/224567445 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600b9820 con 0x55a2231af270 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f004e7fc640 1 -- 172.21.15.83:0/224567445 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f00600b9980 con 0x55a2231af270 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.078+0000 7f006ff93740 1 -- 172.21.15.83:0/224567445 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00600c1300 con 0x55a2231af270 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f004e7fc640 1 -- 172.21.15.83:0/224567445 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00600bd410 con 0x55a2231af270 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f006ff93740 10 client.25552 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f006ff93740 1 -- 172.21.15.83:0/224567445 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a2230d88c0 con 0x55a2231af270 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f004e7fc640 1 -- 172.21.15.83:0/224567445 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00600c1300 con 0x55a2231af270 2024-03-11T19:42:21.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f004e7fc640 1 -- 172.21.15.83:0/224567445 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f00600a1d20 con 0x55a2231af270 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f006ff93740 10 client.25552 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f006ff93740 10 client.25552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f006ff93740 20 client.25552 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f006ff93740 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a22322e470 0x55a223230890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f006ff93740 4 client.25552 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f006ff93740 1 -- 172.21.15.83:0/224567445 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a222ea3890 con 0x55a22322e470 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.079+0000 7f00667fc640 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a22322e470 0x55a223230890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f004e7fc640 10 client.25552 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f00667fc640 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a22322e470 0x55a223230890 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3098eb844cec1bc server_cookie=ba26a76696d8a343 in_seq=0 out_seq=0 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f004e7fc640 1 -- 172.21.15.83:0/224567445 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a222ea3890 con 0x55a22322e470 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f004e7fc640 10 client.25552 handle_command_reply: tid=0 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 1 client.25552 shutdown 2024-03-11T19:42:21.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 1 -- 172.21.15.83:0/224567445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231af270 msgr2=0x55a2231bf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231af270 0x55a2231bf400 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00600bcd10 tx=0x7f00600bda10 comp rx=0 tx=0).stop 2024-03-11T19:42:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 1 -- 172.21.15.83:0/224567445 shutdown_connections 2024-03-11T19:42:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a22322e470 0x55a223230890 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2231b4dd0 0x55a2231bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f004e7fc640 0 client.25552 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2231afc60 0x55a2231bf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 1 --2- 172.21.15.83:0/224567445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a2231af270 0x55a2231bf400 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 1 -- 172.21.15.83:0/224567445 >> 172.21.15.83:0/224567445 conn(0x55a2231aa7e0 msgr2=0x55a2231b9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 1 -- 172.21.15.83:0/224567445 shutdown_connections 2024-03-11T19:42:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 1 -- 172.21.15.83:0/224567445 wait complete. 2024-03-11T19:42:21.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.080+0000 7f006ff93740 20 client.25552 trim_cache size 0 max 16384 2024-03-11T19:42:21.091 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:42:21.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.081+0000 7f006d3bc640 1 -- 172.21.15.83:0/1814071431 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00540a1ab0 msgr2=0x7f00540a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.081+0000 7f006d3bc640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00540a1ab0 0x7f00540a3f70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f006819ed60 tx=0x7f0050005f90 comp rx=0 tx=0).stop 2024-03-11T19:42:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.081+0000 7f006d3bc640 1 -- 172.21.15.83:0/1814071431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00680ff440 msgr2=0x7f006819d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.081+0000 7f006d3bc640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00680ff440 0x7f006819d8a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0060009dc0 tx=0x7f0060034b10 comp rx=0 tx=0).stop 2024-03-11T19:42:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.081+0000 7f006d3bc640 1 -- 172.21.15.83:0/1814071431 shutdown_connections 2024-03-11T19:42:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.081+0000 7f006d3bc640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00540a1ab0 0x7f00540a3f70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.081+0000 7f006d3bc640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0068100b60 0x7f00681a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.081+0000 7f006d3bc640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00681001a0 0x7f006819dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.081+0000 7f006d3bc640 1 --2- 172.21.15.83:0/1814071431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00680ff440 0x7f006819d8a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.081+0000 7f006d3bc640 1 -- 172.21.15.83:0/1814071431 >> 172.21.15.83:0/1814071431 conn(0x7f00680fad70 msgr2=0x7f00681013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.081+0000 7f006d3bc640 1 -- 172.21.15.83:0/1814071431 shutdown_connections 2024-03-11T19:42:21.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:21.083+0000 7f006d3bc640 1 -- 172.21.15.83:0/1814071431 wait complete. 2024-03-11T19:42:21.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:21 smithi083 ceph-mon[31864]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 825 op/s 2024-03-11T19:42:21.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:21 smithi137 ceph-mon[36220]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 825 op/s 2024-03-11T19:42:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:21 smithi204 ceph-mon[36739]: pgmap v443: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 825 op/s 2024-03-11T19:42:22.100 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:42:22.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:42:22.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:42:22.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.266+0000 7f29684f4640 1 -- 172.21.15.83:0/3823598916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2960101c30 msgr2=0x7f29601084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.266+0000 7f29684f4640 1 --2- 172.21.15.83:0/3823598916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2960101c30 0x7f29601084c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f295801acb0 tx=0x7f295803d130 comp rx=0 tx=0).stop 2024-03-11T19:42:22.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.267+0000 7f29684f4640 1 -- 172.21.15.83:0/3823598916 shutdown_connections 2024-03-11T19:42:22.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.267+0000 7f29684f4640 1 --2- 172.21.15.83:0/3823598916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2960101c30 0x7f29601084c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.267+0000 7f29684f4640 1 --2- 172.21.15.83:0/3823598916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2960101270 0x7f29601016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.267+0000 7f29684f4640 1 --2- 172.21.15.83:0/3823598916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2960100080 0x7f29601004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.267+0000 7f29684f4640 1 -- 172.21.15.83:0/3823598916 >> 172.21.15.83:0/3823598916 conn(0x7f29600fb610 msgr2=0x7f29600fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.267+0000 7f29684f4640 1 -- 172.21.15.83:0/3823598916 shutdown_connections 2024-03-11T19:42:22.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.267+0000 7f29684f4640 1 -- 172.21.15.83:0/3823598916 wait complete. 2024-03-11T19:42:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f29684f4640 1 Processor -- start 2024-03-11T19:42:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f29684f4640 1 -- start start 2024-03-11T19:42:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f29684f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2960100080 0x7f296006fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f29684f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2960101270 0x7f296006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f29684f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2960101c30 0x7f296006e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f29684f4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2960192830 con 0x7f2960101c30 2024-03-11T19:42:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f29684f4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296006ed70 con 0x7f2960101270 2024-03-11T19:42:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f29684f4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296006ef30 con 0x7f2960100080 2024-03-11T19:42:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f2966290640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2960100080 0x7f296006fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f2965a8f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2960101270 0x7f296006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f2966a91640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2960101c30 0x7f296006e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f2966a91640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2960101c30 0x7f296006e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54628/0 (socket says 172.21.15.83:54628) 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.268+0000 7f2966a91640 1 -- 172.21.15.83:0/983496654 learned_addr learned my addr 172.21.15.83:0/983496654 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f2966a91640 1 -- 172.21.15.83:0/983496654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2960100080 msgr2=0x7f296006fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f2966a91640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2960100080 0x7f296006fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f2966a91640 1 -- 172.21.15.83:0/983496654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2960101270 msgr2=0x7f296006e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f2966a91640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2960101270 0x7f296006e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f2966a91640 1 -- 172.21.15.83:0/983496654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295801a910 con 0x7f2960101c30 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f2965a8f640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2960101270 0x7f296006e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f2966a91640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2960101c30 0x7f296006e830 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2958043380 tx=0x7f2958003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f29577fe640 1 -- 172.21.15.83:0/983496654 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295803dce0 con 0x7f2960101c30 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f29684f4640 1 -- 172.21.15.83:0/983496654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296006f150 con 0x7f2960101c30 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f29577fe640 1 -- 172.21.15.83:0/983496654 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2958043960 con 0x7f2960101c30 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f29577fe640 1 -- 172.21.15.83:0/983496654 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958045910 con 0x7f2960101c30 2024-03-11T19:42:22.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.269+0000 7f29684f4640 1 -- 172.21.15.83:0/983496654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296019d590 con 0x7f2960101c30 2024-03-11T19:42:22.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.273+0000 7f296b19c740 1 Processor -- start 2024-03-11T19:42:22.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.273+0000 7f29577fe640 1 -- 172.21.15.83:0/983496654 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f295804c070 con 0x7f2960101c30 2024-03-11T19:42:22.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.274+0000 7f296b19c740 1 -- start start 2024-03-11T19:42:22.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.274+0000 7f296b19c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x5620370478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.274+0000 7f296b19c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037048690 0x562037048b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.274+0000 7f296b19c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037049050 0x56203704f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.274+0000 7f296b19c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562036fb6f90 con 0x562037049050 2024-03-11T19:42:22.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.274+0000 7f296b19c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562036fb44c0 con 0x562037048690 2024-03-11T19:42:22.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.274+0000 7f296b19c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562036fafca0 con 0x5620370474c0 2024-03-11T19:42:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.274+0000 7f2965a8f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x5620370478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.274+0000 7f2966290640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037048690 0x562037048b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2965a8f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x5620370478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37598/0 (socket says 172.21.15.83:37598) 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2965a8f640 1 -- 172.21.15.83:0/2647894209 learned_addr learned my addr 172.21.15.83:0/2647894209 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2965a8f640 1 --2- 172.21.15.83:0/2647894209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037049050 0x56203704f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2965a8f640 1 -- 172.21.15.83:0/2647894209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037048690 msgr2=0x562037048b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2965a8f640 1 --2- 172.21.15.83:0/2647894209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037048690 0x562037048b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2965a8f640 1 -- 172.21.15.83:0/2647894209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037049050 msgr2=0x56203704f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2965a8f640 1 --2- 172.21.15.83:0/2647894209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037049050 0x56203704f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2965a8f640 1 -- 172.21.15.83:0/2647894209 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562036f720b0 con 0x5620370474c0 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2965a8f640 1 --2- 172.21.15.83:0/2647894209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x5620370478e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2950013890 tx=0x7f295000b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=908719b56937aec2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2954bfe640 1 -- 172.21.15.83:0/2647894209 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950004220 con 0x5620370474c0 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2954bfe640 1 -- 172.21.15.83:0/2647894209 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2950019070 con 0x5620370474c0 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f2954bfe640 1 -- 172.21.15.83:0/2647894209 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950003850 con 0x5620370474c0 2024-03-11T19:42:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.275+0000 7f29577fe640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f294c0a1ab0 0x7f294c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.276+0000 7f29577fe640 1 -- 172.21.15.83:0/983496654 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f29581003e0 con 0x7f2960101c30 2024-03-11T19:42:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.276+0000 7f296b19c740 1 -- 172.21.15.83:0/2647894209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 msgr2=0x5620370478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.276+0000 7f296b19c740 1 --2- 172.21.15.83:0/2647894209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x5620370478e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2950013890 tx=0x7f295000b420 comp rx=0 tx=0).stop 2024-03-11T19:42:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.277+0000 7f296b19c740 1 -- 172.21.15.83:0/2647894209 shutdown_connections 2024-03-11T19:42:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.277+0000 7f296b19c740 1 --2- 172.21.15.83:0/2647894209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037049050 0x56203704f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.277+0000 7f296b19c740 1 --2- 172.21.15.83:0/2647894209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037048690 0x562037048b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.277+0000 7f296b19c740 1 --2- 172.21.15.83:0/2647894209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x5620370478e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.277+0000 7f296b19c740 1 -- 172.21.15.83:0/2647894209 >> 172.21.15.83:0/2647894209 conn(0x5620370426a0 msgr2=0x562037042aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.277+0000 7f2966290640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f294c0a1ab0 0x7f294c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.277+0000 7f296b19c740 1 -- 172.21.15.83:0/2647894209 shutdown_connections 2024-03-11T19:42:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.277+0000 7f296b19c740 1 -- 172.21.15.83:0/2647894209 wait complete. 2024-03-11T19:42:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.278+0000 7f2966290640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f294c0a1ab0 0x7f294c0a3f70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x562037043610 tx=0x7f295c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.278+0000 7f296b19c740 1 -- start start 2024-03-11T19:42:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.278+0000 7f296b19c740 1 Processor -- start 2024-03-11T19:42:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.278+0000 7f296b19c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x562037051700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.278+0000 7f296b19c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 0x562037051c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f296b19c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 0x562037058c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f296b19c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562036f54f40 con 0x562037048690 2024-03-11T19:42:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f296b19c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562036c834b0 con 0x562037049050 2024-03-11T19:42:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f296b19c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562036f35670 con 0x5620370474c0 2024-03-11T19:42:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f296b19c740 10 client.0 _ll_register_callbacks cb 0x562036fa2f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2966290640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x562037051700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2965a8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 0x562037051c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2966290640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x562037051700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37608/0 (socket says 172.21.15.83:37608) 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2966290640 1 -- 172.21.15.83:0/802631525 learned_addr learned my addr 172.21.15.83:0/802631525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2966a91640 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 0x562037058c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2965a8f640 1 -- 172.21.15.83:0/802631525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 msgr2=0x562037051700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2965a8f640 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x562037051700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2965a8f640 1 -- 172.21.15.83:0/802631525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 msgr2=0x562037058c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2965a8f640 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 0x562037058c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2965a8f640 1 -- 172.21.15.83:0/802631525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2950013540 con 0x562037048690 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2965a8f640 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 0x562037051c40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x562037048590 tx=0x7f295000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6eed82d19c25aa server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.279+0000 7f2949ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.280+0000 7f2949ffb640 1 -- 172.21.15.83:0/802631525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295000eb00 con 0x562037048690 2024-03-11T19:42:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.280+0000 7f2949ffb640 1 -- 172.21.15.83:0/802631525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f295000ee70 con 0x562037048690 2024-03-11T19:42:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.280+0000 7f296b19c740 1 -- 172.21.15.83:0/802631525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562036f89f40 con 0x562037048690 2024-03-11T19:42:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.280+0000 7f2949ffb640 1 -- 172.21.15.83:0/802631525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2950004020 con 0x562037048690 2024-03-11T19:42:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.280+0000 7f296b19c740 10 client.15957 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.280+0000 7f296b19c740 1 -- 172.21.15.83:0/802631525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562036fae320 con 0x562037048690 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.281+0000 7f2949ffb640 1 -- 172.21.15.83:0/802631525 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562036f89f40 con 0x562037048690 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.281+0000 7f2949ffb640 1 -- 172.21.15.83:0/802631525 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2950030070 con 0x562037048690 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.281+0000 7f296b19c740 10 client.15957 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.281+0000 7f296b19c740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.281+0000 7f296b19c740 20 client.15957 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.281+0000 7f296b19c740 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56203705d8d0 0x56203705fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.281+0000 7f296b19c740 4 client.15957 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.281+0000 7f296b19c740 1 -- 172.21.15.83:0/802631525 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562036fa6310 con 0x56203705d8d0 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.281+0000 7f2966290640 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56203705d8d0 0x56203705fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.282+0000 7f2966290640 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56203705d8d0 0x56203705fcf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56da00d0a9ab8a4 server_cookie=e1dd7c2ba093e0e4 in_seq=0 out_seq=0 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.282+0000 7f2949ffb640 10 client.15957 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f2949ffb640 1 -- 172.21.15.83:0/802631525 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562036fa6310 con 0x56203705d8d0 2024-03-11T19:42:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f2949ffb640 10 client.15957 handle_command_reply: tid=0 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 1 client.15957 shutdown 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 1 -- 172.21.15.83:0/802631525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 msgr2=0x562037051c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 0x562037051c40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x562037048590 tx=0x7f295000bdd0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 1 -- 172.21.15.83:0/802631525 shutdown_connections 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56203705d8d0 0x56203705fcf0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 0x562037058c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 0x562037051c40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f2949ffb640 0 client.15957 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 1 --2- 172.21.15.83:0/802631525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x562037051700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 1 -- 172.21.15.83:0/802631525 >> 172.21.15.83:0/802631525 conn(0x5620370426a0 msgr2=0x5620370418e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 1 -- 172.21.15.83:0/802631525 shutdown_connections 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 1 -- 172.21.15.83:0/802631525 wait complete. 2024-03-11T19:42:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.283+0000 7f296b19c740 20 client.15957 trim_cache size 0 max 16384 2024-03-11T19:42:22.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.306+0000 7f296b19c740 1 Processor -- start 2024-03-11T19:42:22.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f296b19c740 1 -- start start 2024-03-11T19:42:22.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f296b19c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620370474c0 0x562037051700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f296b19c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037048690 0x562037051c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f296b19c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562037049050 0x562037058c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f296b19c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562037046f00 con 0x5620370474c0 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f296b19c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562036fa6310 con 0x562037048690 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f296b19c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562036f89f40 con 0x562037049050 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2966290640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620370474c0 0x562037051700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2965a8f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037048690 0x562037051c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2966290640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620370474c0 0x562037051700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54646/0 (socket says 172.21.15.83:54646) 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2966290640 1 -- 172.21.15.83:0/635463065 learned_addr learned my addr 172.21.15.83:0/635463065 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2966a91640 1 --2- 172.21.15.83:0/635463065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562037049050 0x562037058c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2966290640 1 -- 172.21.15.83:0/635463065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562037049050 msgr2=0x562037058c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2966290640 1 --2- 172.21.15.83:0/635463065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562037049050 0x562037058c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2966290640 1 -- 172.21.15.83:0/635463065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037048690 msgr2=0x562037051c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2966290640 1 --2- 172.21.15.83:0/635463065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037048690 0x562037051c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2966290640 1 -- 172.21.15.83:0/635463065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562036f4ac00 con 0x5620370474c0 2024-03-11T19:42:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2965a8f640 1 --2- 172.21.15.83:0/635463065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037048690 0x562037051c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.307+0000 7f2966290640 1 --2- 172.21.15.83:0/635463065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620370474c0 0x562037051700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x562037060600 tx=0x7f295c022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d339d71127c94711 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.308+0000 7f294bfff640 1 -- 172.21.15.83:0/635463065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c025040 con 0x5620370474c0 2024-03-11T19:42:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.308+0000 7f294bfff640 1 -- 172.21.15.83:0/635463065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f295c02a890 con 0x5620370474c0 2024-03-11T19:42:22.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.308+0000 7f294bfff640 1 -- 172.21.15.83:0/635463065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c017070 con 0x5620370474c0 2024-03-11T19:42:22.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.309+0000 7f296b19c740 1 -- 172.21.15.83:0/635463065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620370474c0 msgr2=0x562037051700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.309+0000 7f296b19c740 1 --2- 172.21.15.83:0/635463065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620370474c0 0x562037051700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x562037060600 tx=0x7f295c022040 comp rx=0 tx=0).stop 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.309+0000 7f296b19c740 1 -- 172.21.15.83:0/635463065 shutdown_connections 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.309+0000 7f296b19c740 1 --2- 172.21.15.83:0/635463065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562037049050 0x562037058c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.309+0000 7f296b19c740 1 --2- 172.21.15.83:0/635463065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037048690 0x562037051c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.309+0000 7f296b19c740 1 --2- 172.21.15.83:0/635463065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5620370474c0 0x562037051700 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.309+0000 7f296b19c740 1 -- 172.21.15.83:0/635463065 >> 172.21.15.83:0/635463065 conn(0x5620370426a0 msgr2=0x56203704c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.309+0000 7f296b19c740 1 -- 172.21.15.83:0/635463065 shutdown_connections 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.310+0000 7f296b19c740 1 -- 172.21.15.83:0/635463065 wait complete. 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f296b19c740 1 -- start start 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f296b19c740 1 Processor -- start 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f296b19c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x562037055930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f296b19c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 0x562037055e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f296b19c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 0x5620370b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f296b19c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562037058e90 con 0x562037048690 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f296b19c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562037059000 con 0x562037049050 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f296b19c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620370b4300 con 0x5620370474c0 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f296b19c740 10 client.0 _ll_register_callbacks cb 0x7f295000b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f2965a8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 0x562037055e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f2966a91640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 0x5620370b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f2966290640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x562037055930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f2965a8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 0x562037055e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54648/0 (socket says 172.21.15.83:54648) 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f2966a91640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 0x5620370b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40696/0 (socket says 172.21.15.83:40696) 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f2965a8f640 1 -- 172.21.15.83:0/894029926 learned_addr learned my addr 172.21.15.83:0/894029926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f2965a8f640 1 -- 172.21.15.83:0/894029926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 msgr2=0x562037055930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f2965a8f640 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x562037055930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.311+0000 7f2965a8f640 1 -- 172.21.15.83:0/894029926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 msgr2=0x5620370b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.312+0000 7f2965a8f640 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 0x5620370b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.312+0000 7f2965a8f640 1 -- 172.21.15.83:0/894029926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562036fae320 con 0x562037048690 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.312+0000 7f2966a91640 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 0x5620370b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.312+0000 7f2965a8f640 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 0x562037055e70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f295000ec80 tx=0x7f2950004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7dd389fb9efbc93 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.312+0000 7f2949ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.312+0000 7f2949ffb640 1 -- 172.21.15.83:0/894029926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950013760 con 0x562037048690 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.312+0000 7f2949ffb640 1 -- 172.21.15.83:0/894029926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2950000e40 con 0x562037048690 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.312+0000 7f296b19c740 1 -- 172.21.15.83:0/894029926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562036fd5900 con 0x562037048690 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f2949ffb640 1 -- 172.21.15.83:0/894029926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f295000b600 con 0x562037048690 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f296b19c740 10 client.15963 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f296b19c740 1 -- 172.21.15.83:0/894029926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f295c009de0 con 0x562037048690 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f2949ffb640 1 -- 172.21.15.83:0/894029926 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562036fd5900 con 0x562037048690 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f2949ffb640 1 -- 172.21.15.83:0/894029926 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2950045070 con 0x562037048690 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f296b19c740 10 client.15963 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:22.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f296b19c740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f296b19c740 20 client.15963 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f296b19c740 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5620370b5a90 0x5620370b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f296b19c740 4 client.15963 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:42:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f296b19c740 1 -- 172.21.15.83:0/894029926 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562036caf860 con 0x5620370b5a90 2024-03-11T19:42:22.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.313+0000 7f2966290640 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5620370b5a90 0x5620370b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.314+0000 7f2966290640 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5620370b5a90 0x5620370b7eb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7606004c2fe4fcec server_cookie=55e06a542949589d in_seq=0 out_seq=0 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.315+0000 7f2949ffb640 10 client.15963 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.316+0000 7f2949ffb640 1 -- 172.21.15.83:0/894029926 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562036caf860 con 0x5620370b5a90 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.316+0000 7f2949ffb640 10 client.15963 handle_command_reply: tid=0 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.316+0000 7f296b19c740 1 client.15963 shutdown 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.316+0000 7f296b19c740 1 -- 172.21.15.83:0/894029926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 msgr2=0x562037055e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.316+0000 7f296b19c740 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 0x562037055e70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f295000ec80 tx=0x7f2950004b40 comp rx=0 tx=0).stop 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.316+0000 7f296b19c740 1 -- 172.21.15.83:0/894029926 shutdown_connections 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.316+0000 7f296b19c740 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5620370b5a90 0x5620370b7eb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.317+0000 7f296b19c740 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562037049050 0x5620370b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.317+0000 7f2949ffb640 0 client.15963 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.317+0000 7f296b19c740 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562037048690 0x562037055e70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.317+0000 7f296b19c740 1 --2- 172.21.15.83:0/894029926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620370474c0 0x562037055930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.317+0000 7f296b19c740 1 -- 172.21.15.83:0/894029926 >> 172.21.15.83:0/894029926 conn(0x5620370426a0 msgr2=0x5620370516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.317+0000 7f296b19c740 1 -- 172.21.15.83:0/894029926 shutdown_connections 2024-03-11T19:42:22.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.317+0000 7f296b19c740 1 -- 172.21.15.83:0/894029926 wait complete. 2024-03-11T19:42:22.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.317+0000 7f296b19c740 20 client.15963 trim_cache size 0 max 16384 2024-03-11T19:42:22.324 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:42:22.324 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:42:22.324 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "7b0a48b0-e7a3-433c-92f7-81943aebe7ce", 2024-03-11T19:42:22.324 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:42:22.324 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:42:22.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.318+0000 7f29684f4640 1 -- 172.21.15.83:0/983496654 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f294c0a1ab0 msgr2=0x7f294c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.318+0000 7f29684f4640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f294c0a1ab0 0x7f294c0a3f70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x562037043610 tx=0x7f295c005f90 comp rx=0 tx=0).stop 2024-03-11T19:42:22.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.318+0000 7f29684f4640 1 -- 172.21.15.83:0/983496654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2960101c30 msgr2=0x7f296006e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.318+0000 7f29684f4640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2960101c30 0x7f296006e830 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2958043380 tx=0x7f2958003ba0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.319+0000 7f29684f4640 1 -- 172.21.15.83:0/983496654 shutdown_connections 2024-03-11T19:42:22.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.319+0000 7f29684f4640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f294c0a1ab0 0x7f294c0a3f70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.319+0000 7f29684f4640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2960101c30 0x7f296006e830 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.319+0000 7f29684f4640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2960101270 0x7f296006e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.319+0000 7f29684f4640 1 --2- 172.21.15.83:0/983496654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2960100080 0x7f296006fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.319+0000 7f29684f4640 1 -- 172.21.15.83:0/983496654 >> 172.21.15.83:0/983496654 conn(0x7f29600fb610 msgr2=0x7f29600fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.319+0000 7f29684f4640 1 -- 172.21.15.83:0/983496654 shutdown_connections 2024-03-11T19:42:22.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.319+0000 7f29684f4640 1 -- 172.21.15.83:0/983496654 wait complete. 2024-03-11T19:42:22.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:42:22.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:42:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 185.0 s, 4599.8 tps, lat 21.743 ms stddev 24.605 2024-03-11T19:42:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.504+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/3060410174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc100080 msgr2=0x7f3fcc1004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.504+0000 7f3fd0fbc640 1 --2- 172.21.15.83:0/3060410174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc100080 0x7f3fcc1004a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009be0 tx=0x7f3fc4030070 comp rx=0 tx=0).stop 2024-03-11T19:42:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.504+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/3060410174 shutdown_connections 2024-03-11T19:42:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.504+0000 7f3fd0fbc640 1 --2- 172.21.15.83:0/3060410174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fcc101c30 0x7f3fcc1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.505+0000 7f3fd0fbc640 1 --2- 172.21.15.83:0/3060410174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fcc101270 0x7f3fcc1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.505+0000 7f3fd0fbc640 1 --2- 172.21.15.83:0/3060410174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc100080 0x7f3fcc1004a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.505+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/3060410174 >> 172.21.15.83:0/3060410174 conn(0x7f3fcc0fb610 msgr2=0x7f3fcc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.505+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/3060410174 shutdown_connections 2024-03-11T19:42:22.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.505+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/3060410174 wait complete. 2024-03-11T19:42:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.505+0000 7f3fd0fbc640 1 Processor -- start 2024-03-11T19:42:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fd0fbc640 1 -- start start 2024-03-11T19:42:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fd0fbc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc100080 0x7f3fcc190580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fd0fbc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fcc101270 0x7f3fcc190ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fd0fbc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fcc101c30 0x7f3fcc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fd0fbc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc198070 con 0x7f3fcc101c30 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fd0fbc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1981e0 con 0x7f3fcc100080 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fd0fbc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1983a0 con 0x7f3fcc101270 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fcad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fcc101c30 0x7f3fcc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fcad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fcc101c30 0x7f3fcc197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54658/0 (socket says 172.21.15.83:54658) 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fcad9d640 1 -- 172.21.15.83:0/325933048 learned_addr learned my addr 172.21.15.83:0/325933048 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fca59c640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc100080 0x7f3fcc190580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.506+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fcc101270 0x7f3fcc190ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.507+0000 7f3fcad9d640 1 -- 172.21.15.83:0/325933048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fcc101270 msgr2=0x7f3fcc190ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.507+0000 7f3fcad9d640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fcc101270 0x7f3fcc190ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.507+0000 7f3fcad9d640 1 -- 172.21.15.83:0/325933048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc100080 msgr2=0x7f3fcc190580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.507+0000 7f3fcad9d640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc100080 0x7f3fcc190580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.507+0000 7f3fcad9d640 1 -- 172.21.15.83:0/325933048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc4009840 con 0x7f3fcc101c30 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.507+0000 7f3fca59c640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc100080 0x7f3fcc190580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.507+0000 7f3fcad9d640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fcc101c30 0x7f3fcc197b30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00e970 tx=0x7f3fbc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.507+0000 7f3fbb7fe640 1 -- 172.21.15.83:0/325933048 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc00ccc0 con 0x7f3fcc101c30 2024-03-11T19:42:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.507+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/325933048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc198680 con 0x7f3fcc101c30 2024-03-11T19:42:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.507+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/325933048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc1ae630 con 0x7f3fcc101c30 2024-03-11T19:42:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.508+0000 7f3fbb7fe640 1 -- 172.21.15.83:0/325933048 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fbc00ce20 con 0x7f3fcc101c30 2024-03-11T19:42:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.508+0000 7f3fbb7fe640 1 -- 172.21.15.83:0/325933048 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc010430 con 0x7f3fcc101c30 2024-03-11T19:42:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.510+0000 7f3fbb7fe640 1 -- 172.21.15.83:0/325933048 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3fbc010590 con 0x7f3fcc101c30 2024-03-11T19:42:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.511+0000 7f3fd3b8d740 1 Processor -- start 2024-03-11T19:42:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fd3b8d740 1 -- start start 2024-03-11T19:42:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 0x55d0f5a446f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a497c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a49e30 0x55d0f5a4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f59b4280 con 0x55d0f5a44cc0 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f59b17b0 con 0x55d0f5a49e30 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f59acf90 con 0x55d0f5a442d0 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fc9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a49e30 0x55d0f5a4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fca59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fc9d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a49e30 0x55d0f5a4c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:40714/0 (socket says 172.21.15.83:40714) 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/2914202235 learned_addr learned my addr 172.21.15.83:0/2914202235 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/2914202235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 0x55d0f5a446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.512+0000 7f3fbb7fe640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fb40a1c00 0x7f3fb40a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.513+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/2914202235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 msgr2=0x55d0f5a446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.513+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/2914202235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 0x55d0f5a446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.513+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/2914202235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 msgr2=0x55d0f5a497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.513+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/2914202235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.513+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/2914202235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0f596b570 con 0x55d0f5a49e30 2024-03-11T19:42:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.513+0000 7f3fbb7fe640 1 -- 172.21.15.83:0/325933048 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3fbc014070 con 0x7f3fcc101c30 2024-03-11T19:42:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.513+0000 7f3fcad9d640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fb40a1c00 0x7f3fb40a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.513+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/2914202235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a49e30 0x55d0f5a4c2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00058c0 tx=0x7f3fc000c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a141c0e942c4690 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.513+0000 7f3fb8ff9640 1 -- 172.21.15.83:0/2914202235 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0014310 con 0x55d0f5a49e30 2024-03-11T19:42:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.514+0000 7f3fb8ff9640 1 -- 172.21.15.83:0/2914202235 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fc001a020 con 0x55d0f5a49e30 2024-03-11T19:42:22.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.514+0000 7f3fb8ff9640 1 -- 172.21.15.83:0/2914202235 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0012980 con 0x55d0f5a49e30 2024-03-11T19:42:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.514+0000 7f3fcad9d640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fb40a1c00 0x7f3fb40a40c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc004fd0 tx=0x7f3fbc0ba040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.515+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/2914202235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a49e30 msgr2=0x55d0f5a4c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.515+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/2914202235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a49e30 0x55d0f5a4c2c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00058c0 tx=0x7f3fc000c4d0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.515+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/2914202235 shutdown_connections 2024-03-11T19:42:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.515+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/2914202235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a49e30 0x55d0f5a4c2c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.515+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/2914202235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.515+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/2914202235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 0x55d0f5a446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.515+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/2914202235 >> 172.21.15.83:0/2914202235 conn(0x55d0f5a3f7f0 msgr2=0x55d0f5a3fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.515+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/2914202235 shutdown_connections 2024-03-11T19:42:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.515+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/2914202235 wait complete. 2024-03-11T19:42:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.515+0000 7f3fd3b8d740 1 -- start start 2024-03-11T19:42:22.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.516+0000 7f3fd3b8d740 1 Processor -- start 2024-03-11T19:42:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.516+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a442d0 0x55d0f5a4e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.516+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a4e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.516+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a49e30 0x55d0f5a557c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.516+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f5680430 con 0x55d0f5a44cc0 2024-03-11T19:42:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.516+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f563ce00 con 0x55d0f5a442d0 2024-03-11T19:42:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.516+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f5931fd0 con 0x55d0f5a49e30 2024-03-11T19:42:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.516+0000 7f3fd3b8d740 10 client.0 _ll_register_callbacks cb 0x55d0f5987fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.516+0000 7f3fca59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a4e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.516+0000 7f3fca59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a4e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54674/0 (socket says 172.21.15.83:54674) 2024-03-11T19:42:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.516+0000 7f3fca59c640 1 -- 172.21.15.83:0/1236604536 learned_addr learned my addr 172.21.15.83:0/1236604536 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:22.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.517+0000 7f3fca59c640 1 -- 172.21.15.83:0/1236604536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a49e30 msgr2=0x55d0f5a557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.517+0000 7f3fca59c640 1 --2- 172.21.15.83:0/1236604536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a49e30 0x55d0f5a557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.517+0000 7f3fca59c640 1 -- 172.21.15.83:0/1236604536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a442d0 msgr2=0x55d0f5a4e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.517+0000 7f3fca59c640 1 --2- 172.21.15.83:0/1236604536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a442d0 0x55d0f5a4e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.517+0000 7f3fca59c640 1 -- 172.21.15.83:0/1236604536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc00146a0 con 0x55d0f5a44cc0 2024-03-11T19:42:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.517+0000 7f3fca59c640 1 --2- 172.21.15.83:0/1236604536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a4e780 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc1010d0 tx=0x7f3fc4037820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcc386a29ea50a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.517+0000 7f3fb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:42:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.518+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/1236604536 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4003d80 con 0x55d0f5a44cc0 2024-03-11T19:42:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.518+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/1236604536 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3fc4004040 con 0x55d0f5a44cc0 2024-03-11T19:42:22.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.518+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/1236604536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0f56ac7e0 con 0x55d0f5a44cc0 2024-03-11T19:42:22.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.518+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/1236604536 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc4030be0 con 0x55d0f5a44cc0 2024-03-11T19:42:22.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.518+0000 7f3fd3b8d740 10 client.15975 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:22.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.518+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/1236604536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d0f59ab610 con 0x55d0f5a44cc0 2024-03-11T19:42:22.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.519+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/1236604536 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fc4053020 con 0x55d0f5a44cc0 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.519+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/1236604536 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3fc4040040 con 0x55d0f5a44cc0 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.519+0000 7f3fd3b8d740 10 client.15975 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.519+0000 7f3fd3b8d740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.519+0000 7f3fd3b8d740 20 client.15975 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.519+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/1236604536 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d0f5a5a5e0 0x55d0f5a5ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.519+0000 7f3fd3b8d740 4 client.15975 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.519+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/1236604536 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0f56e9720 con 0x55d0f5a5a5e0 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.519+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/1236604536 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d0f5a5a5e0 0x55d0f5a5ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.520+0000 7f3fb1ffb640 10 client.15975 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.520+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/1236604536 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d0f5a5a5e0 0x55d0f5a5ca00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8dd1f56b6f7b107 server_cookie=7654be2791d2036 in_seq=0 out_seq=0 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.520+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/1236604536 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0f56e9720 con 0x55d0f5a5a5e0 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.520+0000 7f3fb1ffb640 10 client.15975 handle_command_reply: tid=0 2024-03-11T19:42:22.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.520+0000 7f3fd3b8d740 1 client.15975 shutdown 2024-03-11T19:42:22.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/1236604536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 msgr2=0x55d0f5a4e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/1236604536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a4e780 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc1010d0 tx=0x7f3fc4037820 comp rx=0 tx=0).stop 2024-03-11T19:42:22.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/1236604536 shutdown_connections 2024-03-11T19:42:22.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/1236604536 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d0f5a5a5e0 0x55d0f5a5ca00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/1236604536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a49e30 0x55d0f5a557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/1236604536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a4e780 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fb1ffb640 0 client.15975 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:22.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/1236604536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a442d0 0x55d0f5a4e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/1236604536 >> 172.21.15.83:0/1236604536 conn(0x55d0f5a3f7f0 msgr2=0x55d0f5a3ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/1236604536 shutdown_connections 2024-03-11T19:42:22.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/1236604536 wait complete. 2024-03-11T19:42:22.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.521+0000 7f3fd3b8d740 20 client.15975 trim_cache size 0 max 16384 2024-03-11T19:42:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.548+0000 7f3fd3b8d740 1 Processor -- start 2024-03-11T19:42:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.548+0000 7f3fd3b8d740 1 -- start start 2024-03-11T19:42:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.548+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 0x55d0f5a4d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.548+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a55780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.548+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a49e30 0x55d0f5a4d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.548+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f5a58a40 con 0x55d0f5a49e30 2024-03-11T19:42:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.548+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f5a61c10 con 0x55d0f5a44cc0 2024-03-11T19:42:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.548+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f5a491b0 con 0x55d0f5a442d0 2024-03-11T19:42:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.549+0000 7f3fc9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 0x55d0f5a4d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.549+0000 7f3fc9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 0x55d0f5a4d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37668/0 (socket says 172.21.15.83:37668) 2024-03-11T19:42:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.549+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/3404413562 learned_addr learned my addr 172.21.15.83:0/3404413562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.549+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/3404413562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a49e30 0x55d0f5a4d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.549+0000 7f3fca59c640 1 --2- 172.21.15.83:0/3404413562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a55780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.549+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/3404413562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a44cc0 msgr2=0x55d0f5a55780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.549+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/3404413562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a55780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.549+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/3404413562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a49e30 msgr2=0x55d0f5a4d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.549+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/3404413562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a49e30 0x55d0f5a4d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.549+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/3404413562 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0f5947ac0 con 0x55d0f5a442d0 2024-03-11T19:42:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.549+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/3404413562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 0x55d0f5a4d030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0014990 tx=0x7f3fc0014f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d62a419e07ce3dd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.550+0000 7f3fb3fff640 1 -- 172.21.15.83:0/3404413562 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0024070 con 0x55d0f5a442d0 2024-03-11T19:42:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.550+0000 7f3fb3fff640 1 -- 172.21.15.83:0/3404413562 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fc002d070 con 0x55d0f5a442d0 2024-03-11T19:42:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.550+0000 7f3fb3fff640 1 -- 172.21.15.83:0/3404413562 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc000fa20 con 0x55d0f5a442d0 2024-03-11T19:42:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.551+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/3404413562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 msgr2=0x55d0f5a4d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.551+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/3404413562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 0x55d0f5a4d030 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0014990 tx=0x7f3fc0014f10 comp rx=0 tx=0).stop 2024-03-11T19:42:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.551+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/3404413562 shutdown_connections 2024-03-11T19:42:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.551+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/3404413562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a49e30 0x55d0f5a4d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.551+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/3404413562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a55780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.551+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/3404413562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a442d0 0x55d0f5a4d030 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.551+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/3404413562 >> 172.21.15.83:0/3404413562 conn(0x55d0f5a3f7f0 msgr2=0x55d0f5a45f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.552+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/3404413562 shutdown_connections 2024-03-11T19:42:22.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.552+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/3404413562 wait complete. 2024-03-11T19:42:22.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.553+0000 7f3fd3b8d740 1 -- start start 2024-03-11T19:42:22.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.553+0000 7f3fd3b8d740 1 Processor -- start 2024-03-11T19:42:22.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.553+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a442d0 0x55d0f5a53e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fd3b8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a49e30 0x55d0f5a548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f5a55ab0 con 0x55d0f5a442d0 2024-03-11T19:42:22.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f5ab0ad0 con 0x55d0f5a44cc0 2024-03-11T19:42:22.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fd3b8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f5ab0c10 con 0x55d0f5a49e30 2024-03-11T19:42:22.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fd3b8d740 10 client.0 _ll_register_callbacks cb 0x7f3fc000ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:22.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fc9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a442d0 0x55d0f5a53e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fc9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a442d0 0x55d0f5a53e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54690/0 (socket says 172.21.15.83:54690) 2024-03-11T19:42:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/763018456 learned_addr learned my addr 172.21.15.83:0/763018456 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a49e30 0x55d0f5a548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fca59c640 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/763018456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a44cc0 msgr2=0x55d0f5a543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/763018456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a442d0 msgr2=0x55d0f5a53e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a442d0 0x55d0f5a53e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.554+0000 7f3fc9d9b640 1 -- 172.21.15.83:0/763018456 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0f596b570 con 0x55d0f5a49e30 2024-03-11T19:42:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.555+0000 7f3fc9d9b640 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a49e30 0x55d0f5a548e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0001000 tx=0x7f3fc000c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3050620e996a788c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.555+0000 7f3fb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:42:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.555+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/763018456 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0024070 con 0x55d0f5a49e30 2024-03-11T19:42:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.555+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/763018456 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3fc0000e40 con 0x55d0f5a49e30 2024-03-11T19:42:22.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.555+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/763018456 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fc001a020 con 0x55d0f5a49e30 2024-03-11T19:42:22.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.556+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/763018456 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc000fc90 con 0x55d0f5a49e30 2024-03-11T19:42:22.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.556+0000 7f3fd3b8d740 10 client.25585 fetch_fsmap learned FSMap version 26 2024-03-11T19:42:22.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.556+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/763018456 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d0f596d6a0 con 0x55d0f5a49e30 2024-03-11T19:42:22.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.557+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/763018456 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fc001a020 con 0x55d0f5a49e30 2024-03-11T19:42:22.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.557+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/763018456 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3fc0026250 con 0x55d0f5a49e30 2024-03-11T19:42:22.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.557+0000 7f3fd3b8d740 10 client.25585 fetch_fsmap finished waiting for FSMap version 26 2024-03-11T19:42:22.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.557+0000 7f3fd3b8d740 10 client.25585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:22.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.557+0000 7f3fd3b8d740 20 client.25585 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:22.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.557+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d0f5ab22b0 0x55d0f5ab46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:22.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.557+0000 7f3fd3b8d740 4 client.25585 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:42:22.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.557+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/763018456 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d0f5738890 con 0x55d0f5ab22b0 2024-03-11T19:42:22.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.557+0000 7f3fca59c640 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d0f5ab22b0 0x55d0f5ab46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:22.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.557+0000 7f3fb1ffb640 10 client.25585 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:22.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.557+0000 7f3fca59c640 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d0f5ab22b0 0x55d0f5ab46d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b74e4be210486d server_cookie=38b56d22bb4b3296 in_seq=0 out_seq=0 2024-03-11T19:42:22.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.558+0000 7f3fb1ffb640 1 -- 172.21.15.83:0/763018456 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d0f5738890 con 0x55d0f5ab22b0 2024-03-11T19:42:22.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.558+0000 7f3fb1ffb640 10 client.25585 handle_command_reply: tid=0 2024-03-11T19:42:22.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.558+0000 7f3fd3b8d740 1 client.25585 shutdown 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/763018456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a49e30 msgr2=0x55d0f5a548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a49e30 0x55d0f5a548e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0001000 tx=0x7f3fc000c4d0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/763018456 shutdown_connections 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d0f5ab22b0 0x55d0f5ab46d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d0f5a49e30 0x55d0f5a548e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0f5a44cc0 0x55d0f5a543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fd3b8d740 1 --2- 172.21.15.83:0/763018456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0f5a442d0 0x55d0f5a53e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/763018456 >> 172.21.15.83:0/763018456 conn(0x55d0f5a3f7f0 msgr2=0x55d0f5a4d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fb1ffb640 0 client.25585 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/763018456 shutdown_connections 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fd3b8d740 1 -- 172.21.15.83:0/763018456 wait complete. 2024-03-11T19:42:22.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.559+0000 7f3fd3b8d740 20 client.25585 trim_cache size 0 max 16384 2024-03-11T19:42:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:42:22.565 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:42:22.565 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:42:22.565 INFO:teuthology.orchestra.run.smithi083.stdout: "7b0a48b0-e7a3-433c-92f7-81943aebe7ce": { 2024-03-11T19:42:22.565 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:42:22.565 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "7b0a48b0-e7a3-433c-92f7-81943aebe7ce", 2024-03-11T19:42:22.565 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:42:22.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:42:22.565 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:42:22.565 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:42:22.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/325933048 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fb40a1c00 msgr2=0x7f3fb40a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fb40a1c00 0x7f3fb40a40c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc004fd0 tx=0x7f3fbc0ba040 comp rx=0 tx=0).stop 2024-03-11T19:42:22.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/325933048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fcc101c30 msgr2=0x7f3fcc197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:22.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fcc101c30 0x7f3fcc197b30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00e970 tx=0x7f3fbc00ee30 comp rx=0 tx=0).stop 2024-03-11T19:42:22.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/325933048 shutdown_connections 2024-03-11T19:42:22.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3fb40a1c00 0x7f3fb40a40c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fcc101c30 0x7f3fcc197b30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fcc101270 0x7f3fcc190ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 --2- 172.21.15.83:0/325933048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fcc100080 0x7f3fcc190580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:22.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/325933048 >> 172.21.15.83:0/325933048 conn(0x7f3fcc0fb610 msgr2=0x7f3fcc104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:22.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/325933048 shutdown_connections 2024-03-11T19:42:22.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:22.560+0000 7f3fd0fbc640 1 -- 172.21.15.83:0/325933048 wait complete. 2024-03-11T19:42:22.582 INFO:tasks.cephfs.filesystem:scrub status for tag:7b0a48b0-e7a3-433c-92f7-81943aebe7ce - {'path': '/', 'tag': '7b0a48b0-e7a3-433c-92f7-81943aebe7ce', 'options': 'recursive,force'} 2024-03-11T19:42:22.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:22 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:42:22.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:22 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:22.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:22 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:42:22.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:22 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:22.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:22 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:42:22.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:22 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:22.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:22 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:42:22.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:22 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:22.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:22 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:42:22.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:22 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:22.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:22 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:42:22.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:22 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:23.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:23 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:42:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:23 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:23 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:42:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:23 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:42:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:23 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:23 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:23.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:23 smithi083 ceph-mon[31864]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 825 op/s 2024-03-11T19:42:23.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:23 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:42:23.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:23 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:23.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:23 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:42:23.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:23 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:42:23.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:23 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:23.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:23 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:23.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:23 smithi137 ceph-mon[36220]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 825 op/s 2024-03-11T19:42:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:23 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:42:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:23 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:23 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:42:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:23 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:42:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:23 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:23.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:23 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:23.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:23 smithi204 ceph-mon[36739]: pgmap v444: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 825 op/s 2024-03-11T19:42:26.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:25 smithi083 ceph-mon[31864]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 819 op/s 2024-03-11T19:42:26.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:25 smithi137 ceph-mon[36220]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 819 op/s 2024-03-11T19:42:26.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:25 smithi204 ceph-mon[36739]: pgmap v445: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 819 op/s 2024-03-11T19:42:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:26 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:42:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:26 smithi083 ceph-mon[31864]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 804 op/s 2024-03-11T19:42:27.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:26 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:42:27.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:26 smithi137 ceph-mon[36220]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 804 op/s 2024-03-11T19:42:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:26 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:42:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:26 smithi204 ceph-mon[36739]: pgmap v446: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 804 op/s 2024-03-11T19:42:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 190.0 s, 3960.8 tps, lat 25.037 ms stddev 27.104 2024-03-11T19:42:28.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:42:28.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:42:28.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:27 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:42:28.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:42:28.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:42:28.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:27 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:42:28.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:42:28.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:42:28.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:27 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:42:29.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:28 smithi083 ceph-mon[31864]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 827 op/s 2024-03-11T19:42:29.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:28 smithi137 ceph-mon[36220]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 827 op/s 2024-03-11T19:42:29.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:28 smithi204 ceph-mon[36739]: pgmap v447: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 827 op/s 2024-03-11T19:42:30.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:30 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:42:30.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:30 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:42:30.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:30 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:42:30.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:30 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:42:30.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:30 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:42:30.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:30 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:42:31.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:31 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:42:31.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:31 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:42:31.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:31 smithi083 ceph-mon[31864]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 835 op/s 2024-03-11T19:42:31.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:31 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:42:31.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:31 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:42:31.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:31 smithi137 ceph-mon[36220]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 835 op/s 2024-03-11T19:42:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:31 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:42:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:31 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:42:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:31 smithi204 ceph-mon[36739]: pgmap v448: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 835 op/s 2024-03-11T19:42:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 195.0 s, 3664.8 tps, lat 27.487 ms stddev 30.004 2024-03-11T19:42:34.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:33 smithi083 ceph-mon[31864]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 772 op/s 2024-03-11T19:42:34.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:33 smithi137 ceph-mon[36220]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 772 op/s 2024-03-11T19:42:34.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:33 smithi204 ceph-mon[36739]: pgmap v449: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 MiB/s wr, 772 op/s 2024-03-11T19:42:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:34 smithi083 ceph-mon[31864]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 853 op/s 2024-03-11T19:42:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:34 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:42:35.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:34 smithi137 ceph-mon[36220]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 853 op/s 2024-03-11T19:42:35.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:34 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:42:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:34 smithi204 ceph-mon[36739]: pgmap v450: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 853 op/s 2024-03-11T19:42:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:34 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:42:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:36 smithi083 ceph-mon[31864]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.5 MiB/s wr, 761 op/s 2024-03-11T19:42:37.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:36 smithi137 ceph-mon[36220]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.5 MiB/s wr, 761 op/s 2024-03-11T19:42:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:36 smithi204 ceph-mon[36739]: pgmap v451: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.5 MiB/s wr, 761 op/s 2024-03-11T19:42:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 200.0 s, 5393.2 tps, lat 18.526 ms stddev 21.018 2024-03-11T19:42:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:38 smithi083 ceph-mon[31864]: pgmap v452: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.4 MiB/s wr, 847 op/s 2024-03-11T19:42:39.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:38 smithi137 ceph-mon[36220]: pgmap v452: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.4 MiB/s wr, 847 op/s 2024-03-11T19:42:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:38 smithi204 ceph-mon[36739]: pgmap v452: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.4 MiB/s wr, 847 op/s 2024-03-11T19:42:41.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:40 smithi083 ceph-mon[31864]: pgmap v453: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 843 op/s 2024-03-11T19:42:41.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:40 smithi137 ceph-mon[36220]: pgmap v453: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 843 op/s 2024-03-11T19:42:41.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:40 smithi204 ceph-mon[36739]: pgmap v453: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.1 MiB/s wr, 843 op/s 2024-03-11T19:42:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 205.0 s, 5283.2 tps, lat 18.890 ms stddev 20.974 2024-03-11T19:42:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:42 smithi083 ceph-mon[31864]: pgmap v454: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 839 op/s 2024-03-11T19:42:43.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:42 smithi137 ceph-mon[36220]: pgmap v454: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 839 op/s 2024-03-11T19:42:43.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:42 smithi204 ceph-mon[36739]: pgmap v454: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 839 op/s 2024-03-11T19:42:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:44 smithi083 ceph-mon[31864]: pgmap v455: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 924 op/s 2024-03-11T19:42:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:44 smithi137 ceph-mon[36220]: pgmap v455: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 924 op/s 2024-03-11T19:42:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:44 smithi204 ceph-mon[36739]: pgmap v455: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.8 MiB/s wr, 924 op/s 2024-03-11T19:42:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 210.0 s, 3963.2 tps, lat 25.136 ms stddev 27.910 2024-03-11T19:42:48.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:47 smithi083 ceph-mon[31864]: pgmap v456: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 898 op/s 2024-03-11T19:42:48.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:47 smithi137 ceph-mon[36220]: pgmap v456: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 898 op/s 2024-03-11T19:42:48.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:47 smithi204 ceph-mon[36739]: pgmap v456: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 898 op/s 2024-03-11T19:42:49.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:48 smithi083 ceph-mon[31864]: pgmap v457: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-03-11T19:42:49.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:48 smithi137 ceph-mon[36220]: pgmap v457: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-03-11T19:42:49.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:48 smithi204 ceph-mon[36739]: pgmap v457: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-03-11T19:42:51.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:50 smithi083 ceph-mon[31864]: pgmap v458: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 17 MiB/s wr, 955 op/s 2024-03-11T19:42:51.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:50 smithi137 ceph-mon[36220]: pgmap v458: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 17 MiB/s wr, 955 op/s 2024-03-11T19:42:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:50 smithi204 ceph-mon[36739]: pgmap v458: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 17 MiB/s wr, 955 op/s 2024-03-11T19:42:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 215.0 s, 3805.8 tps, lat 26.397 ms stddev 30.275 2024-03-11T19:42:52.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:42:52.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:42:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.745+0000 7f339d75a640 1 -- 172.21.15.83:0/1113663499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3398101290 msgr2=0x7f3398107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.745+0000 7f339d75a640 1 --2- 172.21.15.83:0/1113663499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3398101290 0x7f3398107b20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3390009900 tx=0x7f339002bc30 comp rx=0 tx=0).stop 2024-03-11T19:42:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.745+0000 7f339d75a640 1 -- 172.21.15.83:0/1113663499 shutdown_connections 2024-03-11T19:42:52.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.745+0000 7f339d75a640 1 --2- 172.21.15.83:0/1113663499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3398101290 0x7f3398107b20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.745+0000 7f339d75a640 1 --2- 172.21.15.83:0/1113663499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33981008d0 0x7f3398100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.745+0000 7f339d75a640 1 --2- 172.21.15.83:0/1113663499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33980ff6e0 0x7f33980ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.745+0000 7f339d75a640 1 -- 172.21.15.83:0/1113663499 >> 172.21.15.83:0/1113663499 conn(0x7f33980fac70 msgr2=0x7f33980fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.746+0000 7f339d75a640 1 -- 172.21.15.83:0/1113663499 shutdown_connections 2024-03-11T19:42:52.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.746+0000 7f339d75a640 1 -- 172.21.15.83:0/1113663499 wait complete. 2024-03-11T19:42:52.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.746+0000 7f339d75a640 1 Processor -- start 2024-03-11T19:42:52.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f339d75a640 1 -- start start 2024-03-11T19:42:52.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f339d75a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33980ff6e0 0x7f3398194e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f339d75a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33981008d0 0x7f3398195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f339d75a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3398101290 0x7f339819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f339d75a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339819c940 con 0x7f3398101290 2024-03-11T19:42:52.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f339d75a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339819cab0 con 0x7f33981008d0 2024-03-11T19:42:52.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f339d75a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33981adcd0 con 0x7f33980ff6e0 2024-03-11T19:42:52.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f339cf59640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3398101290 0x7f339819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f3397fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33980ff6e0 0x7f3398194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f339cf59640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3398101290 0x7f339819c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58898/0 (socket says 172.21.15.83:58898) 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f339cf59640 1 -- 172.21.15.83:0/3628186100 learned_addr learned my addr 172.21.15.83:0/3628186100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f338f7fe640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33981008d0 0x7f3398195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f3397fff640 1 -- 172.21.15.83:0/3628186100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33981008d0 msgr2=0x7f3398195390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f3397fff640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33981008d0 0x7f3398195390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f3397fff640 1 -- 172.21.15.83:0/3628186100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3398101290 msgr2=0x7f339819c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f3397fff640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3398101290 0x7f339819c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f3397fff640 1 -- 172.21.15.83:0/3628186100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3388009840 con 0x7f33980ff6e0 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.747+0000 7f338f7fe640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33981008d0 0x7f3398195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.748+0000 7f3397fff640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33980ff6e0 0x7f3398194e50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f338000ea60 tx=0x7f338000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.748+0000 7f3395ffb640 1 -- 172.21.15.83:0/3628186100 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338000cc80 con 0x7f33980ff6e0 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.748+0000 7f3395ffb640 1 -- 172.21.15.83:0/3628186100 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f338000cde0 con 0x7f33980ff6e0 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.748+0000 7f3395ffb640 1 -- 172.21.15.83:0/3628186100 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380016410 con 0x7f33980ff6e0 2024-03-11T19:42:52.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.748+0000 7f339d75a640 1 -- 172.21.15.83:0/3628186100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3390009560 con 0x7f33980ff6e0 2024-03-11T19:42:52.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.748+0000 7f339d75a640 1 -- 172.21.15.83:0/3628186100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33981ae1d0 con 0x7f33980ff6e0 2024-03-11T19:42:52.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.751+0000 7f3395ffb640 1 -- 172.21.15.83:0/3628186100 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3380004000 con 0x7f33980ff6e0 2024-03-11T19:42:52.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.752+0000 7f3395ffb640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33840a1c30 0x7f33840a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.752+0000 7f3395ffb640 1 -- 172.21.15.83:0/3628186100 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f33800cd100 con 0x7f33980ff6e0 2024-03-11T19:42:52.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.752+0000 7f338f7fe640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33840a1c30 0x7f33840a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.752+0000 7f33a1590740 1 Processor -- start 2024-03-11T19:42:52.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f33a1590740 1 -- start start 2024-03-11T19:42:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f338f7fe640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33840a1c30 0x7f33840a40f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3398196360 tx=0x7f3388009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f33a1590740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d237584b0 0x559d237588d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f33a1590740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d23758ea0 0x559d2375d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f33a1590740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d237604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f33a1590740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d236c8280 con 0x559d237584b0 2024-03-11T19:42:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f33a1590740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d236c57b0 con 0x559d2375e010 2024-03-11T19:42:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f33a1590740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d236c0f90 con 0x559d23758ea0 2024-03-11T19:42:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f3397fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d237584b0 0x559d237588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f3397fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d237584b0 0x559d237588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58912/0 (socket says 172.21.15.83:58912) 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f3397fff640 1 -- 172.21.15.83:0/1839244072 learned_addr learned my addr 172.21.15.83:0/1839244072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f339cf59640 1 --2- 172.21.15.83:0/1839244072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d237604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f3397fff640 1 -- 172.21.15.83:0/1839244072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d23758ea0 msgr2=0x559d2375d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f3397fff640 1 --2- 172.21.15.83:0/1839244072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d23758ea0 0x559d2375d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f3397fff640 1 -- 172.21.15.83:0/1839244072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 msgr2=0x559d237604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f3397fff640 1 --2- 172.21.15.83:0/1839244072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d237604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.753+0000 7f3397fff640 1 -- 172.21.15.83:0/1839244072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d2367f570 con 0x559d237584b0 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.754+0000 7f3397fff640 1 --2- 172.21.15.83:0/1839244072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d237584b0 0x559d237588d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f338009a010 tx=0x7f338001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb82c3e4be18726 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.754+0000 7f338e7fc640 1 -- 172.21.15.83:0/1839244072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338009e1d0 con 0x559d237584b0 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.754+0000 7f338e7fc640 1 -- 172.21.15.83:0/1839244072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33800a2820 con 0x559d237584b0 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.754+0000 7f338e7fc640 1 -- 172.21.15.83:0/1839244072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800a25f0 con 0x559d237584b0 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.755+0000 7f33a1590740 1 -- 172.21.15.83:0/1839244072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d237584b0 msgr2=0x559d237588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.755+0000 7f33a1590740 1 --2- 172.21.15.83:0/1839244072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d237584b0 0x559d237588d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f338009a010 tx=0x7f338001d000 comp rx=0 tx=0).stop 2024-03-11T19:42:52.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.755+0000 7f33a1590740 1 -- 172.21.15.83:0/1839244072 shutdown_connections 2024-03-11T19:42:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.755+0000 7f33a1590740 1 --2- 172.21.15.83:0/1839244072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d237604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.755+0000 7f33a1590740 1 --2- 172.21.15.83:0/1839244072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d23758ea0 0x559d2375d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.755+0000 7f33a1590740 1 --2- 172.21.15.83:0/1839244072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d237584b0 0x559d237588d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.755+0000 7f33a1590740 1 -- 172.21.15.83:0/1839244072 >> 172.21.15.83:0/1839244072 conn(0x559d23753a20 msgr2=0x559d23753e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.755+0000 7f33a1590740 1 -- 172.21.15.83:0/1839244072 shutdown_connections 2024-03-11T19:42:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.755+0000 7f33a1590740 1 -- 172.21.15.83:0/1839244072 wait complete. 2024-03-11T19:42:52.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.756+0000 7f33a1590740 1 -- start start 2024-03-11T19:42:52.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.756+0000 7f33a1590740 1 Processor -- start 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.756+0000 7f33a1590740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d23762510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.756+0000 7f33a1590740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d23758ea0 0x559d23762a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.756+0000 7f33a1590740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d2375e010 0x559d23769a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.757+0000 7f338f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d23758ea0 0x559d23762a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.757+0000 7f3397fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d23762510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.757+0000 7f338f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d23758ea0 0x559d23762a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37380/0 (socket says 172.21.15.83:37380) 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.757+0000 7f338f7fe640 1 -- 172.21.15.83:0/2903502961 learned_addr learned my addr 172.21.15.83:0/2903502961 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.757+0000 7f33a1590740 1 -- 172.21.15.83:0/2903502961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d23394430 con 0x559d2375e010 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.757+0000 7f33a1590740 1 -- 172.21.15.83:0/2903502961 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d23350e00 con 0x559d23758ea0 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.757+0000 7f33a1590740 1 -- 172.21.15.83:0/2903502961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d23645fd0 con 0x559d237584b0 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.757+0000 7f33a1590740 10 client.0 _ll_register_callbacks cb 0x559d2369bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:52.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.757+0000 7f3397fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d23762510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:54736/0 (socket says 172.21.15.83:54736) 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.757+0000 7f339cf59640 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d2375e010 0x559d23769a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f3397fff640 1 -- 172.21.15.83:0/2903502961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d23758ea0 msgr2=0x559d23762a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f3397fff640 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d23758ea0 0x559d23762a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f3397fff640 1 -- 172.21.15.83:0/2903502961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d2375e010 msgr2=0x559d23769a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f3397fff640 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d2375e010 0x559d23769a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f3397fff640 1 -- 172.21.15.83:0/2903502961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33981ae1d0 con 0x559d237584b0 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f3397fff640 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d23762510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3380096a50 tx=0x7f338009a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b500678387193f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f33777fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f33777fe640 1 -- 172.21.15.83:0/2903502961 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800994d0 con 0x559d237584b0 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f33777fe640 1 -- 172.21.15.83:0/2903502961 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3380099630 con 0x559d237584b0 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f33a1590740 1 -- 172.21.15.83:0/2903502961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d233c07e0 con 0x559d237584b0 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f33777fe640 1 -- 172.21.15.83:0/2903502961 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3380099830 con 0x559d237584b0 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f33a1590740 10 client.25594 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:52.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.758+0000 7f33a1590740 1 -- 172.21.15.83:0/2903502961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559d236bf610 con 0x559d237584b0 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.759+0000 7f33777fe640 1 -- 172.21.15.83:0/2903502961 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559d233c07e0 con 0x559d237584b0 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.759+0000 7f33777fe640 1 -- 172.21.15.83:0/2903502961 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f338009f0f0 con 0x559d237584b0 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.759+0000 7f33a1590740 10 client.25594 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.759+0000 7f33a1590740 10 client.25594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.759+0000 7f33a1590740 20 client.25594 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.759+0000 7f33a1590740 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559d2376e400 0x559d237707f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.759+0000 7f33a1590740 4 client.25594 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.759+0000 7f33a1590740 1 -- 172.21.15.83:0/2903502961 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d233fd720 con 0x559d2376e400 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.760+0000 7f338f7fe640 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559d2376e400 0x559d237707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.760+0000 7f338f7fe640 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559d2376e400 0x559d237707f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f704782fb0cf1d64 server_cookie=26b325545be37146 in_seq=0 out_seq=0 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33777fe640 10 client.25594 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33777fe640 1 -- 172.21.15.83:0/2903502961 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d233fd720 con 0x559d2376e400 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33777fe640 10 client.25594 handle_command_reply: tid=0 2024-03-11T19:42:52.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33a1590740 1 client.25594 shutdown 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33a1590740 1 -- 172.21.15.83:0/2903502961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 msgr2=0x559d23762510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33a1590740 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d23762510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3380096a50 tx=0x7f338009a200 comp rx=0 tx=0).stop 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33a1590740 1 -- 172.21.15.83:0/2903502961 shutdown_connections 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33a1590740 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559d2376e400 0x559d237707f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33a1590740 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d2375e010 0x559d23769a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33777fe640 0 client.25594 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33a1590740 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d23758ea0 0x559d23762a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33a1590740 1 --2- 172.21.15.83:0/2903502961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d23762510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.761+0000 7f33a1590740 1 -- 172.21.15.83:0/2903502961 >> 172.21.15.83:0/2903502961 conn(0x559d23753a20 msgr2=0x559d23752c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.762+0000 7f33a1590740 1 -- 172.21.15.83:0/2903502961 shutdown_connections 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.762+0000 7f33a1590740 1 -- 172.21.15.83:0/2903502961 wait complete. 2024-03-11T19:42:52.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.762+0000 7f33a1590740 20 client.25594 trim_cache size 0 max 16384 2024-03-11T19:42:52.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.785+0000 7f33a1590740 1 Processor -- start 2024-03-11T19:42:52.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.785+0000 7f33a1590740 1 -- start start 2024-03-11T19:42:52.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.785+0000 7f33a1590740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d2375a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.785+0000 7f33a1590740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d23758ea0 0x559d2375a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f33a1590740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d23769a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f33a1590740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d233fd720 con 0x559d23758ea0 2024-03-11T19:42:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f33a1590740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d233c07e0 con 0x559d2375e010 2024-03-11T19:42:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f33a1590740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2376af80 con 0x559d237584b0 2024-03-11T19:42:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f3397fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d2375a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f3397fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d2375a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:54744/0 (socket says 172.21.15.83:54744) 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f3397fff640 1 -- 172.21.15.83:0/1733032326 learned_addr learned my addr 172.21.15.83:0/1733032326 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f338f7fe640 1 --2- 172.21.15.83:0/1733032326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d23758ea0 0x559d2375a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f3397fff640 1 -- 172.21.15.83:0/1733032326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 msgr2=0x559d23769a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f3397fff640 1 --2- 172.21.15.83:0/1733032326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d23769a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f3397fff640 1 -- 172.21.15.83:0/1733032326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d23758ea0 msgr2=0x559d2375a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f3397fff640 1 --2- 172.21.15.83:0/1733032326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d23758ea0 0x559d2375a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f3397fff640 1 -- 172.21.15.83:0/1733032326 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d2365bac0 con 0x559d237584b0 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.786+0000 7f338f7fe640 1 --2- 172.21.15.83:0/1733032326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d23758ea0 0x559d2375a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.787+0000 7f3397fff640 1 --2- 172.21.15.83:0/1733032326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d2375a2c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f338009a010 tx=0x7f338001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96192a70c4007d2c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.787+0000 7f338dffb640 1 -- 172.21.15.83:0/1733032326 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380096760 con 0x559d237584b0 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.787+0000 7f338dffb640 1 -- 172.21.15.83:0/1733032326 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33800968c0 con 0x559d237584b0 2024-03-11T19:42:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.787+0000 7f338dffb640 1 -- 172.21.15.83:0/1733032326 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3380095dd0 con 0x559d237584b0 2024-03-11T19:42:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.788+0000 7f33a1590740 1 -- 172.21.15.83:0/1733032326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 msgr2=0x559d2375a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.788+0000 7f33a1590740 1 --2- 172.21.15.83:0/1733032326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d2375a2c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f338009a010 tx=0x7f338001d000 comp rx=0 tx=0).stop 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.788+0000 7f33a1590740 1 -- 172.21.15.83:0/1733032326 shutdown_connections 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.788+0000 7f33a1590740 1 --2- 172.21.15.83:0/1733032326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d23769a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.788+0000 7f33a1590740 1 --2- 172.21.15.83:0/1733032326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d23758ea0 0x559d2375a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.788+0000 7f33a1590740 1 --2- 172.21.15.83:0/1733032326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d2375a2c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.788+0000 7f33a1590740 1 -- 172.21.15.83:0/1733032326 >> 172.21.15.83:0/1733032326 conn(0x559d23753a20 msgr2=0x559d2374e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.788+0000 7f33a1590740 1 -- 172.21.15.83:0/1733032326 shutdown_connections 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.789+0000 7f33a1590740 1 -- 172.21.15.83:0/1733032326 wait complete. 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.789+0000 7f33a1590740 1 -- start start 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f33a1590740 1 Processor -- start 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f33a1590740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d23769680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f33a1590740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d23758ea0 0x559d23767c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f33a1590740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d237681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f33a1590740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d23769e70 con 0x559d23758ea0 2024-03-11T19:42:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f33a1590740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d23768710 con 0x559d2375e010 2024-03-11T19:42:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f33a1590740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d237688d0 con 0x559d237584b0 2024-03-11T19:42:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f33a1590740 10 client.0 _ll_register_callbacks cb 0x7f33800035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f3397fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d23769680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f339cf59640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d237681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f338f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d23758ea0 0x559d23767c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f339cf59640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d237681d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37392/0 (socket says 172.21.15.83:37392) 2024-03-11T19:42:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.790+0000 7f339cf59640 1 -- 172.21.15.83:0/1133884094 learned_addr learned my addr 172.21.15.83:0/1133884094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:52.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.791+0000 7f3397fff640 1 -- 172.21.15.83:0/1133884094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 msgr2=0x559d237681d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.791+0000 7f3397fff640 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d237681d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.791+0000 7f3397fff640 1 -- 172.21.15.83:0/1133884094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d23758ea0 msgr2=0x559d23767c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.791+0000 7f3397fff640 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d23758ea0 0x559d23767c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.791+0000 7f3397fff640 1 -- 172.21.15.83:0/1133884094 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33880039f0 con 0x559d237584b0 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.791+0000 7f3397fff640 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d23769680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f338009e390 tx=0x7f338009a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f68eb06a4999b6e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.791+0000 7f33777fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.791+0000 7f339cf59640 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d237681d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.791+0000 7f33777fe640 1 -- 172.21.15.83:0/1133884094 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800b8600 con 0x559d237584b0 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.791+0000 7f33777fe640 1 -- 172.21.15.83:0/1133884094 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f33800966f0 con 0x559d237584b0 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.791+0000 7f33a1590740 1 -- 172.21.15.83:0/1133884094 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33800a2820 con 0x559d237584b0 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.792+0000 7f33777fe640 1 -- 172.21.15.83:0/1133884094 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3380095fe0 con 0x559d237584b0 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.792+0000 7f33a1590740 10 client.25600 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:52.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.792+0000 7f33a1590740 1 -- 172.21.15.83:0/1133884094 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559d236bf610 con 0x559d237584b0 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.792+0000 7f33777fe640 1 -- 172.21.15.83:0/1133884094 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3380098f40 con 0x559d237584b0 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.792+0000 7f33777fe640 1 -- 172.21.15.83:0/1133884094 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f33800b87f0 con 0x559d237584b0 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.792+0000 7f33a1590740 10 client.25600 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.792+0000 7f33a1590740 10 client.25600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.792+0000 7f33a1590740 20 client.25600 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.792+0000 7f33a1590740 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559d237c5a90 0x559d237c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.792+0000 7f33a1590740 4 client.25600 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.792+0000 7f33a1590740 1 -- 172.21.15.83:0/1133884094 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f33800a2820 con 0x559d237c5a90 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.793+0000 7f338f7fe640 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559d237c5a90 0x559d237c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.793+0000 7f338f7fe640 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559d237c5a90 0x559d237c7e80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d216ef7c7afbf7 server_cookie=57f0a56c10377605 in_seq=0 out_seq=0 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.793+0000 7f33777fe640 10 client.25600 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.793+0000 7f33777fe640 1 -- 172.21.15.83:0/1133884094 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f33800a2820 con 0x559d237c5a90 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.793+0000 7f33777fe640 10 client.25600 handle_command_reply: tid=0 2024-03-11T19:42:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.794+0000 7f33a1590740 1 client.25600 shutdown 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.794+0000 7f33a1590740 1 -- 172.21.15.83:0/1133884094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 msgr2=0x559d23769680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f33a1590740 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d23769680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f338009e390 tx=0x7f338009a200 comp rx=0 tx=0).stop 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f33a1590740 1 -- 172.21.15.83:0/1133884094 shutdown_connections 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f33a1590740 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559d237c5a90 0x559d237c7e80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f33a1590740 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559d2375e010 0x559d237681d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f33a1590740 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d23758ea0 0x559d23767c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f33a1590740 1 --2- 172.21.15.83:0/1133884094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559d237584b0 0x559d23769680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f33a1590740 1 -- 172.21.15.83:0/1133884094 >> 172.21.15.83:0/1133884094 conn(0x559d23753a20 msgr2=0x559d2375a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f33777fe640 0 client.25600 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f33a1590740 1 -- 172.21.15.83:0/1133884094 shutdown_connections 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f33a1590740 1 -- 172.21.15.83:0/1133884094 wait complete. 2024-03-11T19:42:52.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f33a1590740 20 client.25600 trim_cache size 0 max 16384 2024-03-11T19:42:52.803 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:42:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:42:52.803 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:42:52.803 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.795+0000 7f339d75a640 1 -- 172.21.15.83:0/3628186100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33840a1c30 msgr2=0x7f33840a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.796+0000 7f339d75a640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33840a1c30 0x7f33840a40f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3398196360 tx=0x7f3388009500 comp rx=0 tx=0).stop 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.796+0000 7f339d75a640 1 -- 172.21.15.83:0/3628186100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33980ff6e0 msgr2=0x7f3398194e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.796+0000 7f339d75a640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33980ff6e0 0x7f3398194e50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f338000ea60 tx=0x7f338000ef20 comp rx=0 tx=0).stop 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.796+0000 7f339d75a640 1 -- 172.21.15.83:0/3628186100 shutdown_connections 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.796+0000 7f339d75a640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f33840a1c30 0x7f33840a40f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.796+0000 7f339d75a640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3398101290 0x7f339819c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.796+0000 7f339d75a640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33981008d0 0x7f3398195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.796+0000 7f339d75a640 1 --2- 172.21.15.83:0/3628186100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33980ff6e0 0x7f3398194e50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.796+0000 7f339d75a640 1 -- 172.21.15.83:0/3628186100 >> 172.21.15.83:0/3628186100 conn(0x7f33980fac70 msgr2=0x7f339810a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.796+0000 7f339d75a640 1 -- 172.21.15.83:0/3628186100 shutdown_connections 2024-03-11T19:42:52.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.796+0000 7f339d75a640 1 -- 172.21.15.83:0/3628186100 wait complete. 2024-03-11T19:42:52.818 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:42:52.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:42:52.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.976+0000 7f65b6ffd640 1 -- 172.21.15.83:0/296567199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac02cbe0 con 0x7f65b8102390 2024-03-11T19:42:52.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.976+0000 7f65be40e640 1 -- 172.21.15.83:0/296567199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b8102390 msgr2=0x7f65b8104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:52.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.976+0000 7f65be40e640 1 --2- 172.21.15.83:0/296567199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b8102390 0x7f65b8104800 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009900 tx=0x7f65ac02bc30 comp rx=0 tx=0).stop 2024-03-11T19:42:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.977+0000 7f65be40e640 1 -- 172.21.15.83:0/296567199 shutdown_connections 2024-03-11T19:42:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.977+0000 7f65be40e640 1 --2- 172.21.15.83:0/296567199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b8104d40 0x7f65b81051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.977+0000 7f65be40e640 1 --2- 172.21.15.83:0/296567199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b8102390 0x7f65b8104800 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.977+0000 7f65be40e640 1 --2- 172.21.15.83:0/296567199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b80ffa20 0x7f65b8101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.977+0000 7f65be40e640 1 -- 172.21.15.83:0/296567199 >> 172.21.15.83:0/296567199 conn(0x7f65b80fb660 msgr2=0x7f65b80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.977+0000 7f65be40e640 1 -- 172.21.15.83:0/296567199 shutdown_connections 2024-03-11T19:42:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.977+0000 7f65be40e640 1 -- 172.21.15.83:0/296567199 wait complete. 2024-03-11T19:42:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.977+0000 7f65be40e640 1 Processor -- start 2024-03-11T19:42:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.977+0000 7f65be40e640 1 -- start start 2024-03-11T19:42:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65be40e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b80ffa20 0x7f65b806f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65be40e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b8102390 0x7f65b806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65be40e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b8104d40 0x7f65b819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65be40e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b8072f40 con 0x7f65b8104d40 2024-03-11T19:42:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65be40e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b80730b0 con 0x7f65b80ffa20 2024-03-11T19:42:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65be40e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b8073270 con 0x7f65b8102390 2024-03-11T19:42:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65bc9ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b8104d40 0x7f65b819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65bc9ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b8104d40 0x7f65b819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58956/0 (socket says 172.21.15.83:58956) 2024-03-11T19:42:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65bc9ab640 1 -- 172.21.15.83:0/2522964570 learned_addr learned my addr 172.21.15.83:0/2522964570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65b7fff640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b80ffa20 0x7f65b806f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65bc9ab640 1 -- 172.21.15.83:0/2522964570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b8102390 msgr2=0x7f65b806fe80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-11T19:42:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.978+0000 7f65bc9ab640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b8102390 0x7f65b806fe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.979+0000 7f65bc9ab640 1 -- 172.21.15.83:0/2522964570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b80ffa20 msgr2=0x7f65b806f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.979+0000 7f65bc9ab640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b80ffa20 0x7f65b806f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.979+0000 7f65bc9ab640 1 -- 172.21.15.83:0/2522964570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a8009840 con 0x7f65b8104d40 2024-03-11T19:42:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.979+0000 7f65bc9ab640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b8104d40 0x7f65b819ca40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f65b000e970 tx=0x7f65b000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.979+0000 7f65b57fa640 1 -- 172.21.15.83:0/2522964570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b0009720 con 0x7f65b8104d40 2024-03-11T19:42:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.979+0000 7f65b57fa640 1 -- 172.21.15.83:0/2522964570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65b0005500 con 0x7f65b8104d40 2024-03-11T19:42:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.980+0000 7f65b57fa640 1 -- 172.21.15.83:0/2522964570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b0010430 con 0x7f65b8104d40 2024-03-11T19:42:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.980+0000 7f65be40e640 1 -- 172.21.15.83:0/2522964570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ac009560 con 0x7f65b8104d40 2024-03-11T19:42:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.980+0000 7f65be40e640 1 -- 172.21.15.83:0/2522964570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b819d420 con 0x7f65b8104d40 2024-03-11T19:42:52.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.983+0000 7f65be40e640 1 -- 172.21.15.83:0/2522964570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b80683c0 con 0x7f65b8104d40 2024-03-11T19:42:52.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.984+0000 7f65b57fa640 1 -- 172.21.15.83:0/2522964570 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f65b000ccc0 con 0x7f65b8104d40 2024-03-11T19:42:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.984+0000 7f65b57fa640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65a00a1a20 0x7f65a00a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.985+0000 7f65b57fa640 1 -- 172.21.15.83:0/2522964570 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f65b0014070 con 0x7f65b8104d40 2024-03-11T19:42:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.985+0000 7f65b7fff640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65a00a1a20 0x7f65a00a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.986+0000 7f65b7fff640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65a00a1a20 0x7f65a00a3ee0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f65b8070e90 tx=0x7f65a8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:52.993+0000 7f65b57fa640 1 -- 172.21.15.83:0/2522964570 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65b0096980 con 0x7f65b8104d40 2024-03-11T19:42:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:52 smithi083 ceph-mon[31864]: pgmap v459: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 18 MiB/s wr, 908 op/s 2024-03-11T19:42:53.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:52 smithi137 ceph-mon[36220]: pgmap v459: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 18 MiB/s wr, 908 op/s 2024-03-11T19:42:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:52 smithi204 ceph-mon[36739]: pgmap v459: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 18 MiB/s wr, 908 op/s 2024-03-11T19:42:53.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.348+0000 7f65be40e640 1 -- 172.21.15.83:0/2522964570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65b802cda0 con 0x7f65b8104d40 2024-03-11T19:42:53.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.349+0000 7f65b57fa640 1 -- 172.21.15.83:0/2522964570 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11351 (secure 0 0 0) 0x7f65b0096980 con 0x7f65b8104d40 2024-03-11T19:42:53.353 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:42:53.353 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:42:34.146352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:42:53.353 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2024-03-11T19:42:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 -- 172.21.15.83:0/2522964570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65a00a1a20 msgr2=0x7f65a00a3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65a00a1a20 0x7f65a00a3ee0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f65b8070e90 tx=0x7f65a8009500 comp rx=0 tx=0).stop 2024-03-11T19:42:53.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 -- 172.21.15.83:0/2522964570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b8104d40 msgr2=0x7f65b819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b8104d40 0x7f65b819ca40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f65b000e970 tx=0x7f65b000ed70 comp rx=0 tx=0).stop 2024-03-11T19:42:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 -- 172.21.15.83:0/2522964570 shutdown_connections 2024-03-11T19:42:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65a00a1a20 0x7f65a00a3ee0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65b8104d40 0x7f65b819ca40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65b8102390 0x7f65b806fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 --2- 172.21.15.83:0/2522964570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b80ffa20 0x7f65b806f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 -- 172.21.15.83:0/2522964570 >> 172.21.15.83:0/2522964570 conn(0x7f65b80fb660 msgr2=0x7f65b81006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 -- 172.21.15.83:0/2522964570 shutdown_connections 2024-03-11T19:42:53.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.351+0000 7f65be40e640 1 -- 172.21.15.83:0/2522964570 wait complete. 2024-03-11T19:42:53.370 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:42:53.370 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:42:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.508+0000 7ff4134df640 1 -- 172.21.15.83:0/3967722778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff40c1011b0 msgr2=0x7ff40c1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.508+0000 7ff4134df640 1 --2- 172.21.15.83:0/3967722778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff40c1011b0 0x7ff40c1015d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff408009900 tx=0x7ff40802bc30 comp rx=0 tx=0).stop 2024-03-11T19:42:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.508+0000 7ff4134df640 1 -- 172.21.15.83:0/3967722778 shutdown_connections 2024-03-11T19:42:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.508+0000 7ff4134df640 1 --2- 172.21.15.83:0/3967722778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff40c102d60 0x7ff40c1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.508+0000 7ff4134df640 1 --2- 172.21.15.83:0/3967722778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff40c1023a0 0x7ff40c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.508+0000 7ff4134df640 1 --2- 172.21.15.83:0/3967722778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff40c1011b0 0x7ff40c1015d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.508+0000 7ff4134df640 1 -- 172.21.15.83:0/3967722778 >> 172.21.15.83:0/3967722778 conn(0x7ff40c0fc720 msgr2=0x7ff40c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.508+0000 7ff4134df640 1 -- 172.21.15.83:0/3967722778 shutdown_connections 2024-03-11T19:42:53.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.508+0000 7ff4134df640 1 -- 172.21.15.83:0/3967722778 wait complete. 2024-03-11T19:42:53.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.509+0000 7ff4134df640 1 Processor -- start 2024-03-11T19:42:53.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.509+0000 7ff4134df640 1 -- start start 2024-03-11T19:42:53.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.509+0000 7ff4134df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff40c1011b0 0x7ff40c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.509+0000 7ff4134df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff40c1023a0 0x7ff40c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.509+0000 7ff4134df640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff40c102d60 0x7ff40c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.509+0000 7ff4134df640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c19ed30 con 0x7ff40c1011b0 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.509+0000 7ff4134df640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c19eea0 con 0x7ff40c1023a0 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.509+0000 7ff4134df640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c19f060 con 0x7ff40c102d60 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.510+0000 7ff41127b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff40c1011b0 0x7ff40c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.510+0000 7ff410a7a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff40c1023a0 0x7ff40c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.510+0000 7ff41127b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff40c1011b0 0x7ff40c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58984/0 (socket says 172.21.15.83:58984) 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.510+0000 7ff41127b640 1 -- 172.21.15.83:0/2113120543 learned_addr learned my addr 172.21.15.83:0/2113120543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.510+0000 7ff410a7a640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff40c1023a0 0x7ff40c197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37424/0 (socket says 172.21.15.83:37424) 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.510+0000 7ff41127b640 1 -- 172.21.15.83:0/2113120543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff40c102d60 msgr2=0x7ff40c19e7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.510+0000 7ff41127b640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff40c102d60 0x7ff40c19e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.510+0000 7ff41127b640 1 -- 172.21.15.83:0/2113120543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff40c1023a0 msgr2=0x7ff40c197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.510+0000 7ff41127b640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff40c1023a0 0x7ff40c197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.510+0000 7ff41127b640 1 -- 172.21.15.83:0/2113120543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff408009560 con 0x7ff40c1011b0 2024-03-11T19:42:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.511+0000 7ff41127b640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff40c1011b0 0x7ff40c197240 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff408000c00 tx=0x7ff408004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.511+0000 7ff4027fc640 1 -- 172.21.15.83:0/2113120543 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408004df0 con 0x7ff40c1011b0 2024-03-11T19:42:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.511+0000 7ff4027fc640 1 -- 172.21.15.83:0/2113120543 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff408005430 con 0x7ff40c1011b0 2024-03-11T19:42:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.511+0000 7ff4134df640 1 -- 172.21.15.83:0/2113120543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c06aba0 con 0x7ff40c1011b0 2024-03-11T19:42:53.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.511+0000 7ff4027fc640 1 -- 172.21.15.83:0/2113120543 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408003e80 con 0x7ff40c1011b0 2024-03-11T19:42:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.512+0000 7ff4134df640 1 -- 172.21.15.83:0/2113120543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c06aff0 con 0x7ff40c1011b0 2024-03-11T19:42:53.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.513+0000 7ff4027fc640 1 -- 172.21.15.83:0/2113120543 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff4080035f0 con 0x7ff40c1011b0 2024-03-11T19:42:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.514+0000 7ff4027fc640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3f80a1c30 0x7ff3f80a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.515+0000 7ff4027fc640 1 -- 172.21.15.83:0/2113120543 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff4080ef9e0 con 0x7ff40c1011b0 2024-03-11T19:42:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.515+0000 7ff410a7a640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3f80a1c30 0x7ff3f80a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.515+0000 7ff41618f740 1 Processor -- start 2024-03-11T19:42:53.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.515+0000 7ff41618f740 1 -- start start 2024-03-11T19:42:53.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.515+0000 7ff41618f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.515+0000 7ff41618f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.515+0000 7ff41618f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.515+0000 7ff41618f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa56d960 con 0x55f1aa5b1be0 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.515+0000 7ff41618f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa521310 con 0x55f1aa5b4d40 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.515+0000 7ff41618f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa51e840 con 0x55f1aa5b11f0 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff410a7a640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3f80a1c30 0x7ff3f80a40f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff40c198750 tx=0x7ff3fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff41127b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff41127b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:54806/0 (socket says 172.21.15.83:54806) 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff41127b640 1 -- 172.21.15.83:0/2419947167 learned_addr learned my addr 172.21.15.83:0/2419947167 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff411a7c640 1 --2- 172.21.15.83:0/2419947167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff410a7a640 1 --2- 172.21.15.83:0/2419947167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff41127b640 1 -- 172.21.15.83:0/2419947167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b4d40 msgr2=0x55f1aa5c1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff41127b640 1 --2- 172.21.15.83:0/2419947167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5c1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff41127b640 1 -- 172.21.15.83:0/2419947167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b1be0 msgr2=0x55f1aa5b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff41127b640 1 --2- 172.21.15.83:0/2419947167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff41127b640 1 -- 172.21.15.83:0/2419947167 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1aa4dc060 con 0x55f1aa5b11f0 2024-03-11T19:42:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff410a7a640 1 --2- 172.21.15.83:0/2419947167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:42:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff41127b640 1 --2- 172.21.15.83:0/2419947167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5b1610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4080b8d30 tx=0x7ff408004b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eda576305d749b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff3f77fe640 1 -- 172.21.15.83:0/2419947167 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080bc960 con 0x55f1aa5b11f0 2024-03-11T19:42:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff3f77fe640 1 -- 172.21.15.83:0/2419947167 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4080c0ab0 con 0x55f1aa5b11f0 2024-03-11T19:42:53.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.516+0000 7ff3f77fe640 1 -- 172.21.15.83:0/2419947167 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080c4e70 con 0x55f1aa5b11f0 2024-03-11T19:42:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.517+0000 7ff41618f740 1 -- 172.21.15.83:0/2419947167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 msgr2=0x55f1aa5b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.517+0000 7ff41618f740 1 --2- 172.21.15.83:0/2419947167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5b1610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4080b8d30 tx=0x7ff408004b70 comp rx=0 tx=0).stop 2024-03-11T19:42:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.517+0000 7ff41618f740 1 -- 172.21.15.83:0/2419947167 shutdown_connections 2024-03-11T19:42:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.517+0000 7ff41618f740 1 --2- 172.21.15.83:0/2419947167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5c1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.517+0000 7ff41618f740 1 --2- 172.21.15.83:0/2419947167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.517+0000 7ff41618f740 1 --2- 172.21.15.83:0/2419947167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5b1610 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.517+0000 7ff41618f740 1 -- 172.21.15.83:0/2419947167 >> 172.21.15.83:0/2419947167 conn(0x55f1aa5ac7e0 msgr2=0x55f1aa5acbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.517+0000 7ff41618f740 1 -- 172.21.15.83:0/2419947167 shutdown_connections 2024-03-11T19:42:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.518+0000 7ff41618f740 1 -- 172.21.15.83:0/2419947167 wait complete. 2024-03-11T19:42:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.518+0000 7ff41618f740 1 -- start start 2024-03-11T19:42:53.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.518+0000 7ff41618f740 1 Processor -- start 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.518+0000 7ff41618f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.518+0000 7ff41618f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.518+0000 7ff41618f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.518+0000 7ff41618f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa51a020 con 0x55f1aa5b11f0 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.518+0000 7ff41618f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa1ed430 con 0x55f1aa5b1be0 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.518+0000 7ff41618f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa1a9e00 con 0x55f1aa5b4d40 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff41618f740 10 client.0 _ll_register_callbacks cb 0x55f1aa4c1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff410a7a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff410a7a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5c3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37442/0 (socket says 172.21.15.83:37442) 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff410a7a640 1 -- 172.21.15.83:0/1607883322 learned_addr learned my addr 172.21.15.83:0/1607883322 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff411a7c640 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff41127b640 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff411a7c640 1 -- 172.21.15.83:0/1607883322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 msgr2=0x55f1aa5c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff411a7c640 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff411a7c640 1 -- 172.21.15.83:0/1607883322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 msgr2=0x55f1aa5c3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff411a7c640 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5c3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff411a7c640 1 -- 172.21.15.83:0/1607883322 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c06aff0 con 0x55f1aa5b4d40 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.519+0000 7ff410a7a640 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff411a7c640 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5cafc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff404005fd0 tx=0x7ff404010ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=577f09034c56c5dc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff3f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/1607883322 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40400e9a0 con 0x55f1aa5b4d40 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff41618f740 1 -- 172.21.15.83:0/1607883322 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1aa2197e0 con 0x55f1aa5b4d40 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/1607883322 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff40400eb00 con 0x55f1aa5b4d40 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/1607883322 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff40400ecc0 con 0x55f1aa5b4d40 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff41618f740 10 client.25612 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff41618f740 1 -- 172.21.15.83:0/1607883322 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f1aa5186a0 con 0x55f1aa5b4d40 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/1607883322 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f1aa2197e0 con 0x55f1aa5b4d40 2024-03-11T19:42:53.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/1607883322 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff404025070 con 0x55f1aa5b4d40 2024-03-11T19:42:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff41618f740 10 client.25612 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff41618f740 10 client.25612 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:42:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff41618f740 20 client.25612 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff41618f740 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f1aa5cfd60 0x55f1aa5d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff41618f740 4 client.25612 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.520+0000 7ff41618f740 1 -- 172.21.15.83:0/1607883322 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1aa256720 con 0x55f1aa5cfd60 2024-03-11T19:42:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.521+0000 7ff41127b640 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f1aa5cfd60 0x55f1aa5d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.521+0000 7ff3f4ff9640 10 client.25612 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:42:53.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.521+0000 7ff41127b640 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f1aa5cfd60 0x55f1aa5d2180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=286022b2dd8c0bcf server_cookie=16af6cc0535c4a11 in_seq=0 out_seq=0 2024-03-11T19:42:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.521+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/1607883322 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f1aa256720 con 0x55f1aa5cfd60 2024-03-11T19:42:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.521+0000 7ff3f4ff9640 10 client.25612 handle_command_reply: tid=0 2024-03-11T19:42:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.521+0000 7ff41618f740 1 client.25612 shutdown 2024-03-11T19:42:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.522+0000 7ff41618f740 1 -- 172.21.15.83:0/1607883322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b4d40 msgr2=0x55f1aa5cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.522+0000 7ff41618f740 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5cafc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff404005fd0 tx=0x7ff404010ad0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.522+0000 7ff41618f740 1 -- 172.21.15.83:0/1607883322 shutdown_connections 2024-03-11T19:42:53.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.522+0000 7ff41618f740 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f1aa5cfd60 0x55f1aa5d2180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.522+0000 7ff41618f740 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5cafc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.522+0000 7ff41618f740 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.522+0000 7ff41618f740 1 --2- 172.21.15.83:0/1607883322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5c3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.522+0000 7ff41618f740 1 -- 172.21.15.83:0/1607883322 >> 172.21.15.83:0/1607883322 conn(0x55f1aa5ac7e0 msgr2=0x55f1aa5abb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.522+0000 7ff3f4ff9640 0 client.25612 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:42:53.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.523+0000 7ff41618f740 1 -- 172.21.15.83:0/1607883322 shutdown_connections 2024-03-11T19:42:53.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.523+0000 7ff41618f740 1 -- 172.21.15.83:0/1607883322 wait complete. 2024-03-11T19:42:53.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.523+0000 7ff41618f740 20 client.25612 trim_cache size 0 max 16384 2024-03-11T19:42:53.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41618f740 1 Processor -- start 2024-03-11T19:42:53.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41618f740 1 -- start start 2024-03-11T19:42:53.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41618f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41618f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5bea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41618f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41618f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa49f130 con 0x55f1aa5b11f0 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41618f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa256720 con 0x55f1aa5b1be0 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41618f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa2197e0 con 0x55f1aa5b4d40 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41127b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41127b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5be510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59008/0 (socket says 172.21.15.83:59008) 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41127b640 1 -- 172.21.15.83:0/2181364802 learned_addr learned my addr 172.21.15.83:0/2181364802 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff410a7a640 1 --2- 172.21.15.83:0/2181364802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff411a7c640 1 --2- 172.21.15.83:0/2181364802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41127b640 1 -- 172.21.15.83:0/2181364802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b4d40 msgr2=0x55f1aa5caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41127b640 1 --2- 172.21.15.83:0/2181364802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41127b640 1 -- 172.21.15.83:0/2181364802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 msgr2=0x55f1aa5bea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41127b640 1 --2- 172.21.15.83:0/2181364802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5bea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.539+0000 7ff41127b640 1 -- 172.21.15.83:0/2181364802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1aa4b4c20 con 0x55f1aa5b11f0 2024-03-11T19:42:53.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.540+0000 7ff41127b640 1 --2- 172.21.15.83:0/2181364802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5be510 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff408009850 tx=0x7ff4080bc5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b4285282b89475 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.540+0000 7ff3f6ffd640 1 -- 172.21.15.83:0/2181364802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080bc960 con 0x55f1aa5b11f0 2024-03-11T19:42:53.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.540+0000 7ff3f6ffd640 1 -- 172.21.15.83:0/2181364802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4080d5250 con 0x55f1aa5b11f0 2024-03-11T19:42:53.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.540+0000 7ff3f6ffd640 1 -- 172.21.15.83:0/2181364802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4080bbdb0 con 0x55f1aa5b11f0 2024-03-11T19:42:53.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.540+0000 7ff41618f740 1 -- 172.21.15.83:0/2181364802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 msgr2=0x55f1aa5be510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.540+0000 7ff41618f740 1 --2- 172.21.15.83:0/2181364802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5be510 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff408009850 tx=0x7ff4080bc5d0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.541+0000 7ff41618f740 1 -- 172.21.15.83:0/2181364802 shutdown_connections 2024-03-11T19:42:53.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.541+0000 7ff41618f740 1 --2- 172.21.15.83:0/2181364802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.541+0000 7ff41618f740 1 --2- 172.21.15.83:0/2181364802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5bea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.541+0000 7ff41618f740 1 --2- 172.21.15.83:0/2181364802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5be510 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.541+0000 7ff41618f740 1 -- 172.21.15.83:0/2181364802 >> 172.21.15.83:0/2181364802 conn(0x55f1aa5ac7e0 msgr2=0x55f1aa5acbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.541+0000 7ff41618f740 1 -- 172.21.15.83:0/2181364802 shutdown_connections 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.541+0000 7ff41618f740 1 -- 172.21.15.83:0/2181364802 wait complete. 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.542+0000 7ff41618f740 1 -- start start 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.542+0000 7ff41618f740 1 Processor -- start 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff41618f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff41618f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff41618f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff41618f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa5ca390 con 0x55f1aa5b4d40 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff41618f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa5ca500 con 0x55f1aa5b1be0 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff41618f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1aa5ca6c0 con 0x55f1aa5b11f0 2024-03-11T19:42:53.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff41618f740 10 client.0 _ll_register_callbacks cb 0x7ff408005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff411a7c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff41127b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff410a7a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff41127b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:54824/0 (socket says 172.21.15.83:54824) 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff41127b640 1 -- 172.21.15.83:0/3663682332 learned_addr learned my addr 172.21.15.83:0/3663682332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff410a7a640 1 -- 172.21.15.83:0/3663682332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 msgr2=0x55f1aa5c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff410a7a640 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff410a7a640 1 -- 172.21.15.83:0/3663682332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b4d40 msgr2=0x55f1aa5c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff410a7a640 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff410a7a640 1 -- 172.21.15.83:0/3663682332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1aa4dc060 con 0x55f1aa5b1be0 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.543+0000 7ff411a7c640 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.544+0000 7ff410a7a640 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5c8e40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0044f0 tx=0x7ff3fc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c88675c4c52f7a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.544+0000 7ff3f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:42:53.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.544+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/3663682332 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc02e8e0 con 0x55f1aa5b1be0 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.544+0000 7ff41618f740 1 -- 172.21.15.83:0/3663682332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4080c0ab0 con 0x55f1aa5b1be0 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.544+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/3663682332 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff3fc02ea40 con 0x55f1aa5b1be0 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.544+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/3663682332 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3fc02ec00 con 0x55f1aa5b1be0 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.544+0000 7ff41618f740 10 client.25511 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.544+0000 7ff41618f740 1 -- 172.21.15.83:0/3663682332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f1aa4da8c0 con 0x55f1aa5b1be0 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.545+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/3663682332 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4080c0ab0 con 0x55f1aa5b1be0 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.545+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/3663682332 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff3fc004b50 con 0x55f1aa5b1be0 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.545+0000 7ff41618f740 10 client.25511 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.545+0000 7ff41618f740 10 client.25511 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.545+0000 7ff41618f740 20 client.25511 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.545+0000 7ff41618f740 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f1aa626e80 0x55f1aa6292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.545+0000 7ff41618f740 4 client.25511 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:42:53.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.545+0000 7ff41618f740 1 -- 172.21.15.83:0/3663682332 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f1aa2a5890 con 0x55f1aa626e80 2024-03-11T19:42:53.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.546+0000 7ff41127b640 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f1aa626e80 0x55f1aa6292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.546+0000 7ff3f4ff9640 10 client.25511 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:42:53.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.546+0000 7ff41127b640 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f1aa626e80 0x55f1aa6292a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5ad887f1105dd61 server_cookie=46dffa9719fb03f2 in_seq=0 out_seq=0 2024-03-11T19:42:53.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.546+0000 7ff3f4ff9640 1 -- 172.21.15.83:0/3663682332 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f1aa2a5890 con 0x55f1aa626e80 2024-03-11T19:42:53.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.546+0000 7ff3f4ff9640 10 client.25511 handle_command_reply: tid=0 2024-03-11T19:42:53.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.546+0000 7ff41618f740 1 client.25511 shutdown 2024-03-11T19:42:53.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff41618f740 1 -- 172.21.15.83:0/3663682332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 msgr2=0x55f1aa5c8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff41618f740 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5c8e40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0044f0 tx=0x7ff3fc004730 comp rx=0 tx=0).stop 2024-03-11T19:42:53.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff41618f740 1 -- 172.21.15.83:0/3663682332 shutdown_connections 2024-03-11T19:42:53.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff41618f740 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f1aa626e80 0x55f1aa6292a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff41618f740 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1aa5b4d40 0x55f1aa5c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff41618f740 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1aa5b1be0 0x55f1aa5c8e40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff41618f740 1 --2- 172.21.15.83:0/3663682332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1aa5b11f0 0x55f1aa5c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff41618f740 1 -- 172.21.15.83:0/3663682332 >> 172.21.15.83:0/3663682332 conn(0x55f1aa5ac7e0 msgr2=0x55f1aa5be510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff3f4ff9640 0 client.25511 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:42:53.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff41618f740 1 -- 172.21.15.83:0/3663682332 shutdown_connections 2024-03-11T19:42:53.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff41618f740 1 -- 172.21.15.83:0/3663682332 wait complete. 2024-03-11T19:42:53.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff41618f740 20 client.25511 trim_cache size 0 max 16384 2024-03-11T19:42:53.555 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:42:53.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff4134df640 1 -- 172.21.15.83:0/2113120543 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3f80a1c30 msgr2=0x7ff3f80a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff4134df640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3f80a1c30 0x7ff3f80a40f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff40c198750 tx=0x7ff3fc005f90 comp rx=0 tx=0).stop 2024-03-11T19:42:53.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff4134df640 1 -- 172.21.15.83:0/2113120543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff40c1011b0 msgr2=0x7ff40c197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.547+0000 7ff4134df640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff40c1011b0 0x7ff40c197240 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff408000c00 tx=0x7ff408004270 comp rx=0 tx=0).stop 2024-03-11T19:42:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.548+0000 7ff4134df640 1 -- 172.21.15.83:0/2113120543 shutdown_connections 2024-03-11T19:42:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.548+0000 7ff4134df640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3f80a1c30 0x7ff3f80a40f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.548+0000 7ff4134df640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff40c102d60 0x7ff40c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.548+0000 7ff4134df640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff40c1023a0 0x7ff40c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.548+0000 7ff4134df640 1 --2- 172.21.15.83:0/2113120543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff40c1011b0 0x7ff40c197240 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.548+0000 7ff4134df640 1 -- 172.21.15.83:0/2113120543 >> 172.21.15.83:0/2113120543 conn(0x7ff40c0fc720 msgr2=0x7ff40c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.548+0000 7ff4134df640 1 -- 172.21.15.83:0/2113120543 shutdown_connections 2024-03-11T19:42:53.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.548+0000 7ff4134df640 1 -- 172.21.15.83:0/2113120543 wait complete. 2024-03-11T19:42:53.567 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:42:53.567 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:42:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.707+0000 7fd03c6c9640 1 -- 172.21.15.83:0/3629732379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd034102e90 msgr2=0x7fd034109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.707+0000 7fd03c6c9640 1 --2- 172.21.15.83:0/3629732379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd034102e90 0x7fd034109720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009be0 tx=0x7fd02c02bef0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.707+0000 7fd03c6c9640 1 -- 172.21.15.83:0/3629732379 shutdown_connections 2024-03-11T19:42:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.707+0000 7fd03c6c9640 1 --2- 172.21.15.83:0/3629732379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd034102e90 0x7fd034109720 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.707+0000 7fd03c6c9640 1 --2- 172.21.15.83:0/3629732379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0341024d0 0x7fd034102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.707+0000 7fd03c6c9640 1 --2- 172.21.15.83:0/3629732379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0340ff460 0x7fd0340ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.707+0000 7fd03c6c9640 1 -- 172.21.15.83:0/3629732379 >> 172.21.15.83:0/3629732379 conn(0x7fd0340fad70 msgr2=0x7fd0340fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.707+0000 7fd03c6c9640 1 -- 172.21.15.83:0/3629732379 shutdown_connections 2024-03-11T19:42:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.707+0000 7fd03c6c9640 1 -- 172.21.15.83:0/3629732379 wait complete. 2024-03-11T19:42:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.708+0000 7fd03c6c9640 1 Processor -- start 2024-03-11T19:42:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.708+0000 7fd03c6c9640 1 -- start start 2024-03-11T19:42:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.708+0000 7fd03c6c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0340ff460 0x7fd0341971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.708+0000 7fd03c6c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0341024d0 0x7fd0341976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.708+0000 7fd03c6c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd034102e90 0x7fd03419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.709+0000 7fd03c6c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03419eca0 con 0x7fd034102e90 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.709+0000 7fd03c6c9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03419ee10 con 0x7fd0340ff460 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.709+0000 7fd03c6c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03419efd0 con 0x7fd0341024d0 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.709+0000 7fd03a465640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0340ff460 0x7fd0341971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.709+0000 7fd03a465640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0340ff460 0x7fd0341971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37488/0 (socket says 172.21.15.83:37488) 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.709+0000 7fd03a465640 1 -- 172.21.15.83:0/2967735699 learned_addr learned my addr 172.21.15.83:0/2967735699 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.709+0000 7fd03ac66640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd034102e90 0x7fd03419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.709+0000 7fd03a465640 1 -- 172.21.15.83:0/2967735699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0341024d0 msgr2=0x7fd0341976f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.710+0000 7fd03a465640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0341024d0 0x7fd0341976f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.710+0000 7fd03a465640 1 -- 172.21.15.83:0/2967735699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd034102e90 msgr2=0x7fd03419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.710+0000 7fd03a465640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd034102e90 0x7fd03419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.710+0000 7fd03a465640 1 -- 172.21.15.83:0/2967735699 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c009840 con 0x7fd0340ff460 2024-03-11T19:42:53.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.710+0000 7fd03a465640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0340ff460 0x7fd0341971b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd030009e10 tx=0x7fd03000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.710+0000 7fd02b7fe640 1 -- 172.21.15.83:0/2967735699 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000ccc0 con 0x7fd0340ff460 2024-03-11T19:42:53.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.710+0000 7fd02b7fe640 1 -- 172.21.15.83:0/2967735699 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd03000ce20 con 0x7fd0340ff460 2024-03-11T19:42:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.711+0000 7fd03c6c9640 1 -- 172.21.15.83:0/2967735699 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd034100320 con 0x7fd0340ff460 2024-03-11T19:42:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.711+0000 7fd03c6c9640 1 -- 172.21.15.83:0/2967735699 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0341007c0 con 0x7fd0340ff460 2024-03-11T19:42:53.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.711+0000 7fd02b7fe640 1 -- 172.21.15.83:0/2967735699 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030013ea0 con 0x7fd0340ff460 2024-03-11T19:42:53.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.713+0000 7fd02b7fe640 1 -- 172.21.15.83:0/2967735699 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd030003850 con 0x7fd0340ff460 2024-03-11T19:42:53.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.715+0000 7fd03f37a740 1 Processor -- start 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.715+0000 7fd02b7fe640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd0200a1ab0 0x7fd0200a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.715+0000 7fd03f37a740 1 -- start start 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.715+0000 7fd03ac66640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd0200a1ab0 0x7fd0200a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.715+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2d5a80 0x55573b2d5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.715+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2d6470 0x55573b2daf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.715+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2db5e0 0x55573b2dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.715+0000 7fd03f37a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573b289960 con 0x55573b2d5a80 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.715+0000 7fd03f37a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573b23d310 con 0x55573b2db5e0 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.715+0000 7fd03f37a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573b23a840 con 0x55573b2d6470 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03a465640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2d5a80 0x55573b2d5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd02b7fe640 1 -- 172.21.15.83:0/2967735699 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd03009fee0 con 0x7fd0340ff460 2024-03-11T19:42:53.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03a465640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2d5a80 0x55573b2d5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59034/0 (socket says 172.21.15.83:59034) 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03a465640 1 -- 172.21.15.83:0/3206401097 learned_addr learned my addr 172.21.15.83:0/3206401097 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd039c64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2d6470 0x55573b2daf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03ac66640 1 --2- 172.21.15.83:0/3206401097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2db5e0 0x55573b2dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03ac66640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd0200a1ab0 0x7fd0200a3f70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd02c004af0 tx=0x7fd02c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03a465640 1 -- 172.21.15.83:0/3206401097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2d6470 msgr2=0x55573b2daf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03a465640 1 --2- 172.21.15.83:0/3206401097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2d6470 0x55573b2daf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03a465640 1 -- 172.21.15.83:0/3206401097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2db5e0 msgr2=0x55573b2dda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03a465640 1 --2- 172.21.15.83:0/3206401097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2db5e0 0x55573b2dda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03a465640 1 -- 172.21.15.83:0/3206401097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55573b1f8060 con 0x55573b2d5a80 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03ac66640 1 --2- 172.21.15.83:0/3206401097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2db5e0 0x55573b2dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd03a465640 1 --2- 172.21.15.83:0/3206401097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2d5a80 0x55573b2d5ea0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd03000e9c0 tx=0x7fd03009b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8749312f91e7e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd028ff9640 1 -- 172.21.15.83:0/3206401097 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0300a7f80 con 0x55573b2d5a80 2024-03-11T19:42:53.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.716+0000 7fd028ff9640 1 -- 172.21.15.83:0/3206401097 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0300d0480 con 0x55573b2d5a80 2024-03-11T19:42:53.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.717+0000 7fd028ff9640 1 -- 172.21.15.83:0/3206401097 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0300a7d50 con 0x55573b2d5a80 2024-03-11T19:42:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.717+0000 7fd03f37a740 1 -- 172.21.15.83:0/3206401097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2d5a80 msgr2=0x55573b2d5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.717+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3206401097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2d5a80 0x55573b2d5ea0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd03000e9c0 tx=0x7fd03009b410 comp rx=0 tx=0).stop 2024-03-11T19:42:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.718+0000 7fd03f37a740 1 -- 172.21.15.83:0/3206401097 shutdown_connections 2024-03-11T19:42:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.718+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3206401097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2db5e0 0x55573b2dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.718+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3206401097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2d6470 0x55573b2daf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.718+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3206401097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2d5a80 0x55573b2d5ea0 secure :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd03000e9c0 tx=0x7fd03009b410 comp rx=0 tx=0).stop 2024-03-11T19:42:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.718+0000 7fd03f37a740 1 -- 172.21.15.83:0/3206401097 >> 172.21.15.83:0/3206401097 conn(0x55573b2c87e0 msgr2=0x55573b2c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.718+0000 7fd03f37a740 1 -- 172.21.15.83:0/3206401097 shutdown_connections 2024-03-11T19:42:53.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.719+0000 7fd03f37a740 1 -- 172.21.15.83:0/3206401097 wait complete. 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.719+0000 7fd03f37a740 1 -- start start 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.719+0000 7fd03f37a740 1 Processor -- start 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 0x55573b2e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2e5410 0x55573b2e7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03f37a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573b236020 con 0x55573b2db5e0 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03f37a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573af09430 con 0x55573b2d6470 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd039c64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 0x55573b2e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03f37a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573aec5e00 con 0x55573b2e5410 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03a465640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03a465640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2dfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37508/0 (socket says 172.21.15.83:37508) 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03ac66640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2e5410 0x55573b2e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd039c64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 0x55573b2e03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59048/0 (socket says 172.21.15.83:59048) 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03a465640 1 -- 172.21.15.83:0/3165352992 learned_addr learned my addr 172.21.15.83:0/3165352992 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03f37a740 10 client.0 _ll_register_callbacks cb 0x55573b1ddcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03ac66640 1 -- 172.21.15.83:0/3165352992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 msgr2=0x55573b2dfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03ac66640 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2dfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03ac66640 1 -- 172.21.15.83:0/3165352992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 msgr2=0x55573b2e03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03ac66640 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 0x55573b2e03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03ac66640 1 -- 172.21.15.83:0/3165352992 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0341007c0 con 0x55573b2e5410 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.720+0000 7fd03a465640 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2dfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd03ac66640 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2e5410 0x55573b2e7800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd02c02c430 tx=0x7fd02c02bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cba7a054a582f750 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd01dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd01dffb640 1 -- 172.21.15.83:0/3165352992 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c004450 con 0x55573b2e5410 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd03f37a740 1 -- 172.21.15.83:0/3165352992 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55573af357e0 con 0x55573b2e5410 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd01dffb640 1 -- 172.21.15.83:0/3165352992 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd02c0045b0 con 0x55573b2e5410 2024-03-11T19:42:53.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd01dffb640 1 -- 172.21.15.83:0/3165352992 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd02c059030 con 0x55573b2e5410 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd03f37a740 10 client.25627 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd03f37a740 1 -- 172.21.15.83:0/3165352992 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55573b2346a0 con 0x55573b2e5410 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd01dffb640 1 -- 172.21.15.83:0/3165352992 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55573af357e0 con 0x55573b2e5410 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd01dffb640 1 -- 172.21.15.83:0/3165352992 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd02c053470 con 0x55573b2e5410 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd03f37a740 10 client.25627 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd03f37a740 10 client.25627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd03f37a740 20 client.25627 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55573b2ec450 0x55573b2ee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd03f37a740 4 client.25627 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.721+0000 7fd03f37a740 1 -- 172.21.15.83:0/3165352992 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55573af72720 con 0x55573b2ec450 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.722+0000 7fd03a465640 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55573b2ec450 0x55573b2ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.722+0000 7fd03a465640 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55573b2ec450 0x55573b2ee890 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b543f9402e587a7 server_cookie=4f78159cebc6ee7f in_seq=0 out_seq=0 2024-03-11T19:42:53.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.722+0000 7fd01dffb640 10 client.25627 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd01dffb640 1 -- 172.21.15.83:0/3165352992 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55573af72720 con 0x55573b2ec450 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd01dffb640 10 client.25627 handle_command_reply: tid=0 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 1 client.25627 shutdown 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 1 -- 172.21.15.83:0/3165352992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2e5410 msgr2=0x55573b2e7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2e5410 0x55573b2e7800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd02c02c430 tx=0x7fd02c02bef0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 1 -- 172.21.15.83:0/3165352992 shutdown_connections 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55573b2ec450 0x55573b2ee890 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2e5410 0x55573b2e7800 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd01dffb640 0 client.25627 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 0x55573b2e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3165352992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2dfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 1 -- 172.21.15.83:0/3165352992 >> 172.21.15.83:0/3165352992 conn(0x55573b2c87e0 msgr2=0x55573b2c7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 1 -- 172.21.15.83:0/3165352992 shutdown_connections 2024-03-11T19:42:53.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 1 -- 172.21.15.83:0/3165352992 wait complete. 2024-03-11T19:42:53.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.723+0000 7fd03f37a740 20 client.25627 trim_cache size 0 max 16384 2024-03-11T19:42:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.738+0000 7fd03f37a740 1 Processor -- start 2024-03-11T19:42:53.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.738+0000 7fd03f37a740 1 -- start start 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.738+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.738+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2db5e0 0x55573b2fccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.738+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2e5410 0x55573b2c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.738+0000 7fd03f37a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573b1bb130 con 0x55573b2e5410 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.738+0000 7fd03f37a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573b2d68a0 con 0x55573b2d6470 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.738+0000 7fd03f37a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573b2e8cf0 con 0x55573b2db5e0 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03ac66640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2e5410 0x55573b2c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03a465640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd039c64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2db5e0 0x55573b2fccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03ac66640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2e5410 0x55573b2c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59058/0 (socket says 172.21.15.83:59058) 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd039c64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2db5e0 0x55573b2fccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:54870/0 (socket says 172.21.15.83:54870) 2024-03-11T19:42:53.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03a465640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2fc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37518/0 (socket says 172.21.15.83:37518) 2024-03-11T19:42:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03ac66640 1 -- 172.21.15.83:0/3271568627 learned_addr learned my addr 172.21.15.83:0/3271568627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03ac66640 1 -- 172.21.15.83:0/3271568627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2db5e0 msgr2=0x55573b2fccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03ac66640 1 --2- 172.21.15.83:0/3271568627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2db5e0 0x55573b2fccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03ac66640 1 -- 172.21.15.83:0/3271568627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 msgr2=0x55573b2fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03ac66640 1 --2- 172.21.15.83:0/3271568627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03ac66640 1 -- 172.21.15.83:0/3271568627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55573b1d0c20 con 0x55573b2e5410 2024-03-11T19:42:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03a465640 1 --2- 172.21.15.83:0/3271568627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd03ac66640 1 --2- 172.21.15.83:0/3271568627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2e5410 0x55573b2c72f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd02c02cbc0 tx=0x7fd02c0333e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92320fee60a207f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd01ffff640 1 -- 172.21.15.83:0/3271568627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c059070 con 0x55573b2e5410 2024-03-11T19:42:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd01ffff640 1 -- 172.21.15.83:0/3271568627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd02c042040 con 0x55573b2e5410 2024-03-11T19:42:53.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.739+0000 7fd01ffff640 1 -- 172.21.15.83:0/3271568627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c034430 con 0x55573b2e5410 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.740+0000 7fd03f37a740 1 -- 172.21.15.83:0/3271568627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2e5410 msgr2=0x55573b2c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.740+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3271568627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2e5410 0x55573b2c72f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd02c02cbc0 tx=0x7fd02c0333e0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.740+0000 7fd03f37a740 1 -- 172.21.15.83:0/3271568627 shutdown_connections 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.740+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3271568627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2e5410 0x55573b2c72f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.740+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3271568627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2db5e0 0x55573b2fccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.740+0000 7fd03f37a740 1 --2- 172.21.15.83:0/3271568627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2d6470 0x55573b2fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.740+0000 7fd03f37a740 1 -- 172.21.15.83:0/3271568627 >> 172.21.15.83:0/3271568627 conn(0x55573b2c87e0 msgr2=0x55573b2d7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.740+0000 7fd03f37a740 1 -- 172.21.15.83:0/3271568627 shutdown_connections 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.740+0000 7fd03f37a740 1 -- 172.21.15.83:0/3271568627 wait complete. 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.740+0000 7fd03f37a740 1 -- start start 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.740+0000 7fd03f37a740 1 Processor -- start 2024-03-11T19:42:53.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2d6470 0x55573b2fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 0x55573b33f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd03f37a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2e5410 0x55573b341a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd03f37a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573b342120 con 0x55573b2db5e0 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd03f37a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573b342290 con 0x55573b2e5410 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd03f37a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55573b342450 con 0x55573b2d6470 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd03f37a740 10 client.0 _ll_register_callbacks cb 0x7fd03009b360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd039c64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 0x55573b33f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd039c64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 0x55573b33f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59070/0 (socket says 172.21.15.83:59070) 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd03a465640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2d6470 0x55573b2fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd039c64640 1 -- 172.21.15.83:0/425760599 learned_addr learned my addr 172.21.15.83:0/425760599 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd03ac66640 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2e5410 0x55573b341a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd039c64640 1 -- 172.21.15.83:0/425760599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2d6470 msgr2=0x55573b2fcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd039c64640 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2d6470 0x55573b2fcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd039c64640 1 -- 172.21.15.83:0/425760599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2e5410 msgr2=0x55573b341a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd039c64640 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2e5410 0x55573b341a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.741+0000 7fd039c64640 1 -- 172.21.15.83:0/425760599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55573b1f8060 con 0x55573b2db5e0 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.742+0000 7fd039c64640 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 0x55573b33f420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55573b2d6ee0 tx=0x7fd024010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d21b085a912fedd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.742+0000 7fd01dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.742+0000 7fd01dffb640 1 -- 172.21.15.83:0/425760599 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024004460 con 0x55573b2db5e0 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.742+0000 7fd01dffb640 1 -- 172.21.15.83:0/425760599 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd02400f870 con 0x55573b2db5e0 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.742+0000 7fd03f37a740 1 -- 172.21.15.83:0/425760599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0300d0480 con 0x55573b2db5e0 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.742+0000 7fd01dffb640 1 -- 172.21.15.83:0/425760599 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd02400fa30 con 0x55573b2db5e0 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.742+0000 7fd03f37a740 10 client.16041 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.742+0000 7fd03f37a740 1 -- 172.21.15.83:0/425760599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55573b2346a0 con 0x55573b2db5e0 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.743+0000 7fd01dffb640 1 -- 172.21.15.83:0/425760599 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd024018040 con 0x55573b2db5e0 2024-03-11T19:42:53.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.743+0000 7fd01dffb640 1 -- 172.21.15.83:0/425760599 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd02401c3f0 con 0x55573b2db5e0 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.743+0000 7fd03f37a740 10 client.16041 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.743+0000 7fd03f37a740 10 client.16041 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.743+0000 7fd03f37a740 20 client.16041 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.743+0000 7fd03f37a740 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55573b2ec450 0x55573b346050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.743+0000 7fd03f37a740 4 client.16041 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.743+0000 7fd03f37a740 1 -- 172.21.15.83:0/425760599 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fd0300d0480 con 0x55573b2ec450 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.743+0000 7fd03a465640 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55573b2ec450 0x55573b346050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd01dffb640 10 client.16041 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd03a465640 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55573b2ec450 0x55573b346050 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0bab2b5ff70908 server_cookie=5c0640facd109e8c in_seq=0 out_seq=0 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd01dffb640 1 -- 172.21.15.83:0/425760599 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fd0300d0480 con 0x55573b2ec450 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd01dffb640 10 client.16041 handle_command_reply: tid=0 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd03f37a740 1 client.16041 shutdown 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd03f37a740 1 -- 172.21.15.83:0/425760599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 msgr2=0x55573b33f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd03f37a740 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 0x55573b33f420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55573b2d6ee0 tx=0x7fd024010bb0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd03f37a740 1 -- 172.21.15.83:0/425760599 shutdown_connections 2024-03-11T19:42:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd03f37a740 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55573b2ec450 0x55573b346050 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd03f37a740 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55573b2e5410 0x55573b341a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd01dffb640 0 client.16041 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:42:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd03f37a740 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55573b2db5e0 0x55573b33f420 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd03f37a740 1 --2- 172.21.15.83:0/425760599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55573b2d6470 0x55573b2fcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.744+0000 7fd03f37a740 1 -- 172.21.15.83:0/425760599 >> 172.21.15.83:0/425760599 conn(0x55573b2c87e0 msgr2=0x55573b2d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.745+0000 7fd03f37a740 1 -- 172.21.15.83:0/425760599 shutdown_connections 2024-03-11T19:42:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.745+0000 7fd03f37a740 1 -- 172.21.15.83:0/425760599 wait complete. 2024-03-11T19:42:53.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.745+0000 7fd03f37a740 20 client.16041 trim_cache size 0 max 16384 2024-03-11T19:42:53.755 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:42:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 -- 172.21.15.83:0/2967735699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd0200a1ab0 msgr2=0x7fd0200a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd0200a1ab0 0x7fd0200a3f70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd02c004af0 tx=0x7fd02c005fd0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 -- 172.21.15.83:0/2967735699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0340ff460 msgr2=0x7fd0341971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0340ff460 0x7fd0341971b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd030009e10 tx=0x7fd03000edc0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 -- 172.21.15.83:0/2967735699 shutdown_connections 2024-03-11T19:42:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd0200a1ab0 0x7fd0200a3f70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd034102e90 0x7fd03419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0341024d0 0x7fd0341976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 --2- 172.21.15.83:0/2967735699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0340ff460 0x7fd0341971b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 -- 172.21.15.83:0/2967735699 >> 172.21.15.83:0/2967735699 conn(0x7fd0340fad70 msgr2=0x7fd034105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 -- 172.21.15.83:0/2967735699 shutdown_connections 2024-03-11T19:42:53.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.746+0000 7fd03c6c9640 1 -- 172.21.15.83:0/2967735699 wait complete. 2024-03-11T19:42:53.765 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:42:53.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:42:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.899+0000 7f396fc96640 1 -- 172.21.15.83:0/1797980893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3968100080 msgr2=0x7f39681004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.899+0000 7f396fc96640 1 --2- 172.21.15.83:0/1797980893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3968100080 0x7f39681004a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3964009900 tx=0x7f396402bc30 comp rx=0 tx=0).stop 2024-03-11T19:42:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.899+0000 7f396fc96640 1 -- 172.21.15.83:0/1797980893 shutdown_connections 2024-03-11T19:42:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.899+0000 7f396fc96640 1 --2- 172.21.15.83:0/1797980893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3968101c30 0x7f39681084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.899+0000 7f396fc96640 1 --2- 172.21.15.83:0/1797980893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3968101270 0x7f39681016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.899+0000 7f396fc96640 1 --2- 172.21.15.83:0/1797980893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3968100080 0x7f39681004a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.899+0000 7f396fc96640 1 -- 172.21.15.83:0/1797980893 >> 172.21.15.83:0/1797980893 conn(0x7f39680fb610 msgr2=0x7f39680fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.899+0000 7f396fc96640 1 -- 172.21.15.83:0/1797980893 shutdown_connections 2024-03-11T19:42:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.899+0000 7f396fc96640 1 -- 172.21.15.83:0/1797980893 wait complete. 2024-03-11T19:42:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396fc96640 1 Processor -- start 2024-03-11T19:42:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396fc96640 1 -- start start 2024-03-11T19:42:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396fc96640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3968100080 0x7f3968196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396fc96640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3968101270 0x7f3968196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396fc96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3968101c30 0x7f396819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396fc96640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396819dcf0 con 0x7f3968101c30 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396fc96640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396819de60 con 0x7f3968101270 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396fc96640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396819e020 con 0x7f3968100080 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396e233640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3968101c30 0x7f396819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396e233640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3968101c30 0x7f396819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59086/0 (socket says 172.21.15.83:59086) 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396e233640 1 -- 172.21.15.83:0/3627019570 learned_addr learned my addr 172.21.15.83:0/3627019570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396d231640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3968101270 0x7f3968196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396da32640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3968100080 0x7f3968196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396e233640 1 -- 172.21.15.83:0/3627019570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3968100080 msgr2=0x7f3968196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396e233640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3968100080 0x7f3968196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396e233640 1 -- 172.21.15.83:0/3627019570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3968101270 msgr2=0x7f3968196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396e233640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3968101270 0x7f3968196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.900+0000 7f396e233640 1 -- 172.21.15.83:0/3627019570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3964009560 con 0x7f3968101c30 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.901+0000 7f396e233640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3968101c30 0x7f396819d7b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3960009fd0 tx=0x7f396000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.901+0000 7f395effd640 1 -- 172.21.15.83:0/3627019570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396000bdd0 con 0x7f3968101c30 2024-03-11T19:42:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.901+0000 7f395effd640 1 -- 172.21.15.83:0/3627019570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3960009940 con 0x7f3968101c30 2024-03-11T19:42:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.901+0000 7f396fc96640 1 -- 172.21.15.83:0/3627019570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39680682e0 con 0x7f3968101c30 2024-03-11T19:42:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.901+0000 7f395effd640 1 -- 172.21.15.83:0/3627019570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39600106c0 con 0x7f3968101c30 2024-03-11T19:42:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.901+0000 7f396fc96640 1 -- 172.21.15.83:0/3627019570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3968068770 con 0x7f3968101c30 2024-03-11T19:42:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.904+0000 7f395effd640 1 -- 172.21.15.83:0/3627019570 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3960010820 con 0x7f3968101c30 2024-03-11T19:42:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f3972861740 1 Processor -- start 2024-03-11T19:42:53.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f3972861740 1 -- start start 2024-03-11T19:42:53.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f395effd640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f39540a1ab0 0x7f39540a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f3972861740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0dfb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f3972861740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0dfbc60 0x5625f0e00760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f3972861740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0e00dd0 0x5625f0e03260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f3972861740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f0db7960 con 0x5625f0e00dd0 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f3972861740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f0d6b310 con 0x5625f0dfb270 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f3972861740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f0d68840 con 0x5625f0dfbc60 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f395effd640 1 -- 172.21.15.83:0/3627019570 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f39600cd110 con 0x7f3968101c30 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f396e233640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0e00dd0 0x5625f0e03260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f396da32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0dfb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f396d231640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f39540a1ab0 0x7f39540a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f396da32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0dfb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37560/0 (socket says 172.21.15.83:37560) 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.905+0000 7f396da32640 1 -- 172.21.15.83:0/3470040172 learned_addr learned my addr 172.21.15.83:0/3470040172 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.906+0000 7f396d231640 1 --2- 172.21.15.83:0/3470040172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0dfbc60 0x5625f0e00760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.906+0000 7f396da32640 1 -- 172.21.15.83:0/3470040172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0dfbc60 msgr2=0x5625f0e00760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.906+0000 7f396da32640 1 --2- 172.21.15.83:0/3470040172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0dfbc60 0x5625f0e00760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.906+0000 7f396da32640 1 -- 172.21.15.83:0/3470040172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0e00dd0 msgr2=0x5625f0e03260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.906+0000 7f396da32640 1 --2- 172.21.15.83:0/3470040172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0e00dd0 0x5625f0e03260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.906+0000 7f396da32640 1 -- 172.21.15.83:0/3470040172 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625f0d26060 con 0x5625f0dfb270 2024-03-11T19:42:53.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.906+0000 7f396d231640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f39540a1ab0 0x7f39540a3f70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3958006e10 tx=0x7f3958009930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.906+0000 7f396da32640 1 --2- 172.21.15.83:0/3470040172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0dfb690 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3964000c00 tx=0x7f396402ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf5d6f1e20e18bcd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.906+0000 7f3953fff640 1 -- 172.21.15.83:0/3470040172 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39640046c0 con 0x5625f0dfb270 2024-03-11T19:42:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.906+0000 7f3953fff640 1 -- 172.21.15.83:0/3470040172 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3964004820 con 0x5625f0dfb270 2024-03-11T19:42:53.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.906+0000 7f3953fff640 1 -- 172.21.15.83:0/3470040172 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964005360 con 0x5625f0dfb270 2024-03-11T19:42:53.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.907+0000 7f3972861740 1 -- 172.21.15.83:0/3470040172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 msgr2=0x5625f0dfb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.907+0000 7f3972861740 1 --2- 172.21.15.83:0/3470040172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0dfb690 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3964000c00 tx=0x7f396402ca10 comp rx=0 tx=0).stop 2024-03-11T19:42:53.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.907+0000 7f3972861740 1 -- 172.21.15.83:0/3470040172 shutdown_connections 2024-03-11T19:42:53.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.907+0000 7f3972861740 1 --2- 172.21.15.83:0/3470040172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0e00dd0 0x5625f0e03260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.907+0000 7f3972861740 1 --2- 172.21.15.83:0/3470040172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0dfbc60 0x5625f0e00760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.907+0000 7f3972861740 1 --2- 172.21.15.83:0/3470040172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0dfb690 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.907+0000 7f3972861740 1 -- 172.21.15.83:0/3470040172 >> 172.21.15.83:0/3470040172 conn(0x5625f0df67e0 msgr2=0x5625f0df6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.908+0000 7f3972861740 1 -- 172.21.15.83:0/3470040172 shutdown_connections 2024-03-11T19:42:53.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.908+0000 7f3972861740 1 -- 172.21.15.83:0/3470040172 wait complete. 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.908+0000 7f3972861740 1 -- start start 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.908+0000 7f3972861740 1 Processor -- start 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.909+0000 7f3972861740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfb270 0x5625f0e0dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.909+0000 7f3972861740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0dfbc60 0x5625f0e0e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.909+0000 7f3972861740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0e00dd0 0x5625f0e15180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f3972861740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f0d64020 con 0x5625f0dfb270 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f3972861740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f0a37430 con 0x5625f0e00dd0 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396da32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfb270 0x5625f0e0dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f3972861740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f09f3e00 con 0x5625f0dfbc60 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396da32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfb270 0x5625f0e0dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59112/0 (socket says 172.21.15.83:59112) 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396da32640 1 -- 172.21.15.83:0/3322987862 learned_addr learned my addr 172.21.15.83:0/3322987862 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396d231640 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0dfbc60 0x5625f0e0e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f3972861740 10 client.0 _ll_register_callbacks cb 0x5625f0d0bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396e233640 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0e00dd0 0x5625f0e15180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396da32640 1 -- 172.21.15.83:0/3322987862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0dfbc60 msgr2=0x5625f0e0e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396da32640 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0dfbc60 0x5625f0e0e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396da32640 1 -- 172.21.15.83:0/3322987862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0e00dd0 msgr2=0x5625f0e15180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396da32640 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0e00dd0 0x5625f0e15180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396da32640 1 -- 172.21.15.83:0/3322987862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3968108dc0 con 0x5625f0dfb270 2024-03-11T19:42:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396d231640 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0dfbc60 0x5625f0e0e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f396da32640 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfb270 0x5625f0e0dc00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3964005ab0 tx=0x7f3964005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a622d7cfda9d280 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f39517fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:42:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f39517fa640 1 -- 172.21.15.83:0/3322987862 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39640046c0 con 0x5625f0dfb270 2024-03-11T19:42:53.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f39517fa640 1 -- 172.21.15.83:0/3322987862 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f396404a650 con 0x5625f0dfb270 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.910+0000 7f3972861740 1 -- 172.21.15.83:0/3322987862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625f0a637e0 con 0x5625f0dfb270 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f39517fa640 1 -- 172.21.15.83:0/3322987862 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f396404a810 con 0x5625f0dfb270 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f3972861740 10 client.16053 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f3972861740 1 -- 172.21.15.83:0/3322987862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5625f0d626a0 con 0x5625f0dfb270 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f39517fa640 1 -- 172.21.15.83:0/3322987862 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5625f0a637e0 con 0x5625f0dfb270 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f39517fa640 1 -- 172.21.15.83:0/3322987862 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f396405d070 con 0x5625f0dfb270 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f3972861740 10 client.16053 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f3972861740 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f3972861740 20 client.16053 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f3972861740 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5625f0e199f0 0x5625f0e1bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f3972861740 4 client.16053 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f3972861740 1 -- 172.21.15.83:0/3322987862 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625f0aa0720 con 0x5625f0e199f0 2024-03-11T19:42:53.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.911+0000 7f396d231640 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5625f0e199f0 0x5625f0e1bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.912+0000 7f396d231640 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5625f0e199f0 0x5625f0e1bde0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f3596a7319b281 server_cookie=3e36cf1c07d436db in_seq=0 out_seq=0 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.912+0000 7f39517fa640 10 client.16053 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f39517fa640 1 -- 172.21.15.83:0/3322987862 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625f0aa0720 con 0x5625f0e199f0 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f39517fa640 10 client.16053 handle_command_reply: tid=0 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f3972861740 1 client.16053 shutdown 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f3972861740 1 -- 172.21.15.83:0/3322987862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfb270 msgr2=0x5625f0e0dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f3972861740 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfb270 0x5625f0e0dc00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3964005ab0 tx=0x7f3964005700 comp rx=0 tx=0).stop 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f3972861740 1 -- 172.21.15.83:0/3322987862 shutdown_connections 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f3972861740 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5625f0e199f0 0x5625f0e1bde0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f39517fa640 0 client.16053 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f3972861740 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0e00dd0 0x5625f0e15180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f3972861740 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0dfbc60 0x5625f0e0e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f3972861740 1 --2- 172.21.15.83:0/3322987862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfb270 0x5625f0e0dc00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.913+0000 7f3972861740 1 -- 172.21.15.83:0/3322987862 >> 172.21.15.83:0/3322987862 conn(0x5625f0df67e0 msgr2=0x5625f0df5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.914+0000 7f3972861740 1 -- 172.21.15.83:0/3322987862 shutdown_connections 2024-03-11T19:42:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.914+0000 7f3972861740 1 -- 172.21.15.83:0/3322987862 wait complete. 2024-03-11T19:42:53.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.914+0000 7f3972861740 20 client.16053 trim_cache size 0 max 16384 2024-03-11T19:42:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f3972861740 1 Processor -- start 2024-03-11T19:42:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f3972861740 1 -- start start 2024-03-11T19:42:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f3972861740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0dfd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f3972861740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 0x5625f0dfd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f3972861740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0e00dd0 0x5625f0e15140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f3972861740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f0ce9130 con 0x5625f0dfbc60 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f3972861740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f0dfc090 con 0x5625f0dfb270 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f3972861740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f0df6650 con 0x5625f0e00dd0 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f396d231640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 0x5625f0dfd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f396da32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0dfd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f396d231640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 0x5625f0dfd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59114/0 (socket says 172.21.15.83:59114) 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f396d231640 1 -- 172.21.15.83:0/3972584148 learned_addr learned my addr 172.21.15.83:0/3972584148 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.929+0000 7f396e233640 1 --2- 172.21.15.83:0/3972584148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0e00dd0 0x5625f0e15140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.930+0000 7f396da32640 1 -- 172.21.15.83:0/3972584148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0e00dd0 msgr2=0x5625f0e15140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.930+0000 7f396da32640 1 --2- 172.21.15.83:0/3972584148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0e00dd0 0x5625f0e15140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.930+0000 7f396da32640 1 -- 172.21.15.83:0/3972584148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 msgr2=0x5625f0dfd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.930+0000 7f396da32640 1 --2- 172.21.15.83:0/3972584148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 0x5625f0dfd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.930+0000 7f396da32640 1 -- 172.21.15.83:0/3972584148 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625f0cfec20 con 0x5625f0dfb270 2024-03-11T19:42:53.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.930+0000 7f396da32640 1 --2- 172.21.15.83:0/3972584148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0dfd120 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f396402c170 tx=0x7f396402c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91487282118c0025 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.930+0000 7f39537fe640 1 -- 172.21.15.83:0/3972584148 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39640046c0 con 0x5625f0dfb270 2024-03-11T19:42:53.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.930+0000 7f39537fe640 1 -- 172.21.15.83:0/3972584148 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f396405a070 con 0x5625f0dfb270 2024-03-11T19:42:53.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.930+0000 7f39537fe640 1 -- 172.21.15.83:0/3972584148 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396405f6d0 con 0x5625f0dfb270 2024-03-11T19:42:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.931+0000 7f3972861740 1 -- 172.21.15.83:0/3972584148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 msgr2=0x5625f0dfd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.931+0000 7f3972861740 1 --2- 172.21.15.83:0/3972584148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0dfd120 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f396402c170 tx=0x7f396402c8e0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.931+0000 7f3972861740 1 -- 172.21.15.83:0/3972584148 shutdown_connections 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.931+0000 7f3972861740 1 --2- 172.21.15.83:0/3972584148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0e00dd0 0x5625f0e15140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.931+0000 7f3972861740 1 --2- 172.21.15.83:0/3972584148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 0x5625f0dfd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.931+0000 7f3972861740 1 --2- 172.21.15.83:0/3972584148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0dfd120 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.931+0000 7f3972861740 1 -- 172.21.15.83:0/3972584148 >> 172.21.15.83:0/3972584148 conn(0x5625f0df67e0 msgr2=0x5625f0df6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.931+0000 7f3972861740 1 -- 172.21.15.83:0/3972584148 shutdown_connections 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.931+0000 7f3972861740 1 -- 172.21.15.83:0/3972584148 wait complete. 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f3972861740 1 -- start start 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f3972861740 1 Processor -- start 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f3972861740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0e13190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f3972861740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 0x5625f0e136d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f3972861740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0e00dd0 0x5625f0e146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f3972861740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f0e14c20 con 0x5625f0dfbc60 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f3972861740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f0e14d90 con 0x5625f0dfb270 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f3972861740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f0e14f50 con 0x5625f0e00dd0 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f3972861740 10 client.0 _ll_register_callbacks cb 0x7f3964005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f396da32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0e13190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f396d231640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 0x5625f0e136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f396da32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0e13190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37578/0 (socket says 172.21.15.83:37578) 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f396da32640 1 -- 172.21.15.83:0/1685265752 learned_addr learned my addr 172.21.15.83:0/1685265752 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.932+0000 7f396e233640 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0e00dd0 0x5625f0e146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f396da32640 1 -- 172.21.15.83:0/1685265752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0e00dd0 msgr2=0x5625f0e146e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f396da32640 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0e00dd0 0x5625f0e146e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f396da32640 1 -- 172.21.15.83:0/1685265752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 msgr2=0x5625f0e136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f396da32640 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 0x5625f0e136d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f396da32640 1 -- 172.21.15.83:0/1685265752 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625f0d626a0 con 0x5625f0dfb270 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f396d231640 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 0x5625f0e136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f396da32640 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0e13190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f396403e000 tx=0x7f396402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=def475d48af983c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f39517fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f39517fa640 1 -- 172.21.15.83:0/1685265752 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396403fbc0 con 0x5625f0dfb270 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f3972861740 1 -- 172.21.15.83:0/1685265752 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3964004820 con 0x5625f0dfb270 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f39517fa640 1 -- 172.21.15.83:0/1685265752 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3964005700 con 0x5625f0dfb270 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f39517fa640 1 -- 172.21.15.83:0/1685265752 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3964004030 con 0x5625f0dfb270 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f3972861740 10 client.25544 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f3972861740 1 -- 172.21.15.83:0/1685265752 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5625f0d248c0 con 0x5625f0dfb270 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.933+0000 7f39517fa640 1 -- 172.21.15.83:0/1685265752 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3964004820 con 0x5625f0dfb270 2024-03-11T19:42:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f39517fa640 1 -- 172.21.15.83:0/1685265752 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3964073070 con 0x5625f0dfb270 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f3972861740 10 client.25544 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f3972861740 10 client.25544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f3972861740 20 client.25544 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f3972861740 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5625f0e199f0 0x5625f0e72d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f3972861740 4 client.25544 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f3972861740 1 -- 172.21.15.83:0/1685265752 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5625f0aef890 con 0x5625f0e199f0 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f396d231640 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5625f0e199f0 0x5625f0e72d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f39517fa640 10 client.25544 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f396d231640 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5625f0e199f0 0x5625f0e72d00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8f6021eba34f2d1 server_cookie=21ae0baecc4a51c9 in_seq=0 out_seq=0 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f39517fa640 1 -- 172.21.15.83:0/1685265752 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5625f0aef890 con 0x5625f0e199f0 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f39517fa640 10 client.25544 handle_command_reply: tid=0 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.934+0000 7f3972861740 1 client.25544 shutdown 2024-03-11T19:42:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f3972861740 1 -- 172.21.15.83:0/1685265752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 msgr2=0x5625f0e13190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f3972861740 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0e13190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f396403e000 tx=0x7f396402bc30 comp rx=0 tx=0).stop 2024-03-11T19:42:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f3972861740 1 -- 172.21.15.83:0/1685265752 shutdown_connections 2024-03-11T19:42:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f3972861740 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5625f0e199f0 0x5625f0e72d00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f3972861740 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625f0e00dd0 0x5625f0e146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f39517fa640 0 client.25544 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f3972861740 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625f0dfbc60 0x5625f0e136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f3972861740 1 --2- 172.21.15.83:0/1685265752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625f0dfb270 0x5625f0e13190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f3972861740 1 -- 172.21.15.83:0/1685265752 >> 172.21.15.83:0/1685265752 conn(0x5625f0df67e0 msgr2=0x5625f0dfd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f3972861740 1 -- 172.21.15.83:0/1685265752 shutdown_connections 2024-03-11T19:42:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f3972861740 1 -- 172.21.15.83:0/1685265752 wait complete. 2024-03-11T19:42:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.935+0000 7f3972861740 20 client.25544 trim_cache size 0 max 16384 2024-03-11T19:42:53.946 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:42:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.937+0000 7f396fc96640 1 -- 172.21.15.83:0/3627019570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f39540a1ab0 msgr2=0x7f39540a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.937+0000 7f396fc96640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f39540a1ab0 0x7f39540a3f70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3958006e10 tx=0x7f3958009930 comp rx=0 tx=0).stop 2024-03-11T19:42:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.937+0000 7f396fc96640 1 -- 172.21.15.83:0/3627019570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3968101c30 msgr2=0x7f396819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.937+0000 7f396fc96640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3968101c30 0x7f396819d7b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3960009fd0 tx=0x7f396000def0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.937+0000 7f396fc96640 1 -- 172.21.15.83:0/3627019570 shutdown_connections 2024-03-11T19:42:53.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.937+0000 7f396fc96640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f39540a1ab0 0x7f39540a3f70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.937+0000 7f396fc96640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3968101c30 0x7f396819d7b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.937+0000 7f396fc96640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3968101270 0x7f3968196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.937+0000 7f396fc96640 1 --2- 172.21.15.83:0/3627019570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3968100080 0x7f3968196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:53.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.937+0000 7f396fc96640 1 -- 172.21.15.83:0/3627019570 >> 172.21.15.83:0/3627019570 conn(0x7f39680fb610 msgr2=0x7f39681045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:53.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.937+0000 7f396fc96640 1 -- 172.21.15.83:0/3627019570 shutdown_connections 2024-03-11T19:42:53.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:53.938+0000 7f396fc96640 1 -- 172.21.15.83:0/3627019570 wait complete. 2024-03-11T19:42:54.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:53 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2522964570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:42:54.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:53 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2522964570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:42:54.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:53 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2522964570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:42:54.955 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:42:54.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:42:54.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:42:54.992 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:54 smithi083 ceph-mon[31864]: pgmap v460: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 23 MiB/s wr, 922 op/s 2024-03-11T19:42:55.040 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:54 smithi137 ceph-mon[36220]: pgmap v460: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 23 MiB/s wr, 922 op/s 2024-03-11T19:42:55.057 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:54 smithi204 ceph-mon[36739]: pgmap v460: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 23 MiB/s wr, 922 op/s 2024-03-11T19:42:55.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.127+0000 7f30bc4dd640 1 -- 172.21.15.83:0/1573213631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30b410bbc0 msgr2=0x7f30b410dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.127+0000 7f30bc4dd640 1 --2- 172.21.15.83:0/1573213631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30b410bbc0 0x7f30b410dfb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009900 tx=0x7f30a402bc60 comp rx=0 tx=0).stop 2024-03-11T19:42:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.129+0000 7f30bc4dd640 1 -- 172.21.15.83:0/1573213631 shutdown_connections 2024-03-11T19:42:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.129+0000 7f30bc4dd640 1 --2- 172.21.15.83:0/1573213631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b41081d0 0x7f30b4108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.129+0000 7f30bc4dd640 1 --2- 172.21.15.83:0/1573213631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30b410bbc0 0x7f30b410dfb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.129+0000 7f30bc4dd640 1 --2- 172.21.15.83:0/1573213631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30b4109290 0x7f30b410b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.129+0000 7f30bc4dd640 1 -- 172.21.15.83:0/1573213631 >> 172.21.15.83:0/1573213631 conn(0x7f30b406dfe0 msgr2=0x7f30b4070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.130+0000 7f30bc4dd640 1 -- 172.21.15.83:0/1573213631 shutdown_connections 2024-03-11T19:42:55.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.130+0000 7f30bc4dd640 1 -- 172.21.15.83:0/1573213631 wait complete. 2024-03-11T19:42:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.131+0000 7f30bc4dd640 1 Processor -- start 2024-03-11T19:42:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.131+0000 7f30bc4dd640 1 -- start start 2024-03-11T19:42:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.131+0000 7f30bc4dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b41081d0 0x7f30b41a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.131+0000 7f30bc4dd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30b4109290 0x7f30b41a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.131+0000 7f30bc4dd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30b410bbc0 0x7f30b41a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.131+0000 7f30bc4dd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b41a9730 con 0x7f30b41081d0 2024-03-11T19:42:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.131+0000 7f30bc4dd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b41a98a0 con 0x7f30b4109290 2024-03-11T19:42:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.131+0000 7f30bc4dd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b41a9a60 con 0x7f30b410bbc0 2024-03-11T19:42:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.132+0000 7f30b9a78640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30b4109290 0x7f30b41a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.132+0000 7f30b9a78640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30b4109290 0x7f30b41a21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37602/0 (socket says 172.21.15.83:37602) 2024-03-11T19:42:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.132+0000 7f30b9a78640 1 -- 172.21.15.83:0/47593546 learned_addr learned my addr 172.21.15.83:0/47593546 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.132+0000 7f30ba279640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b41081d0 0x7f30b41a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.132+0000 7f30baa7a640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30b410bbc0 0x7f30b41a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.132+0000 7f30b9a78640 1 -- 172.21.15.83:0/47593546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30b410bbc0 msgr2=0x7f30b41a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.132+0000 7f30b9a78640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30b410bbc0 0x7f30b41a91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.132+0000 7f30b9a78640 1 -- 172.21.15.83:0/47593546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b41081d0 msgr2=0x7f30b41a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.132+0000 7f30b9a78640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b41081d0 0x7f30b41a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.132+0000 7f30b9a78640 1 -- 172.21.15.83:0/47593546 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a4009560 con 0x7f30b4109290 2024-03-11T19:42:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.132+0000 7f30b9a78640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30b4109290 0x7f30b41a21e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f30a4000c00 tx=0x7f30a4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.133+0000 7f30ab7fe640 1 -- 172.21.15.83:0/47593546 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4004df0 con 0x7f30b4109290 2024-03-11T19:42:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.133+0000 7f30bc4dd640 1 -- 172.21.15.83:0/47593546 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b41a9ce0 con 0x7f30b4109290 2024-03-11T19:42:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.133+0000 7f30bc4dd640 1 -- 172.21.15.83:0/47593546 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b41aa190 con 0x7f30b4109290 2024-03-11T19:42:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.133+0000 7f30ab7fe640 1 -- 172.21.15.83:0/47593546 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30a4005430 con 0x7f30b4109290 2024-03-11T19:42:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.133+0000 7f30ab7fe640 1 -- 172.21.15.83:0/47593546 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4003e50 con 0x7f30b4109290 2024-03-11T19:42:55.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30bf182740 1 Processor -- start 2024-03-11T19:42:55.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30bf182740 1 -- start start 2024-03-11T19:42:55.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30bf182740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6e908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30bf182740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6e91b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30bf182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e92050 0x55ffd6e988e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30bf182740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6dfff90 con 0x55ffd6e92050 2024-03-11T19:42:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30bf182740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6dfd4c0 con 0x55ffd6e904c0 2024-03-11T19:42:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30bf182740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6df8ca0 con 0x55ffd6e91690 2024-03-11T19:42:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30ab7fe640 1 -- 172.21.15.83:0/47593546 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f30a40035f0 con 0x7f30b4109290 2024-03-11T19:42:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30baa7a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6e908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30baa7a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6e908e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37606/0 (socket says 172.21.15.83:37606) 2024-03-11T19:42:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.136+0000 7f30baa7a640 1 -- 172.21.15.83:0/2387571796 learned_addr learned my addr 172.21.15.83:0/2387571796 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.137+0000 7f30b9a78640 1 --2- 172.21.15.83:0/2387571796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e92050 0x55ffd6e988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.137+0000 7f30ba279640 1 --2- 172.21.15.83:0/2387571796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6e91b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.137+0000 7f30ab7fe640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30a00a1b80 0x7f30a00a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30baa7a640 1 -- 172.21.15.83:0/2387571796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 msgr2=0x55ffd6e91b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30baa7a640 1 --2- 172.21.15.83:0/2387571796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6e91b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30baa7a640 1 -- 172.21.15.83:0/2387571796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e92050 msgr2=0x55ffd6e988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30baa7a640 1 --2- 172.21.15.83:0/2387571796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e92050 0x55ffd6e988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30baa7a640 1 -- 172.21.15.83:0/2387571796 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffd6dbb0b0 con 0x55ffd6e904c0 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30b9a78640 1 --2- 172.21.15.83:0/2387571796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e92050 0x55ffd6e988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30ab7fe640 1 -- 172.21.15.83:0/47593546 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f30a40efda0 con 0x7f30b4109290 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30baa7a640 1 --2- 172.21.15.83:0/2387571796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6e908e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009eb0 tx=0x7f30ac010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27600de808e268c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30baa7a640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30a00a1b80 0x7f30a00a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30a8bfe640 1 -- 172.21.15.83:0/2387571796 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac01f040 con 0x55ffd6e904c0 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30a8bfe640 1 -- 172.21.15.83:0/2387571796 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30ac01bc20 con 0x55ffd6e904c0 2024-03-11T19:42:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.138+0000 7f30a8bfe640 1 -- 172.21.15.83:0/2387571796 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0113e0 con 0x55ffd6e904c0 2024-03-11T19:42:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.139+0000 7f30bf182740 1 -- 172.21.15.83:0/2387571796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 msgr2=0x55ffd6e908e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.139+0000 7f30bf182740 1 --2- 172.21.15.83:0/2387571796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6e908e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009eb0 tx=0x7f30ac010740 comp rx=0 tx=0).stop 2024-03-11T19:42:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.140+0000 7f30baa7a640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30a00a1b80 0x7f30a00a4040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f30ac01a000 tx=0x7f30ac011540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.140+0000 7f30bf182740 1 -- 172.21.15.83:0/2387571796 shutdown_connections 2024-03-11T19:42:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.140+0000 7f30bf182740 1 --2- 172.21.15.83:0/2387571796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e92050 0x55ffd6e988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.140+0000 7f30bf182740 1 --2- 172.21.15.83:0/2387571796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6e91b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.140+0000 7f30bf182740 1 --2- 172.21.15.83:0/2387571796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6e908e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.140+0000 7f30bf182740 1 -- 172.21.15.83:0/2387571796 >> 172.21.15.83:0/2387571796 conn(0x55ffd6e8b6a0 msgr2=0x55ffd6e8baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.140+0000 7f30bf182740 1 -- 172.21.15.83:0/2387571796 shutdown_connections 2024-03-11T19:42:55.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.143+0000 7f30bf182740 1 -- 172.21.15.83:0/2387571796 wait complete. 2024-03-11T19:42:55.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.143+0000 7f30bf182740 1 -- start start 2024-03-11T19:42:55.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30bf182740 1 Processor -- start 2024-03-11T19:42:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30bf182740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6ea2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30bf182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e91690 0x55ffd6ea3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30bf182740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e92050 0x55ffd6eaa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30bf182740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6d9df40 con 0x55ffd6e91690 2024-03-11T19:42:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30bf182740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6acc4b0 con 0x55ffd6e904c0 2024-03-11T19:42:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30bf182740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6d7e670 con 0x55ffd6e92050 2024-03-11T19:42:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30bf182740 10 client.0 _ll_register_callbacks cb 0x55ffd6debf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30ba279640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6ea2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30ba279640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6ea2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37614/0 (socket says 172.21.15.83:37614) 2024-03-11T19:42:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30ba279640 1 -- 172.21.15.83:0/1292526678 learned_addr learned my addr 172.21.15.83:0/1292526678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30b9a78640 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e91690 0x55ffd6ea3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.144+0000 7f30baa7a640 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e92050 0x55ffd6eaa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.145+0000 7f30ba279640 1 -- 172.21.15.83:0/1292526678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e92050 msgr2=0x55ffd6eaa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.145+0000 7f30ba279640 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e92050 0x55ffd6eaa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.145+0000 7f30ba279640 1 -- 172.21.15.83:0/1292526678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e91690 msgr2=0x55ffd6ea3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.145+0000 7f30ba279640 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e91690 0x55ffd6ea3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.145+0000 7f30ba279640 1 -- 172.21.15.83:0/1292526678 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac009ae0 con 0x55ffd6e904c0 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.145+0000 7f30b9a78640 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e91690 0x55ffd6ea3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.145+0000 7f30ba279640 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6ea2ee0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f30b00107c0 tx=0x7f30b0010bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c12152bfe105339 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.145+0000 7f309dffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.145+0000 7f309dffb640 1 -- 172.21.15.83:0/1292526678 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b000c910 con 0x55ffd6e904c0 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.145+0000 7f30bf182740 1 -- 172.21.15.83:0/1292526678 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffd6dd2f40 con 0x55ffd6e904c0 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.146+0000 7f309dffb640 1 -- 172.21.15.83:0/1292526678 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f30b000ca70 con 0x55ffd6e904c0 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.146+0000 7f309dffb640 1 -- 172.21.15.83:0/1292526678 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30b000c910 con 0x55ffd6e904c0 2024-03-11T19:42:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.146+0000 7f30bf182740 10 client.25553 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.146+0000 7f30bf182740 1 -- 172.21.15.83:0/1292526678 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ffd6df7320 con 0x55ffd6e904c0 2024-03-11T19:42:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.146+0000 7f309dffb640 1 -- 172.21.15.83:0/1292526678 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ffd6dd2f40 con 0x55ffd6e904c0 2024-03-11T19:42:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.147+0000 7f309dffb640 1 -- 172.21.15.83:0/1292526678 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f30b0011440 con 0x55ffd6e904c0 2024-03-11T19:42:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.147+0000 7f30bf182740 10 client.25553 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.147+0000 7f30bf182740 10 client.25553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.147+0000 7f30bf182740 20 client.25553 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.147+0000 7f30bf182740 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ffd6eaf0a0 0x55ffd6eb14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.147+0000 7f30bf182740 4 client.25553 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.147+0000 7f30bf182740 1 -- 172.21.15.83:0/1292526678 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffd6def310 con 0x55ffd6eaf0a0 2024-03-11T19:42:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.148+0000 7f30b9a78640 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ffd6eaf0a0 0x55ffd6eb14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.148+0000 7f309dffb640 10 client.25553 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.148+0000 7f30b9a78640 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ffd6eaf0a0 0x55ffd6eb14c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745c62b169371b69 server_cookie=80492ed2881f8f7b in_seq=0 out_seq=0 2024-03-11T19:42:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.149+0000 7f309dffb640 1 -- 172.21.15.83:0/1292526678 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffd6def310 con 0x55ffd6eaf0a0 2024-03-11T19:42:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.149+0000 7f309dffb640 10 client.25553 handle_command_reply: tid=0 2024-03-11T19:42:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.149+0000 7f30bf182740 1 client.25553 shutdown 2024-03-11T19:42:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.150+0000 7f30bf182740 1 -- 172.21.15.83:0/1292526678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 msgr2=0x55ffd6ea2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.150+0000 7f30bf182740 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6ea2ee0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f30b00107c0 tx=0x7f30b0010bc0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.152+0000 7f30bf182740 1 -- 172.21.15.83:0/1292526678 shutdown_connections 2024-03-11T19:42:55.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.152+0000 7f30bf182740 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ffd6eaf0a0 0x55ffd6eb14c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.152+0000 7f30bf182740 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e92050 0x55ffd6eaa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.152+0000 7f30bf182740 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e91690 0x55ffd6ea3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.152+0000 7f30bf182740 1 --2- 172.21.15.83:0/1292526678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e904c0 0x55ffd6ea2ee0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.152+0000 7f309dffb640 0 client.25553 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:55.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.152+0000 7f30bf182740 1 -- 172.21.15.83:0/1292526678 >> 172.21.15.83:0/1292526678 conn(0x55ffd6e8b6a0 msgr2=0x55ffd6e8a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.152+0000 7f30bf182740 1 -- 172.21.15.83:0/1292526678 shutdown_connections 2024-03-11T19:42:55.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.152+0000 7f30bf182740 1 -- 172.21.15.83:0/1292526678 wait complete. 2024-03-11T19:42:55.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.152+0000 7f30bf182740 20 client.25553 trim_cache size 0 max 16384 2024-03-11T19:42:55.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30bf182740 1 Processor -- start 2024-03-11T19:42:55.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30bf182740 1 -- start start 2024-03-11T19:42:55.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30bf182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 0x55ffd6e95180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30bf182740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6eaa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30bf182740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e92050 0x55ffd6e956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30bf182740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6e91420 con 0x55ffd6e904c0 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30bf182740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6e8ff00 con 0x55ffd6e92050 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30bf182740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6dd2f40 con 0x55ffd6e91690 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30ba279640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 0x55ffd6e95180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30b9a78640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6eaa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30ba279640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 0x55ffd6e95180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59168/0 (socket says 172.21.15.83:59168) 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30ba279640 1 -- 172.21.15.83:0/1474523941 learned_addr learned my addr 172.21.15.83:0/1474523941 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30baa7a640 1 --2- 172.21.15.83:0/1474523941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e92050 0x55ffd6e956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30b9a78640 1 -- 172.21.15.83:0/1474523941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e92050 msgr2=0x55ffd6e956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30b9a78640 1 --2- 172.21.15.83:0/1474523941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e92050 0x55ffd6e956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30b9a78640 1 -- 172.21.15.83:0/1474523941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 msgr2=0x55ffd6e95180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30b9a78640 1 --2- 172.21.15.83:0/1474523941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 0x55ffd6e95180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.176+0000 7f30b9a78640 1 -- 172.21.15.83:0/1474523941 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffd6d93c00 con 0x55ffd6e91690 2024-03-11T19:42:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.177+0000 7f30ba279640 1 --2- 172.21.15.83:0/1474523941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 0x55ffd6e95180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.177+0000 7f30b9a78640 1 --2- 172.21.15.83:0/1474523941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6eaa420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30a402c270 tx=0x7f30a40f3730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6decd56bf7adae38 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.177+0000 7f309ffff640 1 -- 172.21.15.83:0/1474523941 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40bc680 con 0x55ffd6e91690 2024-03-11T19:42:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.177+0000 7f309ffff640 1 -- 172.21.15.83:0/1474523941 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ffd6def310 con 0x55ffd6e91690 2024-03-11T19:42:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.177+0000 7f309ffff640 1 -- 172.21.15.83:0/1474523941 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40c54c0 con 0x55ffd6e91690 2024-03-11T19:42:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.178+0000 7f30bf182740 1 -- 172.21.15.83:0/1474523941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 msgr2=0x55ffd6eaa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.178+0000 7f30bf182740 1 --2- 172.21.15.83:0/1474523941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6eaa420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30a402c270 tx=0x7f30a40f3730 comp rx=0 tx=0).stop 2024-03-11T19:42:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.178+0000 7f30bf182740 1 -- 172.21.15.83:0/1474523941 shutdown_connections 2024-03-11T19:42:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.178+0000 7f30bf182740 1 --2- 172.21.15.83:0/1474523941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e92050 0x55ffd6e956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.178+0000 7f30bf182740 1 --2- 172.21.15.83:0/1474523941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6eaa420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.178+0000 7f30bf182740 1 --2- 172.21.15.83:0/1474523941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 0x55ffd6e95180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.178+0000 7f30bf182740 1 -- 172.21.15.83:0/1474523941 >> 172.21.15.83:0/1474523941 conn(0x55ffd6e8b6a0 msgr2=0x55ffd6eb5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.178+0000 7f30bf182740 1 -- 172.21.15.83:0/1474523941 shutdown_connections 2024-03-11T19:42:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.178+0000 7f30bf182740 1 -- 172.21.15.83:0/1474523941 wait complete. 2024-03-11T19:42:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.178+0000 7f30bf182740 1 -- start start 2024-03-11T19:42:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.178+0000 7f30bf182740 1 Processor -- start 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.179+0000 7f30bf182740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 0x55ffd6eaa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.179+0000 7f30bf182740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6ea8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30ba279640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 0x55ffd6eaa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30ba279640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 0x55ffd6eaa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59178/0 (socket says 172.21.15.83:59178) 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30ba279640 1 -- 172.21.15.83:0/2554464726 learned_addr learned my addr 172.21.15.83:0/2554464726 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30bf182740 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e92050 0x55ffd6ea9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30b9a78640 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6ea8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30bf182740 1 -- 172.21.15.83:0/2554464726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6ea9800 con 0x55ffd6e904c0 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30bf182740 1 -- 172.21.15.83:0/2554464726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6ea9970 con 0x55ffd6e92050 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30bf182740 1 -- 172.21.15.83:0/2554464726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd6f05600 con 0x55ffd6e91690 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30baa7a640 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e92050 0x55ffd6ea9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30bf182740 10 client.0 _ll_register_callbacks cb 0x7f30ac009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30baa7a640 1 -- 172.21.15.83:0/2554464726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 msgr2=0x55ffd6ea8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30baa7a640 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6ea8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30baa7a640 1 -- 172.21.15.83:0/2554464726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 msgr2=0x55ffd6eaa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30baa7a640 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 0x55ffd6eaa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30baa7a640 1 -- 172.21.15.83:0/2554464726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffd6df7320 con 0x55ffd6e92050 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.180+0000 7f30b9a78640 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6ea8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.181+0000 7f30baa7a640 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e92050 0x55ffd6ea9100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009e80 tx=0x7f30ac01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59bc337d3ec1bf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.181+0000 7f309dffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.181+0000 7f309dffb640 1 -- 172.21.15.83:0/2554464726 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0113e0 con 0x55ffd6e92050 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.181+0000 7f309dffb640 1 -- 172.21.15.83:0/2554464726 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f30ac011790 con 0x55ffd6e92050 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.181+0000 7f30bf182740 1 -- 172.21.15.83:0/2554464726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffd6e1e900 con 0x55ffd6e92050 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.181+0000 7f309dffb640 1 -- 172.21.15.83:0/2554464726 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30ac01ab60 con 0x55ffd6e92050 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.181+0000 7f30bf182740 10 client.25559 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.181+0000 7f30bf182740 1 -- 172.21.15.83:0/2554464726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f30b41a9ce0 con 0x55ffd6e92050 2024-03-11T19:42:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.181+0000 7f309dffb640 1 -- 172.21.15.83:0/2554464726 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ffd6e1e900 con 0x55ffd6e92050 2024-03-11T19:42:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.182+0000 7f309dffb640 1 -- 172.21.15.83:0/2554464726 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f30ac026450 con 0x55ffd6e92050 2024-03-11T19:42:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.182+0000 7f30bf182740 10 client.25559 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.182+0000 7f30bf182740 10 client.25559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.182+0000 7f30bf182740 20 client.25559 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.182+0000 7f30bf182740 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ffd6f070c0 0x55ffd6f094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.182+0000 7f30bf182740 4 client.25559 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:42:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.182+0000 7f30bf182740 1 -- 172.21.15.83:0/2554464726 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ffd6af8860 con 0x55ffd6f070c0 2024-03-11T19:42:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.182+0000 7f30ba279640 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ffd6f070c0 0x55ffd6f094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.184+0000 7f30ba279640 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ffd6f070c0 0x55ffd6f094e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3045ae9a1c7dfb54 server_cookie=c569745d870ad683 in_seq=0 out_seq=0 2024-03-11T19:42:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.184+0000 7f309dffb640 10 client.25559 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f309dffb640 1 -- 172.21.15.83:0/2554464726 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ffd6af8860 con 0x55ffd6f070c0 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f309dffb640 10 client.25559 handle_command_reply: tid=0 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 1 client.25559 shutdown 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 1 -- 172.21.15.83:0/2554464726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e92050 msgr2=0x55ffd6ea9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e92050 0x55ffd6ea9100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009e80 tx=0x7f30ac01b000 comp rx=0 tx=0).stop 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 1 -- 172.21.15.83:0/2554464726 shutdown_connections 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ffd6f070c0 0x55ffd6f094e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffd6e92050 0x55ffd6ea9100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffd6e91690 0x55ffd6ea8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 1 --2- 172.21.15.83:0/2554464726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffd6e904c0 0x55ffd6eaa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f309dffb640 0 client.25559 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 1 -- 172.21.15.83:0/2554464726 >> 172.21.15.83:0/2554464726 conn(0x55ffd6e8b6a0 msgr2=0x55ffd6e95180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 1 -- 172.21.15.83:0/2554464726 shutdown_connections 2024-03-11T19:42:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 1 -- 172.21.15.83:0/2554464726 wait complete. 2024-03-11T19:42:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.185+0000 7f30bf182740 20 client.25559 trim_cache size 0 max 16384 2024-03-11T19:42:55.193 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:42:55.193 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:42:55.193 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "e4062a39-0658-4901-94ae-b08270c4abe9", 2024-03-11T19:42:55.193 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:42:55.193 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:42:55.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 -- 172.21.15.83:0/47593546 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30a00a1b80 msgr2=0x7f30a00a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30a00a1b80 0x7f30a00a4040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f30ac01a000 tx=0x7f30ac011540 comp rx=0 tx=0).stop 2024-03-11T19:42:55.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 -- 172.21.15.83:0/47593546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30b4109290 msgr2=0x7f30b41a21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30b4109290 0x7f30b41a21e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f30a4000c00 tx=0x7f30a4004270 comp rx=0 tx=0).stop 2024-03-11T19:42:55.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 -- 172.21.15.83:0/47593546 shutdown_connections 2024-03-11T19:42:55.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f30a00a1b80 0x7f30a00a4040 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30b410bbc0 0x7f30b41a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30b4109290 0x7f30b41a21e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 --2- 172.21.15.83:0/47593546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30b41081d0 0x7f30b41a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 -- 172.21.15.83:0/47593546 >> 172.21.15.83:0/47593546 conn(0x7f30b406dfe0 msgr2=0x7f30b4109670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 -- 172.21.15.83:0/47593546 shutdown_connections 2024-03-11T19:42:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.186+0000 7f30bc4dd640 1 -- 172.21.15.83:0/47593546 wait complete. 2024-03-11T19:42:55.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:42:55.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:42:55.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.402+0000 7f6607264640 1 -- 172.21.15.83:0/320994881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6600100080 msgr2=0x7f66001004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.402+0000 7f6607264640 1 --2- 172.21.15.83:0/320994881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6600100080 0x7f66001004a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009900 tx=0x7f65f402bc30 comp rx=0 tx=0).stop 2024-03-11T19:42:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.402+0000 7f6607264640 1 -- 172.21.15.83:0/320994881 shutdown_connections 2024-03-11T19:42:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.402+0000 7f6607264640 1 --2- 172.21.15.83:0/320994881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6600101c30 0x7f66001084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.402+0000 7f6607264640 1 --2- 172.21.15.83:0/320994881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6600101270 0x7f66001016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.402+0000 7f6607264640 1 --2- 172.21.15.83:0/320994881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6600100080 0x7f66001004a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.402+0000 7f6607264640 1 -- 172.21.15.83:0/320994881 >> 172.21.15.83:0/320994881 conn(0x7f66000fb5f0 msgr2=0x7f66000fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.402+0000 7f6607264640 1 -- 172.21.15.83:0/320994881 shutdown_connections 2024-03-11T19:42:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.402+0000 7f6607264640 1 -- 172.21.15.83:0/320994881 wait complete. 2024-03-11T19:42:55.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.403+0000 7f6607264640 1 Processor -- start 2024-03-11T19:42:55.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6607264640 1 -- start start 2024-03-11T19:42:55.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6607264640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6600100080 0x7f6600199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6607264640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6600101270 0x7f6600199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6607264640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6600101c30 0x7f66001a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6607264640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001a0f20 con 0x7f6600101270 2024-03-11T19:42:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6607264640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001a1090 con 0x7f6600100080 2024-03-11T19:42:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6607264640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001a1250 con 0x7f6600101c30 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6605000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6600100080 0x7f6600199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6605000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6600100080 0x7f6600199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37658/0 (socket says 172.21.15.83:37658) 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6605000640 1 -- 172.21.15.83:0/4181858084 learned_addr learned my addr 172.21.15.83:0/4181858084 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6605801640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6600101c30 0x7f66001a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f65fffff640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6600101270 0x7f6600199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6605000640 1 -- 172.21.15.83:0/4181858084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6600101c30 msgr2=0x7f66001a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.404+0000 7f6605000640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6600101c30 0x7f66001a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.405+0000 7f6605000640 1 -- 172.21.15.83:0/4181858084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6600101270 msgr2=0x7f6600199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.405+0000 7f6605000640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6600101270 0x7f6600199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.405+0000 7f6605000640 1 -- 172.21.15.83:0/4181858084 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f4009560 con 0x7f6600100080 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.405+0000 7f6605000640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6600100080 0x7f6600199430 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f65f4000c00 tx=0x7f65f40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.405+0000 7f65fdffb640 1 -- 172.21.15.83:0/4181858084 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f40044c0 con 0x7f6600100080 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.405+0000 7f65fdffb640 1 -- 172.21.15.83:0/4181858084 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65f40052d0 con 0x7f6600100080 2024-03-11T19:42:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.405+0000 7f6607264640 1 -- 172.21.15.83:0/4181858084 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66001a14d0 con 0x7f6600100080 2024-03-11T19:42:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.405+0000 7f65fdffb640 1 -- 172.21.15.83:0/4181858084 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f403e450 con 0x7f6600100080 2024-03-11T19:42:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.405+0000 7f6607264640 1 -- 172.21.15.83:0/4181858084 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66001a19e0 con 0x7f6600100080 2024-03-11T19:42:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.409+0000 7f65fdffb640 1 -- 172.21.15.83:0/4181858084 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f65f403e5b0 con 0x7f6600100080 2024-03-11T19:42:55.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.409+0000 7f6609e3c740 1 Processor -- start 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.409+0000 7f6609e3c740 1 -- start start 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.409+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef15f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.409+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef1afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.409+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef1dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.409+0000 7f6609e3c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56ee7d280 con 0x55e56ef164d0 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.409+0000 7f6609e3c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56ee7a7b0 con 0x55e56ef1b640 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.409+0000 7f6609e3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56ee75f90 con 0x55e56ef15ae0 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f6605801640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef15f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f6605801640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef15f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:55022/0 (socket says 172.21.15.83:55022) 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f65fffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef1dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f6605801640 1 -- 172.21.15.83:0/1801924905 learned_addr learned my addr 172.21.15.83:0/1801924905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f65fdffb640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65f00a1a30 0x7f65f00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f6605000640 1 --2- 172.21.15.83:0/1801924905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef1afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f6605801640 1 -- 172.21.15.83:0/1801924905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 msgr2=0x55e56ef1dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f6605801640 1 --2- 172.21.15.83:0/1801924905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef1dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f6605801640 1 -- 172.21.15.83:0/1801924905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 msgr2=0x55e56ef1afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f65fdffb640 1 -- 172.21.15.83:0/4181858084 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f65f40efb50 con 0x7f6600100080 2024-03-11T19:42:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f6605801640 1 --2- 172.21.15.83:0/1801924905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef1afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f6605801640 1 -- 172.21.15.83:0/1801924905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e56ee34570 con 0x55e56ef15ae0 2024-03-11T19:42:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f65fffff640 1 --2- 172.21.15.83:0/1801924905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef1dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.410+0000 7f6605801640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65f00a1a30 0x7f65f00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.411+0000 7f6605801640 1 --2- 172.21.15.83:0/1801924905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef15f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f65f8010510 tx=0x7f65f8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fab507d31b79a0b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.411+0000 7f65eaffd640 1 -- 172.21.15.83:0/1801924905 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8022070 con 0x55e56ef15ae0 2024-03-11T19:42:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.411+0000 7f65eaffd640 1 -- 172.21.15.83:0/1801924905 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65f801d8e0 con 0x55e56ef15ae0 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.411+0000 7f65eaffd640 1 -- 172.21.15.83:0/1801924905 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f801c630 con 0x55e56ef15ae0 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.412+0000 7f6605801640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65f00a1a30 0x7f65f00a3ef0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f65f8021000 tx=0x7f65f801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.412+0000 7f6609e3c740 1 -- 172.21.15.83:0/1801924905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 msgr2=0x55e56ef15f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.412+0000 7f6609e3c740 1 --2- 172.21.15.83:0/1801924905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef15f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f65f8010510 tx=0x7f65f8009f90 comp rx=0 tx=0).stop 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.412+0000 7f6609e3c740 1 -- 172.21.15.83:0/1801924905 shutdown_connections 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.412+0000 7f6609e3c740 1 --2- 172.21.15.83:0/1801924905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef1dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.413+0000 7f6609e3c740 1 --2- 172.21.15.83:0/1801924905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef1afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.413+0000 7f6609e3c740 1 --2- 172.21.15.83:0/1801924905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef15f00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.413+0000 7f6609e3c740 1 -- 172.21.15.83:0/1801924905 >> 172.21.15.83:0/1801924905 conn(0x55e56ef11000 msgr2=0x55e56ef11400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.413+0000 7f6609e3c740 1 -- 172.21.15.83:0/1801924905 shutdown_connections 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.413+0000 7f6609e3c740 1 -- 172.21.15.83:0/1801924905 wait complete. 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.413+0000 7f6609e3c740 1 -- start start 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.413+0000 7f6609e3c740 1 Processor -- start 2024-03-11T19:42:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef1fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef20080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f6609e3c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56eb49430 con 0x55e56ef164d0 2024-03-11T19:42:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f6609e3c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56eb05e00 con 0x55e56ef1b640 2024-03-11T19:42:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f6609e3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56edfafd0 con 0x55e56ef15ae0 2024-03-11T19:42:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f6609e3c740 10 client.0 _ll_register_callbacks cb 0x55e56ee50fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f65fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef20080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f65fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef20080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59226/0 (socket says 172.21.15.83:59226) 2024-03-11T19:42:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f65fffff640 1 -- 172.21.15.83:0/2581907022 learned_addr learned my addr 172.21.15.83:0/2581907022 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f6605000640 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef1fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f6605801640 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f65fffff640 1 -- 172.21.15.83:0/2581907022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 msgr2=0x55e56ef1fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f65fffff640 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef1fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f65fffff640 1 -- 172.21.15.83:0/2581907022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 msgr2=0x55e56ef270c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f65fffff640 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef270c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f65fffff640 1 -- 172.21.15.83:0/2581907022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f8009ca0 con 0x55e56ef164d0 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.414+0000 7f6605801640 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.415+0000 7f6605000640 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef1fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.415+0000 7f65dffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.415+0000 7f65fffff640 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef20080 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f65ec012700 tx=0x7f65ec012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1285fd71eba202e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.416+0000 7f65dffff640 1 -- 172.21.15.83:0/2581907022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec017070 con 0x55e56ef164d0 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.416+0000 7f65dffff640 1 -- 172.21.15.83:0/2581907022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f65ec004030 con 0x55e56ef164d0 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.416+0000 7f6609e3c740 1 -- 172.21.15.83:0/2581907022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e56eb757e0 con 0x55e56ef164d0 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.416+0000 7f65dffff640 1 -- 172.21.15.83:0/2581907022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65ec017070 con 0x55e56ef164d0 2024-03-11T19:42:55.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.416+0000 7f6609e3c740 10 client.16089 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.416+0000 7f6609e3c740 1 -- 172.21.15.83:0/2581907022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e56ee74610 con 0x55e56ef164d0 2024-03-11T19:42:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.416+0000 7f65dffff640 1 -- 172.21.15.83:0/2581907022 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e56eb757e0 con 0x55e56ef164d0 2024-03-11T19:42:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.417+0000 7f65dffff640 1 -- 172.21.15.83:0/2581907022 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f65ec00f5e0 con 0x55e56ef164d0 2024-03-11T19:42:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.417+0000 7f6609e3c740 10 client.16089 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.417+0000 7f6609e3c740 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.417+0000 7f6609e3c740 20 client.16089 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.417+0000 7f6609e3c740 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e56ef2bda0 0x55e56ef2e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.417+0000 7f6609e3c740 4 client.16089 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:42:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.417+0000 7f6609e3c740 1 -- 172.21.15.83:0/2581907022 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e56ebb2720 con 0x55e56ef2bda0 2024-03-11T19:42:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.417+0000 7f6605000640 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e56ef2bda0 0x55e56ef2e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f65dffff640 10 client.16089 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:55.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f6605000640 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e56ef2bda0 0x55e56ef2e1c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50a420f4cc503f23 server_cookie=2829b420f33ba109 in_seq=0 out_seq=0 2024-03-11T19:42:55.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f65dffff640 1 -- 172.21.15.83:0/2581907022 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e56ebb2720 con 0x55e56ef2bda0 2024-03-11T19:42:55.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f65dffff640 10 client.16089 handle_command_reply: tid=0 2024-03-11T19:42:55.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f6609e3c740 1 client.16089 shutdown 2024-03-11T19:42:55.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f6609e3c740 1 -- 172.21.15.83:0/2581907022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 msgr2=0x55e56ef20080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f6609e3c740 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef20080 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f65ec012700 tx=0x7f65ec012b00 comp rx=0 tx=0).stop 2024-03-11T19:42:55.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f6609e3c740 1 -- 172.21.15.83:0/2581907022 shutdown_connections 2024-03-11T19:42:55.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f6609e3c740 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e56ef2bda0 0x55e56ef2e1c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f6609e3c740 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f65dffff640 0 client.16089 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:55.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f6609e3c740 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef20080 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f6609e3c740 1 --2- 172.21.15.83:0/2581907022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef1fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.418+0000 7f6609e3c740 1 -- 172.21.15.83:0/2581907022 >> 172.21.15.83:0/2581907022 conn(0x55e56ef11000 msgr2=0x55e56ef07a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.419+0000 7f6609e3c740 1 -- 172.21.15.83:0/2581907022 shutdown_connections 2024-03-11T19:42:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.419+0000 7f6609e3c740 1 -- 172.21.15.83:0/2581907022 wait complete. 2024-03-11T19:42:55.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.420+0000 7f6609e3c740 20 client.16089 trim_cache size 0 max 16384 2024-03-11T19:42:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.441+0000 7f6609e3c740 1 Processor -- start 2024-03-11T19:42:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.441+0000 7f6609e3c740 1 -- start start 2024-03-11T19:42:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef32b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef26fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef1fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f65fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef26fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f6609e3c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56ef2a370 con 0x55e56ef164d0 2024-03-11T19:42:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f6609e3c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56ef040d0 con 0x55e56ef1b640 2024-03-11T19:42:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f6609e3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56ef27520 con 0x55e56ef15ae0 2024-03-11T19:42:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f65fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef26fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59234/0 (socket says 172.21.15.83:59234) 2024-03-11T19:42:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f65fffff640 1 -- 172.21.15.83:0/4180488138 learned_addr learned my addr 172.21.15.83:0/4180488138 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:55.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f6605000640 1 --2- 172.21.15.83:0/4180488138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef32b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f6605801640 1 --2- 172.21.15.83:0/4180488138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef1fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f65fffff640 1 -- 172.21.15.83:0/4180488138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 msgr2=0x55e56ef32b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f65fffff640 1 --2- 172.21.15.83:0/4180488138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef32b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f65fffff640 1 -- 172.21.15.83:0/4180488138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 msgr2=0x55e56ef1fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f65fffff640 1 --2- 172.21.15.83:0/4180488138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef1fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f65fffff640 1 -- 172.21.15.83:0/4180488138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e56ee10ac0 con 0x55e56ef164d0 2024-03-11T19:42:55.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.442+0000 7f6605000640 1 --2- 172.21.15.83:0/4180488138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef32b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:42:55.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.443+0000 7f65fffff640 1 --2- 172.21.15.83:0/4180488138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef26fe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f65ec01bbb0 tx=0x7f65ec0123e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620443c483eb547 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.443+0000 7f65ea7fc640 1 -- 172.21.15.83:0/4180488138 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec017070 con 0x55e56ef164d0 2024-03-11T19:42:55.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.443+0000 7f65ea7fc640 1 -- 172.21.15.83:0/4180488138 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65ec009850 con 0x55e56ef164d0 2024-03-11T19:42:55.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.443+0000 7f65ea7fc640 1 -- 172.21.15.83:0/4180488138 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec01a520 con 0x55e56ef164d0 2024-03-11T19:42:55.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.444+0000 7f6609e3c740 1 -- 172.21.15.83:0/4180488138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 msgr2=0x55e56ef26fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.444+0000 7f6609e3c740 1 --2- 172.21.15.83:0/4180488138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef26fe0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f65ec01bbb0 tx=0x7f65ec0123e0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.444+0000 7f6609e3c740 1 -- 172.21.15.83:0/4180488138 shutdown_connections 2024-03-11T19:42:55.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.444+0000 7f6609e3c740 1 --2- 172.21.15.83:0/4180488138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef1b640 0x55e56ef1fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.444+0000 7f6609e3c740 1 --2- 172.21.15.83:0/4180488138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef164d0 0x55e56ef26fe0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.444+0000 7f6609e3c740 1 --2- 172.21.15.83:0/4180488138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef15ae0 0x55e56ef32b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.444+0000 7f6609e3c740 1 -- 172.21.15.83:0/4180488138 >> 172.21.15.83:0/4180488138 conn(0x55e56ef11000 msgr2=0x55e56ef113e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.444+0000 7f6609e3c740 1 -- 172.21.15.83:0/4180488138 shutdown_connections 2024-03-11T19:42:55.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.444+0000 7f6609e3c740 1 -- 172.21.15.83:0/4180488138 wait complete. 2024-03-11T19:42:55.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.444+0000 7f6609e3c740 1 -- start start 2024-03-11T19:42:55.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.444+0000 7f6609e3c740 1 Processor -- start 2024-03-11T19:42:55.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.445+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef15ae0 0x55e56ef268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.445+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef164d0 0x55e56ef26e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.445+0000 7f6609e3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef1b640 0x55e56ef24ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.445+0000 7f6609e3c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56ef25530 con 0x55e56ef15ae0 2024-03-11T19:42:55.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.445+0000 7f6609e3c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56ef256a0 con 0x55e56ef164d0 2024-03-11T19:42:55.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.445+0000 7f6609e3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e56ef25860 con 0x55e56ef1b640 2024-03-11T19:42:55.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.446+0000 7f6609e3c740 10 client.0 _ll_register_callbacks cb 0x7f65f8009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:42:55.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.446+0000 7f65fffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef164d0 0x55e56ef26e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.446+0000 7f6605000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef15ae0 0x55e56ef268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.446+0000 7f6605000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef15ae0 0x55e56ef268e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59248/0 (socket says 172.21.15.83:59248) 2024-03-11T19:42:55.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.446+0000 7f6605000640 1 -- 172.21.15.83:0/384804520 learned_addr learned my addr 172.21.15.83:0/384804520 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:42:55.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.447+0000 7f65fffff640 1 -- 172.21.15.83:0/384804520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef1b640 msgr2=0x55e56ef24ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.447+0000 7f65fffff640 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef1b640 0x55e56ef24ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.447+0000 7f65fffff640 1 -- 172.21.15.83:0/384804520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef15ae0 msgr2=0x55e56ef268e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:42:55.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.447+0000 7f65fffff640 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef15ae0 0x55e56ef268e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.447+0000 7f65fffff640 1 -- 172.21.15.83:0/384804520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66001a19e0 con 0x55e56ef164d0 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.447+0000 7f6605000640 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef15ae0 0x55e56ef268e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.447+0000 7f65fffff640 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef164d0 0x55e56ef26e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55e56ef11b60 tx=0x7f65ec012660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c13372932c9f603 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.447+0000 7f65dffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.447+0000 7f65dffff640 1 -- 172.21.15.83:0/384804520 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec017070 con 0x55e56ef164d0 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.447+0000 7f65dffff640 1 -- 172.21.15.83:0/384804520 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f65ec000e40 con 0x55e56ef164d0 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.447+0000 7f6609e3c740 1 -- 172.21.15.83:0/384804520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65f801d8e0 con 0x55e56ef164d0 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.448+0000 7f65dffff640 1 -- 172.21.15.83:0/384804520 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65ec01ba40 con 0x55e56ef164d0 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.448+0000 7f6609e3c740 10 client.25577 fetch_fsmap learned FSMap version 28 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.448+0000 7f6609e3c740 1 -- 172.21.15.83:0/384804520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e56ee74610 con 0x55e56ef164d0 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.448+0000 7f65dffff640 1 -- 172.21.15.83:0/384804520 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65f801d8e0 con 0x55e56ef164d0 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.448+0000 7f65dffff640 1 -- 172.21.15.83:0/384804520 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f65ec00f5e0 con 0x55e56ef164d0 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.448+0000 7f6609e3c740 10 client.25577 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.448+0000 7f6609e3c740 10 client.25577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:42:55.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.448+0000 7f6609e3c740 20 client.25577 populate_metadata read hostname 'smithi083' 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.448+0000 7f6609e3c740 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e56ef831c0 0x55e56ef855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.448+0000 7f6609e3c740 4 client.25577 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.448+0000 7f6609e3c740 1 -- 172.21.15.83:0/384804520 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e56ec01890 con 0x55e56ef831c0 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.449+0000 7f6605000640 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e56ef831c0 0x55e56ef855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.449+0000 7f6605000640 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e56ef831c0 0x55e56ef855e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbe4fb00fccc9ec server_cookie=ccd9887b07bbc72b in_seq=0 out_seq=0 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.449+0000 7f65dffff640 10 client.25577 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.449+0000 7f65dffff640 1 -- 172.21.15.83:0/384804520 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e56ec01890 con 0x55e56ef831c0 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.449+0000 7f65dffff640 10 client.25577 handle_command_reply: tid=0 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.449+0000 7f6609e3c740 1 client.25577 shutdown 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6609e3c740 1 -- 172.21.15.83:0/384804520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef164d0 msgr2=0x55e56ef26e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6609e3c740 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef164d0 0x55e56ef26e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55e56ef11b60 tx=0x7f65ec012660 comp rx=0 tx=0).stop 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6609e3c740 1 -- 172.21.15.83:0/384804520 shutdown_connections 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6609e3c740 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e56ef831c0 0x55e56ef855e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6609e3c740 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e56ef1b640 0x55e56ef24ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6609e3c740 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e56ef164d0 0x55e56ef26e20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6609e3c740 1 --2- 172.21.15.83:0/384804520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e56ef15ae0 0x55e56ef268e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6609e3c740 1 -- 172.21.15.83:0/384804520 >> 172.21.15.83:0/384804520 conn(0x55e56ef11000 msgr2=0x55e56ef32b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f65dffff640 0 client.25577 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:42:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6609e3c740 1 -- 172.21.15.83:0/384804520 shutdown_connections 2024-03-11T19:42:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6609e3c740 1 -- 172.21.15.83:0/384804520 wait complete. 2024-03-11T19:42:55.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6609e3c740 20 client.25577 trim_cache size 0 max 16384 2024-03-11T19:42:55.457 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:42:55.458 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:42:55.458 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:42:55.458 INFO:teuthology.orchestra.run.smithi083.stdout: "e4062a39-0658-4901-94ae-b08270c4abe9": { 2024-03-11T19:42:55.458 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:42:55.458 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "e4062a39-0658-4901-94ae-b08270c4abe9", 2024-03-11T19:42:55.458 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:42:55.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:42:55.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:42:55.458 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:42:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.450+0000 7f6607264640 1 -- 172.21.15.83:0/4181858084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65f00a1a30 msgr2=0x7f65f00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.451+0000 7f6607264640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65f00a1a30 0x7f65f00a3ef0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f65f8021000 tx=0x7f65f801e000 comp rx=0 tx=0).stop 2024-03-11T19:42:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.451+0000 7f6607264640 1 -- 172.21.15.83:0/4181858084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6600100080 msgr2=0x7f6600199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:42:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.451+0000 7f6607264640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6600100080 0x7f6600199430 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f65f4000c00 tx=0x7f65f40040c0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.451+0000 7f6607264640 1 -- 172.21.15.83:0/4181858084 shutdown_connections 2024-03-11T19:42:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.451+0000 7f6607264640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f65f00a1a30 0x7f65f00a3ef0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.451+0000 7f6607264640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6600101c30 0x7f66001a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.451+0000 7f6607264640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6600101270 0x7f6600199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.451+0000 7f6607264640 1 --2- 172.21.15.83:0/4181858084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6600100080 0x7f6600199430 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:42:55.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.451+0000 7f6607264640 1 -- 172.21.15.83:0/4181858084 >> 172.21.15.83:0/4181858084 conn(0x7f66000fb5f0 msgr2=0x7f6600108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:42:55.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.451+0000 7f6607264640 1 -- 172.21.15.83:0/4181858084 shutdown_connections 2024-03-11T19:42:55.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:42:55.451+0000 7f6607264640 1 -- 172.21.15.83:0/4181858084 wait complete. 2024-03-11T19:42:55.469 INFO:tasks.cephfs.filesystem:scrub status for tag:e4062a39-0658-4901-94ae-b08270c4abe9 - {'path': '/', 'tag': 'e4062a39-0658-4901-94ae-b08270c4abe9', 'options': 'recursive,force'} 2024-03-11T19:42:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:55 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:42:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:55 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:55 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:42:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:55 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:56.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:55 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:42:56.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:55 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:56.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:55 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:42:56.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:55 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:56.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:55 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:42:56.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:55 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:56.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:55 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:42:56.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:55 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:56 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:42:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:56 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:56 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:42:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:56 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:42:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:56 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:56 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:56 smithi083 ceph-mon[31864]: pgmap v461: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 909 op/s 2024-03-11T19:42:57.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:56 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:42:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:56 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:56 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:42:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:56 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:42:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:56 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:56 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:56 smithi137 ceph-mon[36220]: pgmap v461: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 909 op/s 2024-03-11T19:42:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:56 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:42:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:56 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:56 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:42:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:56 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:42:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:56 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:56 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:42:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:56 smithi204 ceph-mon[36739]: pgmap v461: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 909 op/s 2024-03-11T19:42:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 220.0 s, 3273.2 tps, lat 30.507 ms stddev 32.803 2024-03-11T19:42:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:42:58 smithi083 ceph-mon[31864]: pgmap v462: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 944 op/s 2024-03-11T19:42:59.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:42:58 smithi137 ceph-mon[36220]: pgmap v462: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 944 op/s 2024-03-11T19:42:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:42:58 smithi204 ceph-mon[36739]: pgmap v462: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 31 MiB/s wr, 944 op/s 2024-03-11T19:43:00.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:00 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:43:00.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:00 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:43:00.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:00 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:43:01.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:01 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:43:01.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:01 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:43:01.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:01 smithi083 ceph-mon[31864]: pgmap v463: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 839 op/s 2024-03-11T19:43:01.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:01 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:43:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:01 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:43:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:01 smithi137 ceph-mon[36220]: pgmap v463: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 839 op/s 2024-03-11T19:43:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:01 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:43:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:01 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:43:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:01 smithi204 ceph-mon[36739]: pgmap v463: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 839 op/s 2024-03-11T19:43:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 225.0 s, 3625.0 tps, lat 27.679 ms stddev 30.610 2024-03-11T19:43:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:02 smithi083 ceph-mon[31864]: pgmap v464: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 806 op/s 2024-03-11T19:43:03.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:02 smithi137 ceph-mon[36220]: pgmap v464: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 806 op/s 2024-03-11T19:43:03.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:02 smithi204 ceph-mon[36739]: pgmap v464: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 806 op/s 2024-03-11T19:43:05.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:04 smithi083 ceph-mon[31864]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 865 op/s 2024-03-11T19:43:05.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:04 smithi137 ceph-mon[36220]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 865 op/s 2024-03-11T19:43:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:04 smithi204 ceph-mon[36739]: pgmap v465: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 26 MiB/s wr, 865 op/s 2024-03-11T19:43:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:06 smithi083 ceph-mon[31864]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 MiB/s wr, 1.07k op/s 2024-03-11T19:43:07.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:06 smithi137 ceph-mon[36220]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 MiB/s wr, 1.07k op/s 2024-03-11T19:43:07.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:06 smithi204 ceph-mon[36739]: pgmap v466: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 34 MiB/s wr, 1.07k op/s 2024-03-11T19:43:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 230.0 s, 3369.2 tps, lat 29.609 ms stddev 41.694 2024-03-11T19:43:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:08 smithi083 ceph-mon[31864]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 51 MiB/s wr, 1.47k op/s 2024-03-11T19:43:09.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:08 smithi137 ceph-mon[36220]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 51 MiB/s wr, 1.47k op/s 2024-03-11T19:43:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:08 smithi204 ceph-mon[36739]: pgmap v467: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 51 MiB/s wr, 1.47k op/s 2024-03-11T19:43:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:10 smithi083 ceph-mon[31864]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 48 MiB/s wr, 1.42k op/s 2024-03-11T19:43:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:10 smithi137 ceph-mon[36220]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 48 MiB/s wr, 1.42k op/s 2024-03-11T19:43:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:10 smithi204 ceph-mon[36739]: pgmap v468: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 48 MiB/s wr, 1.42k op/s 2024-03-11T19:43:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 235.0 s, 3175.4 tps, lat 31.531 ms stddev 33.559 2024-03-11T19:43:13.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:12 smithi083 ceph-mon[31864]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 46 MiB/s wr, 1.41k op/s 2024-03-11T19:43:13.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:12 smithi137 ceph-mon[36220]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 46 MiB/s wr, 1.41k op/s 2024-03-11T19:43:13.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:12 smithi204 ceph-mon[36739]: pgmap v469: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 46 MiB/s wr, 1.41k op/s 2024-03-11T19:43:15.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:14 smithi083 ceph-mon[31864]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 1.52k op/s 2024-03-11T19:43:15.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:14 smithi137 ceph-mon[36220]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 1.52k op/s 2024-03-11T19:43:15.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:14 smithi204 ceph-mon[36739]: pgmap v470: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 1.52k op/s 2024-03-11T19:43:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:16 smithi083 ceph-mon[31864]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 MiB/s wr, 1.45k op/s 2024-03-11T19:43:17.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:16 smithi137 ceph-mon[36220]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 MiB/s wr, 1.45k op/s 2024-03-11T19:43:17.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:16 smithi204 ceph-mon[36739]: pgmap v471: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 MiB/s wr, 1.45k op/s 2024-03-11T19:43:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 240.0 s, 3387.6 tps, lat 29.533 ms stddev 33.645 2024-03-11T19:43:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:18 smithi083 ceph-mon[31864]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.24k op/s 2024-03-11T19:43:19.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:18 smithi137 ceph-mon[36220]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.24k op/s 2024-03-11T19:43:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:18 smithi204 ceph-mon[36739]: pgmap v472: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.24k op/s 2024-03-11T19:43:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:20 smithi083 ceph-mon[31864]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 802 op/s 2024-03-11T19:43:21.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:20 smithi137 ceph-mon[36220]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 802 op/s 2024-03-11T19:43:21.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:20 smithi204 ceph-mon[36739]: pgmap v473: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 15 MiB/s wr, 802 op/s 2024-03-11T19:43:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 245.0 s, 3850.6 tps, lat 25.824 ms stddev 29.360 2024-03-11T19:43:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:22 smithi083 ceph-mon[31864]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-03-11T19:43:23.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:22 smithi137 ceph-mon[36220]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-03-11T19:43:23.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:22 smithi204 ceph-mon[36739]: pgmap v474: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 723 op/s 2024-03-11T19:43:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:24 smithi083 ceph-mon[31864]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 834 op/s 2024-03-11T19:43:25.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:24 smithi137 ceph-mon[36220]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 834 op/s 2024-03-11T19:43:25.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:24 smithi204 ceph-mon[36739]: pgmap v475: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 834 op/s 2024-03-11T19:43:25.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:43:25.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:43:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.622+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3154824636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e10075b40 msgr2=0x7f3e10075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.622+0000 7f3e15e9d640 1 --2- 172.21.15.83:0/3154824636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e10075b40 0x7f3e10075f60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009ca0 tx=0x7f3e0802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.622+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3154824636 shutdown_connections 2024-03-11T19:43:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.622+0000 7f3e15e9d640 1 --2- 172.21.15.83:0/3154824636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e10077720 0x7f3e10077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.622+0000 7f3e15e9d640 1 --2- 172.21.15.83:0/3154824636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e10076d80 0x7f3e100771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.622+0000 7f3e15e9d640 1 --2- 172.21.15.83:0/3154824636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e10075b40 0x7f3e10075f60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.622+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3154824636 >> 172.21.15.83:0/3154824636 conn(0x7f3e100fe940 msgr2=0x7f3e10100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.622+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3154824636 shutdown_connections 2024-03-11T19:43:25.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.622+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3154824636 wait complete. 2024-03-11T19:43:25.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.622+0000 7f3e15e9d640 1 Processor -- start 2024-03-11T19:43:25.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.623+0000 7f3e15e9d640 1 -- start start 2024-03-11T19:43:25.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.623+0000 7f3e15e9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e10075b40 0x7f3e1019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.623+0000 7f3e15e9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e10076d80 0x7f3e1019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.623+0000 7f3e15e9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e10077720 0x7f3e101a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.623+0000 7f3e15e9d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e101a52b0 con 0x7f3e10077720 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.623+0000 7f3e15e9d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e101a5420 con 0x7f3e10076d80 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.623+0000 7f3e15e9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e101a55e0 con 0x7f3e10075b40 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.623+0000 7f3e0f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e10075b40 0x7f3e1019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.623+0000 7f3e0effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e10076d80 0x7f3e1019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.623+0000 7f3e0effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e10076d80 0x7f3e1019dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56596/0 (socket says 172.21.15.83:56596) 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.623+0000 7f3e0effd640 1 -- 172.21.15.83:0/3240049984 learned_addr learned my addr 172.21.15.83:0/3240049984 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.624+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e10077720 0x7f3e101a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.624+0000 7f3e0effd640 1 -- 172.21.15.83:0/3240049984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e10075b40 msgr2=0x7f3e1019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.624+0000 7f3e0effd640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e10075b40 0x7f3e1019d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.624+0000 7f3e0effd640 1 -- 172.21.15.83:0/3240049984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e10077720 msgr2=0x7f3e101a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.624+0000 7f3e0effd640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e10077720 0x7f3e101a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.624+0000 7f3e0effd640 1 -- 172.21.15.83:0/3240049984 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e04009490 con 0x7f3e10076d80 2024-03-11T19:43:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.625+0000 7f3e0effd640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e10076d80 0x7f3e1019dd00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400ea60 tx=0x7f3e0400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.625+0000 7f3e0cff9640 1 -- 172.21.15.83:0/3240049984 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0400cde0 con 0x7f3e10076d80 2024-03-11T19:43:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.625+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3240049984 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e08009900 con 0x7f3e10076d80 2024-03-11T19:43:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.625+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3240049984 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e101a5c90 con 0x7f3e10076d80 2024-03-11T19:43:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.625+0000 7f3e0cff9640 1 -- 172.21.15.83:0/3240049984 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e040035f0 con 0x7f3e10076d80 2024-03-11T19:43:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.625+0000 7f3e0cff9640 1 -- 172.21.15.83:0/3240049984 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0400cde0 con 0x7f3e10076d80 2024-03-11T19:43:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.628+0000 7f3e0cff9640 1 -- 172.21.15.83:0/3240049984 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3e04004000 con 0x7f3e10076d80 2024-03-11T19:43:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.628+0000 7f3e18a74740 1 Processor -- start 2024-03-11T19:43:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.629+0000 7f3e0cff9640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3df80a1ae0 0x7f3df80a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.629+0000 7f3e0cff9640 1 -- 172.21.15.83:0/3240049984 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3e04014070 con 0x7f3e10076d80 2024-03-11T19:43:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.629+0000 7f3e18a74740 1 -- start start 2024-03-11T19:43:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e18a74740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e18a74740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07f5cc0 0x556aa07fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e18a74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 0x556aa07fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e18a74740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa0765280 con 0x556aa07fae30 2024-03-11T19:43:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e18a74740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa07627b0 con 0x556aa07f5cc0 2024-03-11T19:43:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e18a74740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa075df90 con 0x556aa07f52d0 2024-03-11T19:43:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:39666/0 (socket says 172.21.15.83:39666) 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/4040820050 learned_addr learned my addr 172.21.15.83:0/4040820050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3df80a1ae0 0x7f3df80a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0effd640 1 --2- 172.21.15.83:0/4040820050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07f5cc0 0x556aa07fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/4040820050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 0x556aa07fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/4040820050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07f5cc0 msgr2=0x556aa07fa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/4040820050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07f5cc0 0x556aa07fa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/4040820050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 msgr2=0x556aa07fd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/4040820050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 0x556aa07fd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/4040820050 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aa071c570 con 0x556aa07f52d0 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/4040820050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 0x556aa07fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3df80a1ae0 0x7f3df80a3fa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000b840 tx=0x7f3e000041b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.630+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/4040820050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f56f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e08032820 tx=0x7f3e080093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=143f9259624470cb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.631+0000 7f3df5ffb640 1 -- 172.21.15.83:0/4040820050 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08032ab0 con 0x556aa07f52d0 2024-03-11T19:43:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.631+0000 7f3df5ffb640 1 -- 172.21.15.83:0/4040820050 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e08032340 con 0x556aa07f52d0 2024-03-11T19:43:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.631+0000 7f3df5ffb640 1 -- 172.21.15.83:0/4040820050 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08003ca0 con 0x556aa07f52d0 2024-03-11T19:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.632+0000 7f3e18a74740 1 -- 172.21.15.83:0/4040820050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 msgr2=0x556aa07f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.632+0000 7f3e18a74740 1 --2- 172.21.15.83:0/4040820050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f56f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e08032820 tx=0x7f3e080093e0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.632+0000 7f3e18a74740 1 -- 172.21.15.83:0/4040820050 shutdown_connections 2024-03-11T19:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.632+0000 7f3e18a74740 1 --2- 172.21.15.83:0/4040820050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 0x556aa07fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.632+0000 7f3e18a74740 1 --2- 172.21.15.83:0/4040820050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07f5cc0 0x556aa07fa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.632+0000 7f3e18a74740 1 --2- 172.21.15.83:0/4040820050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f56f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.632+0000 7f3e18a74740 1 -- 172.21.15.83:0/4040820050 >> 172.21.15.83:0/4040820050 conn(0x556aa07f07f0 msgr2=0x556aa07f0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.632+0000 7f3e18a74740 1 -- 172.21.15.83:0/4040820050 shutdown_connections 2024-03-11T19:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.632+0000 7f3e18a74740 1 -- 172.21.15.83:0/4040820050 wait complete. 2024-03-11T19:43:25.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.633+0000 7f3e18a74740 1 -- start start 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.633+0000 7f3e18a74740 1 Processor -- start 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.633+0000 7f3e18a74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f52d0 0x556aa07ff190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e18a74740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f5cc0 0x556aa07ff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f52d0 0x556aa07ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e18a74740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 0x556aa0806710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f52d0 0x556aa07ff190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33106/0 (socket says 172.21.15.83:33106) 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e18a74740 1 -- 172.21.15.83:0/3699502564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa0431430 con 0x556aa07f52d0 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/3699502564 learned_addr learned my addr 172.21.15.83:0/3699502564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e18a74740 1 -- 172.21.15.83:0/3699502564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa03ede00 con 0x556aa07fae30 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e18a74740 1 -- 172.21.15.83:0/3699502564 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa06e2fd0 con 0x556aa07f5cc0 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e18a74740 10 client.0 _ll_register_callbacks cb 0x556aa0738fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0effd640 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f5cc0 0x556aa07ff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 0x556aa0806710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/3699502564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f5cc0 msgr2=0x556aa07ff6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f5cc0 0x556aa07ff6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/3699502564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 msgr2=0x556aa0806710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 0x556aa0806710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/3699502564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e100789f0 con 0x556aa07f52d0 2024-03-11T19:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0effd640 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f5cc0 0x556aa07ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 0x556aa0806710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f52d0 0x556aa07ff190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e080325e0 tx=0x7f3e08032fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed5f7e2a7b6a827 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.634+0000 7f3deeffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.635+0000 7f3deeffd640 1 -- 172.21.15.83:0/3699502564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08034650 con 0x556aa07f52d0 2024-03-11T19:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.635+0000 7f3deeffd640 1 -- 172.21.15.83:0/3699502564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3e08049450 con 0x556aa07f52d0 2024-03-11T19:43:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.635+0000 7f3e18a74740 1 -- 172.21.15.83:0/3699502564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556aa045d7e0 con 0x556aa07f52d0 2024-03-11T19:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.636+0000 7f3deeffd640 1 -- 172.21.15.83:0/3699502564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e080347b0 con 0x556aa07f52d0 2024-03-11T19:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.637+0000 7f3e18a74740 10 client.16104 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.637+0000 7f3e18a74740 1 -- 172.21.15.83:0/3699502564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556aa075c610 con 0x556aa07f52d0 2024-03-11T19:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.637+0000 7f3deeffd640 1 -- 172.21.15.83:0/3699502564 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556aa045d7e0 con 0x556aa07f52d0 2024-03-11T19:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.637+0000 7f3deeffd640 1 -- 172.21.15.83:0/3699502564 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3e080587b0 con 0x556aa07f52d0 2024-03-11T19:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.637+0000 7f3e18a74740 10 client.16104 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.637+0000 7f3e18a74740 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.637+0000 7f3e18a74740 20 client.16104 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.637+0000 7f3e18a74740 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556aa080b080 0x556aa080d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.637+0000 7f3e18a74740 4 client.16104 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:43:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.638+0000 7f3e18a74740 1 -- 172.21.15.83:0/3699502564 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556aa049a720 con 0x556aa080b080 2024-03-11T19:43:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.638+0000 7f3e0effd640 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556aa080b080 0x556aa080d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.639+0000 7f3deeffd640 10 client.16104 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.639+0000 7f3e0effd640 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556aa080b080 0x556aa080d4a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7eb3e054ac02c0f server_cookie=7a63ce7e738631da in_seq=0 out_seq=0 2024-03-11T19:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.639+0000 7f3deeffd640 1 -- 172.21.15.83:0/3699502564 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556aa049a720 con 0x556aa080b080 2024-03-11T19:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.639+0000 7f3deeffd640 10 client.16104 handle_command_reply: tid=0 2024-03-11T19:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 1 client.16104 shutdown 2024-03-11T19:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 1 -- 172.21.15.83:0/3699502564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f52d0 msgr2=0x556aa07ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f52d0 0x556aa07ff190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e080325e0 tx=0x7f3e08032fa0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 1 -- 172.21.15.83:0/3699502564 shutdown_connections 2024-03-11T19:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556aa080b080 0x556aa080d4a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3deeffd640 0 client.16104 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 0x556aa0806710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f5cc0 0x556aa07ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 1 --2- 172.21.15.83:0/3699502564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f52d0 0x556aa07ff190 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 1 -- 172.21.15.83:0/3699502564 >> 172.21.15.83:0/3699502564 conn(0x556aa07f07f0 msgr2=0x556aa07efa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 1 -- 172.21.15.83:0/3699502564 shutdown_connections 2024-03-11T19:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 1 -- 172.21.15.83:0/3699502564 wait complete. 2024-03-11T19:43:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.640+0000 7f3e18a74740 20 client.16104 trim_cache size 0 max 16384 2024-03-11T19:43:25.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.663+0000 7f3e18a74740 1 Processor -- start 2024-03-11T19:43:25.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.663+0000 7f3e18a74740 1 -- start start 2024-03-11T19:43:25.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.663+0000 7f3e18a74740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.663+0000 7f3e18a74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f5cc0 0x556aa07f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.664+0000 7f3e0f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.664+0000 7f3e0f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:39676/0 (socket says 172.21.15.83:39676) 2024-03-11T19:43:25.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.664+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/1192226103 learned_addr learned my addr 172.21.15.83:0/1192226103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:25.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.664+0000 7f3e18a74740 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 0x556aa08066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.664+0000 7f3e0effd640 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f5cc0 0x556aa07f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.664+0000 7f3e18a74740 1 -- 172.21.15.83:0/1192226103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa0807b90 con 0x556aa07f5cc0 2024-03-11T19:43:25.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.664+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 0x556aa08066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.664+0000 7f3e18a74740 1 -- 172.21.15.83:0/1192226103 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa07f60a0 con 0x556aa07fae30 2024-03-11T19:43:25.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.664+0000 7f3e18a74740 1 -- 172.21.15.83:0/1192226103 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa07fff50 con 0x556aa07f52d0 2024-03-11T19:43:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.665+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/1192226103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 msgr2=0x556aa08066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.665+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 0x556aa08066d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.665+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/1192226103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f5cc0 msgr2=0x556aa07f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.665+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f5cc0 0x556aa07f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.665+0000 7f3e0f7fe640 1 -- 172.21.15.83:0/1192226103 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aa06f8ac0 con 0x556aa07f52d0 2024-03-11T19:43:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.665+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 0x556aa08066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:43:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.665+0000 7f3e0effd640 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f5cc0 0x556aa07f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.665+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f6ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e08002230 tx=0x7f3e0802c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81851e750f12fd38 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.665+0000 7f3df57fa640 1 -- 172.21.15.83:0/1192226103 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0804d070 con 0x556aa07f52d0 2024-03-11T19:43:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.665+0000 7f3df57fa640 1 -- 172.21.15.83:0/1192226103 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e08058930 con 0x556aa07f52d0 2024-03-11T19:43:25.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.665+0000 7f3df57fa640 1 -- 172.21.15.83:0/1192226103 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08004040 con 0x556aa07f52d0 2024-03-11T19:43:25.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.666+0000 7f3e18a74740 1 -- 172.21.15.83:0/1192226103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 msgr2=0x556aa07f6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.666+0000 7f3e18a74740 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f6ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e08002230 tx=0x7f3e0802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.666+0000 7f3e18a74740 1 -- 172.21.15.83:0/1192226103 shutdown_connections 2024-03-11T19:43:25.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.667+0000 7f3e18a74740 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07fae30 0x556aa08066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.667+0000 7f3e18a74740 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07f5cc0 0x556aa07f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.667+0000 7f3e18a74740 1 --2- 172.21.15.83:0/1192226103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f52d0 0x556aa07f6ea0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.667+0000 7f3e18a74740 1 -- 172.21.15.83:0/1192226103 >> 172.21.15.83:0/1192226103 conn(0x556aa07f07f0 msgr2=0x556aa07f0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:25.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.667+0000 7f3e18a74740 1 -- 172.21.15.83:0/1192226103 shutdown_connections 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.667+0000 7f3e18a74740 1 -- 172.21.15.83:0/1192226103 wait complete. 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.667+0000 7f3e18a74740 1 -- start start 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.667+0000 7f3e18a74740 1 Processor -- start 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e18a74740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07f52d0 0x556aa0805f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e18a74740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f5cc0 0x556aa08064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e18a74740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 0x556aa0804570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e18a74740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa0804ab0 con 0x556aa07fae30 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e18a74740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa0804c20 con 0x556aa07f52d0 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e18a74740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa0804de0 con 0x556aa07f5cc0 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e18a74740 10 client.0 _ll_register_callbacks cb 0x7f3e08037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e0ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 0x556aa0804570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e0ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 0x556aa0804570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33120/0 (socket says 172.21.15.83:33120) 2024-03-11T19:43:25.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e0ffff640 1 -- 172.21.15.83:0/2427924937 learned_addr learned my addr 172.21.15.83:0/2427924937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:25.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e0effd640 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f5cc0 0x556aa08064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07f52d0 0x556aa0805f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e0ffff640 1 -- 172.21.15.83:0/2427924937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f5cc0 msgr2=0x556aa08064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:25.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f5cc0 0x556aa08064a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e0ffff640 1 -- 172.21.15.83:0/2427924937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07f52d0 msgr2=0x556aa0805f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:25.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07f52d0 0x556aa0805f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.668+0000 7f3e0ffff640 1 -- 172.21.15.83:0/2427924937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e101a5c90 con 0x556aa07fae30 2024-03-11T19:43:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.669+0000 7f3e0ffff640 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 0x556aa0804570 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3e00020460 tx=0x7f3e000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e60b40b241809c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.669+0000 7f3deeffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.670+0000 7f3deeffd640 1 -- 172.21.15.83:0/2427924937 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e00025070 con 0x556aa07fae30 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.670+0000 7f3deeffd640 1 -- 172.21.15.83:0/2427924937 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3e000126c0 con 0x556aa07fae30 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.670+0000 7f3e18a74740 1 -- 172.21.15.83:0/2427924937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e08032340 con 0x556aa07fae30 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.670+0000 7f3deeffd640 1 -- 172.21.15.83:0/2427924937 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e00012880 con 0x556aa07fae30 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.670+0000 7f3e18a74740 10 client.16110 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.670+0000 7f3e18a74740 1 -- 172.21.15.83:0/2427924937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556aa075c610 con 0x556aa07fae30 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.670+0000 7f3deeffd640 1 -- 172.21.15.83:0/2427924937 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3e08032340 con 0x556aa07fae30 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.671+0000 7f3deeffd640 1 -- 172.21.15.83:0/2427924937 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3e00029bf0 con 0x556aa07fae30 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.671+0000 7f3e18a74740 10 client.16110 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.671+0000 7f3e18a74740 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.671+0000 7f3e18a74740 20 client.16110 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.671+0000 7f3e18a74740 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556aa080b080 0x556aa0864990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.671+0000 7f3e18a74740 4 client.16110 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.671+0000 7f3e18a74740 1 -- 172.21.15.83:0/2427924937 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556aa04e9890 con 0x556aa080b080 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.671+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556aa080b080 0x556aa0864990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.672+0000 7f3e0f7fe640 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556aa080b080 0x556aa0864990 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21796c6ccfedfc server_cookie=680a322e4aa1be42 in_seq=0 out_seq=0 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.672+0000 7f3deeffd640 10 client.16110 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.672+0000 7f3deeffd640 1 -- 172.21.15.83:0/2427924937 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556aa04e9890 con 0x556aa080b080 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.672+0000 7f3deeffd640 10 client.16110 handle_command_reply: tid=0 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.672+0000 7f3e18a74740 1 client.16110 shutdown 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3e18a74740 1 -- 172.21.15.83:0/2427924937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 msgr2=0x556aa0804570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3e18a74740 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 0x556aa0804570 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3e00020460 tx=0x7f3e000046c0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3e18a74740 1 -- 172.21.15.83:0/2427924937 shutdown_connections 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3e18a74740 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556aa080b080 0x556aa0864990 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3e18a74740 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556aa07fae30 0x556aa0804570 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3deeffd640 0 client.16110 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3e18a74740 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556aa07f5cc0 0x556aa08064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3e18a74740 1 --2- 172.21.15.83:0/2427924937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556aa07f52d0 0x556aa0805f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3e18a74740 1 -- 172.21.15.83:0/2427924937 >> 172.21.15.83:0/2427924937 conn(0x556aa07f07f0 msgr2=0x556aa07f6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3e18a74740 1 -- 172.21.15.83:0/2427924937 shutdown_connections 2024-03-11T19:43:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3e18a74740 1 -- 172.21.15.83:0/2427924937 wait complete. 2024-03-11T19:43:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.673+0000 7f3e18a74740 20 client.16110 trim_cache size 0 max 16384 2024-03-11T19:43:25.680 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:43:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:43:25.680 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:43:25.680 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:43:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3240049984 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3df80a1ae0 msgr2=0x7f3df80a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3df80a1ae0 0x7f3df80a3fa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000b840 tx=0x7f3e000041b0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3240049984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e10076d80 msgr2=0x7f3e1019dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e10076d80 0x7f3e1019dd00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400ea60 tx=0x7f3e0400ef20 comp rx=0 tx=0).stop 2024-03-11T19:43:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3240049984 shutdown_connections 2024-03-11T19:43:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3df80a1ae0 0x7f3df80a3fa0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e10077720 0x7f3e101a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e10076d80 0x7f3e1019dd00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 --2- 172.21.15.83:0/3240049984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e10075b40 0x7f3e1019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3240049984 >> 172.21.15.83:0/3240049984 conn(0x7f3e100fe940 msgr2=0x7f3e10100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3240049984 shutdown_connections 2024-03-11T19:43:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.674+0000 7f3e15e9d640 1 -- 172.21.15.83:0/3240049984 wait complete. 2024-03-11T19:43:25.694 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:43:25.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:43:25.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.849+0000 7fd18156c640 1 -- 172.21.15.83:0/1434526665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd17c106e80 msgr2=0x7fd17c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.849+0000 7fd18156c640 1 --2- 172.21.15.83:0/1434526665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd17c106e80 0x7fd17c1092d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd174009be0 tx=0x7fd17402bef0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.849+0000 7fd18156c640 1 -- 172.21.15.83:0/1434526665 shutdown_connections 2024-03-11T19:43:25.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.849+0000 7fd18156c640 1 --2- 172.21.15.83:0/1434526665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd17c106e80 0x7fd17c1092d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.849+0000 7fd18156c640 1 --2- 172.21.15.83:0/1434526665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd17c104550 0x7fd17c106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.849+0000 7fd18156c640 1 --2- 172.21.15.83:0/1434526665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd17c101c20 0x7fd17c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.849+0000 7fd18156c640 1 -- 172.21.15.83:0/1434526665 >> 172.21.15.83:0/1434526665 conn(0x7fd17c0fb630 msgr2=0x7fd17c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:25.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.849+0000 7fd18156c640 1 -- 172.21.15.83:0/1434526665 shutdown_connections 2024-03-11T19:43:25.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.849+0000 7fd18156c640 1 -- 172.21.15.83:0/1434526665 wait complete. 2024-03-11T19:43:25.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.850+0000 7fd18156c640 1 Processor -- start 2024-03-11T19:43:25.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.850+0000 7fd18156c640 1 -- start start 2024-03-11T19:43:25.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.850+0000 7fd18156c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd17c101c20 0x7fd17c101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.850+0000 7fd18156c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd17c104550 0x7fd17c0ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.850+0000 7fd18156c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd17c106e80 0x7fd17c100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.850+0000 7fd18156c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c100870 con 0x7fd17c104550 2024-03-11T19:43:25.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.850+0000 7fd18156c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c1009e0 con 0x7fd17c106e80 2024-03-11T19:43:25.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.850+0000 7fd18156c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c1a1230 con 0x7fd17c101c20 2024-03-11T19:43:25.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.850+0000 7fd17affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd17c101c20 0x7fd17c101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd17affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd17c101c20 0x7fd17c101590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:39698/0 (socket says 172.21.15.83:39698) 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd17affd640 1 -- 172.21.15.83:0/3372862871 learned_addr learned my addr 172.21.15.83:0/3372862871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd17a7fc640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd17c104550 0x7fd17c0ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd17b7fe640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd17c106e80 0x7fd17c100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd17affd640 1 -- 172.21.15.83:0/3372862871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd17c106e80 msgr2=0x7fd17c100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd17affd640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd17c106e80 0x7fd17c100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd17affd640 1 -- 172.21.15.83:0/3372862871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd17c104550 msgr2=0x7fd17c0ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd17affd640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd17c104550 0x7fd17c0ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd17affd640 1 -- 172.21.15.83:0/3372862871 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd174009840 con 0x7fd17c101c20 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd17a7fc640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd17c104550 0x7fd17c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd17affd640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd17c101c20 0x7fd17c101590 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00e9c0 tx=0x7fd16c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd163fff640 1 -- 172.21.15.83:0/3372862871 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c00ccc0 con 0x7fd17c101c20 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd163fff640 1 -- 172.21.15.83:0/3372862871 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd16c00ce20 con 0x7fd17c101c20 2024-03-11T19:43:25.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.851+0000 7fd163fff640 1 -- 172.21.15.83:0/3372862871 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c010430 con 0x7fd17c101c20 2024-03-11T19:43:25.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.852+0000 7fd18156c640 1 -- 172.21.15.83:0/3372862871 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c1a14e0 con 0x7fd17c101c20 2024-03-11T19:43:25.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.852+0000 7fd18156c640 1 -- 172.21.15.83:0/3372862871 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c1a19a0 con 0x7fd17c101c20 2024-03-11T19:43:25.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.853+0000 7fd18156c640 1 -- 172.21.15.83:0/3372862871 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c0fd120 con 0x7fd17c101c20 2024-03-11T19:43:25.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.854+0000 7fd163fff640 1 -- 172.21.15.83:0/3372862871 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd16c005000 con 0x7fd17c101c20 2024-03-11T19:43:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.855+0000 7fd163fff640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1640a1b30 0x7fd1640a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.856+0000 7fd163fff640 1 -- 172.21.15.83:0/3372862871 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd16c014070 con 0x7fd17c101c20 2024-03-11T19:43:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.856+0000 7fd17a7fc640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1640a1b30 0x7fd1640a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.856+0000 7fd17a7fc640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1640a1b30 0x7fd1640a3ff0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd1700058c0 tx=0x7fd170005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:25.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:25.858+0000 7fd163fff640 1 -- 172.21.15.83:0/3372862871 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd16c096960 con 0x7fd17c101c20 2024-03-11T19:43:26.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.207+0000 7fd18156c640 1 -- 172.21.15.83:0/3372862871 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd17c10ce90 con 0x7fd17c101c20 2024-03-11T19:43:26.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.208+0000 7fd163fff640 1 -- 172.21.15.83:0/3372862871 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11351 (secure 0 0 0) 0x7fd16c09a020 con 0x7fd17c101c20 2024-03-11T19:43:26.213 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:43:26.213 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:42:34.146352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:43:26.213 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2024-03-11T19:43:26.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.211+0000 7fd18156c640 1 -- 172.21.15.83:0/3372862871 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1640a1b30 msgr2=0x7fd1640a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.211+0000 7fd18156c640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1640a1b30 0x7fd1640a3ff0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd1700058c0 tx=0x7fd170005850 comp rx=0 tx=0).stop 2024-03-11T19:43:26.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.211+0000 7fd18156c640 1 -- 172.21.15.83:0/3372862871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd17c101c20 msgr2=0x7fd17c101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.211+0000 7fd18156c640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd17c101c20 0x7fd17c101590 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00e9c0 tx=0x7fd16c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:43:26.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.212+0000 7fd18156c640 1 -- 172.21.15.83:0/3372862871 shutdown_connections 2024-03-11T19:43:26.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.212+0000 7fd18156c640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1640a1b30 0x7fd1640a3ff0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.212+0000 7fd18156c640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd17c106e80 0x7fd17c100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.212+0000 7fd18156c640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd17c104550 0x7fd17c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.212+0000 7fd18156c640 1 --2- 172.21.15.83:0/3372862871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd17c101c20 0x7fd17c101590 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00e9c0 tx=0x7fd16c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:43:26.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.212+0000 7fd18156c640 1 -- 172.21.15.83:0/3372862871 >> 172.21.15.83:0/3372862871 conn(0x7fd17c0fb630 msgr2=0x7fd17c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.212+0000 7fd18156c640 1 -- 172.21.15.83:0/3372862871 shutdown_connections 2024-03-11T19:43:26.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.212+0000 7fd18156c640 1 -- 172.21.15.83:0/3372862871 wait complete. 2024-03-11T19:43:26.235 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:43:26.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:43:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.403+0000 7f5b8e864640 1 -- 172.21.15.83:0/3309918881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b881033a0 msgr2=0x7f5b881037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.403+0000 7f5b8e864640 1 --2- 172.21.15.83:0/3309918881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b881033a0 0x7f5b881037c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009ca0 tx=0x7f5b8002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.403+0000 7f5b8e864640 1 -- 172.21.15.83:0/3309918881 shutdown_connections 2024-03-11T19:43:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.403+0000 7f5b8e864640 1 --2- 172.21.15.83:0/3309918881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b88104f50 0x7f5b8810b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.403+0000 7f5b8e864640 1 --2- 172.21.15.83:0/3309918881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b88104590 0x7f5b88104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.403+0000 7f5b8e864640 1 --2- 172.21.15.83:0/3309918881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b881033a0 0x7f5b881037c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.403+0000 7f5b8e864640 1 -- 172.21.15.83:0/3309918881 >> 172.21.15.83:0/3309918881 conn(0x7f5b880fe930 msgr2=0x7f5b88100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.403+0000 7f5b8e864640 1 -- 172.21.15.83:0/3309918881 shutdown_connections 2024-03-11T19:43:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.403+0000 7f5b8e864640 1 -- 172.21.15.83:0/3309918881 wait complete. 2024-03-11T19:43:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.404+0000 7f5b8e864640 1 Processor -- start 2024-03-11T19:43:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.404+0000 7f5b8e864640 1 -- start start 2024-03-11T19:43:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b8e864640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b881033a0 0x7f5b88078f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b8e864640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b88104590 0x7f5b88079470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b8e864640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b88104f50 0x7f5b88075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b87fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b881033a0 0x7f5b88078f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b87fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b881033a0 0x7f5b88078f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56676/0 (socket says 172.21.15.83:56676) 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b88104590 0x7f5b88079470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b87fff640 1 -- 172.21.15.83:0/1201254724 learned_addr learned my addr 172.21.15.83:0/1201254724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b8e864640 1 -- 172.21.15.83:0/1201254724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b88076230 con 0x7f5b88104f50 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b8e864640 1 -- 172.21.15.83:0/1201254724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880763a0 con 0x7f5b881033a0 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b8e864640 1 -- 172.21.15.83:0/1201254724 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b88076560 con 0x7f5b88104590 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b87fff640 1 -- 172.21.15.83:0/1201254724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b88104590 msgr2=0x7f5b88079470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b87fff640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b88104590 0x7f5b88079470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b87fff640 1 -- 172.21.15.83:0/1201254724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b88104f50 msgr2=0x7f5b88075aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.405+0000 7f5b87fff640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b88104f50 0x7f5b88075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.406+0000 7f5b87fff640 1 -- 172.21.15.83:0/1201254724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b80009900 con 0x7f5b881033a0 2024-03-11T19:43:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.406+0000 7f5b87fff640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b881033a0 0x7f5b88078f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5b800348f0 tx=0x7f5b80034a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.406+0000 7f5b857fa640 1 -- 172.21.15.83:0/1201254724 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800329b0 con 0x7f5b881033a0 2024-03-11T19:43:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.406+0000 7f5b857fa640 1 -- 172.21.15.83:0/1201254724 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b80003e50 con 0x7f5b881033a0 2024-03-11T19:43:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.406+0000 7f5b857fa640 1 -- 172.21.15.83:0/1201254724 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80040450 con 0x7f5b881033a0 2024-03-11T19:43:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.406+0000 7f5b8e864640 1 -- 172.21.15.83:0/1201254724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b881a58b0 con 0x7f5b881033a0 2024-03-11T19:43:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.406+0000 7f5b8e864640 1 -- 172.21.15.83:0/1201254724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b881a5d70 con 0x7f5b881033a0 2024-03-11T19:43:26.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b857fa640 1 -- 172.21.15.83:0/1201254724 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5b8000a040 con 0x7f5b881033a0 2024-03-11T19:43:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b91439740 1 Processor -- start 2024-03-11T19:43:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b91439740 1 -- start start 2024-03-11T19:43:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b91439740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b91439740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b91439740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500b2dd0 0x5642500b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b857fa640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5b700a1ab0 0x7f5b700a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b91439740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564250069960 con 0x5642500ad270 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b91439740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425001d310 con 0x5642500b2dd0 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b91439740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425001a840 con 0x5642500adc60 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b8ce01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b87fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b877fe640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5b700a1ab0 0x7f5b700a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.409+0000 7f5b8ce01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33174/0 (socket says 172.21.15.83:33174) 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b8ce01640 1 -- 172.21.15.83:0/299769702 learned_addr learned my addr 172.21.15.83:0/299769702 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b8ce01640 1 --2- 172.21.15.83:0/299769702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500b2dd0 0x5642500b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b857fa640 1 -- 172.21.15.83:0/1201254724 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5b800efe40 con 0x7f5b881033a0 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b8ce01640 1 -- 172.21.15.83:0/299769702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 msgr2=0x5642500b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b8ce01640 1 --2- 172.21.15.83:0/299769702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b8ce01640 1 -- 172.21.15.83:0/299769702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500b2dd0 msgr2=0x5642500b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b8ce01640 1 --2- 172.21.15.83:0/299769702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500b2dd0 0x5642500b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b8ce01640 1 -- 172.21.15.83:0/299769702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56424ffd8060 con 0x5642500ad270 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b877fe640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5b700a1ab0 0x7f5b700a3f70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5b88076eb0 tx=0x7f5b7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b8ce01640 1 --2- 172.21.15.83:0/299769702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500ad690 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5b78005fd0 tx=0x7f5b780093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a057eea558b18d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b6e7fc640 1 -- 172.21.15.83:0/299769702 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78014230 con 0x5642500ad270 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b6e7fc640 1 -- 172.21.15.83:0/299769702 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b78005500 con 0x5642500ad270 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.410+0000 7f5b6e7fc640 1 -- 172.21.15.83:0/299769702 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800d680 con 0x5642500ad270 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.411+0000 7f5b91439740 1 -- 172.21.15.83:0/299769702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 msgr2=0x5642500ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.411+0000 7f5b91439740 1 --2- 172.21.15.83:0/299769702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500ad690 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5b78005fd0 tx=0x7f5b780093c0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.411+0000 7f5b91439740 1 -- 172.21.15.83:0/299769702 shutdown_connections 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.411+0000 7f5b91439740 1 --2- 172.21.15.83:0/299769702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500b2dd0 0x5642500b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.411+0000 7f5b91439740 1 --2- 172.21.15.83:0/299769702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.411+0000 7f5b91439740 1 --2- 172.21.15.83:0/299769702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500ad690 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.411+0000 7f5b91439740 1 -- 172.21.15.83:0/299769702 >> 172.21.15.83:0/299769702 conn(0x5642500a87e0 msgr2=0x5642500a8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.411+0000 7f5b91439740 1 -- 172.21.15.83:0/299769702 shutdown_connections 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.411+0000 7f5b91439740 1 -- 172.21.15.83:0/299769702 wait complete. 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.411+0000 7f5b91439740 1 -- start start 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.411+0000 7f5b91439740 1 Processor -- start 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b91439740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500ad270 0x5642500b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b91439740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b91439740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500b2dd0 0x5642500bea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b91439740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564250016020 con 0x5642500b2dd0 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b91439740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424fce9430 con 0x5642500ad270 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b91439740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424fca5e00 con 0x5642500adc60 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b91439740 10 client.0 _ll_register_callbacks cb 0x56424ffbdcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b8ce01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500b2dd0 0x5642500bea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b8ce01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500b2dd0 0x5642500bea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33190/0 (socket says 172.21.15.83:33190) 2024-03-11T19:43:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b8ce01640 1 -- 172.21.15.83:0/1150338045 learned_addr learned my addr 172.21.15.83:0/1150338045 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b87fff640 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500ad270 0x5642500b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b877fe640 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b87fff640 1 -- 172.21.15.83:0/1150338045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 msgr2=0x5642500b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b87fff640 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500b79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b87fff640 1 -- 172.21.15.83:0/1150338045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500b2dd0 msgr2=0x5642500bea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b87fff640 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500b2dd0 0x5642500bea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b87fff640 1 -- 172.21.15.83:0/1150338045 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b780144b0 con 0x5642500ad270 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b87fff640 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500ad270 0x5642500b74b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b800eda40 tx=0x7f5b800b78a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fd49360fadd9f57 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b677fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b677fe640 1 -- 172.21.15.83:0/1150338045 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800bc7c0 con 0x5642500ad270 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b677fe640 1 -- 172.21.15.83:0/1150338045 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5b800bf040 con 0x5642500ad270 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.412+0000 7f5b91439740 1 -- 172.21.15.83:0/1150338045 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56424fd157e0 con 0x5642500ad270 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b677fe640 1 -- 172.21.15.83:0/1150338045 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b800c1ed0 con 0x5642500ad270 2024-03-11T19:43:26.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b91439740 10 client.25613 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b91439740 1 -- 172.21.15.83:0/1150338045 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5642500146a0 con 0x5642500ad270 2024-03-11T19:43:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b677fe640 1 -- 172.21.15.83:0/1150338045 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b800d4020 con 0x5642500ad270 2024-03-11T19:43:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b677fe640 1 -- 172.21.15.83:0/1150338045 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5b80050020 con 0x5642500ad270 2024-03-11T19:43:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b91439740 10 client.25613 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b91439740 10 client.25613 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:43:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b91439740 20 client.25613 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b91439740 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5642500c33a0 0x5642500c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b91439740 4 client.25613 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:43:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b91439740 1 -- 172.21.15.83:0/1150338045 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56424fd52720 con 0x5642500c33a0 2024-03-11T19:43:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.413+0000 7f5b877fe640 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5642500c33a0 0x5642500c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.414+0000 7f5b677fe640 10 client.25613 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.414+0000 7f5b877fe640 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5642500c33a0 0x5642500c57c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0c79207996dc55f server_cookie=4926e78f0dbe6df8 in_seq=0 out_seq=0 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.415+0000 7f5b677fe640 1 -- 172.21.15.83:0/1150338045 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56424fd52720 con 0x5642500c33a0 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.415+0000 7f5b677fe640 10 client.25613 handle_command_reply: tid=0 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.415+0000 7f5b91439740 1 client.25613 shutdown 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.416+0000 7f5b91439740 1 -- 172.21.15.83:0/1150338045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500ad270 msgr2=0x5642500b74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.416+0000 7f5b91439740 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500ad270 0x5642500b74b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b800eda40 tx=0x7f5b800b78a0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.416+0000 7f5b91439740 1 -- 172.21.15.83:0/1150338045 shutdown_connections 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.416+0000 7f5b91439740 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5642500c33a0 0x5642500c57c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.416+0000 7f5b91439740 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500b2dd0 0x5642500bea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.416+0000 7f5b91439740 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.416+0000 7f5b91439740 1 --2- 172.21.15.83:0/1150338045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500ad270 0x5642500b74b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.416+0000 7f5b91439740 1 -- 172.21.15.83:0/1150338045 >> 172.21.15.83:0/1150338045 conn(0x5642500a87e0 msgr2=0x5642500a7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.416+0000 7f5b677fe640 0 client.25613 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:43:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.416+0000 7f5b91439740 1 -- 172.21.15.83:0/1150338045 shutdown_connections 2024-03-11T19:43:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.416+0000 7f5b91439740 1 -- 172.21.15.83:0/1150338045 wait complete. 2024-03-11T19:43:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.417+0000 7f5b91439740 20 client.25613 trim_cache size 0 max 16384 2024-03-11T19:43:26.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.440+0000 7f5b91439740 1 Processor -- start 2024-03-11T19:43:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.440+0000 7f5b91439740 1 -- start start 2024-03-11T19:43:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.440+0000 7f5b91439740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.440+0000 7f5b91439740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500adc60 0x5642500be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.440+0000 7f5b91439740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500b2dd0 0x5642500af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.440+0000 7f5b91439740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424ff9b130 con 0x5642500ad270 2024-03-11T19:43:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.440+0000 7f5b91439740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642500ace30 con 0x5642500adc60 2024-03-11T19:43:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.440+0000 7f5b91439740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425009aa40 con 0x5642500b2dd0 2024-03-11T19:43:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.442+0000 7f5b87fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.442+0000 7f5b87fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33200/0 (socket says 172.21.15.83:33200) 2024-03-11T19:43:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.442+0000 7f5b87fff640 1 -- 172.21.15.83:0/794191669 learned_addr learned my addr 172.21.15.83:0/794191669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.442+0000 7f5b87fff640 1 -- 172.21.15.83:0/794191669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500b2dd0 msgr2=0x5642500af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.442+0000 7f5b87fff640 1 --2- 172.21.15.83:0/794191669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500b2dd0 0x5642500af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.442+0000 7f5b87fff640 1 -- 172.21.15.83:0/794191669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500adc60 msgr2=0x5642500be9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.442+0000 7f5b87fff640 1 --2- 172.21.15.83:0/794191669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500adc60 0x5642500be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.442+0000 7f5b87fff640 1 -- 172.21.15.83:0/794191669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56424ffb0c20 con 0x5642500ad270 2024-03-11T19:43:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.442+0000 7f5b87fff640 1 --2- 172.21.15.83:0/794191669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500af1b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5b800b97e0 tx=0x7f5b8009bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d55fc7bdb110e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.443+0000 7f5b6dffb640 1 -- 172.21.15.83:0/794191669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80032b10 con 0x5642500ad270 2024-03-11T19:43:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.443+0000 7f5b6dffb640 1 -- 172.21.15.83:0/794191669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b800d4030 con 0x5642500ad270 2024-03-11T19:43:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.443+0000 7f5b6dffb640 1 -- 172.21.15.83:0/794191669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800bd320 con 0x5642500ad270 2024-03-11T19:43:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.444+0000 7f5b91439740 1 -- 172.21.15.83:0/794191669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 msgr2=0x5642500af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.444+0000 7f5b91439740 1 --2- 172.21.15.83:0/794191669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500af1b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5b800b97e0 tx=0x7f5b8009bf90 comp rx=0 tx=0).stop 2024-03-11T19:43:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.446+0000 7f5b91439740 1 -- 172.21.15.83:0/794191669 shutdown_connections 2024-03-11T19:43:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.446+0000 7f5b91439740 1 --2- 172.21.15.83:0/794191669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500b2dd0 0x5642500af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.446+0000 7f5b91439740 1 --2- 172.21.15.83:0/794191669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500adc60 0x5642500be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.446+0000 7f5b91439740 1 --2- 172.21.15.83:0/794191669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500af1b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.446+0000 7f5b91439740 1 -- 172.21.15.83:0/794191669 >> 172.21.15.83:0/794191669 conn(0x5642500a87e0 msgr2=0x5642500a8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.448+0000 7f5b91439740 1 -- 172.21.15.83:0/794191669 shutdown_connections 2024-03-11T19:43:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.449+0000 7f5b91439740 1 -- 172.21.15.83:0/794191669 wait complete. 2024-03-11T19:43:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.450+0000 7f5b91439740 1 -- start start 2024-03-11T19:43:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.450+0000 7f5b91439740 1 Processor -- start 2024-03-11T19:43:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.451+0000 7f5b91439740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500bc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.451+0000 7f5b91439740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.451+0000 7f5b91439740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500b2dd0 0x5642500bdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.451+0000 7f5b91439740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642500be1b0 con 0x5642500ad270 2024-03-11T19:43:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.451+0000 7f5b91439740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642500be320 con 0x5642500b2dd0 2024-03-11T19:43:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.451+0000 7f5b91439740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642500be4e0 con 0x5642500adc60 2024-03-11T19:43:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.451+0000 7f5b877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.451+0000 7f5b8ce01640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500b2dd0 0x5642500bdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.451+0000 7f5b877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500bcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:39740/0 (socket says 172.21.15.83:39740) 2024-03-11T19:43:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.452+0000 7f5b877fe640 1 -- 172.21.15.83:0/2397637564 learned_addr learned my addr 172.21.15.83:0/2397637564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.452+0000 7f5b91439740 10 client.0 _ll_register_callbacks cb 0x7f5b780098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.452+0000 7f5b87fff640 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500bc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.452+0000 7f5b8ce01640 1 -- 172.21.15.83:0/2397637564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 msgr2=0x5642500bcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.452+0000 7f5b8ce01640 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500bcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.452+0000 7f5b8ce01640 1 -- 172.21.15.83:0/2397637564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 msgr2=0x5642500bc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.452+0000 7f5b8ce01640 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500bc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.452+0000 7f5b8ce01640 1 -- 172.21.15.83:0/2397637564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642500146a0 con 0x5642500b2dd0 2024-03-11T19:43:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.453+0000 7f5b8ce01640 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500b2dd0 0x5642500bdc70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5b780147a0 tx=0x7f5b78005b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e825f86f891e9efe server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.454+0000 7f5b677fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:43:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.454+0000 7f5b677fe640 1 -- 172.21.15.83:0/2397637564 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800da60 con 0x5642500b2dd0 2024-03-11T19:43:26.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.454+0000 7f5b91439740 1 -- 172.21.15.83:0/2397637564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b78005500 con 0x5642500b2dd0 2024-03-11T19:43:26.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.454+0000 7f5b677fe640 1 -- 172.21.15.83:0/2397637564 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5b7800dbc0 con 0x5642500b2dd0 2024-03-11T19:43:26.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.455+0000 7f5b677fe640 1 -- 172.21.15.83:0/2397637564 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b78005670 con 0x5642500b2dd0 2024-03-11T19:43:26.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.457+0000 7f5b91439740 10 client.25616 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:26.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.457+0000 7f5b91439740 1 -- 172.21.15.83:0/2397637564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56424ffd68c0 con 0x5642500b2dd0 2024-03-11T19:43:26.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.458+0000 7f5b677fe640 1 -- 172.21.15.83:0/2397637564 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b78005500 con 0x5642500b2dd0 2024-03-11T19:43:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.458+0000 7f5b677fe640 1 -- 172.21.15.83:0/2397637564 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5b78032070 con 0x5642500b2dd0 2024-03-11T19:43:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.459+0000 7f5b91439740 10 client.25616 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.459+0000 7f5b91439740 10 client.25616 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:43:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.459+0000 7f5b91439740 20 client.25616 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.459+0000 7f5b91439740 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x56425011a730 0x56425011cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.459+0000 7f5b91439740 4 client.25616 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:43:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.459+0000 7f5b91439740 1 -- 172.21.15.83:0/2397637564 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56424fda1890 con 0x56425011a730 2024-03-11T19:43:26.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.459+0000 7f5b87fff640 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x56425011a730 0x56425011cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.459+0000 7f5b677fe640 10 client.25616 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:43:26.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.459+0000 7f5b87fff640 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x56425011a730 0x56425011cb50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd45ee023f0587af server_cookie=857cdbf8faaa5572 in_seq=0 out_seq=0 2024-03-11T19:43:26.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.460+0000 7f5b677fe640 1 -- 172.21.15.83:0/2397637564 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56424fda1890 con 0x56425011a730 2024-03-11T19:43:26.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.460+0000 7f5b677fe640 10 client.25616 handle_command_reply: tid=0 2024-03-11T19:43:26.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.460+0000 7f5b91439740 1 client.25616 shutdown 2024-03-11T19:43:26.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.461+0000 7f5b91439740 1 -- 172.21.15.83:0/2397637564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500b2dd0 msgr2=0x5642500bdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.461+0000 7f5b91439740 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500b2dd0 0x5642500bdc70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5b780147a0 tx=0x7f5b78005b80 comp rx=0 tx=0).stop 2024-03-11T19:43:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.461+0000 7f5b91439740 1 -- 172.21.15.83:0/2397637564 shutdown_connections 2024-03-11T19:43:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.461+0000 7f5b91439740 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x56425011a730 0x56425011cb50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.461+0000 7f5b91439740 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642500b2dd0 0x5642500bdc70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.461+0000 7f5b91439740 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642500adc60 0x5642500bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.461+0000 7f5b91439740 1 --2- 172.21.15.83:0/2397637564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642500ad270 0x5642500bc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.461+0000 7f5b91439740 1 -- 172.21.15.83:0/2397637564 >> 172.21.15.83:0/2397637564 conn(0x5642500a87e0 msgr2=0x5642500af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.461+0000 7f5b677fe640 0 client.25616 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:43:26.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.462+0000 7f5b91439740 1 -- 172.21.15.83:0/2397637564 shutdown_connections 2024-03-11T19:43:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.463+0000 7f5b91439740 1 -- 172.21.15.83:0/2397637564 wait complete. 2024-03-11T19:43:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.463+0000 7f5b91439740 20 client.25616 trim_cache size 0 max 16384 2024-03-11T19:43:26.469 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:43:26.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 -- 172.21.15.83:0/1201254724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5b700a1ab0 msgr2=0x7f5b700a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5b700a1ab0 0x7f5b700a3f70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5b88076eb0 tx=0x7f5b7c005f90 comp rx=0 tx=0).stop 2024-03-11T19:43:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 -- 172.21.15.83:0/1201254724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b881033a0 msgr2=0x7f5b88078f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b881033a0 0x7f5b88078f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5b800348f0 tx=0x7f5b80034a90 comp rx=0 tx=0).stop 2024-03-11T19:43:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 -- 172.21.15.83:0/1201254724 shutdown_connections 2024-03-11T19:43:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5b700a1ab0 0x7f5b700a3f70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b88104f50 0x7f5b88075aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b88104590 0x7f5b88079470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 --2- 172.21.15.83:0/1201254724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b881033a0 0x7f5b88078f30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 -- 172.21.15.83:0/1201254724 >> 172.21.15.83:0/1201254724 conn(0x7f5b880fe930 msgr2=0x7f5b88107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 -- 172.21.15.83:0/1201254724 shutdown_connections 2024-03-11T19:43:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.465+0000 7f5b8e864640 1 -- 172.21.15.83:0/1201254724 wait complete. 2024-03-11T19:43:26.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:26 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3372862871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:43:26.485 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:43:26.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:43:26.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:26 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3372862871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:43:26.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.635+0000 7f7c11448640 1 -- 172.21.15.83:0/817345960 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0402cbf0 con 0x7f7c0c104f70 2024-03-11T19:43:26.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.635+0000 7f7c146ae640 1 -- 172.21.15.83:0/817345960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c0c104f70 msgr2=0x7f7c0c10b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.635+0000 7f7c146ae640 1 --2- 172.21.15.83:0/817345960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c0c104f70 0x7f7c0c10b800 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009900 tx=0x7f7c0402bc40 comp rx=0 tx=0).stop 2024-03-11T19:43:26.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.638+0000 7f7c146ae640 1 -- 172.21.15.83:0/817345960 shutdown_connections 2024-03-11T19:43:26.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.638+0000 7f7c146ae640 1 --2- 172.21.15.83:0/817345960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c0c104f70 0x7f7c0c10b800 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.638+0000 7f7c146ae640 1 --2- 172.21.15.83:0/817345960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c1045b0 0x7f7c0c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.638+0000 7f7c146ae640 1 --2- 172.21.15.83:0/817345960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c0c1033c0 0x7f7c0c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.638+0000 7f7c146ae640 1 -- 172.21.15.83:0/817345960 >> 172.21.15.83:0/817345960 conn(0x7f7c0c0fe930 msgr2=0x7f7c0c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.646+0000 7f7c146ae640 1 -- 172.21.15.83:0/817345960 shutdown_connections 2024-03-11T19:43:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:26 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3372862871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:43:26.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.649+0000 7f7c146ae640 1 -- 172.21.15.83:0/817345960 wait complete. 2024-03-11T19:43:26.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.649+0000 7f7c146ae640 1 Processor -- start 2024-03-11T19:43:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.653+0000 7f7c146ae640 1 -- start start 2024-03-11T19:43:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.653+0000 7f7c146ae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c1033c0 0x7f7c0c06f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.653+0000 7f7c146ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c0c1045b0 0x7f7c0c06fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.653+0000 7f7c146ae640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c0c104f70 0x7f7c0c1ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.653+0000 7f7c146ae640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c0795e0 con 0x7f7c0c1045b0 2024-03-11T19:43:26.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.653+0000 7f7c146ae640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c079750 con 0x7f7c0c1033c0 2024-03-11T19:43:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.653+0000 7f7c146ae640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c072f20 con 0x7f7c0c104f70 2024-03-11T19:43:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.653+0000 7f7c1244a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c1033c0 0x7f7c0c06f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.653+0000 7f7c1244a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c1033c0 0x7f7c0c06f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56722/0 (socket says 172.21.15.83:56722) 2024-03-11T19:43:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.653+0000 7f7c1244a640 1 -- 172.21.15.83:0/2445876931 learned_addr learned my addr 172.21.15.83:0/2445876931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.654+0000 7f7c1244a640 1 -- 172.21.15.83:0/2445876931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c0c104f70 msgr2=0x7f7c0c1ade70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-11T19:43:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.654+0000 7f7c1244a640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c0c104f70 0x7f7c0c1ade70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.654+0000 7f7c1244a640 1 -- 172.21.15.83:0/2445876931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c0c1045b0 msgr2=0x7f7c0c06fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.654+0000 7f7c1244a640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c0c1045b0 0x7f7c0c06fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.654+0000 7f7c1244a640 1 -- 172.21.15.83:0/2445876931 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc009490 con 0x7f7c0c1033c0 2024-03-11T19:43:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.655+0000 7f7c1244a640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c1033c0 0x7f7c0c06f990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00e9c0 tx=0x7f7bfc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.656+0000 7f7c037fe640 1 -- 172.21.15.83:0/2445876931 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc00cde0 con 0x7f7c0c1033c0 2024-03-11T19:43:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.656+0000 7f7c037fe640 1 -- 172.21.15.83:0/2445876931 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bfc010910 con 0x7f7c0c1033c0 2024-03-11T19:43:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.656+0000 7f7c146ae640 1 -- 172.21.15.83:0/2445876931 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c04009560 con 0x7f7c0c1033c0 2024-03-11T19:43:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.656+0000 7f7c146ae640 1 -- 172.21.15.83:0/2445876931 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c1a5840 con 0x7f7c0c1033c0 2024-03-11T19:43:26.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.658+0000 7f7c037fe640 1 -- 172.21.15.83:0/2445876931 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc018a50 con 0x7f7c0c1033c0 2024-03-11T19:43:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.660+0000 7f7c17282740 1 Processor -- start 2024-03-11T19:43:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.663+0000 7f7c17282740 1 -- start start 2024-03-11T19:43:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.664+0000 7f7c17282740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610087270 0x560610087690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.664+0000 7f7c17282740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087c60 0x56061008c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.664+0000 7f7c17282740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56061008cdd0 0x56061008f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.664+0000 7f7c17282740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560610043960 con 0x56061008cdd0 2024-03-11T19:43:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.664+0000 7f7c17282740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060fff7310 con 0x560610087c60 2024-03-11T19:43:26.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.664+0000 7f7c17282740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060fff4840 con 0x560610087270 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.664+0000 7f7c11c49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56061008cdd0 0x56061008f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.664+0000 7f7c11c49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56061008cdd0 0x56061008f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33238/0 (socket says 172.21.15.83:33238) 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.664+0000 7f7c11c49640 1 -- 172.21.15.83:0/663420870 learned_addr learned my addr 172.21.15.83:0/663420870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.665+0000 7f7c11c49640 1 -- 172.21.15.83:0/663420870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610087270 msgr2=0x560610087690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.665+0000 7f7c11c49640 1 --2- 172.21.15.83:0/663420870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610087270 0x560610087690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.665+0000 7f7c11c49640 1 -- 172.21.15.83:0/663420870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087c60 msgr2=0x56061008c760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.665+0000 7f7c11c49640 1 --2- 172.21.15.83:0/663420870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087c60 0x56061008c760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.665+0000 7f7c11c49640 1 -- 172.21.15.83:0/663420870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56060ffb2060 con 0x56061008cdd0 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.665+0000 7f7c11c49640 1 --2- 172.21.15.83:0/663420870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56061008cdd0 0x56061008f260 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009e70 tx=0x7f7c0800d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4d730b44443fb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.668+0000 7f7c037fe640 1 -- 172.21.15.83:0/2445876931 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7bfc010a80 con 0x7f7c0c1033c0 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.666+0000 7f7c00ff9640 1 -- 172.21.15.83:0/663420870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08010660 con 0x56061008cdd0 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.668+0000 7f7c00ff9640 1 -- 172.21.15.83:0/663420870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c0800de80 con 0x56061008cdd0 2024-03-11T19:43:26.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.668+0000 7f7c00ff9640 1 -- 172.21.15.83:0/663420870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0800fcc0 con 0x56061008cdd0 2024-03-11T19:43:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.668+0000 7f7c037fe640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7bf80c3c40 0x7f7bf80c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.669+0000 7f7c037fe640 1 -- 172.21.15.83:0/2445876931 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7bfc014070 con 0x7f7c0c1033c0 2024-03-11T19:43:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.670+0000 7f7c17282740 1 -- 172.21.15.83:0/663420870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56061008cdd0 msgr2=0x56061008f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.670+0000 7f7c17282740 1 --2- 172.21.15.83:0/663420870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56061008cdd0 0x56061008f260 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009e70 tx=0x7f7c0800d6b0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.670+0000 7f7c12c4b640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7bf80c3c40 0x7f7bf80c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.671+0000 7f7c17282740 1 -- 172.21.15.83:0/663420870 shutdown_connections 2024-03-11T19:43:26.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.671+0000 7f7c17282740 1 --2- 172.21.15.83:0/663420870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56061008cdd0 0x56061008f260 secure :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009e70 tx=0x7f7c0800d6b0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.671+0000 7f7c17282740 1 --2- 172.21.15.83:0/663420870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087c60 0x56061008c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.671+0000 7f7c17282740 1 --2- 172.21.15.83:0/663420870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610087270 0x560610087690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.671+0000 7f7c17282740 1 -- 172.21.15.83:0/663420870 >> 172.21.15.83:0/663420870 conn(0x5606100827e0 msgr2=0x560610082be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.671+0000 7f7c12c4b640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7bf80c3c40 0x7f7bf80c6100 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7c04036040 tx=0x7f7c04005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.673+0000 7f7c17282740 1 -- 172.21.15.83:0/663420870 shutdown_connections 2024-03-11T19:43:26.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.676+0000 7f7c17282740 1 -- 172.21.15.83:0/663420870 wait complete. 2024-03-11T19:43:26.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.676+0000 7f7c17282740 1 -- start start 2024-03-11T19:43:26.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.676+0000 7f7c17282740 1 Processor -- start 2024-03-11T19:43:26.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.681+0000 7f7c17282740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087270 0x560610091640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.681+0000 7f7c17282740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 0x560610091b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.681+0000 7f7c17282740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x560610098fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.681+0000 7f7c17282740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060fff0020 con 0x560610087c60 2024-03-11T19:43:26.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.681+0000 7f7c17282740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060fcc3430 con 0x560610087270 2024-03-11T19:43:26.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.681+0000 7f7c17282740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060fc7fe00 con 0x560610096bb0 2024-03-11T19:43:26.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.681+0000 7f7c17282740 10 client.0 _ll_register_callbacks cb 0x56060ff97cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:26.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.681+0000 7f7c11c49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 0x560610091b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.681+0000 7f7c11c49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 0x560610091b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33242/0 (socket says 172.21.15.83:33242) 2024-03-11T19:43:26.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.681+0000 7f7c11c49640 1 -- 172.21.15.83:0/3916592448 learned_addr learned my addr 172.21.15.83:0/3916592448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.681+0000 7f7c1244a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087270 0x560610091640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.682+0000 7f7c11c49640 1 -- 172.21.15.83:0/3916592448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 msgr2=0x560610098fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:43:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.682+0000 7f7c11c49640 1 --2- 172.21.15.83:0/3916592448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x560610098fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.682+0000 7f7c11c49640 1 -- 172.21.15.83:0/3916592448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087270 msgr2=0x560610091640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.682+0000 7f7c11c49640 1 --2- 172.21.15.83:0/3916592448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087270 0x560610091640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.682+0000 7f7c11c49640 1 -- 172.21.15.83:0/3916592448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c08009aa0 con 0x560610087c60 2024-03-11T19:43:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.682+0000 7f7c11c49640 1 --2- 172.21.15.83:0/3916592448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 0x560610091b80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009e70 tx=0x7f7c0800fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbcc5aa85bab80 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.683+0000 7f7bf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:43:26.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.683+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/3916592448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0801f070 con 0x560610087c60 2024-03-11T19:43:26.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.683+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/3916592448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7c08004b20 con 0x560610087c60 2024-03-11T19:43:26.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.683+0000 7f7c17282740 1 -- 172.21.15.83:0/3916592448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56060fd2c720 con 0x560610087c60 2024-03-11T19:43:26.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.685+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/3916592448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c0800ff10 con 0x560610087c60 2024-03-11T19:43:26.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.687+0000 7f7c17282740 10 client.16140 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:26.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.687+0000 7f7c17282740 1 -- 172.21.15.83:0/3916592448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56060ffee6a0 con 0x560610087c60 2024-03-11T19:43:26.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.689+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/3916592448 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56060fd2c720 con 0x560610087c60 2024-03-11T19:43:26.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.689+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/3916592448 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c0802b050 con 0x560610087c60 2024-03-11T19:43:26.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.689+0000 7f7c17282740 10 client.16140 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:26.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.689+0000 7f7c17282740 10 client.16140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:43:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.689+0000 7f7c17282740 20 client.16140 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.689+0000 7f7c17282740 1 --2- 172.21.15.83:0/3916592448 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x56061009db60 0x56061009ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.689+0000 7f7c17282740 4 client.16140 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:43:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.689+0000 7f7c17282740 1 -- 172.21.15.83:0/3916592448 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56060fd7b890 con 0x56061009db60 2024-03-11T19:43:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.690+0000 7f7c1244a640 1 --2- 172.21.15.83:0/3916592448 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x56061009db60 0x56061009ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.690+0000 7f7bf5ffb640 10 client.16140 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:43:26.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.690+0000 7f7c1244a640 1 --2- 172.21.15.83:0/3916592448 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x56061009db60 0x56061009ff80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87c3a95fde3d2ae8 server_cookie=4915ec51d845559f in_seq=0 out_seq=0 2024-03-11T19:43:26.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.691+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/3916592448 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56060fd7b890 con 0x56061009db60 2024-03-11T19:43:26.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.691+0000 7f7bf5ffb640 10 client.16140 handle_command_reply: tid=0 2024-03-11T19:43:26.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.691+0000 7f7c17282740 1 client.16140 shutdown 2024-03-11T19:43:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7c17282740 1 -- 172.21.15.83:0/3916592448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 msgr2=0x560610091b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7c17282740 1 --2- 172.21.15.83:0/3916592448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 0x560610091b80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009e70 tx=0x7f7c0800fc10 comp rx=0 tx=0).stop 2024-03-11T19:43:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7c17282740 1 -- 172.21.15.83:0/3916592448 shutdown_connections 2024-03-11T19:43:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7c17282740 1 --2- 172.21.15.83:0/3916592448 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x56061009db60 0x56061009ff80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7c17282740 1 --2- 172.21.15.83:0/3916592448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x560610098fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7c17282740 1 --2- 172.21.15.83:0/3916592448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 0x560610091b80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7c17282740 1 --2- 172.21.15.83:0/3916592448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087270 0x560610091640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7bf5ffb640 0 client.16140 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:43:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7c17282740 1 -- 172.21.15.83:0/3916592448 >> 172.21.15.83:0/3916592448 conn(0x5606100827e0 msgr2=0x560610081b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7c17282740 1 -- 172.21.15.83:0/3916592448 shutdown_connections 2024-03-11T19:43:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7c17282740 1 -- 172.21.15.83:0/3916592448 wait complete. 2024-03-11T19:43:26.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.693+0000 7f7c17282740 20 client.16140 trim_cache size 0 max 16384 2024-03-11T19:43:26.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.716+0000 7f7c17282740 1 Processor -- start 2024-03-11T19:43:26.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.716+0000 7f7c17282740 1 -- start start 2024-03-11T19:43:26.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.716+0000 7f7c17282740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087270 0x5606100ade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.716+0000 7f7c17282740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087c60 0x5606100ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.716+0000 7f7c17282740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x560610089370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.716+0000 7f7c17282740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060ff75130 con 0x560610087270 2024-03-11T19:43:26.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.716+0000 7f7c17282740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060fd2c720 con 0x560610087c60 2024-03-11T19:43:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.716+0000 7f7c17282740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560610074a40 con 0x560610096bb0 2024-03-11T19:43:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.716+0000 7f7c11c49640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087c60 0x5606100ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.717+0000 7f7c11c49640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087c60 0x5606100ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56736/0 (socket says 172.21.15.83:56736) 2024-03-11T19:43:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.717+0000 7f7c11c49640 1 -- 172.21.15.83:0/3996369094 learned_addr learned my addr 172.21.15.83:0/3996369094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.717+0000 7f7c12c4b640 1 --2- 172.21.15.83:0/3996369094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x560610089370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.717+0000 7f7c1244a640 1 --2- 172.21.15.83:0/3996369094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087270 0x5606100ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.717+0000 7f7c11c49640 1 -- 172.21.15.83:0/3996369094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 msgr2=0x560610089370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.717+0000 7f7c11c49640 1 --2- 172.21.15.83:0/3996369094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x560610089370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.717+0000 7f7c11c49640 1 -- 172.21.15.83:0/3996369094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087270 msgr2=0x5606100ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.717+0000 7f7c11c49640 1 --2- 172.21.15.83:0/3996369094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087270 0x5606100ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.717+0000 7f7c11c49640 1 -- 172.21.15.83:0/3996369094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56060ff8ac20 con 0x560610087c60 2024-03-11T19:43:26.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.717+0000 7f7c11c49640 1 --2- 172.21.15.83:0/3996369094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087c60 0x5606100ae360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c08004830 tx=0x7f7c080021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89ea7396d717d46 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.718+0000 7f7bf7fff640 1 -- 172.21.15.83:0/3996369094 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0801f040 con 0x560610087c60 2024-03-11T19:43:26.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.718+0000 7f7bf7fff640 1 -- 172.21.15.83:0/3996369094 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c0802e690 con 0x560610087c60 2024-03-11T19:43:26.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.718+0000 7f7bf7fff640 1 -- 172.21.15.83:0/3996369094 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0802eab0 con 0x560610087c60 2024-03-11T19:43:26.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.719+0000 7f7c17282740 1 -- 172.21.15.83:0/3996369094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087c60 msgr2=0x5606100ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.719+0000 7f7c17282740 1 --2- 172.21.15.83:0/3996369094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087c60 0x5606100ae360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c08004830 tx=0x7f7c080021d0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.720+0000 7f7c17282740 1 -- 172.21.15.83:0/3996369094 shutdown_connections 2024-03-11T19:43:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.720+0000 7f7c17282740 1 --2- 172.21.15.83:0/3996369094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x560610089370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.720+0000 7f7c17282740 1 --2- 172.21.15.83:0/3996369094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087c60 0x5606100ae360 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.720+0000 7f7c17282740 1 --2- 172.21.15.83:0/3996369094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087270 0x5606100ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.720+0000 7f7c17282740 1 -- 172.21.15.83:0/3996369094 >> 172.21.15.83:0/3996369094 conn(0x5606100827e0 msgr2=0x5606100a48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.720+0000 7f7c17282740 1 -- 172.21.15.83:0/3996369094 shutdown_connections 2024-03-11T19:43:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.720+0000 7f7c17282740 1 -- 172.21.15.83:0/3996369094 wait complete. 2024-03-11T19:43:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.720+0000 7f7c17282740 1 -- start start 2024-03-11T19:43:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c17282740 1 Processor -- start 2024-03-11T19:43:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c17282740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087270 0x5606100ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c17282740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 0x5606100adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c17282740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x5606100f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c17282740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606100f36f0 con 0x560610087c60 2024-03-11T19:43:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c17282740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606100f3830 con 0x560610087270 2024-03-11T19:43:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c17282740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606100f39a0 con 0x560610096bb0 2024-03-11T19:43:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c17282740 10 client.0 _ll_register_callbacks cb 0x7f7c080095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:26.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c11c49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 0x5606100adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c11c49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 0x5606100adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33258/0 (socket says 172.21.15.83:33258) 2024-03-11T19:43:26.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c11c49640 1 -- 172.21.15.83:0/1928966573 learned_addr learned my addr 172.21.15.83:0/1928966573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.721+0000 7f7c12c4b640 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x5606100f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7c1244a640 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087270 0x5606100ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7c12c4b640 1 -- 172.21.15.83:0/1928966573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087270 msgr2=0x5606100ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7c12c4b640 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087270 0x5606100ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7c12c4b640 1 -- 172.21.15.83:0/1928966573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 msgr2=0x5606100adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7c12c4b640 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 0x5606100adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7c12c4b640 1 -- 172.21.15.83:0/1928966573 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56060ffee6a0 con 0x560610096bb0 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7c12c4b640 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x5606100f31b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7c0402c250 tx=0x7f7c0402bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fc3c2733b196d40 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7bf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/1928966573 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c04004df0 con 0x560610096bb0 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/1928966573 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7c04004f50 con 0x560610096bb0 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7c17282740 1 -- 172.21.15.83:0/1928966573 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56060fcef7e0 con 0x560610096bb0 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/1928966573 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c04005110 con 0x560610096bb0 2024-03-11T19:43:26.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7c17282740 10 client.25729 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.722+0000 7f7c17282740 1 -- 172.21.15.83:0/1928966573 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56060ffb08c0 con 0x560610096bb0 2024-03-11T19:43:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.723+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/1928966573 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56060fcef7e0 con 0x560610096bb0 2024-03-11T19:43:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.723+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/1928966573 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c0403c070 con 0x560610096bb0 2024-03-11T19:43:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.723+0000 7f7c17282740 10 client.25729 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.723+0000 7f7c17282740 10 client.25729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:43:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.723+0000 7f7c17282740 20 client.25729 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.723+0000 7f7c17282740 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5606100f53a0 0x5606100f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.723+0000 7f7c17282740 4 client.25729 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:43:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.723+0000 7f7c17282740 1 -- 172.21.15.83:0/1928966573 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56060fc9b700 con 0x5606100f53a0 2024-03-11T19:43:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.723+0000 7f7c1244a640 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5606100f53a0 0x5606100f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.724+0000 7f7bf5ffb640 10 client.25729 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:43:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.724+0000 7f7c1244a640 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5606100f53a0 0x5606100f77c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abd15d7dcd5efc35 server_cookie=65473e0cbb5a053e in_seq=0 out_seq=0 2024-03-11T19:43:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.724+0000 7f7bf5ffb640 1 -- 172.21.15.83:0/1928966573 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56060fc9b700 con 0x5606100f53a0 2024-03-11T19:43:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.724+0000 7f7bf5ffb640 10 client.25729 handle_command_reply: tid=0 2024-03-11T19:43:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.724+0000 7f7c17282740 1 client.25729 shutdown 2024-03-11T19:43:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.725+0000 7f7c17282740 1 -- 172.21.15.83:0/1928966573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 msgr2=0x5606100f31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.725+0000 7f7c17282740 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x5606100f31b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7c0402c250 tx=0x7f7c0402bc40 comp rx=0 tx=0).stop 2024-03-11T19:43:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.726+0000 7f7c17282740 1 -- 172.21.15.83:0/1928966573 shutdown_connections 2024-03-11T19:43:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.726+0000 7f7c17282740 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5606100f53a0 0x5606100f77c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.726+0000 7f7c17282740 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560610096bb0 0x5606100f31b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.726+0000 7f7c17282740 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560610087c60 0x5606100adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.726+0000 7f7c17282740 1 --2- 172.21.15.83:0/1928966573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560610087270 0x5606100ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.726+0000 7f7c17282740 1 -- 172.21.15.83:0/1928966573 >> 172.21.15.83:0/1928966573 conn(0x5606100827e0 msgr2=0x560610089370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.726+0000 7f7bf5ffb640 0 client.25729 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:43:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.726+0000 7f7c17282740 1 -- 172.21.15.83:0/1928966573 shutdown_connections 2024-03-11T19:43:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.726+0000 7f7c17282740 1 -- 172.21.15.83:0/1928966573 wait complete. 2024-03-11T19:43:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.726+0000 7f7c17282740 20 client.25729 trim_cache size 0 max 16384 2024-03-11T19:43:26.733 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:43:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.727+0000 7f7c146ae640 1 -- 172.21.15.83:0/2445876931 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7bf80c3c40 msgr2=0x7f7bf80c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.727+0000 7f7c146ae640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7bf80c3c40 0x7f7bf80c6100 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7c04036040 tx=0x7f7c04005fd0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.727+0000 7f7c146ae640 1 -- 172.21.15.83:0/2445876931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c1033c0 msgr2=0x7f7c0c06f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.727+0000 7f7c146ae640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c1033c0 0x7f7c0c06f990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00e9c0 tx=0x7f7bfc00ee80 comp rx=0 tx=0).stop 2024-03-11T19:43:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.727+0000 7f7c146ae640 1 -- 172.21.15.83:0/2445876931 shutdown_connections 2024-03-11T19:43:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.727+0000 7f7c146ae640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7bf80c3c40 0x7f7bf80c6100 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.727+0000 7f7c146ae640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c0c104f70 0x7f7c0c1ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.727+0000 7f7c146ae640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c0c1045b0 0x7f7c0c06fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.727+0000 7f7c146ae640 1 --2- 172.21.15.83:0/2445876931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c0c1033c0 0x7f7c0c06f990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.727+0000 7f7c146ae640 1 -- 172.21.15.83:0/2445876931 >> 172.21.15.83:0/2445876931 conn(0x7f7c0c0fe930 msgr2=0x7f7c0c107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.727+0000 7f7c146ae640 1 -- 172.21.15.83:0/2445876931 shutdown_connections 2024-03-11T19:43:26.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.728+0000 7f7c146ae640 1 -- 172.21.15.83:0/2445876931 wait complete. 2024-03-11T19:43:26.748 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:43:26.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:43:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.898+0000 7fa5f5763640 1 -- 172.21.15.83:0/3546712416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5f0102d60 msgr2=0x7fa5f01095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.898+0000 7fa5f5763640 1 --2- 172.21.15.83:0/3546712416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5f0102d60 0x7fa5f01095f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009ca0 tx=0x7fa5e802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.898+0000 7fa5f5763640 1 -- 172.21.15.83:0/3546712416 shutdown_connections 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.898+0000 7fa5f5763640 1 --2- 172.21.15.83:0/3546712416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5f0102d60 0x7fa5f01095f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.898+0000 7fa5f5763640 1 --2- 172.21.15.83:0/3546712416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5f01023a0 0x7fa5f0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.898+0000 7fa5f5763640 1 --2- 172.21.15.83:0/3546712416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f01011b0 0x7fa5f01015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.898+0000 7fa5f5763640 1 -- 172.21.15.83:0/3546712416 >> 172.21.15.83:0/3546712416 conn(0x7fa5f00fc720 msgr2=0x7fa5f00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.899+0000 7fa5f5763640 1 -- 172.21.15.83:0/3546712416 shutdown_connections 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.899+0000 7fa5f5763640 1 -- 172.21.15.83:0/3546712416 wait complete. 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.899+0000 7fa5f5763640 1 Processor -- start 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.899+0000 7fa5f5763640 1 -- start start 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5f5763640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5f01011b0 0x7fa5f019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5f5763640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f01023a0 0x7fa5f019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5f5763640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5f0102d60 0x7fa5f01a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5f5763640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f01a52d0 con 0x7fa5f01011b0 2024-03-11T19:43:26.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5f5763640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f01a5440 con 0x7fa5f01023a0 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5f5763640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f01a5600 con 0x7fa5f0102d60 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5f01011b0 0x7fa5f019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f01023a0 0x7fa5f019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5f01011b0 0x7fa5f019d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33272/0 (socket says 172.21.15.83:33272) 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5eeffd640 1 -- 172.21.15.83:0/3774290250 learned_addr learned my addr 172.21.15.83:0/3774290250 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5ef7fe640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5f0102d60 0x7fa5f01a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5e7fff640 1 -- 172.21.15.83:0/3774290250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5f0102d60 msgr2=0x7fa5f01a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5f0102d60 0x7fa5f01a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5e7fff640 1 -- 172.21.15.83:0/3774290250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5f01011b0 msgr2=0x7fa5f019d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5f01011b0 0x7fa5f019d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.900+0000 7fa5e7fff640 1 -- 172.21.15.83:0/3774290250 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e8009900 con 0x7fa5f01023a0 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.901+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f01023a0 0x7fa5f019dd20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e9f0 tx=0x7fa5d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.901+0000 7fa5ecff9640 1 -- 172.21.15.83:0/3774290250 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800ccc0 con 0x7fa5f01023a0 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.901+0000 7fa5ecff9640 1 -- 172.21.15.83:0/3774290250 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5d800ce20 con 0x7fa5f01023a0 2024-03-11T19:43:26.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.901+0000 7fa5ecff9640 1 -- 172.21.15.83:0/3774290250 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8018920 con 0x7fa5f01023a0 2024-03-11T19:43:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.901+0000 7fa5f5763640 1 -- 172.21.15.83:0/3774290250 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f01a58e0 con 0x7fa5f01023a0 2024-03-11T19:43:26.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.901+0000 7fa5f5763640 1 -- 172.21.15.83:0/3774290250 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f01a5da0 con 0x7fa5f01023a0 2024-03-11T19:43:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.904+0000 7fa5ecff9640 1 -- 172.21.15.83:0/3774290250 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa5d8018a80 con 0x7fa5f01023a0 2024-03-11T19:43:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.904+0000 7fa5f840c740 1 Processor -- start 2024-03-11T19:43:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.904+0000 7fa5f840c740 1 -- start start 2024-03-11T19:43:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.904+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 0x55f3fd660690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.904+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660c60 0x55f3fd665760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.904+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 0x55f3fd668260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.904+0000 7fa5f840c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd61c960 con 0x55f3fd660270 2024-03-11T19:43:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.904+0000 7fa5f840c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd5d0310 con 0x55f3fd665dd0 2024-03-11T19:43:26.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.904+0000 7fa5f840c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd5cd840 con 0x55f3fd660c60 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 0x55f3fd660690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 0x55f3fd660690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33288/0 (socket says 172.21.15.83:33288) 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660c60 0x55f3fd665760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5ef7fe640 1 -- 172.21.15.83:0/3129706447 learned_addr learned my addr 172.21.15.83:0/3129706447 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/3129706447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 0x55f3fd668260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5eeffd640 1 -- 172.21.15.83:0/3129706447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 msgr2=0x55f3fd668260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5eeffd640 1 --2- 172.21.15.83:0/3129706447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 0x55f3fd668260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5eeffd640 1 -- 172.21.15.83:0/3129706447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 msgr2=0x55f3fd660690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5eeffd640 1 --2- 172.21.15.83:0/3129706447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 0x55f3fd660690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5eeffd640 1 -- 172.21.15.83:0/3129706447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3fd58b060 con 0x55f3fd660c60 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5eeffd640 1 --2- 172.21.15.83:0/3129706447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660c60 0x55f3fd665760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00058c0 tx=0x7fa5e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1495a29c85b57c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.905+0000 7fa5ecff9640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5dc0a1b30 0x7fa5dc0a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.906+0000 7fa5ef7fe640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5dc0a1b30 0x7fa5dc0a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.906+0000 7fa5e57fa640 1 -- 172.21.15.83:0/3129706447 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0004220 con 0x55f3fd660c60 2024-03-11T19:43:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.906+0000 7fa5e57fa640 1 -- 172.21.15.83:0/3129706447 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5e0004380 con 0x55f3fd660c60 2024-03-11T19:43:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.906+0000 7fa5e57fa640 1 -- 172.21.15.83:0/3129706447 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0004d90 con 0x55f3fd660c60 2024-03-11T19:43:26.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.906+0000 7fa5ecff9640 1 -- 172.21.15.83:0/3774290250 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa5d8014070 con 0x7fa5f01023a0 2024-03-11T19:43:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5f840c740 1 -- 172.21.15.83:0/3129706447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660c60 msgr2=0x55f3fd665760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5f840c740 1 --2- 172.21.15.83:0/3129706447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660c60 0x55f3fd665760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00058c0 tx=0x7fa5e0011810 comp rx=0 tx=0).stop 2024-03-11T19:43:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5ef7fe640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5dc0a1b30 0x7fa5dc0a3ff0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8032710 tx=0x7fa5e8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5f840c740 1 -- 172.21.15.83:0/3129706447 shutdown_connections 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5f840c740 1 --2- 172.21.15.83:0/3129706447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 0x55f3fd668260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5f840c740 1 --2- 172.21.15.83:0/3129706447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660c60 0x55f3fd665760 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5f840c740 1 --2- 172.21.15.83:0/3129706447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 0x55f3fd660690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5f840c740 1 -- 172.21.15.83:0/3129706447 >> 172.21.15.83:0/3129706447 conn(0x55f3fd65b7e0 msgr2=0x55f3fd65bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5f840c740 1 -- 172.21.15.83:0/3129706447 shutdown_connections 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5f840c740 1 -- 172.21.15.83:0/3129706447 wait complete. 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5f840c740 1 -- start start 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.907+0000 7fa5f840c740 1 Processor -- start 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.908+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660270 0x55f3fd66a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.908+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 0x55f3fd66a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.908+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 0x55f3fd6719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.908+0000 7fa5f840c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd5c9020 con 0x55f3fd660c60 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.908+0000 7fa5f840c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd29c430 con 0x55f3fd665dd0 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.908+0000 7fa5f840c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd258e00 con 0x55f3fd660270 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.908+0000 7fa5f840c740 10 client.0 _ll_register_callbacks cb 0x55f3fd570cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.908+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 0x55f3fd6719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.908+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 0x55f3fd6719d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56794/0 (socket says 172.21.15.83:56794) 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.908+0000 7fa5ef7fe640 1 -- 172.21.15.83:0/2460215659 learned_addr learned my addr 172.21.15.83:0/2460215659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.909+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 0x55f3fd66a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.909+0000 7fa5eeffd640 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660270 0x55f3fd66a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.909+0000 7fa5ef7fe640 1 -- 172.21.15.83:0/2460215659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660270 msgr2=0x55f3fd66a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.909+0000 7fa5ef7fe640 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660270 0x55f3fd66a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.909+0000 7fa5ef7fe640 1 -- 172.21.15.83:0/2460215659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 msgr2=0x55f3fd66a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.909+0000 7fa5ef7fe640 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 0x55f3fd66a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.909+0000 7fa5ef7fe640 1 -- 172.21.15.83:0/2460215659 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e00098e0 con 0x55f3fd665dd0 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.909+0000 7fa5eeffd640 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660270 0x55f3fd66a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5ef7fe640 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 0x55f3fd6719d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80325e0 tx=0x7fa5e80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb6a05968d850225 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5d67fc640 1 -- 172.21.15.83:0/2460215659 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8054660 con 0x55f3fd665dd0 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5d67fc640 1 -- 172.21.15.83:0/2460215659 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa5e8032af0 con 0x55f3fd665dd0 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5f840c740 1 -- 172.21.15.83:0/2460215659 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3fd2c87e0 con 0x55f3fd665dd0 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5d67fc640 1 -- 172.21.15.83:0/2460215659 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5e805a030 con 0x55f3fd665dd0 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5f840c740 10 client.25643 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5f840c740 1 -- 172.21.15.83:0/2460215659 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f3fd5c76a0 con 0x55f3fd665dd0 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5d67fc640 1 -- 172.21.15.83:0/2460215659 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f3fd2c87e0 con 0x55f3fd665dd0 2024-03-11T19:43:26.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5d67fc640 1 -- 172.21.15.83:0/2460215659 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa5e8047070 con 0x55f3fd665dd0 2024-03-11T19:43:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5f840c740 10 client.25643 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5f840c740 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5f840c740 20 client.25643 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5f840c740 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f3fd6876a0 0x55f3fd689ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5f840c740 4 client.25643 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:43:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.910+0000 7fa5f840c740 1 -- 172.21.15.83:0/2460215659 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3fd305720 con 0x55f3fd6876a0 2024-03-11T19:43:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.911+0000 7fa5eeffd640 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f3fd6876a0 0x55f3fd689ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.911+0000 7fa5eeffd640 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f3fd6876a0 0x55f3fd689ae0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11bf85dfa2de64f6 server_cookie=29a591d2437b4ab8 in_seq=0 out_seq=0 2024-03-11T19:43:26.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.911+0000 7fa5d67fc640 10 client.25643 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:26.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5d67fc640 1 -- 172.21.15.83:0/2460215659 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f3fd305720 con 0x55f3fd6876a0 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5d67fc640 10 client.25643 handle_command_reply: tid=0 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 1 client.25643 shutdown 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 1 -- 172.21.15.83:0/2460215659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 msgr2=0x55f3fd6719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 0x55f3fd6719d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80325e0 tx=0x7fa5e80095c0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 1 -- 172.21.15.83:0/2460215659 shutdown_connections 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f3fd6876a0 0x55f3fd689ae0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd665dd0 0x55f3fd6719d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5d67fc640 0 client.25643 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 0x55f3fd66a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 1 --2- 172.21.15.83:0/2460215659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd660270 0x55f3fd66a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 1 -- 172.21.15.83:0/2460215659 >> 172.21.15.83:0/2460215659 conn(0x55f3fd65b7e0 msgr2=0x55f3fd65ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 1 -- 172.21.15.83:0/2460215659 shutdown_connections 2024-03-11T19:43:26.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 1 -- 172.21.15.83:0/2460215659 wait complete. 2024-03-11T19:43:26.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.913+0000 7fa5f840c740 20 client.25643 trim_cache size 0 max 16384 2024-03-11T19:43:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.932+0000 7fa5f840c740 1 Processor -- start 2024-03-11T19:43:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.932+0000 7fa5f840c740 1 -- start start 2024-03-11T19:43:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 0x55f3fd65bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660c60 0x55f3fd671990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd665dd0 0x55f3fd662120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5f840c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd54e130 con 0x55f3fd660270 2024-03-11T19:43:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 0x55f3fd65bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 0x55f3fd65bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33306/0 (socket says 172.21.15.83:33306) 2024-03-11T19:43:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5eeffd640 1 -- 172.21.15.83:0/2382604097 learned_addr learned my addr 172.21.15.83:0/2382604097 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/2382604097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660c60 0x55f3fd671990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5f840c740 1 -- 172.21.15.83:0/2382604097 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd2c8810 con 0x55f3fd660c60 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5f840c740 1 -- 172.21.15.83:0/2382604097 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd661090 con 0x55f3fd665dd0 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5ef7fe640 1 --2- 172.21.15.83:0/2382604097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd665dd0 0x55f3fd662120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5e7fff640 1 -- 172.21.15.83:0/2382604097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd665dd0 msgr2=0x55f3fd662120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/2382604097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd665dd0 0x55f3fd662120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5e7fff640 1 -- 172.21.15.83:0/2382604097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 msgr2=0x55f3fd65bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/2382604097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 0x55f3fd65bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5e7fff640 1 -- 172.21.15.83:0/2382604097 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3fd563c20 con 0x55f3fd660c60 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5eeffd640 1 --2- 172.21.15.83:0/2382604097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 0x55f3fd65bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5ef7fe640 1 --2- 172.21.15.83:0/2382604097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd665dd0 0x55f3fd662120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.933+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/2382604097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660c60 0x55f3fd671990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa5d809a5e0 tx=0x7fa5d80ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c39eed4cdf6672d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.934+0000 7fa5e4ff9640 1 -- 172.21.15.83:0/2382604097 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80d09c0 con 0x55f3fd660c60 2024-03-11T19:43:26.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.934+0000 7fa5e4ff9640 1 -- 172.21.15.83:0/2382604097 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5d80968f0 con 0x55f3fd660c60 2024-03-11T19:43:26.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.934+0000 7fa5e4ff9640 1 -- 172.21.15.83:0/2382604097 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d809e830 con 0x55f3fd660c60 2024-03-11T19:43:26.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.934+0000 7fa5f840c740 1 -- 172.21.15.83:0/2382604097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660c60 msgr2=0x55f3fd671990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.934+0000 7fa5f840c740 1 --2- 172.21.15.83:0/2382604097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660c60 0x55f3fd671990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa5d809a5e0 tx=0x7fa5d80ba040 comp rx=0 tx=0).stop 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.935+0000 7fa5f840c740 1 -- 172.21.15.83:0/2382604097 shutdown_connections 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.935+0000 7fa5f840c740 1 --2- 172.21.15.83:0/2382604097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd665dd0 0x55f3fd662120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.935+0000 7fa5f840c740 1 --2- 172.21.15.83:0/2382604097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660c60 0x55f3fd671990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.935+0000 7fa5f840c740 1 --2- 172.21.15.83:0/2382604097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660270 0x55f3fd65bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.935+0000 7fa5f840c740 1 -- 172.21.15.83:0/2382604097 >> 172.21.15.83:0/2382604097 conn(0x55f3fd65b7e0 msgr2=0x55f3fd656a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.936+0000 7fa5f840c740 1 -- 172.21.15.83:0/2382604097 shutdown_connections 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.936+0000 7fa5f840c740 1 -- 172.21.15.83:0/2382604097 wait complete. 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.936+0000 7fa5f840c740 1 -- start start 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5f840c740 1 Processor -- start 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660270 0x55f3fd670240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 0x55f3fd670780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5f840c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd665dd0 0x55f3fd670cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5f840c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd6ddbf0 con 0x55f3fd660c60 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 0x55f3fd670780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660270 0x55f3fd670240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 0x55f3fd670780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33310/0 (socket says 172.21.15.83:33310) 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660270 0x55f3fd670240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56804/0 (socket says 172.21.15.83:56804) 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5e7fff640 1 -- 172.21.15.83:0/3910219789 learned_addr learned my addr 172.21.15.83:0/3910219789 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5f840c740 1 -- 172.21.15.83:0/3910219789 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd6ddd30 con 0x55f3fd660270 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.937+0000 7fa5f840c740 1 -- 172.21.15.83:0/3910219789 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3fd6ddea0 con 0x55f3fd665dd0 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5f840c740 10 client.0 _ll_register_callbacks cb 0x7fa5e0009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5ef7fe640 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd665dd0 0x55f3fd670cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5eeffd640 1 -- 172.21.15.83:0/3910219789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd665dd0 msgr2=0x55f3fd670cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5eeffd640 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd665dd0 0x55f3fd670cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5eeffd640 1 -- 172.21.15.83:0/3910219789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 msgr2=0x55f3fd670780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5eeffd640 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 0x55f3fd670780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5eeffd640 1 -- 172.21.15.83:0/3910219789 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3fd58b060 con 0x55f3fd660270 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 0x55f3fd670780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5eeffd640 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660270 0x55f3fd670240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0000c00 tx=0x7fa5e0004650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae628fb2d808e2cf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5d67fc640 1 -- 172.21.15.83:0/3910219789 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0009e00 con 0x55f3fd660270 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5d67fc640 1 -- 172.21.15.83:0/3910219789 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa5e00047e0 con 0x55f3fd660270 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5f840c740 1 -- 172.21.15.83:0/3910219789 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5e0004380 con 0x55f3fd660270 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5d67fc640 1 -- 172.21.15.83:0/3910219789 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5e00049a0 con 0x55f3fd660270 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5f840c740 10 client.25649 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.938+0000 7fa5f840c740 1 -- 172.21.15.83:0/3910219789 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fa5f01a5da0 con 0x55f3fd660270 2024-03-11T19:43:26.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.939+0000 7fa5d67fc640 1 -- 172.21.15.83:0/3910219789 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5e0004380 con 0x55f3fd660270 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.939+0000 7fa5d67fc640 1 -- 172.21.15.83:0/3910219789 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa5e0025600 con 0x55f3fd660270 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.939+0000 7fa5f840c740 10 client.25649 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.939+0000 7fa5f840c740 10 client.25649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.939+0000 7fa5f840c740 20 client.25649 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.939+0000 7fa5f840c740 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f3fd6df810 0x55f3fd6e1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.939+0000 7fa5f840c740 4 client.25649 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.939+0000 7fa5f840c740 1 -- 172.21.15.83:0/3910219789 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f3fd354890 con 0x55f3fd6df810 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.939+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f3fd6df810 0x55f3fd6e1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.939+0000 7fa5d67fc640 10 client.25649 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.939+0000 7fa5e7fff640 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f3fd6df810 0x55f3fd6e1c30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc0567a01d99e3f3 server_cookie=489f502a84143ce0 in_seq=0 out_seq=0 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.940+0000 7fa5d67fc640 1 -- 172.21.15.83:0/3910219789 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f3fd354890 con 0x55f3fd6df810 2024-03-11T19:43:26.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.940+0000 7fa5d67fc640 10 client.25649 handle_command_reply: tid=0 2024-03-11T19:43:26.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 1 client.25649 shutdown 2024-03-11T19:43:26.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 1 -- 172.21.15.83:0/3910219789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660270 msgr2=0x55f3fd670240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660270 0x55f3fd670240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0000c00 tx=0x7fa5e0004650 comp rx=0 tx=0).stop 2024-03-11T19:43:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 1 -- 172.21.15.83:0/3910219789 shutdown_connections 2024-03-11T19:43:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f3fd6df810 0x55f3fd6e1c30 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5d67fc640 0 client.25649 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3fd665dd0 0x55f3fd670cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3fd660c60 0x55f3fd670780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 1 --2- 172.21.15.83:0/3910219789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3fd660270 0x55f3fd670240 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 1 -- 172.21.15.83:0/3910219789 >> 172.21.15.83:0/3910219789 conn(0x55f3fd65b7e0 msgr2=0x55f3fd65bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 1 -- 172.21.15.83:0/3910219789 shutdown_connections 2024-03-11T19:43:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 1 -- 172.21.15.83:0/3910219789 wait complete. 2024-03-11T19:43:26.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.941+0000 7fa5f840c740 20 client.25649 trim_cache size 0 max 16384 2024-03-11T19:43:26.951 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:43:26.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 -- 172.21.15.83:0/3774290250 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5dc0a1b30 msgr2=0x7fa5dc0a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5dc0a1b30 0x7fa5dc0a3ff0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8032710 tx=0x7fa5e8005da0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 -- 172.21.15.83:0/3774290250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f01023a0 msgr2=0x7fa5f019dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f01023a0 0x7fa5f019dd20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e9f0 tx=0x7fa5d800eeb0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 -- 172.21.15.83:0/3774290250 shutdown_connections 2024-03-11T19:43:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5dc0a1b30 0x7fa5dc0a3ff0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5f0102d60 0x7fa5f01a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5f01023a0 0x7fa5f019dd20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 --2- 172.21.15.83:0/3774290250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5f01011b0 0x7fa5f019d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 -- 172.21.15.83:0/3774290250 >> 172.21.15.83:0/3774290250 conn(0x7fa5f00fc720 msgr2=0x7fa5f00733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 -- 172.21.15.83:0/3774290250 shutdown_connections 2024-03-11T19:43:26.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:26.942+0000 7fa5f5763640 1 -- 172.21.15.83:0/3774290250 wait complete. 2024-03-11T19:43:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 250.0 s, 3786.6 tps, lat 26.427 ms stddev 31.581 2024-03-11T19:43:27.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:27 smithi083 ceph-mon[31864]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 811 op/s 2024-03-11T19:43:27.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:27 smithi137 ceph-mon[36220]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 811 op/s 2024-03-11T19:43:27.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:27 smithi204 ceph-mon[36739]: pgmap v476: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 811 op/s 2024-03-11T19:43:27.963 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:43:27.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:43:27.964 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:43:28.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.161+0000 7fc58055c640 1 -- 172.21.15.83:0/2059704644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578108180 msgr2=0x7fc57810a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.162+0000 7fc58055c640 1 --2- 172.21.15.83:0/2059704644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578108180 0x7fc57810a5b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc568009860 tx=0x7fc56802bb70 comp rx=0 tx=0).stop 2024-03-11T19:43:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.162+0000 7fc58055c640 1 -- 172.21.15.83:0/2059704644 shutdown_connections 2024-03-11T19:43:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.162+0000 7fc58055c640 1 --2- 172.21.15.83:0/2059704644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57810d4a0 0x7fc57810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.162+0000 7fc58055c640 1 --2- 172.21.15.83:0/2059704644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc57810aaf0 0x7fc57810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.162+0000 7fc58055c640 1 --2- 172.21.15.83:0/2059704644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578108180 0x7fc57810a5b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.162+0000 7fc58055c640 1 -- 172.21.15.83:0/2059704644 >> 172.21.15.83:0/2059704644 conn(0x7fc57806dfe0 msgr2=0x7fc578070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.165+0000 7fc58055c640 1 -- 172.21.15.83:0/2059704644 shutdown_connections 2024-03-11T19:43:28.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.167+0000 7fc58055c640 1 -- 172.21.15.83:0/2059704644 wait complete. 2024-03-11T19:43:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.169+0000 7fc58055c640 1 Processor -- start 2024-03-11T19:43:28.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.170+0000 7fc58055c640 1 -- start start 2024-03-11T19:43:28.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.170+0000 7fc58055c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578108180 0x7fc5781a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.170+0000 7fc58055c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57810aaf0 0x7fc5781a6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.170+0000 7fc58055c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc57810d4a0 0x7fc5781ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.170+0000 7fc58055c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5781adbc0 con 0x7fc57810aaf0 2024-03-11T19:43:28.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.170+0000 7fc58055c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5781add30 con 0x7fc57810d4a0 2024-03-11T19:43:28.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.170+0000 7fc58055c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5781adef0 con 0x7fc578108180 2024-03-11T19:43:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.171+0000 7fc57e2f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578108180 0x7fc5781a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.171+0000 7fc57e2f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578108180 0x7fc5781a60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:39848/0 (socket says 172.21.15.83:39848) 2024-03-11T19:43:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.171+0000 7fc57e2f8640 1 -- 172.21.15.83:0/3532644037 learned_addr learned my addr 172.21.15.83:0/3532644037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.171+0000 7fc57eaf9640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc57810d4a0 0x7fc5781ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.171+0000 7fc57daf7640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57810aaf0 0x7fc5781a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.172+0000 7fc57e2f8640 1 -- 172.21.15.83:0/3532644037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc57810d4a0 msgr2=0x7fc5781ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.172+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc57810d4a0 0x7fc5781ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.172+0000 7fc57e2f8640 1 -- 172.21.15.83:0/3532644037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57810aaf0 msgr2=0x7fc5781a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.172+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57810aaf0 0x7fc5781a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.172+0000 7fc57e2f8640 1 -- 172.21.15.83:0/3532644037 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc568009490 con 0x7fc578108180 2024-03-11T19:43:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.172+0000 7fc57daf7640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57810aaf0 0x7fc5781a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.172+0000 7fc57eaf9640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc57810d4a0 0x7fc5781ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.172+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578108180 0x7fc5781a60d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc568000c00 tx=0x7fc568034670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.172+0000 7fc56f7fe640 1 -- 172.21.15.83:0/3532644037 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc568005230 con 0x7fc578108180 2024-03-11T19:43:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.173+0000 7fc56f7fe640 1 -- 172.21.15.83:0/3532644037 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc56802c9f0 con 0x7fc578108180 2024-03-11T19:43:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.173+0000 7fc58055c640 1 -- 172.21.15.83:0/3532644037 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5781ae170 con 0x7fc578108180 2024-03-11T19:43:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.173+0000 7fc56f7fe640 1 -- 172.21.15.83:0/3532644037 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc568004b90 con 0x7fc578108180 2024-03-11T19:43:28.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.173+0000 7fc58055c640 1 -- 172.21.15.83:0/3532644037 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5781ae670 con 0x7fc578108180 2024-03-11T19:43:28.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.177+0000 7fc583205740 1 Processor -- start 2024-03-11T19:43:28.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.177+0000 7fc583205740 1 -- start start 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.177+0000 7fc56f7fe640 1 -- 172.21.15.83:0/3532644037 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc56803b040 con 0x7fc578108180 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.177+0000 7fc583205740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d944c0 0x55b054d948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.177+0000 7fc583205740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 0x55b054d95b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc583205740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d96050 0x55b054d9c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc583205740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b054d03f90 con 0x55b054d944c0 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc583205740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b054d014c0 con 0x55b054d95690 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc583205740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b054cfcca0 con 0x55b054d96050 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc57daf7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d96050 0x55b054d9c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc57e2f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 0x55b054d95b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc57daf7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d96050 0x55b054d9c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:39850/0 (socket says 172.21.15.83:39850) 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc57daf7640 1 -- 172.21.15.83:0/1338699655 learned_addr learned my addr 172.21.15.83:0/1338699655 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc57eaf9640 1 --2- 172.21.15.83:0/1338699655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d944c0 0x55b054d948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc57daf7640 1 -- 172.21.15.83:0/1338699655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 msgr2=0x55b054d95b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc57daf7640 1 --2- 172.21.15.83:0/1338699655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 0x55b054d95b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc57daf7640 1 -- 172.21.15.83:0/1338699655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d944c0 msgr2=0x55b054d948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc57daf7640 1 --2- 172.21.15.83:0/1338699655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d944c0 0x55b054d948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.178+0000 7fc57daf7640 1 -- 172.21.15.83:0/1338699655 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b054cbf0b0 con 0x55b054d96050 2024-03-11T19:43:28.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.179+0000 7fc57daf7640 1 --2- 172.21.15.83:0/1338699655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d96050 0x55b054d9c8e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc574010480 tx=0x7fc574010880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f24838fb7e0cf5db server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.179+0000 7fc56cbfe640 1 -- 172.21.15.83:0/1338699655 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574011700 con 0x55b054d96050 2024-03-11T19:43:28.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.179+0000 7fc56cbfe640 1 -- 172.21.15.83:0/1338699655 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc574015070 con 0x55b054d96050 2024-03-11T19:43:28.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.179+0000 7fc56cbfe640 1 -- 172.21.15.83:0/1338699655 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57401cde0 con 0x55b054d96050 2024-03-11T19:43:28.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.180+0000 7fc56f7fe640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5640a1b80 0x7fc5640a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.180+0000 7fc56f7fe640 1 -- 172.21.15.83:0/3532644037 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc5680ef410 con 0x7fc578108180 2024-03-11T19:43:28.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.181+0000 7fc583205740 1 -- 172.21.15.83:0/1338699655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d96050 msgr2=0x55b054d9c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.181+0000 7fc583205740 1 --2- 172.21.15.83:0/1338699655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d96050 0x55b054d9c8e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc574010480 tx=0x7fc574010880 comp rx=0 tx=0).stop 2024-03-11T19:43:28.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.182+0000 7fc583205740 1 -- 172.21.15.83:0/1338699655 shutdown_connections 2024-03-11T19:43:28.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.182+0000 7fc583205740 1 --2- 172.21.15.83:0/1338699655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d96050 0x55b054d9c8e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.182+0000 7fc583205740 1 --2- 172.21.15.83:0/1338699655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 0x55b054d95b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.182+0000 7fc583205740 1 --2- 172.21.15.83:0/1338699655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d944c0 0x55b054d948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.182+0000 7fc583205740 1 -- 172.21.15.83:0/1338699655 >> 172.21.15.83:0/1338699655 conn(0x55b054d8f6a0 msgr2=0x55b054d8faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.183+0000 7fc57eaf9640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5640a1b80 0x7fc5640a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.183+0000 7fc583205740 1 -- 172.21.15.83:0/1338699655 shutdown_connections 2024-03-11T19:43:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.183+0000 7fc583205740 1 -- 172.21.15.83:0/1338699655 wait complete. 2024-03-11T19:43:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.183+0000 7fc583205740 1 -- start start 2024-03-11T19:43:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc583205740 1 Processor -- start 2024-03-11T19:43:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc583205740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 0x55b054daf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc583205740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 0x55b054dafc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc583205740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d96050 0x55b054db6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc583205740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b054ca1f40 con 0x55b054d96050 2024-03-11T19:43:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc57daf7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 0x55b054dafc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc57daf7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 0x55b054dafc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56846/0 (socket says 172.21.15.83:56846) 2024-03-11T19:43:28.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc57daf7640 1 -- 172.21.15.83:0/3971593992 learned_addr learned my addr 172.21.15.83:0/3971593992 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc583205740 1 -- 172.21.15.83:0/3971593992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0549d04b0 con 0x55b054d95690 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc583205740 1 -- 172.21.15.83:0/3971593992 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b054c82670 con 0x55b054d944c0 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 0x55b054daf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc57eaf9640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5640a1b80 0x7fc5640a4040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55b054d95590 tx=0x7fc570011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.184+0000 7fc57eaf9640 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d96050 0x55b054db6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc57daf7640 1 -- 172.21.15.83:0/3971593992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 msgr2=0x55b054daf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc57daf7640 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 0x55b054daf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc57daf7640 1 -- 172.21.15.83:0/3971593992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d96050 msgr2=0x55b054db6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc57daf7640 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d96050 0x55b054db6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc57daf7640 1 -- 172.21.15.83:0/3971593992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57400bcd0 con 0x55b054d95690 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 0x55b054daf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc583205740 10 client.0 _ll_register_callbacks cb 0x55b054ceff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc57daf7640 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 0x55b054dafc20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc574005940 tx=0x7fc57401dda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700a28bd95ca7a72 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc583205740 1 -- 172.21.15.83:0/3971593992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b054cd6f40 con 0x55b054d95690 2024-03-11T19:43:28.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc561ffb640 10 client.25661 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:43:28.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc561ffb640 1 -- 172.21.15.83:0/3971593992 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574020040 con 0x55b054d95690 2024-03-11T19:43:28.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc561ffb640 1 -- 172.21.15.83:0/3971593992 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc574017070 con 0x55b054d95690 2024-03-11T19:43:28.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc561ffb640 1 -- 172.21.15.83:0/3971593992 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc574005450 con 0x55b054d95690 2024-03-11T19:43:28.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc583205740 10 client.25661 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:28.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.185+0000 7fc583205740 1 -- 172.21.15.83:0/3971593992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=28}) v3 -- 0x55b054cfb320 con 0x55b054d95690 2024-03-11T19:43:28.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.187+0000 7fc561ffb640 1 -- 172.21.15.83:0/3971593992 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc5740046c0 con 0x55b054d95690 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.187+0000 7fc583205740 10 client.25661 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.187+0000 7fc583205740 10 client.25661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.187+0000 7fc583205740 20 client.25661 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.187+0000 7fc583205740 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b054dbb5a0 0x55b054dbda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.187+0000 7fc583205740 4 client.25661 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.187+0000 7fc583205740 1 -- 172.21.15.83:0/3971593992 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b054cf3310 con 0x55b054dbb5a0 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.187+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b054dbb5a0 0x55b054dbda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.188+0000 7fc561ffb640 10 client.25661 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.188+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b054dbb5a0 0x55b054dbda60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd09844103a97c8 server_cookie=4f980dad50f514e5 in_seq=0 out_seq=0 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.188+0000 7fc561ffb640 1 -- 172.21.15.83:0/3971593992 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b054cf3310 con 0x55b054dbb5a0 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.188+0000 7fc561ffb640 10 client.25661 handle_command_reply: tid=0 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.188+0000 7fc583205740 1 client.25661 shutdown 2024-03-11T19:43:28.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc583205740 1 -- 172.21.15.83:0/3971593992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 msgr2=0x55b054dafc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc583205740 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 0x55b054dafc20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc574005940 tx=0x7fc57401dda0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc583205740 1 -- 172.21.15.83:0/3971593992 shutdown_connections 2024-03-11T19:43:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc583205740 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b054dbb5a0 0x55b054dbda60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc583205740 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d96050 0x55b054db6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc561ffb640 0 client.25661 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc583205740 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d95690 0x55b054dafc20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc583205740 1 --2- 172.21.15.83:0/3971593992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 0x55b054daf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc583205740 1 -- 172.21.15.83:0/3971593992 >> 172.21.15.83:0/3971593992 conn(0x55b054d8f6a0 msgr2=0x55b054d8e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc583205740 1 -- 172.21.15.83:0/3971593992 shutdown_connections 2024-03-11T19:43:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc583205740 1 -- 172.21.15.83:0/3971593992 wait complete. 2024-03-11T19:43:28.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.189+0000 7fc583205740 20 client.25661 trim_cache size 0 max 16384 2024-03-11T19:43:28.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc583205740 1 Processor -- start 2024-03-11T19:43:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc583205740 1 -- start start 2024-03-11T19:43:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc583205740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d944c0 0x55b054daf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc583205740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d95690 0x55b054db6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc583205740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d96050 0x55b054d8e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc583205740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b054d8fee0 con 0x55b054d96050 2024-03-11T19:43:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc583205740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b054d84350 con 0x55b054d944c0 2024-03-11T19:43:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc583205740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b054db0410 con 0x55b054d95690 2024-03-11T19:43:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc57e2f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d944c0 0x55b054daf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc57e2f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d944c0 0x55b054daf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56850/0 (socket says 172.21.15.83:56850) 2024-03-11T19:43:28.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc57e2f8640 1 -- 172.21.15.83:0/3572942424 learned_addr learned my addr 172.21.15.83:0/3572942424 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:28.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc57daf7640 1 --2- 172.21.15.83:0/3572942424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d95690 0x55b054db6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc57eaf9640 1 --2- 172.21.15.83:0/3572942424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d96050 0x55b054d8e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc57e2f8640 1 -- 172.21.15.83:0/3572942424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d95690 msgr2=0x55b054db6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/3572942424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d95690 0x55b054db6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc57e2f8640 1 -- 172.21.15.83:0/3572942424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d96050 msgr2=0x55b054d8e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.214+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/3572942424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d96050 0x55b054d8e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.215+0000 7fc57e2f8640 1 -- 172.21.15.83:0/3572942424 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b054c97c00 con 0x55b054d944c0 2024-03-11T19:43:28.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.216+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/3572942424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d944c0 0x55b054daf0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc5680bc320 tx=0x7fc5680ecdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ce10966e0ddc23b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.217+0000 7fc563fff640 1 -- 172.21.15.83:0/3572942424 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5680bc160 con 0x55b054d944c0 2024-03-11T19:43:28.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.217+0000 7fc563fff640 1 -- 172.21.15.83:0/3572942424 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b054cf3310 con 0x55b054d944c0 2024-03-11T19:43:28.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.217+0000 7fc563fff640 1 -- 172.21.15.83:0/3572942424 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56802cb60 con 0x55b054d944c0 2024-03-11T19:43:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.218+0000 7fc583205740 1 -- 172.21.15.83:0/3572942424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d944c0 msgr2=0x55b054daf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.218+0000 7fc583205740 1 --2- 172.21.15.83:0/3572942424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d944c0 0x55b054daf0c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc5680bc320 tx=0x7fc5680ecdf0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.218+0000 7fc583205740 1 -- 172.21.15.83:0/3572942424 shutdown_connections 2024-03-11T19:43:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.218+0000 7fc583205740 1 --2- 172.21.15.83:0/3572942424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d96050 0x55b054d8e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.218+0000 7fc583205740 1 --2- 172.21.15.83:0/3572942424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d95690 0x55b054db6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.218+0000 7fc583205740 1 --2- 172.21.15.83:0/3572942424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d944c0 0x55b054daf0c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.218+0000 7fc583205740 1 -- 172.21.15.83:0/3572942424 >> 172.21.15.83:0/3572942424 conn(0x55b054d8f6a0 msgr2=0x55b054dae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.218+0000 7fc583205740 1 -- 172.21.15.83:0/3572942424 shutdown_connections 2024-03-11T19:43:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.218+0000 7fc583205740 1 -- 172.21.15.83:0/3572942424 wait complete. 2024-03-11T19:43:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.218+0000 7fc583205740 1 -- start start 2024-03-11T19:43:28.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.218+0000 7fc583205740 1 Processor -- start 2024-03-11T19:43:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc583205740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 0x55b054db6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc583205740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d95690 0x55b054db67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc583205740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d96050 0x55b054e12070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc583205740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b054e125b0 con 0x55b054d95690 2024-03-11T19:43:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc583205740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b054e126f0 con 0x55b054d96050 2024-03-11T19:43:28.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc583205740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b054e12880 con 0x55b054d944c0 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc583205740 10 client.0 _ll_register_callbacks cb 0x7fc57400bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc57eaf9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d96050 0x55b054e12070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc57e2f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 0x55b054db6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc57e2f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 0x55b054db6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:39878/0 (socket says 172.21.15.83:39878) 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc57eaf9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d96050 0x55b054e12070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56864/0 (socket says 172.21.15.83:56864) 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc57e2f8640 1 -- 172.21.15.83:0/2965970984 learned_addr learned my addr 172.21.15.83:0/2965970984 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.219+0000 7fc57daf7640 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d95690 0x55b054db67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc57e2f8640 1 -- 172.21.15.83:0/2965970984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d96050 msgr2=0x55b054e12070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d96050 0x55b054e12070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc57e2f8640 1 -- 172.21.15.83:0/2965970984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d95690 msgr2=0x55b054db67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d95690 0x55b054db67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc57e2f8640 1 -- 172.21.15.83:0/2965970984 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5781ae670 con 0x55b054d944c0 2024-03-11T19:43:28.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc57e2f8640 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 0x55b054db6260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc5680a5fd0 tx=0x7fc5680f2980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=371e1c09570db315 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc561ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc561ffb640 1 -- 172.21.15.83:0/2965970984 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5680c0320 con 0x55b054d944c0 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc561ffb640 1 -- 172.21.15.83:0/2965970984 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc568005390 con 0x55b054d944c0 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc583205740 1 -- 172.21.15.83:0/2965970984 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b054d22900 con 0x55b054d944c0 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc561ffb640 1 -- 172.21.15.83:0/2965970984 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5680ba4d0 con 0x55b054d944c0 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc583205740 10 client.25765 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.220+0000 7fc583205740 1 -- 172.21.15.83:0/2965970984 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b054cbd910 con 0x55b054d944c0 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.221+0000 7fc561ffb640 1 -- 172.21.15.83:0/2965970984 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b054d22900 con 0x55b054d944c0 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.221+0000 7fc561ffb640 1 -- 172.21.15.83:0/2965970984 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc5680c46a0 con 0x55b054d944c0 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.221+0000 7fc583205740 10 client.25765 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.221+0000 7fc583205740 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.221+0000 7fc583205740 20 client.25765 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.221+0000 7fc583205740 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b054e13de0 0x55b054e16200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.221+0000 7fc583205740 4 client.25765 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:43:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.221+0000 7fc583205740 1 -- 172.21.15.83:0/2965970984 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b0549fc860 con 0x55b054e13de0 2024-03-11T19:43:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.221+0000 7fc57daf7640 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b054e13de0 0x55b054e16200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.222+0000 7fc57daf7640 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b054e13de0 0x55b054e16200 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbab8edd09b6372b server_cookie=75ae44e2b1a1d393 in_seq=0 out_seq=0 2024-03-11T19:43:28.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.222+0000 7fc561ffb640 10 client.25765 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.223+0000 7fc561ffb640 1 -- 172.21.15.83:0/2965970984 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b0549fc860 con 0x55b054e13de0 2024-03-11T19:43:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.223+0000 7fc561ffb640 10 client.25765 handle_command_reply: tid=0 2024-03-11T19:43:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.223+0000 7fc583205740 1 client.25765 shutdown 2024-03-11T19:43:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc583205740 1 -- 172.21.15.83:0/2965970984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 msgr2=0x55b054db6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc583205740 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 0x55b054db6260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc5680a5fd0 tx=0x7fc5680f2980 comp rx=0 tx=0).stop 2024-03-11T19:43:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc583205740 1 -- 172.21.15.83:0/2965970984 shutdown_connections 2024-03-11T19:43:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc583205740 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b054e13de0 0x55b054e16200 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc583205740 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b054d96050 0x55b054e12070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc583205740 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b054d95690 0x55b054db67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc583205740 1 --2- 172.21.15.83:0/2965970984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b054d944c0 0x55b054db6260 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc583205740 1 -- 172.21.15.83:0/2965970984 >> 172.21.15.83:0/2965970984 conn(0x55b054d8f6a0 msgr2=0x55b054d92900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc561ffb640 0 client.25765 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc583205740 1 -- 172.21.15.83:0/2965970984 shutdown_connections 2024-03-11T19:43:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc583205740 1 -- 172.21.15.83:0/2965970984 wait complete. 2024-03-11T19:43:28.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.224+0000 7fc583205740 20 client.25765 trim_cache size 0 max 16384 2024-03-11T19:43:28.233 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:43:28.233 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:43:28.233 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "1736a5d3-8302-4598-9898-3dd79677ac8a", 2024-03-11T19:43:28.233 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:43:28.233 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:43:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 -- 172.21.15.83:0/3532644037 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5640a1b80 msgr2=0x7fc5640a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5640a1b80 0x7fc5640a4040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55b054d95590 tx=0x7fc570011040 comp rx=0 tx=0).stop 2024-03-11T19:43:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 -- 172.21.15.83:0/3532644037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578108180 msgr2=0x7fc5781a60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578108180 0x7fc5781a60d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc568000c00 tx=0x7fc568034670 comp rx=0 tx=0).stop 2024-03-11T19:43:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 -- 172.21.15.83:0/3532644037 shutdown_connections 2024-03-11T19:43:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5640a1b80 0x7fc5640a4040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc57810d4a0 0x7fc5781ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc57810aaf0 0x7fc5781a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 --2- 172.21.15.83:0/3532644037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578108180 0x7fc5781a60d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 -- 172.21.15.83:0/3532644037 >> 172.21.15.83:0/3532644037 conn(0x7fc57806dfe0 msgr2=0x7fc57810af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 -- 172.21.15.83:0/3532644037 shutdown_connections 2024-03-11T19:43:28.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.225+0000 7fc58055c640 1 -- 172.21.15.83:0/3532644037 wait complete. 2024-03-11T19:43:28.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:43:28.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:43:28.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.403+0000 7f93511b6640 1 -- 172.21.15.83:0/2526528352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f934c101270 msgr2=0x7f934c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.403+0000 7f93511b6640 1 --2- 172.21.15.83:0/2526528352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f934c101270 0x7f934c1016f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9340009900 tx=0x7f934002bc30 comp rx=0 tx=0).stop 2024-03-11T19:43:28.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.403+0000 7f93511b6640 1 -- 172.21.15.83:0/2526528352 shutdown_connections 2024-03-11T19:43:28.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.403+0000 7f93511b6640 1 --2- 172.21.15.83:0/2526528352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f934c101c30 0x7f934c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.403+0000 7f93511b6640 1 --2- 172.21.15.83:0/2526528352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f934c101270 0x7f934c1016f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.403+0000 7f93511b6640 1 --2- 172.21.15.83:0/2526528352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f934c100080 0x7f934c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.404+0000 7f93511b6640 1 -- 172.21.15.83:0/2526528352 >> 172.21.15.83:0/2526528352 conn(0x7f934c0fb5f0 msgr2=0x7f934c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.404+0000 7f93511b6640 1 -- 172.21.15.83:0/2526528352 shutdown_connections 2024-03-11T19:43:28.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.404+0000 7f93511b6640 1 -- 172.21.15.83:0/2526528352 wait complete. 2024-03-11T19:43:28.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.404+0000 7f93511b6640 1 Processor -- start 2024-03-11T19:43:28.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.405+0000 7f93511b6640 1 -- start start 2024-03-11T19:43:28.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.405+0000 7f93511b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f934c100080 0x7f934c195010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.405+0000 7f93511b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f934c101270 0x7f934c195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.406+0000 7f934ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f934c100080 0x7f934c195010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.406+0000 7f934ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f934c100080 0x7f934c195010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56880/0 (socket says 172.21.15.83:56880) 2024-03-11T19:43:28.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.406+0000 7f934ad9d640 1 -- 172.21.15.83:0/4158893451 learned_addr learned my addr 172.21.15.83:0/4158893451 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:28.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.406+0000 7f93511b6640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f934c101c30 0x7f934c19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.406+0000 7f934a59c640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f934c101270 0x7f934c195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.406+0000 7f93511b6640 1 -- 172.21.15.83:0/4158893451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c19cb00 con 0x7f934c101c30 2024-03-11T19:43:28.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.406+0000 7f934b59e640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f934c101c30 0x7f934c19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.406+0000 7f93511b6640 1 -- 172.21.15.83:0/4158893451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c19cc70 con 0x7f934c100080 2024-03-11T19:43:28.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.406+0000 7f93511b6640 1 -- 172.21.15.83:0/4158893451 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c19ce30 con 0x7f934c101270 2024-03-11T19:43:28.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f934a59c640 1 -- 172.21.15.83:0/4158893451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f934c100080 msgr2=0x7f934c195010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f934a59c640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f934c100080 0x7f934c195010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f934a59c640 1 -- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f934c101c30 msgr2=0x7f934c19c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f934a59c640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f934c101c30 0x7f934c19c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f934a59c640 1 -- 172.21.15.83:0/4158893451 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344009840 con 0x7f934c101270 2024-03-11T19:43:28.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f934b59e640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f934c101c30 0x7f934c19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:28.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f934a59c640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f934c101270 0x7f934c195550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f934002c170 tx=0x7f9340005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f9333fff640 1 -- 172.21.15.83:0/4158893451 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93400043d0 con 0x7f934c101270 2024-03-11T19:43:28.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f9333fff640 1 -- 172.21.15.83:0/4158893451 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9340042070 con 0x7f934c101270 2024-03-11T19:43:28.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f93511b6640 1 -- 172.21.15.83:0/4158893451 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9340009560 con 0x7f934c101270 2024-03-11T19:43:28.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f9333fff640 1 -- 172.21.15.83:0/4158893451 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93400052c0 con 0x7f934c101270 2024-03-11T19:43:28.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.407+0000 7f93511b6640 1 -- 172.21.15.83:0/4158893451 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c19d450 con 0x7f934c101270 2024-03-11T19:43:28.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.411+0000 7f9353d86740 1 Processor -- start 2024-03-11T19:43:28.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.411+0000 7f9333fff640 1 -- 172.21.15.83:0/4158893451 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f934003ec30 con 0x7f934c101270 2024-03-11T19:43:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.411+0000 7f9353d86740 1 -- start start 2024-03-11T19:43:28.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.412+0000 7f9353d86740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd716ae0 0x55affd716f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.412+0000 7f934b59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd716ae0 0x55affd716f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.412+0000 7f934b59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd716ae0 0x55affd716f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56894/0 (socket says 172.21.15.83:56894) 2024-03-11T19:43:28.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.412+0000 7f9333fff640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f93340a1b30 0x7f93340a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.412+0000 7f9353d86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd7174d0 0x55affd71bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.412+0000 7f9353d86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd71c640 0x55affd71ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.412+0000 7f9353d86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd67e280 con 0x55affd71c640 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.412+0000 7f9353d86740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd67b7b0 con 0x55affd716ae0 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.412+0000 7f9353d86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd676f90 con 0x55affd7174d0 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.412+0000 7f934b59e640 1 -- 172.21.15.83:0/77663861 learned_addr learned my addr 172.21.15.83:0/77663861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.412+0000 7f934a59c640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f93340a1b30 0x7f93340a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f934b59e640 1 --2- 172.21.15.83:0/77663861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd71c640 0x55affd71ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f934b59e640 1 -- 172.21.15.83:0/77663861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd7174d0 msgr2=0x55affd71bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f934b59e640 1 --2- 172.21.15.83:0/77663861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd7174d0 0x55affd71bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f934b59e640 1 -- 172.21.15.83:0/77663861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd71c640 msgr2=0x55affd71ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f934b59e640 1 --2- 172.21.15.83:0/77663861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd71c640 0x55affd71ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f934b59e640 1 -- 172.21.15.83:0/77663861 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55affd635570 con 0x55affd716ae0 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f934b59e640 1 --2- 172.21.15.83:0/77663861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd716ae0 0x55affd716f00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f934400cc80 tx=0x7f9344009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9318f316499d2a6f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f9333fff640 1 -- 172.21.15.83:0/4158893451 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9340044070 con 0x7f934c101270 2024-03-11T19:43:28.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f934a59c640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f93340a1b30 0x7f93340a3ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f93400bfe20 tx=0x7f93400c3040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f93317fa640 1 -- 172.21.15.83:0/77663861 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344012070 con 0x55affd716ae0 2024-03-11T19:43:28.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f93317fa640 1 -- 172.21.15.83:0/77663861 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f934400abe0 con 0x55affd716ae0 2024-03-11T19:43:28.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.413+0000 7f93317fa640 1 -- 172.21.15.83:0/77663861 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440056e0 con 0x55affd716ae0 2024-03-11T19:43:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.414+0000 7f9353d86740 1 -- 172.21.15.83:0/77663861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd716ae0 msgr2=0x55affd716f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.414+0000 7f9353d86740 1 --2- 172.21.15.83:0/77663861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd716ae0 0x55affd716f00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f934400cc80 tx=0x7f9344009500 comp rx=0 tx=0).stop 2024-03-11T19:43:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.414+0000 7f9353d86740 1 -- 172.21.15.83:0/77663861 shutdown_connections 2024-03-11T19:43:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.414+0000 7f9353d86740 1 --2- 172.21.15.83:0/77663861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd71c640 0x55affd71ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.414+0000 7f9353d86740 1 --2- 172.21.15.83:0/77663861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd7174d0 0x55affd71bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.414+0000 7f9353d86740 1 --2- 172.21.15.83:0/77663861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd716ae0 0x55affd716f00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.414+0000 7f9353d86740 1 -- 172.21.15.83:0/77663861 >> 172.21.15.83:0/77663861 conn(0x55affd7097f0 msgr2=0x55affd709bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.414+0000 7f9353d86740 1 -- 172.21.15.83:0/77663861 shutdown_connections 2024-03-11T19:43:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.414+0000 7f9353d86740 1 -- 172.21.15.83:0/77663861 wait complete. 2024-03-11T19:43:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.414+0000 7f9353d86740 1 -- start start 2024-03-11T19:43:28.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.415+0000 7f9353d86740 1 Processor -- start 2024-03-11T19:43:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.415+0000 7f9353d86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd716ae0 0x55affd720b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.415+0000 7f9353d86740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 0x55affd721050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.415+0000 7f9353d86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd71c640 0x55affd728090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.415+0000 7f9353d86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd34a430 con 0x55affd71c640 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.415+0000 7f9353d86740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd306e00 con 0x55affd7174d0 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.415+0000 7f9353d86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd5fbfd0 con 0x55affd716ae0 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.415+0000 7f9353d86740 10 client.0 _ll_register_callbacks cb 0x55affd651fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.415+0000 7f934ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 0x55affd721050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.415+0000 7f934ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 0x55affd721050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:56910/0 (socket says 172.21.15.83:56910) 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.416+0000 7f934ad9d640 1 -- 172.21.15.83:0/4157074516 learned_addr learned my addr 172.21.15.83:0/4157074516 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.416+0000 7f934b59e640 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd716ae0 0x55affd720b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.416+0000 7f934b59e640 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd71c640 0x55affd728090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.416+0000 7f934ad9d640 1 -- 172.21.15.83:0/4157074516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd716ae0 msgr2=0x55affd720b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.416+0000 7f934ad9d640 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd716ae0 0x55affd720b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.416+0000 7f934ad9d640 1 -- 172.21.15.83:0/4157074516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd71c640 msgr2=0x55affd728090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.416+0000 7f934ad9d640 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd71c640 0x55affd728090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.416+0000 7f934ad9d640 1 -- 172.21.15.83:0/4157074516 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934400c830 con 0x55affd7174d0 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.416+0000 7f934ad9d640 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 0x55affd721050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f933c0116f0 tx=0x7f933c011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fefb07b429856f5c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.417+0000 7f932eff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.417+0000 7f932eff5640 1 -- 172.21.15.83:0/4157074516 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00fd30 con 0x55affd7174d0 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.417+0000 7f932eff5640 1 -- 172.21.15.83:0/4157074516 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f933c004030 con 0x55affd7174d0 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.417+0000 7f9353d86740 1 -- 172.21.15.83:0/4157074516 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55affd3767e0 con 0x55affd7174d0 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.417+0000 7f932eff5640 1 -- 172.21.15.83:0/4157074516 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f933c00fe90 con 0x55affd7174d0 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.418+0000 7f9353d86740 10 client.25679 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.418+0000 7f9353d86740 1 -- 172.21.15.83:0/4157074516 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55affd675610 con 0x55affd7174d0 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.418+0000 7f932eff5640 1 -- 172.21.15.83:0/4157074516 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55affd3767e0 con 0x55affd7174d0 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.418+0000 7f932eff5640 1 -- 172.21.15.83:0/4157074516 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f933c019500 con 0x55affd7174d0 2024-03-11T19:43:28.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.418+0000 7f9353d86740 10 client.25679 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.418+0000 7f9353d86740 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.418+0000 7f9353d86740 20 client.25679 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.418+0000 7f9353d86740 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55affd712750 0x55affd714b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.418+0000 7f9353d86740 4 client.25679 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.418+0000 7f9353d86740 1 -- 172.21.15.83:0/4157074516 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55affd3b3720 con 0x55affd712750 2024-03-11T19:43:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.419+0000 7f934b59e640 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55affd712750 0x55affd714b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.419+0000 7f934b59e640 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55affd712750 0x55affd714b70 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15536614d7589899 server_cookie=3ffcaa44ca2b4be1 in_seq=0 out_seq=0 2024-03-11T19:43:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.419+0000 7f932eff5640 10 client.25679 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.420+0000 7f932eff5640 1 -- 172.21.15.83:0/4157074516 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55affd3b3720 con 0x55affd712750 2024-03-11T19:43:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.420+0000 7f932eff5640 10 client.25679 handle_command_reply: tid=0 2024-03-11T19:43:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.420+0000 7f9353d86740 1 client.25679 shutdown 2024-03-11T19:43:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.421+0000 7f9353d86740 1 -- 172.21.15.83:0/4157074516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 msgr2=0x55affd721050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.421+0000 7f9353d86740 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 0x55affd721050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f933c0116f0 tx=0x7f933c011bb0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.421+0000 7f9353d86740 1 -- 172.21.15.83:0/4157074516 shutdown_connections 2024-03-11T19:43:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.421+0000 7f9353d86740 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55affd712750 0x55affd714b70 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.421+0000 7f9353d86740 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd71c640 0x55affd728090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.421+0000 7f9353d86740 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 0x55affd721050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.421+0000 7f9353d86740 1 --2- 172.21.15.83:0/4157074516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd716ae0 0x55affd720b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.421+0000 7f9353d86740 1 -- 172.21.15.83:0/4157074516 >> 172.21.15.83:0/4157074516 conn(0x55affd7097f0 msgr2=0x55affd708a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.421+0000 7f932eff5640 0 client.25679 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.421+0000 7f9353d86740 1 -- 172.21.15.83:0/4157074516 shutdown_connections 2024-03-11T19:43:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.421+0000 7f9353d86740 1 -- 172.21.15.83:0/4157074516 wait complete. 2024-03-11T19:43:28.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.422+0000 7f9353d86740 20 client.25679 trim_cache size 0 max 16384 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.446+0000 7f9353d86740 1 Processor -- start 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.446+0000 7f9353d86740 1 -- start start 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f9353d86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd716ae0 0x55affd7187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f9353d86740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 0x55affd728050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f9353d86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd71c640 0x55affd718d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f9353d86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd3b3720 con 0x55affd716ae0 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f9353d86740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd3767e0 con 0x55affd7174d0 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f9353d86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd717bc0 con 0x55affd71c640 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f934b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd716ae0 0x55affd7187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f934b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd716ae0 0x55affd7187f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33408/0 (socket says 172.21.15.83:33408) 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f934b59e640 1 -- 172.21.15.83:0/3686933029 learned_addr learned my addr 172.21.15.83:0/3686933029 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f934ad9d640 1 --2- 172.21.15.83:0/3686933029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 0x55affd728050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f934b59e640 1 --2- 172.21.15.83:0/3686933029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd71c640 0x55affd718d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f934b59e640 1 -- 172.21.15.83:0/3686933029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd71c640 msgr2=0x55affd718d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f934b59e640 1 --2- 172.21.15.83:0/3686933029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd71c640 0x55affd718d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f934b59e640 1 -- 172.21.15.83:0/3686933029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 msgr2=0x55affd728050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f934b59e640 1 --2- 172.21.15.83:0/3686933029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 0x55affd728050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f934b59e640 1 -- 172.21.15.83:0/3686933029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55affd611ac0 con 0x55affd716ae0 2024-03-11T19:43:28.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.447+0000 7f934b59e640 1 --2- 172.21.15.83:0/3686933029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd716ae0 0x55affd7187f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9344005100 tx=0x7f934400c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300bbc48e4be7caf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.448+0000 7f9330ff9640 1 -- 172.21.15.83:0/3686933029 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344020040 con 0x55affd716ae0 2024-03-11T19:43:28.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.448+0000 7f9330ff9640 1 -- 172.21.15.83:0/3686933029 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9344003970 con 0x55affd716ae0 2024-03-11T19:43:28.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.448+0000 7f9330ff9640 1 -- 172.21.15.83:0/3686933029 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344012040 con 0x55affd716ae0 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.449+0000 7f9353d86740 1 -- 172.21.15.83:0/3686933029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd716ae0 msgr2=0x55affd7187f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.449+0000 7f9353d86740 1 --2- 172.21.15.83:0/3686933029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd716ae0 0x55affd7187f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9344005100 tx=0x7f934400c400 comp rx=0 tx=0).stop 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.449+0000 7f9353d86740 1 -- 172.21.15.83:0/3686933029 shutdown_connections 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.449+0000 7f9353d86740 1 --2- 172.21.15.83:0/3686933029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd71c640 0x55affd718d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.449+0000 7f9353d86740 1 --2- 172.21.15.83:0/3686933029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd7174d0 0x55affd728050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.449+0000 7f9353d86740 1 --2- 172.21.15.83:0/3686933029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd716ae0 0x55affd7187f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.449+0000 7f9353d86740 1 -- 172.21.15.83:0/3686933029 >> 172.21.15.83:0/3686933029 conn(0x55affd7097f0 msgr2=0x55affd73d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.449+0000 7f9353d86740 1 -- 172.21.15.83:0/3686933029 shutdown_connections 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.449+0000 7f9353d86740 1 -- 172.21.15.83:0/3686933029 wait complete. 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.450+0000 7f9353d86740 1 -- start start 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.450+0000 7f9353d86740 1 Processor -- start 2024-03-11T19:43:28.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.450+0000 7f9353d86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd716ae0 0x55affd725110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.451+0000 7f9353d86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd7174d0 0x55affd725650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.451+0000 7f9353d86740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd71c640 0x55affd726660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.451+0000 7f9353d86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd726ba0 con 0x55affd7174d0 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.451+0000 7f934ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd7174d0 0x55affd725650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.451+0000 7f934ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd7174d0 0x55affd725650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33420/0 (socket says 172.21.15.83:33420) 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.451+0000 7f934ad9d640 1 -- 172.21.15.83:0/261720895 learned_addr learned my addr 172.21.15.83:0/261720895 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.451+0000 7f9353d86740 1 -- 172.21.15.83:0/261720895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd726d10 con 0x55affd71c640 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.451+0000 7f9353d86740 1 -- 172.21.15.83:0/261720895 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd726ed0 con 0x55affd716ae0 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.451+0000 7f9353d86740 10 client.0 _ll_register_callbacks cb 0x7f934400c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.452+0000 7f934b59e640 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd716ae0 0x55affd725110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.452+0000 7f934ad9d640 1 -- 172.21.15.83:0/261720895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd716ae0 msgr2=0x55affd725110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.452+0000 7f934ad9d640 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd716ae0 0x55affd725110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.452+0000 7f934ad9d640 1 -- 172.21.15.83:0/261720895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd71c640 msgr2=0x55affd726660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:28.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.452+0000 7f934ad9d640 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd71c640 0x55affd726660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.452+0000 7f934ad9d640 1 -- 172.21.15.83:0/261720895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55affd635570 con 0x55affd7174d0 2024-03-11T19:43:28.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.452+0000 7f934ad9d640 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd7174d0 0x55affd725650 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f933c009230 tx=0x7f933c005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e1439226b39e5b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:28.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.452+0000 7f932eff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:43:28.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.453+0000 7f932eff5640 1 -- 172.21.15.83:0/261720895 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c019ad0 con 0x55affd7174d0 2024-03-11T19:43:28.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.453+0000 7f932eff5640 1 -- 172.21.15.83:0/261720895 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f933c019c30 con 0x55affd7174d0 2024-03-11T19:43:28.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.453+0000 7f9353d86740 1 -- 172.21.15.83:0/261720895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f934400abe0 con 0x55affd7174d0 2024-03-11T19:43:28.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.454+0000 7f932eff5640 1 -- 172.21.15.83:0/261720895 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f933c019df0 con 0x55affd7174d0 2024-03-11T19:43:28.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.454+0000 7f9353d86740 10 client.16200 fetch_fsmap learned FSMap version 28 2024-03-11T19:43:28.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.454+0000 7f9353d86740 1 -- 172.21.15.83:0/261720895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55affd6376a0 con 0x55affd7174d0 2024-03-11T19:43:28.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.455+0000 7f932eff5640 1 -- 172.21.15.83:0/261720895 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f933c005520 con 0x55affd7174d0 2024-03-11T19:43:28.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.455+0000 7f932eff5640 1 -- 172.21.15.83:0/261720895 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f933c034070 con 0x55affd7174d0 2024-03-11T19:43:28.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.455+0000 7f9353d86740 10 client.16200 fetch_fsmap finished waiting for FSMap version 28 2024-03-11T19:43:28.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.455+0000 7f9353d86740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:28.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.455+0000 7f9353d86740 20 client.16200 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:28.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.455+0000 7f9353d86740 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55affd78d060 0x55affd78f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:28.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.455+0000 7f9353d86740 4 client.16200 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.455+0000 7f9353d86740 1 -- 172.21.15.83:0/261720895 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f934400abe0 con 0x55affd78d060 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.456+0000 7f934b59e640 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55affd78d060 0x55affd78f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.456+0000 7f932eff5640 10 client.16200 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.456+0000 7f934b59e640 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55affd78d060 0x55affd78f480 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a811936b96fee3 server_cookie=9e031e4d456ad887 in_seq=0 out_seq=0 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.456+0000 7f932eff5640 1 -- 172.21.15.83:0/261720895 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f934400abe0 con 0x55affd78d060 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.456+0000 7f932eff5640 10 client.16200 handle_command_reply: tid=0 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.456+0000 7f9353d86740 1 client.16200 shutdown 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.456+0000 7f9353d86740 1 -- 172.21.15.83:0/261720895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd7174d0 msgr2=0x55affd725650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.456+0000 7f9353d86740 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd7174d0 0x55affd725650 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f933c009230 tx=0x7f933c005f80 comp rx=0 tx=0).stop 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.457+0000 7f9353d86740 1 -- 172.21.15.83:0/261720895 shutdown_connections 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.457+0000 7f9353d86740 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55affd78d060 0x55affd78f480 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.457+0000 7f9353d86740 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55affd71c640 0x55affd726660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.457+0000 7f9353d86740 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55affd7174d0 0x55affd725650 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.457+0000 7f9353d86740 1 --2- 172.21.15.83:0/261720895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55affd716ae0 0x55affd725110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.457+0000 7f9353d86740 1 -- 172.21.15.83:0/261720895 >> 172.21.15.83:0/261720895 conn(0x55affd7097f0 msgr2=0x55affd7187f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.457+0000 7f932eff5640 0 client.16200 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:28.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.457+0000 7f9353d86740 1 -- 172.21.15.83:0/261720895 shutdown_connections 2024-03-11T19:43:28.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.457+0000 7f9353d86740 1 -- 172.21.15.83:0/261720895 wait complete. 2024-03-11T19:43:28.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.457+0000 7f9353d86740 20 client.16200 trim_cache size 0 max 16384 2024-03-11T19:43:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:43:28.464 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:43:28.464 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:43:28.464 INFO:teuthology.orchestra.run.smithi083.stdout: "1736a5d3-8302-4598-9898-3dd79677ac8a": { 2024-03-11T19:43:28.464 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:43:28.465 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "1736a5d3-8302-4598-9898-3dd79677ac8a", 2024-03-11T19:43:28.465 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:43:28.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:43:28.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:43:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:43:28.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.459+0000 7f93511b6640 1 -- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f93340a1b30 msgr2=0x7f93340a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.459+0000 7f93511b6640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f93340a1b30 0x7f93340a3ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f93400bfe20 tx=0x7f93400c3040 comp rx=0 tx=0).stop 2024-03-11T19:43:28.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.459+0000 7f93511b6640 1 -- 172.21.15.83:0/4158893451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f934c101270 msgr2=0x7f934c195550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:28.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.459+0000 7f93511b6640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f934c101270 0x7f934c195550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f934002c170 tx=0x7f9340005e80 comp rx=0 tx=0).stop 2024-03-11T19:43:28.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.460+0000 7f93511b6640 1 -- 172.21.15.83:0/4158893451 shutdown_connections 2024-03-11T19:43:28.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.460+0000 7f93511b6640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f93340a1b30 0x7f93340a3ff0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.460+0000 7f93511b6640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f934c101c30 0x7f934c19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.460+0000 7f93511b6640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f934c101270 0x7f934c195550 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.460+0000 7f93511b6640 1 --2- 172.21.15.83:0/4158893451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f934c100080 0x7f934c195010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:28.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.460+0000 7f93511b6640 1 -- 172.21.15.83:0/4158893451 >> 172.21.15.83:0/4158893451 conn(0x7f934c0fb5f0 msgr2=0x7f934c104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:28.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.460+0000 7f93511b6640 1 -- 172.21.15.83:0/4158893451 shutdown_connections 2024-03-11T19:43:28.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:28.460+0000 7f93511b6640 1 -- 172.21.15.83:0/4158893451 wait complete. 2024-03-11T19:43:28.481 INFO:tasks.cephfs.filesystem:scrub status for tag:1736a5d3-8302-4598-9898-3dd79677ac8a - {'path': '/', 'tag': '1736a5d3-8302-4598-9898-3dd79677ac8a', 'options': 'recursive,force'} 2024-03-11T19:43:28.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:28 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:28.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:28 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:28.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:28 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:43:29.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:43:29.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:43:29.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:43:29.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:43:29.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:43:29.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 898 op/s 2024-03-11T19:43:29.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:29.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:29.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:43:29.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 898 op/s 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:29.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:29.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:43:29.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:43:29.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:43:29.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:43:29.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:43:29.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:43:29.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:43:29.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: pgmap v477: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 898 op/s 2024-03-11T19:43:29.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:43:29.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:43:29.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:43:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:30 smithi083 ceph-mon[31864]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 844 op/s 2024-03-11T19:43:31.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:30 smithi137 ceph-mon[36220]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 844 op/s 2024-03-11T19:43:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:30 smithi204 ceph-mon[36739]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 844 op/s 2024-03-11T19:43:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 255.0 s, 2989.4 tps, lat 33.551 ms stddev 35.414 2024-03-11T19:43:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:32 smithi083 ceph-mon[31864]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 843 op/s 2024-03-11T19:43:33.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:32 smithi137 ceph-mon[36220]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 843 op/s 2024-03-11T19:43:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:32 smithi204 ceph-mon[36739]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 843 op/s 2024-03-11T19:43:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:34 smithi083 ceph-mon[31864]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 MiB/s wr, 862 op/s 2024-03-11T19:43:35.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:34 smithi137 ceph-mon[36220]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 MiB/s wr, 862 op/s 2024-03-11T19:43:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:34 smithi204 ceph-mon[36739]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 MiB/s wr, 862 op/s 2024-03-11T19:43:36.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:35 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:43:36.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:35 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:43:36.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:35 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:43:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:36 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:43:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:36 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:43:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:36 smithi083 ceph-mon[31864]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-03-11T19:43:37.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:36 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:43:37.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:36 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:43:37.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:36 smithi137 ceph-mon[36220]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-03-11T19:43:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:36 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:43:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:36 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:43:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:36 smithi204 ceph-mon[36739]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-03-11T19:43:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 260.0 s, 4585.4 tps, lat 21.719 ms stddev 25.298 2024-03-11T19:43:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:38 smithi083 ceph-mon[31864]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 846 op/s 2024-03-11T19:43:39.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:38 smithi137 ceph-mon[36220]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 846 op/s 2024-03-11T19:43:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:38 smithi204 ceph-mon[36739]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 846 op/s 2024-03-11T19:43:41.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:40 smithi083 ceph-mon[31864]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.5 MiB/s wr, 781 op/s 2024-03-11T19:43:41.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:40 smithi137 ceph-mon[36220]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.5 MiB/s wr, 781 op/s 2024-03-11T19:43:41.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:40 smithi204 ceph-mon[36739]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.5 MiB/s wr, 781 op/s 2024-03-11T19:43:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 265.0 s, 4244.8 tps, lat 23.678 ms stddev 26.522 2024-03-11T19:43:43.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:43 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:43:43.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:43 smithi083 ceph-mon[31864]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9.0 MiB/s wr, 770 op/s 2024-03-11T19:43:43.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:43 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:43:43.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:43 smithi137 ceph-mon[36220]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9.0 MiB/s wr, 770 op/s 2024-03-11T19:43:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:43 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:43:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:43 smithi204 ceph-mon[36739]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9.0 MiB/s wr, 770 op/s 2024-03-11T19:43:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:44 smithi083 ceph-mon[31864]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9.2 MiB/s wr, 802 op/s 2024-03-11T19:43:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:44 smithi137 ceph-mon[36220]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9.2 MiB/s wr, 802 op/s 2024-03-11T19:43:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:44 smithi204 ceph-mon[36739]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9.2 MiB/s wr, 802 op/s 2024-03-11T19:43:47.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:46 smithi083 ceph-mon[31864]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 10 MiB/s wr, 851 op/s 2024-03-11T19:43:47.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:46 smithi137 ceph-mon[36220]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 10 MiB/s wr, 851 op/s 2024-03-11T19:43:47.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:46 smithi204 ceph-mon[36739]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 10 MiB/s wr, 851 op/s 2024-03-11T19:43:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 270.0 s, 3088.6 tps, lat 32.245 ms stddev 36.386 2024-03-11T19:43:49.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:48 smithi083 ceph-mon[31864]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.8 MiB/s wr, 858 op/s 2024-03-11T19:43:49.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:48 smithi137 ceph-mon[36220]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.8 MiB/s wr, 858 op/s 2024-03-11T19:43:49.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:48 smithi204 ceph-mon[36739]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.8 MiB/s wr, 858 op/s 2024-03-11T19:43:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:50 smithi083 ceph-mon[31864]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 MiB/s wr, 755 op/s 2024-03-11T19:43:51.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:50 smithi137 ceph-mon[36220]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 MiB/s wr, 755 op/s 2024-03-11T19:43:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:50 smithi204 ceph-mon[36739]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 MiB/s wr, 755 op/s 2024-03-11T19:43:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 275.0 s, 3360.2 tps, lat 29.787 ms stddev 35.756 2024-03-11T19:43:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:52 smithi083 ceph-mon[31864]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 MiB/s wr, 698 op/s 2024-03-11T19:43:53.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:52 smithi137 ceph-mon[36220]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 MiB/s wr, 698 op/s 2024-03-11T19:43:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:52 smithi204 ceph-mon[36739]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 7.3 MiB/s wr, 698 op/s 2024-03-11T19:43:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:54 smithi083 ceph-mon[31864]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.0 MiB/s wr, 747 op/s 2024-03-11T19:43:55.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:54 smithi137 ceph-mon[36220]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.0 MiB/s wr, 747 op/s 2024-03-11T19:43:55.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:54 smithi204 ceph-mon[36739]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.0 MiB/s wr, 747 op/s 2024-03-11T19:43:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:56 smithi083 ceph-mon[31864]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 MiB/s wr, 803 op/s 2024-03-11T19:43:57.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:56 smithi137 ceph-mon[36220]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 MiB/s wr, 803 op/s 2024-03-11T19:43:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:56 smithi204 ceph-mon[36739]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 MiB/s wr, 803 op/s 2024-03-11T19:43:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 280.0 s, 3632.5 tps, lat 27.556 ms stddev 30.111 2024-03-11T19:43:58.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:43:58.482 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:43:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.644+0000 7fc6fd0df640 1 -- 172.21.15.83:0/4143322079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6f81011c0 msgr2=0x7fc6f81015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.644+0000 7fc6fd0df640 1 --2- 172.21.15.83:0/4143322079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6f81011c0 0x7fc6f81015e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009900 tx=0x7fc6e802bc30 comp rx=0 tx=0).stop 2024-03-11T19:43:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.645+0000 7fc6fd0df640 1 -- 172.21.15.83:0/4143322079 shutdown_connections 2024-03-11T19:43:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.645+0000 7fc6fd0df640 1 --2- 172.21.15.83:0/4143322079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f8102d70 0x7fc6f8109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.645+0000 7fc6fd0df640 1 --2- 172.21.15.83:0/4143322079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f81023b0 0x7fc6f8102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.645+0000 7fc6fd0df640 1 --2- 172.21.15.83:0/4143322079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6f81011c0 0x7fc6f81015e0 secure :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009900 tx=0x7fc6e802bc30 comp rx=0 tx=0).stop 2024-03-11T19:43:58.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.645+0000 7fc6fd0df640 1 -- 172.21.15.83:0/4143322079 >> 172.21.15.83:0/4143322079 conn(0x7fc6f80fc710 msgr2=0x7fc6f80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:58.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.645+0000 7fc6fd0df640 1 -- 172.21.15.83:0/4143322079 shutdown_connections 2024-03-11T19:43:58.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.645+0000 7fc6fd0df640 1 -- 172.21.15.83:0/4143322079 wait complete. 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6fd0df640 1 Processor -- start 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6fd0df640 1 -- start start 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6fd0df640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f81023b0 0x7fc6f81973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6fd0df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f8102d70 0x7fc6f81978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6fd0df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6f819c950 0x7fc6f819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6fd0df640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f819f280 con 0x7fc6f819c950 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6fd0df640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f819f3f0 con 0x7fc6f8102d70 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6fd0df640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f819f5b0 con 0x7fc6f81023b0 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6f759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6f819c950 0x7fc6f819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6f659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f8102d70 0x7fc6f81978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6f659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f8102d70 0x7fc6f81978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:57136/0 (socket says 172.21.15.83:57136) 2024-03-11T19:43:58.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6f759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6f819c950 0x7fc6f819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53208/0 (socket says 172.21.15.83:53208) 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6f659c640 1 -- 172.21.15.83:0/2943995559 learned_addr learned my addr 172.21.15.83:0/2943995559 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.646+0000 7fc6f6d9d640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f81023b0 0x7fc6f81973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.647+0000 7fc6f659c640 1 -- 172.21.15.83:0/2943995559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f81023b0 msgr2=0x7fc6f81973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.647+0000 7fc6f659c640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f81023b0 0x7fc6f81973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.647+0000 7fc6f659c640 1 -- 172.21.15.83:0/2943995559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6f819c950 msgr2=0x7fc6f819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.647+0000 7fc6f659c640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6f819c950 0x7fc6f819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.647+0000 7fc6f659c640 1 -- 172.21.15.83:0/2943995559 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8009560 con 0x7fc6f8102d70 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.647+0000 7fc6f659c640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f8102d70 0x7fc6f81978f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00c920 tx=0x7fc6ec00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.647+0000 7fc6dffff640 1 -- 172.21.15.83:0/2943995559 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec007c70 con 0x7fc6f8102d70 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.647+0000 7fc6fd0df640 1 -- 172.21.15.83:0/2943995559 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f8073410 con 0x7fc6f8102d70 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.647+0000 7fc6dffff640 1 -- 172.21.15.83:0/2943995559 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6ec007dd0 con 0x7fc6f8102d70 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.647+0000 7fc6fd0df640 1 -- 172.21.15.83:0/2943995559 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f8073970 con 0x7fc6f8102d70 2024-03-11T19:43:58.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.647+0000 7fc6dffff640 1 -- 172.21.15.83:0/2943995559 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec018860 con 0x7fc6f8102d70 2024-03-11T19:43:58.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.650+0000 7fc6dffff640 1 -- 172.21.15.83:0/2943995559 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc6ec00f450 con 0x7fc6f8102d70 2024-03-11T19:43:58.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6ffd93740 1 Processor -- start 2024-03-11T19:43:58.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6ffd93740 1 -- start start 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d489ae0 0x56530d489f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48a4d0 0x56530d48efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6dffff640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6e00a1ab0 0x7fc6e00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 0x56530d491ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6ffd93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d3f1280 con 0x56530d489ae0 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6ffd93740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d3ee7b0 con 0x56530d48a4d0 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6ffd93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d3e9f90 con 0x56530d48f640 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6dffff640 1 -- 172.21.15.83:0/2943995559 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc6ec0cd3b0 con 0x7fc6f8102d70 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6f6d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48a4d0 0x56530d48efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6f6d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48a4d0 0x56530d48efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:57146/0 (socket says 172.21.15.83:57146) 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6f6d9d640 1 -- 172.21.15.83:0/2331549505 learned_addr learned my addr 172.21.15.83:0/2331549505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:58.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.651+0000 7fc6f759e640 1 --2- 172.21.15.83:0/2331549505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 0x56530d491ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.652+0000 7fc6f759e640 1 -- 172.21.15.83:0/2331549505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48a4d0 msgr2=0x56530d48efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.652+0000 7fc6f759e640 1 --2- 172.21.15.83:0/2331549505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48a4d0 0x56530d48efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.652+0000 7fc6f759e640 1 -- 172.21.15.83:0/2331549505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d489ae0 msgr2=0x56530d489f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.652+0000 7fc6f759e640 1 --2- 172.21.15.83:0/2331549505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d489ae0 0x56530d489f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.652+0000 7fc6f759e640 1 -- 172.21.15.83:0/2331549505 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56530d3a8570 con 0x56530d48f640 2024-03-11T19:43:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.652+0000 7fc6f759e640 1 --2- 172.21.15.83:0/2331549505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 0x56530d491ad0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0005c40 tx=0x7fc6f0010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=297013b4d47bbd7a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.653+0000 7fc6f659c640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6e00a1ab0 0x7fc6e00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.653+0000 7fc6dd7fa640 1 -- 172.21.15.83:0/2331549505 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0015040 con 0x56530d48f640 2024-03-11T19:43:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.653+0000 7fc6dd7fa640 1 -- 172.21.15.83:0/2331549505 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6f0003e50 con 0x56530d48f640 2024-03-11T19:43:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.653+0000 7fc6dd7fa640 1 -- 172.21.15.83:0/2331549505 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0005020 con 0x56530d48f640 2024-03-11T19:43:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.654+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2331549505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 msgr2=0x56530d491ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.654+0000 7fc6f659c640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6e00a1ab0 0x7fc6e00a3f70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0960b0 tx=0x7fc6ec096040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.654+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/2331549505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 0x56530d491ad0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0005c40 tx=0x7fc6f0010860 comp rx=0 tx=0).stop 2024-03-11T19:43:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.654+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2331549505 shutdown_connections 2024-03-11T19:43:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.654+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/2331549505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 0x56530d491ad0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.654+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/2331549505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48a4d0 0x56530d48efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.654+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/2331549505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d489ae0 0x56530d489f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.654+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2331549505 >> 172.21.15.83:0/2331549505 conn(0x56530d47c7f0 msgr2=0x56530d47cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.654+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2331549505 shutdown_connections 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.654+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2331549505 wait complete. 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6ffd93740 1 -- start start 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6ffd93740 1 Processor -- start 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d489ae0 0x56530d493ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48a4d0 0x56530d493ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48f640 0x56530d49b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6ffd93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d0bd430 con 0x56530d489ae0 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6ffd93740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d079e00 con 0x56530d48f640 2024-03-11T19:43:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6ffd93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d36efd0 con 0x56530d48a4d0 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6ffd93740 10 client.0 _ll_register_callbacks cb 0x56530d3c4fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6f759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d489ae0 0x56530d493ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6f759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d489ae0 0x56530d493ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53236/0 (socket says 172.21.15.83:53236) 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6f759e640 1 -- 172.21.15.83:0/822791211 learned_addr learned my addr 172.21.15.83:0/822791211 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6f759e640 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48f640 0x56530d49b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6f759e640 1 -- 172.21.15.83:0/822791211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48a4d0 msgr2=0x56530d493ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6f6d9d640 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48a4d0 0x56530d493ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6f759e640 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48a4d0 0x56530d493ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6f759e640 1 -- 172.21.15.83:0/822791211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48f640 msgr2=0x56530d49b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6f759e640 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48f640 0x56530d49b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.655+0000 7fc6f759e640 1 -- 172.21.15.83:0/822791211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f0009ce0 con 0x56530d489ae0 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.656+0000 7fc6f6d9d640 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48a4d0 0x56530d493ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.656+0000 7fc6f759e640 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d489ae0 0x56530d493ab0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009560 tx=0x7fc6f00042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461e2925b3c7a59 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.656+0000 7fc6daff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.656+0000 7fc6daff5640 1 -- 172.21.15.83:0/822791211 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0015070 con 0x56530d489ae0 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.656+0000 7fc6daff5640 1 -- 172.21.15.83:0/822791211 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc6f0004d50 con 0x56530d489ae0 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.656+0000 7fc6ffd93740 1 -- 172.21.15.83:0/822791211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56530d0e97e0 con 0x56530d489ae0 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.656+0000 7fc6daff5640 1 -- 172.21.15.83:0/822791211 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6f001e070 con 0x56530d489ae0 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.656+0000 7fc6ffd93740 10 client.16209 fetch_fsmap learned FSMap version 29 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.656+0000 7fc6ffd93740 1 -- 172.21.15.83:0/822791211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56530d3e8610 con 0x56530d489ae0 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.657+0000 7fc6daff5640 1 -- 172.21.15.83:0/822791211 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56530d0e97e0 con 0x56530d489ae0 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.657+0000 7fc6daff5640 1 -- 172.21.15.83:0/822791211 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc6f0025070 con 0x56530d489ae0 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.657+0000 7fc6ffd93740 10 client.16209 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.657+0000 7fc6ffd93740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.657+0000 7fc6ffd93740 20 client.16209 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.657+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56530d49fa30 0x56530d4a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.657+0000 7fc6ffd93740 4 client.16209 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.657+0000 7fc6ffd93740 1 -- 172.21.15.83:0/822791211 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56530d126720 con 0x56530d49fa30 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.657+0000 7fc6f6d9d640 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56530d49fa30 0x56530d4a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.658+0000 7fc6daff5640 10 client.16209 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.658+0000 7fc6f6d9d640 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56530d49fa30 0x56530d4a1e50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45699afe1ad81538 server_cookie=cc4f3335961c9006 in_seq=0 out_seq=0 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6daff5640 1 -- 172.21.15.83:0/822791211 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56530d126720 con 0x56530d49fa30 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6daff5640 10 client.16209 handle_command_reply: tid=0 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 1 client.16209 shutdown 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 1 -- 172.21.15.83:0/822791211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d489ae0 msgr2=0x56530d493ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d489ae0 0x56530d493ab0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0009560 tx=0x7fc6f00042b0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 1 -- 172.21.15.83:0/822791211 shutdown_connections 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56530d49fa30 0x56530d4a1e50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48f640 0x56530d49b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48a4d0 0x56530d493ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/822791211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d489ae0 0x56530d493ab0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6daff5640 0 client.16209 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 1 -- 172.21.15.83:0/822791211 >> 172.21.15.83:0/822791211 conn(0x56530d47c7f0 msgr2=0x56530d47ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 1 -- 172.21.15.83:0/822791211 shutdown_connections 2024-03-11T19:43:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 1 -- 172.21.15.83:0/822791211 wait complete. 2024-03-11T19:43:58.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.659+0000 7fc6ffd93740 20 client.16209 trim_cache size 0 max 16384 2024-03-11T19:43:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.683+0000 7fc6ffd93740 1 Processor -- start 2024-03-11T19:43:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.683+0000 7fc6ffd93740 1 -- start start 2024-03-11T19:43:58.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.683+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d489ae0 0x56530d477b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.683+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d49aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.683+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48f640 0x56530d47cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.683+0000 7fc6ffd93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d4780c0 con 0x56530d48a4d0 2024-03-11T19:43:58.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.683+0000 7fc6f6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d49aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6f6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d49aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53250/0 (socket says 172.21.15.83:53250) 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6f6d9d640 1 -- 172.21.15.83:0/663293194 learned_addr learned my addr 172.21.15.83:0/663293194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6f759e640 1 --2- 172.21.15.83:0/663293194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d489ae0 0x56530d477b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6ffd93740 1 -- 172.21.15.83:0/663293194 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d47d7e0 con 0x56530d48f640 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6ffd93740 1 -- 172.21.15.83:0/663293194 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d48a8b0 con 0x56530d489ae0 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6f759e640 1 --2- 172.21.15.83:0/663293194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48f640 0x56530d47cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6f6d9d640 1 -- 172.21.15.83:0/663293194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d489ae0 msgr2=0x56530d477b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6f6d9d640 1 --2- 172.21.15.83:0/663293194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d489ae0 0x56530d477b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6f6d9d640 1 -- 172.21.15.83:0/663293194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48f640 msgr2=0x56530d47cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6f6d9d640 1 --2- 172.21.15.83:0/663293194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48f640 0x56530d47cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6f6d9d640 1 -- 172.21.15.83:0/663293194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56530d492bb0 con 0x56530d48a4d0 2024-03-11T19:43:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6f6d9d640 1 --2- 172.21.15.83:0/663293194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d49aff0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8000c00 tx=0x7fc6e802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b1e97cf7e9d7bc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.684+0000 7fc6dcff9640 1 -- 172.21.15.83:0/663293194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e802cc40 con 0x56530d48a4d0 2024-03-11T19:43:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.685+0000 7fc6dcff9640 1 -- 172.21.15.83:0/663293194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6e8047be0 con 0x56530d48a4d0 2024-03-11T19:43:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.685+0000 7fc6dcff9640 1 -- 172.21.15.83:0/663293194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8004450 con 0x56530d48a4d0 2024-03-11T19:43:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.686+0000 7fc6ffd93740 1 -- 172.21.15.83:0/663293194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 msgr2=0x56530d49aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.686+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/663293194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d49aff0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8000c00 tx=0x7fc6e802c7e0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.686+0000 7fc6ffd93740 1 -- 172.21.15.83:0/663293194 shutdown_connections 2024-03-11T19:43:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.686+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/663293194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d48f640 0x56530d47cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.686+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/663293194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d49aff0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.686+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/663293194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d489ae0 0x56530d477b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.686+0000 7fc6ffd93740 1 -- 172.21.15.83:0/663293194 >> 172.21.15.83:0/663293194 conn(0x56530d47c7f0 msgr2=0x56530d48eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.687+0000 7fc6ffd93740 1 -- 172.21.15.83:0/663293194 shutdown_connections 2024-03-11T19:43:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.687+0000 7fc6ffd93740 1 -- 172.21.15.83:0/663293194 wait complete. 2024-03-11T19:43:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.687+0000 7fc6ffd93740 1 -- start start 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6ffd93740 1 Processor -- start 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d489ae0 0x56530d49aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d4990e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6ffd93740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 0x56530d499620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6ffd93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d49b2c0 con 0x56530d48a4d0 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6ffd93740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d499b60 con 0x56530d489ae0 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6ffd93740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530d499d20 con 0x56530d48f640 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6ffd93740 10 client.0 _ll_register_callbacks cb 0x7fc6f0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d4990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 0x56530d499620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 0x56530d499620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53428/0 (socket says 172.21.15.83:53428) 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d4990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53252/0 (socket says 172.21.15.83:53252) 2024-03-11T19:43:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f759e640 1 -- 172.21.15.83:0/2112411314 learned_addr learned my addr 172.21.15.83:0/2112411314 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f759e640 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d489ae0 0x56530d49aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f759e640 1 -- 172.21.15.83:0/2112411314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d489ae0 msgr2=0x56530d49aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f759e640 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d489ae0 0x56530d49aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f759e640 1 -- 172.21.15.83:0/2112411314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 msgr2=0x56530d4990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f759e640 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d4990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f759e640 1 -- 172.21.15.83:0/2112411314 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f8109f00 con 0x56530d48f640 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.688+0000 7fc6f6d9d640 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d4990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6f759e640 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 0x56530d499620 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0004280 tx=0x7fc6f00042b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c72ed7ca8c00c61 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6daff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6daff5640 1 -- 172.21.15.83:0/2112411314 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0015030 con 0x56530d48f640 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6daff5640 1 -- 172.21.15.83:0/2112411314 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc6f003b040 con 0x56530d48f640 2024-03-11T19:43:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2112411314 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6f0003e50 con 0x56530d48f640 2024-03-11T19:43:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6daff5640 1 -- 172.21.15.83:0/2112411314 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6f001e040 con 0x56530d48f640 2024-03-11T19:43:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6ffd93740 10 client.25792 fetch_fsmap learned FSMap version 29 2024-03-11T19:43:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2112411314 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56530d384ac0 con 0x56530d48f640 2024-03-11T19:43:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6daff5640 1 -- 172.21.15.83:0/2112411314 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6f0003e50 con 0x56530d48f640 2024-03-11T19:43:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6daff5640 1 -- 172.21.15.83:0/2112411314 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc6f004b070 con 0x56530d48f640 2024-03-11T19:43:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6ffd93740 10 client.25792 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:43:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6ffd93740 10 client.25792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6ffd93740 20 client.25792 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56530d4f70e0 0x56530d4f9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6ffd93740 4 client.25792 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:43:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.689+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2112411314 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56530d175890 con 0x56530d4f70e0 2024-03-11T19:43:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.690+0000 7fc6f6d9d640 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56530d4f70e0 0x56530d4f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6f6d9d640 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56530d4f70e0 0x56530d4f9500 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447249efc4e03a89 server_cookie=8ebfce1ee9edcfee in_seq=0 out_seq=0 2024-03-11T19:43:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6daff5640 10 client.25792 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6daff5640 1 -- 172.21.15.83:0/2112411314 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56530d175890 con 0x56530d4f70e0 2024-03-11T19:43:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6daff5640 10 client.25792 handle_command_reply: tid=0 2024-03-11T19:43:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 1 client.25792 shutdown 2024-03-11T19:43:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2112411314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 msgr2=0x56530d499620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 0x56530d499620 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0004280 tx=0x7fc6f00042b0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2112411314 shutdown_connections 2024-03-11T19:43:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56530d4f70e0 0x56530d4f9500 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56530d48f640 0x56530d499620 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6daff5640 0 client.25792 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56530d48a4d0 0x56530d4990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 1 --2- 172.21.15.83:0/2112411314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56530d489ae0 0x56530d49aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2112411314 >> 172.21.15.83:0/2112411314 conn(0x56530d47c7f0 msgr2=0x56530d48ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2112411314 shutdown_connections 2024-03-11T19:43:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 1 -- 172.21.15.83:0/2112411314 wait complete. 2024-03-11T19:43:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.692+0000 7fc6ffd93740 20 client.25792 trim_cache size 0 max 16384 2024-03-11T19:43:58.699 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:43:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:43:58.700 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:43:58.700 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 -- 172.21.15.83:0/2943995559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6e00a1ab0 msgr2=0x7fc6e00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6e00a1ab0 0x7fc6e00a3f70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0960b0 tx=0x7fc6ec096040 comp rx=0 tx=0).stop 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 -- 172.21.15.83:0/2943995559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f8102d70 msgr2=0x7fc6f81978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f8102d70 0x7fc6f81978f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00c920 tx=0x7fc6ec00cde0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 -- 172.21.15.83:0/2943995559 shutdown_connections 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc6e00a1ab0 0x7fc6e00a3f70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6f819c950 0x7fc6f819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6f8102d70 0x7fc6f81978f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 --2- 172.21.15.83:0/2943995559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f81023b0 0x7fc6f81973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 -- 172.21.15.83:0/2943995559 >> 172.21.15.83:0/2943995559 conn(0x7fc6f80fc710 msgr2=0x7fc6f81056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 -- 172.21.15.83:0/2943995559 shutdown_connections 2024-03-11T19:43:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.693+0000 7fc6fd0df640 1 -- 172.21.15.83:0/2943995559 wait complete. 2024-03-11T19:43:58.713 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:43:58.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:43:58.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:58 smithi083 ceph-mon[31864]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.6 MiB/s wr, 845 op/s 2024-03-11T19:43:58.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.870+0000 7f4110b2e640 1 -- 172.21.15.83:0/226567157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410c100f80 msgr2=0x7f410c107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.870+0000 7f4110b2e640 1 --2- 172.21.15.83:0/226567157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410c100f80 0x7f410c107810 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4100009900 tx=0x7f410002bc30 comp rx=0 tx=0).stop 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.870+0000 7f4110b2e640 1 -- 172.21.15.83:0/226567157 shutdown_connections 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.870+0000 7f4110b2e640 1 --2- 172.21.15.83:0/226567157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410c100f80 0x7f410c107810 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.870+0000 7f4110b2e640 1 --2- 172.21.15.83:0/226567157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410c1005c0 0x7f410c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.870+0000 7f4110b2e640 1 --2- 172.21.15.83:0/226567157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f410c0ff3d0 0x7f410c0ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.870+0000 7f4110b2e640 1 -- 172.21.15.83:0/226567157 >> 172.21.15.83:0/226567157 conn(0x7f410c0fa980 msgr2=0x7f410c0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.870+0000 7f4110b2e640 1 -- 172.21.15.83:0/226567157 shutdown_connections 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.870+0000 7f4110b2e640 1 -- 172.21.15.83:0/226567157 wait complete. 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.871+0000 7f4110b2e640 1 Processor -- start 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.871+0000 7f4110b2e640 1 -- start start 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.871+0000 7f4110b2e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410c0ff3d0 0x7f410c1102e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.871+0000 7f4110b2e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f410c1005c0 0x7f410c110820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.871+0000 7f4110b2e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410c100f80 0x7f410c117890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.871+0000 7f4110b2e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c117dd0 con 0x7f410c0ff3d0 2024-03-11T19:43:58.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.871+0000 7f4110b2e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c117f40 con 0x7f410c100f80 2024-03-11T19:43:58.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.871+0000 7f4110b2e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c118100 con 0x7f410c1005c0 2024-03-11T19:43:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.872+0000 7f4109d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f410c1005c0 0x7f410c110820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.872+0000 7f4109d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f410c1005c0 0x7f410c110820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53444/0 (socket says 172.21.15.83:53444) 2024-03-11T19:43:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.872+0000 7f4109d9b640 1 -- 172.21.15.83:0/2855547334 learned_addr learned my addr 172.21.15.83:0/2855547334 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.872+0000 7f410ad9d640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410c100f80 0x7f410c117890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.872+0000 7f4109d9b640 1 -- 172.21.15.83:0/2855547334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410c100f80 msgr2=0x7f410c117890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.872+0000 7f4109d9b640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410c100f80 0x7f410c117890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.872+0000 7f4109d9b640 1 -- 172.21.15.83:0/2855547334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410c0ff3d0 msgr2=0x7f410c1102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.872+0000 7f4109d9b640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410c0ff3d0 0x7f410c1102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:58.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.872+0000 7f4109d9b640 1 -- 172.21.15.83:0/2855547334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100009560 con 0x7f410c1005c0 2024-03-11T19:43:58.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.872+0000 7f4109d9b640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f410c1005c0 0x7f410c110820 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00e9f0 tx=0x7f40fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:58.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.873+0000 7f40fb7fe640 1 -- 172.21.15.83:0/2855547334 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc009700 con 0x7f410c1005c0 2024-03-11T19:43:58.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.873+0000 7f40fb7fe640 1 -- 172.21.15.83:0/2855547334 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40fc00cd80 con 0x7f410c1005c0 2024-03-11T19:43:58.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.873+0000 7f40fb7fe640 1 -- 172.21.15.83:0/2855547334 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc018880 con 0x7f410c1005c0 2024-03-11T19:43:58.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.873+0000 7f4110b2e640 1 -- 172.21.15.83:0/2855547334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410c1183e0 con 0x7f410c1005c0 2024-03-11T19:43:58.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.873+0000 7f4110b2e640 1 -- 172.21.15.83:0/2855547334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c1ae5c0 con 0x7f410c1005c0 2024-03-11T19:43:58.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.874+0000 7f4110b2e640 1 -- 172.21.15.83:0/2855547334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c106620 con 0x7f410c1005c0 2024-03-11T19:43:58.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.876+0000 7f40fb7fe640 1 -- 172.21.15.83:0/2855547334 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f40fc010b10 con 0x7f410c1005c0 2024-03-11T19:43:58.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.877+0000 7f40fb7fe640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40f40a1b80 0x7f40f40a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:58.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.877+0000 7f40fb7fe640 1 -- 172.21.15.83:0/2855547334 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f40fc014070 con 0x7f410c1005c0 2024-03-11T19:43:58.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.877+0000 7f410a59c640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40f40a1b80 0x7f40f40a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:58.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.878+0000 7f410a59c640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40f40a1b80 0x7f40f40a4040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4104005c40 tx=0x7f4104005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:58.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:58.880+0000 7f40fb7fe640 1 -- 172.21.15.83:0/2855547334 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f40fc096950 con 0x7f410c1005c0 2024-03-11T19:43:59.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:58 smithi137 ceph-mon[36220]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.6 MiB/s wr, 845 op/s 2024-03-11T19:43:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:58 smithi204 ceph-mon[36739]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.6 MiB/s wr, 845 op/s 2024-03-11T19:43:59.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.254+0000 7f4110b2e640 1 -- 172.21.15.83:0/2855547334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f410c10ab70 con 0x7f410c1005c0 2024-03-11T19:43:59.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.255+0000 7f40fb7fe640 1 -- 172.21.15.83:0/2855547334 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11350 (secure 0 0 0) 0x7f40fc09a010 con 0x7f410c1005c0 2024-03-11T19:43:59.259 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:43:59.259 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:43:42.002107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:43:59.260 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-03-11T19:43:59.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.258+0000 7f4110b2e640 1 -- 172.21.15.83:0/2855547334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40f40a1b80 msgr2=0x7f40f40a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.258+0000 7f4110b2e640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40f40a1b80 0x7f40f40a4040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4104005c40 tx=0x7f4104005bd0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.259+0000 7f4110b2e640 1 -- 172.21.15.83:0/2855547334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f410c1005c0 msgr2=0x7f410c110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.259+0000 7f4110b2e640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f410c1005c0 0x7f410c110820 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00e9f0 tx=0x7f40fc00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.259+0000 7f4110b2e640 1 -- 172.21.15.83:0/2855547334 shutdown_connections 2024-03-11T19:43:59.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.259+0000 7f4110b2e640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f40f40a1b80 0x7f40f40a4040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.259+0000 7f4110b2e640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f410c100f80 0x7f410c117890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.259+0000 7f4110b2e640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f410c1005c0 0x7f410c110820 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.259+0000 7f4110b2e640 1 --2- 172.21.15.83:0/2855547334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f410c0ff3d0 0x7f410c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.259+0000 7f4110b2e640 1 -- 172.21.15.83:0/2855547334 >> 172.21.15.83:0/2855547334 conn(0x7f410c0fa980 msgr2=0x7f410c1037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.259+0000 7f4110b2e640 1 -- 172.21.15.83:0/2855547334 shutdown_connections 2024-03-11T19:43:59.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.259+0000 7f4110b2e640 1 -- 172.21.15.83:0/2855547334 wait complete. 2024-03-11T19:43:59.279 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:43:59.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:43:59.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.432+0000 7f1199ffb640 1 -- 172.21.15.83:0/1560328391 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190005220 con 0x7f119c101c60 2024-03-11T19:43:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.432+0000 7f11a1765640 1 -- 172.21.15.83:0/1560328391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f119c101c60 msgr2=0x7f119c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.432+0000 7f11a1765640 1 --2- 172.21.15.83:0/1560328391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f119c101c60 0x7f119c104050 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1190009900 tx=0x7f119002bc30 comp rx=0 tx=0).stop 2024-03-11T19:43:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.432+0000 7f11a1765640 1 -- 172.21.15.83:0/1560328391 shutdown_connections 2024-03-11T19:43:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.432+0000 7f11a1765640 1 --2- 172.21.15.83:0/1560328391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119c106ec0 0x7f119c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.432+0000 7f11a1765640 1 --2- 172.21.15.83:0/1560328391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f119c104590 0x7f119c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.432+0000 7f11a1765640 1 --2- 172.21.15.83:0/1560328391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f119c101c60 0x7f119c104050 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.432+0000 7f11a1765640 1 -- 172.21.15.83:0/1560328391 >> 172.21.15.83:0/1560328391 conn(0x7f119c0fb610 msgr2=0x7f119c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.433+0000 7f11a1765640 1 -- 172.21.15.83:0/1560328391 shutdown_connections 2024-03-11T19:43:59.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.433+0000 7f11a1765640 1 -- 172.21.15.83:0/1560328391 wait complete. 2024-03-11T19:43:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.433+0000 7f11a1765640 1 Processor -- start 2024-03-11T19:43:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.433+0000 7f11a1765640 1 -- start start 2024-03-11T19:43:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f11a1765640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f119c101c60 0x7f119c197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f11a1765640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119c104590 0x7f119c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f11a1765640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f119c106ec0 0x7f119c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f11a1765640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c19ed10 con 0x7f119c101c60 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f11a1765640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c19ee80 con 0x7f119c104590 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f11a1765640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c19f040 con 0x7f119c106ec0 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f119affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f119c101c60 0x7f119c197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f119a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119c104590 0x7f119c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f119affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f119c101c60 0x7f119c197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53292/0 (socket says 172.21.15.83:53292) 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f119affd640 1 -- 172.21.15.83:0/740445527 learned_addr learned my addr 172.21.15.83:0/740445527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f119b7fe640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f119c106ec0 0x7f119c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f119a7fc640 1 -- 172.21.15.83:0/740445527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f119c106ec0 msgr2=0x7f119c19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f119a7fc640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f119c106ec0 0x7f119c19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f119a7fc640 1 -- 172.21.15.83:0/740445527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f119c101c60 msgr2=0x7f119c197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f119a7fc640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f119c101c60 0x7f119c197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f119a7fc640 1 -- 172.21.15.83:0/740445527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1194009840 con 0x7f119c104590 2024-03-11T19:43:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.434+0000 7f119b7fe640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f119c106ec0 0x7f119c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:43:59.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.435+0000 7f119a7fc640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119c104590 0x7f119c197760 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c940 tx=0x7f118c00ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.435+0000 7f1183fff640 1 -- 172.21.15.83:0/740445527 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c007bb0 con 0x7f119c104590 2024-03-11T19:43:59.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.435+0000 7f11a1765640 1 -- 172.21.15.83:0/740445527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1190009560 con 0x7f119c104590 2024-03-11T19:43:59.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.435+0000 7f1183fff640 1 -- 172.21.15.83:0/740445527 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f118c007d10 con 0x7f119c104590 2024-03-11T19:43:59.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.435+0000 7f11a1765640 1 -- 172.21.15.83:0/740445527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c19f6f0 con 0x7f119c104590 2024-03-11T19:43:59.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.436+0000 7f1183fff640 1 -- 172.21.15.83:0/740445527 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c007bb0 con 0x7f119c104590 2024-03-11T19:43:59.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.438+0000 7f11a4412740 1 Processor -- start 2024-03-11T19:43:59.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.438+0000 7f1183fff640 1 -- 172.21.15.83:0/740445527 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f118c00f450 con 0x7f119c104590 2024-03-11T19:43:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.438+0000 7f11a4412740 1 -- start start 2024-03-11T19:43:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.439+0000 7f11a4412740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575b270 0x5641c575b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.439+0000 7f11a4412740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575bc60 0x5641c5760760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.439+0000 7f11a4412740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 0x5641c5763260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.439+0000 7f11a4412740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c5717960 con 0x5641c5760dd0 2024-03-11T19:43:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.439+0000 7f11a4412740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c56cb310 con 0x5641c575bc60 2024-03-11T19:43:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.439+0000 7f11a4412740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c56c8840 con 0x5641c575b270 2024-03-11T19:43:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.439+0000 7f119b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575b270 0x5641c575b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.439+0000 7f119b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575b270 0x5641c575b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53470/0 (socket says 172.21.15.83:53470) 2024-03-11T19:43:59.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.439+0000 7f119b7fe640 1 -- 172.21.15.83:0/2321781298 learned_addr learned my addr 172.21.15.83:0/2321781298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.439+0000 7f119a7fc640 1 --2- 172.21.15.83:0/2321781298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 0x5641c5763260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.439+0000 7f119affd640 1 --2- 172.21.15.83:0/2321781298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575bc60 0x5641c5760760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.440+0000 7f1183fff640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11840a1ae0 0x7f11840a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.440+0000 7f1183fff640 1 -- 172.21.15.83:0/740445527 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f118c0cc300 con 0x7f119c104590 2024-03-11T19:43:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.441+0000 7f119b7fe640 1 -- 172.21.15.83:0/2321781298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575bc60 msgr2=0x5641c5760760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.441+0000 7f119b7fe640 1 --2- 172.21.15.83:0/2321781298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575bc60 0x5641c5760760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.441+0000 7f119b7fe640 1 -- 172.21.15.83:0/2321781298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 msgr2=0x5641c5763260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.441+0000 7f119b7fe640 1 --2- 172.21.15.83:0/2321781298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 0x5641c5763260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.441+0000 7f119b7fe640 1 -- 172.21.15.83:0/2321781298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641c5686060 con 0x5641c575b270 2024-03-11T19:43:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.441+0000 7f119b7fe640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11840a1ae0 0x7f11840a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.441+0000 7f119b7fe640 1 --2- 172.21.15.83:0/2321781298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575b270 0x5641c575b690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f119400f550 tx=0x7f119400fa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b5f5cbdf607cfb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.441+0000 7f11817fa640 1 -- 172.21.15.83:0/2321781298 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119400d640 con 0x5641c575b270 2024-03-11T19:43:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.441+0000 7f11817fa640 1 -- 172.21.15.83:0/2321781298 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f119400de70 con 0x5641c575b270 2024-03-11T19:43:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.441+0000 7f11817fa640 1 -- 172.21.15.83:0/2321781298 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119401b3b0 con 0x5641c575b270 2024-03-11T19:43:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.441+0000 7f119b7fe640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11840a1ae0 0x7f11840a3fa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1194020000 tx=0x7f119401e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.442+0000 7f11a4412740 1 -- 172.21.15.83:0/2321781298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575b270 msgr2=0x5641c575b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.442+0000 7f11a4412740 1 --2- 172.21.15.83:0/2321781298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575b270 0x5641c575b690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f119400f550 tx=0x7f119400fa10 comp rx=0 tx=0).stop 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.442+0000 7f11a4412740 1 -- 172.21.15.83:0/2321781298 shutdown_connections 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.442+0000 7f11a4412740 1 --2- 172.21.15.83:0/2321781298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 0x5641c5763260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.442+0000 7f11a4412740 1 --2- 172.21.15.83:0/2321781298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575bc60 0x5641c5760760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.442+0000 7f11a4412740 1 --2- 172.21.15.83:0/2321781298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575b270 0x5641c575b690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.442+0000 7f11a4412740 1 -- 172.21.15.83:0/2321781298 >> 172.21.15.83:0/2321781298 conn(0x5641c57567e0 msgr2=0x5641c5756be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.442+0000 7f11a4412740 1 -- 172.21.15.83:0/2321781298 shutdown_connections 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.442+0000 7f11a4412740 1 -- 172.21.15.83:0/2321781298 wait complete. 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f11a4412740 1 -- start start 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f11a4412740 1 Processor -- start 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f11a4412740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5765460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f11a4412740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575bc60 0x5641c57659a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f11a4412740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 0x5641c576c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f11a4412740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c56c4020 con 0x5641c5760dd0 2024-03-11T19:43:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f11a4412740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c5397430 con 0x5641c575b270 2024-03-11T19:43:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f11a4412740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c5353e00 con 0x5641c575bc60 2024-03-11T19:43:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f11a4412740 10 client.0 _ll_register_callbacks cb 0x5641c566bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f119affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5765460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f119b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 0x5641c576c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f119b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 0x5641c576c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53318/0 (socket says 172.21.15.83:53318) 2024-03-11T19:43:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f119affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5765460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:57228/0 (socket says 172.21.15.83:57228) 2024-03-11T19:43:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f119b7fe640 1 -- 172.21.15.83:0/1019789508 learned_addr learned my addr 172.21.15.83:0/1019789508 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f119a7fc640 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575bc60 0x5641c57659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f119affd640 1 -- 172.21.15.83:0/1019789508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575bc60 msgr2=0x5641c57659a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f119affd640 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575bc60 0x5641c57659a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f119affd640 1 -- 172.21.15.83:0/1019789508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 msgr2=0x5641c576c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.443+0000 7f119affd640 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 0x5641c576c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.444+0000 7f119affd640 1 -- 172.21.15.83:0/1019789508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1194009e20 con 0x5641c575b270 2024-03-11T19:43:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.444+0000 7f119b7fe640 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 0x5641c576c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.444+0000 7f119affd640 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5765460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f119002c680 tx=0x7f11900048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb15dd1a447a4cd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.444+0000 7f117eff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.445+0000 7f117eff5640 1 -- 172.21.15.83:0/1019789508 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190004ae0 con 0x5641c575b270 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.445+0000 7f11a4412740 1 -- 172.21.15.83:0/1019789508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641c53c37e0 con 0x5641c575b270 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.445+0000 7f117eff5640 1 -- 172.21.15.83:0/1019789508 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1190004040 con 0x5641c575b270 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.445+0000 7f117eff5640 1 -- 172.21.15.83:0/1019789508 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f119004f030 con 0x5641c575b270 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.445+0000 7f11a4412740 10 client.25718 fetch_fsmap learned FSMap version 29 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.445+0000 7f11a4412740 1 -- 172.21.15.83:0/1019789508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5641c56c26a0 con 0x5641c575b270 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.446+0000 7f117eff5640 1 -- 172.21.15.83:0/1019789508 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1190044070 con 0x5641c575b270 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.446+0000 7f117eff5640 1 -- 172.21.15.83:0/1019789508 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f119003c040 con 0x5641c575b270 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.446+0000 7f11a4412740 10 client.25718 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.446+0000 7f11a4412740 10 client.25718 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:43:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.446+0000 7f11a4412740 20 client.25718 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.446+0000 7f11a4412740 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5641c57714d0 0x5641c5773960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.446+0000 7f11a4412740 4 client.25718 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.446+0000 7f11a4412740 1 -- 172.21.15.83:0/1019789508 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641c53c37e0 con 0x5641c57714d0 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.447+0000 7f119a7fc640 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5641c57714d0 0x5641c5773960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.447+0000 7f119a7fc640 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5641c57714d0 0x5641c5773960 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33133e091c4f414a server_cookie=e100d0c475145d96 in_seq=0 out_seq=0 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.447+0000 7f117eff5640 10 client.25718 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.448+0000 7f117eff5640 1 -- 172.21.15.83:0/1019789508 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5641c53c37e0 con 0x5641c57714d0 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.448+0000 7f117eff5640 10 client.25718 handle_command_reply: tid=0 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.448+0000 7f11a4412740 1 client.25718 shutdown 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f11a4412740 1 -- 172.21.15.83:0/1019789508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 msgr2=0x5641c5765460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f11a4412740 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5765460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f119002c680 tx=0x7f11900048f0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f11a4412740 1 -- 172.21.15.83:0/1019789508 shutdown_connections 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f11a4412740 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5641c57714d0 0x5641c5773960 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f11a4412740 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c5760dd0 0x5641c576c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f11a4412740 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c575bc60 0x5641c57659a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f11a4412740 1 --2- 172.21.15.83:0/1019789508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5765460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f11a4412740 1 -- 172.21.15.83:0/1019789508 >> 172.21.15.83:0/1019789508 conn(0x5641c57567e0 msgr2=0x5641c5755b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f117eff5640 0 client.25718 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:43:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f11a4412740 1 -- 172.21.15.83:0/1019789508 shutdown_connections 2024-03-11T19:43:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f11a4412740 1 -- 172.21.15.83:0/1019789508 wait complete. 2024-03-11T19:43:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.449+0000 7f11a4412740 20 client.25718 trim_cache size 0 max 16384 2024-03-11T19:43:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.467+0000 7f11a4412740 1 Processor -- start 2024-03-11T19:43:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.467+0000 7f11a4412740 1 -- start start 2024-03-11T19:43:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.467+0000 7f11a4412740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5751a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.467+0000 7f11a4412740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 0x5641c576c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.467+0000 7f11a4412740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c5760dd0 0x5641c57552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f11a4412740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c5649130 con 0x5641c575bc60 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f11a4412740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c576fbd0 con 0x5641c575b270 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f11a4412740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c5748a40 con 0x5641c5760dd0 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f119affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5751a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f119affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5751a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:57242/0 (socket says 172.21.15.83:57242) 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f119affd640 1 -- 172.21.15.83:0/397241321 learned_addr learned my addr 172.21.15.83:0/397241321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f119a7fc640 1 --2- 172.21.15.83:0/397241321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 0x5641c576c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f119b7fe640 1 --2- 172.21.15.83:0/397241321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c5760dd0 0x5641c57552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f119affd640 1 -- 172.21.15.83:0/397241321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c5760dd0 msgr2=0x5641c57552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f119affd640 1 --2- 172.21.15.83:0/397241321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c5760dd0 0x5641c57552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f119affd640 1 -- 172.21.15.83:0/397241321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 msgr2=0x5641c576c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f119affd640 1 --2- 172.21.15.83:0/397241321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 0x5641c576c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f119affd640 1 -- 172.21.15.83:0/397241321 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641c565ec20 con 0x5641c575b270 2024-03-11T19:43:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f119affd640 1 --2- 172.21.15.83:0/397241321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5751a10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f119002c650 tx=0x7f1190005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c8b7f072254f432 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f1180ff9640 1 -- 172.21.15.83:0/397241321 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119004f040 con 0x5641c575b270 2024-03-11T19:43:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f1180ff9640 1 -- 172.21.15.83:0/397241321 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1190044040 con 0x5641c575b270 2024-03-11T19:43:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.468+0000 7f1180ff9640 1 -- 172.21.15.83:0/397241321 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190004190 con 0x5641c575b270 2024-03-11T19:43:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.469+0000 7f11a4412740 1 -- 172.21.15.83:0/397241321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 msgr2=0x5641c5751a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.469+0000 7f11a4412740 1 --2- 172.21.15.83:0/397241321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5751a10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f119002c650 tx=0x7f1190005d50 comp rx=0 tx=0).stop 2024-03-11T19:43:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.470+0000 7f11a4412740 1 -- 172.21.15.83:0/397241321 shutdown_connections 2024-03-11T19:43:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.470+0000 7f11a4412740 1 --2- 172.21.15.83:0/397241321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c5760dd0 0x5641c57552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.470+0000 7f11a4412740 1 --2- 172.21.15.83:0/397241321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 0x5641c576c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.470+0000 7f11a4412740 1 --2- 172.21.15.83:0/397241321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c5751a10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.470+0000 7f11a4412740 1 -- 172.21.15.83:0/397241321 >> 172.21.15.83:0/397241321 conn(0x5641c57567e0 msgr2=0x5641c5756bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.471+0000 7f11a4412740 1 -- 172.21.15.83:0/397241321 shutdown_connections 2024-03-11T19:43:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.471+0000 7f11a4412740 1 -- 172.21.15.83:0/397241321 wait complete. 2024-03-11T19:43:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.471+0000 7f11a4412740 1 -- start start 2024-03-11T19:43:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.471+0000 7f11a4412740 1 Processor -- start 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f11a4412740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c576c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f11a4412740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 0x5641c576ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f11a4412740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c5760dd0 0x5641c576b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f11a4412740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c576cdf0 con 0x5641c575bc60 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f11a4412740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c576b700 con 0x5641c575b270 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f11a4412740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641c576b8c0 con 0x5641c5760dd0 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f11a4412740 10 client.0 _ll_register_callbacks cb 0x7f1194009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 0x5641c576ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 0x5641c576ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53328/0 (socket says 172.21.15.83:53328) 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119a7fc640 1 -- 172.21.15.83:0/1787082159 learned_addr learned my addr 172.21.15.83:0/1787082159 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119affd640 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c576c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119b7fe640 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c5760dd0 0x5641c576b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119a7fc640 1 -- 172.21.15.83:0/1787082159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c5760dd0 msgr2=0x5641c576b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119a7fc640 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c5760dd0 0x5641c576b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119a7fc640 1 -- 172.21.15.83:0/1787082159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 msgr2=0x5641c576c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119a7fc640 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c576c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119a7fc640 1 -- 172.21.15.83:0/1787082159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641c56c26a0 con 0x5641c575bc60 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119affd640 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c576c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f119a7fc640 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 0x5641c576ac80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f118c095810 tx=0x7f118c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef5af126b9de3ea server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.472+0000 7f117eff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.473+0000 7f117eff5640 1 -- 172.21.15.83:0/1787082159 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c007bb0 con 0x5641c575bc60 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.473+0000 7f117eff5640 1 -- 172.21.15.83:0/1787082159 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f118c09d170 con 0x5641c575bc60 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.473+0000 7f11a4412740 1 -- 172.21.15.83:0/1787082159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f119400de70 con 0x5641c575bc60 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.473+0000 7f117eff5640 1 -- 172.21.15.83:0/1787082159 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f118c09d330 con 0x5641c575bc60 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.473+0000 7f11a4412740 10 client.16236 fetch_fsmap learned FSMap version 29 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.473+0000 7f11a4412740 1 -- 172.21.15.83:0/1787082159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5641c56848c0 con 0x5641c575bc60 2024-03-11T19:43:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.474+0000 7f117eff5640 1 -- 172.21.15.83:0/1787082159 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f119400de70 con 0x5641c575bc60 2024-03-11T19:43:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.474+0000 7f117eff5640 1 -- 172.21.15.83:0/1787082159 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f118c016020 con 0x5641c575bc60 2024-03-11T19:43:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.474+0000 7f11a4412740 10 client.16236 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:43:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.474+0000 7f11a4412740 10 client.16236 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:43:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.474+0000 7f11a4412740 20 client.16236 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.474+0000 7f11a4412740 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5641c57c8ff0 0x5641c57cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.474+0000 7f11a4412740 4 client.16236 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:43:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.474+0000 7f11a4412740 1 -- 172.21.15.83:0/1787082159 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5641c5400720 con 0x5641c57c8ff0 2024-03-11T19:43:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.474+0000 7f119affd640 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5641c57c8ff0 0x5641c57cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f119affd640 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5641c57c8ff0 0x5641c57cb410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd32330c00a54870 server_cookie=8c1b649e046a4aee in_seq=0 out_seq=0 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f117eff5640 10 client.16236 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f117eff5640 1 -- 172.21.15.83:0/1787082159 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5641c5400720 con 0x5641c57c8ff0 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f117eff5640 10 client.16236 handle_command_reply: tid=0 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f11a4412740 1 client.16236 shutdown 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f11a4412740 1 -- 172.21.15.83:0/1787082159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 msgr2=0x5641c576ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f11a4412740 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 0x5641c576ac80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f118c095810 tx=0x7f118c005ce0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f11a4412740 1 -- 172.21.15.83:0/1787082159 shutdown_connections 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f11a4412740 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5641c57c8ff0 0x5641c57cb410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f11a4412740 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641c5760dd0 0x5641c576b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f11a4412740 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641c575bc60 0x5641c576ac80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f11a4412740 1 --2- 172.21.15.83:0/1787082159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5641c575b270 0x5641c576c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f11a4412740 1 -- 172.21.15.83:0/1787082159 >> 172.21.15.83:0/1787082159 conn(0x5641c57567e0 msgr2=0x5641c5751a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.475+0000 7f117eff5640 0 client.16236 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:43:59.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.476+0000 7f11a4412740 1 -- 172.21.15.83:0/1787082159 shutdown_connections 2024-03-11T19:43:59.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.476+0000 7f11a4412740 1 -- 172.21.15.83:0/1787082159 wait complete. 2024-03-11T19:43:59.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.476+0000 7f11a4412740 20 client.16236 trim_cache size 0 max 16384 2024-03-11T19:43:59.485 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:43:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.476+0000 7f11a1765640 1 -- 172.21.15.83:0/740445527 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11840a1ae0 msgr2=0x7f11840a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.476+0000 7f11a1765640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11840a1ae0 0x7f11840a3fa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1194020000 tx=0x7f119401e000 comp rx=0 tx=0).stop 2024-03-11T19:43:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.476+0000 7f11a1765640 1 -- 172.21.15.83:0/740445527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119c104590 msgr2=0x7f119c197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.476+0000 7f11a1765640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119c104590 0x7f119c197760 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c940 tx=0x7f118c00ce00 comp rx=0 tx=0).stop 2024-03-11T19:43:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.477+0000 7f11a1765640 1 -- 172.21.15.83:0/740445527 shutdown_connections 2024-03-11T19:43:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.477+0000 7f11a1765640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f11840a1ae0 0x7f11840a3fa0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.477+0000 7f11a1765640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f119c106ec0 0x7f119c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.477+0000 7f11a1765640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f119c104590 0x7f119c197760 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.477+0000 7f11a1765640 1 --2- 172.21.15.83:0/740445527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f119c101c60 0x7f119c197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.477+0000 7f11a1765640 1 -- 172.21.15.83:0/740445527 >> 172.21.15.83:0/740445527 conn(0x7f119c0fb610 msgr2=0x7f119c1025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.477+0000 7f11a1765640 1 -- 172.21.15.83:0/740445527 shutdown_connections 2024-03-11T19:43:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.477+0000 7f11a1765640 1 -- 172.21.15.83:0/740445527 wait complete. 2024-03-11T19:43:59.497 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:43:59.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:43:59.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.656+0000 7f04791c2640 1 -- 172.21.15.83:0/2488704136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047410aaf0 msgr2=0x7f047410cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.656+0000 7f04791c2640 1 --2- 172.21.15.83:0/2488704136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047410aaf0 0x7f047410cf60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f046c009ca0 tx=0x7f046c02c100 comp rx=0 tx=0).stop 2024-03-11T19:43:59.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.656+0000 7f04791c2640 1 -- 172.21.15.83:0/2488704136 shutdown_connections 2024-03-11T19:43:59.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.656+0000 7f04791c2640 1 --2- 172.21.15.83:0/2488704136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f047410d4a0 0x7f047410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.656+0000 7f04791c2640 1 --2- 172.21.15.83:0/2488704136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047410aaf0 0x7f047410cf60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.656+0000 7f04791c2640 1 --2- 172.21.15.83:0/2488704136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474108180 0x7f047410a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.656+0000 7f04791c2640 1 -- 172.21.15.83:0/2488704136 >> 172.21.15.83:0/2488704136 conn(0x7f047406dfe0 msgr2=0x7f0474070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.656+0000 7f04791c2640 1 -- 172.21.15.83:0/2488704136 shutdown_connections 2024-03-11T19:43:59.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.656+0000 7f04791c2640 1 -- 172.21.15.83:0/2488704136 wait complete. 2024-03-11T19:43:59.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.657+0000 7f04791c2640 1 Processor -- start 2024-03-11T19:43:59.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.657+0000 7f04791c2640 1 -- start start 2024-03-11T19:43:59.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f04791c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474108180 0x7f04741a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f04791c2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f047410aaf0 0x7f04741a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f04791c2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047410d4a0 0x7f04741ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f04791c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04741adb60 con 0x7f0474108180 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f04791c2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04741adcd0 con 0x7f047410aaf0 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f04791c2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04741ade90 con 0x7f047410d4a0 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f0472d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474108180 0x7f04741a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f047259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f047410aaf0 0x7f04741a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f0472d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474108180 0x7f04741a6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42968/0 (socket says 172.21.15.83:42968) 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f0472d9d640 1 -- 172.21.15.83:0/469726072 learned_addr learned my addr 172.21.15.83:0/469726072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f047359e640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047410d4a0 0x7f04741ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f047259c640 1 -- 172.21.15.83:0/469726072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047410d4a0 msgr2=0x7f04741ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f047259c640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047410d4a0 0x7f04741ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f047259c640 1 -- 172.21.15.83:0/469726072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474108180 msgr2=0x7f04741a6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f047259c640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474108180 0x7f04741a6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.658+0000 7f047259c640 1 -- 172.21.15.83:0/469726072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c009900 con 0x7f047410aaf0 2024-03-11T19:43:59.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.659+0000 7f0472d9d640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474108180 0x7f04741a6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:43:59.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.659+0000 7f047259c640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f047410aaf0 0x7f04741a6650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f046c009bf0 tx=0x7f046c032fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.659+0000 7f045bfff640 1 -- 172.21.15.83:0/469726072 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c034640 con 0x7f047410aaf0 2024-03-11T19:43:59.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.659+0000 7f045bfff640 1 -- 172.21.15.83:0/469726072 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f046c003e50 con 0x7f047410aaf0 2024-03-11T19:43:59.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.659+0000 7f045bfff640 1 -- 172.21.15.83:0/469726072 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c040450 con 0x7f047410aaf0 2024-03-11T19:43:59.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.659+0000 7f04791c2640 1 -- 172.21.15.83:0/469726072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04741ae110 con 0x7f047410aaf0 2024-03-11T19:43:59.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.659+0000 7f04791c2640 1 -- 172.21.15.83:0/469726072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04741ae540 con 0x7f047410aaf0 2024-03-11T19:43:59.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.662+0000 7f045bfff640 1 -- 172.21.15.83:0/469726072 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f046c00a040 con 0x7f047410aaf0 2024-03-11T19:43:59.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.663+0000 7f047bd8a740 1 Processor -- start 2024-03-11T19:43:59.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.663+0000 7f045bfff640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f045c0a1c00 0x7f045c0a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.663+0000 7f045bfff640 1 -- 172.21.15.83:0/469726072 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f046c0efcd0 con 0x7f047410aaf0 2024-03-11T19:43:59.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047bd8a740 1 -- start start 2024-03-11T19:43:59.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047359e640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f045c0a1c00 0x7f045c0a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0270 0x55f1e50a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 0x55f1e50a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e50a5dd0 0x55f1e50a8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047bd8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e505c960 con 0x55f1e50a5dd0 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047bd8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e5010310 con 0x55f1e50a0c60 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047bd8a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e500d840 con 0x55f1e50a0270 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e50a5dd0 0x55f1e50a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 0x55f1e50a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e50a5dd0 0x55f1e50a8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42978/0 (socket says 172.21.15.83:42978) 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f047359e640 1 -- 172.21.15.83:0/2107738800 learned_addr learned my addr 172.21.15.83:0/2107738800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.664+0000 7f0472d9d640 1 --2- 172.21.15.83:0/2107738800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0270 0x55f1e50a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.665+0000 7f047259c640 1 -- 172.21.15.83:0/2107738800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0270 msgr2=0x55f1e50a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.665+0000 7f047259c640 1 --2- 172.21.15.83:0/2107738800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0270 0x55f1e50a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.665+0000 7f047259c640 1 -- 172.21.15.83:0/2107738800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e50a5dd0 msgr2=0x55f1e50a8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.665+0000 7f047259c640 1 --2- 172.21.15.83:0/2107738800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e50a5dd0 0x55f1e50a8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.665+0000 7f047259c640 1 -- 172.21.15.83:0/2107738800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1e4fcb060 con 0x55f1e50a0c60 2024-03-11T19:43:59.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.665+0000 7f047359e640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f045c0a1c00 0x7f045c0a40c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f04680134b0 tx=0x7f04680138b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.665+0000 7f047259c640 1 --2- 172.21.15.83:0/2107738800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 0x55f1e50a5760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f046c006010 tx=0x7f046c004370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ef55492fd55707 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.665+0000 7f04597fa640 1 -- 172.21.15.83:0/2107738800 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c0c5470 con 0x55f1e50a0c60 2024-03-11T19:43:59.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.665+0000 7f04597fa640 1 -- 172.21.15.83:0/2107738800 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f046c0ed8f0 con 0x55f1e50a0c60 2024-03-11T19:43:59.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.665+0000 7f04597fa640 1 -- 172.21.15.83:0/2107738800 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c0bcf30 con 0x55f1e50a0c60 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.666+0000 7f047bd8a740 1 -- 172.21.15.83:0/2107738800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 msgr2=0x55f1e50a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.666+0000 7f047bd8a740 1 --2- 172.21.15.83:0/2107738800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 0x55f1e50a5760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f046c006010 tx=0x7f046c004370 comp rx=0 tx=0).stop 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.666+0000 7f047bd8a740 1 -- 172.21.15.83:0/2107738800 shutdown_connections 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.666+0000 7f047bd8a740 1 --2- 172.21.15.83:0/2107738800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e50a5dd0 0x55f1e50a8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.666+0000 7f047bd8a740 1 --2- 172.21.15.83:0/2107738800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 0x55f1e50a5760 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.666+0000 7f047bd8a740 1 --2- 172.21.15.83:0/2107738800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0270 0x55f1e50a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.666+0000 7f047bd8a740 1 -- 172.21.15.83:0/2107738800 >> 172.21.15.83:0/2107738800 conn(0x55f1e509b7e0 msgr2=0x55f1e509bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.666+0000 7f047bd8a740 1 -- 172.21.15.83:0/2107738800 shutdown_connections 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.666+0000 7f047bd8a740 1 -- 172.21.15.83:0/2107738800 wait complete. 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.666+0000 7f047bd8a740 1 -- start start 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.666+0000 7f047bd8a740 1 Processor -- start 2024-03-11T19:43:59.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e50a0c60 0x55f1e50aa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a5dd0 0x55f1e50b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f0472d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f0472d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50a9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47150/0 (socket says 172.21.15.83:47150) 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f0472d9d640 1 -- 172.21.15.83:0/3314299066 learned_addr learned my addr 172.21.15.83:0/3314299066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f047359e640 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a5dd0 0x55f1e50b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f047bd8a740 1 -- 172.21.15.83:0/3314299066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e5009020 con 0x55f1e50a0c60 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f047bd8a740 1 -- 172.21.15.83:0/3314299066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e4cdc430 con 0x55f1e50a0270 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f047bd8a740 1 -- 172.21.15.83:0/3314299066 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e4c98e00 con 0x55f1e50a5dd0 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f047259c640 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e50a0c60 0x55f1e50aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f047bd8a740 10 client.0 _ll_register_callbacks cb 0x55f1e4fb0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f0472d9d640 1 -- 172.21.15.83:0/3314299066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a5dd0 msgr2=0x55f1e50b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f0472d9d640 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a5dd0 0x55f1e50b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f0472d9d640 1 -- 172.21.15.83:0/3314299066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e50a0c60 msgr2=0x55f1e50aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f0472d9d640 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e50a0c60 0x55f1e50aa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f0472d9d640 1 -- 172.21.15.83:0/3314299066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04741ae540 con 0x55f1e50a0270 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.667+0000 7f047bd8a740 1 -- 172.21.15.83:0/3314299066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1e4d087e0 con 0x55f1e50a0270 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.668+0000 7f0472d9d640 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50a9fd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0464013710 tx=0x7f0464013bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131aadebacb61dd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.668+0000 7f0456ff5640 10 client.25736 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.668+0000 7f0456ff5640 1 -- 172.21.15.83:0/3314299066 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0464011dd0 con 0x55f1e50a0270 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.668+0000 7f0456ff5640 1 -- 172.21.15.83:0/3314299066 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f046401c440 con 0x55f1e50a0270 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.668+0000 7f0456ff5640 1 -- 172.21.15.83:0/3314299066 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f046401c660 con 0x55f1e50a0270 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.668+0000 7f047bd8a740 10 client.25736 fetch_fsmap learned FSMap version 29 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.668+0000 7f047bd8a740 1 -- 172.21.15.83:0/3314299066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55f1e50076a0 con 0x55f1e50a0270 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.669+0000 7f0456ff5640 1 -- 172.21.15.83:0/3314299066 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0464026470 con 0x55f1e50a0270 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.669+0000 7f047bd8a740 10 client.25736 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.669+0000 7f047bd8a740 10 client.25736 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.669+0000 7f047bd8a740 20 client.25736 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:59.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.669+0000 7f047bd8a740 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55f1e50b6000 0x55f1e50b8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.669+0000 7f047bd8a740 4 client.25736 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.669+0000 7f047bd8a740 1 -- 172.21.15.83:0/3314299066 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1e4d45720 con 0x55f1e50b6000 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.669+0000 7f047259c640 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55f1e50b6000 0x55f1e50b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.669+0000 7f0456ff5640 10 client.25736 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.669+0000 7f047259c640 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55f1e50b6000 0x55f1e50b8410 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4682ab4abf6df1f5 server_cookie=805571dcd1f90e6b in_seq=0 out_seq=0 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.670+0000 7f0456ff5640 1 -- 172.21.15.83:0/3314299066 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f1e4d45720 con 0x55f1e50b6000 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.670+0000 7f0456ff5640 10 client.25736 handle_command_reply: tid=0 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.670+0000 7f047bd8a740 1 client.25736 shutdown 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.670+0000 7f047bd8a740 1 -- 172.21.15.83:0/3314299066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 msgr2=0x55f1e50a9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.670+0000 7f047bd8a740 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50a9fd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0464013710 tx=0x7f0464013bd0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.670+0000 7f047bd8a740 1 -- 172.21.15.83:0/3314299066 shutdown_connections 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.670+0000 7f047bd8a740 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55f1e50b6000 0x55f1e50b8410 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.670+0000 7f047bd8a740 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a5dd0 0x55f1e50b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.670+0000 7f047bd8a740 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e50a0c60 0x55f1e50aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.671+0000 7f047bd8a740 1 --2- 172.21.15.83:0/3314299066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50a9fd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.671+0000 7f047bd8a740 1 -- 172.21.15.83:0/3314299066 >> 172.21.15.83:0/3314299066 conn(0x55f1e509b7e0 msgr2=0x55f1e509ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.671+0000 7f0456ff5640 0 client.25736 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:43:59.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.671+0000 7f047bd8a740 1 -- 172.21.15.83:0/3314299066 shutdown_connections 2024-03-11T19:43:59.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.671+0000 7f047bd8a740 1 -- 172.21.15.83:0/3314299066 wait complete. 2024-03-11T19:43:59.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.671+0000 7f047bd8a740 20 client.25736 trim_cache size 0 max 16384 2024-03-11T19:43:59.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.686+0000 7f047bd8a740 1 Processor -- start 2024-03-11T19:43:59.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.686+0000 7f047bd8a740 1 -- start start 2024-03-11T19:43:59.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.686+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e509b7e0 0x55f1e50a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.686+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0270 0x55f1e50b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.686+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 0x55f1e50a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.687+0000 7f047bd8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e4f8e130 con 0x55f1e509b7e0 2024-03-11T19:43:59.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.687+0000 7f047259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0270 0x55f1e50b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.687+0000 7f047259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0270 0x55f1e50b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:44984/0 (socket says 172.21.15.83:44984) 2024-03-11T19:43:59.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.687+0000 7f047259c640 1 -- 172.21.15.83:0/2848401789 learned_addr learned my addr 172.21.15.83:0/2848401789 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.687+0000 7f047359e640 1 --2- 172.21.15.83:0/2848401789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 0x55f1e50a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.687+0000 7f0472d9d640 1 --2- 172.21.15.83:0/2848401789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e509b7e0 0x55f1e50a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.687+0000 7f047bd8a740 1 -- 172.21.15.83:0/2848401789 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e50aad00 con 0x55f1e50a0c60 2024-03-11T19:43:59.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.687+0000 7f047bd8a740 1 -- 172.21.15.83:0/2848401789 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e50975c0 con 0x55f1e50a0270 2024-03-11T19:43:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.688+0000 7f047359e640 1 -- 172.21.15.83:0/2848401789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0270 msgr2=0x55f1e50b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.688+0000 7f047359e640 1 --2- 172.21.15.83:0/2848401789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0270 0x55f1e50b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.688+0000 7f047359e640 1 -- 172.21.15.83:0/2848401789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e509b7e0 msgr2=0x55f1e50a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.688+0000 7f047359e640 1 --2- 172.21.15.83:0/2848401789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e509b7e0 0x55f1e50a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.688+0000 7f047359e640 1 -- 172.21.15.83:0/2848401789 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1e4fa3c20 con 0x55f1e50a0c60 2024-03-11T19:43:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.688+0000 7f047359e640 1 --2- 172.21.15.83:0/2848401789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 0x55f1e50a9b30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0468013db0 tx=0x7f0468020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319d4e0756671adc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.688+0000 7f0458ff9640 1 -- 172.21.15.83:0/2848401789 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468018040 con 0x55f1e50a0c60 2024-03-11T19:43:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.688+0000 7f0458ff9640 1 -- 172.21.15.83:0/2848401789 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0468010c90 con 0x55f1e50a0c60 2024-03-11T19:43:59.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.688+0000 7f0458ff9640 1 -- 172.21.15.83:0/2848401789 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046802bb50 con 0x55f1e50a0c60 2024-03-11T19:43:59.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.689+0000 7f047bd8a740 1 -- 172.21.15.83:0/2848401789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 msgr2=0x55f1e50a9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.689+0000 7f047bd8a740 1 --2- 172.21.15.83:0/2848401789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 0x55f1e50a9b30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0468013db0 tx=0x7f0468020000 comp rx=0 tx=0).stop 2024-03-11T19:43:59.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.689+0000 7f047bd8a740 1 -- 172.21.15.83:0/2848401789 shutdown_connections 2024-03-11T19:43:59.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.689+0000 7f047bd8a740 1 --2- 172.21.15.83:0/2848401789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0c60 0x55f1e50a9b30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.689+0000 7f047bd8a740 1 --2- 172.21.15.83:0/2848401789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0270 0x55f1e50b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.689+0000 7f047bd8a740 1 --2- 172.21.15.83:0/2848401789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e509b7e0 0x55f1e50a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.689+0000 7f047bd8a740 1 -- 172.21.15.83:0/2848401789 >> 172.21.15.83:0/2848401789 conn(0x55f1e50a58a0 msgr2=0x55f1e50a1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.689+0000 7f047bd8a740 1 -- 172.21.15.83:0/2848401789 shutdown_connections 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.689+0000 7f047bd8a740 1 -- 172.21.15.83:0/2848401789 wait complete. 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.690+0000 7f047bd8a740 1 -- start start 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.690+0000 7f047bd8a740 1 Processor -- start 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.690+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e509b7e0 0x55f1e50b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.690+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50af960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.690+0000 7f047bd8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0c60 0x55f1e50afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.690+0000 7f047bd8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e50b0480 con 0x55f1e509b7e0 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.690+0000 7f0472d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e509b7e0 0x55f1e50b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.690+0000 7f047259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.690+0000 7f047259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50af960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47162/0 (socket says 172.21.15.83:47162) 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.690+0000 7f047259c640 1 -- 172.21.15.83:0/443660486 learned_addr learned my addr 172.21.15.83:0/443660486 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.690+0000 7f047359e640 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0c60 0x55f1e50afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.691+0000 7f047bd8a740 1 -- 172.21.15.83:0/443660486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e50b05f0 con 0x55f1e50a0270 2024-03-11T19:43:59.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.691+0000 7f047bd8a740 1 -- 172.21.15.83:0/443660486 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1e50b07b0 con 0x55f1e50a0c60 2024-03-11T19:43:59.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.691+0000 7f047259c640 1 -- 172.21.15.83:0/443660486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0c60 msgr2=0x55f1e50afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.691+0000 7f047259c640 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0c60 0x55f1e50afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.691+0000 7f047259c640 1 -- 172.21.15.83:0/443660486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e509b7e0 msgr2=0x55f1e50b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.691+0000 7f047259c640 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e509b7e0 0x55f1e50b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.691+0000 7f047259c640 1 -- 172.21.15.83:0/443660486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0468012290 con 0x55f1e50a0270 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f047259c640 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50af960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f046c0c4f40 tx=0x7f046c0042d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b68998092cad710 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f0456ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f047bd8a740 10 client.0 _ll_register_callbacks cb 0x7f0464009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f0456ff5640 1 -- 172.21.15.83:0/443660486 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c0bcaa0 con 0x55f1e50a0270 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f0456ff5640 1 -- 172.21.15.83:0/443660486 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f046c0c5470 con 0x55f1e50a0270 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f047bd8a740 1 -- 172.21.15.83:0/443660486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1e50970f0 con 0x55f1e50a0270 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f0456ff5640 1 -- 172.21.15.83:0/443660486 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f046c0b9340 con 0x55f1e50a0270 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f047bd8a740 10 client.25742 fetch_fsmap learned FSMap version 29 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f047bd8a740 1 -- 172.21.15.83:0/443660486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55f1e4fc98c0 con 0x55f1e50a0270 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f0456ff5640 1 -- 172.21.15.83:0/443660486 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f046c0d0d20 con 0x55f1e50a0270 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f047bd8a740 10 client.25742 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f047bd8a740 10 client.25742 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:43:59.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f047bd8a740 20 client.25742 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f047bd8a740 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55f1e50a5dd0 0x55f1e510f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f047bd8a740 4 client.25742 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.692+0000 7f047bd8a740 1 -- 172.21.15.83:0/443660486 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f046c0ed8f0 con 0x55f1e50a5dd0 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.693+0000 7f0472d9d640 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55f1e50a5dd0 0x55f1e510f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.693+0000 7f0472d9d640 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55f1e50a5dd0 0x55f1e510f5b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51e8114fedf24bbd server_cookie=f90843b6370dc7d6 in_seq=0 out_seq=0 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.693+0000 7f0456ff5640 10 client.25742 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.693+0000 7f0456ff5640 1 -- 172.21.15.83:0/443660486 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f046c0ed8f0 con 0x55f1e50a5dd0 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.693+0000 7f0456ff5640 10 client.25742 handle_command_reply: tid=0 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.693+0000 7f047bd8a740 1 client.25742 shutdown 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.693+0000 7f047bd8a740 1 -- 172.21.15.83:0/443660486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 msgr2=0x55f1e50af960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.693+0000 7f047bd8a740 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50af960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f046c0c4f40 tx=0x7f046c0042d0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.694+0000 7f047bd8a740 1 -- 172.21.15.83:0/443660486 shutdown_connections 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.694+0000 7f047bd8a740 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55f1e50a5dd0 0x55f1e510f5b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.694+0000 7f047bd8a740 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1e50a0c60 0x55f1e50afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.694+0000 7f0456ff5640 0 client.25742 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:43:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.694+0000 7f047bd8a740 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1e50a0270 0x55f1e50af960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.694+0000 7f047bd8a740 1 --2- 172.21.15.83:0/443660486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1e509b7e0 0x55f1e50b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.694+0000 7f047bd8a740 1 -- 172.21.15.83:0/443660486 >> 172.21.15.83:0/443660486 conn(0x55f1e50a58a0 msgr2=0x55f1e509ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.694+0000 7f047bd8a740 1 -- 172.21.15.83:0/443660486 shutdown_connections 2024-03-11T19:43:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.694+0000 7f047bd8a740 1 -- 172.21.15.83:0/443660486 wait complete. 2024-03-11T19:43:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.694+0000 7f047bd8a740 20 client.25742 trim_cache size 0 max 16384 2024-03-11T19:43:59.702 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:43:59.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.695+0000 7f04791c2640 1 -- 172.21.15.83:0/469726072 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f045c0a1c00 msgr2=0x7f045c0a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.695+0000 7f04791c2640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f045c0a1c00 0x7f045c0a40c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f04680134b0 tx=0x7f04680138b0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.695+0000 7f04791c2640 1 -- 172.21.15.83:0/469726072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f047410aaf0 msgr2=0x7f04741a6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.695+0000 7f04791c2640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f047410aaf0 0x7f04741a6650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f046c009bf0 tx=0x7f046c032fa0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.696+0000 7f04791c2640 1 -- 172.21.15.83:0/469726072 shutdown_connections 2024-03-11T19:43:59.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.696+0000 7f04791c2640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f045c0a1c00 0x7f045c0a40c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.696+0000 7f04791c2640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f047410d4a0 0x7f04741ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.696+0000 7f04791c2640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f047410aaf0 0x7f04741a6650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.696+0000 7f04791c2640 1 --2- 172.21.15.83:0/469726072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474108180 0x7f04741a6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.696+0000 7f04791c2640 1 -- 172.21.15.83:0/469726072 >> 172.21.15.83:0/469726072 conn(0x7f047406dfe0 msgr2=0x7f04741085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.696+0000 7f04791c2640 1 -- 172.21.15.83:0/469726072 shutdown_connections 2024-03-11T19:43:59.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.696+0000 7f04791c2640 1 -- 172.21.15.83:0/469726072 wait complete. 2024-03-11T19:43:59.716 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:43:59.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:43:59.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:43:59 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2855547334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:43:59.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.865+0000 7f4247eb5640 1 -- 172.21.15.83:0/2470054764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4240109350 msgr2=0x7f424010fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.865+0000 7f4247eb5640 1 --2- 172.21.15.83:0/2470054764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4240109350 0x7f424010fbe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4238009ca0 tx=0x7f423802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.865+0000 7f4247eb5640 1 -- 172.21.15.83:0/2470054764 shutdown_connections 2024-03-11T19:43:59.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.865+0000 7f4247eb5640 1 --2- 172.21.15.83:0/2470054764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4240109350 0x7f424010fbe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.865+0000 7f4247eb5640 1 --2- 172.21.15.83:0/2470054764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4240108990 0x7f4240108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.865+0000 7f4247eb5640 1 --2- 172.21.15.83:0/2470054764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42401077a0 0x7f4240107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.865+0000 7f4247eb5640 1 -- 172.21.15.83:0/2470054764 >> 172.21.15.83:0/2470054764 conn(0x7f4240076720 msgr2=0x7f4240078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.865+0000 7f4247eb5640 1 -- 172.21.15.83:0/2470054764 shutdown_connections 2024-03-11T19:43:59.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.865+0000 7f4247eb5640 1 -- 172.21.15.83:0/2470054764 wait complete. 2024-03-11T19:43:59.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.866+0000 7f4247eb5640 1 Processor -- start 2024-03-11T19:43:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4247eb5640 1 -- start start 2024-03-11T19:43:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4247eb5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42401077a0 0x7f424019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4247eb5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4240108990 0x7f424019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4247eb5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4240109350 0x7f42401a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4247eb5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401a5300 con 0x7f42401077a0 2024-03-11T19:43:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4247eb5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401a5470 con 0x7f4240109350 2024-03-11T19:43:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4247eb5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401a5630 con 0x7f4240108990 2024-03-11T19:43:59.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4245c51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42401077a0 0x7f424019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4245c51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42401077a0 0x7f424019d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43032/0 (socket says 172.21.15.83:43032) 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4245c51640 1 -- 172.21.15.83:0/4045112563 learned_addr learned my addr 172.21.15.83:0/4045112563 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4245450640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4240108990 0x7f424019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.867+0000 7f4246452640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4240109350 0x7f42401a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4245c51640 1 -- 172.21.15.83:0/4045112563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4240108990 msgr2=0x7f424019dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4245c51640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4240108990 0x7f424019dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4245c51640 1 -- 172.21.15.83:0/4045112563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4240109350 msgr2=0x7f42401a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4245c51640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4240109350 0x7f42401a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4245c51640 1 -- 172.21.15.83:0/4045112563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4238009900 con 0x7f42401077a0 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4246452640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4240109350 0x7f42401a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4245450640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4240108990 0x7f424019dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4245c51640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42401077a0 0x7f424019d810 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f423c00e9c0 tx=0x7f423c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4236ffd640 1 -- 172.21.15.83:0/4045112563 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0096e0 con 0x7f42401077a0 2024-03-11T19:43:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4236ffd640 1 -- 172.21.15.83:0/4045112563 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f423c00cd80 con 0x7f42401077a0 2024-03-11T19:43:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4247eb5640 1 -- 172.21.15.83:0/4045112563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42401a5910 con 0x7f42401077a0 2024-03-11T19:43:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4236ffd640 1 -- 172.21.15.83:0/4045112563 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c010430 con 0x7f42401077a0 2024-03-11T19:43:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.868+0000 7f4247eb5640 1 -- 172.21.15.83:0/4045112563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42401a5e20 con 0x7f42401077a0 2024-03-11T19:43:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.871+0000 7f4236ffd640 1 -- 172.21.15.83:0/4045112563 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f423c003680 con 0x7f42401077a0 2024-03-11T19:43:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.872+0000 7f424ab63740 1 Processor -- start 2024-03-11T19:43:59.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.872+0000 7f424ab63740 1 -- start start 2024-03-11T19:43:59.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.872+0000 7f424ab63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6d5270 0x559dff6d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.872+0000 7f424ab63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 0x559dff6da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.872+0000 7f424ab63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6dadd0 0x559dff6dd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.872+0000 7f424ab63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff691960 con 0x559dff6d5c60 2024-03-11T19:43:59.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.872+0000 7f424ab63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff645310 con 0x559dff6d5270 2024-03-11T19:43:59.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.872+0000 7f424ab63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff642840 con 0x559dff6dadd0 2024-03-11T19:43:59.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4245c51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 0x559dff6da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4245c51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 0x559dff6da760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43046/0 (socket says 172.21.15.83:43046) 2024-03-11T19:43:59.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4245c51640 1 -- 172.21.15.83:0/3409749990 learned_addr learned my addr 172.21.15.83:0/3409749990 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4246452640 1 --2- 172.21.15.83:0/3409749990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6d5270 0x559dff6d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4236ffd640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f422c0a1b30 0x7f422c0a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4246452640 1 -- 172.21.15.83:0/3409749990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6dadd0 msgr2=0x559dff6dd260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:43:59.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4246452640 1 --2- 172.21.15.83:0/3409749990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6dadd0 0x559dff6dd260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4246452640 1 -- 172.21.15.83:0/3409749990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 msgr2=0x559dff6da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4246452640 1 --2- 172.21.15.83:0/3409749990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 0x559dff6da760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4246452640 1 -- 172.21.15.83:0/3409749990 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dff600060 con 0x559dff6d5270 2024-03-11T19:43:59.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4245c51640 1 --2- 172.21.15.83:0/3409749990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 0x559dff6da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:59.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4236ffd640 1 -- 172.21.15.83:0/4045112563 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f423c014070 con 0x7f42401077a0 2024-03-11T19:43:59.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4246452640 1 --2- 172.21.15.83:0/3409749990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6d5270 0x559dff6d5690 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f423802c630 tx=0x7f4238005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=577c873c83346d53 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.873+0000 7f4246452640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f422c0a1b30 0x7f422c0a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.874+0000 7f422bfff640 1 -- 172.21.15.83:0/3409749990 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423804f070 con 0x559dff6d5270 2024-03-11T19:43:59.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.874+0000 7f422bfff640 1 -- 172.21.15.83:0/3409749990 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4238003970 con 0x559dff6d5270 2024-03-11T19:43:59.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.874+0000 7f422bfff640 1 -- 172.21.15.83:0/3409749990 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380043a0 con 0x559dff6d5270 2024-03-11T19:43:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 -- 172.21.15.83:0/3409749990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6d5270 msgr2=0x559dff6d5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f4246452640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f422c0a1b30 0x7f422c0a3ff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f42380098d0 tx=0x7f423804d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 --2- 172.21.15.83:0/3409749990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6d5270 0x559dff6d5690 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f423802c630 tx=0x7f4238005f70 comp rx=0 tx=0).stop 2024-03-11T19:43:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 -- 172.21.15.83:0/3409749990 shutdown_connections 2024-03-11T19:43:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 --2- 172.21.15.83:0/3409749990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6dadd0 0x559dff6dd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 --2- 172.21.15.83:0/3409749990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 0x559dff6da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 --2- 172.21.15.83:0/3409749990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6d5270 0x559dff6d5690 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 -- 172.21.15.83:0/3409749990 >> 172.21.15.83:0/3409749990 conn(0x559dff6d07e0 msgr2=0x559dff6d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 -- 172.21.15.83:0/3409749990 shutdown_connections 2024-03-11T19:43:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 -- 172.21.15.83:0/3409749990 wait complete. 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 -- start start 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 Processor -- start 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.875+0000 7f424ab63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 0x559dff6df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f424ab63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5c60 0x559dff6df990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f424ab63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6dadd0 0x559dff6e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f424ab63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff63e020 con 0x559dff6d5270 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f424ab63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff311430 con 0x559dff6dadd0 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f424ab63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff2cde00 con 0x559dff6d5c60 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f4245c51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 0x559dff6df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f4245c51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 0x559dff6df450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43060/0 (socket says 172.21.15.83:43060) 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f4245c51640 1 -- 172.21.15.83:0/3241034023 learned_addr learned my addr 172.21.15.83:0/3241034023 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f424ab63740 10 client.0 _ll_register_callbacks cb 0x559dff5e5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f4245450640 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5c60 0x559dff6df990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f4246452640 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6dadd0 0x559dff6e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f4245c51640 1 -- 172.21.15.83:0/3241034023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5c60 msgr2=0x559dff6df990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f4245c51640 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5c60 0x559dff6df990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f4245c51640 1 -- 172.21.15.83:0/3241034023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6dadd0 msgr2=0x559dff6e69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f4245c51640 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6dadd0 0x559dff6e69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f4245c51640 1 -- 172.21.15.83:0/3241034023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42401104e0 con 0x559dff6d5270 2024-03-11T19:43:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f4245c51640 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 0x559dff6df450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f423c00c6a0 tx=0x7f423c0cb220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c725fa4a002ed1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f42297fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:43:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f42297fa640 1 -- 172.21.15.83:0/3241034023 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0a2c60 con 0x559dff6d5270 2024-03-11T19:43:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.876+0000 7f42297fa640 1 -- 172.21.15.83:0/3241034023 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f423c09ae50 con 0x559dff6d5270 2024-03-11T19:43:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.877+0000 7f424ab63740 1 -- 172.21.15.83:0/3241034023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dff33d7e0 con 0x559dff6d5270 2024-03-11T19:43:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.877+0000 7f42297fa640 1 -- 172.21.15.83:0/3241034023 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f423c09a290 con 0x559dff6d5270 2024-03-11T19:43:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.878+0000 7f424ab63740 10 client.16263 fetch_fsmap learned FSMap version 29 2024-03-11T19:43:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.878+0000 7f424ab63740 1 -- 172.21.15.83:0/3241034023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559dff63c6a0 con 0x559dff6d5270 2024-03-11T19:43:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.878+0000 7f42297fa640 1 -- 172.21.15.83:0/3241034023 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f423c09a0b0 con 0x559dff6d5270 2024-03-11T19:43:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.878+0000 7f42297fa640 1 -- 172.21.15.83:0/3241034023 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f423c01f060 con 0x559dff6d5270 2024-03-11T19:43:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.879+0000 7f424ab63740 10 client.16263 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:43:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.879+0000 7f424ab63740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.879+0000 7f424ab63740 20 client.16263 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.879+0000 7f424ab63740 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559dff6eb510 0x559dff6ed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.879+0000 7f424ab63740 4 client.16263 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.879+0000 7f424ab63740 1 -- 172.21.15.83:0/3241034023 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dff33d7e0 con 0x559dff6eb510 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.879+0000 7f4245450640 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559dff6eb510 0x559dff6ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.879+0000 7f42297fa640 10 client.16263 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.879+0000 7f4245450640 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559dff6eb510 0x559dff6ed9a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ded76610a3cc8b server_cookie=959b1ad11d91186a in_seq=0 out_seq=0 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f42297fa640 1 -- 172.21.15.83:0/3241034023 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559dff33d7e0 con 0x559dff6eb510 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f42297fa640 10 client.16263 handle_command_reply: tid=0 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f424ab63740 1 client.16263 shutdown 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f424ab63740 1 -- 172.21.15.83:0/3241034023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 msgr2=0x559dff6df450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f424ab63740 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 0x559dff6df450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f423c00c6a0 tx=0x7f423c0cb220 comp rx=0 tx=0).stop 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f424ab63740 1 -- 172.21.15.83:0/3241034023 shutdown_connections 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f424ab63740 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559dff6eb510 0x559dff6ed9a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f424ab63740 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6dadd0 0x559dff6e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f42297fa640 0 client.16263 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f424ab63740 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5c60 0x559dff6df990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f424ab63740 1 --2- 172.21.15.83:0/3241034023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 0x559dff6df450 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f424ab63740 1 -- 172.21.15.83:0/3241034023 >> 172.21.15.83:0/3241034023 conn(0x559dff6d07e0 msgr2=0x559dff6cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f424ab63740 1 -- 172.21.15.83:0/3241034023 shutdown_connections 2024-03-11T19:43:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.880+0000 7f424ab63740 1 -- 172.21.15.83:0/3241034023 wait complete. 2024-03-11T19:43:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.881+0000 7f424ab63740 20 client.16263 trim_cache size 0 max 16384 2024-03-11T19:43:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f424ab63740 1 Processor -- start 2024-03-11T19:43:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f424ab63740 1 -- start start 2024-03-11T19:43:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f424ab63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 0x559dff6d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f424ab63740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6d5c60 0x559dff6e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f424ab63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6dadd0 0x559dff6df060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f424ab63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff5c3130 con 0x559dff6d5270 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f424ab63740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff6cc120 con 0x559dff6d5c60 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f424ab63740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff6d6350 con 0x559dff6dadd0 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f4245c51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 0x559dff6d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f4245c51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 0x559dff6d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43064/0 (socket says 172.21.15.83:43064) 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f4246452640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6dadd0 0x559dff6df060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.897+0000 7f4245c51640 1 -- 172.21.15.83:0/1846620970 learned_addr learned my addr 172.21.15.83:0/1846620970 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.898+0000 7f4245450640 1 --2- 172.21.15.83:0/1846620970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6d5c60 0x559dff6e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.898+0000 7f4246452640 1 -- 172.21.15.83:0/1846620970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6d5c60 msgr2=0x559dff6e6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.898+0000 7f4246452640 1 --2- 172.21.15.83:0/1846620970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6d5c60 0x559dff6e6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.898+0000 7f4246452640 1 -- 172.21.15.83:0/1846620970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 msgr2=0x559dff6d0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.898+0000 7f4246452640 1 --2- 172.21.15.83:0/1846620970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 0x559dff6d0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.898+0000 7f4246452640 1 -- 172.21.15.83:0/1846620970 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dff5d8c20 con 0x559dff6dadd0 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.898+0000 7f4246452640 1 --2- 172.21.15.83:0/1846620970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6dadd0 0x559dff6df060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4238009ca0 tx=0x7f4238051000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b913cec3228a9a0c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.898+0000 7f422b7fe640 1 -- 172.21.15.83:0/1846620970 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423804f040 con 0x559dff6dadd0 2024-03-11T19:43:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.898+0000 7f422b7fe640 1 -- 172.21.15.83:0/1846620970 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42380346e0 con 0x559dff6dadd0 2024-03-11T19:43:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.898+0000 7f422b7fe640 1 -- 172.21.15.83:0/1846620970 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238003ed0 con 0x559dff6dadd0 2024-03-11T19:43:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.899+0000 7f424ab63740 1 -- 172.21.15.83:0/1846620970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6dadd0 msgr2=0x559dff6df060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.899+0000 7f424ab63740 1 --2- 172.21.15.83:0/1846620970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6dadd0 0x559dff6df060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4238009ca0 tx=0x7f4238051000 comp rx=0 tx=0).stop 2024-03-11T19:43:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.899+0000 7f424ab63740 1 -- 172.21.15.83:0/1846620970 shutdown_connections 2024-03-11T19:43:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.899+0000 7f424ab63740 1 --2- 172.21.15.83:0/1846620970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6dadd0 0x559dff6df060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.899+0000 7f424ab63740 1 --2- 172.21.15.83:0/1846620970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6d5c60 0x559dff6e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.899+0000 7f424ab63740 1 --2- 172.21.15.83:0/1846620970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5270 0x559dff6d0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.899+0000 7f424ab63740 1 -- 172.21.15.83:0/1846620970 >> 172.21.15.83:0/1846620970 conn(0x559dff6d07e0 msgr2=0x559dff6cba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.899+0000 7f424ab63740 1 -- 172.21.15.83:0/1846620970 shutdown_connections 2024-03-11T19:43:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.899+0000 7f424ab63740 1 -- 172.21.15.83:0/1846620970 wait complete. 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.900+0000 7f424ab63740 1 -- start start 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.900+0000 7f424ab63740 1 Processor -- start 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.900+0000 7f424ab63740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5270 0x559dff6e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f424ab63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 0x559dff6e4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f4245c51640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5270 0x559dff6e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f4245c51640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5270 0x559dff6e6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45042/0 (socket says 172.21.15.83:45042) 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f4245c51640 1 -- 172.21.15.83:0/544063838 learned_addr learned my addr 172.21.15.83:0/544063838 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f424ab63740 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6dadd0 0x559dff6e5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f424ab63740 1 -- 172.21.15.83:0/544063838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff6e5d10 con 0x559dff6d5c60 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f424ab63740 1 -- 172.21.15.83:0/544063838 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff6e5e80 con 0x559dff6dadd0 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f424ab63740 1 -- 172.21.15.83:0/544063838 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dff741b00 con 0x559dff6d5270 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f424ab63740 10 client.0 _ll_register_callbacks cb 0x7f4238037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:43:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f4245450640 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 0x559dff6e4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f4246452640 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6dadd0 0x559dff6e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f4245450640 1 -- 172.21.15.83:0/544063838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5270 msgr2=0x559dff6e6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f4245450640 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5270 0x559dff6e6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f4245450640 1 -- 172.21.15.83:0/544063838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6dadd0 msgr2=0x559dff6e5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f4245450640 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6dadd0 0x559dff6e5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f4245450640 1 -- 172.21.15.83:0/544063838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559dff600060 con 0x559dff6d5c60 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.901+0000 7f4245c51640 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5270 0x559dff6e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f4245450640 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 0x559dff6e4fe0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x559dff6ee2f0 tx=0x7f423001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987e6e4bc6ff2def server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f42297fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f42297fa640 1 -- 172.21.15.83:0/544063838 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4230016040 con 0x559dff6d5c60 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f42297fa640 1 -- 172.21.15.83:0/544063838 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4230004b40 con 0x559dff6d5c60 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f424ab63740 1 -- 172.21.15.83:0/544063838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4238003970 con 0x559dff6d5c60 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f42297fa640 1 -- 172.21.15.83:0/544063838 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4230004d00 con 0x559dff6d5c60 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f424ab63740 10 client.16269 fetch_fsmap learned FSMap version 29 2024-03-11T19:43:59.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f424ab63740 1 -- 172.21.15.83:0/544063838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559dff5fe8c0 con 0x559dff6d5c60 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f42297fa640 1 -- 172.21.15.83:0/544063838 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4238003970 con 0x559dff6d5c60 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f42297fa640 1 -- 172.21.15.83:0/544063838 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f423002e070 con 0x559dff6d5c60 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f424ab63740 10 client.16269 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f424ab63740 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f424ab63740 20 client.16269 populate_metadata read hostname 'smithi083' 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f424ab63740 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559dff6eb510 0x559dff745590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f424ab63740 4 client.16269 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.902+0000 7f424ab63740 1 -- 172.21.15.83:0/544063838 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559dff37a720 con 0x559dff6eb510 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.903+0000 7f4245c51640 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559dff6eb510 0x559dff745590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.903+0000 7f42297fa640 10 client.16269 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.903+0000 7f4245c51640 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559dff6eb510 0x559dff745590 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=871e4f651165f856 server_cookie=157306068f556908 in_seq=0 out_seq=0 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.903+0000 7f42297fa640 1 -- 172.21.15.83:0/544063838 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559dff37a720 con 0x559dff6eb510 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.903+0000 7f42297fa640 10 client.16269 handle_command_reply: tid=0 2024-03-11T19:43:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.903+0000 7f424ab63740 1 client.16269 shutdown 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.903+0000 7f424ab63740 1 -- 172.21.15.83:0/544063838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 msgr2=0x559dff6e4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.903+0000 7f424ab63740 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 0x559dff6e4fe0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x559dff6ee2f0 tx=0x7f423001c9c0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.904+0000 7f424ab63740 1 -- 172.21.15.83:0/544063838 shutdown_connections 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.904+0000 7f424ab63740 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559dff6eb510 0x559dff745590 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.904+0000 7f424ab63740 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559dff6dadd0 0x559dff6e5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.904+0000 7f424ab63740 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559dff6d5c60 0x559dff6e4fe0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.904+0000 7f42297fa640 0 client.16269 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.904+0000 7f424ab63740 1 --2- 172.21.15.83:0/544063838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559dff6d5270 0x559dff6e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.904+0000 7f424ab63740 1 -- 172.21.15.83:0/544063838 >> 172.21.15.83:0/544063838 conn(0x559dff6d07e0 msgr2=0x559dff6d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.904+0000 7f424ab63740 1 -- 172.21.15.83:0/544063838 shutdown_connections 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.904+0000 7f424ab63740 1 -- 172.21.15.83:0/544063838 wait complete. 2024-03-11T19:43:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.904+0000 7f424ab63740 20 client.16269 trim_cache size 0 max 16384 2024-03-11T19:43:59.913 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:43:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.905+0000 7f4247eb5640 1 -- 172.21.15.83:0/4045112563 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f422c0a1b30 msgr2=0x7f422c0a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.905+0000 7f4247eb5640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f422c0a1b30 0x7f422c0a3ff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f42380098d0 tx=0x7f423804d000 comp rx=0 tx=0).stop 2024-03-11T19:43:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.905+0000 7f4247eb5640 1 -- 172.21.15.83:0/4045112563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42401077a0 msgr2=0x7f424019d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:43:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.905+0000 7f4247eb5640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42401077a0 0x7f424019d810 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f423c00e9c0 tx=0x7f423c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:43:59.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.905+0000 7f4247eb5640 1 -- 172.21.15.83:0/4045112563 shutdown_connections 2024-03-11T19:43:59.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.905+0000 7f4247eb5640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f422c0a1b30 0x7f422c0a3ff0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.905+0000 7f4247eb5640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4240109350 0x7f42401a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.905+0000 7f4247eb5640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4240108990 0x7f424019dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.905+0000 7f4247eb5640 1 --2- 172.21.15.83:0/4045112563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42401077a0 0x7f424019d810 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:43:59.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.905+0000 7f4247eb5640 1 -- 172.21.15.83:0/4045112563 >> 172.21.15.83:0/4045112563 conn(0x7f4240076720 msgr2=0x7f424010bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:43:59.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.906+0000 7f4247eb5640 1 -- 172.21.15.83:0/4045112563 shutdown_connections 2024-03-11T19:43:59.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:43:59.906+0000 7f4247eb5640 1 -- 172.21.15.83:0/4045112563 wait complete. 2024-03-11T19:44:00.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:43:59 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2855547334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:44:00.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:43:59 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2855547334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:44:00.924 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:44:00.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:44:00.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:44:01.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:00 smithi083 ceph-mon[31864]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 MiB/s wr, 781 op/s 2024-03-11T19:44:01.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.077+0000 7fefe4455640 1 -- 172.21.15.83:0/2248818171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefdc101270 msgr2=0x7fefdc1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.077+0000 7fefe4455640 1 --2- 172.21.15.83:0/2248818171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefdc101270 0x7fefdc1016f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fefcc009900 tx=0x7fefcc02bc60 comp rx=0 tx=0).stop 2024-03-11T19:44:01.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.078+0000 7fefe4455640 1 -- 172.21.15.83:0/2248818171 shutdown_connections 2024-03-11T19:44:01.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.078+0000 7fefe4455640 1 --2- 172.21.15.83:0/2248818171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefdc101c30 0x7fefdc0682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.078+0000 7fefe4455640 1 --2- 172.21.15.83:0/2248818171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefdc101270 0x7fefdc1016f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.078+0000 7fefe4455640 1 --2- 172.21.15.83:0/2248818171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefdc100080 0x7fefdc1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.078+0000 7fefe4455640 1 -- 172.21.15.83:0/2248818171 >> 172.21.15.83:0/2248818171 conn(0x7fefdc0fb610 msgr2=0x7fefdc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.082+0000 7fefe4455640 1 -- 172.21.15.83:0/2248818171 shutdown_connections 2024-03-11T19:44:01.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.085+0000 7fefe4455640 1 -- 172.21.15.83:0/2248818171 wait complete. 2024-03-11T19:44:01.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.085+0000 7fefe4455640 1 Processor -- start 2024-03-11T19:44:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.086+0000 7fefe4455640 1 -- start start 2024-03-11T19:44:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.086+0000 7fefe4455640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefdc100080 0x7fefdc06fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.086+0000 7fefe4455640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefdc101270 0x7fefdc070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.086+0000 7fefe4455640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefdc101c30 0x7fefdc1a9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.086+0000 7fefe4455640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc070580 con 0x7fefdc101c30 2024-03-11T19:44:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.086+0000 7fefe4455640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc0706f0 con 0x7fefdc100080 2024-03-11T19:44:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.086+0000 7fefe19f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefdc101270 0x7fefdc070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.086+0000 7fefe19f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefdc101270 0x7fefdc070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45066/0 (socket says 172.21.15.83:45066) 2024-03-11T19:44:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.086+0000 7fefe19f0640 1 -- 172.21.15.83:0/168115306 learned_addr learned my addr 172.21.15.83:0/168115306 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.086+0000 7fefe29f2640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefdc101c30 0x7fefdc1a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.086+0000 7fefe4455640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc1a9cb0 con 0x7fefdc101270 2024-03-11T19:44:01.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.087+0000 7fefe29f2640 1 -- 172.21.15.83:0/168115306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefdc101270 msgr2=0x7fefdc070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.087+0000 7fefe29f2640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefdc101270 0x7fefdc070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.087+0000 7fefe29f2640 1 -- 172.21.15.83:0/168115306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefdc100080 msgr2=0x7fefdc06fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.087+0000 7fefe29f2640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefdc100080 0x7fefdc06fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.087+0000 7fefe29f2640 1 -- 172.21.15.83:0/168115306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc009560 con 0x7fefdc101c30 2024-03-11T19:44:01.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.087+0000 7fefe19f0640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefdc101270 0x7fefdc070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:01.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.088+0000 7fefe29f2640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefdc101c30 0x7fefdc1a9770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fefd800b440 tx=0x7fefd800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.088+0000 7fefd37fe640 1 -- 172.21.15.83:0/168115306 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8011830 con 0x7fefdc101c30 2024-03-11T19:44:01.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.088+0000 7fefd37fe640 1 -- 172.21.15.83:0/168115306 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefd8011e70 con 0x7fefdc101c30 2024-03-11T19:44:01.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.088+0000 7fefd37fe640 1 -- 172.21.15.83:0/168115306 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800f2e0 con 0x7fefdc101c30 2024-03-11T19:44:01.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.088+0000 7fefe4455640 1 -- 172.21.15.83:0/168115306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc1a9f90 con 0x7fefdc101c30 2024-03-11T19:44:01.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.088+0000 7fefe4455640 1 -- 172.21.15.83:0/168115306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc1aa300 con 0x7fefdc101c30 2024-03-11T19:44:01.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.092+0000 7fefe702c740 1 Processor -- start 2024-03-11T19:44:01.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.092+0000 7fefe702c740 1 -- start start 2024-03-11T19:44:01.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.093+0000 7fefe702c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 0x55d722bc9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.093+0000 7fefe702c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bca1b0 0x55d722bca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.093+0000 7fefe702c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bcab70 0x55d722bd1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.093+0000 7fefe702c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d722b2ff90 con 0x55d722bcab70 2024-03-11T19:44:01.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.093+0000 7fefe702c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d722b2d4c0 con 0x55d722bca1b0 2024-03-11T19:44:01.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.093+0000 7fefe702c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d722b28ca0 con 0x55d722bc8fe0 2024-03-11T19:44:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.094+0000 7fefe21f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bca1b0 0x55d722bca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.094+0000 7fefe21f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bca1b0 0x55d722bca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47244/0 (socket says 172.21.15.83:47244) 2024-03-11T19:44:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.094+0000 7fefe21f1640 1 -- 172.21.15.83:0/857439224 learned_addr learned my addr 172.21.15.83:0/857439224 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.094+0000 7fefe19f0640 1 --2- 172.21.15.83:0/857439224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 0x55d722bc9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.094+0000 7fefe19f0640 1 --2- 172.21.15.83:0/857439224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bcab70 0x55d722bd1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.095+0000 7fefe19f0640 1 -- 172.21.15.83:0/857439224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bca1b0 msgr2=0x55d722bca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.095+0000 7fefe19f0640 1 --2- 172.21.15.83:0/857439224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bca1b0 0x55d722bca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.095+0000 7fefe19f0640 1 -- 172.21.15.83:0/857439224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bcab70 msgr2=0x55d722bd1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.095+0000 7fefe19f0640 1 --2- 172.21.15.83:0/857439224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bcab70 0x55d722bd1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.095+0000 7fefe19f0640 1 -- 172.21.15.83:0/857439224 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d722aeb0b0 con 0x55d722bc8fe0 2024-03-11T19:44:01.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.095+0000 7fefe21f1640 1 --2- 172.21.15.83:0/857439224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bca1b0 0x55d722bca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:01.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.095+0000 7fefe19f0640 1 --2- 172.21.15.83:0/857439224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 0x55d722bc9400 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fefcc02c8a0 tx=0x7fefcc02ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b51168d63fb4b13 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.096+0000 7fefd37fe640 1 -- 172.21.15.83:0/168115306 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fefd80105f0 con 0x7fefdc101c30 2024-03-11T19:44:01.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.096+0000 7fefd37fe640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefc80a1b00 0x7fefc80a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.097+0000 7fefd37fe640 1 -- 172.21.15.83:0/168115306 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fefd80cced0 con 0x7fefdc101c30 2024-03-11T19:44:01.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.096+0000 7fefc7fff640 1 -- 172.21.15.83:0/857439224 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc004460 con 0x55d722bc8fe0 2024-03-11T19:44:01.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.097+0000 7fefe21f1640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefc80a1b00 0x7fefc80a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.097+0000 7fefc7fff640 1 -- 172.21.15.83:0/857439224 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefcc003ad0 con 0x55d722bc8fe0 2024-03-11T19:44:01.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.097+0000 7fefc7fff640 1 -- 172.21.15.83:0/857439224 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc005340 con 0x55d722bc8fe0 2024-03-11T19:44:01.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.098+0000 7fefe702c740 1 -- 172.21.15.83:0/857439224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 msgr2=0x55d722bc9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.098+0000 7fefe702c740 1 --2- 172.21.15.83:0/857439224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 0x55d722bc9400 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fefcc02c8a0 tx=0x7fefcc02ccf0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.099+0000 7fefe21f1640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefc80a1b00 0x7fefc80a3fc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009fd0 tx=0x7fefd400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.100+0000 7fefe702c740 1 -- 172.21.15.83:0/857439224 shutdown_connections 2024-03-11T19:44:01.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.100+0000 7fefe702c740 1 --2- 172.21.15.83:0/857439224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bcab70 0x55d722bd1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.100+0000 7fefe702c740 1 --2- 172.21.15.83:0/857439224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bca1b0 0x55d722bca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.100+0000 7fefe702c740 1 --2- 172.21.15.83:0/857439224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 0x55d722bc9400 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.100+0000 7fefe702c740 1 -- 172.21.15.83:0/857439224 >> 172.21.15.83:0/857439224 conn(0x55d722bbb9b0 msgr2=0x55d722bbbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.101+0000 7fefe702c740 1 -- 172.21.15.83:0/857439224 shutdown_connections 2024-03-11T19:44:01.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.102+0000 7fefe702c740 1 -- 172.21.15.83:0/857439224 wait complete. 2024-03-11T19:44:01.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.103+0000 7fefe702c740 1 -- start start 2024-03-11T19:44:01.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.103+0000 7fefe702c740 1 Processor -- start 2024-03-11T19:44:01.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.103+0000 7fefe702c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 0x55d722bc1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.103+0000 7fefe702c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bca1b0 0x55d722bc1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.103+0000 7fefe702c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bcab70 0x55d722bc6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe21f1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 0x55d722bc1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe21f1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 0x55d722bc1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45080/0 (socket says 172.21.15.83:45080) 2024-03-11T19:44:01.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe21f1640 1 -- 172.21.15.83:0/3916885646 learned_addr learned my addr 172.21.15.83:0/3916885646 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:01.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.103+0000 7fefe702c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d722acdf40 con 0x55d722bca1b0 2024-03-11T19:44:01.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe702c740 1 -- 172.21.15.83:0/3916885646 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7227fc4b0 con 0x55d722bcab70 2024-03-11T19:44:01.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe702c740 1 -- 172.21.15.83:0/3916885646 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d722aae670 con 0x55d722bc8fe0 2024-03-11T19:44:01.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe29f2640 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bcab70 0x55d722bc6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe21f1640 1 -- 172.21.15.83:0/3916885646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bcab70 msgr2=0x55d722bc6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe21f1640 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bcab70 0x55d722bc6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe21f1640 1 -- 172.21.15.83:0/3916885646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bca1b0 msgr2=0x55d722bc1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe21f1640 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bca1b0 0x55d722bc1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe21f1640 1 -- 172.21.15.83:0/3916885646 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc068d10 con 0x55d722bc8fe0 2024-03-11T19:44:01.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.105+0000 7fefe29f2640 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bcab70 0x55d722bc6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:44:01.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.105+0000 7fefe21f1640 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 0x55d722bc1a30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fefd4015dc0 tx=0x7fefd4004bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2039afb2d72406 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.105+0000 7fefc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:44:01.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.105+0000 7fefc5ffb640 1 -- 172.21.15.83:0/3916885646 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4013d50 con 0x55d722bc8fe0 2024-03-11T19:44:01.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.104+0000 7fefe702c740 10 client.0 _ll_register_callbacks cb 0x55d722b1bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:01.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.105+0000 7fefe702c740 1 -- 172.21.15.83:0/3916885646 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d722b02f40 con 0x55d722bc8fe0 2024-03-11T19:44:01.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.105+0000 7fefc5ffb640 1 -- 172.21.15.83:0/3916885646 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fefd401f3f0 con 0x55d722bc8fe0 2024-03-11T19:44:01.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.105+0000 7fefc5ffb640 1 -- 172.21.15.83:0/3916885646 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefd4013eb0 con 0x55d722bc8fe0 2024-03-11T19:44:01.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.106+0000 7fefe702c740 10 client.25855 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:01.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.106+0000 7fefe702c740 1 -- 172.21.15.83:0/3916885646 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55d722b27320 con 0x55d722bc8fe0 2024-03-11T19:44:01.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.106+0000 7fefc5ffb640 1 -- 172.21.15.83:0/3916885646 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fefd4026060 con 0x55d722bc8fe0 2024-03-11T19:44:01.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.106+0000 7fefe702c740 10 client.25855 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:01.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.106+0000 7fefe702c740 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:01.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.106+0000 7fefe702c740 20 client.25855 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:01.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.106+0000 7fefe702c740 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d722be78f0 0x55d722be9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.106+0000 7fefe702c740 4 client.25855 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:44:01.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.106+0000 7fefe702c740 1 -- 172.21.15.83:0/3916885646 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d722b1f310 con 0x55d722be78f0 2024-03-11T19:44:01.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.108+0000 7fefe19f0640 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d722be78f0 0x55d722be9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.109+0000 7fefe19f0640 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d722be78f0 0x55d722be9d70 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57098a178a6d512b server_cookie=2c7c808911c3f388 in_seq=0 out_seq=0 2024-03-11T19:44:01.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.109+0000 7fefc5ffb640 10 client.25855 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:01.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.109+0000 7fefc5ffb640 1 -- 172.21.15.83:0/3916885646 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d722b1f310 con 0x55d722be78f0 2024-03-11T19:44:01.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefc5ffb640 10 client.25855 handle_command_reply: tid=0 2024-03-11T19:44:01.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 1 client.25855 shutdown 2024-03-11T19:44:01.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 1 -- 172.21.15.83:0/3916885646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 msgr2=0x55d722bc1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 0x55d722bc1a30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fefd4015dc0 tx=0x7fefd4004bc0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 1 -- 172.21.15.83:0/3916885646 shutdown_connections 2024-03-11T19:44:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d722be78f0 0x55d722be9d70 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bcab70 0x55d722bc6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bca1b0 0x55d722bc1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 1 --2- 172.21.15.83:0/3916885646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bc8fe0 0x55d722bc1a30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 1 -- 172.21.15.83:0/3916885646 >> 172.21.15.83:0/3916885646 conn(0x55d722bbb9b0 msgr2=0x55d722bbac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefc5ffb640 0 client.25855 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 1 -- 172.21.15.83:0/3916885646 shutdown_connections 2024-03-11T19:44:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 1 -- 172.21.15.83:0/3916885646 wait complete. 2024-03-11T19:44:01.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.110+0000 7fefe702c740 20 client.25855 trim_cache size 0 max 16384 2024-03-11T19:44:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe702c740 1 Processor -- start 2024-03-11T19:44:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe702c740 1 -- start start 2024-03-11T19:44:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe702c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bc8fe0 0x55d722bf7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe702c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bca1b0 0x55d722bf8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe702c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bcab70 0x55d722bc1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe21f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bc8fe0 0x55d722bf7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe21f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bc8fe0 0x55d722bf7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43126/0 (socket says 172.21.15.83:43126) 2024-03-11T19:44:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe21f1640 1 -- 172.21.15.83:0/2749025807 learned_addr learned my addr 172.21.15.83:0/2749025807 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe702c740 1 -- 172.21.15.83:0/2749025807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d722bbc1f0 con 0x55d722bc8fe0 2024-03-11T19:44:01.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe702c740 1 -- 172.21.15.83:0/2749025807 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d722bcd6f0 con 0x55d722bcab70 2024-03-11T19:44:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe702c740 1 -- 172.21.15.83:0/2749025807 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d722bb0350 con 0x55d722bca1b0 2024-03-11T19:44:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe21f1640 1 -- 172.21.15.83:0/2749025807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bca1b0 msgr2=0x55d722bf8150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:44:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe21f1640 1 --2- 172.21.15.83:0/2749025807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bca1b0 0x55d722bf8150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe21f1640 1 -- 172.21.15.83:0/2749025807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bcab70 msgr2=0x55d722bc1640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:44:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.127+0000 7fefe21f1640 1 --2- 172.21.15.83:0/2749025807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bcab70 0x55d722bc1640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.128+0000 7fefe21f1640 1 -- 172.21.15.83:0/2749025807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d722ac3c00 con 0x55d722bc8fe0 2024-03-11T19:44:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.128+0000 7fefe21f1640 1 --2- 172.21.15.83:0/2749025807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bc8fe0 0x55d722bf7c10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009420 tx=0x7fefd401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a116be0eb3d70e1e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.128+0000 7fefc77fe640 1 -- 172.21.15.83:0/2749025807 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4013e20 con 0x55d722bc8fe0 2024-03-11T19:44:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.128+0000 7fefc77fe640 1 -- 172.21.15.83:0/2749025807 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefd402e020 con 0x55d722bc8fe0 2024-03-11T19:44:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.128+0000 7fefc77fe640 1 -- 172.21.15.83:0/2749025807 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4035ea0 con 0x55d722bc8fe0 2024-03-11T19:44:01.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.129+0000 7fefe702c740 1 -- 172.21.15.83:0/2749025807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bc8fe0 msgr2=0x55d722bf7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.129+0000 7fefe702c740 1 --2- 172.21.15.83:0/2749025807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bc8fe0 0x55d722bf7c10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009420 tx=0x7fefd401d000 comp rx=0 tx=0).stop 2024-03-11T19:44:01.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.129+0000 7fefe702c740 1 -- 172.21.15.83:0/2749025807 shutdown_connections 2024-03-11T19:44:01.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.129+0000 7fefe702c740 1 --2- 172.21.15.83:0/2749025807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bcab70 0x55d722bc1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.129+0000 7fefe702c740 1 --2- 172.21.15.83:0/2749025807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bca1b0 0x55d722bf8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.129+0000 7fefe702c740 1 --2- 172.21.15.83:0/2749025807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bc8fe0 0x55d722bf7c10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.129+0000 7fefe702c740 1 -- 172.21.15.83:0/2749025807 >> 172.21.15.83:0/2749025807 conn(0x55d722bbb9b0 msgr2=0x55d722bcde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.129+0000 7fefe702c740 1 -- 172.21.15.83:0/2749025807 shutdown_connections 2024-03-11T19:44:01.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.129+0000 7fefe702c740 1 -- 172.21.15.83:0/2749025807 wait complete. 2024-03-11T19:44:01.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe702c740 1 -- start start 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe702c740 1 Processor -- start 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe702c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bc8fe0 0x55d722bf8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe702c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bca1b0 0x55d722bf67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe702c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bcab70 0x55d722bf6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe702c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d722bf7500 con 0x55d722bca1b0 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe702c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d722bf7670 con 0x55d722bc8fe0 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe702c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d722c3e930 con 0x55d722bcab70 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe19f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bca1b0 0x55d722bf67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe702c740 10 client.0 _ll_register_callbacks cb 0x7fefcc0048e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe19f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bca1b0 0x55d722bf67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43128/0 (socket says 172.21.15.83:43128) 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe19f0640 1 -- 172.21.15.83:0/2235054646 learned_addr learned my addr 172.21.15.83:0/2235054646 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe29f2640 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bcab70 0x55d722bf6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.130+0000 7fefe21f1640 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bc8fe0 0x55d722bf8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefe29f2640 1 -- 172.21.15.83:0/2235054646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bc8fe0 msgr2=0x55d722bf8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefe29f2640 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bc8fe0 0x55d722bf8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefe29f2640 1 -- 172.21.15.83:0/2235054646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bca1b0 msgr2=0x55d722bf67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefe29f2640 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bca1b0 0x55d722bf67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefe29f2640 1 -- 172.21.15.83:0/2235054646 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d722b27320 con 0x55d722bcab70 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefe29f2640 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bcab70 0x55d722bf6e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fefd80caad0 tx=0x7fefd80980f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=214e988558c4cae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefc57fa640 1 -- 172.21.15.83:0/2235054646 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd80b1070 con 0x55d722bcab70 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefc57fa640 1 -- 172.21.15.83:0/2235054646 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fefd80a2690 con 0x55d722bcab70 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefe702c740 1 -- 172.21.15.83:0/2235054646 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d722b4e900 con 0x55d722bcab70 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefc57fa640 1 -- 172.21.15.83:0/2235054646 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefd80caea0 con 0x55d722bcab70 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefe702c740 10 client.25858 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefe702c740 1 -- 172.21.15.83:0/2235054646 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d722ae9910 con 0x55d722bcab70 2024-03-11T19:44:01.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.131+0000 7fefc57fa640 1 -- 172.21.15.83:0/2235054646 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefd807c020 con 0x55d722bcab70 2024-03-11T19:44:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.132+0000 7fefc57fa640 1 -- 172.21.15.83:0/2235054646 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fefd8099340 con 0x55d722bcab70 2024-03-11T19:44:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.132+0000 7fefe702c740 10 client.25858 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.132+0000 7fefe702c740 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.132+0000 7fefe702c740 20 client.25858 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.132+0000 7fefe702c740 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d722c40450 0x55d722c42840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.132+0000 7fefe702c740 4 client.25858 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:44:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.132+0000 7fefe702c740 1 -- 172.21.15.83:0/2235054646 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d722b4e900 con 0x55d722c40450 2024-03-11T19:44:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.132+0000 7fefe21f1640 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d722c40450 0x55d722c42840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:00 smithi137 ceph-mon[36220]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 MiB/s wr, 781 op/s 2024-03-11T19:44:01.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.133+0000 7fefe21f1640 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d722c40450 0x55d722c42840 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d554d96915cafeaa server_cookie=e028e36d38363c93 in_seq=0 out_seq=0 2024-03-11T19:44:01.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.133+0000 7fefc57fa640 10 client.25858 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:01.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.134+0000 7fefc57fa640 1 -- 172.21.15.83:0/2235054646 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d722b4e900 con 0x55d722c40450 2024-03-11T19:44:01.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.134+0000 7fefc57fa640 10 client.25858 handle_command_reply: tid=0 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.134+0000 7fefe702c740 1 client.25858 shutdown 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.135+0000 7fefe702c740 1 -- 172.21.15.83:0/2235054646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bcab70 msgr2=0x55d722bf6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.135+0000 7fefe702c740 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bcab70 0x55d722bf6e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fefd80caad0 tx=0x7fefd80980f0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.135+0000 7fefe702c740 1 -- 172.21.15.83:0/2235054646 shutdown_connections 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.135+0000 7fefe702c740 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d722c40450 0x55d722c42840 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.135+0000 7fefc57fa640 0 client.25858 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.135+0000 7fefe702c740 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d722bcab70 0x55d722bf6e00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.135+0000 7fefe702c740 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d722bca1b0 0x55d722bf67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.136+0000 7fefe702c740 1 --2- 172.21.15.83:0/2235054646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d722bc8fe0 0x55d722bf8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.136+0000 7fefe702c740 1 -- 172.21.15.83:0/2235054646 >> 172.21.15.83:0/2235054646 conn(0x55d722bbb9b0 msgr2=0x55d722bc15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.136+0000 7fefe702c740 1 -- 172.21.15.83:0/2235054646 shutdown_connections 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.136+0000 7fefe702c740 1 -- 172.21.15.83:0/2235054646 wait complete. 2024-03-11T19:44:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.136+0000 7fefe702c740 20 client.25858 trim_cache size 0 max 16384 2024-03-11T19:44:01.143 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:44:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:44:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "75d1349e-4f68-41d7-b0f5-6807cc2122cd", 2024-03-11T19:44:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:44:01.143 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:44:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 -- 172.21.15.83:0/168115306 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefc80a1b00 msgr2=0x7fefc80a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefc80a1b00 0x7fefc80a3fc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009fd0 tx=0x7fefd400b040 comp rx=0 tx=0).stop 2024-03-11T19:44:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 -- 172.21.15.83:0/168115306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefdc101c30 msgr2=0x7fefdc1a9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefdc101c30 0x7fefdc1a9770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fefd800b440 tx=0x7fefd800b900 comp rx=0 tx=0).stop 2024-03-11T19:44:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 -- 172.21.15.83:0/168115306 shutdown_connections 2024-03-11T19:44:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefc80a1b00 0x7fefc80a3fc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefdc101c30 0x7fefdc1a9770 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefdc101270 0x7fefdc070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 --2- 172.21.15.83:0/168115306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefdc100080 0x7fefdc06fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 -- 172.21.15.83:0/168115306 >> 172.21.15.83:0/168115306 conn(0x7fefdc0fb610 msgr2=0x7fefdc102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 -- 172.21.15.83:0/168115306 shutdown_connections 2024-03-11T19:44:01.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.137+0000 7fefd13ff640 1 -- 172.21.15.83:0/168115306 wait complete. 2024-03-11T19:44:01.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:44:01.156 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:44:01.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:00 smithi204 ceph-mon[36739]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 MiB/s wr, 781 op/s 2024-03-11T19:44:01.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.310+0000 7faea7cc0640 1 -- 172.21.15.83:0/216548210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea0101c30 msgr2=0x7faea01084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.310+0000 7faea7cc0640 1 --2- 172.21.15.83:0/216548210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea0101c30 0x7faea01084c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fae98009ca0 tx=0x7fae9802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.311+0000 7faea7cc0640 1 -- 172.21.15.83:0/216548210 shutdown_connections 2024-03-11T19:44:01.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.311+0000 7faea7cc0640 1 --2- 172.21.15.83:0/216548210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea0101c30 0x7faea01084c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.311+0000 7faea7cc0640 1 --2- 172.21.15.83:0/216548210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faea0101270 0x7faea01016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.311+0000 7faea7cc0640 1 --2- 172.21.15.83:0/216548210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea0100080 0x7faea01004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.311+0000 7faea7cc0640 1 -- 172.21.15.83:0/216548210 >> 172.21.15.83:0/216548210 conn(0x7faea00fb5f0 msgr2=0x7faea00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.311+0000 7faea7cc0640 1 -- 172.21.15.83:0/216548210 shutdown_connections 2024-03-11T19:44:01.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.311+0000 7faea7cc0640 1 -- 172.21.15.83:0/216548210 wait complete. 2024-03-11T19:44:01.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.312+0000 7faea7cc0640 1 Processor -- start 2024-03-11T19:44:01.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.312+0000 7faea7cc0640 1 -- start start 2024-03-11T19:44:01.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.312+0000 7faea7cc0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea0100080 0x7faea01961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.312+0000 7faea7cc0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea0101270 0x7faea01966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.312+0000 7faea7cc0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faea0101c30 0x7faea019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea5a5c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea0100080 0x7faea01961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea525b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea0101270 0x7faea01966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea525b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea0101270 0x7faea01966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43138/0 (socket says 172.21.15.83:43138) 2024-03-11T19:44:01.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea5a5c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea0100080 0x7faea01961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45112/0 (socket says 172.21.15.83:45112) 2024-03-11T19:44:01.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea525b640 1 -- 172.21.15.83:0/2597985144 learned_addr learned my addr 172.21.15.83:0/2597985144 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea7cc0640 1 -- 172.21.15.83:0/2597985144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea019dc90 con 0x7faea0101270 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea7cc0640 1 -- 172.21.15.83:0/2597985144 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea019de00 con 0x7faea0101c30 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea7cc0640 1 -- 172.21.15.83:0/2597985144 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea019dfc0 con 0x7faea0100080 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea525b640 1 -- 172.21.15.83:0/2597985144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea0100080 msgr2=0x7faea01961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea525b640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea0100080 0x7faea01961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea525b640 1 -- 172.21.15.83:0/2597985144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faea0101c30 msgr2=0x7faea019d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea525b640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faea0101c30 0x7faea019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea525b640 1 -- 172.21.15.83:0/2597985144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae98009900 con 0x7faea0101270 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.313+0000 7faea525b640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea0101270 0x7faea01966e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fae9000b4a0 tx=0x7fae9000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.314+0000 7fae96ffd640 1 -- 172.21.15.83:0/2597985144 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90010610 con 0x7faea0101270 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.314+0000 7faea7cc0640 1 -- 172.21.15.83:0/2597985144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea00682e0 con 0x7faea0101270 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.314+0000 7fae96ffd640 1 -- 172.21.15.83:0/2597985144 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae90010c50 con 0x7faea0101270 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.314+0000 7fae96ffd640 1 -- 172.21.15.83:0/2597985144 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90017e20 con 0x7faea0101270 2024-03-11T19:44:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.314+0000 7faea7cc0640 1 -- 172.21.15.83:0/2597985144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea0068740 con 0x7faea0101270 2024-03-11T19:44:01.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.318+0000 7faeaa96b740 1 Processor -- start 2024-03-11T19:44:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.318+0000 7faeaa96b740 1 -- start start 2024-03-11T19:44:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.319+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3250 0x55642c7c3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.319+0000 7fae96ffd640 1 -- 172.21.15.83:0/2597985144 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fae90017480 con 0x7faea0101270 2024-03-11T19:44:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.319+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 0x55642c7c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.319+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c6da0 0x55642c7d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.319+0000 7faeaa96b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c733280 con 0x55642c7c6da0 2024-03-11T19:44:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.319+0000 7faeaa96b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c7307b0 con 0x55642c7c3250 2024-03-11T19:44:01.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.319+0000 7faeaa96b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c72bf90 con 0x55642c7c3c40 2024-03-11T19:44:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.319+0000 7faea625d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3250 0x55642c7c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.319+0000 7faea625d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3250 0x55642c7c3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47284/0 (socket says 172.21.15.83:47284) 2024-03-11T19:44:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.319+0000 7faea625d640 1 -- 172.21.15.83:0/1717294501 learned_addr learned my addr 172.21.15.83:0/1717294501 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:01.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.319+0000 7faea5a5c640 1 --2- 172.21.15.83:0/1717294501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 0x55642c7c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7fae96ffd640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fae8c0a1a30 0x7fae8c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7faea5a5c640 1 -- 172.21.15.83:0/1717294501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3250 msgr2=0x55642c7c3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7faea5a5c640 1 --2- 172.21.15.83:0/1717294501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3250 0x55642c7c3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7faea5a5c640 1 -- 172.21.15.83:0/1717294501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c6da0 msgr2=0x55642c7d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7faea5a5c640 1 --2- 172.21.15.83:0/1717294501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c6da0 0x55642c7d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7faea5a5c640 1 -- 172.21.15.83:0/1717294501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55642c6ea570 con 0x55642c7c3c40 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7faea625d640 1 --2- 172.21.15.83:0/1717294501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3250 0x55642c7c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7fae96ffd640 1 -- 172.21.15.83:0/2597985144 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fae900cc9b0 con 0x7faea0101270 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7faea625d640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fae8c0a1a30 0x7fae8c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7faea5a5c640 1 --2- 172.21.15.83:0/1717294501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 0x55642c7c6730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009bd0 tx=0x7fae9c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28af4881f4c6289c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7fae8bfff640 1 -- 172.21.15.83:0/1717294501 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c004230 con 0x55642c7c3c40 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7fae8bfff640 1 -- 172.21.15.83:0/1717294501 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae9c009e40 con 0x55642c7c3c40 2024-03-11T19:44:01.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.320+0000 7fae8bfff640 1 -- 172.21.15.83:0/1717294501 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0197e0 con 0x55642c7c3c40 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.321+0000 7faeaa96b740 1 -- 172.21.15.83:0/1717294501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 msgr2=0x55642c7c6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.321+0000 7faeaa96b740 1 --2- 172.21.15.83:0/1717294501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 0x55642c7c6730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009bd0 tx=0x7fae9c011810 comp rx=0 tx=0).stop 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.321+0000 7faea625d640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fae8c0a1a30 0x7fae8c0a3ef0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55642c7c44a0 tx=0x7fae98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.321+0000 7faeaa96b740 1 -- 172.21.15.83:0/1717294501 shutdown_connections 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.321+0000 7faeaa96b740 1 --2- 172.21.15.83:0/1717294501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c6da0 0x55642c7d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.321+0000 7faeaa96b740 1 --2- 172.21.15.83:0/1717294501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 0x55642c7c6730 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.321+0000 7faeaa96b740 1 --2- 172.21.15.83:0/1717294501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3250 0x55642c7c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.321+0000 7faeaa96b740 1 -- 172.21.15.83:0/1717294501 >> 172.21.15.83:0/1717294501 conn(0x55642c7be7f0 msgr2=0x55642c7bebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.321+0000 7faeaa96b740 1 -- 172.21.15.83:0/1717294501 shutdown_connections 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.321+0000 7faeaa96b740 1 -- 172.21.15.83:0/1717294501 wait complete. 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.322+0000 7faeaa96b740 1 -- start start 2024-03-11T19:44:01.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.322+0000 7faeaa96b740 1 Processor -- start 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.322+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 0x55642c7cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.322+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 0x55642c7cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.322+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c6da0 0x55642c7d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.322+0000 7faeaa96b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c3ff430 con 0x55642c7c3250 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.322+0000 7faeaa96b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c3bbe00 con 0x55642c7c6da0 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.322+0000 7faeaa96b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c6b0fd0 con 0x55642c7c3c40 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.322+0000 7faeaa96b740 10 client.0 _ll_register_callbacks cb 0x55642c706fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea525b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 0x55642c7cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea5a5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 0x55642c7cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea525b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 0x55642c7cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45116/0 (socket says 172.21.15.83:45116) 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea525b640 1 -- 172.21.15.83:0/2476884663 learned_addr learned my addr 172.21.15.83:0/2476884663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea625d640 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c6da0 0x55642c7d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea525b640 1 -- 172.21.15.83:0/2476884663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c6da0 msgr2=0x55642c7d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea525b640 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c6da0 0x55642c7d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea525b640 1 -- 172.21.15.83:0/2476884663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 msgr2=0x55642c7cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea525b640 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 0x55642c7cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea525b640 1 -- 172.21.15.83:0/2476884663 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c009830 con 0x55642c7c3c40 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea5a5c640 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 0x55642c7cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faea525b640 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 0x55642c7cd7d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae90095e30 tx=0x7fae900993f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e16552f200bf894a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7fae897fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7fae897fa640 1 -- 172.21.15.83:0/2476884663 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9009db20 con 0x55642c7c3c40 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7fae897fa640 1 -- 172.21.15.83:0/2476884663 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fae900a2170 con 0x55642c7c3c40 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.323+0000 7faeaa96b740 1 -- 172.21.15.83:0/2476884663 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55642c42b7e0 con 0x55642c7c3c40 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7fae897fa640 1 -- 172.21.15.83:0/2476884663 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae90096040 con 0x55642c7c3c40 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7faeaa96b740 10 client.25870 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:01.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7faeaa96b740 1 -- 172.21.15.83:0/2476884663 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55642c72a610 con 0x55642c7c3c40 2024-03-11T19:44:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7fae897fa640 1 -- 172.21.15.83:0/2476884663 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae900b9040 con 0x55642c7c3c40 2024-03-11T19:44:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7fae897fa640 1 -- 172.21.15.83:0/2476884663 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fae900957b0 con 0x55642c7c3c40 2024-03-11T19:44:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7faeaa96b740 10 client.25870 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7faeaa96b740 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7faeaa96b740 20 client.25870 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7faeaa96b740 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55642c7ea430 0x55642c7ec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7faeaa96b740 4 client.25870 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:44:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7faeaa96b740 1 -- 172.21.15.83:0/2476884663 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55642c42b7e0 con 0x55642c7ea430 2024-03-11T19:44:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.324+0000 7faea5a5c640 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55642c7ea430 0x55642c7ec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.325+0000 7faea5a5c640 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55642c7ea430 0x55642c7ec8a0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2441e768347b5bb server_cookie=ebb38dc6837752c9 in_seq=0 out_seq=0 2024-03-11T19:44:01.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.325+0000 7fae897fa640 10 client.25870 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:01.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.325+0000 7fae897fa640 1 -- 172.21.15.83:0/2476884663 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55642c42b7e0 con 0x55642c7ea430 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.325+0000 7fae897fa640 10 client.25870 handle_command_reply: tid=0 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.325+0000 7faeaa96b740 1 client.25870 shutdown 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.325+0000 7faeaa96b740 1 -- 172.21.15.83:0/2476884663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 msgr2=0x55642c7cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.325+0000 7faeaa96b740 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 0x55642c7cd7d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae90095e30 tx=0x7fae900993f0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.327+0000 7faeaa96b740 1 -- 172.21.15.83:0/2476884663 shutdown_connections 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.327+0000 7faeaa96b740 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55642c7ea430 0x55642c7ec8a0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.328+0000 7faeaa96b740 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c6da0 0x55642c7d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.328+0000 7faeaa96b740 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3c40 0x55642c7cd7d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.328+0000 7faeaa96b740 1 --2- 172.21.15.83:0/2476884663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 0x55642c7cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.328+0000 7faeaa96b740 1 -- 172.21.15.83:0/2476884663 >> 172.21.15.83:0/2476884663 conn(0x55642c7be7f0 msgr2=0x55642c7bda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.328+0000 7fae897fa640 0 client.25870 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.328+0000 7faeaa96b740 1 -- 172.21.15.83:0/2476884663 shutdown_connections 2024-03-11T19:44:01.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.328+0000 7faeaa96b740 1 -- 172.21.15.83:0/2476884663 wait complete. 2024-03-11T19:44:01.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.328+0000 7faeaa96b740 20 client.25870 trim_cache size 0 max 16384 2024-03-11T19:44:01.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.349+0000 7faeaa96b740 1 Processor -- start 2024-03-11T19:44:01.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faeaa96b740 1 -- start start 2024-03-11T19:44:01.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3250 0x55642c7fa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 0x55642c7fac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c6da0 0x55642c7c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faeaa96b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c42b7e0 con 0x55642c7c6da0 2024-03-11T19:44:01.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faeaa96b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c7c4090 con 0x55642c7c3c40 2024-03-11T19:44:01.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faeaa96b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c7c4330 con 0x55642c7c3250 2024-03-11T19:44:01.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faea5a5c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3250 0x55642c7fa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faea625d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c6da0 0x55642c7c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faea5a5c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3250 0x55642c7fa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45128/0 (socket says 172.21.15.83:45128) 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faea5a5c640 1 -- 172.21.15.83:0/3348432372 learned_addr learned my addr 172.21.15.83:0/3348432372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faea525b640 1 --2- 172.21.15.83:0/3348432372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 0x55642c7fac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faea5a5c640 1 -- 172.21.15.83:0/3348432372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 msgr2=0x55642c7fac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faea5a5c640 1 --2- 172.21.15.83:0/3348432372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 0x55642c7fac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faea5a5c640 1 -- 172.21.15.83:0/3348432372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c6da0 msgr2=0x55642c7c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faea5a5c640 1 --2- 172.21.15.83:0/3348432372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c6da0 0x55642c7c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.350+0000 7faea5a5c640 1 -- 172.21.15.83:0/3348432372 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55642c6c6ac0 con 0x55642c7c3250 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.351+0000 7faea5a5c640 1 --2- 172.21.15.83:0/3348432372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3250 0x55642c7fa750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0058c0 tx=0x7fae9c0093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60a53b92fd40db39 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.351+0000 7fae8b7fe640 1 -- 172.21.15.83:0/3348432372 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c02a070 con 0x55642c7c3250 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.351+0000 7fae8b7fe640 1 -- 172.21.15.83:0/3348432372 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae9c004410 con 0x55642c7c3250 2024-03-11T19:44:01.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.351+0000 7fae8b7fe640 1 -- 172.21.15.83:0/3348432372 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c003d10 con 0x55642c7c3250 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.352+0000 7faeaa96b740 1 -- 172.21.15.83:0/3348432372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3250 msgr2=0x55642c7fa750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.352+0000 7faeaa96b740 1 --2- 172.21.15.83:0/3348432372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3250 0x55642c7fa750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0058c0 tx=0x7fae9c0093b0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.352+0000 7faeaa96b740 1 -- 172.21.15.83:0/3348432372 shutdown_connections 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.352+0000 7faeaa96b740 1 --2- 172.21.15.83:0/3348432372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c6da0 0x55642c7c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.352+0000 7faeaa96b740 1 --2- 172.21.15.83:0/3348432372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 0x55642c7fac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.352+0000 7faeaa96b740 1 --2- 172.21.15.83:0/3348432372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c3250 0x55642c7fa750 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.352+0000 7faeaa96b740 1 -- 172.21.15.83:0/3348432372 >> 172.21.15.83:0/3348432372 conn(0x55642c7be7f0 msgr2=0x55642c7cd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.352+0000 7faeaa96b740 1 -- 172.21.15.83:0/3348432372 shutdown_connections 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.352+0000 7faeaa96b740 1 -- 172.21.15.83:0/3348432372 wait complete. 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.352+0000 7faeaa96b740 1 -- start start 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faeaa96b740 1 Processor -- start 2024-03-11T19:44:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 0x55642c83b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 0x55642c83d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faeaa96b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c6da0 0x55642c83ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faeaa96b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c840660 con 0x55642c7c3250 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faeaa96b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c8407d0 con 0x55642c7c3c40 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faeaa96b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642c840990 con 0x55642c7c6da0 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faea525b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 0x55642c83d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faea525b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 0x55642c83d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47312/0 (socket says 172.21.15.83:47312) 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faea525b640 1 -- 172.21.15.83:0/1679608623 learned_addr learned my addr 172.21.15.83:0/1679608623 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faea5a5c640 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 0x55642c83b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faeaa96b740 10 client.0 _ll_register_callbacks cb 0x7fae9c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.353+0000 7faea625d640 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c6da0 0x55642c83ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7faea525b640 1 -- 172.21.15.83:0/1679608623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c6da0 msgr2=0x55642c83ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7faea525b640 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c6da0 0x55642c83ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7faea525b640 1 -- 172.21.15.83:0/1679608623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 msgr2=0x55642c83b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7faea525b640 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 0x55642c83b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7faea525b640 1 -- 172.21.15.83:0/1679608623 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55642c6ea570 con 0x55642c7c3c40 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7faea5a5c640 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 0x55642c83b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7faea525b640 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 0x55642c83d940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae9000be60 tx=0x7fae90097ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67dbe9ac0e26427f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7fae897fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7fae897fa640 1 -- 172.21.15.83:0/1679608623 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90098c70 con 0x55642c7c3c40 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7fae897fa640 1 -- 172.21.15.83:0/1679608623 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fae90095930 con 0x55642c7c3c40 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7faeaa96b740 1 -- 172.21.15.83:0/1679608623 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55642c468720 con 0x55642c7c3c40 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7fae897fa640 1 -- 172.21.15.83:0/1679608623 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae90099510 con 0x55642c7c3c40 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.354+0000 7faeaa96b740 10 client.25775 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.355+0000 7faeaa96b740 1 -- 172.21.15.83:0/1679608623 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55642c6ec6a0 con 0x55642c7c3c40 2024-03-11T19:44:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.355+0000 7fae897fa640 1 -- 172.21.15.83:0/1679608623 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae9007f020 con 0x55642c7c3c40 2024-03-11T19:44:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.355+0000 7fae897fa640 1 -- 172.21.15.83:0/1679608623 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fae900b2140 con 0x55642c7c3c40 2024-03-11T19:44:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.355+0000 7faeaa96b740 10 client.25775 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.355+0000 7faeaa96b740 10 client.25775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.355+0000 7faeaa96b740 20 client.25775 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.355+0000 7faeaa96b740 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55642c842030 0x55642c844450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.355+0000 7faeaa96b740 4 client.25775 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:44:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.355+0000 7faeaa96b740 1 -- 172.21.15.83:0/1679608623 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55642c4b7890 con 0x55642c842030 2024-03-11T19:44:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.355+0000 7faea5a5c640 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55642c842030 0x55642c844450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.356+0000 7fae897fa640 10 client.25775 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.356+0000 7faea5a5c640 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55642c842030 0x55642c844450 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc7ac05a4becb82f server_cookie=903972cd965cb762 in_seq=0 out_seq=0 2024-03-11T19:44:01.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.357+0000 7fae897fa640 1 -- 172.21.15.83:0/1679608623 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55642c4b7890 con 0x55642c842030 2024-03-11T19:44:01.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.357+0000 7fae897fa640 10 client.25775 handle_command_reply: tid=0 2024-03-11T19:44:01.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.358+0000 7faeaa96b740 1 client.25775 shutdown 2024-03-11T19:44:01.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.359+0000 7faeaa96b740 1 -- 172.21.15.83:0/1679608623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 msgr2=0x55642c83d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.359+0000 7faeaa96b740 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 0x55642c83d940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae9000be60 tx=0x7fae90097ee0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.360+0000 7faeaa96b740 1 -- 172.21.15.83:0/1679608623 shutdown_connections 2024-03-11T19:44:01.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.360+0000 7faeaa96b740 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55642c842030 0x55642c844450 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.360+0000 7faeaa96b740 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55642c7c6da0 0x55642c83ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.360+0000 7faeaa96b740 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55642c7c3c40 0x55642c83d940 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.360+0000 7faeaa96b740 1 --2- 172.21.15.83:0/1679608623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55642c7c3250 0x55642c83b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.360+0000 7faeaa96b740 1 -- 172.21.15.83:0/1679608623 >> 172.21.15.83:0/1679608623 conn(0x55642c7be7f0 msgr2=0x55642c7cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.360+0000 7fae897fa640 0 client.25775 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.360+0000 7faeaa96b740 1 -- 172.21.15.83:0/1679608623 shutdown_connections 2024-03-11T19:44:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.360+0000 7faeaa96b740 1 -- 172.21.15.83:0/1679608623 wait complete. 2024-03-11T19:44:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.360+0000 7faeaa96b740 20 client.25775 trim_cache size 0 max 16384 2024-03-11T19:44:01.366 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:44:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:44:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:44:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "75d1349e-4f68-41d7-b0f5-6807cc2122cd": { 2024-03-11T19:44:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:44:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "75d1349e-4f68-41d7-b0f5-6807cc2122cd", 2024-03-11T19:44:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:44:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:44:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:44:01.367 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:44:01.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 -- 172.21.15.83:0/2597985144 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fae8c0a1a30 msgr2=0x7fae8c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fae8c0a1a30 0x7fae8c0a3ef0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55642c7c44a0 tx=0x7fae98005fb0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 -- 172.21.15.83:0/2597985144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea0101270 msgr2=0x7faea01966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea0101270 0x7faea01966e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fae9000b4a0 tx=0x7fae9000b960 comp rx=0 tx=0).stop 2024-03-11T19:44:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 -- 172.21.15.83:0/2597985144 shutdown_connections 2024-03-11T19:44:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fae8c0a1a30 0x7fae8c0a3ef0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faea0101c30 0x7faea019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faea0101270 0x7faea01966e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 --2- 172.21.15.83:0/2597985144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea0100080 0x7faea01961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 -- 172.21.15.83:0/2597985144 >> 172.21.15.83:0/2597985144 conn(0x7faea00fb5f0 msgr2=0x7faea0104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 -- 172.21.15.83:0/2597985144 shutdown_connections 2024-03-11T19:44:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:01.361+0000 7faea7cc0640 1 -- 172.21.15.83:0/2597985144 wait complete. 2024-03-11T19:44:01.379 INFO:tasks.cephfs.filesystem:scrub status for tag:75d1349e-4f68-41d7-b0f5-6807cc2122cd - {'path': '/', 'tag': '75d1349e-4f68-41d7-b0f5-6807cc2122cd', 'options': 'recursive,force'} 2024-03-11T19:44:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:01 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:44:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:01 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:01 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:44:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:01 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:44:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:01 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:01 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:44:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:01 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:01 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:01 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:44:02.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:01 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:01 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:44:02.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:01 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:44:02.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:01 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:01 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:44:02.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:01 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:01 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:01 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:44:02.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:01 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:01 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:44:02.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:01 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:44:02.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:01 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:01 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:44:02.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:01 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:01 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 285.0 s, 2548.2 tps, lat 39.167 ms stddev 42.057 2024-03-11T19:44:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:02 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:44:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:02 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:02 smithi083 ceph-mon[31864]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 773 op/s 2024-03-11T19:44:03.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:02 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:44:03.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:02 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:03.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:02 smithi137 ceph-mon[36220]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 773 op/s 2024-03-11T19:44:03.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:02 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:44:03.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:02 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:03.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:02 smithi204 ceph-mon[36739]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 773 op/s 2024-03-11T19:44:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:04 smithi083 ceph-mon[31864]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.7 MiB/s wr, 844 op/s 2024-03-11T19:44:05.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:04 smithi137 ceph-mon[36220]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.7 MiB/s wr, 844 op/s 2024-03-11T19:44:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:04 smithi204 ceph-mon[36739]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.7 MiB/s wr, 844 op/s 2024-03-11T19:44:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:06 smithi083 ceph-mon[31864]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 MiB/s wr, 783 op/s 2024-03-11T19:44:07.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:06 smithi137 ceph-mon[36220]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 MiB/s wr, 783 op/s 2024-03-11T19:44:07.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:06 smithi204 ceph-mon[36739]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.6 MiB/s wr, 783 op/s 2024-03-11T19:44:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 290.0 s, 4359.8 tps, lat 23.012 ms stddev 27.496 2024-03-11T19:44:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:08 smithi083 ceph-mon[31864]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.8 MiB/s wr, 829 op/s 2024-03-11T19:44:09.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:08 smithi137 ceph-mon[36220]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.8 MiB/s wr, 829 op/s 2024-03-11T19:44:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:08 smithi204 ceph-mon[36739]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.8 MiB/s wr, 829 op/s 2024-03-11T19:44:10.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:10 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:44:10.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:10 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:44:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:10 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:44:11.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:11 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:44:11.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:11 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:44:11.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:11 smithi083 ceph-mon[31864]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 MiB/s wr, 738 op/s 2024-03-11T19:44:11.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:11 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:44:11.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:11 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:44:11.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:11 smithi137 ceph-mon[36220]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 MiB/s wr, 738 op/s 2024-03-11T19:44:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:11 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:44:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:11 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:44:11.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:11 smithi204 ceph-mon[36739]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.7 MiB/s wr, 738 op/s 2024-03-11T19:44:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 295.0 s, 3040.6 tps, lat 32.677 ms stddev 37.139 2024-03-11T19:44:13.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:12 smithi083 ceph-mon[31864]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.3 MiB/s wr, 715 op/s 2024-03-11T19:44:13.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:12 smithi137 ceph-mon[36220]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.3 MiB/s wr, 715 op/s 2024-03-11T19:44:13.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:12 smithi204 ceph-mon[36739]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.3 MiB/s wr, 715 op/s 2024-03-11T19:44:15.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:14 smithi083 ceph-mon[31864]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.5 MiB/s wr, 746 op/s 2024-03-11T19:44:15.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:14 smithi137 ceph-mon[36220]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.5 MiB/s wr, 746 op/s 2024-03-11T19:44:15.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:14 smithi204 ceph-mon[36739]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.5 MiB/s wr, 746 op/s 2024-03-11T19:44:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:16 smithi083 ceph-mon[31864]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.4 MiB/s wr, 728 op/s 2024-03-11T19:44:17.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:16 smithi137 ceph-mon[36220]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.4 MiB/s wr, 728 op/s 2024-03-11T19:44:17.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:16 smithi204 ceph-mon[36739]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.4 MiB/s wr, 728 op/s 2024-03-11T19:44:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 300.0 s, 2432.0 tps, lat 41.130 ms stddev 43.049 2024-03-11T19:44:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:18 smithi083 ceph-mon[31864]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-03-11T19:44:19.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:18 smithi137 ceph-mon[36220]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-03-11T19:44:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:18 smithi204 ceph-mon[36739]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-03-11T19:44:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:20 smithi083 ceph-mon[31864]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 637 op/s 2024-03-11T19:44:21.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:20 smithi137 ceph-mon[36220]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 637 op/s 2024-03-11T19:44:21.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:20 smithi204 ceph-mon[36739]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 637 op/s 2024-03-11T19:44:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 305.0 s, 3604.6 tps, lat 27.927 ms stddev 33.866 2024-03-11T19:44:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:22 smithi083 ceph-mon[31864]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 649 op/s 2024-03-11T19:44:23.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:22 smithi137 ceph-mon[36220]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 649 op/s 2024-03-11T19:44:23.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:22 smithi204 ceph-mon[36739]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 MiB/s wr, 649 op/s 2024-03-11T19:44:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:24 smithi083 ceph-mon[31864]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 755 op/s 2024-03-11T19:44:25.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:24 smithi137 ceph-mon[36220]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 755 op/s 2024-03-11T19:44:25.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:24 smithi204 ceph-mon[36739]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 755 op/s 2024-03-11T19:44:27.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:26 smithi083 ceph-mon[31864]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 MiB/s wr, 739 op/s 2024-03-11T19:44:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:26 smithi137 ceph-mon[36220]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 MiB/s wr, 739 op/s 2024-03-11T19:44:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:26 smithi204 ceph-mon[36739]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 MiB/s wr, 739 op/s 2024-03-11T19:44:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 310.0 s, 3291.0 tps, lat 30.231 ms stddev 33.279 2024-03-11T19:44:29.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:28 smithi083 ceph-mon[31864]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-03-11T19:44:29.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:28 smithi137 ceph-mon[36220]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-03-11T19:44:29.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:28 smithi204 ceph-mon[36739]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-03-11T19:44:30.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:29 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:30.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:29 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:30.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:29 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:44:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:44:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:30 smithi083 ceph-mon[31864]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 734 op/s 2024-03-11T19:44:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:44:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:30 smithi137 ceph-mon[36220]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 734 op/s 2024-03-11T19:44:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:44:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:44:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:44:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:30 smithi204 ceph-mon[36739]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 734 op/s 2024-03-11T19:44:31.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:44:31.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:44:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.533+0000 7fd35dd77640 1 -- 172.21.15.83:0/1371832292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd358101c20 msgr2=0x7fd358104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.533+0000 7fd35dd77640 1 --2- 172.21.15.83:0/1371832292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd358101c20 0x7fd358104010 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd348009860 tx=0x7fd34802bb70 comp rx=0 tx=0).stop 2024-03-11T19:44:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.533+0000 7fd35dd77640 1 -- 172.21.15.83:0/1371832292 shutdown_connections 2024-03-11T19:44:31.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.533+0000 7fd35dd77640 1 --2- 172.21.15.83:0/1371832292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd358106e80 0x7fd3581092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.533+0000 7fd35dd77640 1 --2- 172.21.15.83:0/1371832292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358104550 0x7fd358106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.533+0000 7fd35dd77640 1 --2- 172.21.15.83:0/1371832292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd358101c20 0x7fd358104010 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.533+0000 7fd35dd77640 1 -- 172.21.15.83:0/1371832292 >> 172.21.15.83:0/1371832292 conn(0x7fd3580fb650 msgr2=0x7fd3580fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:31.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.533+0000 7fd35dd77640 1 -- 172.21.15.83:0/1371832292 shutdown_connections 2024-03-11T19:44:31.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.533+0000 7fd35dd77640 1 -- 172.21.15.83:0/1371832292 wait complete. 2024-03-11T19:44:31.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.533+0000 7fd35dd77640 1 Processor -- start 2024-03-11T19:44:31.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.534+0000 7fd35dd77640 1 -- start start 2024-03-11T19:44:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.534+0000 7fd35dd77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd358101c20 0x7fd358197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.534+0000 7fd35dd77640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358104550 0x7fd358197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.534+0000 7fd35dd77640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd358106e80 0x7fd35819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.534+0000 7fd35dd77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35819ed30 con 0x7fd358101c20 2024-03-11T19:44:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.534+0000 7fd35dd77640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35819eea0 con 0x7fd358104550 2024-03-11T19:44:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.534+0000 7fd35dd77640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35819f060 con 0x7fd358106e80 2024-03-11T19:44:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.534+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd358101c20 0x7fd358197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.534+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd358101c20 0x7fd358197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40200/0 (socket says 172.21.15.83:40200) 2024-03-11T19:44:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.534+0000 7fd3577fe640 1 -- 172.21.15.83:0/2563586538 learned_addr learned my addr 172.21.15.83:0/2563586538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.535+0000 7fd357fff640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd358106e80 0x7fd35819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.535+0000 7fd356ffd640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358104550 0x7fd358197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.535+0000 7fd3577fe640 1 -- 172.21.15.83:0/2563586538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd358106e80 msgr2=0x7fd35819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.535+0000 7fd3577fe640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd358106e80 0x7fd35819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.535+0000 7fd3577fe640 1 -- 172.21.15.83:0/2563586538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358104550 msgr2=0x7fd358197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.535+0000 7fd3577fe640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358104550 0x7fd358197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.535+0000 7fd3577fe640 1 -- 172.21.15.83:0/2563586538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd348009490 con 0x7fd358101c20 2024-03-11T19:44:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.536+0000 7fd3577fe640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd358101c20 0x7fd358197240 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd348005940 tx=0x7fd348035f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.536+0000 7fd354ff9640 1 -- 172.21.15.83:0/2563586538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34802c710 con 0x7fd358101c20 2024-03-11T19:44:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.536+0000 7fd354ff9640 1 -- 172.21.15.83:0/2563586538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd348034910 con 0x7fd358101c20 2024-03-11T19:44:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.536+0000 7fd354ff9640 1 -- 172.21.15.83:0/2563586538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348032860 con 0x7fd358101c20 2024-03-11T19:44:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.537+0000 7fd35dd77640 1 -- 172.21.15.83:0/2563586538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35819f2e0 con 0x7fd358101c20 2024-03-11T19:44:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.537+0000 7fd35dd77640 1 -- 172.21.15.83:0/2563586538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35819f650 con 0x7fd358101c20 2024-03-11T19:44:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd354ff9640 1 -- 172.21.15.83:0/2563586538 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd34803b040 con 0x7fd358101c20 2024-03-11T19:44:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd360a22740 1 Processor -- start 2024-03-11T19:44:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd360a22740 1 -- start start 2024-03-11T19:44:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd360a22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a1276e0 0x55ce2a127b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd360a22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a12cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd360a22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 0x55ce2a12f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd360a22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2a093280 con 0x55ce2a1280d0 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd360a22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2a0907b0 con 0x55ce2a1276e0 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd360a22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2a08bf90 con 0x55ce2a12d240 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd354ff9640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3400a1b30 0x7fd3400a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a12cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd357fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a1276e0 0x55ce2a127b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd356ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 0x55ce2a12f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a12cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40216/0 (socket says 172.21.15.83:40216) 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd357fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a1276e0 0x55ce2a127b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:38180/0 (socket says 172.21.15.83:38180) 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.540+0000 7fd3577fe640 1 -- 172.21.15.83:0/3292641884 learned_addr learned my addr 172.21.15.83:0/3292641884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd354ff9640 1 -- 172.21.15.83:0/2563586538 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd3480ef0b0 con 0x7fd358101c20 2024-03-11T19:44:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd357fff640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3400a1b30 0x7fd3400a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd356ffd640 1 -- 172.21.15.83:0/3292641884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a1276e0 msgr2=0x55ce2a127b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd356ffd640 1 --2- 172.21.15.83:0/3292641884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a1276e0 0x55ce2a127b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd356ffd640 1 -- 172.21.15.83:0/3292641884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 msgr2=0x55ce2a12cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd356ffd640 1 --2- 172.21.15.83:0/3292641884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a12cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd356ffd640 1 -- 172.21.15.83:0/3292641884 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce2a04a570 con 0x55ce2a12d240 2024-03-11T19:44:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd356ffd640 1 --2- 172.21.15.83:0/3292641884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 0x55ce2a12f6d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd35806a080 tx=0x7fd34c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4783d3bfa17783cf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd357fff640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3400a1b30 0x7fd3400a3ff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55ce2a128930 tx=0x7fd350012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd33dffb640 1 -- 172.21.15.83:0/3292641884 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c004030 con 0x55ce2a12d240 2024-03-11T19:44:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd33dffb640 1 -- 172.21.15.83:0/3292641884 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd34c015070 con 0x55ce2a12d240 2024-03-11T19:44:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.541+0000 7fd33dffb640 1 -- 172.21.15.83:0/3292641884 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c011440 con 0x55ce2a12d240 2024-03-11T19:44:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.542+0000 7fd360a22740 1 -- 172.21.15.83:0/3292641884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 msgr2=0x55ce2a12f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.542+0000 7fd360a22740 1 --2- 172.21.15.83:0/3292641884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 0x55ce2a12f6d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd35806a080 tx=0x7fd34c0107a0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.542+0000 7fd360a22740 1 -- 172.21.15.83:0/3292641884 shutdown_connections 2024-03-11T19:44:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.542+0000 7fd360a22740 1 --2- 172.21.15.83:0/3292641884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 0x55ce2a12f6d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.542+0000 7fd360a22740 1 --2- 172.21.15.83:0/3292641884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a12cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.542+0000 7fd360a22740 1 --2- 172.21.15.83:0/3292641884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a1276e0 0x55ce2a127b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.542+0000 7fd360a22740 1 -- 172.21.15.83:0/3292641884 >> 172.21.15.83:0/3292641884 conn(0x55ce2a11e7f0 msgr2=0x55ce2a11ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.542+0000 7fd360a22740 1 -- 172.21.15.83:0/3292641884 shutdown_connections 2024-03-11T19:44:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.542+0000 7fd360a22740 1 -- 172.21.15.83:0/3292641884 wait complete. 2024-03-11T19:44:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.542+0000 7fd360a22740 1 -- start start 2024-03-11T19:44:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.544+0000 7fd360a22740 1 Processor -- start 2024-03-11T19:44:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.545+0000 7fd360a22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1276e0 0x55ce2a131610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.545+0000 7fd360a22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a1280d0 0x55ce2a131b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.545+0000 7fd360a22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 0x55ce2a138b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.545+0000 7fd357fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 0x55ce2a138b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.545+0000 7fd360a22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce29d5f430 con 0x55ce2a1276e0 2024-03-11T19:44:31.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.545+0000 7fd360a22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce29d1be00 con 0x55ce2a1280d0 2024-03-11T19:44:31.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.545+0000 7fd360a22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2a010fd0 con 0x55ce2a12d240 2024-03-11T19:44:31.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.545+0000 7fd357fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 0x55ce2a138b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:49220/0 (socket says 172.21.15.83:49220) 2024-03-11T19:44:31.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.545+0000 7fd357fff640 1 -- 172.21.15.83:0/641241607 learned_addr learned my addr 172.21.15.83:0/641241607 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:31.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.545+0000 7fd360a22740 10 client.0 _ll_register_callbacks cb 0x55ce2a066fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.546+0000 7fd3577fe640 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1276e0 0x55ce2a131610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.546+0000 7fd357fff640 1 -- 172.21.15.83:0/641241607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a1280d0 msgr2=0x55ce2a131b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.546+0000 7fd357fff640 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a1280d0 0x55ce2a131b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.546+0000 7fd357fff640 1 -- 172.21.15.83:0/641241607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1276e0 msgr2=0x55ce2a131610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.546+0000 7fd357fff640 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1276e0 0x55ce2a131610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.546+0000 7fd357fff640 1 -- 172.21.15.83:0/641241607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34c00b820 con 0x55ce2a12d240 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.546+0000 7fd3577fe640 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1276e0 0x55ce2a131610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.546+0000 7fd357fff640 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 0x55ce2a138b90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd35000de30 tx=0x7fd350004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdd20ee45c5b120b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.546+0000 7fd336ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.546+0000 7fd336ffd640 1 -- 172.21.15.83:0/641241607 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350029420 con 0x55ce2a12d240 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.546+0000 7fd336ffd640 1 -- 172.21.15.83:0/641241607 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd350029580 con 0x55ce2a12d240 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd360a22740 1 -- 172.21.15.83:0/641241607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce29d8b7e0 con 0x55ce2a12d240 2024-03-11T19:44:31.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd336ffd640 1 -- 172.21.15.83:0/641241607 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd350029740 con 0x55ce2a12d240 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd360a22740 10 client.25888 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd360a22740 1 -- 172.21.15.83:0/641241607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ce2a08a610 con 0x55ce2a12d240 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd336ffd640 1 -- 172.21.15.83:0/641241607 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ce29d8b7e0 con 0x55ce2a12d240 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd336ffd640 1 -- 172.21.15.83:0/641241607 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd350032070 con 0x55ce2a12d240 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd360a22740 10 client.25888 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd360a22740 10 client.25888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd360a22740 20 client.25888 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd360a22740 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ce2a13d9e0 0x55ce2a13fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd360a22740 4 client.25888 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd360a22740 1 -- 172.21.15.83:0/641241607 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce29dc8720 con 0x55ce2a13d9e0 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.547+0000 7fd3577fe640 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ce2a13d9e0 0x55ce2a13fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.548+0000 7fd336ffd640 10 client.25888 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:31.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.548+0000 7fd3577fe640 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ce2a13d9e0 0x55ce2a13fe20 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d1be1bd8e948ef8 server_cookie=fa37ca22e09429ea in_seq=0 out_seq=0 2024-03-11T19:44:31.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.548+0000 7fd336ffd640 1 -- 172.21.15.83:0/641241607 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ce29dc8720 con 0x55ce2a13d9e0 2024-03-11T19:44:31.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.548+0000 7fd336ffd640 10 client.25888 handle_command_reply: tid=0 2024-03-11T19:44:31.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 1 client.25888 shutdown 2024-03-11T19:44:31.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 1 -- 172.21.15.83:0/641241607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 msgr2=0x55ce2a138b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 0x55ce2a138b90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd35000de30 tx=0x7fd350004a40 comp rx=0 tx=0).stop 2024-03-11T19:44:31.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 1 -- 172.21.15.83:0/641241607 shutdown_connections 2024-03-11T19:44:31.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ce2a13d9e0 0x55ce2a13fe20 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a12d240 0x55ce2a138b90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd336ffd640 0 client.25888 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:31.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a1280d0 0x55ce2a131b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 1 --2- 172.21.15.83:0/641241607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1276e0 0x55ce2a131610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 1 -- 172.21.15.83:0/641241607 >> 172.21.15.83:0/641241607 conn(0x55ce2a11e7f0 msgr2=0x55ce2a11da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:31.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 1 -- 172.21.15.83:0/641241607 shutdown_connections 2024-03-11T19:44:31.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 1 -- 172.21.15.83:0/641241607 wait complete. 2024-03-11T19:44:31.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.549+0000 7fd360a22740 20 client.25888 trim_cache size 0 max 16384 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.570+0000 7fd360a22740 1 Processor -- start 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.570+0000 7fd360a22740 1 -- start start 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.570+0000 7fd360a22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 0x55ce2a129310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.570+0000 7fd360a22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a129850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.570+0000 7fd360a22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a12d240 0x55ce2a138b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.570+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 0x55ce2a129310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.570+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 0x55ce2a129310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:49224/0 (socket says 172.21.15.83:49224) 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.570+0000 7fd3577fe640 1 -- 172.21.15.83:0/1086718322 learned_addr learned my addr 172.21.15.83:0/1086718322 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.570+0000 7fd360a22740 1 -- 172.21.15.83:0/1086718322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2a119f80 con 0x55ce2a1280d0 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.570+0000 7fd360a22740 1 -- 172.21.15.83:0/1086718322 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2a11f2e0 con 0x55ce2a12d240 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.570+0000 7fd360a22740 1 -- 172.21.15.83:0/1086718322 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce29dc8720 con 0x55ce2a1276e0 2024-03-11T19:44:31.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.571+0000 7fd357fff640 1 --2- 172.21.15.83:0/1086718322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a12d240 0x55ce2a138b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.571+0000 7fd3577fe640 1 -- 172.21.15.83:0/1086718322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a12d240 msgr2=0x55ce2a138b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:31.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.571+0000 7fd3577fe640 1 --2- 172.21.15.83:0/1086718322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a12d240 0x55ce2a138b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.571+0000 7fd3577fe640 1 -- 172.21.15.83:0/1086718322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 msgr2=0x55ce2a129850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:31.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.571+0000 7fd3577fe640 1 --2- 172.21.15.83:0/1086718322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a129850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.571+0000 7fd3577fe640 1 -- 172.21.15.83:0/1086718322 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce2a026ac0 con 0x55ce2a1276e0 2024-03-11T19:44:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.571+0000 7fd3577fe640 1 --2- 172.21.15.83:0/1086718322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 0x55ce2a129310 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd348003730 tx=0x7fd3480eca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4964ab163aa7083d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.571+0000 7fd33d7fa640 1 -- 172.21.15.83:0/1086718322 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3480c0180 con 0x55ce2a1276e0 2024-03-11T19:44:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.571+0000 7fd33d7fa640 1 -- 172.21.15.83:0/1086718322 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3480c47d0 con 0x55ce2a1276e0 2024-03-11T19:44:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.571+0000 7fd33d7fa640 1 -- 172.21.15.83:0/1086718322 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3480b8960 con 0x55ce2a1276e0 2024-03-11T19:44:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.572+0000 7fd360a22740 1 -- 172.21.15.83:0/1086718322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 msgr2=0x55ce2a129310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.572+0000 7fd360a22740 1 --2- 172.21.15.83:0/1086718322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 0x55ce2a129310 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd348003730 tx=0x7fd3480eca90 comp rx=0 tx=0).stop 2024-03-11T19:44:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.573+0000 7fd360a22740 1 -- 172.21.15.83:0/1086718322 shutdown_connections 2024-03-11T19:44:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.573+0000 7fd360a22740 1 --2- 172.21.15.83:0/1086718322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a12d240 0x55ce2a138b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.573+0000 7fd360a22740 1 --2- 172.21.15.83:0/1086718322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a129850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.573+0000 7fd360a22740 1 --2- 172.21.15.83:0/1086718322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 0x55ce2a129310 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.573+0000 7fd360a22740 1 -- 172.21.15.83:0/1086718322 >> 172.21.15.83:0/1086718322 conn(0x55ce2a11e7f0 msgr2=0x55ce2a11ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.573+0000 7fd360a22740 1 -- 172.21.15.83:0/1086718322 shutdown_connections 2024-03-11T19:44:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.573+0000 7fd360a22740 1 -- 172.21.15.83:0/1086718322 wait complete. 2024-03-11T19:44:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.573+0000 7fd360a22740 1 -- start start 2024-03-11T19:44:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd360a22740 1 Processor -- start 2024-03-11T19:44:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd360a22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 0x55ce2a133350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd360a22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a1388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd360a22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a12d240 0x55ce2a133890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd360a22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2a133dd0 con 0x55ce2a1280d0 2024-03-11T19:44:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd360a22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2a133f40 con 0x55ce2a12d240 2024-03-11T19:44:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd360a22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2a1a9730 con 0x55ce2a1276e0 2024-03-11T19:44:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd356ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a1388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd356ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a1388c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40234/0 (socket says 172.21.15.83:40234) 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd356ffd640 1 -- 172.21.15.83:0/684714907 learned_addr learned my addr 172.21.15.83:0/684714907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd360a22740 10 client.0 _ll_register_callbacks cb 0x7fd34c00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 0x55ce2a133350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd357fff640 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a12d240 0x55ce2a133890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd356ffd640 1 -- 172.21.15.83:0/684714907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 msgr2=0x55ce2a133350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd356ffd640 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 0x55ce2a133350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd356ffd640 1 -- 172.21.15.83:0/684714907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a12d240 msgr2=0x55ce2a133890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd356ffd640 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a12d240 0x55ce2a133890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd356ffd640 1 -- 172.21.15.83:0/684714907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35819f650 con 0x55ce2a1280d0 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.574+0000 7fd357fff640 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a12d240 0x55ce2a133890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.575+0000 7fd356ffd640 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a1388c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00beb0 tx=0x7fd34c00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9181e0e405a20eb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.575+0000 7fd336ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:44:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.575+0000 7fd336ffd640 1 -- 172.21.15.83:0/684714907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c020070 con 0x55ce2a1280d0 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.575+0000 7fd336ffd640 1 -- 172.21.15.83:0/684714907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd34c017070 con 0x55ce2a1280d0 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.575+0000 7fd360a22740 1 -- 172.21.15.83:0/684714907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd34c015070 con 0x55ce2a1280d0 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.575+0000 7fd336ffd640 1 -- 172.21.15.83:0/684714907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd34c0052e0 con 0x55ce2a1280d0 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.575+0000 7fd360a22740 10 client.16311 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.575+0000 7fd360a22740 1 -- 172.21.15.83:0/684714907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ce2a04c6a0 con 0x55ce2a1280d0 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.575+0000 7fd336ffd640 1 -- 172.21.15.83:0/684714907 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd34c005440 con 0x55ce2a1280d0 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.575+0000 7fd336ffd640 1 -- 172.21.15.83:0/684714907 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fd34c011440 con 0x55ce2a1280d0 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.575+0000 7fd360a22740 10 client.16311 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.576+0000 7fd360a22740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.576+0000 7fd360a22740 20 client.16311 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.576+0000 7fd360a22740 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ce2a13d9e0 0x55ce2a1ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.576+0000 7fd360a22740 4 client.16311 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.576+0000 7fd360a22740 1 -- 172.21.15.83:0/684714907 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fd34c015070 con 0x55ce2a13d9e0 2024-03-11T19:44:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.576+0000 7fd3577fe640 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ce2a13d9e0 0x55ce2a1ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd3577fe640 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ce2a13d9e0 0x55ce2a1ad0e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3090920c08428f server_cookie=4521f7985d85a772 in_seq=0 out_seq=0 2024-03-11T19:44:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd336ffd640 10 client.16311 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd336ffd640 1 -- 172.21.15.83:0/684714907 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fd34c015070 con 0x55ce2a13d9e0 2024-03-11T19:44:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd336ffd640 10 client.16311 handle_command_reply: tid=0 2024-03-11T19:44:31.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd360a22740 1 client.16311 shutdown 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd360a22740 1 -- 172.21.15.83:0/684714907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 msgr2=0x55ce2a1388c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd360a22740 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a1388c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00beb0 tx=0x7fd34c00bee0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd360a22740 1 -- 172.21.15.83:0/684714907 shutdown_connections 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd360a22740 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ce2a13d9e0 0x55ce2a1ad0e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd360a22740 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ce2a12d240 0x55ce2a133890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd360a22740 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce2a1280d0 0x55ce2a1388c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd360a22740 1 --2- 172.21.15.83:0/684714907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce2a1276e0 0x55ce2a133350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd360a22740 1 -- 172.21.15.83:0/684714907 >> 172.21.15.83:0/684714907 conn(0x55ce2a11e7f0 msgr2=0x55ce2a12c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.577+0000 7fd336ffd640 0 client.16311 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.578+0000 7fd360a22740 1 -- 172.21.15.83:0/684714907 shutdown_connections 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.578+0000 7fd360a22740 1 -- 172.21.15.83:0/684714907 wait complete. 2024-03-11T19:44:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.578+0000 7fd360a22740 20 client.16311 trim_cache size 0 max 16384 2024-03-11T19:44:31.586 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:44:31.586 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:44:31.586 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:44:31.586 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:44:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 -- 172.21.15.83:0/2563586538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3400a1b30 msgr2=0x7fd3400a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3400a1b30 0x7fd3400a3ff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55ce2a128930 tx=0x7fd350012040 comp rx=0 tx=0).stop 2024-03-11T19:44:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 -- 172.21.15.83:0/2563586538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd358101c20 msgr2=0x7fd358197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd358101c20 0x7fd358197240 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd348005940 tx=0x7fd348035f90 comp rx=0 tx=0).stop 2024-03-11T19:44:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 -- 172.21.15.83:0/2563586538 shutdown_connections 2024-03-11T19:44:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3400a1b30 0x7fd3400a3ff0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd358106e80 0x7fd35819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358104550 0x7fd358197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 --2- 172.21.15.83:0/2563586538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd358101c20 0x7fd358197240 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 -- 172.21.15.83:0/2563586538 >> 172.21.15.83:0/2563586538 conn(0x7fd3580fb650 msgr2=0x7fd358102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:31.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 -- 172.21.15.83:0/2563586538 shutdown_connections 2024-03-11T19:44:31.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.579+0000 7fd35dd77640 1 -- 172.21.15.83:0/2563586538 wait complete. 2024-03-11T19:44:31.597 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:44:31.597 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:44:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.740+0000 7f1b11e55640 1 -- 172.21.15.83:0/1544667528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0c0ff790 msgr2=0x7f1b0c106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.740+0000 7f1b11e55640 1 --2- 172.21.15.83:0/1544667528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0c0ff790 0x7f1b0c106970 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009900 tx=0x7f1b0002bc30 comp rx=0 tx=0).stop 2024-03-11T19:44:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.741+0000 7f1b11e55640 1 -- 172.21.15.83:0/1544667528 shutdown_connections 2024-03-11T19:44:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.741+0000 7f1b11e55640 1 --2- 172.21.15.83:0/1544667528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b0c106eb0 0x7f1b0c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.741+0000 7f1b11e55640 1 --2- 172.21.15.83:0/1544667528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0c0ff790 0x7f1b0c106970 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.741+0000 7f1b11e55640 1 --2- 172.21.15.83:0/1544667528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b0c0feda0 0x7f1b0c0ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.741+0000 7f1b11e55640 1 -- 172.21.15.83:0/1544667528 >> 172.21.15.83:0/1544667528 conn(0x7f1b0c0fa980 msgr2=0x7f1b0c0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.741+0000 7f1b11e55640 1 -- 172.21.15.83:0/1544667528 shutdown_connections 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.741+0000 7f1b11e55640 1 -- 172.21.15.83:0/1544667528 wait complete. 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.741+0000 7f1b11e55640 1 Processor -- start 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.741+0000 7f1b11e55640 1 -- start start 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b11e55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0c0feda0 0x7f1b0c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b11e55640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b0c0ff790 0x7f1b0c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b11e55640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b0c106eb0 0x7f1b0c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b11e55640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c19ed40 con 0x7f1b0c0feda0 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b11e55640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c19eeb0 con 0x7f1b0c106eb0 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b11e55640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c19f070 con 0x7f1b0c0ff790 2024-03-11T19:44:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0c0feda0 0x7f1b0c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0c0feda0 0x7f1b0c197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40262/0 (socket says 172.21.15.83:40262) 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0b7fe640 1 -- 172.21.15.83:0/3580789116 learned_addr learned my addr 172.21.15.83:0/3580789116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0affd640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b0c0ff790 0x7f1b0c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0bfff640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b0c106eb0 0x7f1b0c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0b7fe640 1 -- 172.21.15.83:0/3580789116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b0c0ff790 msgr2=0x7f1b0c197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0b7fe640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b0c0ff790 0x7f1b0c197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0b7fe640 1 -- 172.21.15.83:0/3580789116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b0c106eb0 msgr2=0x7f1b0c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0b7fe640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b0c106eb0 0x7f1b0c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0b7fe640 1 -- 172.21.15.83:0/3580789116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b00009560 con 0x7f1b0c0feda0 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0bfff640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b0c106eb0 0x7f1b0c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b0b7fe640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0c0feda0 0x7f1b0c197250 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00d950 tx=0x7f1afc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.742+0000 7f1b08ff9640 1 -- 172.21.15.83:0/3580789116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc009a30 con 0x7f1b0c0feda0 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.743+0000 7f1b08ff9640 1 -- 172.21.15.83:0/3580789116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1afc009b90 con 0x7f1b0c0feda0 2024-03-11T19:44:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.743+0000 7f1b11e55640 1 -- 172.21.15.83:0/3580789116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c19f350 con 0x7f1b0c0feda0 2024-03-11T19:44:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.743+0000 7f1b11e55640 1 -- 172.21.15.83:0/3580789116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c19f880 con 0x7f1b0c0feda0 2024-03-11T19:44:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.743+0000 7f1b08ff9640 1 -- 172.21.15.83:0/3580789116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc004a00 con 0x7f1b0c0feda0 2024-03-11T19:44:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.744+0000 7f1b11e55640 1 -- 172.21.15.83:0/3580789116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0c105780 con 0x7f1b0c0feda0 2024-03-11T19:44:31.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.747+0000 7f1b08ff9640 1 -- 172.21.15.83:0/3580789116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1afc009d00 con 0x7f1b0c0feda0 2024-03-11T19:44:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.748+0000 7f1b08ff9640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1af40a1b30 0x7f1af40a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.748+0000 7f1b08ff9640 1 -- 172.21.15.83:0/3580789116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1afc0cd4a0 con 0x7f1b0c0feda0 2024-03-11T19:44:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.748+0000 7f1b0affd640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1af40a1b30 0x7f1af40a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:31.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.749+0000 7f1b0affd640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1af40a1b30 0x7f1af40a3ff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1b00000c00 tx=0x7f1b0002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:31.751+0000 7f1b08ff9640 1 -- 172.21.15.83:0/3580789116 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1afc096740 con 0x7f1b0c0feda0 2024-03-11T19:44:32.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.110+0000 7f1b11e55640 1 -- 172.21.15.83:0/3580789116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b0c10cbb0 con 0x7f1b0c0feda0 2024-03-11T19:44:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.111+0000 7f1b08ff9640 1 -- 172.21.15.83:0/3580789116 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11350 (secure 0 0 0) 0x7f1afc017070 con 0x7f1b0c0feda0 2024-03-11T19:44:32.115 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:44:32.116 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:43:42.002107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:44:32.116 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-03-11T19:44:32.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 -- 172.21.15.83:0/3580789116 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1af40a1b30 msgr2=0x7f1af40a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1af40a1b30 0x7f1af40a3ff0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1b00000c00 tx=0x7f1b0002c7e0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 -- 172.21.15.83:0/3580789116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0c0feda0 msgr2=0x7f1b0c197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0c0feda0 0x7f1b0c197250 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00d950 tx=0x7f1afc00de10 comp rx=0 tx=0).stop 2024-03-11T19:44:32.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 -- 172.21.15.83:0/3580789116 shutdown_connections 2024-03-11T19:44:32.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1af40a1b30 0x7f1af40a3ff0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b0c106eb0 0x7f1b0c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b0c0ff790 0x7f1b0c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 --2- 172.21.15.83:0/3580789116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b0c0feda0 0x7f1b0c197250 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 -- 172.21.15.83:0/3580789116 >> 172.21.15.83:0/3580789116 conn(0x7f1b0c0fa980 msgr2=0x7f1b0c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 -- 172.21.15.83:0/3580789116 shutdown_connections 2024-03-11T19:44:32.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.114+0000 7f1b11e55640 1 -- 172.21.15.83:0/3580789116 wait complete. 2024-03-11T19:44:32.133 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:44:32.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:44:32.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.292+0000 7f009d20b640 1 -- 172.21.15.83:0/1537839511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f009810aaf0 msgr2=0x7f009810cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.292+0000 7f009d20b640 1 --2- 172.21.15.83:0/1537839511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f009810aaf0 0x7f009810cf60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0088009be0 tx=0x7f008802bf10 comp rx=0 tx=0).stop 2024-03-11T19:44:32.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.294+0000 7f009d20b640 1 -- 172.21.15.83:0/1537839511 shutdown_connections 2024-03-11T19:44:32.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.294+0000 7f009d20b640 1 --2- 172.21.15.83:0/1537839511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f009810d4a0 0x7f009810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.294+0000 7f009d20b640 1 --2- 172.21.15.83:0/1537839511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f009810aaf0 0x7f009810cf60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.294+0000 7f009d20b640 1 --2- 172.21.15.83:0/1537839511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0098108180 0x7f009810a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.294+0000 7f009d20b640 1 -- 172.21.15.83:0/1537839511 >> 172.21.15.83:0/1537839511 conn(0x7f009806dfe0 msgr2=0x7f0098070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.295+0000 7f009d20b640 1 -- 172.21.15.83:0/1537839511 shutdown_connections 2024-03-11T19:44:32.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.296+0000 7f009d20b640 1 -- 172.21.15.83:0/1537839511 wait complete. 2024-03-11T19:44:32.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.297+0000 7f009d20b640 1 Processor -- start 2024-03-11T19:44:32.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.297+0000 7f009d20b640 1 -- start start 2024-03-11T19:44:32.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.297+0000 7f009d20b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0098108180 0x7f0098118730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.297+0000 7f009d20b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f009810aaf0 0x7f0098118c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.297+0000 7f009d20b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f009810d4a0 0x7f00980793e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.297+0000 7f009d20b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0098119500 con 0x7f009810d4a0 2024-03-11T19:44:32.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.297+0000 7f009d20b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0098079920 con 0x7f009810aaf0 2024-03-11T19:44:32.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.297+0000 7f009d20b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0098079ae0 con 0x7f0098108180 2024-03-11T19:44:32.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.298+0000 7f009659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f009810aaf0 0x7f0098118c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.298+0000 7f009659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f009810aaf0 0x7f0098118c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:38256/0 (socket says 172.21.15.83:38256) 2024-03-11T19:44:32.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.298+0000 7f009659c640 1 -- 172.21.15.83:0/3612489909 learned_addr learned my addr 172.21.15.83:0/3612489909 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.298+0000 7f009759e640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f009810d4a0 0x7f00980793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.298+0000 7f0096d9d640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0098108180 0x7f0098118730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.298+0000 7f009659c640 1 -- 172.21.15.83:0/3612489909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0098108180 msgr2=0x7f0098118730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.298+0000 7f009659c640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0098108180 0x7f0098118730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.298+0000 7f009659c640 1 -- 172.21.15.83:0/3612489909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f009810d4a0 msgr2=0x7f00980793e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.298+0000 7f009659c640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f009810d4a0 0x7f00980793e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.298+0000 7f009659c640 1 -- 172.21.15.83:0/3612489909 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0088009840 con 0x7f009810aaf0 2024-03-11T19:44:32.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.298+0000 7f009659c640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f009810aaf0 0x7f0098118c70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0088000f80 tx=0x7f0088003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.299+0000 7f008ffff640 1 -- 172.21.15.83:0/3612489909 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088003e40 con 0x7f009810aaf0 2024-03-11T19:44:32.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.299+0000 7f008ffff640 1 -- 172.21.15.83:0/3612489909 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0088004730 con 0x7f009810aaf0 2024-03-11T19:44:32.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.299+0000 7f008ffff640 1 -- 172.21.15.83:0/3612489909 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008803e450 con 0x7f009810aaf0 2024-03-11T19:44:32.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.299+0000 7f009d20b640 1 -- 172.21.15.83:0/3612489909 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0098079d60 con 0x7f009810aaf0 2024-03-11T19:44:32.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.299+0000 7f009d20b640 1 -- 172.21.15.83:0/3612489909 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009807a220 con 0x7f009810aaf0 2024-03-11T19:44:32.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.302+0000 7f009fdd6740 1 Processor -- start 2024-03-11T19:44:32.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.302+0000 7f009fdd6740 1 -- start start 2024-03-11T19:44:32.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c17525d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c175262760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c175262dd0 0x55c175265260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009fdd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c175219960 con 0x55c17525d270 2024-03-11T19:44:32.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009fdd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1751cd310 con 0x55c17525dc60 2024-03-11T19:44:32.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009fdd6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1751ca840 con 0x55c175262dd0 2024-03-11T19:44:32.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c17525d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c17525d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40282/0 (socket says 172.21.15.83:40282) 2024-03-11T19:44:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009759e640 1 -- 172.21.15.83:0/1917780354 learned_addr learned my addr 172.21.15.83:0/1917780354 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f0096d9d640 1 --2- 172.21.15.83:0/1917780354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c175262760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009759e640 1 -- 172.21.15.83:0/1917780354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c175262dd0 msgr2=0x55c175265260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:44:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009759e640 1 --2- 172.21.15.83:0/1917780354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c175262dd0 0x55c175265260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009759e640 1 -- 172.21.15.83:0/1917780354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 msgr2=0x55c175262760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009759e640 1 --2- 172.21.15.83:0/1917780354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c175262760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.303+0000 7f009759e640 1 -- 172.21.15.83:0/1917780354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c175188060 con 0x55c17525d270 2024-03-11T19:44:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.304+0000 7f0096d9d640 1 --2- 172.21.15.83:0/1917780354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c175262760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.304+0000 7f008ffff640 1 -- 172.21.15.83:0/3612489909 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0088034530 con 0x7f009810aaf0 2024-03-11T19:44:32.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.304+0000 7f009759e640 1 --2- 172.21.15.83:0/1917780354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c17525d690 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f008000dc30 tx=0x7f008000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a117655789c5ecb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.304+0000 7f008d7fa640 1 -- 172.21.15.83:0/1917780354 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080011440 con 0x55c17525d270 2024-03-11T19:44:32.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.304+0000 7f008d7fa640 1 -- 172.21.15.83:0/1917780354 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0080011a80 con 0x55c17525d270 2024-03-11T19:44:32.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.304+0000 7f008d7fa640 1 -- 172.21.15.83:0/1917780354 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008001ba00 con 0x55c17525d270 2024-03-11T19:44:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.305+0000 7f008ffff640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00840a1c40 0x7f00840a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.305+0000 7f0096d9d640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00840a1c40 0x7f00840a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.305+0000 7f008ffff640 1 -- 172.21.15.83:0/3612489909 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f00880f0010 con 0x7f009810aaf0 2024-03-11T19:44:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.305+0000 7f009fdd6740 1 -- 172.21.15.83:0/1917780354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 msgr2=0x55c17525d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.305+0000 7f009fdd6740 1 --2- 172.21.15.83:0/1917780354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c17525d690 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f008000dc30 tx=0x7f008000f730 comp rx=0 tx=0).stop 2024-03-11T19:44:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.305+0000 7f0096d9d640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00840a1c40 0x7f00840a4100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55c17525e060 tx=0x7f009000b4d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.305+0000 7f009fdd6740 1 -- 172.21.15.83:0/1917780354 shutdown_connections 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.305+0000 7f009fdd6740 1 --2- 172.21.15.83:0/1917780354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c175262dd0 0x55c175265260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.306+0000 7f009fdd6740 1 --2- 172.21.15.83:0/1917780354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c175262760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.306+0000 7f009fdd6740 1 --2- 172.21.15.83:0/1917780354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c17525d690 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.306+0000 7f009fdd6740 1 -- 172.21.15.83:0/1917780354 >> 172.21.15.83:0/1917780354 conn(0x55c1752587e0 msgr2=0x55c175258be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.306+0000 7f009fdd6740 1 -- 172.21.15.83:0/1917780354 shutdown_connections 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.306+0000 7f009fdd6740 1 -- 172.21.15.83:0/1917780354 wait complete. 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.306+0000 7f009fdd6740 1 -- start start 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f009fdd6740 1 Processor -- start 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c175267550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c175267a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c175262dd0 0x55c17526ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f009fdd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1751c6020 con 0x55c17525d270 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f0096d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c175267550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f0096d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c175267550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40286/0 (socket says 172.21.15.83:40286) 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f0096d9d640 1 -- 172.21.15.83:0/4186293991 learned_addr learned my addr 172.21.15.83:0/4186293991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f009759e640 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c175262dd0 0x55c17526ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f009fdd6740 1 -- 172.21.15.83:0/4186293991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c174e99430 con 0x55c17525dc60 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f009fdd6740 1 -- 172.21.15.83:0/4186293991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c174e55e00 con 0x55c175262dd0 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f009659c640 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c175267a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.307+0000 7f009fdd6740 10 client.0 _ll_register_callbacks cb 0x55c17516dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f009759e640 1 -- 172.21.15.83:0/4186293991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 msgr2=0x55c175267a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f009759e640 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c175267a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f009759e640 1 -- 172.21.15.83:0/4186293991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 msgr2=0x55c175267550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f009759e640 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c175267550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f009759e640 1 -- 172.21.15.83:0/4186293991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008000d860 con 0x55c175262dd0 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f0096d9d640 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c175267550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f009759e640 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c175262dd0 0x55c17526ead0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0080000c00 tx=0x7f0080012c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=603e5190b5c2303c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f007e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f007e7fc640 1 -- 172.21.15.83:0/4186293991 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008001f040 con 0x55c175262dd0 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f007e7fc640 1 -- 172.21.15.83:0/4186293991 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0080011640 con 0x55c175262dd0 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f009fdd6740 1 -- 172.21.15.83:0/4186293991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c174ec57e0 con 0x55c175262dd0 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f007e7fc640 1 -- 172.21.15.83:0/4186293991 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0080011800 con 0x55c175262dd0 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f009fdd6740 10 client.25909 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f009fdd6740 1 -- 172.21.15.83:0/4186293991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c1751c46a0 con 0x55c175262dd0 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.308+0000 7f007e7fc640 1 -- 172.21.15.83:0/4186293991 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0080035020 con 0x55c175262dd0 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.309+0000 7f007e7fc640 1 -- 172.21.15.83:0/4186293991 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f008001bd20 con 0x55c175262dd0 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.309+0000 7f009fdd6740 10 client.25909 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.309+0000 7f009fdd6740 10 client.25909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:44:32.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.309+0000 7f009fdd6740 20 client.25909 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:32.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.309+0000 7f009fdd6740 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c1752735e0 0x55c175275a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.309+0000 7f009fdd6740 4 client.25909 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:44:32.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.309+0000 7f009fdd6740 1 -- 172.21.15.83:0/4186293991 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c174f02720 con 0x55c1752735e0 2024-03-11T19:44:32.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.309+0000 7f0096d9d640 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c1752735e0 0x55c175275a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.309+0000 7f007e7fc640 10 client.25909 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:44:32.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.309+0000 7f0096d9d640 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c1752735e0 0x55c175275a00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24dfe918bd0193b4 server_cookie=9424e6b6b4bcb967 in_seq=0 out_seq=0 2024-03-11T19:44:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.310+0000 7f007e7fc640 1 -- 172.21.15.83:0/4186293991 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c174f02720 con 0x55c1752735e0 2024-03-11T19:44:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.310+0000 7f007e7fc640 10 client.25909 handle_command_reply: tid=0 2024-03-11T19:44:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.310+0000 7f009fdd6740 1 client.25909 shutdown 2024-03-11T19:44:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f009fdd6740 1 -- 172.21.15.83:0/4186293991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c175262dd0 msgr2=0x55c17526ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f009fdd6740 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c175262dd0 0x55c17526ead0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0080000c00 tx=0x7f0080012c00 comp rx=0 tx=0).stop 2024-03-11T19:44:32.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f009fdd6740 1 -- 172.21.15.83:0/4186293991 shutdown_connections 2024-03-11T19:44:32.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f009fdd6740 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c1752735e0 0x55c175275a00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f009fdd6740 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c175262dd0 0x55c17526ead0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f009fdd6740 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c175267a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f009fdd6740 1 --2- 172.21.15.83:0/4186293991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525d270 0x55c175267550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f009fdd6740 1 -- 172.21.15.83:0/4186293991 >> 172.21.15.83:0/4186293991 conn(0x55c1752587e0 msgr2=0x55c175257b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f007e7fc640 0 client.25909 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:44:32.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f009fdd6740 1 -- 172.21.15.83:0/4186293991 shutdown_connections 2024-03-11T19:44:32.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f009fdd6740 1 -- 172.21.15.83:0/4186293991 wait complete. 2024-03-11T19:44:32.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.311+0000 7f009fdd6740 20 client.25909 trim_cache size 0 max 16384 2024-03-11T19:44:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.328+0000 7f009fdd6740 1 Processor -- start 2024-03-11T19:44:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.328+0000 7f009fdd6740 1 -- start start 2024-03-11T19:44:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.328+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 0x55c17525f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.328+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525dc60 0x55c17525f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.328+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c175262dd0 0x55c17526ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.328+0000 7f009fdd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17514b130 con 0x55c17525dc60 2024-03-11T19:44:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009fdd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c175253a70 con 0x55c175262dd0 2024-03-11T19:44:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009fdd6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c175259780 con 0x55c17525d270 2024-03-11T19:44:32.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525dc60 0x55c17525f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525dc60 0x55c17525f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40294/0 (socket says 172.21.15.83:40294) 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009659c640 1 -- 172.21.15.83:0/3400431428 learned_addr learned my addr 172.21.15.83:0/3400431428 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f0096d9d640 1 --2- 172.21.15.83:0/3400431428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 0x55c17525f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009659c640 1 -- 172.21.15.83:0/3400431428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 msgr2=0x55c17525f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009759e640 1 --2- 172.21.15.83:0/3400431428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c175262dd0 0x55c17526ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009659c640 1 --2- 172.21.15.83:0/3400431428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 0x55c17525f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009659c640 1 -- 172.21.15.83:0/3400431428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c175262dd0 msgr2=0x55c17526ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009659c640 1 --2- 172.21.15.83:0/3400431428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c175262dd0 0x55c17526ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009659c640 1 -- 172.21.15.83:0/3400431428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c175160c20 con 0x55c17525dc60 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f009659c640 1 --2- 172.21.15.83:0/3400431428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525dc60 0x55c17525f770 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0088004950 tx=0x7f00880348d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7436810f7bcdfc6e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f008cff9640 1 -- 172.21.15.83:0/3400431428 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00880bc8f0 con 0x55c17525dc60 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f008cff9640 1 -- 172.21.15.83:0/3400431428 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00880bd690 con 0x55c17525dc60 2024-03-11T19:44:32.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.329+0000 7f008cff9640 1 -- 172.21.15.83:0/3400431428 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00880b9600 con 0x55c17525dc60 2024-03-11T19:44:32.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.330+0000 7f009fdd6740 1 -- 172.21.15.83:0/3400431428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525dc60 msgr2=0x55c17525f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.330+0000 7f009fdd6740 1 --2- 172.21.15.83:0/3400431428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525dc60 0x55c17525f770 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0088004950 tx=0x7f00880348d0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.330+0000 7f009fdd6740 1 -- 172.21.15.83:0/3400431428 shutdown_connections 2024-03-11T19:44:32.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.330+0000 7f009fdd6740 1 --2- 172.21.15.83:0/3400431428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c175262dd0 0x55c17526ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.331+0000 7f009fdd6740 1 --2- 172.21.15.83:0/3400431428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c17525dc60 0x55c17525f770 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.331+0000 7f009fdd6740 1 --2- 172.21.15.83:0/3400431428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 0x55c17525f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.331+0000 7f009fdd6740 1 -- 172.21.15.83:0/3400431428 >> 172.21.15.83:0/3400431428 conn(0x55c1752587e0 msgr2=0x55c175258bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.331+0000 7f009fdd6740 1 -- 172.21.15.83:0/3400431428 shutdown_connections 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.331+0000 7f009fdd6740 1 -- 172.21.15.83:0/3400431428 wait complete. 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.331+0000 7f009fdd6740 1 -- start start 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.331+0000 7f009fdd6740 1 Processor -- start 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.332+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 0x55c17526c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c17526ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f009fdd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c175262dd0 0x55c17526de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f0096d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 0x55c17526c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f009fdd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17526e3c0 con 0x55c175262dd0 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f009fdd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17526e530 con 0x55c17525dc60 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f009fdd6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17526e6f0 con 0x55c17525d270 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f0096d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 0x55c17526c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:49304/0 (socket says 172.21.15.83:49304) 2024-03-11T19:44:32.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f0096d9d640 1 -- 172.21.15.83:0/3149971281 learned_addr learned my addr 172.21.15.83:0/3149971281 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f009fdd6740 10 client.0 _ll_register_callbacks cb 0x7f008000d3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f009659c640 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c17526ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f009759e640 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c175262dd0 0x55c17526de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f0096d9d640 1 -- 172.21.15.83:0/3149971281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 msgr2=0x55c17526ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f0096d9d640 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c17526ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f0096d9d640 1 -- 172.21.15.83:0/3149971281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c175262dd0 msgr2=0x55c17526de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f0096d9d640 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c175262dd0 0x55c17526de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.333+0000 7f0096d9d640 1 -- 172.21.15.83:0/3149971281 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009807a220 con 0x55c17525d270 2024-03-11T19:44:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.334+0000 7f0096d9d640 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 0x55c17526c930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55c175276310 tx=0x7f0090014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afa6af7691fb1d1e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.334+0000 7f007e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:44:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.334+0000 7f007e7fc640 1 -- 172.21.15.83:0/3149971281 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0090027600 con 0x55c17525d270 2024-03-11T19:44:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.334+0000 7f007e7fc640 1 -- 172.21.15.83:0/3149971281 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0090027760 con 0x55c17525d270 2024-03-11T19:44:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.334+0000 7f009fdd6740 1 -- 172.21.15.83:0/3149971281 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0080011a80 con 0x55c17525d270 2024-03-11T19:44:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.334+0000 7f007e7fc640 1 -- 172.21.15.83:0/3149971281 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0090027920 con 0x55c17525d270 2024-03-11T19:44:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.334+0000 7f009fdd6740 10 client.25915 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.334+0000 7f009fdd6740 1 -- 172.21.15.83:0/3149971281 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c1751868c0 con 0x55c17525d270 2024-03-11T19:44:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.335+0000 7f007e7fc640 1 -- 172.21.15.83:0/3149971281 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0080011a80 con 0x55c17525d270 2024-03-11T19:44:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.335+0000 7f007e7fc640 1 -- 172.21.15.83:0/3149971281 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f00900308d0 con 0x55c17525d270 2024-03-11T19:44:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.335+0000 7f009fdd6740 10 client.25915 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:32.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.335+0000 7f009fdd6740 10 client.25915 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.335+0000 7f009fdd6740 20 client.25915 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.335+0000 7f009fdd6740 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c1752cac70 0x55c1752cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.335+0000 7f009fdd6740 4 client.25915 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.335+0000 7f009fdd6740 1 -- 172.21.15.83:0/3149971281 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c174f51890 con 0x55c1752cac70 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f009659c640 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c1752cac70 0x55c1752cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f009659c640 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c1752cac70 0x55c1752cd090 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89c43297389f0fb server_cookie=ed8ea3eaa9202718 in_seq=0 out_seq=0 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f007e7fc640 10 client.25915 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f007e7fc640 1 -- 172.21.15.83:0/3149971281 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c174f51890 con 0x55c1752cac70 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f007e7fc640 10 client.25915 handle_command_reply: tid=0 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f009fdd6740 1 client.25915 shutdown 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f009fdd6740 1 -- 172.21.15.83:0/3149971281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 msgr2=0x55c17526c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f009fdd6740 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 0x55c17526c930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55c175276310 tx=0x7f0090014040 comp rx=0 tx=0).stop 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f009fdd6740 1 -- 172.21.15.83:0/3149971281 shutdown_connections 2024-03-11T19:44:32.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f009fdd6740 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c1752cac70 0x55c1752cd090 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f009fdd6740 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c175262dd0 0x55c17526de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f009fdd6740 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c17525dc60 0x55c17526ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f009fdd6740 1 --2- 172.21.15.83:0/3149971281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17525d270 0x55c17526c930 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f009fdd6740 1 -- 172.21.15.83:0/3149971281 >> 172.21.15.83:0/3149971281 conn(0x55c1752587e0 msgr2=0x55c17525f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.336+0000 7f007e7fc640 0 client.25915 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:44:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009fdd6740 1 -- 172.21.15.83:0/3149971281 shutdown_connections 2024-03-11T19:44:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009fdd6740 1 -- 172.21.15.83:0/3149971281 wait complete. 2024-03-11T19:44:32.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009fdd6740 20 client.25915 trim_cache size 0 max 16384 2024-03-11T19:44:32.345 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:44:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009d20b640 1 -- 172.21.15.83:0/3612489909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00840a1c40 msgr2=0x7f00840a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009d20b640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00840a1c40 0x7f00840a4100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55c17525e060 tx=0x7f009000b4d0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009d20b640 1 -- 172.21.15.83:0/3612489909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f009810aaf0 msgr2=0x7f0098118c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009d20b640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f009810aaf0 0x7f0098118c70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0088000f80 tx=0x7f0088003c10 comp rx=0 tx=0).stop 2024-03-11T19:44:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009d20b640 1 -- 172.21.15.83:0/3612489909 shutdown_connections 2024-03-11T19:44:32.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009d20b640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f00840a1c40 0x7f00840a4100 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009d20b640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f009810d4a0 0x7f00980793e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009d20b640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f009810aaf0 0x7f0098118c70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009d20b640 1 --2- 172.21.15.83:0/3612489909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0098108180 0x7f0098118730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.337+0000 7f009d20b640 1 -- 172.21.15.83:0/3612489909 >> 172.21.15.83:0/3612489909 conn(0x7f009806dfe0 msgr2=0x7f009810b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.338+0000 7f009d20b640 1 -- 172.21.15.83:0/3612489909 shutdown_connections 2024-03-11T19:44:32.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.339+0000 7f009d20b640 1 -- 172.21.15.83:0/3612489909 wait complete. 2024-03-11T19:44:32.355 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:44:32.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:44:32.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:32 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3580789116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:44:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 315.0 s, 2450.5 tps, lat 40.734 ms stddev 44.899 2024-03-11T19:44:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.503+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3009645230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea8102e90 msgr2=0x7f7ea8109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.503+0000 7f7eaddf6640 1 --2- 172.21.15.83:0/3009645230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea8102e90 0x7f7ea8109720 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009ca0 tx=0x7f7ea002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.503+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3009645230 shutdown_connections 2024-03-11T19:44:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.503+0000 7f7eaddf6640 1 --2- 172.21.15.83:0/3009645230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea8102e90 0x7f7ea8109720 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.503+0000 7f7eaddf6640 1 --2- 172.21.15.83:0/3009645230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ea81024d0 0x7f7ea8102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.503+0000 7f7eaddf6640 1 --2- 172.21.15.83:0/3009645230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea80ff460 0x7f7ea80ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.503+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3009645230 >> 172.21.15.83:0/3009645230 conn(0x7f7ea80fad70 msgr2=0x7f7ea80fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.503+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3009645230 shutdown_connections 2024-03-11T19:44:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.503+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3009645230 wait complete. 2024-03-11T19:44:32.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.503+0000 7f7eaddf6640 1 Processor -- start 2024-03-11T19:44:32.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7eaddf6640 1 -- start start 2024-03-11T19:44:32.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7eaddf6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80ff460 0x7f7ea8197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7eaddf6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea81024d0 0x7f7ea8197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7eaddf6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ea8102e90 0x7f7ea819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7eaddf6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea819eb20 con 0x7f7ea80ff460 2024-03-11T19:44:32.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7eaddf6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea819ec90 con 0x7f7ea8102e90 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7eaddf6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea819ee50 con 0x7f7ea81024d0 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80ff460 0x7f7ea8197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7ea7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ea8102e90 0x7f7ea819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80ff460 0x7f7ea8197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40316/0 (socket says 172.21.15.83:40316) 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7ea77fe640 1 -- 172.21.15.83:0/3617379503 learned_addr learned my addr 172.21.15.83:0/3617379503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea81024d0 0x7f7ea8197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7ea7fff640 1 -- 172.21.15.83:0/3617379503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea81024d0 msgr2=0x7f7ea8197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7ea7fff640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea81024d0 0x7f7ea8197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7ea7fff640 1 -- 172.21.15.83:0/3617379503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80ff460 msgr2=0x7f7ea8197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7ea7fff640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80ff460 0x7f7ea8197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.504+0000 7f7ea7fff640 1 -- 172.21.15.83:0/3617379503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0009900 con 0x7f7ea8102e90 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.505+0000 7f7ea7fff640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ea8102e90 0x7f7ea819e5e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009bf0 tx=0x7f7ea0034cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.505+0000 7f7ea4ff9640 1 -- 172.21.15.83:0/3617379503 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0032b30 con 0x7f7ea8102e90 2024-03-11T19:44:32.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.505+0000 7f7ea4ff9640 1 -- 172.21.15.83:0/3617379503 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ea0003e50 con 0x7f7ea8102e90 2024-03-11T19:44:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.505+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3617379503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea819f0d0 con 0x7f7ea8102e90 2024-03-11T19:44:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.505+0000 7f7ea4ff9640 1 -- 172.21.15.83:0/3617379503 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00406b0 con 0x7f7ea8102e90 2024-03-11T19:44:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.505+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3617379503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea819f5a0 con 0x7f7ea8102e90 2024-03-11T19:44:32.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7eb09c5740 1 Processor -- start 2024-03-11T19:44:32.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7ea4ff9640 1 -- 172.21.15.83:0/3617379503 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7ea000a040 con 0x7f7ea8102e90 2024-03-11T19:44:32.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7eb09c5740 1 -- start start 2024-03-11T19:44:32.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446070 0x5604f0446490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446a60 0x5604f044aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f044b660 0x5604f044dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7eb09c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f03f9960 con 0x5604f0446a60 2024-03-11T19:44:32.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7eb09c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f03ad310 con 0x5604f0446070 2024-03-11T19:44:32.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7eb09c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f03aa840 con 0x5604f044b660 2024-03-11T19:44:32.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446a60 0x5604f044aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446a60 0x5604f044aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40318/0 (socket says 172.21.15.83:40318) 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7ea77fe640 1 -- 172.21.15.83:0/3824684339 learned_addr learned my addr 172.21.15.83:0/3824684339 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.508+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/3824684339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f044b660 0x5604f044dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/3824684339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446070 0x5604f0446490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7ea4ff9640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7e900a1ab0 0x7f7e900a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7ea77fe640 1 -- 172.21.15.83:0/3824684339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f044b660 msgr2=0x5604f044dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7ea77fe640 1 --2- 172.21.15.83:0/3824684339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f044b660 0x5604f044dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7ea77fe640 1 -- 172.21.15.83:0/3824684339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446070 msgr2=0x5604f0446490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7ea77fe640 1 --2- 172.21.15.83:0/3824684339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446070 0x5604f0446490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7ea77fe640 1 -- 172.21.15.83:0/3824684339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604f0368060 con 0x5604f0446a60 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7ea7fff640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7e900a1ab0 0x7f7e900a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7ea4ff9640 1 -- 172.21.15.83:0/3617379503 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7ea00f0240 con 0x7f7ea8102e90 2024-03-11T19:44:32.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7ea77fe640 1 --2- 172.21.15.83:0/3824684339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446a60 0x5604f044aff0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009d70 tx=0x7f7e98011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560cacb2dc0ae39b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7e8dffb640 1 -- 172.21.15.83:0/3824684339 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98009840 con 0x5604f0446a60 2024-03-11T19:44:32.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7e8dffb640 1 -- 172.21.15.83:0/3824684339 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e98004220 con 0x5604f0446a60 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.509+0000 7f7e8dffb640 1 -- 172.21.15.83:0/3824684339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e980197f0 con 0x5604f0446a60 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.510+0000 7f7eb09c5740 1 -- 172.21.15.83:0/3824684339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446a60 msgr2=0x5604f044aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.510+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/3824684339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446a60 0x5604f044aff0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009d70 tx=0x7f7e98011810 comp rx=0 tx=0).stop 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.511+0000 7f7ea7fff640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7e900a1ab0 0x7f7e900a3f70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00905f0 tx=0x7f7ea00e7920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.512+0000 7f7eb09c5740 1 -- 172.21.15.83:0/3824684339 shutdown_connections 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.512+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/3824684339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f044b660 0x5604f044dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.512+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/3824684339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446a60 0x5604f044aff0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.512+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/3824684339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446070 0x5604f0446490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.512+0000 7f7eb09c5740 1 -- 172.21.15.83:0/3824684339 >> 172.21.15.83:0/3824684339 conn(0x5604f0438760 msgr2=0x5604f0438b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.512+0000 7f7eb09c5740 1 -- 172.21.15.83:0/3824684339 shutdown_connections 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.512+0000 7f7eb09c5740 1 -- 172.21.15.83:0/3824684339 wait complete. 2024-03-11T19:44:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7eb09c5740 1 -- start start 2024-03-11T19:44:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7eb09c5740 1 Processor -- start 2024-03-11T19:44:32.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f044fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f0446a60 0x5604f04500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f044b660 0x5604f0457110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7eb09c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f03a6020 con 0x5604f0446070 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7eb09c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f0079430 con 0x5604f044b660 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7eb09c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f0035e00 con 0x5604f0446a60 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7eb09c5740 10 client.0 _ll_register_callbacks cb 0x5604f034dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f044fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f044fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40326/0 (socket says 172.21.15.83:40326) 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.513+0000 7f7ea6ffd640 1 -- 172.21.15.83:0/2142279697 learned_addr learned my addr 172.21.15.83:0/2142279697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7ea77fe640 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f0446a60 0x5604f04500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f044b660 0x5604f0457110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7ea6ffd640 1 -- 172.21.15.83:0/2142279697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f0446a60 msgr2=0x5604f04500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f0446a60 0x5604f04500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7ea6ffd640 1 -- 172.21.15.83:0/2142279697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f044b660 msgr2=0x5604f0457110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f044b660 0x5604f0457110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7ea6ffd640 1 -- 172.21.15.83:0/2142279697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e980099a0 con 0x5604f0446070 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f044fb90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c004610 tx=0x7f7e9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f2b60df552754c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7e86ffd640 1 -- 172.21.15.83:0/2142279697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c016070 con 0x5604f0446070 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7e86ffd640 1 -- 172.21.15.83:0/2142279697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7e9c011950 con 0x5604f0446070 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2142279697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604f00a57e0 con 0x5604f0446070 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7e86ffd640 1 -- 172.21.15.83:0/2142279697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e9c011b10 con 0x5604f0446070 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7eb09c5740 10 client.16347 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.514+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2142279697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5604f03a46a0 con 0x5604f0446070 2024-03-11T19:44:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.515+0000 7f7e86ffd640 1 -- 172.21.15.83:0/2142279697 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5604f00a57e0 con 0x5604f0446070 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.515+0000 7f7e86ffd640 1 -- 172.21.15.83:0/2142279697 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7e9c02b070 con 0x5604f0446070 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.515+0000 7f7eb09c5740 10 client.16347 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.515+0000 7f7eb09c5740 10 client.16347 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.515+0000 7f7eb09c5740 20 client.16347 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.515+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5604f045bd90 0x5604f045e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.515+0000 7f7eb09c5740 4 client.16347 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.515+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2142279697 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604f00e2720 con 0x5604f045bd90 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.515+0000 7f7ea77fe640 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5604f045bd90 0x5604f045e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.516+0000 7f7ea77fe640 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5604f045bd90 0x5604f045e1b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=700e70d847ed2345 server_cookie=4d134742250ab4bc in_seq=0 out_seq=0 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.516+0000 7f7e86ffd640 10 client.16347 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.516+0000 7f7e86ffd640 1 -- 172.21.15.83:0/2142279697 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604f00e2720 con 0x5604f045bd90 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.516+0000 7f7e86ffd640 10 client.16347 handle_command_reply: tid=0 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 1 client.16347 shutdown 2024-03-11T19:44:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2142279697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 msgr2=0x5604f044fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f044fb90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c004610 tx=0x7f7e9c004730 comp rx=0 tx=0).stop 2024-03-11T19:44:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2142279697 shutdown_connections 2024-03-11T19:44:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5604f045bd90 0x5604f045e1b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7e86ffd640 0 client.16347 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:44:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f044b660 0x5604f0457110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f0446a60 0x5604f04500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/2142279697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f044fb90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2142279697 >> 172.21.15.83:0/2142279697 conn(0x5604f0438760 msgr2=0x5604f0444e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2142279697 shutdown_connections 2024-03-11T19:44:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2142279697 wait complete. 2024-03-11T19:44:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.517+0000 7f7eb09c5740 20 client.16347 trim_cache size 0 max 16384 2024-03-11T19:44:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7eb09c5740 1 Processor -- start 2024-03-11T19:44:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7eb09c5740 1 -- start start 2024-03-11T19:44:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f0447900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446a60 0x5604f04570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f044b660 0x5604f0447e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7eb09c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f032b130 con 0x5604f0446070 2024-03-11T19:44:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7eb09c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f00e2720 con 0x5604f0446a60 2024-03-11T19:44:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f0447900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f0447900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40340/0 (socket says 172.21.15.83:40340) 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7ea6ffd640 1 -- 172.21.15.83:0/2517607855 learned_addr learned my addr 172.21.15.83:0/2517607855 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2517607855 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f042aa40 con 0x5604f044b660 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/2517607855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f044b660 0x5604f0447e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.536+0000 7f7ea77fe640 1 --2- 172.21.15.83:0/2517607855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446a60 0x5604f04570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.537+0000 7f7ea6ffd640 1 -- 172.21.15.83:0/2517607855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f044b660 msgr2=0x5604f0447e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.537+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/2517607855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f044b660 0x5604f0447e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.537+0000 7f7ea6ffd640 1 -- 172.21.15.83:0/2517607855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446a60 msgr2=0x5604f04570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.537+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/2517607855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446a60 0x5604f04570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.537+0000 7f7ea6ffd640 1 -- 172.21.15.83:0/2517607855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604f0340c20 con 0x5604f0446070 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.537+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/2517607855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f0447900 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c001eb0 tx=0x7f7e9c005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd882f7799ab425b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.537+0000 7f7e8d7fa640 1 -- 172.21.15.83:0/2517607855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c016070 con 0x5604f0446070 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.537+0000 7f7e8d7fa640 1 -- 172.21.15.83:0/2517607855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e9c028070 con 0x5604f0446070 2024-03-11T19:44:32.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.537+0000 7f7e8d7fa640 1 -- 172.21.15.83:0/2517607855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c005690 con 0x5604f0446070 2024-03-11T19:44:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.538+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2517607855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 msgr2=0x5604f0447900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.538+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/2517607855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f0447900 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c001eb0 tx=0x7f7e9c005d70 comp rx=0 tx=0).stop 2024-03-11T19:44:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.538+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2517607855 shutdown_connections 2024-03-11T19:44:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.538+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/2517607855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f044b660 0x5604f0447e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.538+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/2517607855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446a60 0x5604f04570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.538+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/2517607855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f0446070 0x5604f0447900 secure :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c001eb0 tx=0x7f7e9c005d70 comp rx=0 tx=0).stop 2024-03-11T19:44:32.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.538+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2517607855 >> 172.21.15.83:0/2517607855 conn(0x5604f0438760 msgr2=0x5604f0433a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.540+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2517607855 shutdown_connections 2024-03-11T19:44:32.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.540+0000 7f7eb09c5740 1 -- 172.21.15.83:0/2517607855 wait complete. 2024-03-11T19:44:32.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.540+0000 7f7eb09c5740 1 -- start start 2024-03-11T19:44:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.540+0000 7f7eb09c5740 1 Processor -- start 2024-03-11T19:44:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446a60 0x5604f04562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f044b660 0x5604f0456820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7eb09c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f04b03e0 0x5604f04b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7eb09c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f04b2d10 con 0x5604f044b660 2024-03-11T19:44:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7eb09c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f04b2e50 con 0x5604f0446a60 2024-03-11T19:44:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7eb09c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604f04b2fc0 con 0x5604f04b03e0 2024-03-11T19:44:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7eb09c5740 10 client.0 _ll_register_callbacks cb 0x7f7e9c0035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:32.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f044b660 0x5604f0456820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f04b03e0 0x5604f04b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f04b03e0 0x5604f04b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:49362/0 (socket says 172.21.15.83:49362) 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7ea6ffd640 1 -- 172.21.15.83:0/1942161808 learned_addr learned my addr 172.21.15.83:0/1942161808 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446a60 0x5604f04562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7ea77fe640 1 -- 172.21.15.83:0/1942161808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f04b03e0 msgr2=0x5604f04b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7ea77fe640 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f04b03e0 0x5604f04b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7ea77fe640 1 -- 172.21.15.83:0/1942161808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446a60 msgr2=0x5604f04562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7ea77fe640 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446a60 0x5604f04562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7ea77fe640 1 -- 172.21.15.83:0/1942161808 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604f03a46a0 con 0x5604f044b660 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.541+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446a60 0x5604f04562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.542+0000 7f7ea77fe640 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f044b660 0x5604f0456820 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5604f04396d0 tx=0x7f7e98005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4605946d18c9e45 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.542+0000 7f7e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:44:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.542+0000 7f7e86ffd640 1 -- 172.21.15.83:0/1942161808 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98029040 con 0x5604f044b660 2024-03-11T19:44:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.542+0000 7f7e86ffd640 1 -- 172.21.15.83:0/1942161808 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7e9802bb70 con 0x5604f044b660 2024-03-11T19:44:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.542+0000 7f7eb09c5740 1 -- 172.21.15.83:0/1942161808 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e98004220 con 0x5604f044b660 2024-03-11T19:44:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.542+0000 7f7e86ffd640 1 -- 172.21.15.83:0/1942161808 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e9802bd50 con 0x5604f044b660 2024-03-11T19:44:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7eb09c5740 10 client.16353 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7eb09c5740 1 -- 172.21.15.83:0/1942161808 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5604f03668c0 con 0x5604f044b660 2024-03-11T19:44:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7e86ffd640 1 -- 172.21.15.83:0/1942161808 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e9803b020 con 0x5604f044b660 2024-03-11T19:44:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7e86ffd640 1 -- 172.21.15.83:0/1942161808 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7e98005410 con 0x5604f044b660 2024-03-11T19:44:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7eb09c5740 10 client.16353 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7eb09c5740 10 client.16353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:44:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7eb09c5740 20 client.16353 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5604f04b49e0 0x5604f04b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7eb09c5740 4 client.16353 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7eb09c5740 1 -- 172.21.15.83:0/1942161808 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5604f0131890 con 0x5604f04b49e0 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5604f04b49e0 0x5604f04b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7e86ffd640 10 client.16353 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.543+0000 7f7ea6ffd640 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5604f04b49e0 0x5604f04b6e00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d27c56887855254 server_cookie=351227baa194a525 in_seq=0 out_seq=0 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7e86ffd640 1 -- 172.21.15.83:0/1942161808 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5604f0131890 con 0x5604f04b49e0 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7e86ffd640 10 client.16353 handle_command_reply: tid=0 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7eb09c5740 1 client.16353 shutdown 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7eb09c5740 1 -- 172.21.15.83:0/1942161808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f044b660 msgr2=0x5604f0456820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f044b660 0x5604f0456820 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5604f04396d0 tx=0x7f7e98005d30 comp rx=0 tx=0).stop 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7eb09c5740 1 -- 172.21.15.83:0/1942161808 shutdown_connections 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5604f04b49e0 0x5604f04b6e00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5604f04b03e0 0x5604f04b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7e86ffd640 0 client.16353 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:44:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604f044b660 0x5604f0456820 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7eb09c5740 1 --2- 172.21.15.83:0/1942161808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604f0446a60 0x5604f04562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7eb09c5740 1 -- 172.21.15.83:0/1942161808 >> 172.21.15.83:0/1942161808 conn(0x5604f0438760 msgr2=0x5604f0447900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7eb09c5740 1 -- 172.21.15.83:0/1942161808 shutdown_connections 2024-03-11T19:44:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.544+0000 7f7eb09c5740 1 -- 172.21.15.83:0/1942161808 wait complete. 2024-03-11T19:44:32.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eb09c5740 20 client.16353 trim_cache size 0 max 16384 2024-03-11T19:44:32.552 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:44:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3617379503 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7e900a1ab0 msgr2=0x7f7e900a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eaddf6640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7e900a1ab0 0x7f7e900a3f70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00905f0 tx=0x7f7ea00e7920 comp rx=0 tx=0).stop 2024-03-11T19:44:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3617379503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ea8102e90 msgr2=0x7f7ea819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eaddf6640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ea8102e90 0x7f7ea819e5e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009bf0 tx=0x7f7ea0034cc0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3617379503 shutdown_connections 2024-03-11T19:44:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eaddf6640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7e900a1ab0 0x7f7e900a3f70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eaddf6640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ea8102e90 0x7f7ea819e5e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eaddf6640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ea81024d0 0x7f7ea8197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eaddf6640 1 --2- 172.21.15.83:0/3617379503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ea80ff460 0x7f7ea8197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3617379503 >> 172.21.15.83:0/3617379503 conn(0x7f7ea80fad70 msgr2=0x7f7ea8105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.545+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3617379503 shutdown_connections 2024-03-11T19:44:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.546+0000 7f7eaddf6640 1 -- 172.21.15.83:0/3617379503 wait complete. 2024-03-11T19:44:32.566 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:44:32.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:44:32.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:32 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3580789116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:44:32.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:32 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3580789116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:44:32.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.753+0000 7f1d8f54b640 1 -- 172.21.15.83:0/226060707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d881077a0 msgr2=0x7f1d88107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.753+0000 7f1d8f54b640 1 --2- 172.21.15.83:0/226060707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d881077a0 0x7f1d88107bc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009900 tx=0x7f1d7802bc30 comp rx=0 tx=0).stop 2024-03-11T19:44:32.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.753+0000 7f1d8f54b640 1 -- 172.21.15.83:0/226060707 shutdown_connections 2024-03-11T19:44:32.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.753+0000 7f1d8f54b640 1 --2- 172.21.15.83:0/226060707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d88109350 0x7f1d8810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.753+0000 7f1d8f54b640 1 --2- 172.21.15.83:0/226060707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d88108990 0x7f1d88108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.753+0000 7f1d8f54b640 1 --2- 172.21.15.83:0/226060707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d881077a0 0x7f1d88107bc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.753+0000 7f1d8f54b640 1 -- 172.21.15.83:0/226060707 >> 172.21.15.83:0/226060707 conn(0x7f1d88076740 msgr2=0x7f1d88078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.753+0000 7f1d8f54b640 1 -- 172.21.15.83:0/226060707 shutdown_connections 2024-03-11T19:44:32.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.754+0000 7f1d8f54b640 1 -- 172.21.15.83:0/226060707 wait complete. 2024-03-11T19:44:32.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.754+0000 7f1d8f54b640 1 Processor -- start 2024-03-11T19:44:32.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8f54b640 1 -- start start 2024-03-11T19:44:32.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8f54b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d881077a0 0x7f1d8819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8f54b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d88108990 0x7f1d8819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8f54b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d88109350 0x7f1d881a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8f54b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881a5340 con 0x7f1d88108990 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8f54b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881a54b0 con 0x7f1d881077a0 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8f54b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881a5670 con 0x7f1d88109350 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8cae6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d88108990 0x7f1d8819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8cae6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d88108990 0x7f1d8819dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40380/0 (socket says 172.21.15.83:40380) 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8cae6640 1 -- 172.21.15.83:0/1701491640 learned_addr learned my addr 172.21.15.83:0/1701491640 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d88109350 0x7f1d881a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.755+0000 7f1d8d2e7640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d881077a0 0x7f1d8819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.756+0000 7f1d8dae8640 1 -- 172.21.15.83:0/1701491640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d881077a0 msgr2=0x7f1d8819d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.756+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d881077a0 0x7f1d8819d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.756+0000 7f1d8dae8640 1 -- 172.21.15.83:0/1701491640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d88108990 msgr2=0x7f1d8819dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.756+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d88108990 0x7f1d8819dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.756+0000 7f1d8dae8640 1 -- 172.21.15.83:0/1701491640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d78009560 con 0x7f1d88109350 2024-03-11T19:44:32.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.756+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d88109350 0x7f1d881a4e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009fd0 tx=0x7f1d8000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.756+0000 7f1d867fc640 1 -- 172.21.15.83:0/1701491640 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000ce20 con 0x7f1d88109350 2024-03-11T19:44:32.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.756+0000 7f1d867fc640 1 -- 172.21.15.83:0/1701491640 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d80005020 con 0x7f1d88109350 2024-03-11T19:44:32.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.756+0000 7f1d8f54b640 1 -- 172.21.15.83:0/1701491640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d881a5950 con 0x7f1d88109350 2024-03-11T19:44:32.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.756+0000 7f1d867fc640 1 -- 172.21.15.83:0/1701491640 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80010690 con 0x7f1d88109350 2024-03-11T19:44:32.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.756+0000 7f1d8f54b640 1 -- 172.21.15.83:0/1701491640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d881a5e60 con 0x7f1d88109350 2024-03-11T19:44:32.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.758+0000 7f1d867fc640 1 -- 172.21.15.83:0/1701491640 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1d80005500 con 0x7f1d88109350 2024-03-11T19:44:32.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.760+0000 7f1d867fc640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d700a1b30 0x7f1d700a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.760+0000 7f1d921f4740 1 Processor -- start 2024-03-11T19:44:32.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.760+0000 7f1d921f4740 1 -- start start 2024-03-11T19:44:32.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.760+0000 7f1d867fc640 1 -- 172.21.15.83:0/1701491640 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1d80014070 con 0x7f1d88109350 2024-03-11T19:44:32.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.760+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f4280 0x56029e2f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.760+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4c70 0x56029e2f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.760+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f9de0 0x56029e2fc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.760+0000 7f1d921f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029e29f960 con 0x56029e2f9de0 2024-03-11T19:44:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.760+0000 7f1d921f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029e253310 con 0x56029e2f4c70 2024-03-11T19:44:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.760+0000 7f1d921f4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029e250840 con 0x56029e2f4280 2024-03-11T19:44:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.760+0000 7f1d8dae8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f9de0 0x56029e2fc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d8dae8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f9de0 0x56029e2fc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40386/0 (socket says 172.21.15.83:40386) 2024-03-11T19:44:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d8dae8640 1 -- 172.21.15.83:0/1232220669 learned_addr learned my addr 172.21.15.83:0/1232220669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d8cae6640 1 --2- 172.21.15.83:0/1232220669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4c70 0x56029e2f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d8dae8640 1 -- 172.21.15.83:0/1232220669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f4280 msgr2=0x56029e2f46a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:44:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/1232220669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f4280 0x56029e2f46a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d8dae8640 1 -- 172.21.15.83:0/1232220669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4c70 msgr2=0x56029e2f9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/1232220669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4c70 0x56029e2f9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d8d2e7640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d700a1b30 0x7f1d700a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d8dae8640 1 -- 172.21.15.83:0/1232220669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56029e20e060 con 0x56029e2f9de0 2024-03-11T19:44:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/1232220669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f9de0 0x56029e2fc270 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1d800974d0 tx=0x7f1d8009a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7a013273e39d2c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d6f7fe640 1 -- 172.21.15.83:0/1232220669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800b2070 con 0x56029e2f9de0 2024-03-11T19:44:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d6f7fe640 1 -- 172.21.15.83:0/1232220669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d8009ab00 con 0x56029e2f9de0 2024-03-11T19:44:32.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.761+0000 7f1d6f7fe640 1 -- 172.21.15.83:0/1232220669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800a2e10 con 0x56029e2f9de0 2024-03-11T19:44:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.762+0000 7f1d921f4740 1 -- 172.21.15.83:0/1232220669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f9de0 msgr2=0x56029e2fc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.762+0000 7f1d8d2e7640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d700a1b30 0x7f1d700a3ff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1d78006010 tx=0x7f1d78000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.762+0000 7f1d921f4740 1 --2- 172.21.15.83:0/1232220669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f9de0 0x56029e2fc270 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1d800974d0 tx=0x7f1d8009a460 comp rx=0 tx=0).stop 2024-03-11T19:44:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.762+0000 7f1d921f4740 1 -- 172.21.15.83:0/1232220669 shutdown_connections 2024-03-11T19:44:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.762+0000 7f1d921f4740 1 --2- 172.21.15.83:0/1232220669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f9de0 0x56029e2fc270 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.762+0000 7f1d921f4740 1 --2- 172.21.15.83:0/1232220669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4c70 0x56029e2f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.762+0000 7f1d921f4740 1 --2- 172.21.15.83:0/1232220669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f4280 0x56029e2f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.762+0000 7f1d921f4740 1 -- 172.21.15.83:0/1232220669 >> 172.21.15.83:0/1232220669 conn(0x56029e2de7e0 msgr2=0x56029e2debe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.762+0000 7f1d921f4740 1 -- 172.21.15.83:0/1232220669 shutdown_connections 2024-03-11T19:44:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.762+0000 7f1d921f4740 1 -- 172.21.15.83:0/1232220669 wait complete. 2024-03-11T19:44:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.763+0000 7f1d921f4740 1 -- start start 2024-03-11T19:44:32.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.763+0000 7f1d921f4740 1 Processor -- start 2024-03-11T19:44:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.763+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4280 0x56029e2fe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.763+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e2fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f9de0 0x56029e3059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d921f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029e24c020 con 0x56029e2f4c70 2024-03-11T19:44:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d921f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029df1f430 con 0x56029e2f4280 2024-03-11T19:44:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d921f4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029dedbe00 con 0x56029e2f9de0 2024-03-11T19:44:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d921f4740 10 client.0 _ll_register_callbacks cb 0x56029e1f3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d8cae6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e2fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d8cae6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e2fe9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40400/0 (socket says 172.21.15.83:40400) 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d8cae6640 1 -- 172.21.15.83:0/3945137449 learned_addr learned my addr 172.21.15.83:0/3945137449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f9de0 0x56029e3059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d8cae6640 1 -- 172.21.15.83:0/3945137449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f9de0 msgr2=0x56029e3059f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d8cae6640 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f9de0 0x56029e3059f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d8cae6640 1 -- 172.21.15.83:0/3945137449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4280 msgr2=0x56029e2fe470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d8cae6640 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4280 0x56029e2fe470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.764+0000 7f1d8cae6640 1 -- 172.21.15.83:0/3945137449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d881a5e60 con 0x56029e2f4c70 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.765+0000 7f1d8cae6640 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e2fe9b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0126b0 tx=0x7f1d7c012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63eb6aec5a1b44 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.765+0000 7f1d6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d6cff9640 1 -- 172.21.15.83:0/3945137449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c017070 con 0x56029e2f4c70 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d921f4740 1 -- 172.21.15.83:0/3945137449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56029df4b7e0 con 0x56029e2f4c70 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d6cff9640 1 -- 172.21.15.83:0/3945137449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1d7c004050 con 0x56029e2f4c70 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d6cff9640 1 -- 172.21.15.83:0/3945137449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d7c004210 con 0x56029e2f4c70 2024-03-11T19:44:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d921f4740 10 client.16365 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d921f4740 1 -- 172.21.15.83:0/3945137449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56029e24a6a0 con 0x56029e2f4c70 2024-03-11T19:44:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d6cff9640 1 -- 172.21.15.83:0/3945137449 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56029df4b7e0 con 0x56029e2f4c70 2024-03-11T19:44:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d6cff9640 1 -- 172.21.15.83:0/3945137449 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1d7c025070 con 0x56029e2f4c70 2024-03-11T19:44:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d921f4740 10 client.16365 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d921f4740 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d921f4740 20 client.16365 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d921f4740 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56029e30a6c0 0x56029e30cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d921f4740 4 client.16365 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:44:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.766+0000 7f1d921f4740 1 -- 172.21.15.83:0/3945137449 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56029df88720 con 0x56029e30a6c0 2024-03-11T19:44:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.767+0000 7f1d8d2e7640 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56029e30a6c0 0x56029e30cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.767+0000 7f1d6cff9640 10 client.16365 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.767+0000 7f1d8d2e7640 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56029e30a6c0 0x56029e30cae0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf939375e55c524 server_cookie=df79072187a5eb8c in_seq=0 out_seq=0 2024-03-11T19:44:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.768+0000 7f1d6cff9640 1 -- 172.21.15.83:0/3945137449 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56029df88720 con 0x56029e30a6c0 2024-03-11T19:44:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.768+0000 7f1d6cff9640 10 client.16365 handle_command_reply: tid=0 2024-03-11T19:44:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.768+0000 7f1d921f4740 1 client.16365 shutdown 2024-03-11T19:44:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d921f4740 1 -- 172.21.15.83:0/3945137449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 msgr2=0x56029e2fe9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d921f4740 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e2fe9b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0126b0 tx=0x7f1d7c012b70 comp rx=0 tx=0).stop 2024-03-11T19:44:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d921f4740 1 -- 172.21.15.83:0/3945137449 shutdown_connections 2024-03-11T19:44:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d921f4740 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56029e30a6c0 0x56029e30cae0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d921f4740 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f9de0 0x56029e3059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d921f4740 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e2fe9b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d921f4740 1 --2- 172.21.15.83:0/3945137449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4280 0x56029e2fe470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d921f4740 1 -- 172.21.15.83:0/3945137449 >> 172.21.15.83:0/3945137449 conn(0x56029e2de7e0 msgr2=0x56029e2ddb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d6cff9640 0 client.16365 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d921f4740 1 -- 172.21.15.83:0/3945137449 shutdown_connections 2024-03-11T19:44:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d921f4740 1 -- 172.21.15.83:0/3945137449 wait complete. 2024-03-11T19:44:32.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.769+0000 7f1d921f4740 20 client.16365 trim_cache size 0 max 16384 2024-03-11T19:44:32.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d921f4740 1 Processor -- start 2024-03-11T19:44:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d921f4740 1 -- start start 2024-03-11T19:44:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4280 0x56029e311480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e3059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f9de0 0x56029e2fe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d921f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029e1d1130 con 0x56029e2f4c70 2024-03-11T19:44:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d921f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029e306ee0 con 0x56029e2f4280 2024-03-11T19:44:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d921f4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029e2f5d00 con 0x56029e2f9de0 2024-03-11T19:44:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d8d2e7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4280 0x56029e311480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d8d2e7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4280 0x56029e311480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:38408/0 (socket says 172.21.15.83:38408) 2024-03-11T19:44:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d8d2e7640 1 -- 172.21.15.83:0/4222165907 learned_addr learned my addr 172.21.15.83:0/4222165907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/4222165907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f9de0 0x56029e2fe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.786+0000 7f1d8cae6640 1 --2- 172.21.15.83:0/4222165907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e3059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.787+0000 7f1d8dae8640 1 -- 172.21.15.83:0/4222165907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4280 msgr2=0x56029e311480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.787+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/4222165907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4280 0x56029e311480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.787+0000 7f1d8dae8640 1 -- 172.21.15.83:0/4222165907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 msgr2=0x56029e3059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.787+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/4222165907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e3059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.787+0000 7f1d8dae8640 1 -- 172.21.15.83:0/4222165907 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56029e1e6c20 con 0x56029e2f9de0 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.787+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/4222165907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f9de0 0x56029e2fe470 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d80096f60 tx=0x7f1d8009b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e180d5241c6daef server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.787+0000 7f1d6effd640 1 -- 172.21.15.83:0/4222165907 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800b2030 con 0x56029e2f9de0 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.787+0000 7f1d6effd640 1 -- 172.21.15.83:0/4222165907 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d8009ec50 con 0x56029e2f9de0 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.787+0000 7f1d6effd640 1 -- 172.21.15.83:0/4222165907 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800aedb0 con 0x56029e2f9de0 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.788+0000 7f1d921f4740 1 -- 172.21.15.83:0/4222165907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f9de0 msgr2=0x56029e2fe470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.788+0000 7f1d921f4740 1 --2- 172.21.15.83:0/4222165907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f9de0 0x56029e2fe470 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d80096f60 tx=0x7f1d8009b370 comp rx=0 tx=0).stop 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.788+0000 7f1d921f4740 1 -- 172.21.15.83:0/4222165907 shutdown_connections 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.788+0000 7f1d921f4740 1 --2- 172.21.15.83:0/4222165907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f9de0 0x56029e2fe470 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.788+0000 7f1d921f4740 1 --2- 172.21.15.83:0/4222165907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e3059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.788+0000 7f1d921f4740 1 --2- 172.21.15.83:0/4222165907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f4280 0x56029e311480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.788+0000 7f1d921f4740 1 -- 172.21.15.83:0/4222165907 >> 172.21.15.83:0/4222165907 conn(0x56029e2de7e0 msgr2=0x56029e2debc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.788+0000 7f1d921f4740 1 -- 172.21.15.83:0/4222165907 shutdown_connections 2024-03-11T19:44:32.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.788+0000 7f1d921f4740 1 -- 172.21.15.83:0/4222165907 wait complete. 2024-03-11T19:44:32.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.788+0000 7f1d921f4740 1 -- start start 2024-03-11T19:44:32.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.788+0000 7f1d921f4740 1 Processor -- start 2024-03-11T19:44:32.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f4280 0x56029e303760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e303ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d921f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f9de0 0x56029e304cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d921f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029e3051f0 con 0x56029e2f4c70 2024-03-11T19:44:32.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d921f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029e305360 con 0x56029e2f9de0 2024-03-11T19:44:32.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d921f4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56029e305520 con 0x56029e2f4280 2024-03-11T19:44:32.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d921f4740 10 client.0 _ll_register_callbacks cb 0x7f1d80005800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:32.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d8d2e7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f4280 0x56029e303760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d8cae6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e303ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d8d2e7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f4280 0x56029e303760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:49404/0 (socket says 172.21.15.83:49404) 2024-03-11T19:44:32.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d8d2e7640 1 -- 172.21.15.83:0/2690375538 learned_addr learned my addr 172.21.15.83:0/2690375538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:32.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f9de0 0x56029e304cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.789+0000 7f1d8d2e7640 1 -- 172.21.15.83:0/2690375538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f9de0 msgr2=0x56029e304cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.790+0000 7f1d8d2e7640 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f9de0 0x56029e304cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.790+0000 7f1d8d2e7640 1 -- 172.21.15.83:0/2690375538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 msgr2=0x56029e303ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.790+0000 7f1d8d2e7640 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e303ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.790+0000 7f1d8d2e7640 1 -- 172.21.15.83:0/2690375538 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56029e20e060 con 0x56029e2f4280 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.790+0000 7f1d8dae8640 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f9de0 0x56029e304cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.790+0000 7f1d8d2e7640 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f4280 0x56029e303760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1d78036040 tx=0x7f1d78009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed6175657d66090 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.791+0000 7f1d6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.791+0000 7f1d6cff9640 1 -- 172.21.15.83:0/2690375538 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780043c0 con 0x56029e2f4280 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.791+0000 7f1d6cff9640 1 -- 172.21.15.83:0/2690375538 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1d78046040 con 0x56029e2f4280 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.791+0000 7f1d921f4740 1 -- 172.21.15.83:0/2690375538 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d8009ab00 con 0x56029e2f4280 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.791+0000 7f1d6cff9640 1 -- 172.21.15.83:0/2690375538 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d78004520 con 0x56029e2f4280 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.791+0000 7f1d921f4740 10 client.25948 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.791+0000 7f1d921f4740 1 -- 172.21.15.83:0/2690375538 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56029e20c8c0 con 0x56029e2f4280 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.792+0000 7f1d6cff9640 1 -- 172.21.15.83:0/2690375538 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d78051070 con 0x56029e2f4280 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.792+0000 7f1d6cff9640 1 -- 172.21.15.83:0/2690375538 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1d7803c040 con 0x56029e2f4280 2024-03-11T19:44:32.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.792+0000 7f1d921f4740 10 client.25948 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.792+0000 7f1d921f4740 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.792+0000 7f1d921f4740 20 client.25948 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.792+0000 7f1d921f4740 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56029e361c80 0x56029e3640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.792+0000 7f1d921f4740 4 client.25948 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.792+0000 7f1d921f4740 1 -- 172.21.15.83:0/2690375538 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f1d8009ab00 con 0x56029e361c80 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d8cae6640 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56029e361c80 0x56029e3640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d8cae6640 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56029e361c80 0x56029e3640a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f269fb2e4e37668 server_cookie=6421b63831ff0324 in_seq=0 out_seq=0 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d6cff9640 10 client.25948 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d6cff9640 1 -- 172.21.15.83:0/2690375538 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f1d8009ab00 con 0x56029e361c80 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d6cff9640 10 client.25948 handle_command_reply: tid=0 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d921f4740 1 client.25948 shutdown 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d921f4740 1 -- 172.21.15.83:0/2690375538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f4280 msgr2=0x56029e303760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d921f4740 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f4280 0x56029e303760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1d78036040 tx=0x7f1d78009260 comp rx=0 tx=0).stop 2024-03-11T19:44:32.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d921f4740 1 -- 172.21.15.83:0/2690375538 shutdown_connections 2024-03-11T19:44:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d921f4740 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56029e361c80 0x56029e3640a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d921f4740 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56029e2f9de0 0x56029e304cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d921f4740 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56029e2f4c70 0x56029e303ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d921f4740 1 --2- 172.21.15.83:0/2690375538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56029e2f4280 0x56029e303760 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d6cff9640 0 client.25948 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.793+0000 7f1d921f4740 1 -- 172.21.15.83:0/2690375538 >> 172.21.15.83:0/2690375538 conn(0x56029e2de7e0 msgr2=0x56029e311480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.794+0000 7f1d921f4740 1 -- 172.21.15.83:0/2690375538 shutdown_connections 2024-03-11T19:44:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.794+0000 7f1d921f4740 1 -- 172.21.15.83:0/2690375538 wait complete. 2024-03-11T19:44:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.794+0000 7f1d921f4740 20 client.25948 trim_cache size 0 max 16384 2024-03-11T19:44:32.801 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:44:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.795+0000 7f1d8f54b640 1 -- 172.21.15.83:0/1701491640 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d700a1b30 msgr2=0x7f1d700a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.795+0000 7f1d8f54b640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d700a1b30 0x7f1d700a3ff0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1d78006010 tx=0x7f1d78000bc0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.795+0000 7f1d8f54b640 1 -- 172.21.15.83:0/1701491640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d88109350 msgr2=0x7f1d881a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.795+0000 7f1d8f54b640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d88109350 0x7f1d881a4e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009fd0 tx=0x7f1d8000ef10 comp rx=0 tx=0).stop 2024-03-11T19:44:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.795+0000 7f1d8f54b640 1 -- 172.21.15.83:0/1701491640 shutdown_connections 2024-03-11T19:44:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.795+0000 7f1d8f54b640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1d700a1b30 0x7f1d700a3ff0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.795+0000 7f1d8f54b640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d88109350 0x7f1d881a4e00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.795+0000 7f1d8f54b640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d88108990 0x7f1d8819dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.795+0000 7f1d8f54b640 1 --2- 172.21.15.83:0/1701491640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d881077a0 0x7f1d8819d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.796+0000 7f1d8f54b640 1 -- 172.21.15.83:0/1701491640 >> 172.21.15.83:0/1701491640 conn(0x7f1d88076740 msgr2=0x7f1d8810bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:32.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.796+0000 7f1d8f54b640 1 -- 172.21.15.83:0/1701491640 shutdown_connections 2024-03-11T19:44:32.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:32.796+0000 7f1d8f54b640 1 -- 172.21.15.83:0/1701491640 wait complete. 2024-03-11T19:44:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:33 smithi083 ceph-mon[31864]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 810 op/s 2024-03-11T19:44:33.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:33 smithi137 ceph-mon[36220]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 810 op/s 2024-03-11T19:44:33.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:33 smithi204 ceph-mon[36739]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 810 op/s 2024-03-11T19:44:33.815 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:44:33.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:44:33.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:44:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.962+0000 7f7a515e2640 1 -- 172.21.15.83:0/4229916908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a4c1003d0 msgr2=0x7f7a4c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.962+0000 7f7a515e2640 1 --2- 172.21.15.83:0/4229916908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a4c1003d0 0x7f7a4c100830 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009900 tx=0x7f7a4002bcd0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.962+0000 7f7a515e2640 1 -- 172.21.15.83:0/4229916908 shutdown_connections 2024-03-11T19:44:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.962+0000 7f7a515e2640 1 --2- 172.21.15.83:0/4229916908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c100d70 0x7f7a4c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.962+0000 7f7a515e2640 1 --2- 172.21.15.83:0/4229916908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a4c1003d0 0x7f7a4c100830 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.962+0000 7f7a515e2640 1 --2- 172.21.15.83:0/4229916908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a4c0ffa70 0x7f7a4c0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.962+0000 7f7a515e2640 1 -- 172.21.15.83:0/4229916908 >> 172.21.15.83:0/4229916908 conn(0x7f7a4c0fb610 msgr2=0x7f7a4c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.962+0000 7f7a515e2640 1 -- 172.21.15.83:0/4229916908 shutdown_connections 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.962+0000 7f7a515e2640 1 -- 172.21.15.83:0/4229916908 wait complete. 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a515e2640 1 Processor -- start 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a515e2640 1 -- start start 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a515e2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a4c0ffa70 0x7f7a4c1055f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a515e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a4c1003d0 0x7f7a4c105b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a515e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c100d70 0x7f7a4c102160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a515e2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c1028f0 con 0x7f7a4c1003d0 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a515e2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c102a60 con 0x7f7a4c100d70 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a515e2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c102c20 con 0x7f7a4c0ffa70 2024-03-11T19:44:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a4a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a4c1003d0 0x7f7a4c105b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a4b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c100d70 0x7f7a4c102160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a4affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a4c0ffa70 0x7f7a4c1055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a4a7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a4c1003d0 0x7f7a4c105b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40432/0 (socket says 172.21.15.83:40432) 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a4a7fc640 1 -- 172.21.15.83:0/3899615947 learned_addr learned my addr 172.21.15.83:0/3899615947 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a4a7fc640 1 -- 172.21.15.83:0/3899615947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a4c0ffa70 msgr2=0x7f7a4c1055f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a4a7fc640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a4c0ffa70 0x7f7a4c1055f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a4a7fc640 1 -- 172.21.15.83:0/3899615947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c100d70 msgr2=0x7f7a4c102160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a4a7fc640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c100d70 0x7f7a4c102160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.963+0000 7f7a4a7fc640 1 -- 172.21.15.83:0/3899615947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a40009560 con 0x7f7a4c1003d0 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.964+0000 7f7a4a7fc640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a4c1003d0 0x7f7a4c105b30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7a40000c00 tx=0x7f7a4002dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.964+0000 7f7a33fff640 1 -- 172.21.15.83:0/3899615947 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a400044f0 con 0x7f7a4c1003d0 2024-03-11T19:44:33.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.964+0000 7f7a515e2640 1 -- 172.21.15.83:0/3899615947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c1a5930 con 0x7f7a4c1003d0 2024-03-11T19:44:33.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.964+0000 7f7a33fff640 1 -- 172.21.15.83:0/3899615947 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a400052d0 con 0x7f7a4c1003d0 2024-03-11T19:44:33.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.964+0000 7f7a33fff640 1 -- 172.21.15.83:0/3899615947 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40003d50 con 0x7f7a4c1003d0 2024-03-11T19:44:33.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.964+0000 7f7a515e2640 1 -- 172.21.15.83:0/3899615947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c1a5d80 con 0x7f7a4c1003d0 2024-03-11T19:44:33.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a541bb740 1 Processor -- start 2024-03-11T19:44:33.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a541bb740 1 -- start start 2024-03-11T19:44:33.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a33fff640 1 -- 172.21.15.83:0/3899615947 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7a40005440 con 0x7f7a4c1003d0 2024-03-11T19:44:33.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 0x55ff478668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 0x55ff47867b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff4786e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a541bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff477d5f90 con 0x55ff47868050 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a541bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff477d34c0 con 0x55ff47867690 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a541bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff477ceca0 con 0x55ff478664c0 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a4affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 0x55ff47867b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a4affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 0x55ff47867b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:38432/0 (socket says 172.21.15.83:38432) 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a4affd640 1 -- 172.21.15.83:0/768838465 learned_addr learned my addr 172.21.15.83:0/768838465 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a4a7fc640 1 --2- 172.21.15.83:0/768838465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff4786e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a4b7fe640 1 --2- 172.21.15.83:0/768838465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 0x55ff478668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a4b7fe640 1 -- 172.21.15.83:0/768838465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 msgr2=0x55ff47867b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:33.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a4b7fe640 1 --2- 172.21.15.83:0/768838465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 0x55ff47867b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.968+0000 7f7a4b7fe640 1 -- 172.21.15.83:0/768838465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 msgr2=0x55ff4786e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.969+0000 7f7a4b7fe640 1 --2- 172.21.15.83:0/768838465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff4786e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.969+0000 7f7a4b7fe640 1 -- 172.21.15.83:0/768838465 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff477910b0 con 0x55ff478664c0 2024-03-11T19:44:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.969+0000 7f7a4affd640 1 --2- 172.21.15.83:0/768838465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 0x55ff47867b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.969+0000 7f7a4b7fe640 1 --2- 172.21.15.83:0/768838465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 0x55ff478668e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009f50 tx=0x7f7a44003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d17f73d9472330c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.969+0000 7f7a33fff640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a340a1a80 0x7f7a340a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.969+0000 7f7a33fff640 1 -- 172.21.15.83:0/3899615947 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7a400efc10 con 0x7f7a4c1003d0 2024-03-11T19:44:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.969+0000 7f7a4affd640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a340a1a80 0x7f7a340a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.969+0000 7f7a317fa640 1 -- 172.21.15.83:0/768838465 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44011440 con 0x55ff478664c0 2024-03-11T19:44:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.969+0000 7f7a317fa640 1 -- 172.21.15.83:0/768838465 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a44011a80 con 0x55ff478664c0 2024-03-11T19:44:33.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.969+0000 7f7a317fa640 1 -- 172.21.15.83:0/768838465 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440109f0 con 0x55ff478664c0 2024-03-11T19:44:33.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.970+0000 7f7a541bb740 1 -- 172.21.15.83:0/768838465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 msgr2=0x55ff478668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:33.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.970+0000 7f7a541bb740 1 --2- 172.21.15.83:0/768838465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 0x55ff478668e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009f50 tx=0x7f7a44003e00 comp rx=0 tx=0).stop 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.970+0000 7f7a4affd640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a340a1a80 0x7f7a340a3f40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00a5c0 tx=0x7f7a3c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.970+0000 7f7a541bb740 1 -- 172.21.15.83:0/768838465 shutdown_connections 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.970+0000 7f7a541bb740 1 --2- 172.21.15.83:0/768838465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff4786e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.970+0000 7f7a541bb740 1 --2- 172.21.15.83:0/768838465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 0x55ff47867b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.970+0000 7f7a541bb740 1 --2- 172.21.15.83:0/768838465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 0x55ff478668e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.970+0000 7f7a541bb740 1 -- 172.21.15.83:0/768838465 >> 172.21.15.83:0/768838465 conn(0x55ff478616a0 msgr2=0x55ff47861aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.970+0000 7f7a541bb740 1 -- 172.21.15.83:0/768838465 shutdown_connections 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.970+0000 7f7a541bb740 1 -- 172.21.15.83:0/768838465 wait complete. 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.971+0000 7f7a541bb740 1 -- start start 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.971+0000 7f7a541bb740 1 Processor -- start 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.971+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 0x55ff478706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:33.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.971+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 0x55ff47870c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.971+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff47877c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.971+0000 7f7a541bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff47773f40 con 0x55ff47868050 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.971+0000 7f7a4b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff47877c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.971+0000 7f7a4b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff47877c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40444/0 (socket says 172.21.15.83:40444) 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.971+0000 7f7a4b7fe640 1 -- 172.21.15.83:0/1135219660 learned_addr learned my addr 172.21.15.83:0/1135219660 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.971+0000 7f7a541bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff474a24b0 con 0x55ff478664c0 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.972+0000 7f7a541bb740 1 -- 172.21.15.83:0/1135219660 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff47754670 con 0x55ff47867690 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.972+0000 7f7a541bb740 10 client.0 _ll_register_callbacks cb 0x55ff477c1f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.972+0000 7f7a4affd640 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 0x55ff478706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.972+0000 7f7a4a7fc640 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 0x55ff47870c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.972+0000 7f7a4affd640 1 -- 172.21.15.83:0/1135219660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 msgr2=0x55ff47870c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.972+0000 7f7a4affd640 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 0x55ff47870c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.972+0000 7f7a4affd640 1 -- 172.21.15.83:0/1135219660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 msgr2=0x55ff47877c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:33.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.972+0000 7f7a4affd640 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff47877c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.972+0000 7f7a4affd640 1 -- 172.21.15.83:0/1135219660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a44009bb0 con 0x55ff478664c0 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.973+0000 7f7a541bb740 1 -- 172.21.15.83:0/1135219660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff477a8f40 con 0x55ff478664c0 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.973+0000 7f7a4a7fc640 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 0x55ff47870c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.973+0000 7f7a4affd640 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 0x55ff478706d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00a550 tx=0x7f7a3c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c998f65b8c4b683a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.973+0000 7f7a2eff5640 10 client.25856 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.973+0000 7f7a2eff5640 1 -- 172.21.15.83:0/1135219660 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c021070 con 0x55ff478664c0 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.973+0000 7f7a2eff5640 1 -- 172.21.15.83:0/1135219660 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7a3c005020 con 0x55ff478664c0 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.973+0000 7f7a2eff5640 1 -- 172.21.15.83:0/1135219660 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a3c015070 con 0x55ff478664c0 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.973+0000 7f7a541bb740 10 client.25856 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.973+0000 7f7a541bb740 1 -- 172.21.15.83:0/1135219660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55ff477cd320 con 0x55ff478664c0 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.974+0000 7f7a2eff5640 1 -- 172.21.15.83:0/1135219660 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7a3c02d020 con 0x55ff478664c0 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.974+0000 7f7a541bb740 10 client.25856 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:33.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.974+0000 7f7a541bb740 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.974+0000 7f7a541bb740 20 client.25856 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.974+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ff4787c740 0x55ff4787eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.974+0000 7f7a541bb740 4 client.25856 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.974+0000 7f7a541bb740 1 -- 172.21.15.83:0/1135219660 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff477c5310 con 0x55ff4787c740 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.974+0000 7f7a4a7fc640 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ff4787c740 0x55ff4787eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.974+0000 7f7a2eff5640 10 client.25856 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.974+0000 7f7a4a7fc640 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ff4787c740 0x55ff4787eb50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5113000da8864629 server_cookie=c67d3da5f6c32013 in_seq=0 out_seq=0 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a2eff5640 1 -- 172.21.15.83:0/1135219660 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff477c5310 con 0x55ff4787c740 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a2eff5640 10 client.25856 handle_command_reply: tid=0 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 1 client.25856 shutdown 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 1 -- 172.21.15.83:0/1135219660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 msgr2=0x55ff478706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 0x55ff478706d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00a550 tx=0x7f7a3c004730 comp rx=0 tx=0).stop 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 1 -- 172.21.15.83:0/1135219660 shutdown_connections 2024-03-11T19:44:33.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ff4787c740 0x55ff4787eb50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a2eff5640 0 client.25856 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:33.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff47877c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 0x55ff47870c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1135219660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 0x55ff478706d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:33.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 1 -- 172.21.15.83:0/1135219660 >> 172.21.15.83:0/1135219660 conn(0x55ff478616a0 msgr2=0x55ff478608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:33.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 1 -- 172.21.15.83:0/1135219660 shutdown_connections 2024-03-11T19:44:33.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 1 -- 172.21.15.83:0/1135219660 wait complete. 2024-03-11T19:44:33.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.975+0000 7f7a541bb740 20 client.25856 trim_cache size 0 max 16384 2024-03-11T19:44:34.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a541bb740 1 Processor -- start 2024-03-11T19:44:34.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a541bb740 1 -- start start 2024-03-11T19:44:34.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 0x55ff478702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 0x55ff47877c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff47870820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a541bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff47862190 con 0x55ff47868050 2024-03-11T19:44:34.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a541bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff477c5310 con 0x55ff47867690 2024-03-11T19:44:34.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a541bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff47871850 con 0x55ff478664c0 2024-03-11T19:44:34.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a4b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff47870820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a4b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff47870820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40456/0 (socket says 172.21.15.83:40456) 2024-03-11T19:44:34.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a4affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 0x55ff478702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a4b7fe640 1 -- 172.21.15.83:0/859815633 learned_addr learned my addr 172.21.15.83:0/859815633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:34.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a4a7fc640 1 --2- 172.21.15.83:0/859815633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 0x55ff47877c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a4affd640 1 -- 172.21.15.83:0/859815633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 msgr2=0x55ff47877c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a4affd640 1 --2- 172.21.15.83:0/859815633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 0x55ff47877c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a4affd640 1 -- 172.21.15.83:0/859815633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 msgr2=0x55ff47870820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a4affd640 1 --2- 172.21.15.83:0/859815633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff47870820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a4affd640 1 -- 172.21.15.83:0/859815633 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff47769c00 con 0x55ff478664c0 2024-03-11T19:44:34.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.998+0000 7f7a4b7fe640 1 --2- 172.21.15.83:0/859815633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff47870820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:34.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.999+0000 7f7a4affd640 1 --2- 172.21.15.83:0/859815633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 0x55ff478702e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c010010 tx=0x7f7a3c021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b5ea02155801c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:34.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.999+0000 7f7a30ff9640 1 -- 172.21.15.83:0/859815633 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c005370 con 0x55ff478664c0 2024-03-11T19:44:34.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.999+0000 7f7a30ff9640 1 -- 172.21.15.83:0/859815633 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ff477a8f40 con 0x55ff478664c0 2024-03-11T19:44:34.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.999+0000 7f7a30ff9640 1 -- 172.21.15.83:0/859815633 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c004970 con 0x55ff478664c0 2024-03-11T19:44:34.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.999+0000 7f7a541bb740 1 -- 172.21.15.83:0/859815633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 msgr2=0x55ff478702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:33.999+0000 7f7a541bb740 1 --2- 172.21.15.83:0/859815633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 0x55ff478702e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c010010 tx=0x7f7a3c021000 comp rx=0 tx=0).stop 2024-03-11T19:44:34.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.000+0000 7f7a541bb740 1 -- 172.21.15.83:0/859815633 shutdown_connections 2024-03-11T19:44:34.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.000+0000 7f7a541bb740 1 --2- 172.21.15.83:0/859815633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff47870820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.000+0000 7f7a541bb740 1 --2- 172.21.15.83:0/859815633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff47867690 0x55ff47877c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.000+0000 7f7a541bb740 1 --2- 172.21.15.83:0/859815633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff478664c0 0x55ff478702e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.000+0000 7f7a541bb740 1 -- 172.21.15.83:0/859815633 >> 172.21.15.83:0/859815633 conn(0x55ff478616a0 msgr2=0x55ff47861a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.000+0000 7f7a541bb740 1 -- 172.21.15.83:0/859815633 shutdown_connections 2024-03-11T19:44:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.000+0000 7f7a541bb740 1 -- 172.21.15.83:0/859815633 wait complete. 2024-03-11T19:44:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.000+0000 7f7a541bb740 1 -- start start 2024-03-11T19:44:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.000+0000 7f7a541bb740 1 Processor -- start 2024-03-11T19:44:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.000+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 0x55ff47876b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.000+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 0x55ff478770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a541bb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff478d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a541bb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff478777f0 con 0x55ff47868050 2024-03-11T19:44:34.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a541bb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff478d31a0 con 0x55ff478664c0 2024-03-11T19:44:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a541bb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff478d3360 con 0x55ff47867690 2024-03-11T19:44:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a541bb740 10 client.0 _ll_register_callbacks cb 0x7f7a44009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff478d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff478d2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40472/0 (socket says 172.21.15.83:40472) 2024-03-11T19:44:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4b7fe640 1 -- 172.21.15.83:0/1863284621 learned_addr learned my addr 172.21.15.83:0/1863284621 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 0x55ff478770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4affd640 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 0x55ff47876b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4b7fe640 1 -- 172.21.15.83:0/1863284621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 msgr2=0x55ff478770b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4b7fe640 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 0x55ff478770b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4b7fe640 1 -- 172.21.15.83:0/1863284621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 msgr2=0x55ff47876b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4b7fe640 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 0x55ff47876b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4b7fe640 1 -- 172.21.15.83:0/1863284621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff477910b0 con 0x55ff47868050 2024-03-11T19:44:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4affd640 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 0x55ff47876b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:44:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4a7fc640 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 0x55ff478770b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a4b7fe640 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff478d2c30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7a44005850 tx=0x7f7a44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe30770047137fb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.001+0000 7f7a2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:44:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a2eff5640 1 -- 172.21.15.83:0/1863284621 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44011850 con 0x55ff47868050 2024-03-11T19:44:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a2eff5640 1 -- 172.21.15.83:0/1863284621 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7a440119b0 con 0x55ff47868050 2024-03-11T19:44:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a541bb740 1 -- 172.21.15.83:0/1863284621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff477f4900 con 0x55ff47868050 2024-03-11T19:44:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a2eff5640 1 -- 172.21.15.83:0/1863284621 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a44011b70 con 0x55ff47868050 2024-03-11T19:44:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a541bb740 10 client.16389 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a541bb740 1 -- 172.21.15.83:0/1863284621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ff477cd320 con 0x55ff47868050 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a2eff5640 1 -- 172.21.15.83:0/1863284621 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ff477f4900 con 0x55ff47868050 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a2eff5640 1 -- 172.21.15.83:0/1863284621 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7a44025040 con 0x55ff47868050 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a541bb740 10 client.16389 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a541bb740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a541bb740 20 client.16389 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ff4787c740 0x55ff478d6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a541bb740 4 client.16389 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.002+0000 7f7a541bb740 1 -- 172.21.15.83:0/1863284621 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ff474ce860 con 0x55ff4787c740 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.003+0000 7f7a4affd640 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ff4787c740 0x55ff478d6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.003+0000 7f7a2eff5640 10 client.16389 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:34.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.003+0000 7f7a4affd640 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ff4787c740 0x55ff478d6ea0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8d482120c372f5 server_cookie=70cfeb48018196c6 in_seq=0 out_seq=0 2024-03-11T19:44:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a2eff5640 1 -- 172.21.15.83:0/1863284621 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ff474ce860 con 0x55ff4787c740 2024-03-11T19:44:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a2eff5640 10 client.16389 handle_command_reply: tid=0 2024-03-11T19:44:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a541bb740 1 client.16389 shutdown 2024-03-11T19:44:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a541bb740 1 -- 172.21.15.83:0/1863284621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 msgr2=0x55ff478d2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff478d2c30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7a44005850 tx=0x7f7a44005880 comp rx=0 tx=0).stop 2024-03-11T19:44:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a541bb740 1 -- 172.21.15.83:0/1863284621 shutdown_connections 2024-03-11T19:44:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ff4787c740 0x55ff478d6ea0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff47868050 0x55ff478d2c30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff47867690 0x55ff478770b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a2eff5640 0 client.16389 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a541bb740 1 --2- 172.21.15.83:0/1863284621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff478664c0 0x55ff47876b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.004+0000 7f7a541bb740 1 -- 172.21.15.83:0/1863284621 >> 172.21.15.83:0/1863284621 conn(0x55ff478616a0 msgr2=0x55ff47870250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.005+0000 7f7a541bb740 1 -- 172.21.15.83:0/1863284621 shutdown_connections 2024-03-11T19:44:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.005+0000 7f7a541bb740 1 -- 172.21.15.83:0/1863284621 wait complete. 2024-03-11T19:44:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.005+0000 7f7a541bb740 20 client.16389 trim_cache size 0 max 16384 2024-03-11T19:44:34.016 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:44:34.016 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:44:34.016 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "226b0507-3f0f-4cb0-aa99-a19257cc9f80", 2024-03-11T19:44:34.017 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:44:34.017 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:44:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 -- 172.21.15.83:0/3899615947 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a340a1a80 msgr2=0x7f7a340a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a340a1a80 0x7f7a340a3f40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00a5c0 tx=0x7f7a3c00a2d0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 -- 172.21.15.83:0/3899615947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a4c1003d0 msgr2=0x7f7a4c105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a4c1003d0 0x7f7a4c105b30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7a40000c00 tx=0x7f7a4002dbc0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 -- 172.21.15.83:0/3899615947 shutdown_connections 2024-03-11T19:44:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a340a1a80 0x7f7a340a3f40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c100d70 0x7f7a4c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a4c1003d0 0x7f7a4c105b30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 --2- 172.21.15.83:0/3899615947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a4c0ffa70 0x7f7a4c1055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 -- 172.21.15.83:0/3899615947 >> 172.21.15.83:0/3899615947 conn(0x7f7a4c0fb610 msgr2=0x7f7a4c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 -- 172.21.15.83:0/3899615947 shutdown_connections 2024-03-11T19:44:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.007+0000 7f7a515e2640 1 -- 172.21.15.83:0/3899615947 wait complete. 2024-03-11T19:44:34.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:44:34.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:44:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.188+0000 7f910f7fe640 1 -- 172.21.15.83:0/2551995081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8004c20 con 0x7f9110102360 2024-03-11T19:44:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.188+0000 7f9116d77640 1 -- 172.21.15.83:0/2551995081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9110102360 msgr2=0x7f91101047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.188+0000 7f9116d77640 1 --2- 172.21.15.83:0/2551995081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9110102360 0x7f91101047d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f90f801a910 tx=0x7f90f803cc60 comp rx=0 tx=0).stop 2024-03-11T19:44:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.188+0000 7f9116d77640 1 -- 172.21.15.83:0/2551995081 shutdown_connections 2024-03-11T19:44:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.188+0000 7f9116d77640 1 --2- 172.21.15.83:0/2551995081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9110104d10 0x7f9110105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.188+0000 7f9116d77640 1 --2- 172.21.15.83:0/2551995081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9110102360 0x7f91101047d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.188+0000 7f9116d77640 1 --2- 172.21.15.83:0/2551995081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91100ff9f0 0x7f9110101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.188+0000 7f9116d77640 1 -- 172.21.15.83:0/2551995081 >> 172.21.15.83:0/2551995081 conn(0x7f91100fb5f0 msgr2=0x7f91100fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:34.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.189+0000 7f9116d77640 1 -- 172.21.15.83:0/2551995081 shutdown_connections 2024-03-11T19:44:34.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.189+0000 7f9116d77640 1 -- 172.21.15.83:0/2551995081 wait complete. 2024-03-11T19:44:34.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.189+0000 7f9116d77640 1 Processor -- start 2024-03-11T19:44:34.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.189+0000 7f9116d77640 1 -- start start 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f9116d77640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91100ff9f0 0x7f911006ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f9116d77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9110104d10 0x7f911006b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f9116d77640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f911006fe40 0x7f9110072300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f9116d77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9110072840 con 0x7f9110104d10 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f9116d77640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100729b0 con 0x7f911006fe40 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f9116d77640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9110072b70 con 0x7f91100ff9f0 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f910ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9110104d10 0x7f911006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f910ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9110104d10 0x7f911006b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40484/0 (socket says 172.21.15.83:40484) 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f910ffff640 1 -- 172.21.15.83:0/997386672 learned_addr learned my addr 172.21.15.83:0/997386672 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f9115314640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f911006fe40 0x7f9110072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f9114b13640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91100ff9f0 0x7f911006ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f910ffff640 1 -- 172.21.15.83:0/997386672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91100ff9f0 msgr2=0x7f911006ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f910ffff640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91100ff9f0 0x7f911006ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f910ffff640 1 -- 172.21.15.83:0/997386672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f911006fe40 msgr2=0x7f9110072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f910ffff640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f911006fe40 0x7f9110072300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f910ffff640 1 -- 172.21.15.83:0/997386672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f801a570 con 0x7f9110104d10 2024-03-11T19:44:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f910ffff640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9110104d10 0x7f911006b320 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f90f803ffd0 tx=0x7f90f8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f910dffb640 1 -- 172.21.15.83:0/997386672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f803dc20 con 0x7f9110104d10 2024-03-11T19:44:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.190+0000 7f9116d77640 1 -- 172.21.15.83:0/997386672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9110072df0 con 0x7f9110104d10 2024-03-11T19:44:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.191+0000 7f9116d77640 1 -- 172.21.15.83:0/997386672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91101a6550 con 0x7f9110104d10 2024-03-11T19:44:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.191+0000 7f910dffb640 1 -- 172.21.15.83:0/997386672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f90f803dd80 con 0x7f9110104d10 2024-03-11T19:44:34.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.191+0000 7f910dffb640 1 -- 172.21.15.83:0/997386672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8045440 con 0x7f9110104d10 2024-03-11T19:44:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.193+0000 7f9119a29740 1 Processor -- start 2024-03-11T19:44:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9119a29740 1 -- start start 2024-03-11T19:44:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f910dffb640 1 -- 172.21.15.83:0/997386672 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f90f80035f0 con 0x7f9110104d10 2024-03-11T19:44:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9119a29740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453c2d0 0x56097453c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9119a29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 0x5609745417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9119a29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560974541e30 0x5609745442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9119a29740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609744ac280 con 0x56097453c2d0 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9119a29740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609744a97b0 con 0x56097453ccc0 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9119a29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609744a4f90 con 0x560974541e30 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9115314640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453c2d0 0x56097453c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f910ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560974541e30 0x5609745442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9115314640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453c2d0 0x56097453c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40486/0 (socket says 172.21.15.83:40486) 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9115314640 1 -- 172.21.15.83:0/1545338082 learned_addr learned my addr 172.21.15.83:0/1545338082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9114b13640 1 --2- 172.21.15.83:0/1545338082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 0x5609745417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9115314640 1 -- 172.21.15.83:0/1545338082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560974541e30 msgr2=0x5609745442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9115314640 1 --2- 172.21.15.83:0/1545338082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560974541e30 0x5609745442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9115314640 1 -- 172.21.15.83:0/1545338082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 msgr2=0x5609745417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9115314640 1 --2- 172.21.15.83:0/1545338082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 0x5609745417c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9115314640 1 -- 172.21.15.83:0/1545338082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560974463570 con 0x56097453c2d0 2024-03-11T19:44:34.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.194+0000 7f9114b13640 1 --2- 172.21.15.83:0/1545338082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 0x5609745417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.195+0000 7f9115314640 1 --2- 172.21.15.83:0/1545338082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453c2d0 0x56097453c6f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9108009fd0 tx=0x7f91080118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8db4dcae85dc73 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.195+0000 7f910dffb640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f90fc0a1ab0 0x7f90fc0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.195+0000 7f9106ffd640 1 -- 172.21.15.83:0/1545338082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108004d20 con 0x56097453c2d0 2024-03-11T19:44:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.195+0000 7f9114b13640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f90fc0a1ab0 0x7f90fc0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.195+0000 7f9106ffd640 1 -- 172.21.15.83:0/1545338082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9108004e80 con 0x56097453c2d0 2024-03-11T19:44:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.195+0000 7f9106ffd640 1 -- 172.21.15.83:0/1545338082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108005280 con 0x56097453c2d0 2024-03-11T19:44:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.195+0000 7f910dffb640 1 -- 172.21.15.83:0/997386672 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f90f81006c0 con 0x7f9110104d10 2024-03-11T19:44:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.196+0000 7f9119a29740 1 -- 172.21.15.83:0/1545338082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453c2d0 msgr2=0x56097453c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.196+0000 7f9119a29740 1 --2- 172.21.15.83:0/1545338082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453c2d0 0x56097453c6f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9108009fd0 tx=0x7f91080118d0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.196+0000 7f9114b13640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f90fc0a1ab0 0x7f90fc0a3f70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x56097453d0c0 tx=0x7f9100009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.199+0000 7f9119a29740 1 -- 172.21.15.83:0/1545338082 shutdown_connections 2024-03-11T19:44:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.199+0000 7f9119a29740 1 --2- 172.21.15.83:0/1545338082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560974541e30 0x5609745442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.199+0000 7f9119a29740 1 --2- 172.21.15.83:0/1545338082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 0x5609745417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.199+0000 7f9119a29740 1 --2- 172.21.15.83:0/1545338082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453c2d0 0x56097453c6f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.199+0000 7f9119a29740 1 -- 172.21.15.83:0/1545338082 >> 172.21.15.83:0/1545338082 conn(0x5609745377f0 msgr2=0x560974537bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.199+0000 7f9119a29740 1 -- 172.21.15.83:0/1545338082 shutdown_connections 2024-03-11T19:44:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.199+0000 7f9119a29740 1 -- 172.21.15.83:0/1545338082 wait complete. 2024-03-11T19:44:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.199+0000 7f9119a29740 1 -- start start 2024-03-11T19:44:34.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9119a29740 1 Processor -- start 2024-03-11T19:44:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9119a29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453c2d0 0x560974546300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9119a29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 0x560974546840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9119a29740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560974541e30 0x56097454d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9119a29740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974178430 con 0x560974541e30 2024-03-11T19:44:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9119a29740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974134e00 con 0x56097453ccc0 2024-03-11T19:44:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9119a29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974429fd0 con 0x56097453c2d0 2024-03-11T19:44:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9119a29740 10 client.0 _ll_register_callbacks cb 0x56097447ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9115314640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560974541e30 0x56097454d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9115314640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560974541e30 0x56097454d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40498/0 (socket says 172.21.15.83:40498) 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9115314640 1 -- 172.21.15.83:0/3852180774 learned_addr learned my addr 172.21.15.83:0/3852180774 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f910ffff640 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 0x560974546840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.200+0000 7f9114b13640 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453c2d0 0x560974546300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f9114b13640 1 -- 172.21.15.83:0/3852180774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 msgr2=0x560974546840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f9114b13640 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 0x560974546840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f9114b13640 1 -- 172.21.15.83:0/3852180774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560974541e30 msgr2=0x56097454d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f9114b13640 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560974541e30 0x56097454d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f9114b13640 1 -- 172.21.15.83:0/3852180774 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9108009c80 con 0x56097453c2d0 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f9115314640 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560974541e30 0x56097454d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f910ffff640 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 0x560974546840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f9114b13640 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453c2d0 0x560974546300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5609745387b0 tx=0x7f91000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83137a8edb96292 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f90effff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:44:34.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f90effff640 1 -- 172.21.15.83:0/3852180774 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100011ee0 con 0x56097453c2d0 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f9119a29740 1 -- 172.21.15.83:0/3852180774 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609741e1720 con 0x56097453c2d0 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f90effff640 1 -- 172.21.15.83:0/3852180774 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f910001c040 con 0x56097453c2d0 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f90effff640 1 -- 172.21.15.83:0/3852180774 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9100014e20 con 0x56097453c2d0 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f9119a29740 10 client.25972 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.201+0000 7f9119a29740 1 -- 172.21.15.83:0/3852180774 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5609744a3610 con 0x56097453c2d0 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.202+0000 7f90effff640 1 -- 172.21.15.83:0/3852180774 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5609741e1720 con 0x56097453c2d0 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.202+0000 7f90effff640 1 -- 172.21.15.83:0/3852180774 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f9100029810 con 0x56097453c2d0 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.202+0000 7f9119a29740 10 client.25972 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.202+0000 7f9119a29740 10 client.25972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.202+0000 7f9119a29740 20 client.25972 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.202+0000 7f9119a29740 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5609745524a0 0x5609745548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.202+0000 7f9119a29740 4 client.25972 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.202+0000 7f9119a29740 1 -- 172.21.15.83:0/3852180774 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560974230890 con 0x5609745524a0 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.202+0000 7f910ffff640 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5609745524a0 0x5609745548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.203+0000 7f90effff640 10 client.25972 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.203+0000 7f910ffff640 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5609745524a0 0x5609745548c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5286ad3bc5952298 server_cookie=fa2483a17ded83e7 in_seq=0 out_seq=0 2024-03-11T19:44:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.203+0000 7f90effff640 1 -- 172.21.15.83:0/3852180774 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560974230890 con 0x5609745524a0 2024-03-11T19:44:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.203+0000 7f90effff640 10 client.25972 handle_command_reply: tid=0 2024-03-11T19:44:34.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.203+0000 7f9119a29740 1 client.25972 shutdown 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f9119a29740 1 -- 172.21.15.83:0/3852180774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453c2d0 msgr2=0x560974546300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f9119a29740 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453c2d0 0x560974546300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5609745387b0 tx=0x7f91000046c0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f9119a29740 1 -- 172.21.15.83:0/3852180774 shutdown_connections 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f9119a29740 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5609745524a0 0x5609745548c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f9119a29740 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560974541e30 0x56097454d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f9119a29740 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453ccc0 0x560974546840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f9119a29740 1 --2- 172.21.15.83:0/3852180774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453c2d0 0x560974546300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f9119a29740 1 -- 172.21.15.83:0/3852180774 >> 172.21.15.83:0/3852180774 conn(0x5609745377f0 msgr2=0x560974536a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f90effff640 0 client.25972 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f9119a29740 1 -- 172.21.15.83:0/3852180774 shutdown_connections 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f9119a29740 1 -- 172.21.15.83:0/3852180774 wait complete. 2024-03-11T19:44:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.204+0000 7f9119a29740 20 client.25972 trim_cache size 0 max 16384 2024-03-11T19:44:34.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.226+0000 7f9119a29740 1 Processor -- start 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.226+0000 7f9119a29740 1 -- start start 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9119a29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609745377f0 0x560974559240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9119a29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453c2d0 0x56097454d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9119a29740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453ccc0 0x56097453df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9119a29740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560974533320 con 0x56097453ccc0 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9119a29740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56097453d0f0 con 0x5609745377f0 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9119a29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56097454ed00 con 0x56097453c2d0 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9115314640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453ccc0 0x56097453df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9115314640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453ccc0 0x56097453df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40502/0 (socket says 172.21.15.83:40502) 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9115314640 1 -- 172.21.15.83:0/3966273245 learned_addr learned my addr 172.21.15.83:0/3966273245 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f910ffff640 1 --2- 172.21.15.83:0/3966273245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453c2d0 0x56097454d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9114b13640 1 --2- 172.21.15.83:0/3966273245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609745377f0 0x560974559240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9115314640 1 -- 172.21.15.83:0/3966273245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453c2d0 msgr2=0x56097454d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9115314640 1 --2- 172.21.15.83:0/3966273245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453c2d0 0x56097454d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9115314640 1 -- 172.21.15.83:0/3966273245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609745377f0 msgr2=0x560974559240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9115314640 1 --2- 172.21.15.83:0/3966273245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609745377f0 0x560974559240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9115314640 1 -- 172.21.15.83:0/3966273245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56097443fac0 con 0x56097453ccc0 2024-03-11T19:44:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.227+0000 7f9115314640 1 --2- 172.21.15.83:0/3966273245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453ccc0 0x56097453df60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9108011dd0 tx=0x7f9108004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8249fedf06d21aa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.228+0000 7f91067fc640 1 -- 172.21.15.83:0/3966273245 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108003970 con 0x56097453ccc0 2024-03-11T19:44:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.228+0000 7f91067fc640 1 -- 172.21.15.83:0/3966273245 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9108004d00 con 0x56097453ccc0 2024-03-11T19:44:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.228+0000 7f91067fc640 1 -- 172.21.15.83:0/3966273245 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910800faa0 con 0x56097453ccc0 2024-03-11T19:44:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.228+0000 7f9119a29740 1 -- 172.21.15.83:0/3966273245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453ccc0 msgr2=0x56097453df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.228+0000 7f9119a29740 1 --2- 172.21.15.83:0/3966273245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453ccc0 0x56097453df60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9108011dd0 tx=0x7f9108004380 comp rx=0 tx=0).stop 2024-03-11T19:44:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.229+0000 7f9119a29740 1 -- 172.21.15.83:0/3966273245 shutdown_connections 2024-03-11T19:44:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.229+0000 7f9119a29740 1 --2- 172.21.15.83:0/3966273245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56097453ccc0 0x56097453df60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.229+0000 7f9119a29740 1 --2- 172.21.15.83:0/3966273245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453c2d0 0x56097454d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.229+0000 7f9119a29740 1 --2- 172.21.15.83:0/3966273245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609745377f0 0x560974559240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.229+0000 7f9119a29740 1 -- 172.21.15.83:0/3966273245 >> 172.21.15.83:0/3966273245 conn(0x560974541900 msgr2=0x560974532840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.229+0000 7f9119a29740 1 -- 172.21.15.83:0/3966273245 shutdown_connections 2024-03-11T19:44:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.229+0000 7f9119a29740 1 -- 172.21.15.83:0/3966273245 wait complete. 2024-03-11T19:44:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.229+0000 7f9119a29740 1 -- start start 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.229+0000 7f9119a29740 1 Processor -- start 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.229+0000 7f9119a29740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609745377f0 0x56097454b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9119a29740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453c2d0 0x56097454bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9119a29740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453ccc0 0x56097454cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9114b13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609745377f0 0x56097454b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9119a29740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56097454d050 con 0x5609745377f0 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9119a29740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56097454d1c0 con 0x56097453c2d0 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9119a29740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56097454d380 con 0x56097453ccc0 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9114b13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609745377f0 0x56097454b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40512/0 (socket says 172.21.15.83:40512) 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9114b13640 1 -- 172.21.15.83:0/1847041340 learned_addr learned my addr 172.21.15.83:0/1847041340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9119a29740 10 client.0 _ll_register_callbacks cb 0x7f9108009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9115314640 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453ccc0 0x56097454cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f910ffff640 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453c2d0 0x56097454bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9114b13640 1 -- 172.21.15.83:0/1847041340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453ccc0 msgr2=0x56097454cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9114b13640 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453ccc0 0x56097454cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9114b13640 1 -- 172.21.15.83:0/1847041340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453c2d0 msgr2=0x56097454bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9114b13640 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453c2d0 0x56097454bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9114b13640 1 -- 172.21.15.83:0/1847041340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560974463570 con 0x5609745377f0 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f910ffff640 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453c2d0 0x56097454bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f9114b13640 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609745377f0 0x56097454b5c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9100029cc0 tx=0x7f9100011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2600e74772d3375 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f90effff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.230+0000 7f90effff640 1 -- 172.21.15.83:0/1847041340 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000297c0 con 0x5609745377f0 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f9119a29740 1 -- 172.21.15.83:0/1847041340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609741934e0 con 0x5609745377f0 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f90effff640 1 -- 172.21.15.83:0/1847041340 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9100029e30 con 0x5609745377f0 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f90effff640 1 -- 172.21.15.83:0/1847041340 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9100006a30 con 0x5609745377f0 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f9119a29740 10 client.16407 fetch_fsmap learned FSMap version 29 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f9119a29740 1 -- 172.21.15.83:0/1847041340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5609744656a0 con 0x5609745377f0 2024-03-11T19:44:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f90effff640 1 -- 172.21.15.83:0/1847041340 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9100047020 con 0x5609745377f0 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f90effff640 1 -- 172.21.15.83:0/1847041340 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f91000135c0 con 0x5609745377f0 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f9119a29740 10 client.16407 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f9119a29740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f9119a29740 20 client.16407 populate_metadata read hostname 'smithi083' 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f9119a29740 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560974541e30 0x5609745ab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f9119a29740 4 client.16407 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f9119a29740 1 -- 172.21.15.83:0/1847041340 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560974150700 con 0x560974541e30 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.231+0000 7f910ffff640 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560974541e30 0x5609745ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.232+0000 7f90effff640 10 client.16407 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.232+0000 7f910ffff640 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560974541e30 0x5609745ab910 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e233394619211a65 server_cookie=8acb214690c10bca in_seq=0 out_seq=0 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.233+0000 7f90effff640 1 -- 172.21.15.83:0/1847041340 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560974150700 con 0x560974541e30 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.233+0000 7f90effff640 10 client.16407 handle_command_reply: tid=0 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.233+0000 7f9119a29740 1 client.16407 shutdown 2024-03-11T19:44:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.233+0000 7f9119a29740 1 -- 172.21.15.83:0/1847041340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609745377f0 msgr2=0x56097454b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.233+0000 7f9119a29740 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609745377f0 0x56097454b5c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9100029cc0 tx=0x7f9100011bf0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.233+0000 7f9119a29740 1 -- 172.21.15.83:0/1847041340 shutdown_connections 2024-03-11T19:44:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.233+0000 7f9119a29740 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x560974541e30 0x5609745ab910 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.234+0000 7f90effff640 0 client.16407 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:44:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.234+0000 7f9119a29740 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56097453ccc0 0x56097454cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.234+0000 7f9119a29740 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56097453c2d0 0x56097454bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.234+0000 7f9119a29740 1 --2- 172.21.15.83:0/1847041340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609745377f0 0x56097454b5c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.234+0000 7f9119a29740 1 -- 172.21.15.83:0/1847041340 >> 172.21.15.83:0/1847041340 conn(0x560974541900 msgr2=0x560974559240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.235+0000 7f9119a29740 1 -- 172.21.15.83:0/1847041340 shutdown_connections 2024-03-11T19:44:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.235+0000 7f9119a29740 1 -- 172.21.15.83:0/1847041340 wait complete. 2024-03-11T19:44:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.235+0000 7f9119a29740 20 client.16407 trim_cache size 0 max 16384 2024-03-11T19:44:34.243 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:44:34.243 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:44:34.243 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:44:34.243 INFO:teuthology.orchestra.run.smithi083.stdout: "226b0507-3f0f-4cb0-aa99-a19257cc9f80": { 2024-03-11T19:44:34.243 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:44:34.243 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "226b0507-3f0f-4cb0-aa99-a19257cc9f80", 2024-03-11T19:44:34.243 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:44:34.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:44:34.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:44:34.243 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:44:34.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 -- 172.21.15.83:0/997386672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f90fc0a1ab0 msgr2=0x7f90fc0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f90fc0a1ab0 0x7f90fc0a3f70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x56097453d0c0 tx=0x7f9100009110 comp rx=0 tx=0).stop 2024-03-11T19:44:34.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 -- 172.21.15.83:0/997386672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9110104d10 msgr2=0x7f911006b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:44:34.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9110104d10 0x7f911006b320 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f90f803ffd0 tx=0x7f90f8004430 comp rx=0 tx=0).stop 2024-03-11T19:44:34.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 -- 172.21.15.83:0/997386672 shutdown_connections 2024-03-11T19:44:34.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f90fc0a1ab0 0x7f90fc0a3f70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f911006fe40 0x7f9110072300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9110104d10 0x7f911006b320 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 --2- 172.21.15.83:0/997386672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91100ff9f0 0x7f911006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:44:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 -- 172.21.15.83:0/997386672 >> 172.21.15.83:0/997386672 conn(0x7f91100fb5f0 msgr2=0x7f91100fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:44:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 -- 172.21.15.83:0/997386672 shutdown_connections 2024-03-11T19:44:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:44:34.236+0000 7f9116d77640 1 -- 172.21.15.83:0/997386672 wait complete. 2024-03-11T19:44:34.253 INFO:tasks.cephfs.filesystem:scrub status for tag:226b0507-3f0f-4cb0-aa99-a19257cc9f80 - {'path': '/', 'tag': '226b0507-3f0f-4cb0-aa99-a19257cc9f80', 'options': 'recursive,force'} 2024-03-11T19:44:34.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:34 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:44:34.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:34 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:34.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:34 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:44:34.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:34 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:34.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:34 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:44:34.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:34 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:35 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:44:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:35 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:35 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:44:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:35 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:44:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:35 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:35 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:44:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:35 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:35 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:35 smithi083 ceph-mon[31864]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.8 MiB/s wr, 820 op/s 2024-03-11T19:44:35.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:35 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:44:35.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:35 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:35 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:44:35.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:35 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:44:35.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:35 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:35 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:44:35.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:35 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:35 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:35 smithi137 ceph-mon[36220]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.8 MiB/s wr, 820 op/s 2024-03-11T19:44:35.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:35 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:44:35.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:35 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:35 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:44:35.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:35 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:44:35.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:35 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:35 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:44:35.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:35 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:35 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:44:35.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:35 smithi204 ceph-mon[36739]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.8 MiB/s wr, 820 op/s 2024-03-11T19:44:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:36 smithi083 ceph-mon[31864]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 MiB/s wr, 708 op/s 2024-03-11T19:44:37.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:36 smithi137 ceph-mon[36220]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 MiB/s wr, 708 op/s 2024-03-11T19:44:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:36 smithi204 ceph-mon[36739]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 MiB/s wr, 708 op/s 2024-03-11T19:44:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 320.0 s, 2556.3 tps, lat 39.295 ms stddev 40.545 2024-03-11T19:44:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:38 smithi083 ceph-mon[31864]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 775 op/s 2024-03-11T19:44:39.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:38 smithi137 ceph-mon[36220]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 775 op/s 2024-03-11T19:44:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:38 smithi204 ceph-mon[36739]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 775 op/s 2024-03-11T19:44:40.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:40 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:44:40.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:40 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:44:40.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:40 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:44:41.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:41 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:44:41.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:41 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:44:41.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:41 smithi083 ceph-mon[31864]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.0 MiB/s wr, 655 op/s 2024-03-11T19:44:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:41 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:44:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:41 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:44:41.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:41 smithi137 ceph-mon[36220]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.0 MiB/s wr, 655 op/s 2024-03-11T19:44:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:41 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:44:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:41 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:44:41.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:41 smithi204 ceph-mon[36739]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.0 MiB/s wr, 655 op/s 2024-03-11T19:44:42.497 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 325.0 s, 3084.8 tps, lat 32.402 ms stddev 35.326 2024-03-11T19:44:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:42 smithi083 ceph-mon[31864]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.9 MiB/s wr, 682 op/s 2024-03-11T19:44:43.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:42 smithi137 ceph-mon[36220]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.9 MiB/s wr, 682 op/s 2024-03-11T19:44:43.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:42 smithi204 ceph-mon[36739]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 6.9 MiB/s wr, 682 op/s 2024-03-11T19:44:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:44 smithi083 ceph-mon[31864]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.6 MiB/s wr, 690 op/s 2024-03-11T19:44:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:44 smithi137 ceph-mon[36220]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.6 MiB/s wr, 690 op/s 2024-03-11T19:44:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:44 smithi204 ceph-mon[36739]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.6 MiB/s wr, 690 op/s 2024-03-11T19:44:47.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:46 smithi083 ceph-mon[31864]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.2 MiB/s wr, 720 op/s 2024-03-11T19:44:47.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:46 smithi137 ceph-mon[36220]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.2 MiB/s wr, 720 op/s 2024-03-11T19:44:47.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:46 smithi204 ceph-mon[36739]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.2 MiB/s wr, 720 op/s 2024-03-11T19:44:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 330.0 s, 3628.8 tps, lat 27.620 ms stddev 30.795 2024-03-11T19:44:49.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:48 smithi083 ceph-mon[31864]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.7 MiB/s wr, 769 op/s 2024-03-11T19:44:49.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:48 smithi137 ceph-mon[36220]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.7 MiB/s wr, 769 op/s 2024-03-11T19:44:49.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:48 smithi204 ceph-mon[36739]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.7 MiB/s wr, 769 op/s 2024-03-11T19:44:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:50 smithi083 ceph-mon[31864]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.2 MiB/s wr, 703 op/s 2024-03-11T19:44:51.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:50 smithi137 ceph-mon[36220]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.2 MiB/s wr, 703 op/s 2024-03-11T19:44:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:50 smithi204 ceph-mon[36739]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 7.2 MiB/s wr, 703 op/s 2024-03-11T19:44:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 335.0 s, 3095.6 tps, lat 32.302 ms stddev 38.930 2024-03-11T19:44:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:52 smithi083 ceph-mon[31864]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 MiB/s wr, 694 op/s 2024-03-11T19:44:53.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:52 smithi137 ceph-mon[36220]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 MiB/s wr, 694 op/s 2024-03-11T19:44:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:52 smithi204 ceph-mon[36739]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 MiB/s wr, 694 op/s 2024-03-11T19:44:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:54 smithi083 ceph-mon[31864]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 MiB/s wr, 770 op/s 2024-03-11T19:44:55.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:54 smithi137 ceph-mon[36220]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 MiB/s wr, 770 op/s 2024-03-11T19:44:55.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:54 smithi204 ceph-mon[36739]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.8 MiB/s wr, 770 op/s 2024-03-11T19:44:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:56 smithi083 ceph-mon[31864]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.6 MiB/s wr, 729 op/s 2024-03-11T19:44:57.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:56 smithi137 ceph-mon[36220]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.6 MiB/s wr, 729 op/s 2024-03-11T19:44:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:56 smithi204 ceph-mon[36739]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.6 MiB/s wr, 729 op/s 2024-03-11T19:44:57.497 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 340.0 s, 3315.6 tps, lat 30.069 ms stddev 33.612 2024-03-11T19:44:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:44:58 smithi083 ceph-mon[31864]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 715 op/s 2024-03-11T19:44:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:44:58 smithi137 ceph-mon[36220]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 715 op/s 2024-03-11T19:44:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:44:58 smithi204 ceph-mon[36739]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 715 op/s 2024-03-11T19:45:01.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:00 smithi083 ceph-mon[31864]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.2 MiB/s wr, 723 op/s 2024-03-11T19:45:01.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:00 smithi137 ceph-mon[36220]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.2 MiB/s wr, 723 op/s 2024-03-11T19:45:01.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:00 smithi204 ceph-mon[36739]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.2 MiB/s wr, 723 op/s 2024-03-11T19:45:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 345.0 s, 4215.1 tps, lat 23.718 ms stddev 28.607 2024-03-11T19:45:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:02 smithi083 ceph-mon[31864]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-03-11T19:45:03.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:02 smithi137 ceph-mon[36220]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-03-11T19:45:03.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:02 smithi204 ceph-mon[36739]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-03-11T19:45:04.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:45:04.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:45:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.397+0000 7fed03f4f640 1 -- 172.21.15.83:0/1888221434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecfc101270 msgr2=0x7fecfc1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.397+0000 7fed03f4f640 1 --2- 172.21.15.83:0/1888221434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecfc101270 0x7fecfc1016f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fecec009860 tx=0x7fecec02bb70 comp rx=0 tx=0).stop 2024-03-11T19:45:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.397+0000 7fed03f4f640 1 -- 172.21.15.83:0/1888221434 shutdown_connections 2024-03-11T19:45:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.397+0000 7fed03f4f640 1 --2- 172.21.15.83:0/1888221434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecfc101c30 0x7fecfc1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.397+0000 7fed03f4f640 1 --2- 172.21.15.83:0/1888221434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecfc101270 0x7fecfc1016f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.397+0000 7fed03f4f640 1 --2- 172.21.15.83:0/1888221434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecfc100080 0x7fecfc1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.397+0000 7fed03f4f640 1 -- 172.21.15.83:0/1888221434 >> 172.21.15.83:0/1888221434 conn(0x7fecfc0fb5f0 msgr2=0x7fecfc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:04.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.398+0000 7fed03f4f640 1 -- 172.21.15.83:0/1888221434 shutdown_connections 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.398+0000 7fed03f4f640 1 -- 172.21.15.83:0/1888221434 wait complete. 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.398+0000 7fed03f4f640 1 Processor -- start 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.398+0000 7fed03f4f640 1 -- start start 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed03f4f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecfc100080 0x7fecfc1961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed03f4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecfc101270 0x7fecfc196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed03f4f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecfc101c30 0x7fecfc19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed014ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecfc101270 0x7fecfc196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed03f4f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc19dcc0 con 0x7fecfc101270 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed03f4f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc19de30 con 0x7fecfc100080 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed03f4f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc19dff0 con 0x7fecfc101c30 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed014ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecfc101270 0x7fecfc196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49232/0 (socket says 172.21.15.83:49232) 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed014ea640 1 -- 172.21.15.83:0/3080915676 learned_addr learned my addr 172.21.15.83:0/3080915676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:04.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed01ceb640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecfc100080 0x7fecfc1961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed01ceb640 1 -- 172.21.15.83:0/3080915676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecfc101c30 msgr2=0x7fecfc19d780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-11T19:45:04.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed01ceb640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecfc101c30 0x7fecfc19d780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed01ceb640 1 -- 172.21.15.83:0/3080915676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecfc101270 msgr2=0x7fecfc196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed01ceb640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecfc101270 0x7fecfc196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.399+0000 7fed01ceb640 1 -- 172.21.15.83:0/3080915676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec009490 con 0x7fecfc100080 2024-03-11T19:45:04.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.400+0000 7fed01ceb640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecfc100080 0x7fecfc1961d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fecfc1010d0 tx=0x7fecf800dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:04.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.400+0000 7fecf2ffd640 1 -- 172.21.15.83:0/3080915676 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf8009a30 con 0x7fecfc100080 2024-03-11T19:45:04.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.400+0000 7fecf2ffd640 1 -- 172.21.15.83:0/3080915676 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecf8016070 con 0x7fecfc100080 2024-03-11T19:45:04.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.400+0000 7fecf2ffd640 1 -- 172.21.15.83:0/3080915676 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf801c820 con 0x7fecfc100080 2024-03-11T19:45:04.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.400+0000 7fed03f4f640 1 -- 172.21.15.83:0/3080915676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecfc068340 con 0x7fecfc100080 2024-03-11T19:45:04.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.400+0000 7fed03f4f640 1 -- 172.21.15.83:0/3080915676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecfc068880 con 0x7fecfc100080 2024-03-11T19:45:04.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.403+0000 7fecf2ffd640 1 -- 172.21.15.83:0/3080915676 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fecf8009b90 con 0x7fecfc100080 2024-03-11T19:45:04.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.404+0000 7fecf2ffd640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fece80a1bb0 0x7fece80a4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.404+0000 7fed06c01740 1 Processor -- start 2024-03-11T19:45:04.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.404+0000 7fecf2ffd640 1 -- 172.21.15.83:0/3080915676 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fecf80ccfe0 con 0x7fecfc100080 2024-03-11T19:45:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed014ea640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fece80a1bb0 0x7fece80a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed06c01740 1 -- start start 2024-03-11T19:45:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed06c01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb982d0 0x55649bb986f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed06c01740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb98cc0 0x55649bb9d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed06c01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb9de30 0x55649bba02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed06c01740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649bb08280 con 0x55649bb98cc0 2024-03-11T19:45:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed06c01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649bb057b0 con 0x55649bb9de30 2024-03-11T19:45:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed06c01740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649bb00f90 con 0x55649bb982d0 2024-03-11T19:45:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed024ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb9de30 0x55649bba02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed014ea640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fece80a1bb0 0x7fece80a4070 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fecec032530 tx=0x7fecec034be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed014ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb98cc0 0x55649bb9d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed024ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb9de30 0x55649bba02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47004/0 (socket says 172.21.15.83:47004) 2024-03-11T19:45:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed024ec640 1 -- 172.21.15.83:0/1428140340 learned_addr learned my addr 172.21.15.83:0/1428140340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.405+0000 7fed01ceb640 1 --2- 172.21.15.83:0/1428140340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb982d0 0x55649bb986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.406+0000 7fed024ec640 1 -- 172.21.15.83:0/1428140340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb982d0 msgr2=0x55649bb986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.406+0000 7fed024ec640 1 --2- 172.21.15.83:0/1428140340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb982d0 0x55649bb986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.406+0000 7fed024ec640 1 -- 172.21.15.83:0/1428140340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb98cc0 msgr2=0x55649bb9d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.406+0000 7fed024ec640 1 --2- 172.21.15.83:0/1428140340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb98cc0 0x55649bb9d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.406+0000 7fed024ec640 1 -- 172.21.15.83:0/1428140340 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55649babf570 con 0x55649bb9de30 2024-03-11T19:45:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.406+0000 7fed024ec640 1 --2- 172.21.15.83:0/1428140340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb9de30 0x55649bba02c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009d00 tx=0x7fecf400b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72240c2e70a90e44 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.406+0000 7fece7fff640 1 -- 172.21.15.83:0/1428140340 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf400e070 con 0x55649bb9de30 2024-03-11T19:45:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.406+0000 7fece7fff640 1 -- 172.21.15.83:0/1428140340 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecf4009830 con 0x55649bb9de30 2024-03-11T19:45:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.406+0000 7fece7fff640 1 -- 172.21.15.83:0/1428140340 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4011690 con 0x55649bb9de30 2024-03-11T19:45:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.407+0000 7fed06c01740 1 -- 172.21.15.83:0/1428140340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb9de30 msgr2=0x55649bba02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.407+0000 7fed06c01740 1 --2- 172.21.15.83:0/1428140340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb9de30 0x55649bba02c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009d00 tx=0x7fecf400b420 comp rx=0 tx=0).stop 2024-03-11T19:45:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.407+0000 7fed06c01740 1 -- 172.21.15.83:0/1428140340 shutdown_connections 2024-03-11T19:45:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.407+0000 7fed06c01740 1 --2- 172.21.15.83:0/1428140340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb9de30 0x55649bba02c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.407+0000 7fed06c01740 1 --2- 172.21.15.83:0/1428140340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb98cc0 0x55649bb9d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.407+0000 7fed06c01740 1 --2- 172.21.15.83:0/1428140340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb982d0 0x55649bb986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.407+0000 7fed06c01740 1 -- 172.21.15.83:0/1428140340 >> 172.21.15.83:0/1428140340 conn(0x55649bb937f0 msgr2=0x55649bb93bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.407+0000 7fed06c01740 1 -- 172.21.15.83:0/1428140340 shutdown_connections 2024-03-11T19:45:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.407+0000 7fed06c01740 1 -- 172.21.15.83:0/1428140340 wait complete. 2024-03-11T19:45:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.407+0000 7fed06c01740 1 -- start start 2024-03-11T19:45:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed06c01740 1 Processor -- start 2024-03-11T19:45:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed06c01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bba2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed06c01740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb98cc0 0x55649bba2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed06c01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb9de30 0x55649bba97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed06c01740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649b7d4430 con 0x55649bb98cc0 2024-03-11T19:45:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed06c01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649b790e00 con 0x55649bb982d0 2024-03-11T19:45:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed06c01740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649ba85fd0 con 0x55649bb9de30 2024-03-11T19:45:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed014ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb98cc0 0x55649bba2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed06c01740 10 client.0 _ll_register_callbacks cb 0x55649badbfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed014ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb98cc0 0x55649bba2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49240/0 (socket says 172.21.15.83:49240) 2024-03-11T19:45:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed014ea640 1 -- 172.21.15.83:0/2967978418 learned_addr learned my addr 172.21.15.83:0/2967978418 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed01ceb640 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bba2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.408+0000 7fed024ec640 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb9de30 0x55649bba97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.409+0000 7fed01ceb640 1 -- 172.21.15.83:0/2967978418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb9de30 msgr2=0x55649bba97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:04.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.409+0000 7fed01ceb640 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb9de30 0x55649bba97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.409+0000 7fed01ceb640 1 -- 172.21.15.83:0/2967978418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb98cc0 msgr2=0x55649bba2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.409+0000 7fed01ceb640 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb98cc0 0x55649bba2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.409+0000 7fed01ceb640 1 -- 172.21.15.83:0/2967978418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf40099b0 con 0x55649bb982d0 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.409+0000 7fed01ceb640 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bba2220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fecf80cabc0 tx=0x7fecf809a820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb223e5b9dd2ee55 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.409+0000 7fece57fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.410+0000 7fece57fa640 1 -- 172.21.15.83:0/2967978418 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80b1020 con 0x55649bb982d0 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.410+0000 7fece57fa640 1 -- 172.21.15.83:0/2967978418 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fecf809a240 con 0x55649bb982d0 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.410+0000 7fed06c01740 1 -- 172.21.15.83:0/2967978418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55649b83d720 con 0x55649bb982d0 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.410+0000 7fece57fa640 1 -- 172.21.15.83:0/2967978418 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecf809a400 con 0x55649bb982d0 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.411+0000 7fed06c01740 10 client.25892 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.411+0000 7fed06c01740 1 -- 172.21.15.83:0/2967978418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55649baff610 con 0x55649bb982d0 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.411+0000 7fece57fa640 1 -- 172.21.15.83:0/2967978418 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55649b83d720 con 0x55649bb982d0 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.411+0000 7fece57fa640 1 -- 172.21.15.83:0/2967978418 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fecf801a030 con 0x55649bb982d0 2024-03-11T19:45:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.411+0000 7fed06c01740 10 client.25892 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.411+0000 7fed06c01740 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.411+0000 7fed06c01740 20 client.25892 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.411+0000 7fed06c01740 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55649bbae460 0x55649bbb0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.411+0000 7fed06c01740 4 client.25892 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.411+0000 7fed06c01740 1 -- 172.21.15.83:0/2967978418 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55649b88c890 con 0x55649bbae460 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.412+0000 7fed014ea640 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55649bbae460 0x55649bbb0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.412+0000 7fed014ea640 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55649bbae460 0x55649bbb0850 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f4b554e986f7e2f server_cookie=410e9fffca1b47af in_seq=0 out_seq=0 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.412+0000 7fece57fa640 10 client.25892 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fece57fa640 1 -- 172.21.15.83:0/2967978418 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55649b88c890 con 0x55649bbae460 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fece57fa640 10 client.25892 handle_command_reply: tid=0 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 1 client.25892 shutdown 2024-03-11T19:45:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 1 -- 172.21.15.83:0/2967978418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 msgr2=0x55649bba2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bba2220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fecf80cabc0 tx=0x7fecf809a820 comp rx=0 tx=0).stop 2024-03-11T19:45:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 1 -- 172.21.15.83:0/2967978418 shutdown_connections 2024-03-11T19:45:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55649bbae460 0x55649bbb0850 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb9de30 0x55649bba97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb98cc0 0x55649bba2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 1 --2- 172.21.15.83:0/2967978418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bba2220 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fece57fa640 0 client.25892 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 1 -- 172.21.15.83:0/2967978418 >> 172.21.15.83:0/2967978418 conn(0x55649bb937f0 msgr2=0x55649bb92a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 1 -- 172.21.15.83:0/2967978418 shutdown_connections 2024-03-11T19:45:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 1 -- 172.21.15.83:0/2967978418 wait complete. 2024-03-11T19:45:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.413+0000 7fed06c01740 20 client.25892 trim_cache size 0 max 16384 2024-03-11T19:45:04.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed06c01740 1 Processor -- start 2024-03-11T19:45:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed06c01740 1 -- start start 2024-03-11T19:45:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed06c01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bb99f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed06c01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb98cc0 0x55649bba9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed06c01740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 0x55649bb9a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed06c01740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649bbaca50 con 0x55649bb9de30 2024-03-11T19:45:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed06c01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649bbaac90 con 0x55649bb982d0 2024-03-11T19:45:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed06c01740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649bb8de20 con 0x55649bb98cc0 2024-03-11T19:45:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed024ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 0x55649bb9a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed024ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 0x55649bb9a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49252/0 (socket says 172.21.15.83:49252) 2024-03-11T19:45:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed024ec640 1 -- 172.21.15.83:0/1050412560 learned_addr learned my addr 172.21.15.83:0/1050412560 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:04.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.434+0000 7fed01ceb640 1 --2- 172.21.15.83:0/1050412560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bb99f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.435+0000 7fed014ea640 1 --2- 172.21.15.83:0/1050412560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb98cc0 0x55649bba9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.435+0000 7fed024ec640 1 -- 172.21.15.83:0/1050412560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb98cc0 msgr2=0x55649bba9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:04.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.435+0000 7fed024ec640 1 --2- 172.21.15.83:0/1050412560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb98cc0 0x55649bba9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.435+0000 7fed024ec640 1 -- 172.21.15.83:0/1050412560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 msgr2=0x55649bb99f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:04.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.435+0000 7fed024ec640 1 --2- 172.21.15.83:0/1050412560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bb99f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.435+0000 7fed024ec640 1 -- 172.21.15.83:0/1050412560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55649ba9bac0 con 0x55649bb9de30 2024-03-11T19:45:04.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.435+0000 7fed01ceb640 1 --2- 172.21.15.83:0/1050412560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bb99f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:04.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.435+0000 7fed024ec640 1 --2- 172.21.15.83:0/1050412560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 0x55649bb9a440 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009d00 tx=0x7fecf4007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b84291fd575ddd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:04.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.435+0000 7fece77fe640 1 -- 172.21.15.83:0/1050412560 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf400e040 con 0x55649bb9de30 2024-03-11T19:45:04.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.435+0000 7fece77fe640 1 -- 172.21.15.83:0/1050412560 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecf40070b0 con 0x55649bb9de30 2024-03-11T19:45:04.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.435+0000 7fece77fe640 1 -- 172.21.15.83:0/1050412560 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4025d30 con 0x55649bb9de30 2024-03-11T19:45:04.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.436+0000 7fed06c01740 1 -- 172.21.15.83:0/1050412560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 msgr2=0x55649bb9a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.436+0000 7fed06c01740 1 --2- 172.21.15.83:0/1050412560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 0x55649bb9a440 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009d00 tx=0x7fecf4007040 comp rx=0 tx=0).stop 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.436+0000 7fed06c01740 1 -- 172.21.15.83:0/1050412560 shutdown_connections 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.436+0000 7fed06c01740 1 --2- 172.21.15.83:0/1050412560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 0x55649bb9a440 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.436+0000 7fed06c01740 1 --2- 172.21.15.83:0/1050412560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb98cc0 0x55649bba9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.436+0000 7fed06c01740 1 --2- 172.21.15.83:0/1050412560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bb99f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.436+0000 7fed06c01740 1 -- 172.21.15.83:0/1050412560 >> 172.21.15.83:0/1050412560 conn(0x55649bb937f0 msgr2=0x55649bb93bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.436+0000 7fed06c01740 1 -- 172.21.15.83:0/1050412560 shutdown_connections 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.436+0000 7fed06c01740 1 -- 172.21.15.83:0/1050412560 wait complete. 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.437+0000 7fed06c01740 1 -- start start 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.437+0000 7fed06c01740 1 Processor -- start 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.438+0000 7fed06c01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bba8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.438+0000 7fed06c01740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb98cc0 0x55649bba8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.438+0000 7fed01ceb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bba8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.438+0000 7fed01ceb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bba8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47050/0 (socket says 172.21.15.83:47050) 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.438+0000 7fed01ceb640 1 -- 172.21.15.83:0/1159570484 learned_addr learned my addr 172.21.15.83:0/1159570484 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.438+0000 7fed014ea640 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb98cc0 0x55649bba8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.438+0000 7fed06c01740 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 0x55649bba8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.438+0000 7fed06c01740 1 -- 172.21.15.83:0/1159570484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649bc04960 con 0x55649bb9de30 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.438+0000 7fed06c01740 1 -- 172.21.15.83:0/1159570484 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649bc04aa0 con 0x55649bb982d0 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.438+0000 7fed06c01740 1 -- 172.21.15.83:0/1159570484 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649bc04c60 con 0x55649bb98cc0 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.438+0000 7fed06c01740 10 client.0 _ll_register_callbacks cb 0x7fecf4009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.439+0000 7fed024ec640 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 0x55649bba8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.439+0000 7fed01ceb640 1 -- 172.21.15.83:0/1159570484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb98cc0 msgr2=0x55649bba8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.439+0000 7fed01ceb640 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb98cc0 0x55649bba8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.439+0000 7fed01ceb640 1 -- 172.21.15.83:0/1159570484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 msgr2=0x55649bba8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.439+0000 7fed01ceb640 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 0x55649bba8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.439+0000 7fed01ceb640 1 -- 172.21.15.83:0/1159570484 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55649baff610 con 0x55649bb982d0 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.439+0000 7fed014ea640 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb98cc0 0x55649bba8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.440+0000 7fed01ceb640 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bba8010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009610 tx=0x7fecf80a2520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=658edba875b468a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.440+0000 7fece57fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.440+0000 7fece57fa640 1 -- 172.21.15.83:0/1159570484 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80b1040 con 0x55649bb982d0 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.440+0000 7fece57fa640 1 -- 172.21.15.83:0/1159570484 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fecf8005b70 con 0x55649bb982d0 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.440+0000 7fed06c01740 1 -- 172.21.15.83:0/1159570484 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55649b7ef4e0 con 0x55649bb982d0 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.440+0000 7fece57fa640 1 -- 172.21.15.83:0/1159570484 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecf8099940 con 0x55649bb982d0 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.440+0000 7fed06c01740 10 client.25898 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.440+0000 7fed06c01740 1 -- 172.21.15.83:0/1159570484 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55649babf570 con 0x55649bb982d0 2024-03-11T19:45:04.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.441+0000 7fece57fa640 1 -- 172.21.15.83:0/1159570484 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecf807d080 con 0x55649bb982d0 2024-03-11T19:45:04.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.441+0000 7fece57fa640 1 -- 172.21.15.83:0/1159570484 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fecf801a040 con 0x55649bb982d0 2024-03-11T19:45:04.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.441+0000 7fed06c01740 10 client.25898 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:04.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.441+0000 7fed06c01740 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:04.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.441+0000 7fed06c01740 20 client.25898 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:04.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.441+0000 7fed06c01740 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55649bc06460 0x55649bc08850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.441+0000 7fed06c01740 4 client.25898 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:45:04.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.441+0000 7fed06c01740 1 -- 172.21.15.83:0/1159570484 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55649b7ac700 con 0x55649bc06460 2024-03-11T19:45:04.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.442+0000 7fed014ea640 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55649bc06460 0x55649bc08850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.442+0000 7fed014ea640 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55649bc06460 0x55649bc08850 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ae4a0e6846d35d6 server_cookie=db217602f7131721 in_seq=0 out_seq=0 2024-03-11T19:45:04.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.442+0000 7fece57fa640 10 client.25898 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:04.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.442+0000 7fece57fa640 1 -- 172.21.15.83:0/1159570484 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55649b7ac700 con 0x55649bc06460 2024-03-11T19:45:04.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.442+0000 7fece57fa640 10 client.25898 handle_command_reply: tid=0 2024-03-11T19:45:04.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 1 client.25898 shutdown 2024-03-11T19:45:04.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 1 -- 172.21.15.83:0/1159570484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 msgr2=0x55649bba8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bba8010 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009610 tx=0x7fecf80a2520 comp rx=0 tx=0).stop 2024-03-11T19:45:04.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 1 -- 172.21.15.83:0/1159570484 shutdown_connections 2024-03-11T19:45:04.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55649bc06460 0x55649bc08850 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55649bb9de30 0x55649bba8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55649bb98cc0 0x55649bba8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 1 --2- 172.21.15.83:0/1159570484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55649bb982d0 0x55649bba8010 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fece57fa640 0 client.25898 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 1 -- 172.21.15.83:0/1159570484 >> 172.21.15.83:0/1159570484 conn(0x55649bb937f0 msgr2=0x55649bb99f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 1 -- 172.21.15.83:0/1159570484 shutdown_connections 2024-03-11T19:45:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 1 -- 172.21.15.83:0/1159570484 wait complete. 2024-03-11T19:45:04.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed06c01740 20 client.25898 trim_cache size 0 max 16384 2024-03-11T19:45:04.451 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:45:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:45:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:45:04.452 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:45:04.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed03f4f640 1 -- 172.21.15.83:0/3080915676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fece80a1bb0 msgr2=0x7fece80a4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed03f4f640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fece80a1bb0 0x7fece80a4070 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fecec032530 tx=0x7fecec034be0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed03f4f640 1 -- 172.21.15.83:0/3080915676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecfc100080 msgr2=0x7fecfc1961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.443+0000 7fed03f4f640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecfc100080 0x7fecfc1961d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fecfc1010d0 tx=0x7fecf800dbb0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.444+0000 7fed03f4f640 1 -- 172.21.15.83:0/3080915676 shutdown_connections 2024-03-11T19:45:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.444+0000 7fed03f4f640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fece80a1bb0 0x7fece80a4070 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.444+0000 7fed03f4f640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecfc101c30 0x7fecfc19d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.444+0000 7fed03f4f640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecfc101270 0x7fecfc196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.444+0000 7fed03f4f640 1 --2- 172.21.15.83:0/3080915676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecfc100080 0x7fecfc1961d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.444+0000 7fed03f4f640 1 -- 172.21.15.83:0/3080915676 >> 172.21.15.83:0/3080915676 conn(0x7fecfc0fb5f0 msgr2=0x7fecfc1045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.444+0000 7fed03f4f640 1 -- 172.21.15.83:0/3080915676 shutdown_connections 2024-03-11T19:45:04.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.444+0000 7fed03f4f640 1 -- 172.21.15.83:0/3080915676 wait complete. 2024-03-11T19:45:04.462 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:45:04.462 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:45:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.615+0000 7ff28e1c2640 1 -- 172.21.15.83:0/1081244318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff288101c50 msgr2=0x7ff2881084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.615+0000 7ff28e1c2640 1 --2- 172.21.15.83:0/1081244318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff288101c50 0x7ff2881084e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff280009ca0 tx=0x7ff28002dfe0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.615+0000 7ff28e1c2640 1 -- 172.21.15.83:0/1081244318 shutdown_connections 2024-03-11T19:45:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.615+0000 7ff28e1c2640 1 --2- 172.21.15.83:0/1081244318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff288101c50 0x7ff2881084e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.615+0000 7ff28e1c2640 1 --2- 172.21.15.83:0/1081244318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff288101290 0x7ff288101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.615+0000 7ff28e1c2640 1 --2- 172.21.15.83:0/1081244318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2881000a0 0x7ff2881004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.615+0000 7ff28e1c2640 1 -- 172.21.15.83:0/1081244318 >> 172.21.15.83:0/1081244318 conn(0x7ff2880fb610 msgr2=0x7ff2880fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.616+0000 7ff28e1c2640 1 -- 172.21.15.83:0/1081244318 shutdown_connections 2024-03-11T19:45:04.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.616+0000 7ff28e1c2640 1 -- 172.21.15.83:0/1081244318 wait complete. 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.616+0000 7ff28e1c2640 1 Processor -- start 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff28e1c2640 1 -- start start 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff28e1c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2881000a0 0x7ff288199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff28e1c2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288101290 0x7ff2881998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff28e1c2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff288101c50 0x7ff2881a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff28e1c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2881a0e80 con 0x7ff2881000a0 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff28e1c2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2881a0ff0 con 0x7ff288101290 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2881000a0 0x7ff288199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff28e1c2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2881a11b0 con 0x7ff288101c50 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2881000a0 0x7ff288199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49270/0 (socket says 172.21.15.83:49270) 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff2877fe640 1 -- 172.21.15.83:0/3689131544 learned_addr learned my addr 172.21.15.83:0/3689131544 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff286ffd640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288101290 0x7ff2881998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff287fff640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff288101c50 0x7ff2881a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff2877fe640 1 -- 172.21.15.83:0/3689131544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff288101c50 msgr2=0x7ff2881a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff2877fe640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff288101c50 0x7ff2881a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff2877fe640 1 -- 172.21.15.83:0/3689131544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288101290 msgr2=0x7ff2881998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff2877fe640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288101290 0x7ff2881998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff2877fe640 1 -- 172.21.15.83:0/3689131544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280009900 con 0x7ff2881000a0 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.617+0000 7ff286ffd640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288101290 0x7ff2881998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.618+0000 7ff2877fe640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2881000a0 0x7ff288199390 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff27800e9c0 tx=0x7ff27800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.618+0000 7ff284ff9640 1 -- 172.21.15.83:0/3689131544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278009700 con 0x7ff2881000a0 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.618+0000 7ff284ff9640 1 -- 172.21.15.83:0/3689131544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff27800cd80 con 0x7ff2881000a0 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.618+0000 7ff284ff9640 1 -- 172.21.15.83:0/3689131544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278010430 con 0x7ff2881000a0 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.619+0000 7ff28e1c2640 1 -- 172.21.15.83:0/3689131544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2881a1490 con 0x7ff2881000a0 2024-03-11T19:45:04.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.619+0000 7ff28e1c2640 1 -- 172.21.15.83:0/3689131544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2881a1950 con 0x7ff2881000a0 2024-03-11T19:45:04.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.620+0000 7ff28e1c2640 1 -- 172.21.15.83:0/3689131544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2740052c0 con 0x7ff2881000a0 2024-03-11T19:45:04.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.621+0000 7ff284ff9640 1 -- 172.21.15.83:0/3689131544 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff278004020 con 0x7ff2881000a0 2024-03-11T19:45:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.622+0000 7ff284ff9640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2700a1ab0 0x7ff2700a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:04.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.622+0000 7ff284ff9640 1 -- 172.21.15.83:0/3689131544 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff278014070 con 0x7ff2881000a0 2024-03-11T19:45:04.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.623+0000 7ff286ffd640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2700a1ab0 0x7ff2700a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:04.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.623+0000 7ff286ffd640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2700a1ab0 0x7ff2700a3f70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff28819a8a0 tx=0x7ff27c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:04.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.629+0000 7ff284ff9640 1 -- 172.21.15.83:0/3689131544 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff278096980 con 0x7ff2881000a0 2024-03-11T19:45:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.977+0000 7ff28e1c2640 1 -- 172.21.15.83:0/3689131544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff274005840 con 0x7ff2881000a0 2024-03-11T19:45:04.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.978+0000 7ff284ff9640 1 -- 172.21.15.83:0/3689131544 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+11350 (secure 0 0 0) 0x7ff27809a040 con 0x7ff2881000a0 2024-03-11T19:45:04.983 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:45:04.983 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:43:42.002107+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:45:04.983 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-03-11T19:45:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 -- 172.21.15.83:0/3689131544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2700a1ab0 msgr2=0x7ff2700a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2700a1ab0 0x7ff2700a3f70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff28819a8a0 tx=0x7ff27c00b390 comp rx=0 tx=0).stop 2024-03-11T19:45:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 -- 172.21.15.83:0/3689131544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2881000a0 msgr2=0x7ff288199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2881000a0 0x7ff288199390 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff27800e9c0 tx=0x7ff27800ee80 comp rx=0 tx=0).stop 2024-03-11T19:45:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 -- 172.21.15.83:0/3689131544 shutdown_connections 2024-03-11T19:45:04.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2700a1ab0 0x7ff2700a3f70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff288101c50 0x7ff2881a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff288101290 0x7ff2881998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 --2- 172.21.15.83:0/3689131544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2881000a0 0x7ff288199390 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 -- 172.21.15.83:0/3689131544 >> 172.21.15.83:0/3689131544 conn(0x7ff2880fb610 msgr2=0x7ff288104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 -- 172.21.15.83:0/3689131544 shutdown_connections 2024-03-11T19:45:04.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:04.982+0000 7ff28e1c2640 1 -- 172.21.15.83:0/3689131544 wait complete. 2024-03-11T19:45:05.001 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:04 smithi083 ceph-mon[31864]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.8 MiB/s wr, 775 op/s 2024-03-11T19:45:05.003 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:45:05.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:45:05.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:04 smithi137 ceph-mon[36220]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.8 MiB/s wr, 775 op/s 2024-03-11T19:45:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:04 smithi204 ceph-mon[36739]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.8 MiB/s wr, 775 op/s 2024-03-11T19:45:05.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.165+0000 7f0d92efb640 1 -- 172.21.15.83:0/2752256892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d8c108990 msgr2=0x7f0d8c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.165+0000 7f0d92efb640 1 --2- 172.21.15.83:0/2752256892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d8c108990 0x7f0d8c108e10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009ca0 tx=0x7f0d8402c140 comp rx=0 tx=0).stop 2024-03-11T19:45:05.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.167+0000 7f0d92efb640 1 -- 172.21.15.83:0/2752256892 shutdown_connections 2024-03-11T19:45:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.167+0000 7f0d92efb640 1 --2- 172.21.15.83:0/2752256892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8c109350 0x7f0d8c10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.167+0000 7f0d92efb640 1 --2- 172.21.15.83:0/2752256892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d8c108990 0x7f0d8c108e10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.167+0000 7f0d92efb640 1 --2- 172.21.15.83:0/2752256892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8c1077a0 0x7f0d8c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.167+0000 7f0d92efb640 1 -- 172.21.15.83:0/2752256892 >> 172.21.15.83:0/2752256892 conn(0x7f0d8c076740 msgr2=0x7f0d8c078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.171+0000 7f0d92efb640 1 -- 172.21.15.83:0/2752256892 shutdown_connections 2024-03-11T19:45:05.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.172+0000 7f0d92efb640 1 -- 172.21.15.83:0/2752256892 wait complete. 2024-03-11T19:45:05.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.173+0000 7f0d92efb640 1 Processor -- start 2024-03-11T19:45:05.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.173+0000 7f0d92efb640 1 -- start start 2024-03-11T19:45:05.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.173+0000 7f0d92efb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d8c1077a0 0x7f0d8c081400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.173+0000 7f0d92efb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8c108990 0x7f0d8c081940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.173+0000 7f0d92efb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8c109350 0x7f0d8c07fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.174+0000 7f0d8bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8c108990 0x7f0d8c081940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.174+0000 7f0d91498640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8c109350 0x7f0d8c07fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.174+0000 7f0d91498640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8c109350 0x7f0d8c07fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49286/0 (socket says 172.21.15.83:49286) 2024-03-11T19:45:05.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.174+0000 7f0d91498640 1 -- 172.21.15.83:0/1188579833 learned_addr learned my addr 172.21.15.83:0/1188579833 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.174+0000 7f0d92efb640 1 -- 172.21.15.83:0/1188579833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c07ffb0 con 0x7f0d8c109350 2024-03-11T19:45:05.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.174+0000 7f0d92efb640 1 -- 172.21.15.83:0/1188579833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c080120 con 0x7f0d8c1077a0 2024-03-11T19:45:05.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.174+0000 7f0d92efb640 1 -- 172.21.15.83:0/1188579833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c0802e0 con 0x7f0d8c108990 2024-03-11T19:45:05.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.176+0000 7f0d90c97640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d8c1077a0 0x7f0d8c081400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.176+0000 7f0d8bfff640 1 -- 172.21.15.83:0/1188579833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d8c1077a0 msgr2=0x7f0d8c081400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.176+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d8c1077a0 0x7f0d8c081400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.176+0000 7f0d8bfff640 1 -- 172.21.15.83:0/1188579833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8c109350 msgr2=0x7f0d8c07fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.176+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8c109350 0x7f0d8c07fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.176+0000 7f0d8bfff640 1 -- 172.21.15.83:0/1188579833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d84009900 con 0x7f0d8c108990 2024-03-11T19:45:05.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.179+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8c108990 0x7f0d8c081940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009bf0 tx=0x7f0d84032a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.180+0000 7f0d89ffb640 1 -- 172.21.15.83:0/1188579833 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84034850 con 0x7f0d8c108990 2024-03-11T19:45:05.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.180+0000 7f0d89ffb640 1 -- 172.21.15.83:0/1188579833 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d84032b80 con 0x7f0d8c108990 2024-03-11T19:45:05.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.180+0000 7f0d92efb640 1 -- 172.21.15.83:0/1188579833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8c080560 con 0x7f0d8c108990 2024-03-11T19:45:05.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.180+0000 7f0d92efb640 1 -- 172.21.15.83:0/1188579833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8c1ae8d0 con 0x7f0d8c108990 2024-03-11T19:45:05.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.181+0000 7f0d89ffb640 1 -- 172.21.15.83:0/1188579833 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8403f440 con 0x7f0d8c108990 2024-03-11T19:45:05.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.184+0000 7f0d89ffb640 1 -- 172.21.15.83:0/1188579833 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0d8403b040 con 0x7f0d8c108990 2024-03-11T19:45:05.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.185+0000 7f0d89ffb640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d740a1b80 0x7f0d740a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.186+0000 7f0d89ffb640 1 -- 172.21.15.83:0/1188579833 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0d840efe30 con 0x7f0d8c108990 2024-03-11T19:45:05.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.186+0000 7f0d90c97640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d740a1b80 0x7f0d740a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.187+0000 7f0d90c97640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d740a1b80 0x7f0d740a4040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0d80005fd0 tx=0x7f0d80005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.189+0000 7f0d95bab740 1 Processor -- start 2024-03-11T19:45:05.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.189+0000 7f0d95bab740 1 -- start start 2024-03-11T19:45:05.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.189+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abb270 0x55fc75abb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.189+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abbc60 0x55fc75ac0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.189+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75ac0dd0 0x55fc75ac3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.189+0000 7f0d95bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc75a77960 con 0x55fc75abb270 2024-03-11T19:45:05.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.189+0000 7f0d95bab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc75a2b310 con 0x55fc75abbc60 2024-03-11T19:45:05.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.189+0000 7f0d95bab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc75a28840 con 0x55fc75ac0dd0 2024-03-11T19:45:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d91498640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75ac0dd0 0x55fc75ac3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d8bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abbc60 0x55fc75ac0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d8bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abbc60 0x55fc75ac0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47086/0 (socket says 172.21.15.83:47086) 2024-03-11T19:45:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d91498640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75ac0dd0 0x55fc75ac3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:44946/0 (socket says 172.21.15.83:44946) 2024-03-11T19:45:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d8bfff640 1 -- 172.21.15.83:0/2813259580 learned_addr learned my addr 172.21.15.83:0/2813259580 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d90c97640 1 --2- 172.21.15.83:0/2813259580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abb270 0x55fc75abb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d8bfff640 1 -- 172.21.15.83:0/2813259580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75ac0dd0 msgr2=0x55fc75ac3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/2813259580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75ac0dd0 0x55fc75ac3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d8bfff640 1 -- 172.21.15.83:0/2813259580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abb270 msgr2=0x55fc75abb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/2813259580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abb270 0x55fc75abb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d8bfff640 1 -- 172.21.15.83:0/2813259580 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc759e6060 con 0x55fc75abbc60 2024-03-11T19:45:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d91498640 1 --2- 172.21.15.83:0/2813259580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75ac0dd0 0x55fc75ac3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d90c97640 1 --2- 172.21.15.83:0/2813259580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abb270 0x55fc75abb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.190+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/2813259580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abbc60 0x55fc75ac0760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d840b9210 tx=0x7f0d8402c140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58accd620f0ff32 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.191+0000 7f0d72ffd640 1 -- 172.21.15.83:0/2813259580 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d840bcdb0 con 0x55fc75abbc60 2024-03-11T19:45:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.191+0000 7f0d72ffd640 1 -- 172.21.15.83:0/2813259580 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d840c0f00 con 0x55fc75abbc60 2024-03-11T19:45:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.191+0000 7f0d72ffd640 1 -- 172.21.15.83:0/2813259580 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d840bd010 con 0x55fc75abbc60 2024-03-11T19:45:05.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.192+0000 7f0d95bab740 1 -- 172.21.15.83:0/2813259580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abbc60 msgr2=0x55fc75ac0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.192+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2813259580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abbc60 0x55fc75ac0760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d840b9210 tx=0x7f0d8402c140 comp rx=0 tx=0).stop 2024-03-11T19:45:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.192+0000 7f0d95bab740 1 -- 172.21.15.83:0/2813259580 shutdown_connections 2024-03-11T19:45:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.192+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2813259580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75ac0dd0 0x55fc75ac3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.192+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2813259580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abbc60 0x55fc75ac0760 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.192+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2813259580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abb270 0x55fc75abb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.192+0000 7f0d95bab740 1 -- 172.21.15.83:0/2813259580 >> 172.21.15.83:0/2813259580 conn(0x55fc75ab67e0 msgr2=0x55fc75ab6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.193+0000 7f0d95bab740 1 -- 172.21.15.83:0/2813259580 shutdown_connections 2024-03-11T19:45:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.193+0000 7f0d95bab740 1 -- 172.21.15.83:0/2813259580 wait complete. 2024-03-11T19:45:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.194+0000 7f0d95bab740 1 -- start start 2024-03-11T19:45:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.194+0000 7f0d95bab740 1 Processor -- start 2024-03-11T19:45:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.194+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abb270 0x55fc75ac51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.194+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abbc60 0x55fc75ac5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.194+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75acc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.194+0000 7f0d95bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc75a24020 con 0x55fc75ac0dd0 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.194+0000 7f0d95bab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc756f7430 con 0x55fc75abb270 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.194+0000 7f0d95bab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc756b3e00 con 0x55fc75abbc60 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.194+0000 7f0d95bab740 10 client.0 _ll_register_callbacks cb 0x55fc759cbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d91498640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75acc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d91498640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75acc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49300/0 (socket says 172.21.15.83:49300) 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d91498640 1 -- 172.21.15.83:0/3931037379 learned_addr learned my addr 172.21.15.83:0/3931037379 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d90c97640 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abb270 0x55fc75ac51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abbc60 0x55fc75ac5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d90c97640 1 -- 172.21.15.83:0/3931037379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abbc60 msgr2=0x55fc75ac5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d90c97640 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abbc60 0x55fc75ac5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d90c97640 1 -- 172.21.15.83:0/3931037379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 msgr2=0x55fc75acc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d90c97640 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75acc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d90c97640 1 -- 172.21.15.83:0/3931037379 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8c1ae8d0 con 0x55fc75abb270 2024-03-11T19:45:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d91498640 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75acc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d90c97640 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abb270 0x55fc75ac51c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55fc75abc4c0 tx=0x7f0d800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc42a6c3a69da105 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.195+0000 7f0d6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:45:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.196+0000 7f0d6bfff640 1 -- 172.21.15.83:0/3931037379 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80010920 con 0x55fc75abb270 2024-03-11T19:45:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.196+0000 7f0d95bab740 1 -- 172.21.15.83:0/3931037379 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc757237e0 con 0x55fc75abb270 2024-03-11T19:45:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.196+0000 7f0d6bfff640 1 -- 172.21.15.83:0/3931037379 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0d80010a80 con 0x55fc75abb270 2024-03-11T19:45:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.196+0000 7f0d6bfff640 1 -- 172.21.15.83:0/3931037379 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d80010c40 con 0x55fc75abb270 2024-03-11T19:45:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.196+0000 7f0d95bab740 10 client.25913 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.196+0000 7f0d95bab740 1 -- 172.21.15.83:0/3931037379 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fc75a226a0 con 0x55fc75abb270 2024-03-11T19:45:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.196+0000 7f0d6bfff640 1 -- 172.21.15.83:0/3931037379 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fc757237e0 con 0x55fc75abb270 2024-03-11T19:45:05.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.197+0000 7f0d6bfff640 1 -- 172.21.15.83:0/3931037379 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0d80025020 con 0x55fc75abb270 2024-03-11T19:45:05.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.197+0000 7f0d95bab740 10 client.25913 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:05.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.197+0000 7f0d95bab740 10 client.25913 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:45:05.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.197+0000 7f0d95bab740 20 client.25913 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:05.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.197+0000 7f0d95bab740 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fc75ad13d0 0x55fc75ad37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.197+0000 7f0d95bab740 4 client.25913 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.197+0000 7f0d95bab740 1 -- 172.21.15.83:0/3931037379 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc75760720 con 0x55fc75ad13d0 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.197+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fc75ad13d0 0x55fc75ad37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.198+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fc75ad13d0 0x55fc75ad37f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef8e73ad2462bede server_cookie=981e14195cfd4373 in_seq=0 out_seq=0 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.198+0000 7f0d6bfff640 10 client.25913 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.199+0000 7f0d6bfff640 1 -- 172.21.15.83:0/3931037379 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fc75760720 con 0x55fc75ad13d0 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.199+0000 7f0d6bfff640 10 client.25913 handle_command_reply: tid=0 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.199+0000 7f0d95bab740 1 client.25913 shutdown 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.200+0000 7f0d95bab740 1 -- 172.21.15.83:0/3931037379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abb270 msgr2=0x55fc75ac51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.200+0000 7f0d95bab740 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abb270 0x55fc75ac51c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55fc75abc4c0 tx=0x7f0d800046c0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.200+0000 7f0d95bab740 1 -- 172.21.15.83:0/3931037379 shutdown_connections 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.200+0000 7f0d95bab740 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fc75ad13d0 0x55fc75ad37f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.200+0000 7f0d95bab740 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75acc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.200+0000 7f0d6bfff640 0 client.25913 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:45:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.200+0000 7f0d95bab740 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abbc60 0x55fc75ac5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.200+0000 7f0d95bab740 1 --2- 172.21.15.83:0/3931037379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abb270 0x55fc75ac51c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.200+0000 7f0d95bab740 1 -- 172.21.15.83:0/3931037379 >> 172.21.15.83:0/3931037379 conn(0x55fc75ab67e0 msgr2=0x55fc75ab5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.200+0000 7f0d95bab740 1 -- 172.21.15.83:0/3931037379 shutdown_connections 2024-03-11T19:45:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.201+0000 7f0d95bab740 1 -- 172.21.15.83:0/3931037379 wait complete. 2024-03-11T19:45:05.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.201+0000 7f0d95bab740 20 client.25913 trim_cache size 0 max 16384 2024-03-11T19:45:05.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d95bab740 1 Processor -- start 2024-03-11T19:45:05.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d95bab740 1 -- start start 2024-03-11T19:45:05.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abb270 0x55fc75abd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abbc60 0x55fc75acc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75abd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d95bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc759a9130 con 0x55fc75ac0dd0 2024-03-11T19:45:05.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d95bab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc757237e0 con 0x55fc75abb270 2024-03-11T19:45:05.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d95bab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc75aa8a40 con 0x55fc75abbc60 2024-03-11T19:45:05.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d91498640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75abd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d90c97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abb270 0x55fc75abd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d91498640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75abd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49306/0 (socket says 172.21.15.83:49306) 2024-03-11T19:45:05.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.217+0000 7f0d91498640 1 -- 172.21.15.83:0/2535314022 learned_addr learned my addr 172.21.15.83:0/2535314022 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.218+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/2535314022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abbc60 0x55fc75acc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.218+0000 7f0d91498640 1 -- 172.21.15.83:0/2535314022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abbc60 msgr2=0x55fc75acc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.218+0000 7f0d91498640 1 --2- 172.21.15.83:0/2535314022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abbc60 0x55fc75acc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.218+0000 7f0d91498640 1 -- 172.21.15.83:0/2535314022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abb270 msgr2=0x55fc75abd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.218+0000 7f0d91498640 1 --2- 172.21.15.83:0/2535314022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abb270 0x55fc75abd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.218+0000 7f0d91498640 1 -- 172.21.15.83:0/2535314022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc759bec20 con 0x55fc75ac0dd0 2024-03-11T19:45:05.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.218+0000 7f0d91498640 1 --2- 172.21.15.83:0/2535314022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75abd5c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00b700 tx=0x7f0d7c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf7b0887ef243b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.218+0000 7f0d727fc640 1 -- 172.21.15.83:0/2535314022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c012070 con 0x55fc75ac0dd0 2024-03-11T19:45:05.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.218+0000 7f0d727fc640 1 -- 172.21.15.83:0/2535314022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d7c013040 con 0x55fc75ac0dd0 2024-03-11T19:45:05.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.219+0000 7f0d727fc640 1 -- 172.21.15.83:0/2535314022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c016760 con 0x55fc75ac0dd0 2024-03-11T19:45:05.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.219+0000 7f0d95bab740 1 -- 172.21.15.83:0/2535314022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 msgr2=0x55fc75abd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.219+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2535314022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75abd5c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00b700 tx=0x7f0d7c008040 comp rx=0 tx=0).stop 2024-03-11T19:45:05.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.219+0000 7f0d95bab740 1 -- 172.21.15.83:0/2535314022 shutdown_connections 2024-03-11T19:45:05.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.219+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2535314022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75ac0dd0 0x55fc75abd5c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.219+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2535314022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abbc60 0x55fc75acc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.219+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2535314022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75abb270 0x55fc75abd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.219+0000 7f0d95bab740 1 -- 172.21.15.83:0/2535314022 >> 172.21.15.83:0/2535314022 conn(0x55fc75ab67e0 msgr2=0x55fc75ab1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.219+0000 7f0d95bab740 1 -- 172.21.15.83:0/2535314022 shutdown_connections 2024-03-11T19:45:05.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.219+0000 7f0d95bab740 1 -- 172.21.15.83:0/2535314022 wait complete. 2024-03-11T19:45:05.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.220+0000 7f0d95bab740 1 -- start start 2024-03-11T19:45:05.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.220+0000 7f0d95bab740 1 Processor -- start 2024-03-11T19:45:05.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.220+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abb270 0x55fc75acc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.220+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abbc60 0x55fc75acaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.220+0000 7f0d95bab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75ac0dd0 0x55fc75acafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.220+0000 7f0d95bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc75acb500 con 0x55fc75abbc60 2024-03-11T19:45:05.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.220+0000 7f0d95bab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc75acb670 con 0x55fc75ac0dd0 2024-03-11T19:45:05.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.220+0000 7f0d95bab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc75acb830 con 0x55fc75abb270 2024-03-11T19:45:05.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.220+0000 7f0d95bab740 10 client.0 _ll_register_callbacks cb 0x7f0d8000ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:05.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.221+0000 7f0d8bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abbc60 0x55fc75acaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.221+0000 7f0d8bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abbc60 0x55fc75acaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49310/0 (socket says 172.21.15.83:49310) 2024-03-11T19:45:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.221+0000 7f0d8bfff640 1 -- 172.21.15.83:0/2463672732 learned_addr learned my addr 172.21.15.83:0/2463672732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.221+0000 7f0d90c97640 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abb270 0x55fc75acc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.221+0000 7f0d91498640 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75ac0dd0 0x55fc75acafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.221+0000 7f0d90c97640 1 -- 172.21.15.83:0/2463672732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75ac0dd0 msgr2=0x55fc75acafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.221+0000 7f0d90c97640 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75ac0dd0 0x55fc75acafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.221+0000 7f0d90c97640 1 -- 172.21.15.83:0/2463672732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abbc60 msgr2=0x55fc75acaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.221+0000 7f0d90c97640 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abbc60 0x55fc75acaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.221+0000 7f0d90c97640 1 -- 172.21.15.83:0/2463672732 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c00e040 con 0x55fc75abb270 2024-03-11T19:45:05.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.222+0000 7f0d90c97640 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abb270 0x55fc75acc3d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0d800098c0 tx=0x7f0d80011470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59034eac6c386544 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.222+0000 7f0d6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:45:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.222+0000 7f0d6bfff640 1 -- 172.21.15.83:0/2463672732 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80005040 con 0x55fc75abb270 2024-03-11T19:45:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.222+0000 7f0d6bfff640 1 -- 172.21.15.83:0/2463672732 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0d8003b070 con 0x55fc75abb270 2024-03-11T19:45:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.222+0000 7f0d95bab740 1 -- 172.21.15.83:0/2463672732 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d840c0f00 con 0x55fc75abb270 2024-03-11T19:45:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.222+0000 7f0d6bfff640 1 -- 172.21.15.83:0/2463672732 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d80027a80 con 0x55fc75abb270 2024-03-11T19:45:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.222+0000 7f0d95bab740 10 client.26011 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.222+0000 7f0d95bab740 1 -- 172.21.15.83:0/2463672732 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fc759e48c0 con 0x55fc75abb270 2024-03-11T19:45:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.222+0000 7f0d6bfff640 1 -- 172.21.15.83:0/2463672732 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d840c0f00 con 0x55fc75abb270 2024-03-11T19:45:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d6bfff640 1 -- 172.21.15.83:0/2463672732 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f0d80048070 con 0x55fc75abb270 2024-03-11T19:45:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d95bab740 10 client.26011 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:05.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d95bab740 10 client.26011 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:45:05.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d95bab740 20 client.26011 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:05.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fc75ad13d0 0x55fc75b2af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d95bab740 4 client.26011 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:45:05.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d95bab740 1 -- 172.21.15.83:0/2463672732 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fc757af890 con 0x55fc75ad13d0 2024-03-11T19:45:05.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fc75ad13d0 0x55fc75b2af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d6bfff640 10 client.26011 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:45:05.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d8bfff640 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fc75ad13d0 0x55fc75b2af90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=267361e4a872e44a server_cookie=de3f01fc7c3e3cf2 in_seq=0 out_seq=0 2024-03-11T19:45:05.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d6bfff640 1 -- 172.21.15.83:0/2463672732 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fc757af890 con 0x55fc75ad13d0 2024-03-11T19:45:05.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d6bfff640 10 client.26011 handle_command_reply: tid=0 2024-03-11T19:45:05.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.223+0000 7f0d95bab740 1 client.26011 shutdown 2024-03-11T19:45:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d95bab740 1 -- 172.21.15.83:0/2463672732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abb270 msgr2=0x55fc75acc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abb270 0x55fc75acc3d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0d800098c0 tx=0x7f0d80011470 comp rx=0 tx=0).stop 2024-03-11T19:45:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d95bab740 1 -- 172.21.15.83:0/2463672732 shutdown_connections 2024-03-11T19:45:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fc75ad13d0 0x55fc75b2af90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc75ac0dd0 0x55fc75acafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fc75abbc60 0x55fc75acaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d95bab740 1 --2- 172.21.15.83:0/2463672732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc75abb270 0x55fc75acc3d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d6bfff640 0 client.26011 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:45:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d95bab740 1 -- 172.21.15.83:0/2463672732 >> 172.21.15.83:0/2463672732 conn(0x55fc75ab67e0 msgr2=0x55fc75abd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d95bab740 1 -- 172.21.15.83:0/2463672732 shutdown_connections 2024-03-11T19:45:05.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d95bab740 1 -- 172.21.15.83:0/2463672732 wait complete. 2024-03-11T19:45:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.224+0000 7f0d95bab740 20 client.26011 trim_cache size 0 max 16384 2024-03-11T19:45:05.235 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:45:05.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.225+0000 7f0d92efb640 1 -- 172.21.15.83:0/1188579833 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d740a1b80 msgr2=0x7f0d740a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.225+0000 7f0d92efb640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d740a1b80 0x7f0d740a4040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0d80005fd0 tx=0x7f0d80005850 comp rx=0 tx=0).stop 2024-03-11T19:45:05.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.225+0000 7f0d92efb640 1 -- 172.21.15.83:0/1188579833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8c108990 msgr2=0x7f0d8c081940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.225+0000 7f0d92efb640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8c108990 0x7f0d8c081940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0d84009bf0 tx=0x7f0d84032a10 comp rx=0 tx=0).stop 2024-03-11T19:45:05.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.225+0000 7f0d92efb640 1 -- 172.21.15.83:0/1188579833 shutdown_connections 2024-03-11T19:45:05.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.225+0000 7f0d92efb640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d740a1b80 0x7f0d740a4040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.225+0000 7f0d92efb640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8c109350 0x7f0d8c07fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.225+0000 7f0d92efb640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8c108990 0x7f0d8c081940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.225+0000 7f0d92efb640 1 --2- 172.21.15.83:0/1188579833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d8c1077a0 0x7f0d8c081400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.225+0000 7f0d92efb640 1 -- 172.21.15.83:0/1188579833 >> 172.21.15.83:0/1188579833 conn(0x7f0d8c076740 msgr2=0x7f0d8c10bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.225+0000 7f0d92efb640 1 -- 172.21.15.83:0/1188579833 shutdown_connections 2024-03-11T19:45:05.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.226+0000 7f0d92efb640 1 -- 172.21.15.83:0/1188579833 wait complete. 2024-03-11T19:45:05.244 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:45:05.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:45:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.391+0000 7f17ede76640 1 -- 172.21.15.83:0/3476316505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17e8102d40 msgr2=0x7f17e81095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.391+0000 7f17ede76640 1 --2- 172.21.15.83:0/3476316505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17e8102d40 0x7f17e81095d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009ca0 tx=0x7f17e002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.391+0000 7f17ede76640 1 -- 172.21.15.83:0/3476316505 shutdown_connections 2024-03-11T19:45:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.391+0000 7f17ede76640 1 --2- 172.21.15.83:0/3476316505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17e8102d40 0x7f17e81095d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.391+0000 7f17ede76640 1 --2- 172.21.15.83:0/3476316505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8102380 0x7f17e8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.391+0000 7f17ede76640 1 --2- 172.21.15.83:0/3476316505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17e8101190 0x7f17e81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.391+0000 7f17ede76640 1 -- 172.21.15.83:0/3476316505 >> 172.21.15.83:0/3476316505 conn(0x7f17e80fc700 msgr2=0x7f17e80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.391+0000 7f17ede76640 1 -- 172.21.15.83:0/3476316505 shutdown_connections 2024-03-11T19:45:05.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.391+0000 7f17ede76640 1 -- 172.21.15.83:0/3476316505 wait complete. 2024-03-11T19:45:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.392+0000 7f17ede76640 1 Processor -- start 2024-03-11T19:45:05.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.392+0000 7f17ede76640 1 -- start start 2024-03-11T19:45:05.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.392+0000 7f17ede76640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17e8101190 0x7f17e8074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.392+0000 7f17ede76640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8102380 0x7f17e8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17e6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8102380 0x7f17e8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17e6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8102380 0x7f17e8073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47132/0 (socket says 172.21.15.83:47132) 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17e6ffd640 1 -- 172.21.15.83:0/3222896852 learned_addr learned my addr 172.21.15.83:0/3222896852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17e77fe640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17e8101190 0x7f17e8074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17ede76640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17e8102d40 0x7f17e8073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17ede76640 1 -- 172.21.15.83:0/3222896852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e819b050 con 0x7f17e8102d40 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17ede76640 1 -- 172.21.15.83:0/3222896852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e8073ff0 con 0x7f17e8102380 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17ede76640 1 -- 172.21.15.83:0/3222896852 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e80741b0 con 0x7f17e8101190 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17e6ffd640 1 -- 172.21.15.83:0/3222896852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17e8101190 msgr2=0x7f17e8074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17e6ffd640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17e8101190 0x7f17e8074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17e6ffd640 1 -- 172.21.15.83:0/3222896852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17e8102d40 msgr2=0x7f17e8073ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17e6ffd640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17e8102d40 0x7f17e8073ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.393+0000 7f17e6ffd640 1 -- 172.21.15.83:0/3222896852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e0009900 con 0x7f17e8102380 2024-03-11T19:45:05.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.394+0000 7f17e6ffd640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8102380 0x7f17e8073570 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00e9f0 tx=0x7f17dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.394+0000 7f17e4ff9640 1 -- 172.21.15.83:0/3222896852 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc009700 con 0x7f17e8102380 2024-03-11T19:45:05.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.394+0000 7f17e4ff9640 1 -- 172.21.15.83:0/3222896852 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17dc00cd80 con 0x7f17e8102380 2024-03-11T19:45:05.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.394+0000 7f17ede76640 1 -- 172.21.15.83:0/3222896852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e81a5a40 con 0x7f17e8102380 2024-03-11T19:45:05.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.394+0000 7f17e4ff9640 1 -- 172.21.15.83:0/3222896852 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc018920 con 0x7f17e8102380 2024-03-11T19:45:05.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.394+0000 7f17ede76640 1 -- 172.21.15.83:0/3222896852 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e81a5db0 con 0x7f17e8102380 2024-03-11T19:45:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17f0a49740 1 Processor -- start 2024-03-11T19:45:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17f0a49740 1 -- start start 2024-03-11T19:45:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17e4ff9640 1 -- 172.21.15.83:0/3222896852 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f17dc018a80 con 0x7f17e8102380 2024-03-11T19:45:05.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 0x559653ac2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 0x559653ac7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653aca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17f0a49740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559653a7e960 con 0x559653ac7dd0 2024-03-11T19:45:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17f0a49740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559653a32310 con 0x559653ac2270 2024-03-11T19:45:05.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17f0a49740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559653a2f840 con 0x559653ac2c60 2024-03-11T19:45:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17e77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 0x559653ac7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17e77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 0x559653ac7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45006/0 (socket says 172.21.15.83:45006) 2024-03-11T19:45:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17e77fe640 1 -- 172.21.15.83:0/4055229655 learned_addr learned my addr 172.21.15.83:0/4055229655 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.397+0000 7f17e7fff640 1 --2- 172.21.15.83:0/4055229655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 0x559653ac2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.398+0000 7f17e6ffd640 1 --2- 172.21.15.83:0/4055229655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653aca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.398+0000 7f17e7fff640 1 -- 172.21.15.83:0/4055229655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 msgr2=0x559653ac7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.398+0000 7f17e7fff640 1 --2- 172.21.15.83:0/4055229655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 0x559653ac7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.398+0000 7f17e7fff640 1 -- 172.21.15.83:0/4055229655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 msgr2=0x559653aca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.398+0000 7f17e7fff640 1 --2- 172.21.15.83:0/4055229655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653aca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.398+0000 7f17e7fff640 1 -- 172.21.15.83:0/4055229655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596539ed060 con 0x559653ac2270 2024-03-11T19:45:05.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.398+0000 7f17e77fe640 1 --2- 172.21.15.83:0/4055229655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 0x559653ac7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:05.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.398+0000 7f17e7fff640 1 --2- 172.21.15.83:0/4055229655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 0x559653ac2690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f17e0037040 tx=0x7f17e0009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4681685099feb66 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.399+0000 7f17e4ff9640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17d00a1b30 0x7f17d00a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.399+0000 7f17e77fe640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17d00a1b30 0x7f17d00a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.400+0000 7f17e4ff9640 1 -- 172.21.15.83:0/3222896852 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f17dc014070 con 0x7f17e8102380 2024-03-11T19:45:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.400+0000 7f17cdffb640 1 -- 172.21.15.83:0/4055229655 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0032d70 con 0x559653ac2270 2024-03-11T19:45:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.400+0000 7f17e77fe640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17d00a1b30 0x7f17d00a3ff0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x559653ac3060 tx=0x7f17d800b5a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.400+0000 7f17cdffb640 1 -- 172.21.15.83:0/4055229655 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17e00327b0 con 0x559653ac2270 2024-03-11T19:45:05.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.400+0000 7f17cdffb640 1 -- 172.21.15.83:0/4055229655 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0003bd0 con 0x559653ac2270 2024-03-11T19:45:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.401+0000 7f17f0a49740 1 -- 172.21.15.83:0/4055229655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 msgr2=0x559653ac2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.401+0000 7f17f0a49740 1 --2- 172.21.15.83:0/4055229655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 0x559653ac2690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f17e0037040 tx=0x7f17e0009380 comp rx=0 tx=0).stop 2024-03-11T19:45:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.401+0000 7f17f0a49740 1 -- 172.21.15.83:0/4055229655 shutdown_connections 2024-03-11T19:45:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.401+0000 7f17f0a49740 1 --2- 172.21.15.83:0/4055229655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653aca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.401+0000 7f17f0a49740 1 --2- 172.21.15.83:0/4055229655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 0x559653ac7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.401+0000 7f17f0a49740 1 --2- 172.21.15.83:0/4055229655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 0x559653ac2690 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.401+0000 7f17f0a49740 1 -- 172.21.15.83:0/4055229655 >> 172.21.15.83:0/4055229655 conn(0x559653abd7e0 msgr2=0x559653abdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.401+0000 7f17f0a49740 1 -- 172.21.15.83:0/4055229655 shutdown_connections 2024-03-11T19:45:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.401+0000 7f17f0a49740 1 -- 172.21.15.83:0/4055229655 wait complete. 2024-03-11T19:45:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.401+0000 7f17f0a49740 1 -- start start 2024-03-11T19:45:05.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.402+0000 7f17f0a49740 1 Processor -- start 2024-03-11T19:45:05.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.403+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2270 0x559653add4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.403+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 0x559653adda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.403+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653ae4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.403+0000 7f17e77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2270 0x559653add4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.403+0000 7f17e77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2270 0x559653add4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45012/0 (socket says 172.21.15.83:45012) 2024-03-11T19:45:05.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.403+0000 7f17e6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 0x559653adda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.403+0000 7f17e6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 0x559653adda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47156/0 (socket says 172.21.15.83:47156) 2024-03-11T19:45:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.403+0000 7f17e77fe640 1 -- 172.21.15.83:0/3762157671 learned_addr learned my addr 172.21.15.83:0/3762157671 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17f0a49740 1 -- 172.21.15.83:0/3762157671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559653a2b020 con 0x559653ac7dd0 2024-03-11T19:45:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17f0a49740 1 -- 172.21.15.83:0/3762157671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596536fe430 con 0x559653ac2c60 2024-03-11T19:45:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17f0a49740 1 -- 172.21.15.83:0/3762157671 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596536bae00 con 0x559653ac2270 2024-03-11T19:45:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17e7fff640 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653ae4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17f0a49740 10 client.0 _ll_register_callbacks cb 0x5596539d2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17e7fff640 1 -- 172.21.15.83:0/3762157671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2270 msgr2=0x559653add4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17e7fff640 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2270 0x559653add4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17e7fff640 1 -- 172.21.15.83:0/3762157671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 msgr2=0x559653adda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17e7fff640 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 0x559653adda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17e7fff640 1 -- 172.21.15.83:0/3762157671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e8109ed0 con 0x559653ac7dd0 2024-03-11T19:45:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17e6ffd640 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 0x559653adda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17e7fff640 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653ae4a70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009ca0 tx=0x7f17e0034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec06952d9844c30 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17f0a49740 1 -- 172.21.15.83:0/3762157671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55965372a7e0 con 0x559653ac7dd0 2024-03-11T19:45:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.404+0000 7f17c6ffd640 10 client.16458 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:45:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17c6ffd640 1 -- 172.21.15.83:0/3762157671 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0034640 con 0x559653ac7dd0 2024-03-11T19:45:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17c6ffd640 1 -- 172.21.15.83:0/3762157671 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f17e0003970 con 0x559653ac7dd0 2024-03-11T19:45:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17c6ffd640 1 -- 172.21.15.83:0/3762157671 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17e0003b30 con 0x559653ac7dd0 2024-03-11T19:45:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17f0a49740 10 client.16458 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17f0a49740 1 -- 172.21.15.83:0/3762157671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x559653a296a0 con 0x559653ac7dd0 2024-03-11T19:45:05.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17c6ffd640 1 -- 172.21.15.83:0/3762157671 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f17e0003c90 con 0x559653ac7dd0 2024-03-11T19:45:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17f0a49740 10 client.16458 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17f0a49740 10 client.16458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:45:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17f0a49740 20 client.16458 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17f0a49740 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559653ae9020 0x559653aeb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17f0a49740 4 client.16458 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17f0a49740 1 -- 172.21.15.83:0/3762157671 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559653767720 con 0x559653ae9020 2024-03-11T19:45:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.405+0000 7f17e77fe640 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559653ae9020 0x559653aeb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.406+0000 7f17c6ffd640 10 client.16458 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:45:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.406+0000 7f17e77fe640 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559653ae9020 0x559653aeb410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b27d334f2975061c server_cookie=e37ed5b3aa07d8a in_seq=0 out_seq=0 2024-03-11T19:45:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.406+0000 7f17c6ffd640 1 -- 172.21.15.83:0/3762157671 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559653767720 con 0x559653ae9020 2024-03-11T19:45:05.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.406+0000 7f17c6ffd640 10 client.16458 handle_command_reply: tid=0 2024-03-11T19:45:05.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.407+0000 7f17f0a49740 1 client.16458 shutdown 2024-03-11T19:45:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.407+0000 7f17f0a49740 1 -- 172.21.15.83:0/3762157671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 msgr2=0x559653ae4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.407+0000 7f17f0a49740 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653ae4a70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009ca0 tx=0x7f17e0034a90 comp rx=0 tx=0).stop 2024-03-11T19:45:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.408+0000 7f17f0a49740 1 -- 172.21.15.83:0/3762157671 shutdown_connections 2024-03-11T19:45:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.408+0000 7f17f0a49740 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559653ae9020 0x559653aeb410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.408+0000 7f17f0a49740 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653ae4a70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.408+0000 7f17f0a49740 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 0x559653adda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.408+0000 7f17f0a49740 1 --2- 172.21.15.83:0/3762157671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2270 0x559653add4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.408+0000 7f17f0a49740 1 -- 172.21.15.83:0/3762157671 >> 172.21.15.83:0/3762157671 conn(0x559653abd7e0 msgr2=0x559653abcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.408+0000 7f17c6ffd640 0 client.16458 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:45:05.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.408+0000 7f17f0a49740 1 -- 172.21.15.83:0/3762157671 shutdown_connections 2024-03-11T19:45:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.408+0000 7f17f0a49740 1 -- 172.21.15.83:0/3762157671 wait complete. 2024-03-11T19:45:05.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.409+0000 7f17f0a49740 20 client.16458 trim_cache size 0 max 16384 2024-03-11T19:45:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.426+0000 7f17f0a49740 1 Processor -- start 2024-03-11T19:45:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.426+0000 7f17f0a49740 1 -- start start 2024-03-11T19:45:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.426+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac2270 0x559653ac1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.426+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 0x559653ae4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.426+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac7dd0 0x559653ac5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.426+0000 7f17e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac2270 0x559653ac1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.426+0000 7f17e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac2270 0x559653ac1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49342/0 (socket says 172.21.15.83:49342) 2024-03-11T19:45:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17e77fe640 1 -- 172.21.15.83:0/2361433669 learned_addr learned my addr 172.21.15.83:0/2361433669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17f0a49740 1 -- 172.21.15.83:0/2361433669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596539b0130 con 0x559653ac2270 2024-03-11T19:45:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17f0a49740 1 -- 172.21.15.83:0/2361433669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559653767720 con 0x559653ac2c60 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17f0a49740 1 -- 172.21.15.83:0/2361433669 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559653aafa40 con 0x559653ac7dd0 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17e7fff640 1 --2- 172.21.15.83:0/2361433669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac7dd0 0x559653ac5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17e6ffd640 1 --2- 172.21.15.83:0/2361433669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 0x559653ae4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17e7fff640 1 -- 172.21.15.83:0/2361433669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 msgr2=0x559653ae4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17e7fff640 1 --2- 172.21.15.83:0/2361433669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 0x559653ae4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17e7fff640 1 -- 172.21.15.83:0/2361433669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac2270 msgr2=0x559653ac1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17e7fff640 1 --2- 172.21.15.83:0/2361433669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac2270 0x559653ac1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17e7fff640 1 -- 172.21.15.83:0/2361433669 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596539c5c20 con 0x559653ac7dd0 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17e6ffd640 1 --2- 172.21.15.83:0/2361433669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 0x559653ae4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17e7fff640 1 --2- 172.21.15.83:0/2361433669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac7dd0 0x559653ac5200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f17e0004af0 tx=0x7f17e004a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b420d8e0e1910c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17cd7fa640 1 -- 172.21.15.83:0/2361433669 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e00583d0 con 0x559653ac7dd0 2024-03-11T19:45:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17cd7fa640 1 -- 172.21.15.83:0/2361433669 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55965372a7e0 con 0x559653ac7dd0 2024-03-11T19:45:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.427+0000 7f17cd7fa640 1 -- 172.21.15.83:0/2361433669 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0003df0 con 0x559653ac7dd0 2024-03-11T19:45:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.428+0000 7f17f0a49740 1 -- 172.21.15.83:0/2361433669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac7dd0 msgr2=0x559653ac5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.428+0000 7f17f0a49740 1 --2- 172.21.15.83:0/2361433669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac7dd0 0x559653ac5200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f17e0004af0 tx=0x7f17e004a000 comp rx=0 tx=0).stop 2024-03-11T19:45:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.428+0000 7f17f0a49740 1 -- 172.21.15.83:0/2361433669 shutdown_connections 2024-03-11T19:45:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.428+0000 7f17f0a49740 1 --2- 172.21.15.83:0/2361433669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac7dd0 0x559653ac5200 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.428+0000 7f17f0a49740 1 --2- 172.21.15.83:0/2361433669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2c60 0x559653ae4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.428+0000 7f17f0a49740 1 --2- 172.21.15.83:0/2361433669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac2270 0x559653ac1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.428+0000 7f17f0a49740 1 -- 172.21.15.83:0/2361433669 >> 172.21.15.83:0/2361433669 conn(0x559653abd7e0 msgr2=0x559653ac0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.428+0000 7f17f0a49740 1 -- 172.21.15.83:0/2361433669 shutdown_connections 2024-03-11T19:45:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.428+0000 7f17f0a49740 1 -- 172.21.15.83:0/2361433669 wait complete. 2024-03-11T19:45:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.429+0000 7f17f0a49740 1 -- start start 2024-03-11T19:45:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.429+0000 7f17f0a49740 1 Processor -- start 2024-03-11T19:45:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.429+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 0x559653b376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.429+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 0x559653b37c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.429+0000 7f17f0a49740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653b3e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17f0a49740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559653b3edf0 con 0x559653ac7dd0 2024-03-11T19:45:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17f0a49740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559653b3ef30 con 0x559653ac2270 2024-03-11T19:45:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653b3e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17e77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 0x559653b376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17e7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653b3e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49358/0 (socket says 172.21.15.83:49358) 2024-03-11T19:45:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17e7fff640 1 -- 172.21.15.83:0/1564377290 learned_addr learned my addr 172.21.15.83:0/1564377290 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17f0a49740 1 -- 172.21.15.83:0/1564377290 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559653b3f0a0 con 0x559653ac2c60 2024-03-11T19:45:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17e6ffd640 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 0x559653b37c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17f0a49740 10 client.0 _ll_register_callbacks cb 0x7f17e0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17e77fe640 1 -- 172.21.15.83:0/1564377290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 msgr2=0x559653b37c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17e77fe640 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 0x559653b37c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17e77fe640 1 -- 172.21.15.83:0/1564377290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 msgr2=0x559653b3e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17e77fe640 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653b3e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17e77fe640 1 -- 172.21.15.83:0/1564377290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559653a296a0 con 0x559653ac2270 2024-03-11T19:45:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17e77fe640 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 0x559653b376f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f17d800ba40 tx=0x7f17d8015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638e5cbea0a1901 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.430+0000 7f17c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:45:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17c6ffd640 1 -- 172.21.15.83:0/1564377290 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d80106d0 con 0x559653ac2270 2024-03-11T19:45:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17c6ffd640 1 -- 172.21.15.83:0/1564377290 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f17d8010830 con 0x559653ac2270 2024-03-11T19:45:05.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17f0a49740 1 -- 172.21.15.83:0/1564377290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17e00327b0 con 0x559653ac2270 2024-03-11T19:45:05.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17c6ffd640 1 -- 172.21.15.83:0/1564377290 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17d8036030 con 0x559653ac2270 2024-03-11T19:45:05.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17f0a49740 10 client.25934 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:05.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17f0a49740 1 -- 172.21.15.83:0/1564377290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5596539eb8c0 con 0x559653ac2270 2024-03-11T19:45:05.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17c6ffd640 1 -- 172.21.15.83:0/1564377290 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17e00327b0 con 0x559653ac2270 2024-03-11T19:45:05.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17c6ffd640 1 -- 172.21.15.83:0/1564377290 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f17d8030ba0 con 0x559653ac2270 2024-03-11T19:45:05.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17f0a49740 10 client.25934 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:05.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17f0a49740 10 client.25934 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:45:05.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17f0a49740 20 client.25934 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:05.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.431+0000 7f17f0a49740 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559653b40b50 0x559653b42f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.432+0000 7f17f0a49740 4 client.25934 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:45:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.432+0000 7f17f0a49740 1 -- 172.21.15.83:0/1564377290 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5596537b6890 con 0x559653b40b50 2024-03-11T19:45:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.432+0000 7f17e6ffd640 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559653b40b50 0x559653b42f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.432+0000 7f17e6ffd640 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559653b40b50 0x559653b42f70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45c9ea60ff4eb139 server_cookie=e80885b56ee6471c in_seq=0 out_seq=0 2024-03-11T19:45:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.432+0000 7f17c6ffd640 10 client.25934 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:45:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.432+0000 7f17c6ffd640 1 -- 172.21.15.83:0/1564377290 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5596537b6890 con 0x559653b40b50 2024-03-11T19:45:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.432+0000 7f17c6ffd640 10 client.25934 handle_command_reply: tid=0 2024-03-11T19:45:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.432+0000 7f17f0a49740 1 client.25934 shutdown 2024-03-11T19:45:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17f0a49740 1 -- 172.21.15.83:0/1564377290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 msgr2=0x559653b376f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17f0a49740 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 0x559653b376f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f17d800ba40 tx=0x7f17d8015000 comp rx=0 tx=0).stop 2024-03-11T19:45:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17f0a49740 1 -- 172.21.15.83:0/1564377290 shutdown_connections 2024-03-11T19:45:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17f0a49740 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559653b40b50 0x559653b42f70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17f0a49740 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559653ac7dd0 0x559653b3e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17c6ffd640 0 client.25934 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:45:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17f0a49740 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559653ac2c60 0x559653b37c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17f0a49740 1 --2- 172.21.15.83:0/1564377290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559653ac2270 0x559653b376f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17f0a49740 1 -- 172.21.15.83:0/1564377290 >> 172.21.15.83:0/1564377290 conn(0x559653abd7e0 msgr2=0x559653ac12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17f0a49740 1 -- 172.21.15.83:0/1564377290 shutdown_connections 2024-03-11T19:45:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17f0a49740 1 -- 172.21.15.83:0/1564377290 wait complete. 2024-03-11T19:45:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.433+0000 7f17f0a49740 20 client.25934 trim_cache size 0 max 16384 2024-03-11T19:45:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:45:05.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.435+0000 7f17ede76640 1 -- 172.21.15.83:0/3222896852 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17d00a1b30 msgr2=0x7f17d00a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.435+0000 7f17ede76640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17d00a1b30 0x7f17d00a3ff0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x559653ac3060 tx=0x7f17d800b5a0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.435+0000 7f17ede76640 1 -- 172.21.15.83:0/3222896852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8102380 msgr2=0x7f17e8073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.435+0000 7f17ede76640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8102380 0x7f17e8073570 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00e9f0 tx=0x7f17dc00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.435+0000 7f17ede76640 1 -- 172.21.15.83:0/3222896852 shutdown_connections 2024-03-11T19:45:05.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.435+0000 7f17ede76640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17d00a1b30 0x7f17d00a3ff0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.435+0000 7f17ede76640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17e8102d40 0x7f17e8073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.435+0000 7f17ede76640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17e8102380 0x7f17e8073570 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.435+0000 7f17ede76640 1 --2- 172.21.15.83:0/3222896852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17e8101190 0x7f17e8074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.435+0000 7f17ede76640 1 -- 172.21.15.83:0/3222896852 >> 172.21.15.83:0/3222896852 conn(0x7f17e80fc700 msgr2=0x7f17e8105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.436+0000 7f17ede76640 1 -- 172.21.15.83:0/3222896852 shutdown_connections 2024-03-11T19:45:05.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.436+0000 7f17ede76640 1 -- 172.21.15.83:0/3222896852 wait complete. 2024-03-11T19:45:05.453 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:45:05.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:45:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.614+0000 7fc9690d4640 1 -- 172.21.15.83:0/678019668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc96410d4a0 msgr2=0x7fc96410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.614+0000 7fc9690d4640 1 --2- 172.21.15.83:0/678019668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc96410d4a0 0x7fc96410d920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc95c01acb0 tx=0x7fc95c03d170 comp rx=0 tx=0).stop 2024-03-11T19:45:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.618+0000 7fc9690d4640 1 -- 172.21.15.83:0/678019668 shutdown_connections 2024-03-11T19:45:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.618+0000 7fc9690d4640 1 --2- 172.21.15.83:0/678019668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc96410d4a0 0x7fc96410d920 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.618+0000 7fc9690d4640 1 --2- 172.21.15.83:0/678019668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96410aaf0 0x7fc96410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.618+0000 7fc9690d4640 1 --2- 172.21.15.83:0/678019668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc964108180 0x7fc96410a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.618+0000 7fc9690d4640 1 -- 172.21.15.83:0/678019668 >> 172.21.15.83:0/678019668 conn(0x7fc96406dfe0 msgr2=0x7fc964070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.618+0000 7fc9690d4640 1 -- 172.21.15.83:0/678019668 shutdown_connections 2024-03-11T19:45:05.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.618+0000 7fc9690d4640 1 -- 172.21.15.83:0/678019668 wait complete. 2024-03-11T19:45:05.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc9690d4640 1 Processor -- start 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc9690d4640 1 -- start start 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc9690d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc964108180 0x7fc96419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc9690d4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96410aaf0 0x7fc96419dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc9690d4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc96410d4a0 0x7fc9641a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc9690d4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9641a5390 con 0x7fc964108180 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc9690d4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9641a5500 con 0x7fc96410d4a0 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc9690d4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9641a56c0 con 0x7fc96410aaf0 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc962d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc964108180 0x7fc96419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc96359e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc96410d4a0 0x7fc9641a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc96359e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc96410d4a0 0x7fc9641a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47194/0 (socket says 172.21.15.83:47194) 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc96359e640 1 -- 172.21.15.83:0/247465179 learned_addr learned my addr 172.21.15.83:0/247465179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.619+0000 7fc96259c640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96410aaf0 0x7fc96419dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc96359e640 1 -- 172.21.15.83:0/247465179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96410aaf0 msgr2=0x7fc96419dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc96359e640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96410aaf0 0x7fc96419dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc96359e640 1 -- 172.21.15.83:0/247465179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc964108180 msgr2=0x7fc96419d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc96359e640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc964108180 0x7fc96419d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc96359e640 1 -- 172.21.15.83:0/247465179 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c01a910 con 0x7fc96410d4a0 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc962d9d640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc964108180 0x7fc96419d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc96359e640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc96410d4a0 0x7fc9641a4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc95c01ac00 tx=0x7fc95c004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc96259c640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96410aaf0 0x7fc96419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc94bfff640 1 -- 172.21.15.83:0/247465179 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c03dd20 con 0x7fc96410d4a0 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc9690d4640 1 -- 172.21.15.83:0/247465179 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9641a5940 con 0x7fc96410d4a0 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc94bfff640 1 -- 172.21.15.83:0/247465179 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc95c004830 con 0x7fc96410d4a0 2024-03-11T19:45:05.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc94bfff640 1 -- 172.21.15.83:0/247465179 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c045b50 con 0x7fc96410d4a0 2024-03-11T19:45:05.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.620+0000 7fc9690d4640 1 -- 172.21.15.83:0/247465179 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9641a5df0 con 0x7fc96410d4a0 2024-03-11T19:45:05.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.623+0000 7fc94bfff640 1 -- 172.21.15.83:0/247465179 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc95c04c070 con 0x7fc96410d4a0 2024-03-11T19:45:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96bd7c740 1 Processor -- start 2024-03-11T19:45:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96bd7c740 1 -- start start 2024-03-11T19:45:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 0x55603aca7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 0x55603acac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 0x55603acaf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96bd7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603ac63960 con 0x55603acacdd0 2024-03-11T19:45:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96bd7c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603ac17310 con 0x55603aca7c60 2024-03-11T19:45:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96bd7c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603ac14840 con 0x55603aca7270 2024-03-11T19:45:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 0x55603acaf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc962d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 0x55603acac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc94bfff640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc94c0a1b80 0x7fc94c0a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 0x55603acaf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49386/0 (socket says 172.21.15.83:49386) 2024-03-11T19:45:05.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc962d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 0x55603acac760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47210/0 (socket says 172.21.15.83:47210) 2024-03-11T19:45:05.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96259c640 1 -- 172.21.15.83:0/237478117 learned_addr learned my addr 172.21.15.83:0/237478117 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc96259c640 1 --2- 172.21.15.83:0/237478117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 0x55603aca7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.624+0000 7fc94bfff640 1 -- 172.21.15.83:0/247465179 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc95c1006f0 con 0x7fc96410d4a0 2024-03-11T19:45:05.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc96259c640 1 -- 172.21.15.83:0/237478117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 msgr2=0x55603aca7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc96259c640 1 --2- 172.21.15.83:0/237478117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 0x55603aca7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc96259c640 1 -- 172.21.15.83:0/237478117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 msgr2=0x55603acac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc96259c640 1 --2- 172.21.15.83:0/237478117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 0x55603acac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc96359e640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc94c0a1b80 0x7fc94c0a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc96259c640 1 -- 172.21.15.83:0/237478117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55603abd2060 con 0x55603acacdd0 2024-03-11T19:45:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc962d9d640 1 --2- 172.21.15.83:0/237478117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 0x55603acac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc96259c640 1 --2- 172.21.15.83:0/237478117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 0x55603acaf260 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc9540128c0 tx=0x7fc95400a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15de82ef2f83fc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc96359e640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc94c0a1b80 0x7fc94c0a4040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc95c043d80 tx=0x7fc95c03d690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc9497fa640 1 -- 172.21.15.83:0/237478117 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95400daf0 con 0x55603acacdd0 2024-03-11T19:45:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc9497fa640 1 -- 172.21.15.83:0/237478117 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc954018070 con 0x55603acacdd0 2024-03-11T19:45:05.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.625+0000 7fc9497fa640 1 -- 172.21.15.83:0/237478117 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954004a00 con 0x55603acacdd0 2024-03-11T19:45:05.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.626+0000 7fc96bd7c740 1 -- 172.21.15.83:0/237478117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 msgr2=0x55603acaf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.626+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/237478117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 0x55603acaf260 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc9540128c0 tx=0x7fc95400a490 comp rx=0 tx=0).stop 2024-03-11T19:45:05.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 -- 172.21.15.83:0/237478117 shutdown_connections 2024-03-11T19:45:05.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/237478117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 0x55603acaf260 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/237478117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 0x55603acac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/237478117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 0x55603aca7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 -- 172.21.15.83:0/237478117 >> 172.21.15.83:0/237478117 conn(0x55603aca27e0 msgr2=0x55603aca2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 -- 172.21.15.83:0/237478117 shutdown_connections 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 -- 172.21.15.83:0/237478117 wait complete. 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 -- start start 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 Processor -- start 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 0x55603acb14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7c60 0x55603acb1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603acacdd0 0x55603acb8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603ac10020 con 0x55603aca7270 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603a8e3430 con 0x55603acacdd0 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.627+0000 7fc96bd7c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603a89fe00 con 0x55603aca7c60 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc96259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 0x55603acb14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc96259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 0x55603acb14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49396/0 (socket says 172.21.15.83:49396) 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc96259c640 1 -- 172.21.15.83:0/2812359400 learned_addr learned my addr 172.21.15.83:0/2812359400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc96bd7c740 10 client.0 _ll_register_callbacks cb 0x55603abb7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc962d9d640 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7c60 0x55603acb1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc96259c640 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603acacdd0 0x55603acb8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc962d9d640 1 -- 172.21.15.83:0/2812359400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603acacdd0 msgr2=0x55603acb8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc962d9d640 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603acacdd0 0x55603acb8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc962d9d640 1 -- 172.21.15.83:0/2812359400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 msgr2=0x55603acb14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc962d9d640 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 0x55603acb14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc962d9d640 1 -- 172.21.15.83:0/2812359400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9540125a0 con 0x55603aca7c60 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc962d9d640 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7c60 0x55603acb1a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55603aca8060 tx=0x7fc95800f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4002062989e89cc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc946ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc946ff5640 1 -- 172.21.15.83:0/2812359400 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958015070 con 0x55603aca7c60 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc96bd7c740 1 -- 172.21.15.83:0/2812359400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55603a90f7e0 con 0x55603aca7c60 2024-03-11T19:45:05.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.628+0000 7fc946ff5640 1 -- 172.21.15.83:0/2812359400 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc958005020 con 0x55603aca7c60 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc946ff5640 1 -- 172.21.15.83:0/2812359400 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc95800ae90 con 0x55603aca7c60 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc96bd7c740 10 client.26035 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc96bd7c740 1 -- 172.21.15.83:0/2812359400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55603ac0e6a0 con 0x55603aca7c60 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc946ff5640 1 -- 172.21.15.83:0/2812359400 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55603a90f7e0 con 0x55603aca7c60 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc946ff5640 1 -- 172.21.15.83:0/2812359400 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc958019410 con 0x55603aca7c60 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc96bd7c740 10 client.26035 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc96bd7c740 10 client.26035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc96bd7c740 20 client.26035 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55603acbd7d0 0x55603acbfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc96bd7c740 4 client.26035 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc96bd7c740 1 -- 172.21.15.83:0/2812359400 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55603a94c720 con 0x55603acbd7d0 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.629+0000 7fc96259c640 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55603acbd7d0 0x55603acbfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.630+0000 7fc946ff5640 10 client.26035 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:05.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.630+0000 7fc96259c640 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55603acbd7d0 0x55603acbfc10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eaccbb4ea31347f server_cookie=fff5f706da85d5b6 in_seq=0 out_seq=0 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc946ff5640 1 -- 172.21.15.83:0/2812359400 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55603a94c720 con 0x55603acbd7d0 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc946ff5640 10 client.26035 handle_command_reply: tid=0 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc96bd7c740 1 client.26035 shutdown 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc96bd7c740 1 -- 172.21.15.83:0/2812359400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7c60 msgr2=0x55603acb1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7c60 0x55603acb1a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55603aca8060 tx=0x7fc95800f7f0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc96bd7c740 1 -- 172.21.15.83:0/2812359400 shutdown_connections 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55603acbd7d0 0x55603acbfc10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603acacdd0 0x55603acb8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7c60 0x55603acb1a00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/2812359400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 0x55603acb14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc96bd7c740 1 -- 172.21.15.83:0/2812359400 >> 172.21.15.83:0/2812359400 conn(0x55603aca27e0 msgr2=0x55603aca1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc946ff5640 0 client.26035 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc96bd7c740 1 -- 172.21.15.83:0/2812359400 shutdown_connections 2024-03-11T19:45:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.631+0000 7fc96bd7c740 1 -- 172.21.15.83:0/2812359400 wait complete. 2024-03-11T19:45:05.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.632+0000 7fc96bd7c740 20 client.26035 trim_cache size 0 max 16384 2024-03-11T19:45:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.650+0000 7fc96bd7c740 1 Processor -- start 2024-03-11T19:45:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.650+0000 7fc96bd7c740 1 -- start start 2024-03-11T19:45:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.650+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 0x55603ac9da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.650+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 0x55603acb8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.650+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 0x55603acb1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.650+0000 7fc96259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 0x55603ac9da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.650+0000 7fc96259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 0x55603ac9da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45056/0 (socket says 172.21.15.83:45056) 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.650+0000 7fc96259c640 1 -- 172.21.15.83:0/1180781150 learned_addr learned my addr 172.21.15.83:0/1180781150 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.650+0000 7fc96bd7c740 1 -- 172.21.15.83:0/1180781150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603ab95130 con 0x55603acacdd0 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.651+0000 7fc96bd7c740 1 -- 172.21.15.83:0/1180781150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603ac9e5c0 con 0x55603aca7c60 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.651+0000 7fc96bd7c740 1 -- 172.21.15.83:0/1180781150 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603acb1300 con 0x55603aca7270 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.651+0000 7fc962d9d640 1 --2- 172.21.15.83:0/1180781150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 0x55603acb8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.651+0000 7fc96259c640 1 --2- 172.21.15.83:0/1180781150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 0x55603acb1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.651+0000 7fc96259c640 1 -- 172.21.15.83:0/1180781150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 msgr2=0x55603acb8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.651+0000 7fc96259c640 1 --2- 172.21.15.83:0/1180781150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 0x55603acb8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.651+0000 7fc96259c640 1 -- 172.21.15.83:0/1180781150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 msgr2=0x55603acb1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.651+0000 7fc96259c640 1 --2- 172.21.15.83:0/1180781150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 0x55603acb1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.651+0000 7fc96259c640 1 -- 172.21.15.83:0/1180781150 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55603abaac20 con 0x55603aca7270 2024-03-11T19:45:05.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.651+0000 7fc96259c640 1 --2- 172.21.15.83:0/1180781150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 0x55603ac9da10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc95400de60 tx=0x7fc95400a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921680c23e67211e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.651+0000 7fc948ff9640 1 -- 172.21.15.83:0/1180781150 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95400d970 con 0x55603aca7270 2024-03-11T19:45:05.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.652+0000 7fc948ff9640 1 -- 172.21.15.83:0/1180781150 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc95402c840 con 0x55603aca7270 2024-03-11T19:45:05.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.652+0000 7fc948ff9640 1 -- 172.21.15.83:0/1180781150 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95402b880 con 0x55603aca7270 2024-03-11T19:45:05.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.652+0000 7fc96bd7c740 1 -- 172.21.15.83:0/1180781150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 msgr2=0x55603ac9da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.652+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/1180781150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 0x55603ac9da10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc95400de60 tx=0x7fc95400a330 comp rx=0 tx=0).stop 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.653+0000 7fc96bd7c740 1 -- 172.21.15.83:0/1180781150 shutdown_connections 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.653+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/1180781150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603acacdd0 0x55603acb1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.653+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/1180781150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603aca7c60 0x55603acb8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.653+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/1180781150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7270 0x55603ac9da10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.653+0000 7fc96bd7c740 1 -- 172.21.15.83:0/1180781150 >> 172.21.15.83:0/1180781150 conn(0x55603aca27e0 msgr2=0x55603aca86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.653+0000 7fc96bd7c740 1 -- 172.21.15.83:0/1180781150 shutdown_connections 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.653+0000 7fc96bd7c740 1 -- 172.21.15.83:0/1180781150 wait complete. 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96bd7c740 1 -- start start 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96bd7c740 1 Processor -- start 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 0x55603acb8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7c60 0x55603acb6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96bd7c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603acacdd0 0x55603acb7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96bd7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603acb7840 con 0x55603aca7270 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96bd7c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603acb79b0 con 0x55603acacdd0 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96bd7c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603acb7b70 con 0x55603aca7c60 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96bd7c740 10 client.0 _ll_register_callbacks cb 0x7fc95400aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 0x55603acb8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 0x55603acb8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49404/0 (socket says 172.21.15.83:49404) 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.654+0000 7fc96259c640 1 -- 172.21.15.83:0/3814706665 learned_addr learned my addr 172.21.15.83:0/3814706665 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc962d9d640 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7c60 0x55603acb6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc96259c640 1 -- 172.21.15.83:0/3814706665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7c60 msgr2=0x55603acb6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc96259c640 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7c60 0x55603acb6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc96259c640 1 -- 172.21.15.83:0/3814706665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603acacdd0 msgr2=0x55603acb7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc96259c640 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603acacdd0 0x55603acb7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc96259c640 1 -- 172.21.15.83:0/3814706665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc954025040 con 0x55603aca7270 2024-03-11T19:45:05.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc96259c640 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 0x55603acb8740 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc95400e010 tx=0x7fc95400ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d525409cacd86f82 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc946ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc946ff5640 1 -- 172.21.15.83:0/3814706665 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95400d970 con 0x55603aca7270 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc946ff5640 1 -- 172.21.15.83:0/3814706665 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc95400db80 con 0x55603aca7270 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc96bd7c740 1 -- 172.21.15.83:0/3814706665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc954018070 con 0x55603aca7270 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc946ff5640 1 -- 172.21.15.83:0/3814706665 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc95402c570 con 0x55603aca7270 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc96bd7c740 10 client.16479 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.655+0000 7fc96bd7c740 1 -- 172.21.15.83:0/3814706665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55603abd08c0 con 0x55603aca7270 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.656+0000 7fc946ff5640 1 -- 172.21.15.83:0/3814706665 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc954048020 con 0x55603aca7270 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.656+0000 7fc946ff5640 1 -- 172.21.15.83:0/3814706665 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc954035070 con 0x55603aca7270 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.656+0000 7fc96bd7c740 10 client.16479 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.656+0000 7fc96bd7c740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.656+0000 7fc96bd7c740 20 client.16479 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.656+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55603ad15070 0x55603ad17490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:05.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.656+0000 7fc96bd7c740 4 client.16479 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:45:05.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.656+0000 7fc96bd7c740 1 -- 172.21.15.83:0/3814706665 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55603a99b890 con 0x55603ad15070 2024-03-11T19:45:05.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.656+0000 7fc962d9d640 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55603ad15070 0x55603ad17490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:05.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.657+0000 7fc946ff5640 10 client.16479 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:05.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.657+0000 7fc962d9d640 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55603ad15070 0x55603ad17490 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e286a0ea5f51865 server_cookie=84d2142ae50de791 in_seq=0 out_seq=0 2024-03-11T19:45:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.657+0000 7fc946ff5640 1 -- 172.21.15.83:0/3814706665 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55603a99b890 con 0x55603ad15070 2024-03-11T19:45:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.657+0000 7fc946ff5640 10 client.16479 handle_command_reply: tid=0 2024-03-11T19:45:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.657+0000 7fc96bd7c740 1 client.16479 shutdown 2024-03-11T19:45:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc96bd7c740 1 -- 172.21.15.83:0/3814706665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 msgr2=0x55603acb8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 0x55603acb8740 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc95400e010 tx=0x7fc95400ad60 comp rx=0 tx=0).stop 2024-03-11T19:45:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc96bd7c740 1 -- 172.21.15.83:0/3814706665 shutdown_connections 2024-03-11T19:45:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55603ad15070 0x55603ad17490 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55603acacdd0 0x55603acb7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55603aca7c60 0x55603acb6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc96bd7c740 1 --2- 172.21.15.83:0/3814706665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55603aca7270 0x55603acb8740 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc96bd7c740 1 -- 172.21.15.83:0/3814706665 >> 172.21.15.83:0/3814706665 conn(0x55603aca27e0 msgr2=0x55603ac9da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc946ff5640 0 client.16479 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:05.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc96bd7c740 1 -- 172.21.15.83:0/3814706665 shutdown_connections 2024-03-11T19:45:05.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc96bd7c740 1 -- 172.21.15.83:0/3814706665 wait complete. 2024-03-11T19:45:05.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.658+0000 7fc96bd7c740 20 client.16479 trim_cache size 0 max 16384 2024-03-11T19:45:05.668 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:45:05.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 -- 172.21.15.83:0/247465179 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc94c0a1b80 msgr2=0x7fc94c0a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc94c0a1b80 0x7fc94c0a4040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc95c043d80 tx=0x7fc95c03d690 comp rx=0 tx=0).stop 2024-03-11T19:45:05.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 -- 172.21.15.83:0/247465179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc96410d4a0 msgr2=0x7fc9641a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:05.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc96410d4a0 0x7fc9641a4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc95c01ac00 tx=0x7fc95c004530 comp rx=0 tx=0).stop 2024-03-11T19:45:05.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 -- 172.21.15.83:0/247465179 shutdown_connections 2024-03-11T19:45:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc94c0a1b80 0x7fc94c0a4040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc96410d4a0 0x7fc9641a4e50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96410aaf0 0x7fc96419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 --2- 172.21.15.83:0/247465179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc964108180 0x7fc96419d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 -- 172.21.15.83:0/247465179 >> 172.21.15.83:0/247465179 conn(0x7fc96406dfe0 msgr2=0x7fc96410b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 -- 172.21.15.83:0/247465179 shutdown_connections 2024-03-11T19:45:05.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:05.659+0000 7fc9690d4640 1 -- 172.21.15.83:0/247465179 wait complete. 2024-03-11T19:45:06.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:05 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3689131544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:45:06.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:05 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3689131544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:45:06.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:05 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3689131544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:45:06.678 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:45:06.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:45:06.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:45:06.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.813+0000 7f2d291a0640 1 -- 172.21.15.83:0/3306779065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d241001a0 msgr2=0x7f2d24100600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:06.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.813+0000 7f2d291a0640 1 --2- 172.21.15.83:0/3306779065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d241001a0 0x7f2d24100600 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009900 tx=0x7f2d1802bc30 comp rx=0 tx=0).stop 2024-03-11T19:45:06.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.813+0000 7f2d291a0640 1 -- 172.21.15.83:0/3306779065 shutdown_connections 2024-03-11T19:45:06.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.813+0000 7f2d291a0640 1 --2- 172.21.15.83:0/3306779065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d24100b40 0x7f2d24108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.813+0000 7f2d291a0640 1 --2- 172.21.15.83:0/3306779065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d241001a0 0x7f2d24100600 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.813+0000 7f2d291a0640 1 --2- 172.21.15.83:0/3306779065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24101550 0x7f2d24101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.813+0000 7f2d291a0640 1 -- 172.21.15.83:0/3306779065 >> 172.21.15.83:0/3306779065 conn(0x7f2d240facb0 msgr2=0x7f2d240fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:06.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.813+0000 7f2d291a0640 1 -- 172.21.15.83:0/3306779065 shutdown_connections 2024-03-11T19:45:06.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.813+0000 7f2d291a0640 1 -- 172.21.15.83:0/3306779065 wait complete. 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d291a0640 1 Processor -- start 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d291a0640 1 -- start start 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d291a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d241001a0 0x7f2d24194f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d291a0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d24100b40 0x7f2d24195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d291a0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24101550 0x7f2d2419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d291a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2419ca20 con 0x7f2d241001a0 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d291a0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2419cb90 con 0x7f2d24100b40 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d291a0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2419cd50 con 0x7f2d24101550 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d23fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d241001a0 0x7f2d24194f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d23fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d241001a0 0x7f2d24194f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49418/0 (socket says 172.21.15.83:49418) 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d23fff640 1 -- 172.21.15.83:0/1653244275 learned_addr learned my addr 172.21.15.83:0/1653244275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d237fe640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d24100b40 0x7f2d24195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d2899f640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24101550 0x7f2d2419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d23fff640 1 -- 172.21.15.83:0/1653244275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24101550 msgr2=0x7f2d2419c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:06.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.814+0000 7f2d23fff640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24101550 0x7f2d2419c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.815+0000 7f2d23fff640 1 -- 172.21.15.83:0/1653244275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d24100b40 msgr2=0x7f2d24195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:06.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.815+0000 7f2d23fff640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d24100b40 0x7f2d24195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.815+0000 7f2d23fff640 1 -- 172.21.15.83:0/1653244275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d18009560 con 0x7f2d241001a0 2024-03-11T19:45:06.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.815+0000 7f2d23fff640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d241001a0 0x7f2d24194f30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9f0 tx=0x7f2d1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:06.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.816+0000 7f2d217fa640 1 -- 172.21.15.83:0/1653244275 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14009720 con 0x7f2d241001a0 2024-03-11T19:45:06.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.816+0000 7f2d217fa640 1 -- 172.21.15.83:0/1653244275 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d1400cd80 con 0x7f2d241001a0 2024-03-11T19:45:06.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.816+0000 7f2d291a0640 1 -- 172.21.15.83:0/1653244275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2419cfd0 con 0x7f2d241001a0 2024-03-11T19:45:06.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.816+0000 7f2d291a0640 1 -- 172.21.15.83:0/1653244275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2419d4b0 con 0x7f2d241001a0 2024-03-11T19:45:06.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.816+0000 7f2d217fa640 1 -- 172.21.15.83:0/1653244275 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14010490 con 0x7f2d241001a0 2024-03-11T19:45:06.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.818+0000 7f2d217fa640 1 -- 172.21.15.83:0/1653244275 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2d14010730 con 0x7f2d241001a0 2024-03-11T19:45:06.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.818+0000 7f2d2cfce740 1 Processor -- start 2024-03-11T19:45:06.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2cfce740 1 -- start start 2024-03-11T19:45:06.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2cfce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426dd1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d217fa640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2d0c0a3d90 0x7f2d0c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2cfce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd2860 0x564426dd2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2cfce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd3220 0x564426dd9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2cfce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426d40f90 con 0x564426dd1690 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2cfce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426d3e4c0 con 0x564426dd2860 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2cfce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426d39ca0 con 0x564426dd3220 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2899f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426dd1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2899f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426dd1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49430/0 (socket says 172.21.15.83:49430) 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2899f640 1 -- 172.21.15.83:0/1935984947 learned_addr learned my addr 172.21.15.83:0/1935984947 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d23fff640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2d0c0a3d90 0x7f2d0c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2899f640 1 --2- 172.21.15.83:0/1935984947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd3220 0x564426dd9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d237fe640 1 --2- 172.21.15.83:0/1935984947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd2860 0x564426dd2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d217fa640 1 -- 172.21.15.83:0/1653244275 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2d14014070 con 0x7f2d241001a0 2024-03-11T19:45:06.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2899f640 1 -- 172.21.15.83:0/1935984947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd3220 msgr2=0x564426dd9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:06.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2899f640 1 --2- 172.21.15.83:0/1935984947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd3220 0x564426dd9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2899f640 1 -- 172.21.15.83:0/1935984947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd2860 msgr2=0x564426dd2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:06.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2899f640 1 --2- 172.21.15.83:0/1935984947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd2860 0x564426dd2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2899f640 1 -- 172.21.15.83:0/1935984947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564426cfc0b0 con 0x564426dd1690 2024-03-11T19:45:06.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d237fe640 1 --2- 172.21.15.83:0/1935984947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd2860 0x564426dd2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:06.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d23fff640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2d0c0a3d90 0x7f2d0c0a6250 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2d14099590 tx=0x7f2d14013b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:06.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d2899f640 1 --2- 172.21.15.83:0/1935984947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426dd1ab0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c013b60 tx=0x7f2d1c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1e649a3846f357 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:06.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.819+0000 7f2d0a7fc640 1 -- 172.21.15.83:0/1935984947 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c013130 con 0x564426dd1690 2024-03-11T19:45:06.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.820+0000 7f2d0a7fc640 1 -- 172.21.15.83:0/1935984947 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c013290 con 0x564426dd1690 2024-03-11T19:45:06.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.820+0000 7f2d0a7fc640 1 -- 172.21.15.83:0/1935984947 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c0053a0 con 0x564426dd1690 2024-03-11T19:45:06.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.821+0000 7f2d2cfce740 1 -- 172.21.15.83:0/1935984947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 msgr2=0x564426dd1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:06.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.821+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/1935984947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426dd1ab0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c013b60 tx=0x7f2d1c009b30 comp rx=0 tx=0).stop 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.821+0000 7f2d2cfce740 1 -- 172.21.15.83:0/1935984947 shutdown_connections 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.821+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/1935984947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd3220 0x564426dd9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.821+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/1935984947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd2860 0x564426dd2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.821+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/1935984947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426dd1ab0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.821+0000 7f2d2cfce740 1 -- 172.21.15.83:0/1935984947 >> 172.21.15.83:0/1935984947 conn(0x564426dcc870 msgr2=0x564426dccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.821+0000 7f2d2cfce740 1 -- 172.21.15.83:0/1935984947 shutdown_connections 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.821+0000 7f2d2cfce740 1 -- 172.21.15.83:0/1935984947 wait complete. 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.821+0000 7f2d2cfce740 1 -- start start 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d2cfce740 1 Processor -- start 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d2cfce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426ddb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d2cfce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426ddbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d2cfce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd3220 0x564426de2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d2cfce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426cdef40 con 0x564426dd1690 2024-03-11T19:45:06.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d2cfce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426a0d4b0 con 0x564426dd3220 2024-03-11T19:45:06.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d2cfce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426cbf670 con 0x564426dd2860 2024-03-11T19:45:06.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d2cfce740 10 client.0 _ll_register_callbacks cb 0x564426d2cf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:06.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d237fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426ddbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d237fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426ddbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45106/0 (socket says 172.21.15.83:45106) 2024-03-11T19:45:06.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d237fe640 1 -- 172.21.15.83:0/282305371 learned_addr learned my addr 172.21.15.83:0/282305371 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:06.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.822+0000 7f2d2899f640 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd3220 0x564426de2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.823+0000 7f2d2899f640 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426ddb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.824+0000 7f2d237fe640 1 -- 172.21.15.83:0/282305371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd3220 msgr2=0x564426de2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.824+0000 7f2d237fe640 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd3220 0x564426de2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.824+0000 7f2d237fe640 1 -- 172.21.15.83:0/282305371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 msgr2=0x564426ddb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:06.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.824+0000 7f2d237fe640 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426ddb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.824+0000 7f2d237fe640 1 -- 172.21.15.83:0/282305371 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c01b040 con 0x564426dd2860 2024-03-11T19:45:06.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.837+0000 7f2d237fe640 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426ddbde0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2d1802c170 tx=0x7f2d180048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ac1712018d06d1f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:06.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.839+0000 7f2d037fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:45:06.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.839+0000 7f2d037fe640 1 -- 172.21.15.83:0/282305371 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18004b20 con 0x564426dd2860 2024-03-11T19:45:06.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.839+0000 7f2d037fe640 1 -- 172.21.15.83:0/282305371 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2d18004250 con 0x564426dd2860 2024-03-11T19:45:06.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.840+0000 7f2d2cfce740 1 -- 172.21.15.83:0/282305371 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564426d13f40 con 0x564426dd2860 2024-03-11T19:45:06.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.842+0000 7f2d037fe640 1 -- 172.21.15.83:0/282305371 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d18004410 con 0x564426dd2860 2024-03-11T19:45:06.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.842+0000 7f2d2cfce740 10 client.26053 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:06.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.842+0000 7f2d2cfce740 1 -- 172.21.15.83:0/282305371 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564426d38320 con 0x564426dd2860 2024-03-11T19:45:06.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.843+0000 7f2d037fe640 1 -- 172.21.15.83:0/282305371 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564426d13f40 con 0x564426dd2860 2024-03-11T19:45:06.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.843+0000 7f2d037fe640 1 -- 172.21.15.83:0/282305371 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2d1803c040 con 0x564426dd2860 2024-03-11T19:45:06.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.843+0000 7f2d2cfce740 10 client.26053 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:06.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.843+0000 7f2d2cfce740 10 client.26053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:06.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.843+0000 7f2d2cfce740 20 client.26053 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:06.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.843+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564426de7a40 0x564426de9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.843+0000 7f2d2cfce740 4 client.26053 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:06.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.843+0000 7f2d2cfce740 1 -- 172.21.15.83:0/282305371 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564426d30310 con 0x564426de7a40 2024-03-11T19:45:06.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.847+0000 7f2d2899f640 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564426de7a40 0x564426de9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.847+0000 7f2d2899f640 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564426de7a40 0x564426de9e60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8d5b8156b07958 server_cookie=cce029ceee34657a in_seq=0 out_seq=0 2024-03-11T19:45:06.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.847+0000 7f2d037fe640 10 client.26053 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:06.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.848+0000 7f2d037fe640 1 -- 172.21.15.83:0/282305371 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564426d30310 con 0x564426de7a40 2024-03-11T19:45:06.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.848+0000 7f2d037fe640 10 client.26053 handle_command_reply: tid=0 2024-03-11T19:45:06.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.848+0000 7f2d2cfce740 1 client.26053 shutdown 2024-03-11T19:45:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.849+0000 7f2d2cfce740 1 -- 172.21.15.83:0/282305371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 msgr2=0x564426ddbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.849+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426ddbde0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2d1802c170 tx=0x7f2d180048f0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.849+0000 7f2d2cfce740 1 -- 172.21.15.83:0/282305371 shutdown_connections 2024-03-11T19:45:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.849+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564426de7a40 0x564426de9e60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.849+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd3220 0x564426de2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.849+0000 7f2d037fe640 0 client.26053 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.849+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426ddbde0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.849+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/282305371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426ddb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.849+0000 7f2d2cfce740 1 -- 172.21.15.83:0/282305371 >> 172.21.15.83:0/282305371 conn(0x564426dcc870 msgr2=0x564426dcfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.849+0000 7f2d2cfce740 1 -- 172.21.15.83:0/282305371 shutdown_connections 2024-03-11T19:45:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.850+0000 7f2d2cfce740 1 -- 172.21.15.83:0/282305371 wait complete. 2024-03-11T19:45:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.850+0000 7f2d2cfce740 20 client.26053 trim_cache size 0 max 16384 2024-03-11T19:45:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.871+0000 7f2d2cfce740 1 Processor -- start 2024-03-11T19:45:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.871+0000 7f2d2cfce740 1 -- start start 2024-03-11T19:45:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2cfce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd1690 0x564426dd6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2cfce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426dd6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2cfce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd3220 0x564426dcb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2cfce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426ddb6e0 con 0x564426dd3220 2024-03-11T19:45:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2cfce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426d30340 con 0x564426dd1690 2024-03-11T19:45:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2cfce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426dd25f0 con 0x564426dd2860 2024-03-11T19:45:06.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2899f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd3220 0x564426dcb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2899f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd3220 0x564426dcb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49442/0 (socket says 172.21.15.83:49442) 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2899f640 1 -- 172.21.15.83:0/2793087562 learned_addr learned my addr 172.21.15.83:0/2793087562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d237fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426dd6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2899f640 1 --2- 172.21.15.83:0/2793087562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd1690 0x564426dd6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2899f640 1 -- 172.21.15.83:0/2793087562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 msgr2=0x564426dd6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2899f640 1 --2- 172.21.15.83:0/2793087562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426dd6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2899f640 1 -- 172.21.15.83:0/2793087562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd1690 msgr2=0x564426dd6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2899f640 1 --2- 172.21.15.83:0/2793087562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd1690 0x564426dd6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2899f640 1 -- 172.21.15.83:0/2793087562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564426cd4c00 con 0x564426dd3220 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d2899f640 1 --2- 172.21.15.83:0/2793087562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd3220 0x564426dcb400 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c005cc0 tx=0x7f2d1c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1339ef8e348c60e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.872+0000 7f2d09ffb640 1 -- 172.21.15.83:0/2793087562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c00fe80 con 0x564426dd3220 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.873+0000 7f2d09ffb640 1 -- 172.21.15.83:0/2793087562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c003e50 con 0x564426dd3220 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.873+0000 7f2d09ffb640 1 -- 172.21.15.83:0/2793087562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c02b5d0 con 0x564426dd3220 2024-03-11T19:45:06.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.873+0000 7f2d2cfce740 1 -- 172.21.15.83:0/2793087562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd3220 msgr2=0x564426dcb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:06.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.873+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/2793087562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd3220 0x564426dcb400 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c005cc0 tx=0x7f2d1c009500 comp rx=0 tx=0).stop 2024-03-11T19:45:06.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.873+0000 7f2d2cfce740 1 -- 172.21.15.83:0/2793087562 shutdown_connections 2024-03-11T19:45:06.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.873+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/2793087562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd3220 0x564426dcb400 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.873+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/2793087562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426dd6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.873+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/2793087562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd1690 0x564426dd6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.873+0000 7f2d2cfce740 1 -- 172.21.15.83:0/2793087562 >> 172.21.15.83:0/2793087562 conn(0x564426dcc870 msgr2=0x564426dccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:06.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.873+0000 7f2d2cfce740 1 -- 172.21.15.83:0/2793087562 shutdown_connections 2024-03-11T19:45:06.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.873+0000 7f2d2cfce740 1 -- 172.21.15.83:0/2793087562 wait complete. 2024-03-11T19:45:06.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.874+0000 7f2d2cfce740 1 -- start start 2024-03-11T19:45:06.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.874+0000 7f2d2cfce740 1 Processor -- start 2024-03-11T19:45:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.875+0000 7f2d2cfce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426e3ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.875+0000 7f2d2cfce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426e3b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.875+0000 7f2d2899f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426e3ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.875+0000 7f2d2899f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426e3ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49456/0 (socket says 172.21.15.83:49456) 2024-03-11T19:45:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.875+0000 7f2d2899f640 1 -- 172.21.15.83:0/851673354 learned_addr learned my addr 172.21.15.83:0/851673354 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.875+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd3220 0x564426e422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.875+0000 7f2d2cfce740 1 -- 172.21.15.83:0/851673354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426e42810 con 0x564426dd1690 2024-03-11T19:45:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.875+0000 7f2d2cfce740 1 -- 172.21.15.83:0/851673354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426e42950 con 0x564426dd3220 2024-03-11T19:45:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.875+0000 7f2d2cfce740 1 -- 172.21.15.83:0/851673354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564426e42ac0 con 0x564426dd2860 2024-03-11T19:45:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.875+0000 7f2d237fe640 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426e3b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.875+0000 7f2d2cfce740 10 client.0 _ll_register_callbacks cb 0x7f2d1c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d2899f640 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd3220 0x564426e422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d237fe640 1 -- 172.21.15.83:0/851673354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd3220 msgr2=0x564426e422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d237fe640 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd3220 0x564426e422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d237fe640 1 -- 172.21.15.83:0/851673354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 msgr2=0x564426e3ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d237fe640 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426e3ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d237fe640 1 -- 172.21.15.83:0/851673354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c02b040 con 0x564426dd2860 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d237fe640 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426e3b290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2d18000c00 tx=0x7f2d18005300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf0f4bda0e6c6e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d037fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d037fe640 1 -- 172.21.15.83:0/851673354 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18004b20 con 0x564426dd2860 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d2cfce740 1 -- 172.21.15.83:0/851673354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564426d5f900 con 0x564426dd2860 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d037fe640 1 -- 172.21.15.83:0/851673354 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2d18000e40 con 0x564426dd2860 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d037fe640 1 -- 172.21.15.83:0/851673354 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d1802cc80 con 0x564426dd2860 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d2cfce740 10 client.26059 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d2cfce740 1 -- 172.21.15.83:0/851673354 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564426cfa910 con 0x564426dd2860 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d037fe640 1 -- 172.21.15.83:0/851673354 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564426d5f900 con 0x564426dd2860 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.876+0000 7f2d037fe640 1 -- 172.21.15.83:0/851673354 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2d1803c040 con 0x564426dd2860 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.877+0000 7f2d2cfce740 10 client.26059 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.877+0000 7f2d2cfce740 10 client.26059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.877+0000 7f2d2cfce740 20 client.26059 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.877+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564426de7a40 0x564426e46440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.877+0000 7f2d2cfce740 4 client.26059 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.877+0000 7f2d2cfce740 1 -- 172.21.15.83:0/851673354 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564426a39860 con 0x564426de7a40 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.877+0000 7f2d2899f640 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564426de7a40 0x564426e46440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.878+0000 7f2d2899f640 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564426de7a40 0x564426e46440 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b44d961bd246a5d server_cookie=467673677b4ca14b in_seq=0 out_seq=0 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.878+0000 7f2d037fe640 10 client.26059 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d037fe640 1 -- 172.21.15.83:0/851673354 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564426a39860 con 0x564426de7a40 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d037fe640 10 client.26059 handle_command_reply: tid=0 2024-03-11T19:45:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 1 client.26059 shutdown 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 1 -- 172.21.15.83:0/851673354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 msgr2=0x564426e3b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426e3b290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2d18000c00 tx=0x7f2d18005300 comp rx=0 tx=0).stop 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 1 -- 172.21.15.83:0/851673354 shutdown_connections 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564426de7a40 0x564426e46440 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564426dd3220 0x564426e422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564426dd2860 0x564426e3b290 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 1 --2- 172.21.15.83:0/851673354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564426dd1690 0x564426e3ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d037fe640 0 client.26059 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 1 -- 172.21.15.83:0/851673354 >> 172.21.15.83:0/851673354 conn(0x564426dcc870 msgr2=0x564426ddb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 1 -- 172.21.15.83:0/851673354 shutdown_connections 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 1 -- 172.21.15.83:0/851673354 wait complete. 2024-03-11T19:45:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.879+0000 7f2d2cfce740 20 client.26059 trim_cache size 0 max 16384 2024-03-11T19:45:06.888 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:45:06.888 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:45:06.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "957c1fd1-7475-421a-8d63-6d499ff639e3", 2024-03-11T19:45:06.888 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:45:06.888 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 -- 172.21.15.83:0/1653244275 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2d0c0a3d90 msgr2=0x7f2d0c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2d0c0a3d90 0x7f2d0c0a6250 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2d14099590 tx=0x7f2d14013b60 comp rx=0 tx=0).stop 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 -- 172.21.15.83:0/1653244275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d241001a0 msgr2=0x7f2d24194f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d241001a0 0x7f2d24194f30 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9f0 tx=0x7f2d1400edf0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 -- 172.21.15.83:0/1653244275 shutdown_connections 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f2d0c0a3d90 0x7f2d0c0a6250 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24101550 0x7f2d2419c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d24100b40 0x7f2d24195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 --2- 172.21.15.83:0/1653244275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d241001a0 0x7f2d24194f30 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 -- 172.21.15.83:0/1653244275 >> 172.21.15.83:0/1653244275 conn(0x7f2d240facb0 msgr2=0x7f2d241044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 -- 172.21.15.83:0/1653244275 shutdown_connections 2024-03-11T19:45:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:06.880+0000 7f2d291a0640 1 -- 172.21.15.83:0/1653244275 wait complete. 2024-03-11T19:45:06.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:45:06.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:45:07.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.031+0000 7f4372302640 1 -- 172.21.15.83:0/1308550470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c100080 msgr2=0x7f436c1004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:07.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.031+0000 7f4372302640 1 --2- 172.21.15.83:0/1308550470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c100080 0x7f436c1004a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f435c009900 tx=0x7f435c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:45:07.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.031+0000 7f4372302640 1 -- 172.21.15.83:0/1308550470 shutdown_connections 2024-03-11T19:45:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.031+0000 7f4372302640 1 --2- 172.21.15.83:0/1308550470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c101c30 0x7f436c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.031+0000 7f4372302640 1 --2- 172.21.15.83:0/1308550470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f436c101270 0x7f436c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.031+0000 7f4372302640 1 --2- 172.21.15.83:0/1308550470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c100080 0x7f436c1004a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.031+0000 7f4372302640 1 -- 172.21.15.83:0/1308550470 >> 172.21.15.83:0/1308550470 conn(0x7f436c0fb5f0 msgr2=0x7f436c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.032+0000 7f4372302640 1 -- 172.21.15.83:0/1308550470 shutdown_connections 2024-03-11T19:45:07.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.032+0000 7f4372302640 1 -- 172.21.15.83:0/1308550470 wait complete. 2024-03-11T19:45:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.033+0000 7f4372302640 1 Processor -- start 2024-03-11T19:45:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.033+0000 7f4372302640 1 -- start start 2024-03-11T19:45:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.033+0000 7f4372302640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f436c100080 0x7f436c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.033+0000 7f4372302640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c101270 0x7f436c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.033+0000 7f4372302640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c101c30 0x7f436c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.034+0000 7f4372302640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c1a0f90 con 0x7f436c101c30 2024-03-11T19:45:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.034+0000 7f4372302640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c1a1100 con 0x7f436c101270 2024-03-11T19:45:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.034+0000 7f4372302640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c1a12c0 con 0x7f436c100080 2024-03-11T19:45:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.034+0000 7f437089f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c101c30 0x7f436c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.034+0000 7f437089f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c101c30 0x7f436c1a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49468/0 (socket says 172.21.15.83:49468) 2024-03-11T19:45:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.034+0000 7f437089f640 1 -- 172.21.15.83:0/2046049840 learned_addr learned my addr 172.21.15.83:0/2046049840 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.038+0000 7f436b7fe640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c101270 0x7f436c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.039+0000 7f436bfff640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f436c100080 0x7f436c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.040+0000 7f437089f640 1 -- 172.21.15.83:0/2046049840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f436c100080 msgr2=0x7f436c1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.040+0000 7f437089f640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f436c100080 0x7f436c1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.040+0000 7f437089f640 1 -- 172.21.15.83:0/2046049840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c101270 msgr2=0x7f436c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.040+0000 7f437089f640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c101270 0x7f436c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.040+0000 7f437089f640 1 -- 172.21.15.83:0/2046049840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c009560 con 0x7f436c101c30 2024-03-11T19:45:07.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.049+0000 7f437089f640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c101c30 0x7f436c1a0a50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4364009fd0 tx=0x7f436400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:07.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.052+0000 7f43697fa640 1 -- 172.21.15.83:0/2046049840 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364003b70 con 0x7f436c101c30 2024-03-11T19:45:07.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.052+0000 7f43697fa640 1 -- 172.21.15.83:0/2046049840 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f436401a070 con 0x7f436c101c30 2024-03-11T19:45:07.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.053+0000 7f43697fa640 1 -- 172.21.15.83:0/2046049840 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364004630 con 0x7f436c101c30 2024-03-11T19:45:07.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.055+0000 7f4372302640 1 -- 172.21.15.83:0/2046049840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c191330 con 0x7f436c101c30 2024-03-11T19:45:07.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.055+0000 7f4372302640 1 -- 172.21.15.83:0/2046049840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c191760 con 0x7f436c101c30 2024-03-11T19:45:07.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.059+0000 7f4374fb1740 1 Processor -- start 2024-03-11T19:45:07.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:06 smithi083 ceph-mon[31864]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 804 op/s 2024-03-11T19:45:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.066+0000 7f4374fb1740 1 -- start start 2024-03-11T19:45:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.066+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a9398556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.066+0000 7f43697fa640 1 -- 172.21.15.83:0/2046049840 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4364003cd0 con 0x7f436c101c30 2024-03-11T19:45:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.066+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a939855cc0 0x55a93985a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.066+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a93985ae30 0x55a93985d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.066+0000 7f4374fb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9397c5280 con 0x55a939855cc0 2024-03-11T19:45:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.066+0000 7f4374fb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9397c27b0 con 0x55a9398552d0 2024-03-11T19:45:07.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.066+0000 7f4374fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9397bdf90 con 0x55a93985ae30 2024-03-11T19:45:07.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.067+0000 7f43697fa640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43540a1ae0 0x7f43540a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.067+0000 7f43697fa640 1 -- 172.21.15.83:0/2046049840 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f43640d5630 con 0x7f436c101c30 2024-03-11T19:45:07.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.069+0000 7f436bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a939855cc0 0x55a93985a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.069+0000 7f436bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a939855cc0 0x55a93985a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49470/0 (socket says 172.21.15.83:49470) 2024-03-11T19:45:07.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.069+0000 7f436bfff640 1 -- 172.21.15.83:0/846168230 learned_addr learned my addr 172.21.15.83:0/846168230 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:07.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.070+0000 7f436b7fe640 1 --2- 172.21.15.83:0/846168230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a9398556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.070+0000 7f436b7fe640 1 --2- 172.21.15.83:0/846168230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a93985ae30 0x55a93985d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.071+0000 7f436b7fe640 1 -- 172.21.15.83:0/846168230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a93985ae30 msgr2=0x55a93985d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:07.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.071+0000 7f436b7fe640 1 --2- 172.21.15.83:0/846168230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a93985ae30 0x55a93985d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.071+0000 7f436b7fe640 1 -- 172.21.15.83:0/846168230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a939855cc0 msgr2=0x55a93985a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:07.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.071+0000 7f436b7fe640 1 --2- 172.21.15.83:0/846168230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a939855cc0 0x55a93985a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.071+0000 7f436b7fe640 1 -- 172.21.15.83:0/846168230 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a93977c570 con 0x55a9398552d0 2024-03-11T19:45:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.072+0000 7f436b7fe640 1 --2- 172.21.15.83:0/846168230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a9398556f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f436c19a9b0 tx=0x7f43600095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83fddb650ea86d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.072+0000 7f43527fc640 1 -- 172.21.15.83:0/846168230 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360004030 con 0x55a9398552d0 2024-03-11T19:45:07.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.072+0000 7f43527fc640 1 -- 172.21.15.83:0/846168230 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4360005020 con 0x55a9398552d0 2024-03-11T19:45:07.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.073+0000 7f43527fc640 1 -- 172.21.15.83:0/846168230 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360004cd0 con 0x55a9398552d0 2024-03-11T19:45:07.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.073+0000 7f4374fb1740 1 -- 172.21.15.83:0/846168230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 msgr2=0x55a9398556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:07.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.073+0000 7f4374fb1740 1 --2- 172.21.15.83:0/846168230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a9398556f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f436c19a9b0 tx=0x7f43600095c0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.073+0000 7f4374fb1740 1 -- 172.21.15.83:0/846168230 shutdown_connections 2024-03-11T19:45:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.073+0000 7f4374fb1740 1 --2- 172.21.15.83:0/846168230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a93985ae30 0x55a93985d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.074+0000 7f4374fb1740 1 --2- 172.21.15.83:0/846168230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a939855cc0 0x55a93985a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.074+0000 7f4374fb1740 1 --2- 172.21.15.83:0/846168230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a9398556f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.074+0000 7f4374fb1740 1 -- 172.21.15.83:0/846168230 >> 172.21.15.83:0/846168230 conn(0x55a9398507f0 msgr2=0x55a939850bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:07.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.074+0000 7f4374fb1740 1 -- 172.21.15.83:0/846168230 shutdown_connections 2024-03-11T19:45:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.074+0000 7f4374fb1740 1 -- 172.21.15.83:0/846168230 wait complete. 2024-03-11T19:45:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.074+0000 7f4374fb1740 1 -- start start 2024-03-11T19:45:07.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.075+0000 7f4374fb1740 1 Processor -- start 2024-03-11T19:45:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.075+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9398552d0 0x55a9398702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.075+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a939870810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.075+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93985ae30 0x55a939877850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.075+0000 7f4374fb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a939491430 con 0x55a9398552d0 2024-03-11T19:45:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.075+0000 7f4374fb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93944de00 con 0x55a93985ae30 2024-03-11T19:45:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.075+0000 7f4374fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a939742fd0 con 0x55a939855cc0 2024-03-11T19:45:07.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.075+0000 7f4374fb1740 10 client.0 _ll_register_callbacks cb 0x55a939798fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:07.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.076+0000 7f436b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93985ae30 0x55a939877850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.076+0000 7f436b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93985ae30 0x55a939877850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47366/0 (socket says 172.21.15.83:47366) 2024-03-11T19:45:07.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.076+0000 7f436b7fe640 1 -- 172.21.15.83:0/1131120556 learned_addr learned my addr 172.21.15.83:0/1131120556 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:07.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.076+0000 7f436b7fe640 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9398552d0 0x55a9398702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.076+0000 7f436bfff640 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a939870810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.076+0000 7f436b7fe640 1 -- 172.21.15.83:0/1131120556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 msgr2=0x55a939870810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:07.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.076+0000 7f436b7fe640 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a939870810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.076+0000 7f436b7fe640 1 -- 172.21.15.83:0/1131120556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9398552d0 msgr2=0x55a9398702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:07.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.076+0000 7f436b7fe640 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9398552d0 0x55a9398702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.076+0000 7f436b7fe640 1 -- 172.21.15.83:0/1131120556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436001b040 con 0x55a93985ae30 2024-03-11T19:45:07.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.077+0000 7f437089f640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43540a1ae0 0x7f43540a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.079+0000 7f436b7fe640 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93985ae30 0x55a939877850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4360016020 tx=0x7f4360015f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a840022763a70cb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:07.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.079+0000 7f434b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:45:07.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.079+0000 7f434b7fe640 1 -- 172.21.15.83:0/1131120556 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000f540 con 0x55a93985ae30 2024-03-11T19:45:07.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.079+0000 7f434b7fe640 1 -- 172.21.15.83:0/1131120556 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f436000f6a0 con 0x55a93985ae30 2024-03-11T19:45:07.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.080+0000 7f4374fb1740 1 -- 172.21.15.83:0/1131120556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9394bd7e0 con 0x55a93985ae30 2024-03-11T19:45:07.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.081+0000 7f434b7fe640 1 -- 172.21.15.83:0/1131120556 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f436000f860 con 0x55a93985ae30 2024-03-11T19:45:07.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.081+0000 7f4374fb1740 10 client.25976 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:07.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.081+0000 7f4374fb1740 1 -- 172.21.15.83:0/1131120556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a9397bc610 con 0x55a93985ae30 2024-03-11T19:45:07.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.082+0000 7f434b7fe640 1 -- 172.21.15.83:0/1131120556 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a9394bd7e0 con 0x55a93985ae30 2024-03-11T19:45:07.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.082+0000 7f434b7fe640 1 -- 172.21.15.83:0/1131120556 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f4360020af0 con 0x55a93985ae30 2024-03-11T19:45:07.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.082+0000 7f4374fb1740 10 client.25976 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:07.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.082+0000 7f4374fb1740 10 client.25976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:07.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.082+0000 7f4374fb1740 20 client.25976 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:07.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.082+0000 7f4374fb1740 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a93987c2f0 0x55a93987e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.082+0000 7f4374fb1740 4 client.25976 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:07.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.082+0000 7f4374fb1740 1 -- 172.21.15.83:0/1131120556 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9394fa720 con 0x55a93987c2f0 2024-03-11T19:45:07.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.083+0000 7f436bfff640 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a93987c2f0 0x55a93987e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.084+0000 7f436bfff640 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a93987c2f0 0x55a93987e6e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e2e31ecfce86ec5 server_cookie=159b297c2514c9a8 in_seq=0 out_seq=0 2024-03-11T19:45:07.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.084+0000 7f434b7fe640 10 client.25976 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:07.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.086+0000 7f434b7fe640 1 -- 172.21.15.83:0/1131120556 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9394fa720 con 0x55a93987c2f0 2024-03-11T19:45:07.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.086+0000 7f434b7fe640 10 client.25976 handle_command_reply: tid=0 2024-03-11T19:45:07.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.086+0000 7f4374fb1740 1 client.25976 shutdown 2024-03-11T19:45:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.090+0000 7f4374fb1740 1 -- 172.21.15.83:0/1131120556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93985ae30 msgr2=0x55a939877850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:07.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.090+0000 7f4374fb1740 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93985ae30 0x55a939877850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4360016020 tx=0x7f4360015f90 comp rx=0 tx=0).stop 2024-03-11T19:45:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.095+0000 7f437089f640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43540a1ae0 0x7f43540a3fa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4364003fa0 tx=0x7f4364004000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.095+0000 7f4374fb1740 1 -- 172.21.15.83:0/1131120556 shutdown_connections 2024-03-11T19:45:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.095+0000 7f4374fb1740 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a93987c2f0 0x55a93987e6e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.095+0000 7f4374fb1740 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a93985ae30 0x55a939877850 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.095+0000 7f4374fb1740 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a939870810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.095+0000 7f4374fb1740 1 --2- 172.21.15.83:0/1131120556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9398552d0 0x55a9398702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.095+0000 7f4374fb1740 1 -- 172.21.15.83:0/1131120556 >> 172.21.15.83:0/1131120556 conn(0x55a9398507f0 msgr2=0x55a93984fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.096+0000 7f434b7fe640 0 client.25976 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:07.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.096+0000 7f4374fb1740 1 -- 172.21.15.83:0/1131120556 shutdown_connections 2024-03-11T19:45:07.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.096+0000 7f4374fb1740 1 -- 172.21.15.83:0/1131120556 wait complete. 2024-03-11T19:45:07.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.096+0000 7f4374fb1740 20 client.25976 trim_cache size 0 max 16384 2024-03-11T19:45:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f4374fb1740 1 Processor -- start 2024-03-11T19:45:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f4374fb1740 1 -- start start 2024-03-11T19:45:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a939856f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a9398574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a93985ae30 0x55a939877770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f4374fb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9394bd7e0 con 0x55a93985ae30 2024-03-11T19:45:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f4374fb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a939877cb0 con 0x55a9398552d0 2024-03-11T19:45:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f4374fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a939871450 con 0x55a939855cc0 2024-03-11T19:45:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f436b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a93985ae30 0x55a939877770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f436b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a93985ae30 0x55a939877770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49490/0 (socket says 172.21.15.83:49490) 2024-03-11T19:45:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f436b7fe640 1 -- 172.21.15.83:0/1736730785 learned_addr learned my addr 172.21.15.83:0/1736730785 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f436bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a9398574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.120+0000 7f436b7fe640 1 --2- 172.21.15.83:0/1736730785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a939856f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.121+0000 7f436bfff640 1 -- 172.21.15.83:0/1736730785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 msgr2=0x55a939856f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.121+0000 7f436bfff640 1 --2- 172.21.15.83:0/1736730785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a939856f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.121+0000 7f436bfff640 1 -- 172.21.15.83:0/1736730785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a93985ae30 msgr2=0x55a939877770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.121+0000 7f436bfff640 1 --2- 172.21.15.83:0/1736730785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a93985ae30 0x55a939877770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.121+0000 7f436bfff640 1 -- 172.21.15.83:0/1736730785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a939758ac0 con 0x55a939855cc0 2024-03-11T19:45:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.121+0000 7f436bfff640 1 --2- 172.21.15.83:0/1736730785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a9398574a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f435c000c00 tx=0x7f435c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dca0a7e6fc5f3092 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.121+0000 7f4351ffb640 1 -- 172.21.15.83:0/1736730785 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c004040 con 0x55a939855cc0 2024-03-11T19:45:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.122+0000 7f4351ffb640 1 -- 172.21.15.83:0/1736730785 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f435c02c7e0 con 0x55a939855cc0 2024-03-11T19:45:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.122+0000 7f4351ffb640 1 -- 172.21.15.83:0/1736730785 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c0468d0 con 0x55a939855cc0 2024-03-11T19:45:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.122+0000 7f4374fb1740 1 -- 172.21.15.83:0/1736730785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 msgr2=0x55a9398574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.122+0000 7f4374fb1740 1 --2- 172.21.15.83:0/1736730785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a9398574a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f435c000c00 tx=0x7f435c005fd0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.122+0000 7f4374fb1740 1 -- 172.21.15.83:0/1736730785 shutdown_connections 2024-03-11T19:45:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.122+0000 7f4374fb1740 1 --2- 172.21.15.83:0/1736730785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a93985ae30 0x55a939877770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.122+0000 7f4374fb1740 1 --2- 172.21.15.83:0/1736730785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a9398574a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.122+0000 7f4374fb1740 1 --2- 172.21.15.83:0/1736730785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a939856f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.122+0000 7f4374fb1740 1 -- 172.21.15.83:0/1736730785 >> 172.21.15.83:0/1736730785 conn(0x55a9398507f0 msgr2=0x55a9398702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.122+0000 7f4374fb1740 1 -- 172.21.15.83:0/1736730785 shutdown_connections 2024-03-11T19:45:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.122+0000 7f4374fb1740 1 -- 172.21.15.83:0/1736730785 wait complete. 2024-03-11T19:45:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f4374fb1740 1 -- start start 2024-03-11T19:45:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f4374fb1740 1 Processor -- start 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a939876a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a939876f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f4374fb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a93985ae30 0x55a9398d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f4374fb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9398d2f30 con 0x55a93985ae30 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f4374fb1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9398d3070 con 0x55a9398552d0 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f4374fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9398d3200 con 0x55a939855cc0 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f4374fb1740 10 client.0 _ll_register_callbacks cb 0x7f4360009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f436b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a93985ae30 0x55a9398d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f436b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a939876a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f436bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a939876f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f436b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a939876a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47388/0 (socket says 172.21.15.83:47388) 2024-03-11T19:45:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.123+0000 7f436b7fe640 1 -- 172.21.15.83:0/3886740882 learned_addr learned my addr 172.21.15.83:0/3886740882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:07.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.124+0000 7f436bfff640 1 -- 172.21.15.83:0/3886740882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 msgr2=0x55a939876a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:07.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.124+0000 7f436bfff640 1 --2- 172.21.15.83:0/3886740882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a939876a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.124+0000 7f436bfff640 1 -- 172.21.15.83:0/3886740882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a93985ae30 msgr2=0x55a9398d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:07.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.124+0000 7f436bfff640 1 --2- 172.21.15.83:0/3886740882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a93985ae30 0x55a9398d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.124+0000 7f436bfff640 1 -- 172.21.15.83:0/3886740882 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c108dc0 con 0x55a939855cc0 2024-03-11T19:45:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.125+0000 7f436bfff640 1 --2- 172.21.15.83:0/3886740882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a939876f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f435c009230 tx=0x7f435c004940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc591db74bfaf00e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.125+0000 7f434b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:45:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f434b7fe640 1 -- 172.21.15.83:0/3886740882 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c004ab0 con 0x55a939855cc0 2024-03-11T19:45:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f434b7fe640 1 -- 172.21.15.83:0/3886740882 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f435c043040 con 0x55a939855cc0 2024-03-11T19:45:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f4374fb1740 1 -- 172.21.15.83:0/3886740882 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4360005020 con 0x55a939855cc0 2024-03-11T19:45:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f434b7fe640 1 -- 172.21.15.83:0/3886740882 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f435c046860 con 0x55a939855cc0 2024-03-11T19:45:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f4374fb1740 10 client.26077 fetch_fsmap learned FSMap version 29 2024-03-11T19:45:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f4374fb1740 1 -- 172.21.15.83:0/3886740882 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a93977e6a0 con 0x55a939855cc0 2024-03-11T19:45:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f434b7fe640 1 -- 172.21.15.83:0/3886740882 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4360005020 con 0x55a939855cc0 2024-03-11T19:45:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f434b7fe640 1 -- 172.21.15.83:0/3886740882 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f435c003790 con 0x55a939855cc0 2024-03-11T19:45:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f4374fb1740 10 client.26077 fetch_fsmap finished waiting for FSMap version 29 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f4374fb1740 10 client.26077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f4374fb1740 20 client.26077 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f4374fb1740 1 --2- 172.21.15.83:0/3886740882 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9398d4970 0x55a9398d6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f4374fb1740 4 client.26077 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f4374fb1740 1 -- 172.21.15.83:0/3886740882 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a939549890 con 0x55a9398d4970 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.126+0000 7f436b7fe640 1 --2- 172.21.15.83:0/3886740882 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9398d4970 0x55a9398d6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.127+0000 7f436b7fe640 1 --2- 172.21.15.83:0/3886740882 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9398d4970 0x55a9398d6d90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d5bc1c390dbe79 server_cookie=b1ca631d753b8296 in_seq=0 out_seq=0 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.127+0000 7f434b7fe640 10 client.26077 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.127+0000 7f434b7fe640 1 -- 172.21.15.83:0/3886740882 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a939549890 con 0x55a9398d4970 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.127+0000 7f434b7fe640 10 client.26077 handle_command_reply: tid=0 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.127+0000 7f4374fb1740 1 client.26077 shutdown 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f4374fb1740 1 -- 172.21.15.83:0/3886740882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 msgr2=0x55a939876f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f4374fb1740 1 --2- 172.21.15.83:0/3886740882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a939876f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f435c009230 tx=0x7f435c004940 comp rx=0 tx=0).stop 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f4374fb1740 1 -- 172.21.15.83:0/3886740882 shutdown_connections 2024-03-11T19:45:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f4374fb1740 1 --2- 172.21.15.83:0/3886740882 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9398d4970 0x55a9398d6d90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f434b7fe640 0 client.26077 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f4374fb1740 1 --2- 172.21.15.83:0/3886740882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a93985ae30 0x55a9398d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f4374fb1740 1 --2- 172.21.15.83:0/3886740882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a939855cc0 0x55a939876f80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f4374fb1740 1 --2- 172.21.15.83:0/3886740882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9398552d0 0x55a939876a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f4374fb1740 1 -- 172.21.15.83:0/3886740882 >> 172.21.15.83:0/3886740882 conn(0x55a9398507f0 msgr2=0x55a9398701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f4374fb1740 1 -- 172.21.15.83:0/3886740882 shutdown_connections 2024-03-11T19:45:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f4374fb1740 1 -- 172.21.15.83:0/3886740882 wait complete. 2024-03-11T19:45:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.128+0000 7f4374fb1740 20 client.26077 trim_cache size 0 max 16384 2024-03-11T19:45:07.136 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:45:07.136 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:45:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:45:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "957c1fd1-7475-421a-8d63-6d499ff639e3": { 2024-03-11T19:45:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:45:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "957c1fd1-7475-421a-8d63-6d499ff639e3", 2024-03-11T19:45:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:45:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:45:07.137 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:45:07.137 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:45:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 -- 172.21.15.83:0/2046049840 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43540a1ae0 msgr2=0x7f43540a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43540a1ae0 0x7f43540a3fa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4364003fa0 tx=0x7f4364004000 comp rx=0 tx=0).stop 2024-03-11T19:45:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 -- 172.21.15.83:0/2046049840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c101c30 msgr2=0x7f436c1a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c101c30 0x7f436c1a0a50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4364009fd0 tx=0x7f436400b260 comp rx=0 tx=0).stop 2024-03-11T19:45:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 -- 172.21.15.83:0/2046049840 shutdown_connections 2024-03-11T19:45:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f43540a1ae0 0x7f43540a3fa0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f436c101c30 0x7f436c1a0a50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f436c101270 0x7f436c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 --2- 172.21.15.83:0/2046049840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f436c100080 0x7f436c1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:07.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 -- 172.21.15.83:0/2046049840 >> 172.21.15.83:0/2046049840 conn(0x7f436c0fb5f0 msgr2=0x7f436c0fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:07.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 -- 172.21.15.83:0/2046049840 shutdown_connections 2024-03-11T19:45:07.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:07.129+0000 7f4372302640 1 -- 172.21.15.83:0/2046049840 wait complete. 2024-03-11T19:45:07.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:06 smithi137 ceph-mon[36220]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 804 op/s 2024-03-11T19:45:07.146 INFO:tasks.cephfs.filesystem:scrub status for tag:957c1fd1-7475-421a-8d63-6d499ff639e3 - {'path': '/', 'tag': '957c1fd1-7475-421a-8d63-6d499ff639e3', 'options': 'recursive,force'} 2024-03-11T19:45:07.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:06 smithi204 ceph-mon[36739]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 804 op/s 2024-03-11T19:45:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 350.0 s, 2541.8 tps, lat 39.114 ms stddev 44.426 2024-03-11T19:45:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:07 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:45:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:07 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:07 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:45:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:07 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:45:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:07 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:07 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:45:08.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:07 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:45:08.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:07 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:07 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:07 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:07 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:45:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:07 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:07 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:45:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:07 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:45:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:07 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:07 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:45:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:07 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:45:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:07 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:07 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:07 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:07 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:45:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:07 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:07 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:45:08.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:07 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:45:08.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:07 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:07 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:45:08.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:07 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:45:08.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:07 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:07 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:08.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:07 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:08 smithi083 ceph-mon[31864]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 819 op/s 2024-03-11T19:45:09.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:08 smithi137 ceph-mon[36220]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 819 op/s 2024-03-11T19:45:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:08 smithi204 ceph-mon[36739]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 819 op/s 2024-03-11T19:45:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:10 smithi083 ceph-mon[31864]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 722 op/s 2024-03-11T19:45:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:10 smithi137 ceph-mon[36220]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 722 op/s 2024-03-11T19:45:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:10 smithi204 ceph-mon[36739]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.0 MiB/s wr, 722 op/s 2024-03-11T19:45:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 355.0 s, 3878.4 tps, lat 26.029 ms stddev 30.455 2024-03-11T19:45:13.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:12 smithi083 ceph-mon[31864]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.4 MiB/s wr, 716 op/s 2024-03-11T19:45:13.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:12 smithi137 ceph-mon[36220]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.4 MiB/s wr, 716 op/s 2024-03-11T19:45:13.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:12 smithi204 ceph-mon[36739]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.4 MiB/s wr, 716 op/s 2024-03-11T19:45:15.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:15 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:45:15.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:15 smithi083 ceph-mon[31864]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 7.0 MiB/s wr, 720 op/s 2024-03-11T19:45:15.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:15 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:45:15.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:15 smithi137 ceph-mon[36220]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 7.0 MiB/s wr, 720 op/s 2024-03-11T19:45:15.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:15 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:45:15.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:15 smithi204 ceph-mon[36739]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 7.0 MiB/s wr, 720 op/s 2024-03-11T19:45:16.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:16 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:45:16.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:16 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:45:16.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:16 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:45:16.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:16 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:45:16.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:16 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:45:16.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:16 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:45:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:16 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:45:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:16 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:45:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:16 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:45:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 360.0 s, 2616.0 tps, lat 38.052 ms stddev 40.755 2024-03-11T19:45:17.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:17 smithi083 ceph-mon[31864]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8.8 MiB/s wr, 803 op/s 2024-03-11T19:45:17.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:17 smithi137 ceph-mon[36220]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8.8 MiB/s wr, 803 op/s 2024-03-11T19:45:17.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:17 smithi204 ceph-mon[36739]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8.8 MiB/s wr, 803 op/s 2024-03-11T19:45:19.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:18 smithi083 ceph-mon[31864]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.1 MiB/s wr, 781 op/s 2024-03-11T19:45:19.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:18 smithi137 ceph-mon[36220]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.1 MiB/s wr, 781 op/s 2024-03-11T19:45:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:18 smithi204 ceph-mon[36739]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.1 MiB/s wr, 781 op/s 2024-03-11T19:45:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:20 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:45:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:20 smithi083 ceph-mon[31864]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 10 MiB/s wr, 750 op/s 2024-03-11T19:45:21.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:20 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:45:21.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:20 smithi137 ceph-mon[36220]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 10 MiB/s wr, 750 op/s 2024-03-11T19:45:21.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:20 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:45:21.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:20 smithi204 ceph-mon[36739]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 10 MiB/s wr, 750 op/s 2024-03-11T19:45:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 365.0 s, 2645.4 tps, lat 37.777 ms stddev 42.281 2024-03-11T19:45:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:22 smithi083 ceph-mon[31864]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 839 op/s 2024-03-11T19:45:23.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:22 smithi137 ceph-mon[36220]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 839 op/s 2024-03-11T19:45:23.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:22 smithi204 ceph-mon[36739]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 839 op/s 2024-03-11T19:45:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:24 smithi083 ceph-mon[31864]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 884 op/s 2024-03-11T19:45:25.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:24 smithi137 ceph-mon[36220]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 884 op/s 2024-03-11T19:45:25.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:24 smithi204 ceph-mon[36739]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 MiB/s wr, 884 op/s 2024-03-11T19:45:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:26 smithi083 ceph-mon[31864]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 874 op/s 2024-03-11T19:45:27.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:26 smithi137 ceph-mon[36220]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 874 op/s 2024-03-11T19:45:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:26 smithi204 ceph-mon[36739]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 874 op/s 2024-03-11T19:45:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 370.0 s, 2874.6 tps, lat 34.682 ms stddev 40.582 2024-03-11T19:45:29.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:28 smithi083 ceph-mon[31864]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 847 op/s 2024-03-11T19:45:29.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:28 smithi137 ceph-mon[36220]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 847 op/s 2024-03-11T19:45:29.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:28 smithi204 ceph-mon[36739]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 23 MiB/s wr, 847 op/s 2024-03-11T19:45:30.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:30 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:30.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:30 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:30.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:30 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:45:31.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:31 smithi137 ceph-mon[36220]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 768 op/s 2024-03-11T19:45:31.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:31 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:31 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:31 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:31 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:31 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:31.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:31 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:31.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:31 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:45:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:31 smithi204 ceph-mon[36739]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 768 op/s 2024-03-11T19:45:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:31 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:31 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:31 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:31 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:31 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:31 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:31.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:31 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:45:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:31 smithi083 ceph-mon[31864]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 MiB/s wr, 768 op/s 2024-03-11T19:45:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:45:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:45:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:45:31.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:45:32.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:32 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-03-11T19:45:32.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:32 smithi137 ceph-mon[36220]: Unable to set osd_memory_target on smithi083 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-03-11T19:45:32.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:32 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-03-11T19:45:32.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:32 smithi204 ceph-mon[36739]: Unable to set osd_memory_target on smithi083 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-03-11T19:45:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 375.0 s, 2809.4 tps, lat 35.703 ms stddev 40.896 2024-03-11T19:45:32.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:32 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-03-11T19:45:32.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:32 smithi083 ceph-mon[31864]: Unable to set osd_memory_target on smithi083 to 184783872: error parsing value: Value '184783872' is below minimum 939524096 2024-03-11T19:45:33.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:33 smithi137 ceph-mon[36220]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 787 op/s 2024-03-11T19:45:33.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:33 smithi204 ceph-mon[36739]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 787 op/s 2024-03-11T19:45:33.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:33 smithi083 ceph-mon[31864]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 23 MiB/s wr, 787 op/s 2024-03-11T19:45:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:34 smithi083 ceph-mon[31864]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 23 MiB/s wr, 811 op/s 2024-03-11T19:45:35.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:34 smithi137 ceph-mon[36220]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 23 MiB/s wr, 811 op/s 2024-03-11T19:45:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:34 smithi204 ceph-mon[36739]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 23 MiB/s wr, 811 op/s 2024-03-11T19:45:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:36 smithi083 ceph-mon[31864]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 21 MiB/s wr, 767 op/s 2024-03-11T19:45:37.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:36 smithi137 ceph-mon[36220]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 21 MiB/s wr, 767 op/s 2024-03-11T19:45:37.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:45:37.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:45:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:36 smithi204 ceph-mon[36739]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 21 MiB/s wr, 767 op/s 2024-03-11T19:45:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.304+0000 7fbecf440640 1 -- 172.21.15.83:0/100625445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec8106ea0 msgr2=0x7fbec81092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.304+0000 7fbecf440640 1 --2- 172.21.15.83:0/100625445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec8106ea0 0x7fbec81092f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009ca0 tx=0x7fbec002c100 comp rx=0 tx=0).stop 2024-03-11T19:45:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.304+0000 7fbecf440640 1 -- 172.21.15.83:0/100625445 shutdown_connections 2024-03-11T19:45:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.304+0000 7fbecf440640 1 --2- 172.21.15.83:0/100625445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec8106ea0 0x7fbec81092f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.304+0000 7fbecf440640 1 --2- 172.21.15.83:0/100625445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec8104570 0x7fbec8106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.304+0000 7fbecf440640 1 --2- 172.21.15.83:0/100625445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbec8101c40 0x7fbec8104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.304+0000 7fbecf440640 1 -- 172.21.15.83:0/100625445 >> 172.21.15.83:0/100625445 conn(0x7fbec80fb650 msgr2=0x7fbec80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.304+0000 7fbecf440640 1 -- 172.21.15.83:0/100625445 shutdown_connections 2024-03-11T19:45:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.305+0000 7fbecf440640 1 -- 172.21.15.83:0/100625445 wait complete. 2024-03-11T19:45:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.305+0000 7fbecf440640 1 Processor -- start 2024-03-11T19:45:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.305+0000 7fbecf440640 1 -- start start 2024-03-11T19:45:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.306+0000 7fbecf440640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbec8101c40 0x7fbec8196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.306+0000 7fbecf440640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec8104570 0x7fbec8197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.306+0000 7fbecf440640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec8106ea0 0x7fbec819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.306+0000 7fbecf440640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec819ea30 con 0x7fbec8106ea0 2024-03-11T19:45:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.306+0000 7fbecf440640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec819eba0 con 0x7fbec8101c40 2024-03-11T19:45:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.306+0000 7fbecf440640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec819ed60 con 0x7fbec8104570 2024-03-11T19:45:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.306+0000 7fbecc9db640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec8104570 0x7fbec8197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.306+0000 7fbecc9db640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec8104570 0x7fbec8197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:54910/0 (socket says 172.21.15.83:54910) 2024-03-11T19:45:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.306+0000 7fbecc9db640 1 -- 172.21.15.83:0/26235784 learned_addr learned my addr 172.21.15.83:0/26235784 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.307+0000 7fbecc9db640 1 -- 172.21.15.83:0/26235784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbec8101c40 msgr2=0x7fbec8196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.307+0000 7fbecc9db640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbec8101c40 0x7fbec8196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.307+0000 7fbecc9db640 1 -- 172.21.15.83:0/26235784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec8106ea0 msgr2=0x7fbec819e4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:45:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.307+0000 7fbecc9db640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec8106ea0 0x7fbec819e4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.307+0000 7fbecc9db640 1 -- 172.21.15.83:0/26235784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb8009490 con 0x7fbec8104570 2024-03-11T19:45:37.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.311+0000 7fbecc9db640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec8104570 0x7fbec8197480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800e9a0 tx=0x7fbeb800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.313+0000 7fbebe7fc640 1 -- 172.21.15.83:0/26235784 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800cd80 con 0x7fbec8104570 2024-03-11T19:45:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.313+0000 7fbebe7fc640 1 -- 172.21.15.83:0/26235784 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbeb80035f0 con 0x7fbec8104570 2024-03-11T19:45:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.313+0000 7fbebe7fc640 1 -- 172.21.15.83:0/26235784 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8010690 con 0x7fbec8104570 2024-03-11T19:45:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.313+0000 7fbecf440640 1 -- 172.21.15.83:0/26235784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec0009900 con 0x7fbec8104570 2024-03-11T19:45:37.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.313+0000 7fbecf440640 1 -- 172.21.15.83:0/26235784 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec819f280 con 0x7fbec8104570 2024-03-11T19:45:37.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.320+0000 7fbebe7fc640 1 -- 172.21.15.83:0/26235784 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fbeb8003760 con 0x7fbec8104570 2024-03-11T19:45:37.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.322+0000 7fbed2014740 1 Processor -- start 2024-03-11T19:45:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.323+0000 7fbed2014740 1 -- start start 2024-03-11T19:45:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.323+0000 7fbed2014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fb52e0 0x55e6a8fb5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.323+0000 7fbed2014740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fba7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.323+0000 7fbed2014740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fbae40 0x55e6a8fbd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.323+0000 7fbed2014740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8f14280 con 0x55e6a8fb52e0 2024-03-11T19:45:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.323+0000 7fbed2014740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8f117b0 con 0x55e6a8fbae40 2024-03-11T19:45:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.323+0000 7fbed2014740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8f0cf90 con 0x55e6a8fb5cd0 2024-03-11T19:45:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.323+0000 7fbebe7fc640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbeb40a1c00 0x7fbeb40a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.324+0000 7fbebe7fc640 1 -- 172.21.15.83:0/26235784 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fbeb8014070 con 0x7fbec8104570 2024-03-11T19:45:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.324+0000 7fbecd9dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fb52e0 0x55e6a8fb5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.324+0000 7fbecd9dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fb52e0 0x55e6a8fb5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34396/0 (socket says 172.21.15.83:34396) 2024-03-11T19:45:37.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.324+0000 7fbecd9dd640 1 -- 172.21.15.83:0/2647035305 learned_addr learned my addr 172.21.15.83:0/2647035305 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.326+0000 7fbecc9db640 1 --2- 172.21.15.83:0/2647035305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fbae40 0x55e6a8fbd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.328+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2647035305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fba7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.328+0000 7fbecd9dd640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbeb40a1c00 0x7fbeb40a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.329+0000 7fbecd1dc640 1 -- 172.21.15.83:0/2647035305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fbae40 msgr2=0x55e6a8fbd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:37.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.329+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2647035305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fbae40 0x55e6a8fbd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.329+0000 7fbecd1dc640 1 -- 172.21.15.83:0/2647035305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fb52e0 msgr2=0x55e6a8fb5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:37.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.329+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2647035305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fb52e0 0x55e6a8fb5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.329+0000 7fbecd1dc640 1 -- 172.21.15.83:0/2647035305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6a8ecb570 con 0x55e6a8fb5cd0 2024-03-11T19:45:37.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.334+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2647035305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fba7d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbec400a9e0 tx=0x7fbec400aea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d12382019fce84f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:37.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.335+0000 7fbecd9dd640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbeb40a1c00 0x7fbeb40a40c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbec0004af0 tx=0x7fbec00093a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.336+0000 7fbeb37fe640 1 -- 172.21.15.83:0/2647035305 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec400cdb0 con 0x55e6a8fb5cd0 2024-03-11T19:45:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.336+0000 7fbeb37fe640 1 -- 172.21.15.83:0/2647035305 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbec4011c30 con 0x55e6a8fb5cd0 2024-03-11T19:45:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.336+0000 7fbeb37fe640 1 -- 172.21.15.83:0/2647035305 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4019c90 con 0x55e6a8fb5cd0 2024-03-11T19:45:37.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.339+0000 7fbed2014740 1 -- 172.21.15.83:0/2647035305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb5cd0 msgr2=0x55e6a8fba7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.339+0000 7fbed2014740 1 --2- 172.21.15.83:0/2647035305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fba7d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbec400a9e0 tx=0x7fbec400aea0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.341+0000 7fbed2014740 1 -- 172.21.15.83:0/2647035305 shutdown_connections 2024-03-11T19:45:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.341+0000 7fbed2014740 1 --2- 172.21.15.83:0/2647035305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fbae40 0x55e6a8fbd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.341+0000 7fbed2014740 1 --2- 172.21.15.83:0/2647035305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fba7d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.341+0000 7fbed2014740 1 --2- 172.21.15.83:0/2647035305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fb52e0 0x55e6a8fb5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.341+0000 7fbed2014740 1 -- 172.21.15.83:0/2647035305 >> 172.21.15.83:0/2647035305 conn(0x55e6a8f9f7f0 msgr2=0x55e6a8f9fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.342+0000 7fbed2014740 1 -- 172.21.15.83:0/2647035305 shutdown_connections 2024-03-11T19:45:37.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.345+0000 7fbed2014740 1 -- 172.21.15.83:0/2647035305 wait complete. 2024-03-11T19:45:37.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.345+0000 7fbed2014740 1 -- start start 2024-03-11T19:45:37.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.345+0000 7fbed2014740 1 Processor -- start 2024-03-11T19:45:37.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.350+0000 7fbed2014740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb52e0 0x55e6a8fc7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.350+0000 7fbed2014740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fc8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.350+0000 7fbed2014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.350+0000 7fbed2014740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8be0430 con 0x55e6a8fbae40 2024-03-11T19:45:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.350+0000 7fbed2014740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8b9ce00 con 0x55e6a8fb5cd0 2024-03-11T19:45:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.350+0000 7fbed2014740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8e91fd0 con 0x55e6a8fb52e0 2024-03-11T19:45:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.350+0000 7fbed2014740 10 client.0 _ll_register_callbacks cb 0x55e6a8ee7fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.353+0000 7fbecd9dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.353+0000 7fbecd9dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34406/0 (socket says 172.21.15.83:34406) 2024-03-11T19:45:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.353+0000 7fbecd9dd640 1 -- 172.21.15.83:0/2018849014 learned_addr learned my addr 172.21.15.83:0/2018849014 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:37.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.355+0000 7fbecd9dd640 1 -- 172.21.15.83:0/2018849014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb52e0 msgr2=0x55e6a8fc7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:37.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.355+0000 7fbecd9dd640 1 --2- 172.21.15.83:0/2018849014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb52e0 0x55e6a8fc7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.355+0000 7fbecd9dd640 1 -- 172.21.15.83:0/2018849014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fb5cd0 msgr2=0x55e6a8fc8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:37.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.355+0000 7fbecd9dd640 1 --2- 172.21.15.83:0/2018849014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fc8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.355+0000 7fbecd9dd640 1 -- 172.21.15.83:0/2018849014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec400a610 con 0x55e6a8fbae40 2024-03-11T19:45:37.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.355+0000 7fbecd9dd640 1 --2- 172.21.15.83:0/2018849014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcf0b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbec002c640 tx=0x7fbec0044040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf2c76330382a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:37.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.355+0000 7fbeb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:45:37.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.356+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2018849014 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0003c40 con 0x55e6a8fbae40 2024-03-11T19:45:37.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.356+0000 7fbed2014740 1 -- 172.21.15.83:0/2018849014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6a8c0c7e0 con 0x55e6a8fbae40 2024-03-11T19:45:37.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.356+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2018849014 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fbec0003da0 con 0x55e6a8fbae40 2024-03-11T19:45:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.356+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2018849014 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbec0004b40 con 0x55e6a8fbae40 2024-03-11T19:45:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.356+0000 7fbed2014740 10 client.16524 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.356+0000 7fbed2014740 1 -- 172.21.15.83:0/2018849014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e6a8f0b610 con 0x55e6a8fbae40 2024-03-11T19:45:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.357+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2018849014 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e6a8c0c7e0 con 0x55e6a8fbae40 2024-03-11T19:45:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.357+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2018849014 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fbec004a070 con 0x55e6a8fbae40 2024-03-11T19:45:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.357+0000 7fbed2014740 10 client.16524 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.357+0000 7fbed2014740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.357+0000 7fbed2014740 20 client.16524 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:37.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.357+0000 7fbed2014740 1 --2- 172.21.15.83:0/2018849014 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e6a8fd3f40 0x55e6a8fd6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.357+0000 7fbed2014740 4 client.16524 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:37.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.357+0000 7fbed2014740 1 -- 172.21.15.83:0/2018849014 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6a8c49720 con 0x55e6a8fd3f40 2024-03-11T19:45:37.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.361+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2018849014 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e6a8fd3f40 0x55e6a8fd6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.371+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2018849014 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e6a8fd3f40 0x55e6a8fd6400 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab43555276229e36 server_cookie=2271f32dc4613a48 in_seq=0 out_seq=0 2024-03-11T19:45:37.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.371+0000 7fbeb0ff9640 10 client.16524 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.372+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2018849014 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e6a8c49720 con 0x55e6a8fd3f40 2024-03-11T19:45:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.372+0000 7fbeb0ff9640 10 client.16524 handle_command_reply: tid=0 2024-03-11T19:45:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.372+0000 7fbed2014740 1 client.16524 shutdown 2024-03-11T19:45:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.374+0000 7fbed2014740 1 -- 172.21.15.83:0/2018849014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 msgr2=0x55e6a8fcf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.374+0000 7fbed2014740 1 --2- 172.21.15.83:0/2018849014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcf0b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbec002c640 tx=0x7fbec0044040 comp rx=0 tx=0).stop 2024-03-11T19:45:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.375+0000 7fbed2014740 1 -- 172.21.15.83:0/2018849014 shutdown_connections 2024-03-11T19:45:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.375+0000 7fbed2014740 1 --2- 172.21.15.83:0/2018849014 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e6a8fd3f40 0x55e6a8fd6400 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.375+0000 7fbed2014740 1 --2- 172.21.15.83:0/2018849014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcf0b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.375+0000 7fbed2014740 1 --2- 172.21.15.83:0/2018849014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fc8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.375+0000 7fbed2014740 1 --2- 172.21.15.83:0/2018849014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb52e0 0x55e6a8fc7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.375+0000 7fbed2014740 1 -- 172.21.15.83:0/2018849014 >> 172.21.15.83:0/2018849014 conn(0x55e6a8f9f7f0 msgr2=0x55e6a8f9ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.375+0000 7fbeb0ff9640 0 client.16524 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.375+0000 7fbed2014740 1 -- 172.21.15.83:0/2018849014 shutdown_connections 2024-03-11T19:45:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.376+0000 7fbed2014740 1 -- 172.21.15.83:0/2018849014 wait complete. 2024-03-11T19:45:37.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.376+0000 7fbed2014740 20 client.16524 trim_cache size 0 max 16384 2024-03-11T19:45:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.407+0000 7fbed2014740 1 Processor -- start 2024-03-11T19:45:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.407+0000 7fbed2014740 1 -- start start 2024-03-11T19:45:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.407+0000 7fbed2014740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb52e0 0x55e6a8f9e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.407+0000 7fbed2014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fcefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.408+0000 7fbecd1dc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb52e0 0x55e6a8f9e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.408+0000 7fbecd1dc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb52e0 0x55e6a8f9e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:54940/0 (socket says 172.21.15.83:54940) 2024-03-11T19:45:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.408+0000 7fbecd1dc640 1 -- 172.21.15.83:0/2952653872 learned_addr learned my addr 172.21.15.83:0/2952653872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.408+0000 7fbed2014740 1 --2- 172.21.15.83:0/2952653872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fbae40 0x55e6a8f9e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.408+0000 7fbed2014740 1 -- 172.21.15.83:0/2952653872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8c49720 con 0x55e6a8fb5cd0 2024-03-11T19:45:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.408+0000 7fbed2014740 1 -- 172.21.15.83:0/2952653872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8c0c7e0 con 0x55e6a8fbae40 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.408+0000 7fbed2014740 1 -- 172.21.15.83:0/2952653872 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8fc8d20 con 0x55e6a8fb52e0 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.408+0000 7fbecd9dd640 1 --2- 172.21.15.83:0/2952653872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fbae40 0x55e6a8f9e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.409+0000 7fbecc9db640 1 --2- 172.21.15.83:0/2952653872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fcefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.409+0000 7fbecd1dc640 1 -- 172.21.15.83:0/2952653872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fbae40 msgr2=0x55e6a8f9e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.409+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2952653872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fbae40 0x55e6a8f9e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.409+0000 7fbecd1dc640 1 -- 172.21.15.83:0/2952653872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fb5cd0 msgr2=0x55e6a8fcefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.409+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2952653872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fcefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.409+0000 7fbecd1dc640 1 -- 172.21.15.83:0/2952653872 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6a8ea7ac0 con 0x55e6a8fb52e0 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.409+0000 7fbecd9dd640 1 --2- 172.21.15.83:0/2952653872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fbae40 0x55e6a8f9e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.409+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2952653872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb52e0 0x55e6a8f9e1c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbec40195c0 tx=0x7fbec4019440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ecd3b1fac953af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.409+0000 7fbeb2ffd640 1 -- 172.21.15.83:0/2952653872 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec400cbb0 con 0x55e6a8fb52e0 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.409+0000 7fbeb2ffd640 1 -- 172.21.15.83:0/2952653872 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbec400cd90 con 0x55e6a8fb52e0 2024-03-11T19:45:37.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.409+0000 7fbeb2ffd640 1 -- 172.21.15.83:0/2952653872 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4005db0 con 0x55e6a8fb52e0 2024-03-11T19:45:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.410+0000 7fbed2014740 1 -- 172.21.15.83:0/2952653872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb52e0 msgr2=0x55e6a8f9e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.410+0000 7fbed2014740 1 --2- 172.21.15.83:0/2952653872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb52e0 0x55e6a8f9e1c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbec40195c0 tx=0x7fbec4019440 comp rx=0 tx=0).stop 2024-03-11T19:45:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.410+0000 7fbed2014740 1 -- 172.21.15.83:0/2952653872 shutdown_connections 2024-03-11T19:45:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.411+0000 7fbed2014740 1 --2- 172.21.15.83:0/2952653872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fbae40 0x55e6a8f9e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.411+0000 7fbed2014740 1 --2- 172.21.15.83:0/2952653872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fcefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.411+0000 7fbed2014740 1 --2- 172.21.15.83:0/2952653872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fb52e0 0x55e6a8f9e1c0 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbec40195c0 tx=0x7fbec4019440 comp rx=0 tx=0).stop 2024-03-11T19:45:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.411+0000 7fbed2014740 1 -- 172.21.15.83:0/2952653872 >> 172.21.15.83:0/2952653872 conn(0x55e6a8f9f7f0 msgr2=0x55e6a8fb6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.411+0000 7fbed2014740 1 -- 172.21.15.83:0/2952653872 shutdown_connections 2024-03-11T19:45:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.412+0000 7fbed2014740 1 -- 172.21.15.83:0/2952653872 wait complete. 2024-03-11T19:45:37.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.412+0000 7fbed2014740 1 -- start start 2024-03-11T19:45:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.412+0000 7fbed2014740 1 Processor -- start 2024-03-11T19:45:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.412+0000 7fbed2014740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.412+0000 7fbed2014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbed2014740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fcce80 0x55e6a8fcd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbed2014740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8fcd840 con 0x55e6a8fbae40 2024-03-11T19:45:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbed2014740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8fcd9b0 con 0x55e6a8fb5cd0 2024-03-11T19:45:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbed2014740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6a8fcdb70 con 0x55e6a8fcce80 2024-03-11T19:45:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbecc9db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbecc9db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34430/0 (socket says 172.21.15.83:34430) 2024-03-11T19:45:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbecc9db640 1 -- 172.21.15.83:0/2182179320 learned_addr learned my addr 172.21.15.83:0/2182179320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbecd9dd640 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fcce80 0x55e6a8fcd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbed2014740 10 client.0 _ll_register_callbacks cb 0x7fbec4005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:37.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbecd9dd640 1 -- 172.21.15.83:0/2182179320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fb5cd0 msgr2=0x55e6a8fce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbecd9dd640 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fce870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbecd9dd640 1 -- 172.21.15.83:0/2182179320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 msgr2=0x55e6a8fcedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbecd9dd640 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.413+0000 7fbecd9dd640 1 -- 172.21.15.83:0/2182179320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6a8f0b610 con 0x55e6a8fcce80 2024-03-11T19:45:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.414+0000 7fbecc9db640 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.414+0000 7fbecd9dd640 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fcce80 0x55e6a8fcd300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006010 tx=0x7fbec00057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de75b11087aa7e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.414+0000 7fbeb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:45:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.414+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2182179320 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec002ee30 con 0x55e6a8fcce80 2024-03-11T19:45:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.414+0000 7fbed2014740 1 -- 172.21.15.83:0/2182179320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbec4011c30 con 0x55e6a8fcce80 2024-03-11T19:45:37.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.414+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2182179320 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fbec00052e0 con 0x55e6a8fcce80 2024-03-11T19:45:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.415+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2182179320 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbec0005cf0 con 0x55e6a8fcce80 2024-03-11T19:45:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.415+0000 7fbed2014740 10 client.26092 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.415+0000 7fbed2014740 1 -- 172.21.15.83:0/2182179320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e6a8ecb570 con 0x55e6a8fcce80 2024-03-11T19:45:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.415+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2182179320 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbec4011c30 con 0x55e6a8fcce80 2024-03-11T19:45:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.416+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2182179320 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fbec0068070 con 0x55e6a8fcce80 2024-03-11T19:45:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.416+0000 7fbed2014740 10 client.26092 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.416+0000 7fbed2014740 10 client.26092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.416+0000 7fbed2014740 20 client.26092 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.416+0000 7fbed2014740 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e6a902b980 0x55e6a902dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.416+0000 7fbed2014740 4 client.26092 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:45:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.416+0000 7fbed2014740 1 -- 172.21.15.83:0/2182179320 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e6a8c98890 con 0x55e6a902b980 2024-03-11T19:45:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.416+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e6a902b980 0x55e6a902dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.417+0000 7fbeb0ff9640 10 client.26092 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:37.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.417+0000 7fbecd1dc640 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e6a902b980 0x55e6a902dda0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3bc55855436cea5 server_cookie=2af4254d59aa2043 in_seq=0 out_seq=0 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.417+0000 7fbeb0ff9640 1 -- 172.21.15.83:0/2182179320 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e6a8c98890 con 0x55e6a902b980 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.417+0000 7fbeb0ff9640 10 client.26092 handle_command_reply: tid=0 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.417+0000 7fbed2014740 1 client.26092 shutdown 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.417+0000 7fbed2014740 1 -- 172.21.15.83:0/2182179320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fcce80 msgr2=0x55e6a8fcd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.418+0000 7fbed2014740 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fcce80 0x55e6a8fcd300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbec0006010 tx=0x7fbec00057c0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.418+0000 7fbed2014740 1 -- 172.21.15.83:0/2182179320 shutdown_connections 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.418+0000 7fbed2014740 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e6a902b980 0x55e6a902dda0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.418+0000 7fbed2014740 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6a8fcce80 0x55e6a8fcd300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.418+0000 7fbeb0ff9640 0 client.26092 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.418+0000 7fbed2014740 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e6a8fbae40 0x55e6a8fcedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.418+0000 7fbed2014740 1 --2- 172.21.15.83:0/2182179320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6a8fb5cd0 0x55e6a8fce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.418+0000 7fbed2014740 1 -- 172.21.15.83:0/2182179320 >> 172.21.15.83:0/2182179320 conn(0x55e6a8f9f7f0 msgr2=0x55e6a8fa2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:37.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.418+0000 7fbed2014740 1 -- 172.21.15.83:0/2182179320 shutdown_connections 2024-03-11T19:45:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.418+0000 7fbed2014740 1 -- 172.21.15.83:0/2182179320 wait complete. 2024-03-11T19:45:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.418+0000 7fbed2014740 20 client.26092 trim_cache size 0 max 16384 2024-03-11T19:45:37.425 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:45:37.425 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:45:37.425 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:45:37.425 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:45:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.419+0000 7fbecf440640 1 -- 172.21.15.83:0/26235784 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbeb40a1c00 msgr2=0x7fbeb40a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.419+0000 7fbecf440640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbeb40a1c00 0x7fbeb40a40c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbec0004af0 tx=0x7fbec00093a0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.419+0000 7fbecf440640 1 -- 172.21.15.83:0/26235784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec8104570 msgr2=0x7fbec8197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.419+0000 7fbecf440640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec8104570 0x7fbec8197480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800e9a0 tx=0x7fbeb800ee60 comp rx=0 tx=0).stop 2024-03-11T19:45:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.419+0000 7fbecf440640 1 -- 172.21.15.83:0/26235784 shutdown_connections 2024-03-11T19:45:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.419+0000 7fbecf440640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fbeb40a1c00 0x7fbeb40a40c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.420+0000 7fbecf440640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbec8106ea0 0x7fbec819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.420+0000 7fbecf440640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbec8104570 0x7fbec8197480 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.420+0000 7fbecf440640 1 --2- 172.21.15.83:0/26235784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbec8101c40 0x7fbec8196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.420+0000 7fbecf440640 1 -- 172.21.15.83:0/26235784 >> 172.21.15.83:0/26235784 conn(0x7fbec80fb650 msgr2=0x7fbec8104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.420+0000 7fbecf440640 1 -- 172.21.15.83:0/26235784 shutdown_connections 2024-03-11T19:45:37.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.420+0000 7fbecf440640 1 -- 172.21.15.83:0/26235784 wait complete. 2024-03-11T19:45:37.439 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:45:37.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:45:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 380.0 s, 2655.2 tps, lat 37.727 ms stddev 45.626 2024-03-11T19:45:37.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.581+0000 7ff5112ae640 1 -- 172.21.15.83:0/1480753270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101290 msgr2=0x7ff50c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.581+0000 7ff5112ae640 1 --2- 172.21.15.83:0/1480753270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101290 0x7ff50c101710 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff500009900 tx=0x7ff50002bc30 comp rx=0 tx=0).stop 2024-03-11T19:45:37.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.581+0000 7ff5112ae640 1 -- 172.21.15.83:0/1480753270 shutdown_connections 2024-03-11T19:45:37.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.581+0000 7ff5112ae640 1 --2- 172.21.15.83:0/1480753270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff50c101c50 0x7ff50c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.581+0000 7ff5112ae640 1 --2- 172.21.15.83:0/1480753270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101290 0x7ff50c101710 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.581+0000 7ff5112ae640 1 --2- 172.21.15.83:0/1480753270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c1000a0 0x7ff50c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.581+0000 7ff5112ae640 1 -- 172.21.15.83:0/1480753270 >> 172.21.15.83:0/1480753270 conn(0x7ff50c0fb630 msgr2=0x7ff50c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:37.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.581+0000 7ff5112ae640 1 -- 172.21.15.83:0/1480753270 shutdown_connections 2024-03-11T19:45:37.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.582+0000 7ff5112ae640 1 -- 172.21.15.83:0/1480753270 wait complete. 2024-03-11T19:45:37.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.583+0000 7ff5112ae640 1 Processor -- start 2024-03-11T19:45:37.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.583+0000 7ff5112ae640 1 -- start start 2024-03-11T19:45:37.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.583+0000 7ff5112ae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff50c1000a0 0x7ff50c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.583+0000 7ff5112ae640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c101290 0x7ff50c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.583+0000 7ff5112ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101c50 0x7ff50c1a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.583+0000 7ff5112ae640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c1a0f10 con 0x7ff50c101c50 2024-03-11T19:45:37.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.583+0000 7ff5112ae640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c1a1080 con 0x7ff50c1000a0 2024-03-11T19:45:37.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.583+0000 7ff5112ae640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c1a1240 con 0x7ff50c101290 2024-03-11T19:45:37.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.583+0000 7ff50affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff50c1000a0 0x7ff50c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff50affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff50c1000a0 0x7ff50c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:38348/0 (socket says 172.21.15.83:38348) 2024-03-11T19:45:37.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff50affd640 1 -- 172.21.15.83:0/2439685513 learned_addr learned my addr 172.21.15.83:0/2439685513 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff50a7fc640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c101290 0x7ff50c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff50b7fe640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101c50 0x7ff50c1a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff50affd640 1 -- 172.21.15.83:0/2439685513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c101290 msgr2=0x7ff50c199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff50affd640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c101290 0x7ff50c199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff50affd640 1 -- 172.21.15.83:0/2439685513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101c50 msgr2=0x7ff50c1a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff50affd640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101c50 0x7ff50c1a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff50affd640 1 -- 172.21.15.83:0/2439685513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff500009560 con 0x7ff50c1000a0 2024-03-11T19:45:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff50affd640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff50c1000a0 0x7ff50c199420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00d950 tx=0x7ff4fc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff4f3fff640 1 -- 172.21.15.83:0/2439685513 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc009c30 con 0x7ff50c1000a0 2024-03-11T19:45:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff4f3fff640 1 -- 172.21.15.83:0/2439685513 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4fc009d90 con 0x7ff50c1000a0 2024-03-11T19:45:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.584+0000 7ff4f3fff640 1 -- 172.21.15.83:0/2439685513 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00f850 con 0x7ff50c1000a0 2024-03-11T19:45:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.585+0000 7ff50b7fe640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101c50 0x7ff50c1a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:45:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.585+0000 7ff5112ae640 1 -- 172.21.15.83:0/2439685513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c1a1520 con 0x7ff50c1000a0 2024-03-11T19:45:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.585+0000 7ff5112ae640 1 -- 172.21.15.83:0/2439685513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c1a1950 con 0x7ff50c1000a0 2024-03-11T19:45:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.586+0000 7ff5112ae640 1 -- 172.21.15.83:0/2439685513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50c105bd0 con 0x7ff50c1000a0 2024-03-11T19:45:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.587+0000 7ff4f3fff640 1 -- 172.21.15.83:0/2439685513 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff4fc00f9b0 con 0x7ff50c1000a0 2024-03-11T19:45:37.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.587+0000 7ff4f3fff640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4f40a1a30 0x7ff4f40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:37.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.588+0000 7ff4f3fff640 1 -- 172.21.15.83:0/2439685513 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff4fc0cd690 con 0x7ff50c1000a0 2024-03-11T19:45:37.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.589+0000 7ff50a7fc640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4f40a1a30 0x7ff4f40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.590+0000 7ff50a7fc640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4f40a1a30 0x7ff4f40a3ef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff500009230 tx=0x7ff500005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:37.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.592+0000 7ff4f3fff640 1 -- 172.21.15.83:0/2439685513 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4fc0968b0 con 0x7ff50c1000a0 2024-03-11T19:45:37.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.927+0000 7ff5112ae640 1 -- 172.21.15.83:0/2439685513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff50c02ce70 con 0x7ff50c1000a0 2024-03-11T19:45:37.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.928+0000 7ff4f3fff640 1 -- 172.21.15.83:0/2439685513 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11350 (secure 0 0 0) 0x7ff4fc099f70 con 0x7ff50c1000a0 2024-03-11T19:45:37.933 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:45:37.933 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:45:19.152624+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:45:37.933 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 31 2024-03-11T19:45:37.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.931+0000 7ff5112ae640 1 -- 172.21.15.83:0/2439685513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4f40a1a30 msgr2=0x7ff4f40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.931+0000 7ff5112ae640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4f40a1a30 0x7ff4f40a3ef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff500009230 tx=0x7ff500005fd0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.931+0000 7ff5112ae640 1 -- 172.21.15.83:0/2439685513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff50c1000a0 msgr2=0x7ff50c199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:37.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.931+0000 7ff5112ae640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff50c1000a0 0x7ff50c199420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00d950 tx=0x7ff4fc00de10 comp rx=0 tx=0).stop 2024-03-11T19:45:37.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.931+0000 7ff5112ae640 1 -- 172.21.15.83:0/2439685513 shutdown_connections 2024-03-11T19:45:37.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.931+0000 7ff5112ae640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4f40a1a30 0x7ff4f40a3ef0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.931+0000 7ff5112ae640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff50c101c50 0x7ff50c1a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.931+0000 7ff5112ae640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c101290 0x7ff50c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.931+0000 7ff5112ae640 1 --2- 172.21.15.83:0/2439685513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff50c1000a0 0x7ff50c199420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:37.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.931+0000 7ff5112ae640 1 -- 172.21.15.83:0/2439685513 >> 172.21.15.83:0/2439685513 conn(0x7ff50c0fb630 msgr2=0x7ff50c0fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:37.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.932+0000 7ff5112ae640 1 -- 172.21.15.83:0/2439685513 shutdown_connections 2024-03-11T19:45:37.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:37.932+0000 7ff5112ae640 1 -- 172.21.15.83:0/2439685513 wait complete. 2024-03-11T19:45:37.950 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:45:37.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:45:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.109+0000 7ff686943640 1 -- 172.21.15.83:0/1722100847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff680101270 msgr2=0x7ff6801016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.109+0000 7ff686943640 1 --2- 172.21.15.83:0/1722100847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff680101270 0x7ff6801016f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff670009900 tx=0x7ff67002bc30 comp rx=0 tx=0).stop 2024-03-11T19:45:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.109+0000 7ff686943640 1 -- 172.21.15.83:0/1722100847 shutdown_connections 2024-03-11T19:45:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.109+0000 7ff686943640 1 --2- 172.21.15.83:0/1722100847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff680101c30 0x7ff6801084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.109+0000 7ff686943640 1 --2- 172.21.15.83:0/1722100847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff680101270 0x7ff6801016f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.109+0000 7ff686943640 1 --2- 172.21.15.83:0/1722100847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff680100080 0x7ff6801004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.109+0000 7ff686943640 1 -- 172.21.15.83:0/1722100847 >> 172.21.15.83:0/1722100847 conn(0x7ff6800fb610 msgr2=0x7ff6800fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.109+0000 7ff686943640 1 -- 172.21.15.83:0/1722100847 shutdown_connections 2024-03-11T19:45:38.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.109+0000 7ff686943640 1 -- 172.21.15.83:0/1722100847 wait complete. 2024-03-11T19:45:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.109+0000 7ff686943640 1 Processor -- start 2024-03-11T19:45:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.109+0000 7ff686943640 1 -- start start 2024-03-11T19:45:38.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff686943640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff680100080 0x7ff68010a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff686943640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff680101270 0x7ff680108e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff686943640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff680101c30 0x7ff680109380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff686943640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff680109ae0 con 0x7ff680101c30 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff686943640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff680109c50 con 0x7ff680100080 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff686943640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6801a1280 con 0x7ff680101270 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff684ee0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff680101c30 0x7ff680109380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff684ee0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff680101c30 0x7ff680109380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34446/0 (socket says 172.21.15.83:34446) 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff684ee0640 1 -- 172.21.15.83:0/3727628386 learned_addr learned my addr 172.21.15.83:0/3727628386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff67ffff640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff680100080 0x7ff68010a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff67f7fe640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff680101270 0x7ff680108e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff684ee0640 1 -- 172.21.15.83:0/3727628386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff680101270 msgr2=0x7ff680108e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff684ee0640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff680101270 0x7ff680108e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff684ee0640 1 -- 172.21.15.83:0/3727628386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff680100080 msgr2=0x7ff68010a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff684ee0640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff680100080 0x7ff68010a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff684ee0640 1 -- 172.21.15.83:0/3727628386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff670009560 con 0x7ff680101c30 2024-03-11T19:45:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff67ffff640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff680100080 0x7ff68010a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff67f7fe640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff680101270 0x7ff680108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:45:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff684ee0640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff680101c30 0x7ff680109380 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff67800ead0 tx=0x7ff67800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.110+0000 7ff67d7fa640 1 -- 172.21.15.83:0/3727628386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800cde0 con 0x7ff680101c30 2024-03-11T19:45:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.111+0000 7ff686943640 1 -- 172.21.15.83:0/3727628386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6801a1530 con 0x7ff680101c30 2024-03-11T19:45:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.111+0000 7ff67d7fa640 1 -- 172.21.15.83:0/3727628386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff678005020 con 0x7ff680101c30 2024-03-11T19:45:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.111+0000 7ff67d7fa640 1 -- 172.21.15.83:0/3727628386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6780116b0 con 0x7ff680101c30 2024-03-11T19:45:38.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.111+0000 7ff686943640 1 -- 172.21.15.83:0/3727628386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6801a1a40 con 0x7ff680101c30 2024-03-11T19:45:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.114+0000 7ff6895ea740 1 Processor -- start 2024-03-11T19:45:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.114+0000 7ff6895ea740 1 -- start start 2024-03-11T19:45:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.114+0000 7ff67d7fa640 1 -- 172.21.15.83:0/3727628386 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff678011810 con 0x7ff680101c30 2024-03-11T19:45:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.114+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244f280 0x55651244f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.114+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55651244fc70 0x556512454770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.114+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556512454de0 0x556512457270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.114+0000 7ff6895ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565123fa960 con 0x55651244fc70 2024-03-11T19:45:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.114+0000 7ff6895ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565123ae310 con 0x556512454de0 2024-03-11T19:45:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.114+0000 7ff6895ea740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565123ab840 con 0x55651244f280 2024-03-11T19:45:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.115+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55651244fc70 0x556512454770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.115+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55651244fc70 0x556512454770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34448/0 (socket says 172.21.15.83:34448) 2024-03-11T19:45:38.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.115+0000 7ff67ffff640 1 -- 172.21.15.83:0/3339872818 learned_addr learned my addr 172.21.15.83:0/3339872818 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.115+0000 7ff67d7fa640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6680a1b30 0x7ff6680a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.115+0000 7ff67ffff640 1 -- 172.21.15.83:0/3339872818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244f280 msgr2=0x55651244f6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:45:38.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.115+0000 7ff67ffff640 1 --2- 172.21.15.83:0/3339872818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244f280 0x55651244f6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.115+0000 7ff67ffff640 1 -- 172.21.15.83:0/3339872818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556512454de0 msgr2=0x556512457270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:45:38.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.115+0000 7ff67ffff640 1 --2- 172.21.15.83:0/3339872818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556512454de0 0x556512457270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.115+0000 7ff67ffff640 1 -- 172.21.15.83:0/3339872818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556512369060 con 0x55651244fc70 2024-03-11T19:45:38.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.115+0000 7ff67d7fa640 1 -- 172.21.15.83:0/3727628386 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff6780cd600 con 0x7ff680101c30 2024-03-11T19:45:38.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.115+0000 7ff684ee0640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6680a1b30 0x7ff6680a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.116+0000 7ff67ffff640 1 --2- 172.21.15.83:0/3339872818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55651244fc70 0x556512454770 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff674032460 tx=0x7ff674032920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6593d13700cb4f56 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.116+0000 7ff6667fc640 1 -- 172.21.15.83:0/3339872818 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674033770 con 0x55651244fc70 2024-03-11T19:45:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.116+0000 7ff6667fc640 1 -- 172.21.15.83:0/3339872818 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff674037070 con 0x55651244fc70 2024-03-11T19:45:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.116+0000 7ff684ee0640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6680a1b30 0x7ff6680a3ff0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff6780a27f0 tx=0x7ff678005890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.116+0000 7ff6667fc640 1 -- 172.21.15.83:0/3339872818 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674033770 con 0x55651244fc70 2024-03-11T19:45:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.117+0000 7ff6895ea740 1 -- 172.21.15.83:0/3339872818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55651244fc70 msgr2=0x556512454770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.117+0000 7ff6895ea740 1 --2- 172.21.15.83:0/3339872818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55651244fc70 0x556512454770 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff674032460 tx=0x7ff674032920 comp rx=0 tx=0).stop 2024-03-11T19:45:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.117+0000 7ff6895ea740 1 -- 172.21.15.83:0/3339872818 shutdown_connections 2024-03-11T19:45:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.117+0000 7ff6895ea740 1 --2- 172.21.15.83:0/3339872818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556512454de0 0x556512457270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.117+0000 7ff6895ea740 1 --2- 172.21.15.83:0/3339872818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55651244fc70 0x556512454770 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.117+0000 7ff6895ea740 1 --2- 172.21.15.83:0/3339872818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244f280 0x55651244f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.117+0000 7ff6895ea740 1 -- 172.21.15.83:0/3339872818 >> 172.21.15.83:0/3339872818 conn(0x5565124397e0 msgr2=0x556512439be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.117+0000 7ff6895ea740 1 -- 172.21.15.83:0/3339872818 shutdown_connections 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.117+0000 7ff6895ea740 1 -- 172.21.15.83:0/3339872818 wait complete. 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.117+0000 7ff6895ea740 1 -- start start 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff6895ea740 1 Processor -- start 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55651244f280 0x55651243f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x55651243fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512446e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff6895ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565123a7020 con 0x556512454de0 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff6895ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651207a430 con 0x55651244f280 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff6895ea740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556512036e00 con 0x55651244fc70 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff6895ea740 10 client.0 _ll_register_callbacks cb 0x55651234ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff67f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512446e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff67f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512446e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34464/0 (socket says 172.21.15.83:34464) 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff67f7fe640 1 -- 172.21.15.83:0/3105446723 learned_addr learned my addr 172.21.15.83:0/3105446723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff67f7fe640 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55651244f280 0x55651243f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.118+0000 7ff67ffff640 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x55651243fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff67f7fe640 1 -- 172.21.15.83:0/3105446723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 msgr2=0x55651243fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff67f7fe640 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x55651243fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff67f7fe640 1 -- 172.21.15.83:0/3105446723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55651244f280 msgr2=0x55651243f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff67f7fe640 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55651244f280 0x55651243f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff67f7fe640 1 -- 172.21.15.83:0/3105446723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67400bcd0 con 0x556512454de0 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff67f7fe640 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512446e00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff670005940 tx=0x7ff670005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684b560eb2d1feea server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff65f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff65f7fe640 1 -- 172.21.15.83:0/3105446723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670049070 con 0x556512454de0 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff65f7fe640 1 -- 172.21.15.83:0/3105446723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff6700035f0 con 0x556512454de0 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff6895ea740 1 -- 172.21.15.83:0/3105446723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565120a67e0 con 0x556512454de0 2024-03-11T19:45:38.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff65f7fe640 1 -- 172.21.15.83:0/3105446723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff670004510 con 0x556512454de0 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff6895ea740 10 client.16548 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff6895ea740 1 -- 172.21.15.83:0/3105446723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5565123a56a0 con 0x556512454de0 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff65f7fe640 1 -- 172.21.15.83:0/3105446723 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff670054080 con 0x556512454de0 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff65f7fe640 1 -- 172.21.15.83:0/3105446723 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff67004c030 con 0x556512454de0 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff6895ea740 10 client.16548 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff6895ea740 10 client.16548 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff6895ea740 20 client.16548 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff6895ea740 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55651244ba70 0x55651244de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff6895ea740 4 client.16548 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.119+0000 7ff6895ea740 1 -- 172.21.15.83:0/3105446723 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565120e3720 con 0x55651244ba70 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.120+0000 7ff67ffff640 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55651244ba70 0x55651244de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.120+0000 7ff65f7fe640 10 client.16548 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.120+0000 7ff67ffff640 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55651244ba70 0x55651244de60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33d323cc065bff6 server_cookie=9fc260f525d016e2 in_seq=0 out_seq=0 2024-03-11T19:45:38.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff65f7fe640 1 -- 172.21.15.83:0/3105446723 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565120e3720 con 0x55651244ba70 2024-03-11T19:45:38.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff65f7fe640 10 client.16548 handle_command_reply: tid=0 2024-03-11T19:45:38.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 1 client.16548 shutdown 2024-03-11T19:45:38.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 1 -- 172.21.15.83:0/3105446723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 msgr2=0x556512446e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512446e00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff670005940 tx=0x7ff670005a20 comp rx=0 tx=0).stop 2024-03-11T19:45:38.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 1 -- 172.21.15.83:0/3105446723 shutdown_connections 2024-03-11T19:45:38.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55651244ba70 0x55651244de60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512446e00 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x55651243fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff65f7fe640 0 client.16548 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:45:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 1 --2- 172.21.15.83:0/3105446723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55651244f280 0x55651243f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 1 -- 172.21.15.83:0/3105446723 >> 172.21.15.83:0/3105446723 conn(0x5565124397e0 msgr2=0x556512438b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 1 -- 172.21.15.83:0/3105446723 shutdown_connections 2024-03-11T19:45:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 1 -- 172.21.15.83:0/3105446723 wait complete. 2024-03-11T19:45:38.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.121+0000 7ff6895ea740 20 client.16548 trim_cache size 0 max 16384 2024-03-11T19:45:38.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.137+0000 7ff6895ea740 1 Processor -- start 2024-03-11T19:45:38.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.137+0000 7ff6895ea740 1 -- start start 2024-03-11T19:45:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.137+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55651244f280 0x556512451130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.137+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x556512451670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.137+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556512454de0 0x556512446dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.137+0000 7ff6895ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651232c130 con 0x55651244f280 2024-03-11T19:45:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.137+0000 7ff6895ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565124408f0 con 0x556512454de0 2024-03-11T19:45:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.137+0000 7ff6895ea740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651243f510 con 0x55651244fc70 2024-03-11T19:45:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.137+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x556512451670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.138+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x556512451670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:55018/0 (socket says 172.21.15.83:55018) 2024-03-11T19:45:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.138+0000 7ff67ffff640 1 -- 172.21.15.83:0/291782642 learned_addr learned my addr 172.21.15.83:0/291782642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.138+0000 7ff67f7fe640 1 --2- 172.21.15.83:0/291782642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556512454de0 0x556512446dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.138+0000 7ff67ffff640 1 -- 172.21.15.83:0/291782642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556512454de0 msgr2=0x556512446dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.138+0000 7ff67ffff640 1 --2- 172.21.15.83:0/291782642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556512454de0 0x556512446dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.138+0000 7ff67ffff640 1 -- 172.21.15.83:0/291782642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55651244f280 msgr2=0x556512451130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.138+0000 7ff67ffff640 1 --2- 172.21.15.83:0/291782642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55651244f280 0x556512451130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.138+0000 7ff67ffff640 1 -- 172.21.15.83:0/291782642 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556512341c20 con 0x55651244fc70 2024-03-11T19:45:38.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.138+0000 7ff67ffff640 1 --2- 172.21.15.83:0/291782642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x556512451670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff674032460 tx=0x7ff674040040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c52acf63e68e3ee server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.138+0000 7ff665ffb640 1 -- 172.21.15.83:0/291782642 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674043070 con 0x55651244fc70 2024-03-11T19:45:38.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.139+0000 7ff665ffb640 1 -- 172.21.15.83:0/291782642 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff674044040 con 0x55651244fc70 2024-03-11T19:45:38.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.139+0000 7ff665ffb640 1 -- 172.21.15.83:0/291782642 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674033770 con 0x55651244fc70 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.139+0000 7ff6895ea740 1 -- 172.21.15.83:0/291782642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 msgr2=0x556512451670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.139+0000 7ff6895ea740 1 --2- 172.21.15.83:0/291782642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x556512451670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff674032460 tx=0x7ff674040040 comp rx=0 tx=0).stop 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.139+0000 7ff6895ea740 1 -- 172.21.15.83:0/291782642 shutdown_connections 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.139+0000 7ff6895ea740 1 --2- 172.21.15.83:0/291782642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556512454de0 0x556512446dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.139+0000 7ff6895ea740 1 --2- 172.21.15.83:0/291782642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x556512451670 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.139+0000 7ff6895ea740 1 --2- 172.21.15.83:0/291782642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55651244f280 0x556512451130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.139+0000 7ff6895ea740 1 -- 172.21.15.83:0/291782642 >> 172.21.15.83:0/291782642 conn(0x5565124397e0 msgr2=0x55651243e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.140+0000 7ff6895ea740 1 -- 172.21.15.83:0/291782642 shutdown_connections 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.140+0000 7ff6895ea740 1 -- 172.21.15.83:0/291782642 wait complete. 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.140+0000 7ff6895ea740 1 -- start start 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.140+0000 7ff6895ea740 1 Processor -- start 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.140+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55651244f280 0x556512446810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.140+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x556512446d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.140+0000 7ff6895ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512444e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.140+0000 7ff6895ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556512445540 con 0x556512454de0 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.140+0000 7ff6895ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565124456b0 con 0x55651244f280 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.140+0000 7ff6895ea740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556512445870 con 0x55651244fc70 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.140+0000 7ff6895ea740 10 client.0 _ll_register_callbacks cb 0x7ff67400bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff67f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512444e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff67f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512444e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34478/0 (socket says 172.21.15.83:34478) 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff67f7fe640 1 -- 172.21.15.83:0/995836943 learned_addr learned my addr 172.21.15.83:0/995836943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff67f7fe640 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55651244f280 0x556512446810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff67ffff640 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x556512446d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff67f7fe640 1 -- 172.21.15.83:0/995836943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 msgr2=0x556512446d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff67f7fe640 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x556512446d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff67f7fe640 1 -- 172.21.15.83:0/995836943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55651244f280 msgr2=0x556512446810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff67f7fe640 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55651244f280 0x556512446810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff67f7fe640 1 -- 172.21.15.83:0/995836943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556512369060 con 0x556512454de0 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff67f7fe640 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512444e70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff67002c840 tx=0x7ff670000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8068ca30c00721 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff65f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff65f7fe640 1 -- 172.21.15.83:0/995836943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670049030 con 0x556512454de0 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff65f7fe640 1 -- 172.21.15.83:0/995836943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff67004a030 con 0x556512454de0 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.141+0000 7ff6895ea740 1 -- 172.21.15.83:0/995836943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff674037070 con 0x556512454de0 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff65f7fe640 1 -- 172.21.15.83:0/995836943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff670004510 con 0x556512454de0 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff6895ea740 10 client.16551 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff6895ea740 1 -- 172.21.15.83:0/995836943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5565123678c0 con 0x556512454de0 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff65f7fe640 1 -- 172.21.15.83:0/995836943 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff674037070 con 0x556512454de0 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff65f7fe640 1 -- 172.21.15.83:0/995836943 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff67004c040 con 0x556512454de0 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff6895ea740 10 client.16551 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff6895ea740 10 client.16551 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:45:38.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff6895ea740 20 client.16551 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff6895ea740 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5565124b4c30 0x5565124b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff6895ea740 4 client.16551 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:45:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff6895ea740 1 -- 172.21.15.83:0/995836943 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556512132890 con 0x5565124b4c30 2024-03-11T19:45:38.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.142+0000 7ff67ffff640 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5565124b4c30 0x5565124b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.143+0000 7ff67ffff640 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5565124b4c30 0x5565124b7050 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9754de042c581e2 server_cookie=afa18769ecff23fc in_seq=0 out_seq=0 2024-03-11T19:45:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.143+0000 7ff65f7fe640 10 client.16551 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:45:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.144+0000 7ff65f7fe640 1 -- 172.21.15.83:0/995836943 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556512132890 con 0x5565124b4c30 2024-03-11T19:45:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.144+0000 7ff65f7fe640 10 client.16551 handle_command_reply: tid=0 2024-03-11T19:45:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.144+0000 7ff6895ea740 1 client.16551 shutdown 2024-03-11T19:45:38.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.144+0000 7ff6895ea740 1 -- 172.21.15.83:0/995836943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 msgr2=0x556512444e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.144+0000 7ff6895ea740 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512444e70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff67002c840 tx=0x7ff670000bc0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.145+0000 7ff6895ea740 1 -- 172.21.15.83:0/995836943 shutdown_connections 2024-03-11T19:45:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.145+0000 7ff6895ea740 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5565124b4c30 0x5565124b7050 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.145+0000 7ff6895ea740 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556512454de0 0x556512444e70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.145+0000 7ff6895ea740 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55651244fc70 0x556512446d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.145+0000 7ff6895ea740 1 --2- 172.21.15.83:0/995836943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55651244f280 0x556512446810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.145+0000 7ff6895ea740 1 -- 172.21.15.83:0/995836943 >> 172.21.15.83:0/995836943 conn(0x5565124397e0 msgr2=0x55651243e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.145+0000 7ff65f7fe640 0 client.16551 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:45:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.145+0000 7ff6895ea740 1 -- 172.21.15.83:0/995836943 shutdown_connections 2024-03-11T19:45:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.145+0000 7ff6895ea740 1 -- 172.21.15.83:0/995836943 wait complete. 2024-03-11T19:45:38.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.145+0000 7ff6895ea740 20 client.16551 trim_cache size 0 max 16384 2024-03-11T19:45:38.154 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:45:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 -- 172.21.15.83:0/3727628386 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6680a1b30 msgr2=0x7ff6680a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6680a1b30 0x7ff6680a3ff0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff6780a27f0 tx=0x7ff678005890 comp rx=0 tx=0).stop 2024-03-11T19:45:38.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 -- 172.21.15.83:0/3727628386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff680101c30 msgr2=0x7ff680109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff680101c30 0x7ff680109380 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff67800ead0 tx=0x7ff67800ef90 comp rx=0 tx=0).stop 2024-03-11T19:45:38.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 -- 172.21.15.83:0/3727628386 shutdown_connections 2024-03-11T19:45:38.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff6680a1b30 0x7ff6680a3ff0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff680101c30 0x7ff680109380 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff680101270 0x7ff680108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 --2- 172.21.15.83:0/3727628386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff680100080 0x7ff68010a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 -- 172.21.15.83:0/3727628386 >> 172.21.15.83:0/3727628386 conn(0x7ff6800fb610 msgr2=0x7ff680104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 -- 172.21.15.83:0/3727628386 shutdown_connections 2024-03-11T19:45:38.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.146+0000 7ff686943640 1 -- 172.21.15.83:0/3727628386 wait complete. 2024-03-11T19:45:38.167 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:45:38.167 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:45:38.224 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:37 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2439685513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:45:38.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.333+0000 7f179309a640 1 -- 172.21.15.83:0/1380934772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178c1024d0 msgr2=0x7f178c102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.333+0000 7f179309a640 1 --2- 172.21.15.83:0/1380934772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178c1024d0 0x7f178c102950 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1778009860 tx=0x7f177802bb70 comp rx=0 tx=0).stop 2024-03-11T19:45:38.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.333+0000 7f179309a640 1 -- 172.21.15.83:0/1380934772 shutdown_connections 2024-03-11T19:45:38.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.333+0000 7f179309a640 1 --2- 172.21.15.83:0/1380934772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f178c102e90 0x7f178c109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.333+0000 7f179309a640 1 --2- 172.21.15.83:0/1380934772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178c1024d0 0x7f178c102950 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.333+0000 7f179309a640 1 --2- 172.21.15.83:0/1380934772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f178c0ff460 0x7f178c0ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.333+0000 7f179309a640 1 -- 172.21.15.83:0/1380934772 >> 172.21.15.83:0/1380934772 conn(0x7f178c0fad70 msgr2=0x7f178c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.333+0000 7f179309a640 1 -- 172.21.15.83:0/1380934772 shutdown_connections 2024-03-11T19:45:38.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.333+0000 7f179309a640 1 -- 172.21.15.83:0/1380934772 wait complete. 2024-03-11T19:45:38.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.334+0000 7f179309a640 1 Processor -- start 2024-03-11T19:45:38.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.334+0000 7f179309a640 1 -- start start 2024-03-11T19:45:38.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.335+0000 7f179309a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f178c0ff460 0x7f178c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.335+0000 7f179309a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178c1024d0 0x7f178c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.335+0000 7f179309a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f178c102e90 0x7f178c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.335+0000 7f179309a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c197e90 con 0x7f178c102e90 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.335+0000 7f179309a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c1a1120 con 0x7f178c0ff460 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.335+0000 7f179309a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c1a1290 con 0x7f178c1024d0 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.335+0000 7f1791637640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f178c102e90 0x7f178c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.335+0000 7f1791637640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f178c102e90 0x7f178c1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34498/0 (socket says 172.21.15.83:34498) 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.335+0000 7f1791637640 1 -- 172.21.15.83:0/81921129 learned_addr learned my addr 172.21.15.83:0/81921129 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.335+0000 7f178bfff640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178c1024d0 0x7f178c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.336+0000 7f1790e36640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f178c0ff460 0x7f178c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.336+0000 7f1791637640 1 -- 172.21.15.83:0/81921129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178c1024d0 msgr2=0x7f178c197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.336+0000 7f1791637640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178c1024d0 0x7f178c197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.336+0000 7f1791637640 1 -- 172.21.15.83:0/81921129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f178c0ff460 msgr2=0x7f178c197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.336+0000 7f1791637640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f178c0ff460 0x7f178c197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.336+0000 7f1791637640 1 -- 172.21.15.83:0/81921129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1784009840 con 0x7f178c102e90 2024-03-11T19:45:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.336+0000 7f1790e36640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f178c0ff460 0x7f178c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:45:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.336+0000 7f1791637640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f178c102e90 0x7f178c1a0be0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f178400efd0 tx=0x7f178400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.337+0000 7f1789ffb640 1 -- 172.21.15.83:0/81921129 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784003b70 con 0x7f178c102e90 2024-03-11T19:45:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.337+0000 7f1789ffb640 1 -- 172.21.15.83:0/81921129 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1784003cd0 con 0x7f178c102e90 2024-03-11T19:45:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.337+0000 7f1789ffb640 1 -- 172.21.15.83:0/81921129 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784010960 con 0x7f178c102e90 2024-03-11T19:45:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.337+0000 7f179309a640 1 -- 172.21.15.83:0/81921129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1778009490 con 0x7f178c102e90 2024-03-11T19:45:38.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.337+0000 7f179309a640 1 -- 172.21.15.83:0/81921129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c19cc70 con 0x7f178c102e90 2024-03-11T19:45:38.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.340+0000 7f1789ffb640 1 -- 172.21.15.83:0/81921129 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1784010ac0 con 0x7f178c102e90 2024-03-11T19:45:38.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1795c6e740 1 Processor -- start 2024-03-11T19:45:38.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1795c6e740 1 -- start start 2024-03-11T19:45:38.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1789ffb640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f177c0a1c00 0x7f177c0a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 0x5579623ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623aec60 0x5579623b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623b3dd0 0x5579623b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1789ffb640 1 -- 172.21.15.83:0/81921129 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1784014070 con 0x7f178c102e90 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1795c6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796236a960 con 0x5579623aec60 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1795c6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796231e310 con 0x5579623b3dd0 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1795c6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796231b840 con 0x5579623ae270 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f178bfff640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f177c0a1c00 0x7f177c0a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1790e36640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 0x5579623ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1790e36640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 0x5579623ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:55040/0 (socket says 172.21.15.83:55040) 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1790e36640 1 -- 172.21.15.83:0/4212249417 learned_addr learned my addr 172.21.15.83:0/4212249417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f178bfff640 1 --2- 172.21.15.83:0/4212249417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623aec60 0x5579623b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.341+0000 7f1791637640 1 --2- 172.21.15.83:0/4212249417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623b3dd0 0x5579623b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.342+0000 7f1790e36640 1 -- 172.21.15.83:0/4212249417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623b3dd0 msgr2=0x5579623b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.342+0000 7f1790e36640 1 --2- 172.21.15.83:0/4212249417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623b3dd0 0x5579623b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.342+0000 7f1790e36640 1 -- 172.21.15.83:0/4212249417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623aec60 msgr2=0x5579623b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.342+0000 7f1790e36640 1 --2- 172.21.15.83:0/4212249417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623aec60 0x5579623b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.342+0000 7f1790e36640 1 -- 172.21.15.83:0/4212249417 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579622d9060 con 0x5579623ae270 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.342+0000 7f178bfff640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f177c0a1c00 0x7f177c0a40c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f177802c180 tx=0x7f1778035cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.342+0000 7f1790e36640 1 --2- 172.21.15.83:0/4212249417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 0x5579623ae690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1780009c10 tx=0x7f1780010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b04063f9e5d4e56 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.342+0000 7f1791637640 1 --2- 172.21.15.83:0/4212249417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623b3dd0 0x5579623b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.342+0000 7f1776ffd640 1 -- 172.21.15.83:0/4212249417 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178001b480 con 0x5579623ae270 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.342+0000 7f1776ffd640 1 -- 172.21.15.83:0/4212249417 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f178001bac0 con 0x5579623ae270 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.342+0000 7f1776ffd640 1 -- 172.21.15.83:0/4212249417 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780012a70 con 0x5579623ae270 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.343+0000 7f1795c6e740 1 -- 172.21.15.83:0/4212249417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 msgr2=0x5579623ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.343+0000 7f1795c6e740 1 --2- 172.21.15.83:0/4212249417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 0x5579623ae690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1780009c10 tx=0x7f1780010740 comp rx=0 tx=0).stop 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.343+0000 7f1795c6e740 1 -- 172.21.15.83:0/4212249417 shutdown_connections 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.343+0000 7f1795c6e740 1 --2- 172.21.15.83:0/4212249417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623b3dd0 0x5579623b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.343+0000 7f1795c6e740 1 --2- 172.21.15.83:0/4212249417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623aec60 0x5579623b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.343+0000 7f1795c6e740 1 --2- 172.21.15.83:0/4212249417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 0x5579623ae690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.343+0000 7f1795c6e740 1 -- 172.21.15.83:0/4212249417 >> 172.21.15.83:0/4212249417 conn(0x5579623a97e0 msgr2=0x5579623a9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.343+0000 7f1795c6e740 1 -- 172.21.15.83:0/4212249417 shutdown_connections 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.343+0000 7f1795c6e740 1 -- 172.21.15.83:0/4212249417 wait complete. 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1795c6e740 1 -- start start 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1795c6e740 1 Processor -- start 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623ae270 0x5579623b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623aec60 0x5579623b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623b3dd0 0x5579623bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1795c6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557962317020 con 0x5579623ae270 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1795c6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557961fea430 con 0x5579623b3dd0 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1795c6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557961fa6e00 con 0x5579623aec60 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1795c6e740 10 client.0 _ll_register_callbacks cb 0x5579622becb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1790e36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623ae270 0x5579623b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1791637640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623b3dd0 0x5579623bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1790e36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623ae270 0x5579623b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34506/0 (socket says 172.21.15.83:34506) 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.344+0000 7f1790e36640 1 -- 172.21.15.83:0/4028044926 learned_addr learned my addr 172.21.15.83:0/4028044926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f178bfff640 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623aec60 0x5579623b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f1790e36640 1 -- 172.21.15.83:0/4028044926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623aec60 msgr2=0x5579623b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f1790e36640 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623aec60 0x5579623b89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f1790e36640 1 -- 172.21.15.83:0/4028044926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623b3dd0 msgr2=0x5579623bf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f1790e36640 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623b3dd0 0x5579623bf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f1790e36640 1 -- 172.21.15.83:0/4028044926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1780009840 con 0x5579623ae270 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f178bfff640 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623aec60 0x5579623b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f1790e36640 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623ae270 0x5579623b8480 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1780010c40 tx=0x7f178001b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ea472f02290d86 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f176bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f176bfff640 1 -- 172.21.15.83:0/4028044926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780021070 con 0x5579623ae270 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f176bfff640 1 -- 172.21.15.83:0/4028044926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1780016070 con 0x5579623ae270 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f1795c6e740 1 -- 172.21.15.83:0/4028044926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579620167e0 con 0x5579623ae270 2024-03-11T19:45:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f176bfff640 1 -- 172.21.15.83:0/4028044926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1780005040 con 0x5579623ae270 2024-03-11T19:45:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f1795c6e740 10 client.16563 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.345+0000 7f1795c6e740 1 -- 172.21.15.83:0/4028044926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5579623156a0 con 0x5579623ae270 2024-03-11T19:45:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.346+0000 7f176bfff640 1 -- 172.21.15.83:0/4028044926 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5579620167e0 con 0x5579623ae270 2024-03-11T19:45:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.346+0000 7f176bfff640 1 -- 172.21.15.83:0/4028044926 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1780004990 con 0x5579623ae270 2024-03-11T19:45:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.346+0000 7f1795c6e740 10 client.16563 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.346+0000 7f1795c6e740 10 client.16563 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:45:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.346+0000 7f1795c6e740 20 client.16563 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.346+0000 7f1795c6e740 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5579623c4450 0x5579623c6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.347+0000 7f1795c6e740 4 client.16563 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.347+0000 7f1795c6e740 1 -- 172.21.15.83:0/4028044926 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557962053720 con 0x5579623c4450 2024-03-11T19:45:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.348+0000 7f178bfff640 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5579623c4450 0x5579623c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.349+0000 7f178bfff640 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5579623c4450 0x5579623c6870 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f020aae9a0eaac71 server_cookie=6c563ce9f21b1103 in_seq=0 out_seq=0 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.349+0000 7f176bfff640 10 client.16563 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.349+0000 7f176bfff640 1 -- 172.21.15.83:0/4028044926 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557962053720 con 0x5579623c4450 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.349+0000 7f176bfff640 10 client.16563 handle_command_reply: tid=0 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.349+0000 7f1795c6e740 1 client.16563 shutdown 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f1795c6e740 1 -- 172.21.15.83:0/4028044926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623ae270 msgr2=0x5579623b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f1795c6e740 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623ae270 0x5579623b8480 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1780010c40 tx=0x7f178001b6b0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f1795c6e740 1 -- 172.21.15.83:0/4028044926 shutdown_connections 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f1795c6e740 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5579623c4450 0x5579623c6870 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f1795c6e740 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623b3dd0 0x5579623bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f176bfff640 0 client.16563 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f1795c6e740 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623aec60 0x5579623b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f1795c6e740 1 --2- 172.21.15.83:0/4028044926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623ae270 0x5579623b8480 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f1795c6e740 1 -- 172.21.15.83:0/4028044926 >> 172.21.15.83:0/4028044926 conn(0x5579623a97e0 msgr2=0x5579623a8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f1795c6e740 1 -- 172.21.15.83:0/4028044926 shutdown_connections 2024-03-11T19:45:38.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f1795c6e740 1 -- 172.21.15.83:0/4028044926 wait complete. 2024-03-11T19:45:38.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.350+0000 7f1795c6e740 20 client.16563 trim_cache size 0 max 16384 2024-03-11T19:45:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.366+0000 7f1795c6e740 1 Processor -- start 2024-03-11T19:45:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.366+0000 7f1795c6e740 1 -- start start 2024-03-11T19:45:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.366+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623ae270 0x5579623a82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.366+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623aec60 0x5579623bf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.366+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623b3dd0 0x5579623a8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.366+0000 7f1795c6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796229c130 con 0x5579623aec60 2024-03-11T19:45:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.366+0000 7f1795c6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579623a4a10 con 0x5579623ae270 2024-03-11T19:45:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.366+0000 7f1795c6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579623c3fc0 con 0x5579623b3dd0 2024-03-11T19:45:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.366+0000 7f1790e36640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623ae270 0x5579623a82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.366+0000 7f1790e36640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623ae270 0x5579623a82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:38464/0 (socket says 172.21.15.83:38464) 2024-03-11T19:45:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.366+0000 7f1790e36640 1 -- 172.21.15.83:0/174925849 learned_addr learned my addr 172.21.15.83:0/174925849 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.367+0000 7f1790e36640 1 -- 172.21.15.83:0/174925849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623b3dd0 msgr2=0x5579623a8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.367+0000 7f1790e36640 1 --2- 172.21.15.83:0/174925849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623b3dd0 0x5579623a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.367+0000 7f1790e36640 1 -- 172.21.15.83:0/174925849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623aec60 msgr2=0x5579623bf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.367+0000 7f1790e36640 1 --2- 172.21.15.83:0/174925849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623aec60 0x5579623bf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.367+0000 7f1790e36640 1 -- 172.21.15.83:0/174925849 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579622b1c20 con 0x5579623ae270 2024-03-11T19:45:38.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.367+0000 7f1790e36640 1 --2- 172.21.15.83:0/174925849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623ae270 0x5579623a82f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1780009c10 tx=0x7f1780009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f45d7152f40407c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.370+0000 7f17767fc640 1 -- 172.21.15.83:0/174925849 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780021070 con 0x5579623ae270 2024-03-11T19:45:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.370+0000 7f17767fc640 1 -- 172.21.15.83:0/174925849 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1780004b20 con 0x5579623ae270 2024-03-11T19:45:38.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.370+0000 7f17767fc640 1 -- 172.21.15.83:0/174925849 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780026990 con 0x5579623ae270 2024-03-11T19:45:38.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.372+0000 7f1795c6e740 1 -- 172.21.15.83:0/174925849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623ae270 msgr2=0x5579623a82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.373+0000 7f1795c6e740 1 --2- 172.21.15.83:0/174925849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623ae270 0x5579623a82f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1780009c10 tx=0x7f1780009260 comp rx=0 tx=0).stop 2024-03-11T19:45:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.378+0000 7f1795c6e740 1 -- 172.21.15.83:0/174925849 shutdown_connections 2024-03-11T19:45:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.378+0000 7f1795c6e740 1 --2- 172.21.15.83:0/174925849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623b3dd0 0x5579623a8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.378+0000 7f1795c6e740 1 --2- 172.21.15.83:0/174925849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623aec60 0x5579623bf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.378+0000 7f1795c6e740 1 --2- 172.21.15.83:0/174925849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623ae270 0x5579623a82f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.378+0000 7f1795c6e740 1 -- 172.21.15.83:0/174925849 >> 172.21.15.83:0/174925849 conn(0x5579623a97e0 msgr2=0x5579623af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:37 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2439685513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:45:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.385+0000 7f1795c6e740 1 -- 172.21.15.83:0/174925849 shutdown_connections 2024-03-11T19:45:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.385+0000 7f1795c6e740 1 -- 172.21.15.83:0/174925849 wait complete. 2024-03-11T19:45:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.385+0000 7f1795c6e740 1 -- start start 2024-03-11T19:45:38.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.386+0000 7f1795c6e740 1 Processor -- start 2024-03-11T19:45:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.386+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 0x5579623bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.386+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623aec60 0x5579623bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.386+0000 7f1795c6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623b3dd0 0x5579623bc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.387+0000 7f1795c6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579623bcb50 con 0x5579623b3dd0 2024-03-11T19:45:38.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.387+0000 7f178bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623aec60 0x5579623bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.387+0000 7f1790e36640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 0x5579623bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.387+0000 7f178bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623aec60 0x5579623bc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:38478/0 (socket says 172.21.15.83:38478) 2024-03-11T19:45:38.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.387+0000 7f178bfff640 1 -- 172.21.15.83:0/827110933 learned_addr learned my addr 172.21.15.83:0/827110933 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.387+0000 7f1795c6e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579623bccc0 con 0x5579623aec60 2024-03-11T19:45:38.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.387+0000 7f1795c6e740 1 -- 172.21.15.83:0/827110933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55796242b940 con 0x5579623ae270 2024-03-11T19:45:38.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.387+0000 7f1795c6e740 10 client.0 _ll_register_callbacks cb 0x7f1780009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:38.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.388+0000 7f1791637640 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623b3dd0 0x5579623bc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.388+0000 7f1791637640 1 -- 172.21.15.83:0/827110933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 msgr2=0x5579623bf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.388+0000 7f1791637640 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 0x5579623bf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.388+0000 7f1791637640 1 -- 172.21.15.83:0/827110933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623aec60 msgr2=0x5579623bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.389+0000 7f1791637640 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623aec60 0x5579623bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.389+0000 7f1791637640 1 -- 172.21.15.83:0/827110933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579623156a0 con 0x5579623b3dd0 2024-03-11T19:45:38.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.394+0000 7f1791637640 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623b3dd0 0x5579623bc610 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f17840cb3c0 tx=0x7f178409af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf6762909b8e600 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.394+0000 7f176bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:45:38.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.396+0000 7f176bfff640 1 -- 172.21.15.83:0/827110933 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17840a3000 con 0x5579623b3dd0 2024-03-11T19:45:38.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.396+0000 7f1795c6e740 1 -- 172.21.15.83:0/827110933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f178001bac0 con 0x5579623b3dd0 2024-03-11T19:45:38.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.396+0000 7f176bfff640 1 -- 172.21.15.83:0/827110933 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f17840b3260 con 0x5579623b3dd0 2024-03-11T19:45:38.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.398+0000 7f176bfff640 1 -- 172.21.15.83:0/827110933 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1784004b40 con 0x5579623b3dd0 2024-03-11T19:45:38.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:37 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2439685513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:45:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.401+0000 7f1795c6e740 10 client.16566 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:38.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.402+0000 7f1795c6e740 1 -- 172.21.15.83:0/827110933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5579622d78c0 con 0x5579623b3dd0 2024-03-11T19:45:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.403+0000 7f176bfff640 1 -- 172.21.15.83:0/827110933 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1784004ca0 con 0x5579623b3dd0 2024-03-11T19:45:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.403+0000 7f176bfff640 1 -- 172.21.15.83:0/827110933 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f17840a5070 con 0x5579623b3dd0 2024-03-11T19:45:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.403+0000 7f1795c6e740 10 client.16566 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.403+0000 7f1795c6e740 10 client.16566 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:45:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.403+0000 7f1795c6e740 20 client.16566 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.403+0000 7f1795c6e740 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55796242d2a0 0x55796242f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.404+0000 7f1795c6e740 4 client.16566 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:45:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.404+0000 7f1795c6e740 1 -- 172.21.15.83:0/827110933 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f178001bac0 con 0x55796242d2a0 2024-03-11T19:45:38.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.404+0000 7f1790e36640 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55796242d2a0 0x55796242f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f176bfff640 10 client.16566 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:45:38.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f1790e36640 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55796242d2a0 0x55796242f6c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=871a70f99576aabf server_cookie=14f42dc081ff124a in_seq=0 out_seq=0 2024-03-11T19:45:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f176bfff640 1 -- 172.21.15.83:0/827110933 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f178001bac0 con 0x55796242d2a0 2024-03-11T19:45:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f176bfff640 10 client.16566 handle_command_reply: tid=0 2024-03-11T19:45:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f1795c6e740 1 client.16566 shutdown 2024-03-11T19:45:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f1795c6e740 1 -- 172.21.15.83:0/827110933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623b3dd0 msgr2=0x5579623bc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f1795c6e740 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623b3dd0 0x5579623bc610 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f17840cb3c0 tx=0x7f178409af60 comp rx=0 tx=0).stop 2024-03-11T19:45:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f1795c6e740 1 -- 172.21.15.83:0/827110933 shutdown_connections 2024-03-11T19:45:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f1795c6e740 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55796242d2a0 0x55796242f6c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f1795c6e740 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579623b3dd0 0x5579623bc610 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f1795c6e740 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579623aec60 0x5579623bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f1795c6e740 1 --2- 172.21.15.83:0/827110933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579623ae270 0x5579623bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f1795c6e740 1 -- 172.21.15.83:0/827110933 >> 172.21.15.83:0/827110933 conn(0x5579623a97e0 msgr2=0x5579623aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.405+0000 7f176bfff640 0 client.16566 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:45:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f1795c6e740 1 -- 172.21.15.83:0/827110933 shutdown_connections 2024-03-11T19:45:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f1795c6e740 1 -- 172.21.15.83:0/827110933 wait complete. 2024-03-11T19:45:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f1795c6e740 20 client.16566 trim_cache size 0 max 16384 2024-03-11T19:45:38.412 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:45:38.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 -- 172.21.15.83:0/81921129 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f177c0a1c00 msgr2=0x7f177c0a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f177c0a1c00 0x7f177c0a40c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f177802c180 tx=0x7f1778035cf0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 -- 172.21.15.83:0/81921129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f178c102e90 msgr2=0x7f178c1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f178c102e90 0x7f178c1a0be0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f178400efd0 tx=0x7f178400c6a0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 -- 172.21.15.83:0/81921129 shutdown_connections 2024-03-11T19:45:38.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f177c0a1c00 0x7f177c0a40c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f178c102e90 0x7f178c1a0be0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178c1024d0 0x7f178c197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 --2- 172.21.15.83:0/81921129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f178c0ff460 0x7f178c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 -- 172.21.15.83:0/81921129 >> 172.21.15.83:0/81921129 conn(0x7f178c0fad70 msgr2=0x7f178c1057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 -- 172.21.15.83:0/81921129 shutdown_connections 2024-03-11T19:45:38.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.406+0000 7f179309a640 1 -- 172.21.15.83:0/81921129 wait complete. 2024-03-11T19:45:38.424 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:45:38.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:45:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.554+0000 7eff33181640 1 -- 172.21.15.83:0/2899913391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c101c70 msgr2=0x7eff2c108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.554+0000 7eff33181640 1 --2- 172.21.15.83:0/2899913391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c101c70 0x7eff2c108500 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7eff24009cc0 tx=0x7eff2402e310 comp rx=0 tx=0).stop 2024-03-11T19:45:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.554+0000 7eff33181640 1 -- 172.21.15.83:0/2899913391 shutdown_connections 2024-03-11T19:45:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.554+0000 7eff33181640 1 --2- 172.21.15.83:0/2899913391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c101c70 0x7eff2c108500 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.554+0000 7eff33181640 1 --2- 172.21.15.83:0/2899913391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c1012b0 0x7eff2c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.554+0000 7eff33181640 1 --2- 172.21.15.83:0/2899913391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c1005a0 0x7eff2c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.554+0000 7eff33181640 1 -- 172.21.15.83:0/2899913391 >> 172.21.15.83:0/2899913391 conn(0x7eff2c0fad70 msgr2=0x7eff2c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.554+0000 7eff33181640 1 -- 172.21.15.83:0/2899913391 shutdown_connections 2024-03-11T19:45:38.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.554+0000 7eff33181640 1 -- 172.21.15.83:0/2899913391 wait complete. 2024-03-11T19:45:38.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.555+0000 7eff33181640 1 Processor -- start 2024-03-11T19:45:38.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.555+0000 7eff33181640 1 -- start start 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff33181640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c1005a0 0x7eff2c195010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff33181640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c1012b0 0x7eff2c195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff33181640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c101c70 0x7eff2c19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff33181640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c19cb00 con 0x7eff2c101c70 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff33181640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c19cc70 con 0x7eff2c1005a0 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff33181640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c19ce30 con 0x7eff2c1012b0 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff3171e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c101c70 0x7eff2c19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff3171e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c101c70 0x7eff2c19c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34530/0 (socket says 172.21.15.83:34530) 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff30f1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c1005a0 0x7eff2c195010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff3171e640 1 -- 172.21.15.83:0/3362469465 learned_addr learned my addr 172.21.15.83:0/3362469465 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff2bfff640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c1012b0 0x7eff2c195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff3171e640 1 -- 172.21.15.83:0/3362469465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c1012b0 msgr2=0x7eff2c195550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff3171e640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c1012b0 0x7eff2c195550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff3171e640 1 -- 172.21.15.83:0/3362469465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c1005a0 msgr2=0x7eff2c195010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff3171e640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c1005a0 0x7eff2c195010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff3171e640 1 -- 172.21.15.83:0/3362469465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff24009900 con 0x7eff2c101c70 2024-03-11T19:45:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.556+0000 7eff3171e640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c101c70 0x7eff2c19c5c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7eff24039040 tx=0x7eff24003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.557+0000 7eff29ffb640 1 -- 172.21.15.83:0/3362469465 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24036640 con 0x7eff2c101c70 2024-03-11T19:45:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.557+0000 7eff29ffb640 1 -- 172.21.15.83:0/3362469465 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff24003e40 con 0x7eff2c101c70 2024-03-11T19:45:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.557+0000 7eff33181640 1 -- 172.21.15.83:0/3362469465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c19d0b0 con 0x7eff2c101c70 2024-03-11T19:45:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.557+0000 7eff29ffb640 1 -- 172.21.15.83:0/3362469465 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2403f650 con 0x7eff2c101c70 2024-03-11T19:45:38.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.557+0000 7eff33181640 1 -- 172.21.15.83:0/3362469465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c19d470 con 0x7eff2c101c70 2024-03-11T19:45:38.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.561+0000 7eff35e32740 1 Processor -- start 2024-03-11T19:45:38.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.561+0000 7eff29ffb640 1 -- 172.21.15.83:0/3362469465 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7eff2403d040 con 0x7eff2c101c70 2024-03-11T19:45:38.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.561+0000 7eff35e32740 1 -- start start 2024-03-11T19:45:38.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff35e32740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 0x557350a2d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff35e32740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2dc60 0x557350a32760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff35e32740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a32dd0 0x557350a35260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff35e32740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573509e9960 con 0x557350a2dc60 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff35e32740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735099d310 con 0x557350a32dd0 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff35e32740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735099a840 con 0x557350a2d270 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff30f1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2dc60 0x557350a32760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a32dd0 0x557350a35260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff30f1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2dc60 0x557350a32760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34542/0 (socket says 172.21.15.83:34542) 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff30f1d640 1 -- 172.21.15.83:0/481291468 learned_addr learned my addr 172.21.15.83:0/481291468 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff2bfff640 1 --2- 172.21.15.83:0/481291468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 0x557350a2d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff29ffb640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff140a1ab0 0x7eff140a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff2bfff640 1 -- 172.21.15.83:0/481291468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 msgr2=0x557350a2d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff2bfff640 1 --2- 172.21.15.83:0/481291468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 0x557350a2d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff2bfff640 1 -- 172.21.15.83:0/481291468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2dc60 msgr2=0x557350a32760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff2bfff640 1 --2- 172.21.15.83:0/481291468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2dc60 0x557350a32760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff2bfff640 1 -- 172.21.15.83:0/481291468 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557350958060 con 0x557350a32dd0 2024-03-11T19:45:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff3171e640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff140a1ab0 0x7eff140a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.562+0000 7eff29ffb640 1 -- 172.21.15.83:0/3362469465 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7eff240f1b70 con 0x7eff2c101c70 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.563+0000 7eff2bfff640 1 --2- 172.21.15.83:0/481291468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a32dd0 0x557350a35260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7eff2c196520 tx=0x7eff2000c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d11a0c7245e99793 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.563+0000 7eff12ffd640 1 -- 172.21.15.83:0/481291468 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff200142d0 con 0x557350a32dd0 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.563+0000 7eff12ffd640 1 -- 172.21.15.83:0/481291468 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff2001a030 con 0x557350a32dd0 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.563+0000 7eff12ffd640 1 -- 172.21.15.83:0/481291468 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2000fea0 con 0x557350a32dd0 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff35e32740 1 -- 172.21.15.83:0/481291468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a32dd0 msgr2=0x557350a35260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff3171e640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff140a1ab0 0x7eff140a3f70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7eff24003fd0 tx=0x7eff24004490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff35e32740 1 --2- 172.21.15.83:0/481291468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a32dd0 0x557350a35260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7eff2c196520 tx=0x7eff2000c430 comp rx=0 tx=0).stop 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff35e32740 1 -- 172.21.15.83:0/481291468 shutdown_connections 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff35e32740 1 --2- 172.21.15.83:0/481291468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a32dd0 0x557350a35260 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff35e32740 1 --2- 172.21.15.83:0/481291468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2dc60 0x557350a32760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff35e32740 1 --2- 172.21.15.83:0/481291468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 0x557350a2d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff35e32740 1 -- 172.21.15.83:0/481291468 >> 172.21.15.83:0/481291468 conn(0x557350a287e0 msgr2=0x557350a28be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff35e32740 1 -- 172.21.15.83:0/481291468 shutdown_connections 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff35e32740 1 -- 172.21.15.83:0/481291468 wait complete. 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff35e32740 1 -- start start 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.564+0000 7eff35e32740 1 Processor -- start 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff35e32740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2d270 0x557350a37520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff35e32740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2dc60 0x557350a37a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff35e32740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a32dd0 0x557350a3eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff35e32740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557350996020 con 0x557350a2dc60 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff35e32740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557350669430 con 0x557350a2d270 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff35e32740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557350625e00 con 0x557350a32dd0 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2d270 0x557350a37520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2d270 0x557350a37520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:38514/0 (socket says 172.21.15.83:38514) 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff2bfff640 1 -- 172.21.15.83:0/411903058 learned_addr learned my addr 172.21.15.83:0/411903058 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff35e32740 10 client.0 _ll_register_callbacks cb 0x55735093dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:38.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff30f1d640 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2dc60 0x557350a37a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.565+0000 7eff2bfff640 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a32dd0 0x557350a3eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff2bfff640 1 -- 172.21.15.83:0/411903058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a32dd0 msgr2=0x557350a3eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff2bfff640 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a32dd0 0x557350a3eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff2bfff640 1 -- 172.21.15.83:0/411903058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2dc60 msgr2=0x557350a37a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff2bfff640 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2dc60 0x557350a37a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff2bfff640 1 -- 172.21.15.83:0/411903058 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff200145e0 con 0x557350a2d270 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff2bfff640 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2d270 0x557350a37520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff20001eb0 tx=0x7eff20005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f4a55e495726b8a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff0bfff640 1 -- 172.21.15.83:0/411903058 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff20005230 con 0x557350a2d270 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff0bfff640 1 -- 172.21.15.83:0/411903058 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7eff20000e40 con 0x557350a2d270 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff35e32740 1 -- 172.21.15.83:0/411903058 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573506957e0 con 0x557350a2d270 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff0bfff640 1 -- 172.21.15.83:0/411903058 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff2000fea0 con 0x557350a2d270 2024-03-11T19:45:38.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff35e32740 10 client.26033 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff35e32740 1 -- 172.21.15.83:0/411903058 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5573509946a0 con 0x557350a2d270 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff0bfff640 1 -- 172.21.15.83:0/411903058 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5573506957e0 con 0x557350a2d270 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.566+0000 7eff0bfff640 1 -- 172.21.15.83:0/411903058 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7eff2001f440 con 0x557350a2d270 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.567+0000 7eff35e32740 10 client.26033 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.567+0000 7eff35e32740 10 client.26033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.567+0000 7eff35e32740 20 client.26033 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.567+0000 7eff35e32740 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557350a54560 0x557350a56980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.567+0000 7eff35e32740 4 client.26033 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.567+0000 7eff35e32740 1 -- 172.21.15.83:0/411903058 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573506d2720 con 0x557350a54560 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.567+0000 7eff30f1d640 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557350a54560 0x557350a56980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.567+0000 7eff0bfff640 10 client.26033 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:38.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.567+0000 7eff30f1d640 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557350a54560 0x557350a56980 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc21598372a676e server_cookie=ad969c8ebfa8ebf0 in_seq=0 out_seq=0 2024-03-11T19:45:38.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff0bfff640 1 -- 172.21.15.83:0/411903058 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5573506d2720 con 0x557350a54560 2024-03-11T19:45:38.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff0bfff640 10 client.26033 handle_command_reply: tid=0 2024-03-11T19:45:38.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 1 client.26033 shutdown 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 1 -- 172.21.15.83:0/411903058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2d270 msgr2=0x557350a37520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2d270 0x557350a37520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff20001eb0 tx=0x7eff20005c50 comp rx=0 tx=0).stop 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 1 -- 172.21.15.83:0/411903058 shutdown_connections 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557350a54560 0x557350a56980 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a32dd0 0x557350a3eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2dc60 0x557350a37a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff0bfff640 0 client.26033 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 1 --2- 172.21.15.83:0/411903058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2d270 0x557350a37520 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 1 -- 172.21.15.83:0/411903058 >> 172.21.15.83:0/411903058 conn(0x557350a287e0 msgr2=0x557350a27b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 1 -- 172.21.15.83:0/411903058 shutdown_connections 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 1 -- 172.21.15.83:0/411903058 wait complete. 2024-03-11T19:45:38.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.568+0000 7eff35e32740 20 client.26033 trim_cache size 0 max 16384 2024-03-11T19:45:38.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.591+0000 7eff35e32740 1 Processor -- start 2024-03-11T19:45:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.594+0000 7eff35e32740 1 -- start start 2024-03-11T19:45:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.595+0000 7eff35e32740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2d270 0x557350a28bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.595+0000 7eff35e32740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2dc60 0x557350a3ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.595+0000 7eff35e32740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a32dd0 0x557350a37130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.595+0000 7eff35e32740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735091b130 con 0x557350a2d270 2024-03-11T19:45:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.595+0000 7eff35e32740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557350a38710 con 0x557350a2dc60 2024-03-11T19:45:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.595+0000 7eff35e32740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557350a2e350 con 0x557350a32dd0 2024-03-11T19:45:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.595+0000 7eff30f1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2dc60 0x557350a3ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.595+0000 7eff30f1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2dc60 0x557350a3ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:38524/0 (socket says 172.21.15.83:38524) 2024-03-11T19:45:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.595+0000 7eff30f1d640 1 -- 172.21.15.83:0/65155236 learned_addr learned my addr 172.21.15.83:0/65155236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.595+0000 7eff2bfff640 1 --2- 172.21.15.83:0/65155236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a32dd0 0x557350a37130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.596+0000 7eff30f1d640 1 -- 172.21.15.83:0/65155236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a32dd0 msgr2=0x557350a37130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.596+0000 7eff30f1d640 1 --2- 172.21.15.83:0/65155236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a32dd0 0x557350a37130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.596+0000 7eff30f1d640 1 -- 172.21.15.83:0/65155236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2d270 msgr2=0x557350a28bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.596+0000 7eff30f1d640 1 --2- 172.21.15.83:0/65155236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2d270 0x557350a28bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.596+0000 7eff30f1d640 1 -- 172.21.15.83:0/65155236 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557350930c20 con 0x557350a2dc60 2024-03-11T19:45:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.596+0000 7eff30f1d640 1 --2- 172.21.15.83:0/65155236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2dc60 0x557350a3ea30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x557350a29750 tx=0x7eff1c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23be7bd5ce6eb42d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.597+0000 7eff127fc640 1 -- 172.21.15.83:0/65155236 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c017040 con 0x557350a2dc60 2024-03-11T19:45:38.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.597+0000 7eff127fc640 1 -- 172.21.15.83:0/65155236 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7eff1c018040 con 0x557350a2dc60 2024-03-11T19:45:38.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.598+0000 7eff127fc640 1 -- 172.21.15.83:0/65155236 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c01f2b0 con 0x557350a2dc60 2024-03-11T19:45:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.600+0000 7eff35e32740 1 -- 172.21.15.83:0/65155236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2dc60 msgr2=0x557350a3ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.600+0000 7eff35e32740 1 --2- 172.21.15.83:0/65155236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2dc60 0x557350a3ea30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x557350a29750 tx=0x7eff1c005f90 comp rx=0 tx=0).stop 2024-03-11T19:45:38.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.601+0000 7eff35e32740 1 -- 172.21.15.83:0/65155236 shutdown_connections 2024-03-11T19:45:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.601+0000 7eff35e32740 1 --2- 172.21.15.83:0/65155236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a32dd0 0x557350a37130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.601+0000 7eff35e32740 1 --2- 172.21.15.83:0/65155236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a2dc60 0x557350a3ea30 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x557350a29750 tx=0x7eff1c005f90 comp rx=0 tx=0).stop 2024-03-11T19:45:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.601+0000 7eff35e32740 1 --2- 172.21.15.83:0/65155236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a2d270 0x557350a28bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.601+0000 7eff35e32740 1 -- 172.21.15.83:0/65155236 >> 172.21.15.83:0/65155236 conn(0x557350a287e0 msgr2=0x557350a23a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.606+0000 7eff35e32740 1 -- 172.21.15.83:0/65155236 shutdown_connections 2024-03-11T19:45:38.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.609+0000 7eff35e32740 1 -- 172.21.15.83:0/65155236 wait complete. 2024-03-11T19:45:38.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.609+0000 7eff35e32740 1 -- start start 2024-03-11T19:45:38.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.609+0000 7eff35e32740 1 Processor -- start 2024-03-11T19:45:38.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff35e32740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 0x557350a3d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff35e32740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a32dd0 0x557350a3d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff35e32740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a3dd40 0x557350aaaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 0x557350a3d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 0x557350a3d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:55136/0 (socket says 172.21.15.83:55136) 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff2bfff640 1 -- 172.21.15.83:0/3391441012 learned_addr learned my addr 172.21.15.83:0/3391441012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff35e32740 1 -- 172.21.15.83:0/3391441012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557350a3edd0 con 0x557350a3dd40 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff35e32740 1 -- 172.21.15.83:0/3391441012 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557350aaafb0 con 0x557350a32dd0 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff35e32740 1 -- 172.21.15.83:0/3391441012 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557350aab140 con 0x557350a2d270 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff2bfff640 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a3dd40 0x557350aaaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff30f1d640 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a32dd0 0x557350a3d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.610+0000 7eff35e32740 10 client.0 _ll_register_callbacks cb 0x7eff2000c9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.611+0000 7eff2bfff640 1 -- 172.21.15.83:0/3391441012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a32dd0 msgr2=0x557350a3d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.611+0000 7eff2bfff640 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a32dd0 0x557350a3d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.611+0000 7eff2bfff640 1 -- 172.21.15.83:0/3391441012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a3dd40 msgr2=0x557350aaaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.611+0000 7eff2bfff640 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a3dd40 0x557350aaaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.611+0000 7eff2bfff640 1 -- 172.21.15.83:0/3391441012 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c009b90 con 0x557350a2d270 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.611+0000 7eff35e32740 1 -- 172.21.15.83:0/3391441012 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff2001a030 con 0x557350a2d270 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.611+0000 7eff2bfff640 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 0x557350a3d2c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7eff2001f840 tx=0x7eff2000c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b812f9d8eb5b9e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.612+0000 7eff0bfff640 10 client.26143 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.615+0000 7eff0bfff640 1 -- 172.21.15.83:0/3391441012 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2000f730 con 0x557350a2d270 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.615+0000 7eff0bfff640 1 -- 172.21.15.83:0/3391441012 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7eff20000e40 con 0x557350a2d270 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.615+0000 7eff0bfff640 1 -- 172.21.15.83:0/3391441012 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff2000f890 con 0x557350a2d270 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.618+0000 7eff35e32740 10 client.26143 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.618+0000 7eff35e32740 1 -- 172.21.15.83:0/3391441012 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=31}) v3 -- 0x5573509568c0 con 0x557350a2d270 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.618+0000 7eff0bfff640 1 -- 172.21.15.83:0/3391441012 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7eff20012660 con 0x557350a2d270 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.619+0000 7eff35e32740 10 client.26143 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.619+0000 7eff35e32740 10 client.26143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:38.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.619+0000 7eff35e32740 20 client.26143 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:38.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.619+0000 7eff35e32740 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557350aac520 0x557350aae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:38.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.619+0000 7eff35e32740 4 client.26143 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:45:38.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.619+0000 7eff35e32740 1 -- 172.21.15.83:0/3391441012 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557350721890 con 0x557350aac520 2024-03-11T19:45:38.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.620+0000 7eff30f1d640 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557350aac520 0x557350aae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:38.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.623+0000 7eff30f1d640 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557350aac520 0x557350aae910 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471ffdae6e10ef7f server_cookie=f5c0b352a57adb90 in_seq=0 out_seq=0 2024-03-11T19:45:38.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.623+0000 7eff0bfff640 10 client.26143 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:38.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.625+0000 7eff0bfff640 1 -- 172.21.15.83:0/3391441012 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557350721890 con 0x557350aac520 2024-03-11T19:45:38.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.626+0000 7eff0bfff640 10 client.26143 handle_command_reply: tid=0 2024-03-11T19:45:38.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.626+0000 7eff35e32740 1 client.26143 shutdown 2024-03-11T19:45:38.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.627+0000 7eff35e32740 1 -- 172.21.15.83:0/3391441012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 msgr2=0x557350a3d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.627+0000 7eff35e32740 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 0x557350a3d2c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7eff2001f840 tx=0x7eff2000c430 comp rx=0 tx=0).stop 2024-03-11T19:45:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.628+0000 7eff35e32740 1 -- 172.21.15.83:0/3391441012 shutdown_connections 2024-03-11T19:45:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.628+0000 7eff35e32740 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557350aac520 0x557350aae910 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.629+0000 7eff35e32740 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557350a3dd40 0x557350aaaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.629+0000 7eff35e32740 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557350a32dd0 0x557350a3d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.629+0000 7eff35e32740 1 --2- 172.21.15.83:0/3391441012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557350a2d270 0x557350a3d2c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.629+0000 7eff35e32740 1 -- 172.21.15.83:0/3391441012 >> 172.21.15.83:0/3391441012 conn(0x557350a287e0 msgr2=0x557350a28bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.629+0000 7eff0bfff640 0 client.26143 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:38.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.633+0000 7eff35e32740 1 -- 172.21.15.83:0/3391441012 shutdown_connections 2024-03-11T19:45:38.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.635+0000 7eff35e32740 1 -- 172.21.15.83:0/3391441012 wait complete. 2024-03-11T19:45:38.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.635+0000 7eff35e32740 20 client.26143 trim_cache size 0 max 16384 2024-03-11T19:45:38.641 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:45:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.636+0000 7eff33181640 1 -- 172.21.15.83:0/3362469465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff140a1ab0 msgr2=0x7eff140a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.636+0000 7eff33181640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff140a1ab0 0x7eff140a3f70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7eff24003fd0 tx=0x7eff24004490 comp rx=0 tx=0).stop 2024-03-11T19:45:38.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.636+0000 7eff33181640 1 -- 172.21.15.83:0/3362469465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c101c70 msgr2=0x7eff2c19c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.636+0000 7eff33181640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c101c70 0x7eff2c19c5c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7eff24039040 tx=0x7eff24003c20 comp rx=0 tx=0).stop 2024-03-11T19:45:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.637+0000 7eff33181640 1 -- 172.21.15.83:0/3362469465 shutdown_connections 2024-03-11T19:45:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.637+0000 7eff33181640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7eff140a1ab0 0x7eff140a3f70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.637+0000 7eff33181640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c101c70 0x7eff2c19c5c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.637+0000 7eff33181640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c1012b0 0x7eff2c195550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.637+0000 7eff33181640 1 --2- 172.21.15.83:0/3362469465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c1005a0 0x7eff2c195010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.637+0000 7eff33181640 1 -- 172.21.15.83:0/3362469465 >> 172.21.15.83:0/3362469465 conn(0x7eff2c0fad70 msgr2=0x7eff2c104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.637+0000 7eff33181640 1 -- 172.21.15.83:0/3362469465 shutdown_connections 2024-03-11T19:45:38.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:38.637+0000 7eff33181640 1 -- 172.21.15.83:0/3362469465 wait complete. 2024-03-11T19:45:39.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:38 smithi083 ceph-mon[31864]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 19 MiB/s wr, 795 op/s 2024-03-11T19:45:39.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:38 smithi137 ceph-mon[36220]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 19 MiB/s wr, 795 op/s 2024-03-11T19:45:39.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:38 smithi204 ceph-mon[36739]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 19 MiB/s wr, 795 op/s 2024-03-11T19:45:39.654 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:45:39.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:45:39.654 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:45:39.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.821+0000 7fc7b99a0640 1 -- 172.21.15.83:0/2684653335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7b41010e0 msgr2=0x7fc7b4101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.821+0000 7fc7b99a0640 1 --2- 172.21.15.83:0/2684653335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7b41010e0 0x7fc7b4101560 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009890 tx=0x7fc7a802bb70 comp rx=0 tx=0).stop 2024-03-11T19:45:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.821+0000 7fc7b99a0640 1 -- 172.21.15.83:0/2684653335 shutdown_connections 2024-03-11T19:45:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.821+0000 7fc7b99a0640 1 --2- 172.21.15.83:0/2684653335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b4101aa0 0x7fc7b41083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.821+0000 7fc7b99a0640 1 --2- 172.21.15.83:0/2684653335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7b41010e0 0x7fc7b4101560 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.821+0000 7fc7b99a0640 1 --2- 172.21.15.83:0/2684653335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7b40fff10 0x7fc7b4100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.821+0000 7fc7b99a0640 1 -- 172.21.15.83:0/2684653335 >> 172.21.15.83:0/2684653335 conn(0x7fc7b40fb4f0 msgr2=0x7fc7b40fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.821+0000 7fc7b99a0640 1 -- 172.21.15.83:0/2684653335 shutdown_connections 2024-03-11T19:45:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.821+0000 7fc7b99a0640 1 -- 172.21.15.83:0/2684653335 wait complete. 2024-03-11T19:45:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.822+0000 7fc7b99a0640 1 Processor -- start 2024-03-11T19:45:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.822+0000 7fc7b99a0640 1 -- start start 2024-03-11T19:45:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.822+0000 7fc7b99a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b40fff10 0x7fc7b4196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.822+0000 7fc7b99a0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7b41010e0 0x7fc7b4196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b99a0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7b4101aa0 0x7fc7b419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b99a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b419db10 con 0x7fc7b40fff10 2024-03-11T19:45:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b99a0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b419dc80 con 0x7fc7b41010e0 2024-03-11T19:45:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b99a0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b419de40 con 0x7fc7b4101aa0 2024-03-11T19:45:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b919f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7b4101aa0 0x7fc7b419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b919f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7b4101aa0 0x7fc7b419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53906/0 (socket says 172.21.15.83:53906) 2024-03-11T19:45:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b919f640 1 -- 172.21.15.83:0/3427751045 learned_addr learned my addr 172.21.15.83:0/3427751045 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b3fff640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7b41010e0 0x7fc7b4196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b899e640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b40fff10 0x7fc7b4196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b919f640 1 -- 172.21.15.83:0/3427751045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7b41010e0 msgr2=0x7fc7b4196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b919f640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7b41010e0 0x7fc7b4196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b919f640 1 -- 172.21.15.83:0/3427751045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b40fff10 msgr2=0x7fc7b4196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b919f640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b40fff10 0x7fc7b4196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.823+0000 7fc7b919f640 1 -- 172.21.15.83:0/3427751045 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a8009490 con 0x7fc7b4101aa0 2024-03-11T19:45:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.824+0000 7fc7b919f640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7b4101aa0 0x7fc7b419d5d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00eaa0 tx=0x7fc7ac00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.824+0000 7fc7b1ffb640 1 -- 172.21.15.83:0/3427751045 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac004d20 con 0x7fc7b4101aa0 2024-03-11T19:45:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.825+0000 7fc7b1ffb640 1 -- 172.21.15.83:0/3427751045 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7ac004e80 con 0x7fc7b4101aa0 2024-03-11T19:45:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.825+0000 7fc7b99a0640 1 -- 172.21.15.83:0/3427751045 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b4068340 con 0x7fc7b4101aa0 2024-03-11T19:45:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.825+0000 7fc7b1ffb640 1 -- 172.21.15.83:0/3427751045 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac005a10 con 0x7fc7b4101aa0 2024-03-11T19:45:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.825+0000 7fc7b99a0640 1 -- 172.21.15.83:0/3427751045 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b4068820 con 0x7fc7b4101aa0 2024-03-11T19:45:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.827+0000 7fc7b1ffb640 1 -- 172.21.15.83:0/3427751045 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc7ac005020 con 0x7fc7b4101aa0 2024-03-11T19:45:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.828+0000 7fc7b1ffb640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc79c0a1ab0 0x7fc79c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.829+0000 7fc7b1ffb640 1 -- 172.21.15.83:0/3427751045 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc7ac0cd540 con 0x7fc7b4101aa0 2024-03-11T19:45:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.829+0000 7fc7bd7d5740 1 Processor -- start 2024-03-11T19:45:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.829+0000 7fc7bd7d5740 1 -- start start 2024-03-11T19:45:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.829+0000 7fc7b899e640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc79c0a1ab0 0x7fc79c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.829+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d415cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.830+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415d860 0x55a9d415dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.830+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415e220 0x55a9d4164ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.830+0000 7fc7b899e640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc79c0a1ab0 0x7fc79c0a3f70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40ffc30 tx=0x7fc7a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.830+0000 7fc7b899e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d415cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.830+0000 7fc7b899e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d415cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53902/0 (socket says 172.21.15.83:53902) 2024-03-11T19:45:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.830+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d40cbf90 con 0x55a9d415c690 2024-03-11T19:45:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.830+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d40c94c0 con 0x55a9d415e220 2024-03-11T19:45:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.830+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d40c4ca0 con 0x55a9d415d860 2024-03-11T19:45:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.830+0000 7fc7b899e640 1 -- 172.21.15.83:0/4292804410 learned_addr learned my addr 172.21.15.83:0/4292804410 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.830+0000 7fc7b919f640 1 --2- 172.21.15.83:0/4292804410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415e220 0x55a9d4164ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.830+0000 7fc7b3fff640 1 --2- 172.21.15.83:0/4292804410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415d860 0x55a9d415dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.831+0000 7fc7b899e640 1 -- 172.21.15.83:0/4292804410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415d860 msgr2=0x55a9d415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.831+0000 7fc7b899e640 1 --2- 172.21.15.83:0/4292804410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415d860 0x55a9d415dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.831+0000 7fc7b899e640 1 -- 172.21.15.83:0/4292804410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415e220 msgr2=0x55a9d4164ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.831+0000 7fc7b899e640 1 --2- 172.21.15.83:0/4292804410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415e220 0x55a9d4164ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.831+0000 7fc7b899e640 1 -- 172.21.15.83:0/4292804410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9d40870b0 con 0x55a9d415c690 2024-03-11T19:45:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.831+0000 7fc7b899e640 1 --2- 172.21.15.83:0/4292804410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d415cab0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400e630 tx=0x7fc7a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3e72fc3ce84a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.832+0000 7fc79affd640 1 -- 172.21.15.83:0/4292804410 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4005020 con 0x55a9d415c690 2024-03-11T19:45:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.832+0000 7fc79affd640 1 -- 172.21.15.83:0/4292804410 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7a4021070 con 0x55a9d415c690 2024-03-11T19:45:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.833+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/4292804410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 msgr2=0x55a9d415cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.833+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/4292804410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d415cab0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400e630 tx=0x7fc7a4014000 comp rx=0 tx=0).stop 2024-03-11T19:45:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.836+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/4292804410 shutdown_connections 2024-03-11T19:45:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.836+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/4292804410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415e220 0x55a9d4164ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.836+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/4292804410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415d860 0x55a9d415dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.836+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/4292804410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d415cab0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.836+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/4292804410 >> 172.21.15.83:0/4292804410 conn(0x55a9d4157870 msgr2=0x55a9d4157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:39.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.841+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/4292804410 shutdown_connections 2024-03-11T19:45:39.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.841+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/4292804410 wait complete. 2024-03-11T19:45:39.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.841+0000 7fc7bd7d5740 1 -- start start 2024-03-11T19:45:39.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.841+0000 7fc7bd7d5740 1 Processor -- start 2024-03-11T19:45:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.842+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415c690 0x55a9d416acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.842+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415d860 0x55a9d416b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.842+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d4172270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.842+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d4069f40 con 0x55a9d415d860 2024-03-11T19:45:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.843+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d3d984b0 con 0x55a9d415c690 2024-03-11T19:45:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.843+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d404a670 con 0x55a9d415e220 2024-03-11T19:45:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.843+0000 7fc7b919f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d4172270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.843+0000 7fc7b919f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d4172270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53928/0 (socket says 172.21.15.83:53928) 2024-03-11T19:45:39.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.843+0000 7fc7b919f640 1 -- 172.21.15.83:0/2576099245 learned_addr learned my addr 172.21.15.83:0/2576099245 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:39.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.843+0000 7fc7bd7d5740 10 client.0 _ll_register_callbacks cb 0x55a9d40b7f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:39.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.843+0000 7fc7b919f640 1 -- 172.21.15.83:0/2576099245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415c690 msgr2=0x55a9d416acf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:45:39.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.843+0000 7fc7b919f640 1 --2- 172.21.15.83:0/2576099245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415c690 0x55a9d416acf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.843+0000 7fc7b919f640 1 -- 172.21.15.83:0/2576099245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415d860 msgr2=0x55a9d416b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:39.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.843+0000 7fc7b919f640 1 --2- 172.21.15.83:0/2576099245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415d860 0x55a9d416b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.843+0000 7fc7b919f640 1 -- 172.21.15.83:0/2576099245 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a4017070 con 0x55a9d415e220 2024-03-11T19:45:39.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.846+0000 7fc7b919f640 1 --2- 172.21.15.83:0/2576099245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d4172270 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0caff0 tx=0x7fc7ac09ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa9911fdc478009c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:39.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.846+0000 7fc793fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:45:39.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.846+0000 7fc793fff640 1 -- 172.21.15.83:0/2576099245 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0a2c60 con 0x55a9d415e220 2024-03-11T19:45:39.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.847+0000 7fc793fff640 1 -- 172.21.15.83:0/2576099245 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc7ac0987c0 con 0x55a9d415e220 2024-03-11T19:45:39.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.848+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/2576099245 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9d409ef40 con 0x55a9d415e220 2024-03-11T19:45:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.849+0000 7fc793fff640 1 -- 172.21.15.83:0/2576099245 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7ac005340 con 0x55a9d415e220 2024-03-11T19:45:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.851+0000 7fc7bd7d5740 10 client.26155 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.851+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/2576099245 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a9d40c3320 con 0x55a9d415e220 2024-03-11T19:45:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.853+0000 7fc793fff640 1 -- 172.21.15.83:0/2576099245 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7ac0cd320 con 0x55a9d415e220 2024-03-11T19:45:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.853+0000 7fc793fff640 1 -- 172.21.15.83:0/2576099245 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc7ac018030 con 0x55a9d415e220 2024-03-11T19:45:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.853+0000 7fc7bd7d5740 10 client.26155 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.853+0000 7fc7bd7d5740 10 client.26155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.853+0000 7fc7bd7d5740 20 client.26155 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.853+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/2576099245 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9d4176f80 0x55a9d4179410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.854+0000 7fc7bd7d5740 4 client.26155 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.854+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/2576099245 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9d409ef40 con 0x55a9d4176f80 2024-03-11T19:45:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.854+0000 7fc7b899e640 1 --2- 172.21.15.83:0/2576099245 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9d4176f80 0x55a9d4179410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.855+0000 7fc7b899e640 1 --2- 172.21.15.83:0/2576099245 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9d4176f80 0x55a9d4179410 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487feee76c3cd662 server_cookie=76705173b627a944 in_seq=0 out_seq=0 2024-03-11T19:45:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.855+0000 7fc793fff640 10 client.26155 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:39.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.857+0000 7fc793fff640 1 -- 172.21.15.83:0/2576099245 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9d409ef40 con 0x55a9d4176f80 2024-03-11T19:45:39.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.857+0000 7fc793fff640 10 client.26155 handle_command_reply: tid=0 2024-03-11T19:45:39.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.857+0000 7fc7bd7d5740 1 client.26155 shutdown 2024-03-11T19:45:39.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.857+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/2576099245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 msgr2=0x55a9d4172270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:39.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.857+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/2576099245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d4172270 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0caff0 tx=0x7fc7ac09ac80 comp rx=0 tx=0).stop 2024-03-11T19:45:39.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.858+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/2576099245 shutdown_connections 2024-03-11T19:45:39.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.858+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/2576099245 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9d4176f80 0x55a9d4179410 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.858+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/2576099245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d4172270 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.858+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/2576099245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415d860 0x55a9d416b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.858+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/2576099245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415c690 0x55a9d416acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.858+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/2576099245 >> 172.21.15.83:0/2576099245 conn(0x55a9d4157870 msgr2=0x55a9d415aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.858+0000 7fc793fff640 0 client.26155 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.862+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/2576099245 shutdown_connections 2024-03-11T19:45:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.862+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/2576099245 wait complete. 2024-03-11T19:45:39.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.864+0000 7fc7bd7d5740 20 client.26155 trim_cache size 0 max 16384 2024-03-11T19:45:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.890+0000 7fc7bd7d5740 1 Processor -- start 2024-03-11T19:45:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.890+0000 7fc7bd7d5740 1 -- start start 2024-03-11T19:45:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.891+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d416acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.891+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415d860 0x55a9d416b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.891+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d4172190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.891+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d41726d0 con 0x55a9d415c690 2024-03-11T19:45:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.891+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d409ef40 con 0x55a9d415d860 2024-03-11T19:45:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.891+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d416bee0 con 0x55a9d415e220 2024-03-11T19:45:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.891+0000 7fc7b919f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d4172190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.891+0000 7fc7b919f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d4172190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53938/0 (socket says 172.21.15.83:53938) 2024-03-11T19:45:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.891+0000 7fc7b919f640 1 -- 172.21.15.83:0/3726677815 learned_addr learned my addr 172.21.15.83:0/3726677815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.891+0000 7fc7b899e640 1 --2- 172.21.15.83:0/3726677815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d416acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.891+0000 7fc7b3fff640 1 --2- 172.21.15.83:0/3726677815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415d860 0x55a9d416b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.892+0000 7fc7b3fff640 1 -- 172.21.15.83:0/3726677815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 msgr2=0x55a9d4172190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.892+0000 7fc7b3fff640 1 --2- 172.21.15.83:0/3726677815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d4172190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.892+0000 7fc7b3fff640 1 -- 172.21.15.83:0/3726677815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 msgr2=0x55a9d416acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.892+0000 7fc7b3fff640 1 --2- 172.21.15.83:0/3726677815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d416acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.892+0000 7fc7b3fff640 1 -- 172.21.15.83:0/3726677815 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9d405fc00 con 0x55a9d415d860 2024-03-11T19:45:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.893+0000 7fc7b3fff640 1 --2- 172.21.15.83:0/3726677815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415d860 0x55a9d416b230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8037040 tx=0x7fc7a802c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23655b79191c0395 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.894+0000 7fc79a7fc640 1 -- 172.21.15.83:0/3726677815 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a802cbd0 con 0x55a9d415d860 2024-03-11T19:45:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.894+0000 7fc79a7fc640 1 -- 172.21.15.83:0/3726677815 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7a8034440 con 0x55a9d415d860 2024-03-11T19:45:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.894+0000 7fc79a7fc640 1 -- 172.21.15.83:0/3726677815 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8003b20 con 0x55a9d415d860 2024-03-11T19:45:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.895+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/3726677815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415d860 msgr2=0x55a9d416b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.895+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/3726677815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415d860 0x55a9d416b230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8037040 tx=0x7fc7a802c690 comp rx=0 tx=0).stop 2024-03-11T19:45:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.895+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/3726677815 shutdown_connections 2024-03-11T19:45:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.895+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/3726677815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d4172190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.895+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/3726677815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415d860 0x55a9d416b230 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.895+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/3726677815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d416acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.895+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/3726677815 >> 172.21.15.83:0/3726677815 conn(0x55a9d4157870 msgr2=0x55a9d417ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.895+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/3726677815 shutdown_connections 2024-03-11T19:45:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.896+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/3726677815 wait complete. 2024-03-11T19:45:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.896+0000 7fc7bd7d5740 1 -- start start 2024-03-11T19:45:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.896+0000 7fc7bd7d5740 1 Processor -- start 2024-03-11T19:45:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.896+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d4170830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.896+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415d860 0x55a9d4170d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.896+0000 7fc7bd7d5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d41712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.896+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d4172480 con 0x55a9d415c690 2024-03-11T19:45:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.896+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d41cd520 con 0x55a9d415d860 2024-03-11T19:45:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.896+0000 7fc7bd7d5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d41cd660 con 0x55a9d415e220 2024-03-11T19:45:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.896+0000 7fc7bd7d5740 10 client.0 _ll_register_callbacks cb 0x7fc7a4009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.896+0000 7fc7b899e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d4170830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc7b899e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d4170830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53930/0 (socket says 172.21.15.83:53930) 2024-03-11T19:45:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc7b899e640 1 -- 172.21.15.83:0/113588074 learned_addr learned my addr 172.21.15.83:0/113588074 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc7b919f640 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d41712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc7b3fff640 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415d860 0x55a9d4170d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc7b919f640 1 -- 172.21.15.83:0/113588074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415d860 msgr2=0x55a9d4170d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc7b919f640 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415d860 0x55a9d4170d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc7b919f640 1 -- 172.21.15.83:0/113588074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 msgr2=0x55a9d4170830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc7b919f640 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d4170830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc7b919f640 1 -- 172.21.15.83:0/113588074 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b4108c80 con 0x55a9d415e220 2024-03-11T19:45:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc7b919f640 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d41712b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac09a1c0 tx=0x7fc7ac0989d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e0f52b107d902d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc793fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:45:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.897+0000 7fc793fff640 1 -- 172.21.15.83:0/113588074 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac099e20 con 0x55a9d415e220 2024-03-11T19:45:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.898+0000 7fc793fff640 1 -- 172.21.15.83:0/113588074 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc7ac0cd320 con 0x55a9d415e220 2024-03-11T19:45:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.898+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/113588074 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9d40ea900 con 0x55a9d415e220 2024-03-11T19:45:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.898+0000 7fc793fff640 1 -- 172.21.15.83:0/113588074 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7ac0a2c60 con 0x55a9d415e220 2024-03-11T19:45:39.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.898+0000 7fc7bd7d5740 10 client.26161 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.898+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/113588074 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a9d4085910 con 0x55a9d415e220 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.898+0000 7fc793fff640 1 -- 172.21.15.83:0/113588074 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a9d40ea900 con 0x55a9d415e220 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.898+0000 7fc793fff640 1 -- 172.21.15.83:0/113588074 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc7ac018070 con 0x55a9d415e220 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.898+0000 7fc7bd7d5740 10 client.26161 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.898+0000 7fc7bd7d5740 10 client.26161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.898+0000 7fc7bd7d5740 20 client.26161 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.899+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9d41cec80 0x55a9d41d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.899+0000 7fc7bd7d5740 4 client.26161 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.899+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/113588074 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a9d40bb310 con 0x55a9d41cec80 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.899+0000 7fc7b899e640 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9d41cec80 0x55a9d41d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.899+0000 7fc793fff640 10 client.26161 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.899+0000 7fc7b899e640 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9d41cec80 0x55a9d41d10a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=203c97c5532e524f server_cookie=21245d0f302238a1 in_seq=0 out_seq=0 2024-03-11T19:45:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.900+0000 7fc793fff640 1 -- 172.21.15.83:0/113588074 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a9d40bb310 con 0x55a9d41cec80 2024-03-11T19:45:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.900+0000 7fc793fff640 10 client.26161 handle_command_reply: tid=0 2024-03-11T19:45:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.900+0000 7fc7bd7d5740 1 client.26161 shutdown 2024-03-11T19:45:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.900+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/113588074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 msgr2=0x55a9d41712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.900+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d41712b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac09a1c0 tx=0x7fc7ac0989d0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.900+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/113588074 shutdown_connections 2024-03-11T19:45:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.900+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a9d41cec80 0x55a9d41d10a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.900+0000 7fc793fff640 0 client.26161 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.900+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9d415e220 0x55a9d41712b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.901+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9d415d860 0x55a9d4170d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.901+0000 7fc7bd7d5740 1 --2- 172.21.15.83:0/113588074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d415c690 0x55a9d4170830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.901+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/113588074 >> 172.21.15.83:0/113588074 conn(0x55a9d4157870 msgr2=0x55a9d416ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.901+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/113588074 shutdown_connections 2024-03-11T19:45:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.901+0000 7fc7bd7d5740 1 -- 172.21.15.83:0/113588074 wait complete. 2024-03-11T19:45:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.901+0000 7fc7bd7d5740 20 client.26161 trim_cache size 0 max 16384 2024-03-11T19:45:39.911 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:45:39.911 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:45:39.911 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "a42bfc12-c74e-490e-bcda-06b5fa791a6f", 2024-03-11T19:45:39.911 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:45:39.911 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:45:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.902+0000 7fc7b99a0640 1 -- 172.21.15.83:0/3427751045 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc79c0a1ab0 msgr2=0x7fc79c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:39.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.902+0000 7fc7b99a0640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc79c0a1ab0 0x7fc79c0a3f70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40ffc30 tx=0x7fc7a4009260 comp rx=0 tx=0).stop 2024-03-11T19:45:39.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.903+0000 7fc7b99a0640 1 -- 172.21.15.83:0/3427751045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7b4101aa0 msgr2=0x7fc7b419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:39.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.903+0000 7fc7b99a0640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7b4101aa0 0x7fc7b419d5d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00eaa0 tx=0x7fc7ac00eea0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.903+0000 7fc7b99a0640 1 -- 172.21.15.83:0/3427751045 shutdown_connections 2024-03-11T19:45:39.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.903+0000 7fc7b99a0640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc79c0a1ab0 0x7fc79c0a3f70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.903+0000 7fc7b99a0640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7b4101aa0 0x7fc7b419d5d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.903+0000 7fc7b99a0640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7b41010e0 0x7fc7b4196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.903+0000 7fc7b99a0640 1 --2- 172.21.15.83:0/3427751045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b40fff10 0x7fc7b4196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:39.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.903+0000 7fc7b99a0640 1 -- 172.21.15.83:0/3427751045 >> 172.21.15.83:0/3427751045 conn(0x7fc7b40fb4f0 msgr2=0x7fc7b4104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:39.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.903+0000 7fc7b99a0640 1 -- 172.21.15.83:0/3427751045 shutdown_connections 2024-03-11T19:45:39.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:39.903+0000 7fc7b99a0640 1 -- 172.21.15.83:0/3427751045 wait complete. 2024-03-11T19:45:39.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:45:39.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:45:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.095+0000 7f7bc110d640 1 -- 172.21.15.83:0/1480749313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bbc108180 msgr2=0x7f7bbc10a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.095+0000 7f7bc110d640 1 --2- 172.21.15.83:0/1480749313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bbc108180 0x7f7bbc10a5b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009be0 tx=0x7f7bb402bf30 comp rx=0 tx=0).stop 2024-03-11T19:45:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.095+0000 7f7bc110d640 1 -- 172.21.15.83:0/1480749313 shutdown_connections 2024-03-11T19:45:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.095+0000 7f7bc110d640 1 --2- 172.21.15.83:0/1480749313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc10d4a0 0x7f7bbc10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.095+0000 7f7bc110d640 1 --2- 172.21.15.83:0/1480749313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bbc10aaf0 0x7f7bbc10cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.095+0000 7f7bc110d640 1 --2- 172.21.15.83:0/1480749313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bbc108180 0x7f7bbc10a5b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.095+0000 7f7bc110d640 1 -- 172.21.15.83:0/1480749313 >> 172.21.15.83:0/1480749313 conn(0x7f7bbc06e200 msgr2=0x7f7bbc070640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.096+0000 7f7bc110d640 1 -- 172.21.15.83:0/1480749313 shutdown_connections 2024-03-11T19:45:40.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.096+0000 7f7bc110d640 1 -- 172.21.15.83:0/1480749313 wait complete. 2024-03-11T19:45:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.097+0000 7f7bc110d640 1 Processor -- start 2024-03-11T19:45:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.097+0000 7f7bc110d640 1 -- start start 2024-03-11T19:45:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bc110d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bbc108180 0x7f7bbc1146b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bc110d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bbc10aaf0 0x7f7bbc114bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bc110d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc10d4a0 0x7f7bbc1b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bc110d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc117c40 con 0x7f7bbc10d4a0 2024-03-11T19:45:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bc110d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc117db0 con 0x7f7bbc108180 2024-03-11T19:45:40.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bc110d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc117f70 con 0x7f7bbc10aaf0 2024-03-11T19:45:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bbad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bbc108180 0x7f7bbc1146b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bbb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc10d4a0 0x7f7bbc1b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bbb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc10d4a0 0x7f7bbc1b62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53948/0 (socket says 172.21.15.83:53948) 2024-03-11T19:45:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bbb59e640 1 -- 172.21.15.83:0/918651758 learned_addr learned my addr 172.21.15.83:0/918651758 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:40.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bba59c640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bbc10aaf0 0x7f7bbc114bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bbb59e640 1 -- 172.21.15.83:0/918651758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bbc10aaf0 msgr2=0x7f7bbc114bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bbb59e640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bbc10aaf0 0x7f7bbc114bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bbb59e640 1 -- 172.21.15.83:0/918651758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bbc108180 msgr2=0x7f7bbc1146b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bbb59e640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bbc108180 0x7f7bbc1146b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.098+0000 7f7bbb59e640 1 -- 172.21.15.83:0/918651758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4009840 con 0x7f7bbc10d4a0 2024-03-11T19:45:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.099+0000 7f7bbb59e640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc10d4a0 0x7f7bbc1b62a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009fd0 tx=0x7f7bac00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.100+0000 7f7ba3fff640 1 -- 172.21.15.83:0/918651758 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac011830 con 0x7f7bbc10d4a0 2024-03-11T19:45:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.100+0000 7f7ba3fff640 1 -- 172.21.15.83:0/918651758 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bac011e70 con 0x7f7bbc10d4a0 2024-03-11T19:45:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.100+0000 7f7bc110d640 1 -- 172.21.15.83:0/918651758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc1b6980 con 0x7f7bbc10d4a0 2024-03-11T19:45:40.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.100+0000 7f7bc110d640 1 -- 172.21.15.83:0/918651758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc1b6e30 con 0x7f7bbc10d4a0 2024-03-11T19:45:40.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.104+0000 7f7bc3dbc740 1 Processor -- start 2024-03-11T19:45:40.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.104+0000 7f7bc3dbc740 1 -- start start 2024-03-11T19:45:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.104+0000 7f7bc3dbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac02d0 0x559ae8ac06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.104+0000 7f7bc3dbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac0cc0 0x559ae8ac57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.105+0000 7f7bba59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac02d0 0x559ae8ac06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.105+0000 7f7bba59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac02d0 0x559ae8ac06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:53968/0 (socket says 172.21.15.83:53968) 2024-03-11T19:45:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.105+0000 7f7bba59c640 1 -- 172.21.15.83:0/3205983862 learned_addr learned my addr 172.21.15.83:0/3205983862 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.105+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/3205983862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac5e30 0x559ae8ac82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.105+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/3205983862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae8a30280 con 0x559ae8ac0cc0 2024-03-11T19:45:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.105+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/3205983862 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae8a2d7b0 con 0x559ae8ac5e30 2024-03-11T19:45:40.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.105+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/3205983862 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae8a28f90 con 0x559ae8ac02d0 2024-03-11T19:45:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.106+0000 7f7bba59c640 1 --2- 172.21.15.83:0/3205983862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac5e30 0x559ae8ac82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.106+0000 7f7ba3fff640 1 -- 172.21.15.83:0/918651758 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00f2e0 con 0x7f7bbc10d4a0 2024-03-11T19:45:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.106+0000 7f7bba59c640 1 -- 172.21.15.83:0/3205983862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac5e30 msgr2=0x559ae8ac82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:40.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.106+0000 7f7bba59c640 1 --2- 172.21.15.83:0/3205983862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac5e30 0x559ae8ac82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.106+0000 7f7bba59c640 1 -- 172.21.15.83:0/3205983862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac0cc0 msgr2=0x559ae8ac57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.106+0000 7f7bba59c640 1 --2- 172.21.15.83:0/3205983862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac0cc0 0x559ae8ac57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.106+0000 7f7bba59c640 1 -- 172.21.15.83:0/3205983862 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ae89e7570 con 0x559ae8ac02d0 2024-03-11T19:45:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.107+0000 7f7bba59c640 1 --2- 172.21.15.83:0/3205983862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac02d0 0x559ae8ac06f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0013960 tx=0x7f7bb000df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a7c0740dfb5439 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.107+0000 7f7ba17fa640 1 -- 172.21.15.83:0/3205983862 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000b8c0 con 0x559ae8ac02d0 2024-03-11T19:45:40.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.107+0000 7f7ba17fa640 1 -- 172.21.15.83:0/3205983862 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bb0010450 con 0x559ae8ac02d0 2024-03-11T19:45:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.107+0000 7f7ba17fa640 1 -- 172.21.15.83:0/3205983862 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0011620 con 0x559ae8ac02d0 2024-03-11T19:45:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.108+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/3205983862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac02d0 msgr2=0x559ae8ac06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:40.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.108+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/3205983862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac02d0 0x559ae8ac06f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0013960 tx=0x7f7bb000df90 comp rx=0 tx=0).stop 2024-03-11T19:45:40.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.110+0000 7f7ba3fff640 1 -- 172.21.15.83:0/918651758 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7bac011990 con 0x7f7bbc10d4a0 2024-03-11T19:45:40.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.110+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/3205983862 shutdown_connections 2024-03-11T19:45:40.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.110+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/3205983862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac5e30 0x559ae8ac82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.110+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/3205983862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac0cc0 0x559ae8ac57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.111+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/3205983862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac02d0 0x559ae8ac06f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.111+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/3205983862 >> 172.21.15.83:0/3205983862 conn(0x559ae8abb7f0 msgr2=0x559ae8abbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:40.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.111+0000 7f7ba3fff640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7ba40a1b20 0x7f7ba40a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.111+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/3205983862 shutdown_connections 2024-03-11T19:45:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.111+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/3205983862 wait complete. 2024-03-11T19:45:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.112+0000 7f7ba3fff640 1 -- 172.21.15.83:0/918651758 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7bac0ccb20 con 0x7f7bbc10d4a0 2024-03-11T19:45:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.112+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7ba40a1b20 0x7f7ba40a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.112+0000 7f7bc3dbc740 1 -- start start 2024-03-11T19:45:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.112+0000 7f7bc3dbc740 1 Processor -- start 2024-03-11T19:45:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bc3dbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac02d0 0x559ae8aca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bc3dbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac0cc0 0x559ae8aca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bc3dbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac5e30 0x559ae8ad15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bc3dbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae86fc430 con 0x559ae8ac02d0 2024-03-11T19:45:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bc3dbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae86b8e00 con 0x559ae8ac0cc0 2024-03-11T19:45:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bc3dbc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae89adfd0 con 0x559ae8ac5e30 2024-03-11T19:45:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7ba40a1b20 0x7f7ba40a3fe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4034f50 tx=0x7f7bb4034ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bba59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac0cc0 0x559ae8aca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bc3dbc740 10 client.0 _ll_register_callbacks cb 0x559ae8a03fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bba59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac0cc0 0x559ae8aca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47152/0 (socket says 172.21.15.83:47152) 2024-03-11T19:45:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bba59c640 1 -- 172.21.15.83:0/1046632634 learned_addr learned my addr 172.21.15.83:0/1046632634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bbb59e640 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac5e30 0x559ae8ad15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bba59c640 1 -- 172.21.15.83:0/1046632634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac5e30 msgr2=0x559ae8ad15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac02d0 0x559ae8aca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bba59c640 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac5e30 0x559ae8ad15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bba59c640 1 -- 172.21.15.83:0/1046632634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac02d0 msgr2=0x559ae8aca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bba59c640 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac02d0 0x559ae8aca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bba59c640 1 -- 172.21.15.83:0/1046632634 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb0013640 con 0x559ae8ac0cc0 2024-03-11T19:45:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.113+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac02d0 0x559ae8aca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:45:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.114+0000 7f7bba59c640 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac0cc0 0x559ae8aca5b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0005320 tx=0x7f7bb0005350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c775162a923384de server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.114+0000 7f7b9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:45:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.114+0000 7f7b9eff5640 1 -- 172.21.15.83:0/1046632634 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00048c0 con 0x559ae8ac0cc0 2024-03-11T19:45:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.114+0000 7f7b9eff5640 1 -- 172.21.15.83:0/1046632634 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7bb0010930 con 0x559ae8ac0cc0 2024-03-11T19:45:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.114+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/1046632634 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ae87287e0 con 0x559ae8ac0cc0 2024-03-11T19:45:40.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.115+0000 7f7b9eff5640 1 -- 172.21.15.83:0/1046632634 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bb0010c50 con 0x559ae8ac0cc0 2024-03-11T19:45:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.116+0000 7f7bc3dbc740 10 client.26066 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.116+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/1046632634 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559ae8a27610 con 0x559ae8ac0cc0 2024-03-11T19:45:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.116+0000 7f7b9eff5640 1 -- 172.21.15.83:0/1046632634 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bb002f070 con 0x559ae8ac0cc0 2024-03-11T19:45:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.117+0000 7f7b9eff5640 1 -- 172.21.15.83:0/1046632634 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7bb0032070 con 0x559ae8ac0cc0 2024-03-11T19:45:40.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.117+0000 7f7bc3dbc740 10 client.26066 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.117+0000 7f7bc3dbc740 10 client.26066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.117+0000 7f7bc3dbc740 20 client.26066 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.117+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559ae8ad6010 0x559ae8ad8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.117+0000 7f7bc3dbc740 4 client.26066 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:45:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.117+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/1046632634 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ae87287e0 con 0x559ae8ad6010 2024-03-11T19:45:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.118+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559ae8ad6010 0x559ae8ad8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.119+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559ae8ad6010 0x559ae8ad8430 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c65a7f21a6d553 server_cookie=25d47719285ce254 in_seq=0 out_seq=0 2024-03-11T19:45:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.119+0000 7f7b9eff5640 10 client.26066 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.119+0000 7f7b9eff5640 1 -- 172.21.15.83:0/1046632634 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559ae87287e0 con 0x559ae8ad6010 2024-03-11T19:45:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.119+0000 7f7b9eff5640 10 client.26066 handle_command_reply: tid=0 2024-03-11T19:45:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.119+0000 7f7bc3dbc740 1 client.26066 shutdown 2024-03-11T19:45:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.120+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/1046632634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac0cc0 msgr2=0x559ae8aca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.120+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac0cc0 0x559ae8aca5b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0005320 tx=0x7f7bb0005350 comp rx=0 tx=0).stop 2024-03-11T19:45:40.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.120+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/1046632634 shutdown_connections 2024-03-11T19:45:40.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.120+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559ae8ad6010 0x559ae8ad8430 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.120+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac5e30 0x559ae8ad15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.120+0000 7f7b9eff5640 0 client.26066 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:40.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.120+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac0cc0 0x559ae8aca5b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.120+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/1046632634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac02d0 0x559ae8aca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.120+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/1046632634 >> 172.21.15.83:0/1046632634 conn(0x559ae8abb7f0 msgr2=0x559ae8abaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.120+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/1046632634 shutdown_connections 2024-03-11T19:45:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.120+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/1046632634 wait complete. 2024-03-11T19:45:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.121+0000 7f7bc3dbc740 20 client.26066 trim_cache size 0 max 16384 2024-03-11T19:45:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.149+0000 7f7bc3dbc740 1 Processor -- start 2024-03-11T19:45:40.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.149+0000 7f7bc3dbc740 1 -- start start 2024-03-11T19:45:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.149+0000 7f7bc3dbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac02d0 0x559ae8abf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.149+0000 7f7bc3dbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac0cc0 0x559ae8ad15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.149+0000 7f7bc3dbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac5e30 0x559ae8abb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bbad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac02d0 0x559ae8abf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bbad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac02d0 0x559ae8abf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47164/0 (socket says 172.21.15.83:47164) 2024-03-11T19:45:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bbad9d640 1 -- 172.21.15.83:0/787246723 learned_addr learned my addr 172.21.15.83:0/787246723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:40.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bbb59e640 1 --2- 172.21.15.83:0/787246723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac5e30 0x559ae8abb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/787246723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae8abc2e0 con 0x559ae8ac0cc0 2024-03-11T19:45:40.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bba59c640 1 --2- 172.21.15.83:0/787246723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac0cc0 0x559ae8ad15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/787246723 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae87287e0 con 0x559ae8ac02d0 2024-03-11T19:45:40.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/787246723 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae8ac1100 con 0x559ae8ac5e30 2024-03-11T19:45:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bbad9d640 1 -- 172.21.15.83:0/787246723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac5e30 msgr2=0x559ae8abb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/787246723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac5e30 0x559ae8abb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bbad9d640 1 -- 172.21.15.83:0/787246723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac0cc0 msgr2=0x559ae8ad15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/787246723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac0cc0 0x559ae8ad15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.150+0000 7f7bbad9d640 1 -- 172.21.15.83:0/787246723 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ae89c3ac0 con 0x559ae8ac02d0 2024-03-11T19:45:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.151+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/787246723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac02d0 0x559ae8abf550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009600 tx=0x7f7bb403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8322c57fb31fa769 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.151+0000 7f7ba0ff9640 1 -- 172.21.15.83:0/787246723 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4052070 con 0x559ae8ac02d0 2024-03-11T19:45:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.151+0000 7f7ba0ff9640 1 -- 172.21.15.83:0/787246723 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bb4003e50 con 0x559ae8ac02d0 2024-03-11T19:45:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.151+0000 7f7ba0ff9640 1 -- 172.21.15.83:0/787246723 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4004860 con 0x559ae8ac02d0 2024-03-11T19:45:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.152+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/787246723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac02d0 msgr2=0x559ae8abf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:40.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.152+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/787246723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac02d0 0x559ae8abf550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009600 tx=0x7f7bb403f000 comp rx=0 tx=0).stop 2024-03-11T19:45:40.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.152+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/787246723 shutdown_connections 2024-03-11T19:45:40.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.152+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/787246723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac5e30 0x559ae8abb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.152+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/787246723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac0cc0 0x559ae8ad15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.153+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/787246723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac02d0 0x559ae8abf550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.153+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/787246723 >> 172.21.15.83:0/787246723 conn(0x559ae8abb7f0 msgr2=0x559ae8abbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:40.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.153+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/787246723 shutdown_connections 2024-03-11T19:45:40.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.153+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/787246723 wait complete. 2024-03-11T19:45:40.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.153+0000 7f7bc3dbc740 1 -- start start 2024-03-11T19:45:40.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.153+0000 7f7bc3dbc740 1 Processor -- start 2024-03-11T19:45:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.154+0000 7f7bc3dbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac02d0 0x559ae8ad1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.154+0000 7f7bc3dbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac0cc0 0x559ae8acf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.154+0000 7f7bc3dbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac5e30 0x559ae8acfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.154+0000 7f7bc3dbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae8ad18f0 con 0x559ae8ac02d0 2024-03-11T19:45:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.154+0000 7f7bc3dbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae8ad0190 con 0x559ae8ac5e30 2024-03-11T19:45:40.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.154+0000 7f7bc3dbc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ae8ad0350 con 0x559ae8ac0cc0 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.154+0000 7f7bc3dbc740 10 client.0 _ll_register_callbacks cb 0x7f7bb000d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.154+0000 7f7bbb59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac5e30 0x559ae8acfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.155+0000 7f7bba59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac0cc0 0x559ae8acf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.155+0000 7f7bba59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac0cc0 0x559ae8acf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:54006/0 (socket says 172.21.15.83:54006) 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.155+0000 7f7bba59c640 1 -- 172.21.15.83:0/575930283 learned_addr learned my addr 172.21.15.83:0/575930283 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.155+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac02d0 0x559ae8ad1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.155+0000 7f7bba59c640 1 -- 172.21.15.83:0/575930283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac5e30 msgr2=0x559ae8acfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.155+0000 7f7bba59c640 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac5e30 0x559ae8acfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.155+0000 7f7bba59c640 1 -- 172.21.15.83:0/575930283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac02d0 msgr2=0x559ae8ad1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.155+0000 7f7bba59c640 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac02d0 0x559ae8ad1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.155+0000 7f7bba59c640 1 -- 172.21.15.83:0/575930283 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc10e460 con 0x559ae8ac0cc0 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.155+0000 7f7bba59c640 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac0cc0 0x559ae8acf710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x559ae8acb550 tx=0x7f7bb00138d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e4ba3c136ab243a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.155+0000 7f7b9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:45:40.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.156+0000 7f7b9eff5640 1 -- 172.21.15.83:0/575930283 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000d570 con 0x559ae8ac0cc0 2024-03-11T19:45:40.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.156+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/575930283 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bb0010450 con 0x559ae8ac0cc0 2024-03-11T19:45:40.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.156+0000 7f7b9eff5640 1 -- 172.21.15.83:0/575930283 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7bb000de60 con 0x559ae8ac0cc0 2024-03-11T19:45:40.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.156+0000 7f7b9eff5640 1 -- 172.21.15.83:0/575930283 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bb0010c20 con 0x559ae8ac0cc0 2024-03-11T19:45:40.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.156+0000 7f7bc3dbc740 10 client.26179 fetch_fsmap learned FSMap version 31 2024-03-11T19:45:40.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.156+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/575930283 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559ae89e96a0 con 0x559ae8ac0cc0 2024-03-11T19:45:40.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.156+0000 7f7b9eff5640 1 -- 172.21.15.83:0/575930283 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bb0010450 con 0x559ae8ac0cc0 2024-03-11T19:45:40.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.157+0000 7f7b9eff5640 1 -- 172.21.15.83:0/575930283 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7bb0032040 con 0x559ae8ac0cc0 2024-03-11T19:45:40.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.157+0000 7f7bc3dbc740 10 client.26179 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:45:40.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.157+0000 7f7bc3dbc740 10 client.26179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:45:40.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.157+0000 7f7bc3dbc740 20 client.26179 populate_metadata read hostname 'smithi083' 2024-03-11T19:45:40.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.157+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559ae8ad6010 0x559ae8b2f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:45:40.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.157+0000 7f7bc3dbc740 4 client.26179 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:45:40.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.157+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/575930283 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559ae8765720 con 0x559ae8ad6010 2024-03-11T19:45:40.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.158+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559ae8ad6010 0x559ae8b2f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.158+0000 7f7b9eff5640 10 client.26179 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.158+0000 7f7bbad9d640 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559ae8ad6010 0x559ae8b2f8f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61247d4f40ad51b9 server_cookie=5f29a208cff4d9d0 in_seq=0 out_seq=0 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.158+0000 7f7b9eff5640 1 -- 172.21.15.83:0/575930283 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559ae8765720 con 0x559ae8ad6010 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.158+0000 7f7b9eff5640 10 client.26179 handle_command_reply: tid=0 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.158+0000 7f7bc3dbc740 1 client.26179 shutdown 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/575930283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac0cc0 msgr2=0x559ae8acf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac0cc0 0x559ae8acf710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x559ae8acb550 tx=0x7f7bb00138d0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/575930283 shutdown_connections 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559ae8ad6010 0x559ae8b2f8f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559ae8ac5e30 0x559ae8acfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559ae8ac0cc0 0x559ae8acf710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7bc3dbc740 1 --2- 172.21.15.83:0/575930283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ae8ac02d0 0x559ae8ad1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/575930283 >> 172.21.15.83:0/575930283 conn(0x559ae8abb7f0 msgr2=0x559ae8adcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7b9eff5640 0 client.26179 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:45:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/575930283 shutdown_connections 2024-03-11T19:45:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7bc3dbc740 1 -- 172.21.15.83:0/575930283 wait complete. 2024-03-11T19:45:40.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.159+0000 7f7bc3dbc740 20 client.26179 trim_cache size 0 max 16384 2024-03-11T19:45:40.167 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:45:40.167 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:45:40.167 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:45:40.167 INFO:teuthology.orchestra.run.smithi083.stdout: "a42bfc12-c74e-490e-bcda-06b5fa791a6f": { 2024-03-11T19:45:40.167 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:45:40.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "a42bfc12-c74e-490e-bcda-06b5fa791a6f", 2024-03-11T19:45:40.167 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:45:40.168 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:45:40.168 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:45:40.168 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:45:40.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.160+0000 7f7bc110d640 1 -- 172.21.15.83:0/918651758 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7ba40a1b20 msgr2=0x7f7ba40a3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.160+0000 7f7bc110d640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7ba40a1b20 0x7f7ba40a3fe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4034f50 tx=0x7f7bb4034ee0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.160+0000 7f7bc110d640 1 -- 172.21.15.83:0/918651758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc10d4a0 msgr2=0x7f7bbc1b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:45:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.160+0000 7f7bc110d640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc10d4a0 0x7f7bbc1b62a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009fd0 tx=0x7f7bac00b8e0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.161+0000 7f7bc110d640 1 -- 172.21.15.83:0/918651758 shutdown_connections 2024-03-11T19:45:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.161+0000 7f7bc110d640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7ba40a1b20 0x7f7ba40a3fe0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.161+0000 7f7bc110d640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc10d4a0 0x7f7bbc1b62a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.161+0000 7f7bc110d640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bbc10aaf0 0x7f7bbc114bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.161+0000 7f7bc110d640 1 --2- 172.21.15.83:0/918651758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bbc108180 0x7f7bbc1146b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:45:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.161+0000 7f7bc110d640 1 -- 172.21.15.83:0/918651758 >> 172.21.15.83:0/918651758 conn(0x7f7bbc06e200 msgr2=0x7f7bbc10ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:45:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.161+0000 7f7bc110d640 1 -- 172.21.15.83:0/918651758 shutdown_connections 2024-03-11T19:45:40.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:45:40.161+0000 7f7bc110d640 1 -- 172.21.15.83:0/918651758 wait complete. 2024-03-11T19:45:40.181 INFO:tasks.cephfs.filesystem:scrub status for tag:a42bfc12-c74e-490e-bcda-06b5fa791a6f - {'path': '/', 'tag': 'a42bfc12-c74e-490e-bcda-06b5fa791a6f', 'options': 'recursive,force'} 2024-03-11T19:45:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:40 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:45:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:40 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:40.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:40 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:45:40.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:40 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:40.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:40 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:45:40.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:40 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:41.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:41 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:45:41.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:41 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:41.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:41 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:45:41.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:41 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:45:41.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:41 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:41.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:41 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:41.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:41 smithi137 ceph-mon[36220]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 17 MiB/s wr, 710 op/s 2024-03-11T19:45:41.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:41 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:45:41.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:41 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:41.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:41 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:45:41.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:41 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:45:41.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:41 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:41.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:41 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:41.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:41 smithi204 ceph-mon[36739]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 17 MiB/s wr, 710 op/s 2024-03-11T19:45:41.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:41 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:45:41.571 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:41 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:41.571 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:41 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:45:41.571 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:41 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:45:41.571 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:41 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:41.571 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:41 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:45:41.571 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:41 smithi083 ceph-mon[31864]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 17 MiB/s wr, 710 op/s 2024-03-11T19:45:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 385.0 s, 2274.4 tps, lat 43.767 ms stddev 71.732 2024-03-11T19:45:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:42 smithi083 ceph-mon[31864]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 23 MiB/s wr, 951 op/s 2024-03-11T19:45:43.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:42 smithi137 ceph-mon[36220]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 23 MiB/s wr, 951 op/s 2024-03-11T19:45:43.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:42 smithi204 ceph-mon[36739]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 23 MiB/s wr, 951 op/s 2024-03-11T19:45:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:44 smithi083 ceph-mon[31864]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 38 MiB/s wr, 1.41k op/s 2024-03-11T19:45:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:44 smithi137 ceph-mon[36220]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 38 MiB/s wr, 1.41k op/s 2024-03-11T19:45:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:44 smithi204 ceph-mon[36739]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 38 MiB/s wr, 1.41k op/s 2024-03-11T19:45:46.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:45 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:45:46.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:45 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:45:46.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:45 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:45:47.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:46 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:45:47.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:46 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:45:47.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:46 smithi083 ceph-mon[31864]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 MiB/s wr, 1.49k op/s 2024-03-11T19:45:47.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:46 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:45:47.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:46 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:45:47.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:46 smithi137 ceph-mon[36220]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 MiB/s wr, 1.49k op/s 2024-03-11T19:45:47.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:46 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:45:47.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:46 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:45:47.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:46 smithi204 ceph-mon[36739]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 MiB/s wr, 1.49k op/s 2024-03-11T19:45:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 390.0 s, 2397.0 tps, lat 41.836 ms stddev 47.236 2024-03-11T19:45:49.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:48 smithi083 ceph-mon[31864]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 1.60k op/s 2024-03-11T19:45:49.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:48 smithi137 ceph-mon[36220]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 1.60k op/s 2024-03-11T19:45:49.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:48 smithi204 ceph-mon[36739]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 44 MiB/s wr, 1.60k op/s 2024-03-11T19:45:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:50 smithi083 ceph-mon[31864]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 1.49k op/s 2024-03-11T19:45:51.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:50 smithi137 ceph-mon[36220]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 1.49k op/s 2024-03-11T19:45:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:50 smithi204 ceph-mon[36739]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 40 MiB/s wr, 1.49k op/s 2024-03-11T19:45:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 395.0 s, 2666.4 tps, lat 37.435 ms stddev 43.403 2024-03-11T19:45:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:52 smithi083 ceph-mon[31864]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 40 MiB/s wr, 1.50k op/s 2024-03-11T19:45:53.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:52 smithi137 ceph-mon[36220]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 40 MiB/s wr, 1.50k op/s 2024-03-11T19:45:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:52 smithi204 ceph-mon[36739]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 40 MiB/s wr, 1.50k op/s 2024-03-11T19:45:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:54 smithi083 ceph-mon[31864]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 1.36k op/s 2024-03-11T19:45:55.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:54 smithi137 ceph-mon[36220]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 1.36k op/s 2024-03-11T19:45:55.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:54 smithi204 ceph-mon[36739]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 35 MiB/s wr, 1.36k op/s 2024-03-11T19:45:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:56 smithi083 ceph-mon[31864]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 863 op/s 2024-03-11T19:45:57.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:56 smithi137 ceph-mon[36220]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 863 op/s 2024-03-11T19:45:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:56 smithi204 ceph-mon[36739]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 18 MiB/s wr, 863 op/s 2024-03-11T19:45:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 400.0 s, 2930.4 tps, lat 33.965 ms stddev 40.595 2024-03-11T19:45:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:45:58 smithi083 ceph-mon[31864]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 876 op/s 2024-03-11T19:45:59.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:45:58 smithi137 ceph-mon[36220]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 876 op/s 2024-03-11T19:45:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:45:58 smithi204 ceph-mon[36739]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 876 op/s 2024-03-11T19:46:01.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:00 smithi083 ceph-mon[31864]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 734 op/s 2024-03-11T19:46:01.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:00 smithi137 ceph-mon[36220]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 734 op/s 2024-03-11T19:46:01.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:00 smithi204 ceph-mon[36739]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 734 op/s 2024-03-11T19:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 405.0 s, 1905.2 tps, lat 52.490 ms stddev 58.150 2024-03-11T19:46:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:02 smithi083 ceph-mon[31864]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 752 op/s 2024-03-11T19:46:03.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:02 smithi137 ceph-mon[36220]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 752 op/s 2024-03-11T19:46:03.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:02 smithi204 ceph-mon[36739]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 752 op/s 2024-03-11T19:46:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:04 smithi083 ceph-mon[31864]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 MiB/s wr, 898 op/s 2024-03-11T19:46:05.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:04 smithi137 ceph-mon[36220]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 MiB/s wr, 898 op/s 2024-03-11T19:46:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:04 smithi204 ceph-mon[36739]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 12 MiB/s wr, 898 op/s 2024-03-11T19:46:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:06 smithi083 ceph-mon[31864]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 MiB/s wr, 770 op/s 2024-03-11T19:46:07.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:06 smithi137 ceph-mon[36220]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 MiB/s wr, 770 op/s 2024-03-11T19:46:07.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:06 smithi204 ceph-mon[36739]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 10 MiB/s wr, 770 op/s 2024-03-11T19:46:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 410.0 s, 2577.6 tps, lat 39.111 ms stddev 42.279 2024-03-11T19:46:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:08 smithi083 ceph-mon[31864]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 MiB/s wr, 895 op/s 2024-03-11T19:46:09.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:08 smithi137 ceph-mon[36220]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 MiB/s wr, 895 op/s 2024-03-11T19:46:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:08 smithi204 ceph-mon[36739]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 MiB/s wr, 895 op/s 2024-03-11T19:46:10.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:46:10.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:46:10.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.320+0000 7f91e7e9d640 1 -- 172.21.15.83:0/1429817552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e00ff9f0 msgr2=0x7f91e00ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.320+0000 7f91e7e9d640 1 --2- 172.21.15.83:0/1429817552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e00ff9f0 0x7f91e00ffe10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009860 tx=0x7f91dc02bb70 comp rx=0 tx=0).stop 2024-03-11T19:46:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.320+0000 7f91e7e9d640 1 -- 172.21.15.83:0/1429817552 shutdown_connections 2024-03-11T19:46:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.320+0000 7f91e7e9d640 1 --2- 172.21.15.83:0/1429817552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e0100cf0 0x7f91e0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.320+0000 7f91e7e9d640 1 --2- 172.21.15.83:0/1429817552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91e0100350 0x7f91e01007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.320+0000 7f91e7e9d640 1 --2- 172.21.15.83:0/1429817552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e00ff9f0 0x7f91e00ffe10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.320+0000 7f91e7e9d640 1 -- 172.21.15.83:0/1429817552 >> 172.21.15.83:0/1429817552 conn(0x7f91e00fb5f0 msgr2=0x7f91e00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.320+0000 7f91e7e9d640 1 -- 172.21.15.83:0/1429817552 shutdown_connections 2024-03-11T19:46:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.320+0000 7f91e7e9d640 1 -- 172.21.15.83:0/1429817552 wait complete. 2024-03-11T19:46:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e7e9d640 1 Processor -- start 2024-03-11T19:46:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e7e9d640 1 -- start start 2024-03-11T19:46:10.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e7e9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e00ff9f0 0x7f91e0199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e7e9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91e0100350 0x7f91e0199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e7e9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e0100cf0 0x7f91e01a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e7e9d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e01a0ef0 con 0x7f91e0100350 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e7e9d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e01a1060 con 0x7f91e0100cf0 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e7e9d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e01a1220 con 0x7f91e00ff9f0 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e643a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e0100cf0 0x7f91e01a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e643a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e0100cf0 0x7f91e01a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41676/0 (socket says 172.21.15.83:41676) 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e5c39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e00ff9f0 0x7f91e0199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e643a640 1 -- 172.21.15.83:0/3848844125 learned_addr learned my addr 172.21.15.83:0/3848844125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.321+0000 7f91e5438640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91e0100350 0x7f91e0199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91e5c39640 1 -- 172.21.15.83:0/3848844125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e0100cf0 msgr2=0x7f91e01a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91e5c39640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e0100cf0 0x7f91e01a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91e5c39640 1 -- 172.21.15.83:0/3848844125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91e0100350 msgr2=0x7f91e0199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91e5c39640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91e0100350 0x7f91e0199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91e5c39640 1 -- 172.21.15.83:0/3848844125 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc009490 con 0x7f91e00ff9f0 2024-03-11T19:46:10.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91e643a640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e0100cf0 0x7f91e01a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:10.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91e5c39640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e00ff9f0 0x7f91e0199400 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91dc02c0b0 tx=0x7f91dc0045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:10.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91d6ffd640 1 -- 172.21.15.83:0/3848844125 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc02c510 con 0x7f91e00ff9f0 2024-03-11T19:46:10.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91d6ffd640 1 -- 172.21.15.83:0/3848844125 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91dc02cb50 con 0x7f91e00ff9f0 2024-03-11T19:46:10.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91e7e9d640 1 -- 172.21.15.83:0/3848844125 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e01a14a0 con 0x7f91e00ff9f0 2024-03-11T19:46:10.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91d6ffd640 1 -- 172.21.15.83:0/3848844125 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc032600 con 0x7f91e00ff9f0 2024-03-11T19:46:10.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.322+0000 7f91e7e9d640 1 -- 172.21.15.83:0/3848844125 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e01a1810 con 0x7f91e00ff9f0 2024-03-11T19:46:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.324+0000 7f91d6ffd640 1 -- 172.21.15.83:0/3848844125 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f91dc03b040 con 0x7f91e00ff9f0 2024-03-11T19:46:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.325+0000 7f91eaa7a740 1 Processor -- start 2024-03-11T19:46:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.325+0000 7f91d6ffd640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f91cc0a1a30 0x7f91cc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.325+0000 7f91d6ffd640 1 -- 172.21.15.83:0/3848844125 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f91dc0ef1a0 con 0x7f91e00ff9f0 2024-03-11T19:46:10.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.326+0000 7f91eaa7a740 1 -- start start 2024-03-11T19:46:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.326+0000 7f91eaa7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed37f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.326+0000 7f91eaa7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed3cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.326+0000 7f91eaa7a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed3fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.326+0000 7f91eaa7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00ec9f280 con 0x55a00ed37ae0 2024-03-11T19:46:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.326+0000 7f91eaa7a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00ec9c7b0 con 0x55a00ed3d640 2024-03-11T19:46:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.326+0000 7f91eaa7a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00ec97f90 con 0x55a00ed384d0 2024-03-11T19:46:10.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.326+0000 7f91e643a640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f91cc0a1a30 0x7f91cc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e5c39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed37f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e5c39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed37f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43732/0 (socket says 172.21.15.83:43732) 2024-03-11T19:46:10.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e5c39640 1 -- 172.21.15.83:0/3530226243 learned_addr learned my addr 172.21.15.83:0/3530226243 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:10.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e643a640 1 --2- 172.21.15.83:0/3530226243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed3fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e5438640 1 --2- 172.21.15.83:0/3530226243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed3cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e5c39640 1 -- 172.21.15.83:0/3530226243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 msgr2=0x55a00ed3cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e5c39640 1 --2- 172.21.15.83:0/3530226243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed3cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e5c39640 1 -- 172.21.15.83:0/3530226243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 msgr2=0x55a00ed3fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e5c39640 1 --2- 172.21.15.83:0/3530226243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed3fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e5c39640 1 -- 172.21.15.83:0/3530226243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a00ec56570 con 0x55a00ed37ae0 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e643a640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f91cc0a1a30 0x7f91cc0a3ef0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009fd0 tx=0x7f91d8011810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.327+0000 7f91e5c39640 1 --2- 172.21.15.83:0/3530226243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed37f00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0c4430 tx=0x7f91dc02bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844832d02ee1735b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.328+0000 7f91cbfff640 1 -- 172.21.15.83:0/3530226243 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0bc1c0 con 0x55a00ed37ae0 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.328+0000 7f91cbfff640 1 -- 172.21.15.83:0/3530226243 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91dc0c0310 con 0x55a00ed37ae0 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.328+0000 7f91cbfff640 1 -- 172.21.15.83:0/3530226243 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0bc420 con 0x55a00ed37ae0 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.328+0000 7f91eaa7a740 1 -- 172.21.15.83:0/3530226243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 msgr2=0x55a00ed37f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.328+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/3530226243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed37f00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0c4430 tx=0x7f91dc02bb70 comp rx=0 tx=0).stop 2024-03-11T19:46:10.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.328+0000 7f91eaa7a740 1 -- 172.21.15.83:0/3530226243 shutdown_connections 2024-03-11T19:46:10.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.328+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/3530226243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed3fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.328+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/3530226243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed3cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.328+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/3530226243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed37f00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.328+0000 7f91eaa7a740 1 -- 172.21.15.83:0/3530226243 >> 172.21.15.83:0/3530226243 conn(0x55a00ed33000 msgr2=0x55a00ed33400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:10.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.329+0000 7f91eaa7a740 1 -- 172.21.15.83:0/3530226243 shutdown_connections 2024-03-11T19:46:10.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.329+0000 7f91eaa7a740 1 -- 172.21.15.83:0/3530226243 wait complete. 2024-03-11T19:46:10.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.329+0000 7f91eaa7a740 1 -- start start 2024-03-11T19:46:10.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.329+0000 7f91eaa7a740 1 Processor -- start 2024-03-11T19:46:10.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.329+0000 7f91eaa7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed2b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.329+0000 7f91eaa7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed2bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.329+0000 7f91eaa7a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed30d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91eaa7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e96b430 con 0x55a00ed37ae0 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91eaa7a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e927e00 con 0x55a00ed3d640 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91eaa7a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00ec1cfd0 con 0x55a00ed384d0 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e5c39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed2b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91eaa7a740 10 client.0 _ll_register_callbacks cb 0x55a00ec72fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e5438640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed2bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e5c39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed2b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43748/0 (socket says 172.21.15.83:43748) 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e5c39640 1 -- 172.21.15.83:0/921907928 learned_addr learned my addr 172.21.15.83:0/921907928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e643a640 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed30d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e5438640 1 -- 172.21.15.83:0/921907928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 msgr2=0x55a00ed30d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e5438640 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed30d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e5438640 1 -- 172.21.15.83:0/921907928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 msgr2=0x55a00ed2b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:10.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e5438640 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed2b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e5438640 1 -- 172.21.15.83:0/921907928 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e01a1810 con 0x55a00ed384d0 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e5c39640 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed2b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91e5438640 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed2bd00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55a00ed388d0 tx=0x7f91d0012aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=906dfdf108c01a5b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91c97fa640 1 -- 172.21.15.83:0/921907928 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0017070 con 0x55a00ed384d0 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91eaa7a740 1 -- 172.21.15.83:0/921907928 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a00e9977e0 con 0x55a00ed384d0 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.330+0000 7f91c97fa640 1 -- 172.21.15.83:0/921907928 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f91d0004050 con 0x55a00ed384d0 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.331+0000 7f91c97fa640 1 -- 172.21.15.83:0/921907928 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91d0004210 con 0x55a00ed384d0 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.331+0000 7f91eaa7a740 10 client.26191 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.331+0000 7f91eaa7a740 1 -- 172.21.15.83:0/921907928 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a00ec96610 con 0x55a00ed384d0 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.331+0000 7f91c97fa640 1 -- 172.21.15.83:0/921907928 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a00e9977e0 con 0x55a00ed384d0 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.331+0000 7f91c97fa640 1 -- 172.21.15.83:0/921907928 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f91d0025070 con 0x55a00ed384d0 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.331+0000 7f91eaa7a740 10 client.26191 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:10.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.331+0000 7f91eaa7a740 10 client.26191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.331+0000 7f91eaa7a740 20 client.26191 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.331+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a00ed560c0 0x55a00ed584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.332+0000 7f91eaa7a740 4 client.26191 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.332+0000 7f91eaa7a740 1 -- 172.21.15.83:0/921907928 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a00e9d4720 con 0x55a00ed560c0 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.332+0000 7f91e5c39640 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a00ed560c0 0x55a00ed584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.332+0000 7f91c97fa640 10 client.26191 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.332+0000 7f91e5c39640 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a00ed560c0 0x55a00ed584e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a76a92f8ee069fe5 server_cookie=38c83b08e0176925 in_seq=0 out_seq=0 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91c97fa640 1 -- 172.21.15.83:0/921907928 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a00e9d4720 con 0x55a00ed560c0 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91c97fa640 10 client.26191 handle_command_reply: tid=0 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 1 client.26191 shutdown 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 1 -- 172.21.15.83:0/921907928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 msgr2=0x55a00ed2bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed2bd00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55a00ed388d0 tx=0x7f91d0012aa0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 1 -- 172.21.15.83:0/921907928 shutdown_connections 2024-03-11T19:46:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a00ed560c0 0x55a00ed584e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed30d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed2bd00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/921907928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed2b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 1 -- 172.21.15.83:0/921907928 >> 172.21.15.83:0/921907928 conn(0x55a00ed33000 msgr2=0x55a00ed29a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91c97fa640 0 client.26191 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 1 -- 172.21.15.83:0/921907928 shutdown_connections 2024-03-11T19:46:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 1 -- 172.21.15.83:0/921907928 wait complete. 2024-03-11T19:46:10.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.333+0000 7f91eaa7a740 20 client.26191 trim_cache size 0 max 16384 2024-03-11T19:46:10.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.352+0000 7f91eaa7a740 1 Processor -- start 2024-03-11T19:46:10.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.352+0000 7f91eaa7a740 1 -- start start 2024-03-11T19:46:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.352+0000 7f91eaa7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.352+0000 7f91eaa7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed66900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.352+0000 7f91eaa7a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed2b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.352+0000 7f91e5c39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91eaa7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00ed39170 con 0x55a00ed37ae0 2024-03-11T19:46:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91e5c39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed663c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43756/0 (socket says 172.21.15.83:43756) 2024-03-11T19:46:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91e5c39640 1 -- 172.21.15.83:0/806561385 learned_addr learned my addr 172.21.15.83:0/806561385 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91e5438640 1 --2- 172.21.15.83:0/806561385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed66900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91eaa7a740 1 -- 172.21.15.83:0/806561385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00ed376a0 con 0x55a00ed3d640 2024-03-11T19:46:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91eaa7a740 1 -- 172.21.15.83:0/806561385 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00e9d4720 con 0x55a00ed384d0 2024-03-11T19:46:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91e643a640 1 --2- 172.21.15.83:0/806561385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed2b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91e5c39640 1 -- 172.21.15.83:0/806561385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 msgr2=0x55a00ed66900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91e5c39640 1 --2- 172.21.15.83:0/806561385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed66900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91e5c39640 1 -- 172.21.15.83:0/806561385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 msgr2=0x55a00ed2b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91e5c39640 1 --2- 172.21.15.83:0/806561385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed2b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91e5c39640 1 -- 172.21.15.83:0/806561385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a00ec32ac0 con 0x55a00ed37ae0 2024-03-11T19:46:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91e5c39640 1 --2- 172.21.15.83:0/806561385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed663c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0b8b40 tx=0x7f91dc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f048ef12c7902e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91cb7fe640 1 -- 172.21.15.83:0/806561385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0bc1c0 con 0x55a00ed37ae0 2024-03-11T19:46:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91cb7fe640 1 -- 172.21.15.83:0/806561385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91dc0c4960 con 0x55a00ed37ae0 2024-03-11T19:46:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.353+0000 7f91cb7fe640 1 -- 172.21.15.83:0/806561385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0c44d0 con 0x55a00ed37ae0 2024-03-11T19:46:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.354+0000 7f91eaa7a740 1 -- 172.21.15.83:0/806561385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 msgr2=0x55a00ed663c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.354+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/806561385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed663c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0b8b40 tx=0x7f91dc005f90 comp rx=0 tx=0).stop 2024-03-11T19:46:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.354+0000 7f91eaa7a740 1 -- 172.21.15.83:0/806561385 shutdown_connections 2024-03-11T19:46:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.354+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/806561385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00ed2b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.354+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/806561385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed384d0 0x55a00ed66900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.354+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/806561385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed37ae0 0x55a00ed663c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.354+0000 7f91eaa7a740 1 -- 172.21.15.83:0/806561385 >> 172.21.15.83:0/806561385 conn(0x55a00ed33000 msgr2=0x55a00ed5ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.354+0000 7f91eaa7a740 1 -- 172.21.15.83:0/806561385 shutdown_connections 2024-03-11T19:46:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.354+0000 7f91eaa7a740 1 -- 172.21.15.83:0/806561385 wait complete. 2024-03-11T19:46:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.355+0000 7f91eaa7a740 1 -- start start 2024-03-11T19:46:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.355+0000 7f91eaa7a740 1 Processor -- start 2024-03-11T19:46:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.356+0000 7f91eaa7a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed37ae0 0x55a00eda7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.356+0000 7f91eaa7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed384d0 0x55a00eda95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.356+0000 7f91e5c39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed37ae0 0x55a00eda7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.356+0000 7f91e5c39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed37ae0 0x55a00eda7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47140/0 (socket says 172.21.15.83:47140) 2024-03-11T19:46:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.356+0000 7f91e5c39640 1 -- 172.21.15.83:0/869667809 learned_addr learned my addr 172.21.15.83:0/869667809 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.356+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00edabb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.356+0000 7f91eaa7a740 1 -- 172.21.15.83:0/869667809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00ed66c30 con 0x55a00ed384d0 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.356+0000 7f91eaa7a740 1 -- 172.21.15.83:0/869667809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00edac040 con 0x55a00ed3d640 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.357+0000 7f91eaa7a740 1 -- 172.21.15.83:0/869667809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a00edac180 con 0x55a00ed37ae0 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.357+0000 7f91eaa7a740 10 client.0 _ll_register_callbacks cb 0x7f91d0009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.357+0000 7f91e5438640 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed384d0 0x55a00eda95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.357+0000 7f91e643a640 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00edabb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.357+0000 7f91e5438640 1 -- 172.21.15.83:0/869667809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed37ae0 msgr2=0x55a00eda7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.357+0000 7f91e5438640 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed37ae0 0x55a00eda7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.357+0000 7f91e5438640 1 -- 172.21.15.83:0/869667809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 msgr2=0x55a00edabb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.357+0000 7f91e5438640 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00edabb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.357+0000 7f91e5438640 1 -- 172.21.15.83:0/869667809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a00ec56570 con 0x55a00ed384d0 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.357+0000 7f91e5438640 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed384d0 0x55a00eda95b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f91d000f5b0 tx=0x7f91d0005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929b273dc5f073d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.357+0000 7f91c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:46:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.358+0000 7f91c97fa640 1 -- 172.21.15.83:0/869667809 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0017040 con 0x55a00ed384d0 2024-03-11T19:46:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.358+0000 7f91c97fa640 1 -- 172.21.15.83:0/869667809 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f91d0000e40 con 0x55a00ed384d0 2024-03-11T19:46:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.358+0000 7f91eaa7a740 1 -- 172.21.15.83:0/869667809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91dc0c0310 con 0x55a00ed384d0 2024-03-11T19:46:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.358+0000 7f91c97fa640 1 -- 172.21.15.83:0/869667809 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91d000fa20 con 0x55a00ed384d0 2024-03-11T19:46:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.358+0000 7f91eaa7a740 10 client.16623 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.358+0000 7f91eaa7a740 1 -- 172.21.15.83:0/869667809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a00ec586a0 con 0x55a00ed384d0 2024-03-11T19:46:10.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.358+0000 7f91c97fa640 1 -- 172.21.15.83:0/869667809 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91dc0c0310 con 0x55a00ed384d0 2024-03-11T19:46:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91c97fa640 1 -- 172.21.15.83:0/869667809 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f91d002f070 con 0x55a00ed384d0 2024-03-11T19:46:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91eaa7a740 10 client.16623 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91eaa7a740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91eaa7a740 20 client.16623 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a00edadbf0 0x55a00edb0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91eaa7a740 4 client.16623 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91eaa7a740 1 -- 172.21.15.83:0/869667809 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a00ea23890 con 0x55a00edadbf0 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91e5c39640 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a00edadbf0 0x55a00edb0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91e5c39640 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a00edadbf0 0x55a00edb0010 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfbbeeb1d77b8b70 server_cookie=48726ab0fdacbb13 in_seq=0 out_seq=0 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91c97fa640 10 client.16623 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91c97fa640 1 -- 172.21.15.83:0/869667809 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a00ea23890 con 0x55a00edadbf0 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.359+0000 7f91c97fa640 10 client.16623 handle_command_reply: tid=0 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 1 client.16623 shutdown 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 1 -- 172.21.15.83:0/869667809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed384d0 msgr2=0x55a00eda95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed384d0 0x55a00eda95b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f91d000f5b0 tx=0x7f91d0005610 comp rx=0 tx=0).stop 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 1 -- 172.21.15.83:0/869667809 shutdown_connections 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a00edadbf0 0x55a00edb0010 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a00ed3d640 0x55a00edabb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91c97fa640 0 client.16623 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a00ed384d0 0x55a00eda95b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 1 --2- 172.21.15.83:0/869667809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a00ed37ae0 0x55a00eda7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 1 -- 172.21.15.83:0/869667809 >> 172.21.15.83:0/869667809 conn(0x55a00ed33000 msgr2=0x55a00ed5ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 1 -- 172.21.15.83:0/869667809 shutdown_connections 2024-03-11T19:46:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 1 -- 172.21.15.83:0/869667809 wait complete. 2024-03-11T19:46:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.360+0000 7f91eaa7a740 20 client.16623 trim_cache size 0 max 16384 2024-03-11T19:46:10.369 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:46:10.369 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:46:10.369 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:46:10.369 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:46:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 -- 172.21.15.83:0/3848844125 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f91cc0a1a30 msgr2=0x7f91cc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f91cc0a1a30 0x7f91cc0a3ef0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009fd0 tx=0x7f91d8011810 comp rx=0 tx=0).stop 2024-03-11T19:46:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 -- 172.21.15.83:0/3848844125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e00ff9f0 msgr2=0x7f91e0199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e00ff9f0 0x7f91e0199400 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f91dc02c0b0 tx=0x7f91dc0045a0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 -- 172.21.15.83:0/3848844125 shutdown_connections 2024-03-11T19:46:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f91cc0a1a30 0x7f91cc0a3ef0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91e0100cf0 0x7f91e01a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91e0100350 0x7f91e0199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 --2- 172.21.15.83:0/3848844125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91e00ff9f0 0x7f91e0199400 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 -- 172.21.15.83:0/3848844125 >> 172.21.15.83:0/3848844125 conn(0x7f91e00fb5f0 msgr2=0x7f91e00fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 -- 172.21.15.83:0/3848844125 shutdown_connections 2024-03-11T19:46:10.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.361+0000 7f91e7e9d640 1 -- 172.21.15.83:0/3848844125 wait complete. 2024-03-11T19:46:10.378 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:46:10.379 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:46:10.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.535+0000 7f51c5ffb640 1 -- 172.21.15.83:0/2964708069 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0032b40 con 0x7f51c80ffa30 2024-03-11T19:46:10.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.535+0000 7f51cd481640 1 -- 172.21.15.83:0/2964708069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c80ffa30 msgr2=0x7f51c8101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.535+0000 7f51cd481640 1 --2- 172.21.15.83:0/2964708069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c80ffa30 0x7f51c8101e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009ca0 tx=0x7f51c002c120 comp rx=0 tx=0).stop 2024-03-11T19:46:10.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.535+0000 7f51cd481640 1 -- 172.21.15.83:0/2964708069 shutdown_connections 2024-03-11T19:46:10.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.535+0000 7f51cd481640 1 --2- 172.21.15.83:0/2964708069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d50 0x7f51c81051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.535+0000 7f51cd481640 1 --2- 172.21.15.83:0/2964708069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c81023a0 0x7f51c8104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.535+0000 7f51cd481640 1 --2- 172.21.15.83:0/2964708069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c80ffa30 0x7f51c8101e60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.535+0000 7f51cd481640 1 -- 172.21.15.83:0/2964708069 >> 172.21.15.83:0/2964708069 conn(0x7f51c80fb630 msgr2=0x7f51c80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:10.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.538+0000 7f51cd481640 1 -- 172.21.15.83:0/2964708069 shutdown_connections 2024-03-11T19:46:10.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.538+0000 7f51cd481640 1 -- 172.21.15.83:0/2964708069 wait complete. 2024-03-11T19:46:10.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.538+0000 7f51cd481640 1 Processor -- start 2024-03-11T19:46:10.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.539+0000 7f51cd481640 1 -- start start 2024-03-11T19:46:10.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.539+0000 7f51cd481640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c80ffa30 0x7f51c806fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.539+0000 7f51cd481640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c81023a0 0x7f51c806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.539+0000 7f51cd481640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d50 0x7f51c81adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.539+0000 7f51cd481640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c8072f90 con 0x7f51c80ffa30 2024-03-11T19:46:10.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.539+0000 7f51cd481640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c8073100 con 0x7f51c8104d50 2024-03-11T19:46:10.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.539+0000 7f51cd481640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c8106050 con 0x7f51c81023a0 2024-03-11T19:46:10.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.540+0000 7f51c67fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c81023a0 0x7f51c806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.540+0000 7f51c67fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c81023a0 0x7f51c806ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47168/0 (socket says 172.21.15.83:47168) 2024-03-11T19:46:10.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.540+0000 7f51c67fc640 1 -- 172.21.15.83:0/814543260 learned_addr learned my addr 172.21.15.83:0/814543260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:10.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.540+0000 7f51c77fe640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d50 0x7f51c81adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.540+0000 7f51c67fc640 1 -- 172.21.15.83:0/814543260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d50 msgr2=0x7f51c81adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.540+0000 7f51c67fc640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d50 0x7f51c81adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.540+0000 7f51c67fc640 1 -- 172.21.15.83:0/814543260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c80ffa30 msgr2=0x7f51c806fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.540+0000 7f51c67fc640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c80ffa30 0x7f51c806fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.540+0000 7f51c67fc640 1 -- 172.21.15.83:0/814543260 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c0009900 con 0x7f51c81023a0 2024-03-11T19:46:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.541+0000 7f51c67fc640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c81023a0 0x7f51c806ff40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f51b800b4a0 tx=0x7f51b800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.541+0000 7f51affff640 1 -- 172.21.15.83:0/814543260 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8010610 con 0x7f51c81023a0 2024-03-11T19:46:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.542+0000 7f51affff640 1 -- 172.21.15.83:0/814543260 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51b8010c50 con 0x7f51c81023a0 2024-03-11T19:46:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.542+0000 7f51affff640 1 -- 172.21.15.83:0/814543260 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8017e20 con 0x7f51c81023a0 2024-03-11T19:46:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.543+0000 7f51cd481640 1 -- 172.21.15.83:0/814543260 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c8106330 con 0x7f51c81023a0 2024-03-11T19:46:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.543+0000 7f51cd481640 1 -- 172.21.15.83:0/814543260 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c8106870 con 0x7f51c81023a0 2024-03-11T19:46:10.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.545+0000 7f51cd481640 1 -- 172.21.15.83:0/814543260 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c80676e0 con 0x7f51c81023a0 2024-03-11T19:46:10.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.546+0000 7f51affff640 1 -- 172.21.15.83:0/814543260 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f51b8017480 con 0x7f51c81023a0 2024-03-11T19:46:10.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.547+0000 7f51affff640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f51b00a1b80 0x7f51b00a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:10.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.547+0000 7f51affff640 1 -- 172.21.15.83:0/814543260 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f51b80cc9e0 con 0x7f51c81023a0 2024-03-11T19:46:10.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.552+0000 7f51c6ffd640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f51b00a1b80 0x7f51b00a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:10.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.553+0000 7f51c6ffd640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f51b00a1b80 0x7f51b00a4040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f51c0006010 tx=0x7f51c00097e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:10.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.553+0000 7f51affff640 1 -- 172.21.15.83:0/814543260 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f51b8095c00 con 0x7f51c81023a0 2024-03-11T19:46:10.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.863+0000 7f51cd481640 1 -- 172.21.15.83:0/814543260 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51c802cda0 con 0x7f51c81023a0 2024-03-11T19:46:10.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.864+0000 7f51affff640 1 -- 172.21.15.83:0/814543260 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11350 (secure 0 0 0) 0x7f51b80092b0 con 0x7f51c81023a0 2024-03-11T19:46:10.868 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:46:10.868 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:45:19.152624+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:46:10.868 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 31 2024-03-11T19:46:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.866+0000 7f51cd481640 1 -- 172.21.15.83:0/814543260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f51b00a1b80 msgr2=0x7f51b00a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.866+0000 7f51cd481640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f51b00a1b80 0x7f51b00a4040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f51c0006010 tx=0x7f51c00097e0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.866+0000 7f51cd481640 1 -- 172.21.15.83:0/814543260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c81023a0 msgr2=0x7f51c806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.866+0000 7f51cd481640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c81023a0 0x7f51c806ff40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f51b800b4a0 tx=0x7f51b800b960 comp rx=0 tx=0).stop 2024-03-11T19:46:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.866+0000 7f51cd481640 1 -- 172.21.15.83:0/814543260 shutdown_connections 2024-03-11T19:46:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.866+0000 7f51cd481640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f51b00a1b80 0x7f51b00a4040 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.866+0000 7f51cd481640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51c8104d50 0x7f51c81adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.866+0000 7f51cd481640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51c81023a0 0x7f51c806ff40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.866+0000 7f51cd481640 1 --2- 172.21.15.83:0/814543260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51c80ffa30 0x7f51c806fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.866+0000 7f51cd481640 1 -- 172.21.15.83:0/814543260 >> 172.21.15.83:0/814543260 conn(0x7f51c80fb630 msgr2=0x7f51c8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:10.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.867+0000 7f51cd481640 1 -- 172.21.15.83:0/814543260 shutdown_connections 2024-03-11T19:46:10.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:10.867+0000 7f51cd481640 1 -- 172.21.15.83:0/814543260 wait complete. 2024-03-11T19:46:10.885 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:46:10.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:46:11.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.040+0000 7f659755e640 1 -- 172.21.15.83:0/2259966437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6590109020 msgr2=0x7f659010b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.040+0000 7f659755e640 1 --2- 172.21.15.83:0/2259966437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6590109020 0x7f659010b4b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6580009900 tx=0x7f658002bc60 comp rx=0 tx=0).stop 2024-03-11T19:46:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.040+0000 7f659755e640 1 -- 172.21.15.83:0/2259966437 shutdown_connections 2024-03-11T19:46:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.040+0000 7f659755e640 1 --2- 172.21.15.83:0/2259966437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6590109020 0x7f659010b4b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.040+0000 7f659755e640 1 --2- 172.21.15.83:0/2259966437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6590073bc0 0x7f6590074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.040+0000 7f659755e640 1 --2- 172.21.15.83:0/2259966437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65900751a0 0x7f65900735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.040+0000 7f659755e640 1 -- 172.21.15.83:0/2259966437 >> 172.21.15.83:0/2259966437 conn(0x7f65900fc6e0 msgr2=0x7f65900feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.040+0000 7f659755e640 1 -- 172.21.15.83:0/2259966437 shutdown_connections 2024-03-11T19:46:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.040+0000 7f659755e640 1 -- 172.21.15.83:0/2259966437 wait complete. 2024-03-11T19:46:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f659755e640 1 Processor -- start 2024-03-11T19:46:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f659755e640 1 -- start start 2024-03-11T19:46:11.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f659755e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6590073bc0 0x7f6590199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f659755e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65900751a0 0x7f65901999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f659755e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6590109020 0x7f65901a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f659755e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901a0f60 con 0x7f65900751a0 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f659755e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901a10d0 con 0x7f6590073bc0 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f659755e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901a1290 con 0x7f6590109020 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f6594af9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65900751a0 0x7f65901999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f6594af9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65900751a0 0x7f65901999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43792/0 (socket says 172.21.15.83:43792) 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f6594af9640 1 -- 172.21.15.83:0/1052501086 learned_addr learned my addr 172.21.15.83:0/1052501086 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f65952fa640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6590073bc0 0x7f6590199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f6595afb640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6590109020 0x7f65901a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f6594af9640 1 -- 172.21.15.83:0/1052501086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6590109020 msgr2=0x7f65901a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f6594af9640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6590109020 0x7f65901a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f6594af9640 1 -- 172.21.15.83:0/1052501086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6590073bc0 msgr2=0x7f6590199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f6594af9640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6590073bc0 0x7f6590199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.041+0000 7f6594af9640 1 -- 172.21.15.83:0/1052501086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6580009560 con 0x7f65900751a0 2024-03-11T19:46:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.042+0000 7f6594af9640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65900751a0 0x7f65901999b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f658c00d950 tx=0x7f658c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.042+0000 7f65867fc640 1 -- 172.21.15.83:0/1052501086 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c009c30 con 0x7f65900751a0 2024-03-11T19:46:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.043+0000 7f659755e640 1 -- 172.21.15.83:0/1052501086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65901a1570 con 0x7f65900751a0 2024-03-11T19:46:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.043+0000 7f659755e640 1 -- 172.21.15.83:0/1052501086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65901a1b50 con 0x7f65900751a0 2024-03-11T19:46:11.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.043+0000 7f65867fc640 1 -- 172.21.15.83:0/1052501086 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f658c009d90 con 0x7f65900751a0 2024-03-11T19:46:11.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.043+0000 7f65867fc640 1 -- 172.21.15.83:0/1052501086 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c00f650 con 0x7f65900751a0 2024-03-11T19:46:11.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.045+0000 7f65867fc640 1 -- 172.21.15.83:0/1052501086 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f658c00f7b0 con 0x7f65900751a0 2024-03-11T19:46:11.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.045+0000 7f659a20d740 1 Processor -- start 2024-03-11T19:46:11.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f659a20d740 1 -- start start 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f659a20d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1270 0x562de4cf1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f659a20d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1c60 0x562de4cf6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f659a20d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4cf9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f659a20d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4cad960 con 0x562de4cf6dd0 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f659a20d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4c61310 con 0x562de4cf1270 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f659a20d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4c5e840 con 0x562de4cf1c60 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f65867fc640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f657c0a1a30 0x7f657c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f6594af9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4cf9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f6595afb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1270 0x562de4cf1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f65952fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1c60 0x562de4cf6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f6595afb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1270 0x562de4cf1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41750/0 (socket says 172.21.15.83:41750) 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f6594af9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4cf9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43808/0 (socket says 172.21.15.83:43808) 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f6595afb640 1 -- 172.21.15.83:0/1097840035 learned_addr learned my addr 172.21.15.83:0/1097840035 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f6595afb640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f657c0a1a30 0x7f657c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f6595afb640 1 -- 172.21.15.83:0/1097840035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1c60 msgr2=0x562de4cf6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f6595afb640 1 --2- 172.21.15.83:0/1097840035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1c60 0x562de4cf6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f6595afb640 1 -- 172.21.15.83:0/1097840035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 msgr2=0x562de4cf9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f6595afb640 1 --2- 172.21.15.83:0/1097840035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4cf9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.046+0000 7f6595afb640 1 -- 172.21.15.83:0/1097840035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562de4c1c060 con 0x562de4cf1270 2024-03-11T19:46:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.047+0000 7f65867fc640 1 -- 172.21.15.83:0/1052501086 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f658c0ccba0 con 0x7f65900751a0 2024-03-11T19:46:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.047+0000 7f6595afb640 1 --2- 172.21.15.83:0/1097840035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1270 0x562de4cf1690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6580009900 tx=0x7f6580004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98593aafb2d56eb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.047+0000 7f657b7fe640 1 -- 172.21.15.83:0/1097840035 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658002cb60 con 0x562de4cf1270 2024-03-11T19:46:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.047+0000 7f6595afb640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f657c0a1a30 0x7f657c0a3ef0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f658003b000 tx=0x7f6580048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.047+0000 7f657b7fe640 1 -- 172.21.15.83:0/1097840035 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65800044d0 con 0x562de4cf1270 2024-03-11T19:46:11.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.047+0000 7f657b7fe640 1 -- 172.21.15.83:0/1097840035 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580003cd0 con 0x562de4cf1270 2024-03-11T19:46:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.048+0000 7f659a20d740 1 -- 172.21.15.83:0/1097840035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1270 msgr2=0x562de4cf1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.048+0000 7f659a20d740 1 --2- 172.21.15.83:0/1097840035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1270 0x562de4cf1690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6580009900 tx=0x7f6580004870 comp rx=0 tx=0).stop 2024-03-11T19:46:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.048+0000 7f659a20d740 1 -- 172.21.15.83:0/1097840035 shutdown_connections 2024-03-11T19:46:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.048+0000 7f659a20d740 1 --2- 172.21.15.83:0/1097840035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4cf9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.048+0000 7f659a20d740 1 --2- 172.21.15.83:0/1097840035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1c60 0x562de4cf6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.048+0000 7f659a20d740 1 --2- 172.21.15.83:0/1097840035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1270 0x562de4cf1690 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.048+0000 7f659a20d740 1 -- 172.21.15.83:0/1097840035 >> 172.21.15.83:0/1097840035 conn(0x562de4cec7e0 msgr2=0x562de4cecbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.049+0000 7f659a20d740 1 -- 172.21.15.83:0/1097840035 shutdown_connections 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.049+0000 7f659a20d740 1 -- 172.21.15.83:0/1097840035 wait complete. 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.049+0000 7f659a20d740 1 -- start start 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.049+0000 7f659a20d740 1 Processor -- start 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f659a20d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 0x562de4cfb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f659a20d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4cfb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f659a20d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4d02a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f6594af9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4cfb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f659a20d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4c5a020 con 0x562de4cf6dd0 2024-03-11T19:46:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f659a20d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de492d430 con 0x562de4cf1c60 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f659a20d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de48e9e00 con 0x562de4cf1270 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f6594af9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4cfb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41764/0 (socket says 172.21.15.83:41764) 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f6594af9640 1 -- 172.21.15.83:0/1736730873 learned_addr learned my addr 172.21.15.83:0/1736730873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f65952fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 0x562de4cfb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f6595afb640 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4d02a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.050+0000 7f659a20d740 10 client.0 _ll_register_callbacks cb 0x562de4c01cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f6594af9640 1 -- 172.21.15.83:0/1736730873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 msgr2=0x562de4cfb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f6594af9640 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 0x562de4cfb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f6594af9640 1 -- 172.21.15.83:0/1736730873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 msgr2=0x562de4d02a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f6594af9640 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4d02a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f6594af9640 1 -- 172.21.15.83:0/1736730873 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659010c090 con 0x562de4cf1c60 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f6594af9640 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4cfb9f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f658c005fd0 tx=0x7f658c06a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e9c3ff86de86109 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f6578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:46:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f6578ff9640 1 -- 172.21.15.83:0/1736730873 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c09a2c0 con 0x562de4cf1c60 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f6578ff9640 1 -- 172.21.15.83:0/1736730873 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f658c06ae40 con 0x562de4cf1c60 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f659a20d740 1 -- 172.21.15.83:0/1736730873 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562de49597e0 con 0x562de4cf1c60 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f6578ff9640 1 -- 172.21.15.83:0/1736730873 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f658c099bc0 con 0x562de4cf1c60 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f659a20d740 10 client.26102 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f659a20d740 1 -- 172.21.15.83:0/1736730873 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562de4c586a0 con 0x562de4cf1c60 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.051+0000 7f6578ff9640 1 -- 172.21.15.83:0/1736730873 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562de49597e0 con 0x562de4cf1c60 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.052+0000 7f6578ff9640 1 -- 172.21.15.83:0/1736730873 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f658c099010 con 0x562de4cf1c60 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.052+0000 7f659a20d740 10 client.26102 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.052+0000 7f659a20d740 10 client.26102 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.052+0000 7f659a20d740 20 client.26102 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.052+0000 7f659a20d740 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562de4d07690 0x562de4d09ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.052+0000 7f659a20d740 4 client.26102 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:11.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.052+0000 7f659a20d740 1 -- 172.21.15.83:0/1736730873 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562de4996720 con 0x562de4d07690 2024-03-11T19:46:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.052+0000 7f65952fa640 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562de4d07690 0x562de4d09ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.053+0000 7f65952fa640 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562de4d07690 0x562de4d09ad0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2fd10f69cd5266fd server_cookie=bb40dfe653567965 in_seq=0 out_seq=0 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.053+0000 7f6578ff9640 10 client.26102 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.053+0000 7f6578ff9640 1 -- 172.21.15.83:0/1736730873 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562de4996720 con 0x562de4d07690 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.053+0000 7f6578ff9640 10 client.26102 handle_command_reply: tid=0 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 1 client.26102 shutdown 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 1 -- 172.21.15.83:0/1736730873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 msgr2=0x562de4cfb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4cfb9f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f658c005fd0 tx=0x7f658c06a110 comp rx=0 tx=0).stop 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 1 -- 172.21.15.83:0/1736730873 shutdown_connections 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562de4d07690 0x562de4d09ad0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4d02a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4cfb9f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 1 --2- 172.21.15.83:0/1736730873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 0x562de4cfb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f6578ff9640 0 client.26102 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:46:11.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 1 -- 172.21.15.83:0/1736730873 >> 172.21.15.83:0/1736730873 conn(0x562de4cec7e0 msgr2=0x562de4cebb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 1 -- 172.21.15.83:0/1736730873 shutdown_connections 2024-03-11T19:46:11.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 1 -- 172.21.15.83:0/1736730873 wait complete. 2024-03-11T19:46:11.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.054+0000 7f659a20d740 20 client.26102 trim_cache size 0 max 16384 2024-03-11T19:46:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:10 smithi083 ceph-mon[31864]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-03-11T19:46:11.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.070+0000 7f659a20d740 1 Processor -- start 2024-03-11T19:46:11.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.070+0000 7f659a20d740 1 -- start start 2024-03-11T19:46:11.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.070+0000 7f659a20d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 0x562de4cfb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.070+0000 7f659a20d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4d029f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.070+0000 7f659a20d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4cfb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f659a20d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4bdf130 con 0x562de4cf6dd0 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f659a20d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4d05c70 con 0x562de4cf1c60 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f659a20d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4cfb2f0 con 0x562de4cf1270 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f6595afb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4cfb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f6595afb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4cfb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43818/0 (socket says 172.21.15.83:43818) 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f6594af9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4d029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f6595afb640 1 -- 172.21.15.83:0/1272191588 learned_addr learned my addr 172.21.15.83:0/1272191588 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f6595afb640 1 -- 172.21.15.83:0/1272191588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 msgr2=0x562de4cfb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f6595afb640 1 --2- 172.21.15.83:0/1272191588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 0x562de4cfb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f6595afb640 1 -- 172.21.15.83:0/1272191588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 msgr2=0x562de4d029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f6595afb640 1 --2- 172.21.15.83:0/1272191588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4d029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f6595afb640 1 -- 172.21.15.83:0/1272191588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562de4bf4c20 con 0x562de4cf6dd0 2024-03-11T19:46:11.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f6594af9640 1 --2- 172.21.15.83:0/1272191588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4d029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:11.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f6595afb640 1 --2- 172.21.15.83:0/1272191588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4cfb9d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6580036040 tx=0x7f6580005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0136906b4669c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f657affd640 1 -- 172.21.15.83:0/1272191588 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658002c2e0 con 0x562de4cf6dd0 2024-03-11T19:46:11.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f657affd640 1 -- 172.21.15.83:0/1272191588 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6580003cd0 con 0x562de4cf6dd0 2024-03-11T19:46:11.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.071+0000 7f657affd640 1 -- 172.21.15.83:0/1272191588 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65800436b0 con 0x562de4cf6dd0 2024-03-11T19:46:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.072+0000 7f659a20d740 1 -- 172.21.15.83:0/1272191588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 msgr2=0x562de4cfb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.072+0000 7f659a20d740 1 --2- 172.21.15.83:0/1272191588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4cfb9d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6580036040 tx=0x7f6580005a20 comp rx=0 tx=0).stop 2024-03-11T19:46:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.072+0000 7f659a20d740 1 -- 172.21.15.83:0/1272191588 shutdown_connections 2024-03-11T19:46:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.072+0000 7f659a20d740 1 --2- 172.21.15.83:0/1272191588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4cfb9d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.072+0000 7f659a20d740 1 --2- 172.21.15.83:0/1272191588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4d029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.072+0000 7f659a20d740 1 --2- 172.21.15.83:0/1272191588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 0x562de4cfb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.072+0000 7f659a20d740 1 -- 172.21.15.83:0/1272191588 >> 172.21.15.83:0/1272191588 conn(0x562de4cec7e0 msgr2=0x562de4d0e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.072+0000 7f659a20d740 1 -- 172.21.15.83:0/1272191588 shutdown_connections 2024-03-11T19:46:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.073+0000 7f659a20d740 1 -- 172.21.15.83:0/1272191588 wait complete. 2024-03-11T19:46:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.073+0000 7f659a20d740 1 -- start start 2024-03-11T19:46:11.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.073+0000 7f659a20d740 1 Processor -- start 2024-03-11T19:46:11.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.074+0000 7f659a20d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 0x562de4d02540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.074+0000 7f659a20d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4d00b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.074+0000 7f659a20d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4d01090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.074+0000 7f659a20d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4d02d30 con 0x562de4cf6dd0 2024-03-11T19:46:11.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.074+0000 7f659a20d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4d015d0 con 0x562de4cf1c60 2024-03-11T19:46:11.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.074+0000 7f659a20d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562de4d01790 con 0x562de4cf1270 2024-03-11T19:46:11.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.074+0000 7f659a20d740 10 client.0 _ll_register_callbacks cb 0x7f6580009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6595afb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4d01090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6594af9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4d00b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6595afb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4d01090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43820/0 (socket says 172.21.15.83:43820) 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6595afb640 1 -- 172.21.15.83:0/2504130127 learned_addr learned my addr 172.21.15.83:0/2504130127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f65952fa640 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 0x562de4d02540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6594af9640 1 -- 172.21.15.83:0/2504130127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 msgr2=0x562de4d02540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6594af9640 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 0x562de4d02540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6594af9640 1 -- 172.21.15.83:0/2504130127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 msgr2=0x562de4d01090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6594af9640 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4d01090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6594af9640 1 -- 172.21.15.83:0/2504130127 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562de4c586a0 con 0x562de4cf1c60 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6594af9640 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4d00b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f658c096590 tx=0x7f658c010de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b07f423223c8335 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6578ff9640 1 -- 172.21.15.83:0/2504130127 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c0b2050 con 0x562de4cf1c60 2024-03-11T19:46:11.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f6578ff9640 1 -- 172.21.15.83:0/2504130127 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f658c06ae40 con 0x562de4cf1c60 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.075+0000 7f659a20d740 1 -- 172.21.15.83:0/2504130127 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65800044d0 con 0x562de4cf1c60 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f6578ff9640 1 -- 172.21.15.83:0/2504130127 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f658c099700 con 0x562de4cf1c60 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f659a20d740 10 client.26108 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f659a20d740 1 -- 172.21.15.83:0/2504130127 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562de4c1c060 con 0x562de4cf1c60 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f6578ff9640 1 -- 172.21.15.83:0/2504130127 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65800044d0 con 0x562de4cf1c60 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f6578ff9640 1 -- 172.21.15.83:0/2504130127 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f658c0cab70 con 0x562de4cf1c60 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f659a20d740 10 client.26108 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f659a20d740 10 client.26108 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f659a20d740 20 client.26108 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f659a20d740 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562de4d67600 0x562de4d69a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f659a20d740 4 client.26108 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f659a20d740 1 -- 172.21.15.83:0/2504130127 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562de49e5890 con 0x562de4d67600 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.076+0000 7f65952fa640 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562de4d67600 0x562de4d69a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.077+0000 7f6578ff9640 10 client.26108 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.077+0000 7f65952fa640 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562de4d67600 0x562de4d69a20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8251542149ba4be server_cookie=de8e374d1b6c19ae in_seq=0 out_seq=0 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.077+0000 7f6578ff9640 1 -- 172.21.15.83:0/2504130127 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562de49e5890 con 0x562de4d67600 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.077+0000 7f6578ff9640 10 client.26108 handle_command_reply: tid=0 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.077+0000 7f659a20d740 1 client.26108 shutdown 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.077+0000 7f659a20d740 1 -- 172.21.15.83:0/2504130127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 msgr2=0x562de4d00b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.077+0000 7f659a20d740 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4d00b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f658c096590 tx=0x7f658c010de0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.077+0000 7f659a20d740 1 -- 172.21.15.83:0/2504130127 shutdown_connections 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.078+0000 7f659a20d740 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x562de4d67600 0x562de4d69a20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.078+0000 7f659a20d740 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562de4cf6dd0 0x562de4d01090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.078+0000 7f6578ff9640 0 client.26108 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.078+0000 7f659a20d740 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562de4cf1c60 0x562de4d00b50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.078+0000 7f659a20d740 1 --2- 172.21.15.83:0/2504130127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562de4cf1270 0x562de4d02540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.078+0000 7f659a20d740 1 -- 172.21.15.83:0/2504130127 >> 172.21.15.83:0/2504130127 conn(0x562de4cec7e0 msgr2=0x562de4cf31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.078+0000 7f659a20d740 1 -- 172.21.15.83:0/2504130127 shutdown_connections 2024-03-11T19:46:11.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.080+0000 7f659a20d740 1 -- 172.21.15.83:0/2504130127 wait complete. 2024-03-11T19:46:11.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.080+0000 7f659a20d740 20 client.26108 trim_cache size 0 max 16384 2024-03-11T19:46:11.087 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:46:11.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 -- 172.21.15.83:0/1052501086 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f657c0a1a30 msgr2=0x7f657c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f657c0a1a30 0x7f657c0a3ef0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f658003b000 tx=0x7f6580048000 comp rx=0 tx=0).stop 2024-03-11T19:46:11.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 -- 172.21.15.83:0/1052501086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65900751a0 msgr2=0x7f65901999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65900751a0 0x7f65901999b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f658c00d950 tx=0x7f658c00de10 comp rx=0 tx=0).stop 2024-03-11T19:46:11.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 -- 172.21.15.83:0/1052501086 shutdown_connections 2024-03-11T19:46:11.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f657c0a1a30 0x7f657c0a3ef0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6590109020 0x7f65901a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65900751a0 0x7f65901999b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 --2- 172.21.15.83:0/1052501086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6590073bc0 0x7f6590199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 -- 172.21.15.83:0/1052501086 >> 172.21.15.83:0/1052501086 conn(0x7f65900fc6e0 msgr2=0x7f6590104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 -- 172.21.15.83:0/1052501086 shutdown_connections 2024-03-11T19:46:11.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.082+0000 7f659755e640 1 -- 172.21.15.83:0/1052501086 wait complete. 2024-03-11T19:46:11.101 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:46:11.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:46:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:10 smithi137 ceph-mon[36220]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-03-11T19:46:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:10 smithi204 ceph-mon[36739]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-03-11T19:46:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.255+0000 7fa2243e9640 1 -- 172.21.15.83:0/1038731697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c106e60 msgr2=0x7fa21c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.255+0000 7fa2243e9640 1 --2- 172.21.15.83:0/1038731697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c106e60 0x7fa21c1092b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa214009ca0 tx=0x7fa21402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.256+0000 7fa2243e9640 1 -- 172.21.15.83:0/1038731697 shutdown_connections 2024-03-11T19:46:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.256+0000 7fa2243e9640 1 --2- 172.21.15.83:0/1038731697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c106e60 0x7fa21c1092b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.256+0000 7fa2243e9640 1 --2- 172.21.15.83:0/1038731697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa21c104530 0x7fa21c106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.256+0000 7fa2243e9640 1 --2- 172.21.15.83:0/1038731697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa21c101c00 0x7fa21c103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.256+0000 7fa2243e9640 1 -- 172.21.15.83:0/1038731697 >> 172.21.15.83:0/1038731697 conn(0x7fa21c0fb610 msgr2=0x7fa21c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.256+0000 7fa2243e9640 1 -- 172.21.15.83:0/1038731697 shutdown_connections 2024-03-11T19:46:11.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.256+0000 7fa2243e9640 1 -- 172.21.15.83:0/1038731697 wait complete. 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa2243e9640 1 Processor -- start 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa2243e9640 1 -- start start 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa2243e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa21c101c00 0x7fa21c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa2243e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa21c104530 0x7fa21c197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa2243e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c106e60 0x7fa21c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa2243e9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c19eb10 con 0x7fa21c106e60 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa2243e9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c19ec80 con 0x7fa21c104530 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa2243e9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c19ee40 con 0x7fa21c101c00 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa222986640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c106e60 0x7fa21c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa221984640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa21c104530 0x7fa21c197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa222986640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c106e60 0x7fa21c19e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43826/0 (socket says 172.21.15.83:43826) 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa222986640 1 -- 172.21.15.83:0/1638550298 learned_addr learned my addr 172.21.15.83:0/1638550298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.257+0000 7fa222185640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa21c101c00 0x7fa21c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa221984640 1 -- 172.21.15.83:0/1638550298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa21c101c00 msgr2=0x7fa21c197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa221984640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa21c101c00 0x7fa21c197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa221984640 1 -- 172.21.15.83:0/1638550298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c106e60 msgr2=0x7fa21c19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa221984640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c106e60 0x7fa21c19e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa221984640 1 -- 172.21.15.83:0/1638550298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa214009900 con 0x7fa21c104530 2024-03-11T19:46:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa222986640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c106e60 0x7fa21c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa221984640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa21c104530 0x7fa21c197560 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa21c068e40 tx=0x7fa20c009e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa2137fe640 1 -- 172.21.15.83:0/1638550298 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c014070 con 0x7fa21c104530 2024-03-11T19:46:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa2243e9640 1 -- 172.21.15.83:0/1638550298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c0ffa50 con 0x7fa21c104530 2024-03-11T19:46:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa2137fe640 1 -- 172.21.15.83:0/1638550298 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa20c007860 con 0x7fa21c104530 2024-03-11T19:46:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa2137fe640 1 -- 172.21.15.83:0/1638550298 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c005660 con 0x7fa21c104530 2024-03-11T19:46:11.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.258+0000 7fa2243e9640 1 -- 172.21.15.83:0/1638550298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c0fff10 con 0x7fa21c104530 2024-03-11T19:46:11.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.260+0000 7fa2137fe640 1 -- 172.21.15.83:0/1638550298 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa20c018070 con 0x7fa21c104530 2024-03-11T19:46:11.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.261+0000 7fa2137fe640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2080a1a30 0x7fa2080a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.261+0000 7fa226fc3740 1 Processor -- start 2024-03-11T19:46:11.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.261+0000 7fa2137fe640 1 -- 172.21.15.83:0/1638550298 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa20c0d0580 con 0x7fa21c104530 2024-03-11T19:46:11.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.262+0000 7fa226fc3740 1 -- start start 2024-03-11T19:46:11.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.262+0000 7fa222185640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2080a1a30 0x7fa2080a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.262+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559738169a80 0x559738169ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.262+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973816a470 0x55973816ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.262+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816f5e0 0x559738171a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.262+0000 7fa226fc3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973811d960 con 0x55973816f5e0 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.262+0000 7fa226fc3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597380d1310 con 0x559738169a80 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.262+0000 7fa226fc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597380ce840 con 0x55973816a470 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.262+0000 7fa222986640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816f5e0 0x559738171a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.262+0000 7fa222185640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559738169a80 0x559738169ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa221984640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973816a470 0x55973816ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa222185640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559738169a80 0x559738169ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41810/0 (socket says 172.21.15.83:41810) 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa222986640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816f5e0 0x559738171a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43828/0 (socket says 172.21.15.83:43828) 2024-03-11T19:46:11.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa222185640 1 -- 172.21.15.83:0/788371619 learned_addr learned my addr 172.21.15.83:0/788371619 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa222185640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2080a1a30 0x7fa2080a3ef0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa218009c20 tx=0x7fa218011810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa222185640 1 -- 172.21.15.83:0/788371619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973816a470 msgr2=0x55973816ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa222185640 1 --2- 172.21.15.83:0/788371619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973816a470 0x55973816ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa222185640 1 -- 172.21.15.83:0/788371619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816f5e0 msgr2=0x559738171a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa222185640 1 --2- 172.21.15.83:0/788371619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816f5e0 0x559738171a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa222185640 1 -- 172.21.15.83:0/788371619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55973808c060 con 0x559738169a80 2024-03-11T19:46:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa222185640 1 --2- 172.21.15.83:0/788371619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559738169a80 0x559738169ea0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa21801d4f0 tx=0x7fa21801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bd979b1964b83 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa210bfe640 1 -- 172.21.15.83:0/788371619 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21800f680 con 0x559738169a80 2024-03-11T19:46:11.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa210bfe640 1 -- 172.21.15.83:0/788371619 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa21800fcc0 con 0x559738169a80 2024-03-11T19:46:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.263+0000 7fa210bfe640 1 -- 172.21.15.83:0/788371619 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218026e60 con 0x559738169a80 2024-03-11T19:46:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.264+0000 7fa226fc3740 1 -- 172.21.15.83:0/788371619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559738169a80 msgr2=0x559738169ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.264+0000 7fa226fc3740 1 --2- 172.21.15.83:0/788371619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559738169a80 0x559738169ea0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa21801d4f0 tx=0x7fa21801c000 comp rx=0 tx=0).stop 2024-03-11T19:46:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.264+0000 7fa226fc3740 1 -- 172.21.15.83:0/788371619 shutdown_connections 2024-03-11T19:46:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.264+0000 7fa226fc3740 1 --2- 172.21.15.83:0/788371619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816f5e0 0x559738171a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.264+0000 7fa226fc3740 1 --2- 172.21.15.83:0/788371619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973816a470 0x55973816ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.264+0000 7fa226fc3740 1 --2- 172.21.15.83:0/788371619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559738169a80 0x559738169ea0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.264+0000 7fa226fc3740 1 -- 172.21.15.83:0/788371619 >> 172.21.15.83:0/788371619 conn(0x55973815c7e0 msgr2=0x55973815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.264+0000 7fa226fc3740 1 -- 172.21.15.83:0/788371619 shutdown_connections 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.264+0000 7fa226fc3740 1 -- 172.21.15.83:0/788371619 wait complete. 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.264+0000 7fa226fc3740 1 -- start start 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa226fc3740 1 Processor -- start 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559738169a80 0x559738162b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816a470 0x5597381630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x5597381680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa226fc3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597380ca020 con 0x55973816a470 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa226fc3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559737d9d430 con 0x55973816f5e0 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa226fc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559737d59e00 con 0x559738169a80 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa226fc3740 10 client.0 _ll_register_callbacks cb 0x559738071cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa222986640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x5597381680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa222185640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559738169a80 0x559738162b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa222986640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x5597381680d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41822/0 (socket says 172.21.15.83:41822) 2024-03-11T19:46:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa222986640 1 -- 172.21.15.83:0/3495806507 learned_addr learned my addr 172.21.15.83:0/3495806507 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa221984640 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816a470 0x5597381630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa222185640 1 -- 172.21.15.83:0/3495806507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 msgr2=0x5597381680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa222185640 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x5597381680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa222185640 1 -- 172.21.15.83:0/3495806507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816a470 msgr2=0x5597381630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa222185640 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816a470 0x5597381630a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.265+0000 7fa222185640 1 -- 172.21.15.83:0/3495806507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa218009e30 con 0x559738169a80 2024-03-11T19:46:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.266+0000 7fa222185640 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559738169a80 0x559738162b60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa21801d9b0 tx=0x7fa218005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f5b6d61b968e146 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.266+0000 7fa205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:46:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.266+0000 7fa205ffb640 1 -- 172.21.15.83:0/3495806507 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218026650 con 0x559738169a80 2024-03-11T19:46:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.266+0000 7fa205ffb640 1 -- 172.21.15.83:0/3495806507 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa218005040 con 0x559738169a80 2024-03-11T19:46:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.266+0000 7fa226fc3740 1 -- 172.21.15.83:0/3495806507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559737dc97e0 con 0x559738169a80 2024-03-11T19:46:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.266+0000 7fa205ffb640 1 -- 172.21.15.83:0/3495806507 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2180267b0 con 0x559738169a80 2024-03-11T19:46:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.266+0000 7fa226fc3740 10 client.26230 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.266+0000 7fa226fc3740 1 -- 172.21.15.83:0/3495806507 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5597380c86a0 con 0x559738169a80 2024-03-11T19:46:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.266+0000 7fa205ffb640 1 -- 172.21.15.83:0/3495806507 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559737dc97e0 con 0x559738169a80 2024-03-11T19:46:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.267+0000 7fa205ffb640 1 -- 172.21.15.83:0/3495806507 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa2180046c0 con 0x559738169a80 2024-03-11T19:46:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.267+0000 7fa226fc3740 10 client.26230 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.267+0000 7fa226fc3740 10 client.26230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:46:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.267+0000 7fa226fc3740 20 client.26230 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:11.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.267+0000 7fa226fc3740 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559738188540 0x55973818a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.267+0000 7fa226fc3740 4 client.26230 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.267+0000 7fa226fc3740 1 -- 172.21.15.83:0/3495806507 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559737e06720 con 0x559738188540 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.267+0000 7fa221984640 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559738188540 0x55973818a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.269+0000 7fa205ffb640 10 client.26230 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.269+0000 7fa221984640 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559738188540 0x55973818a960 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a16e834b37240775 server_cookie=b684a849a4790a27 in_seq=0 out_seq=0 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.269+0000 7fa205ffb640 1 -- 172.21.15.83:0/3495806507 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559737e06720 con 0x559738188540 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.269+0000 7fa205ffb640 10 client.26230 handle_command_reply: tid=0 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.269+0000 7fa226fc3740 1 client.26230 shutdown 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa226fc3740 1 -- 172.21.15.83:0/3495806507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559738169a80 msgr2=0x559738162b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa226fc3740 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559738169a80 0x559738162b60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa21801d9b0 tx=0x7fa218005db0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa226fc3740 1 -- 172.21.15.83:0/3495806507 shutdown_connections 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa226fc3740 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x559738188540 0x55973818a960 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa226fc3740 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x5597381680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa205ffb640 0 client.26230 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:46:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa226fc3740 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816a470 0x5597381630a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa226fc3740 1 --2- 172.21.15.83:0/3495806507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559738169a80 0x559738162b60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa226fc3740 1 -- 172.21.15.83:0/3495806507 >> 172.21.15.83:0/3495806507 conn(0x55973815c7e0 msgr2=0x55973815bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa226fc3740 1 -- 172.21.15.83:0/3495806507 shutdown_connections 2024-03-11T19:46:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa226fc3740 1 -- 172.21.15.83:0/3495806507 wait complete. 2024-03-11T19:46:11.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.270+0000 7fa226fc3740 20 client.26230 trim_cache size 0 max 16384 2024-03-11T19:46:11.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.285+0000 7fa226fc3740 1 Processor -- start 2024-03-11T19:46:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.285+0000 7fa226fc3740 1 -- start start 2024-03-11T19:46:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.285+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559738169a80 0x559738198820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.285+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816a470 0x559738198d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.285+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x55973816ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.285+0000 7fa226fc3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973804f130 con 0x55973816a470 2024-03-11T19:46:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.285+0000 7fa226fc3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597381585c0 con 0x55973816f5e0 2024-03-11T19:46:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.285+0000 7fa226fc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973818fb90 con 0x559738169a80 2024-03-11T19:46:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.285+0000 7fa222986640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x55973816ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.285+0000 7fa221984640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816a470 0x559738198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.285+0000 7fa222986640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x55973816ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41824/0 (socket says 172.21.15.83:41824) 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.286+0000 7fa222986640 1 -- 172.21.15.83:0/2213642028 learned_addr learned my addr 172.21.15.83:0/2213642028 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.286+0000 7fa222185640 1 --2- 172.21.15.83:0/2213642028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559738169a80 0x559738198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.286+0000 7fa222986640 1 -- 172.21.15.83:0/2213642028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559738169a80 msgr2=0x559738198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.286+0000 7fa222986640 1 --2- 172.21.15.83:0/2213642028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559738169a80 0x559738198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.286+0000 7fa222986640 1 -- 172.21.15.83:0/2213642028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816a470 msgr2=0x559738198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.286+0000 7fa222986640 1 --2- 172.21.15.83:0/2213642028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816a470 0x559738198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.286+0000 7fa222986640 1 -- 172.21.15.83:0/2213642028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559738064c20 con 0x55973816f5e0 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.286+0000 7fa222986640 1 --2- 172.21.15.83:0/2213642028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x55973816ab60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa214032eb0 tx=0x7fa214032ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e9e9bc403c1bbd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.286+0000 7fa207fff640 1 -- 172.21.15.83:0/2213642028 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214032480 con 0x55973816f5e0 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.286+0000 7fa207fff640 1 -- 172.21.15.83:0/2213642028 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa214034ca0 con 0x55973816f5e0 2024-03-11T19:46:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.286+0000 7fa207fff640 1 -- 172.21.15.83:0/2213642028 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140042a0 con 0x55973816f5e0 2024-03-11T19:46:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.287+0000 7fa226fc3740 1 -- 172.21.15.83:0/2213642028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 msgr2=0x55973816ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.287+0000 7fa226fc3740 1 --2- 172.21.15.83:0/2213642028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x55973816ab60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa214032eb0 tx=0x7fa214032ee0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.287+0000 7fa226fc3740 1 -- 172.21.15.83:0/2213642028 shutdown_connections 2024-03-11T19:46:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.287+0000 7fa226fc3740 1 --2- 172.21.15.83:0/2213642028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x55973816ab60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.287+0000 7fa226fc3740 1 --2- 172.21.15.83:0/2213642028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55973816a470 0x559738198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.287+0000 7fa226fc3740 1 --2- 172.21.15.83:0/2213642028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559738169a80 0x559738198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.287+0000 7fa226fc3740 1 -- 172.21.15.83:0/2213642028 >> 172.21.15.83:0/2213642028 conn(0x55973815c7e0 msgr2=0x5597381615c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.288+0000 7fa226fc3740 1 -- 172.21.15.83:0/2213642028 shutdown_connections 2024-03-11T19:46:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.289+0000 7fa226fc3740 1 -- 172.21.15.83:0/2213642028 wait complete. 2024-03-11T19:46:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.289+0000 7fa226fc3740 1 -- start start 2024-03-11T19:46:11.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.289+0000 7fa226fc3740 1 Processor -- start 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.289+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559738169a80 0x5597381d94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.289+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973816a470 0x5597381dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa226fc3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x5597381ddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa226fc3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597381de6c0 con 0x559738169a80 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa226fc3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597381de830 con 0x55973816f5e0 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa226fc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597381de9f0 con 0x55973816a470 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa222185640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559738169a80 0x5597381d94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa226fc3740 10 client.0 _ll_register_callbacks cb 0x7fa218009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa222185640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559738169a80 0x5597381d94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43862/0 (socket says 172.21.15.83:43862) 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa222185640 1 -- 172.21.15.83:0/982102108 learned_addr learned my addr 172.21.15.83:0/982102108 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa222986640 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x5597381ddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa221984640 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973816a470 0x5597381dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa222185640 1 -- 172.21.15.83:0/982102108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973816a470 msgr2=0x5597381dba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa222185640 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973816a470 0x5597381dba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa222185640 1 -- 172.21.15.83:0/982102108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 msgr2=0x5597381ddfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa222185640 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x5597381ddfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa222185640 1 -- 172.21.15.83:0/982102108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c109e90 con 0x559738169a80 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa222986640 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x5597381ddfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa222185640 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559738169a80 0x5597381d94c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa218011d10 tx=0x7fa21800ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390224881350e516 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa205ffb640 1 -- 172.21.15.83:0/982102108 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21800f3c0 con 0x559738169a80 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.290+0000 7fa205ffb640 1 -- 172.21.15.83:0/982102108 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa21800f520 con 0x559738169a80 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa226fc3740 1 -- 172.21.15.83:0/982102108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa21800fcc0 con 0x559738169a80 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa205ffb640 1 -- 172.21.15.83:0/982102108 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa21800f6e0 con 0x559738169a80 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa226fc3740 10 client.16659 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa226fc3740 1 -- 172.21.15.83:0/982102108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55973808a8c0 con 0x559738169a80 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa205ffb640 1 -- 172.21.15.83:0/982102108 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa21800fcc0 con 0x559738169a80 2024-03-11T19:46:11.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa205ffb640 1 -- 172.21.15.83:0/982102108 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa218027030 con 0x559738169a80 2024-03-11T19:46:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa226fc3740 10 client.16659 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa226fc3740 10 client.16659 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:46:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa226fc3740 20 client.16659 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa226fc3740 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5597381e01a0 0x5597381e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa226fc3740 4 client.16659 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:46:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa226fc3740 1 -- 172.21.15.83:0/982102108 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559737e55890 con 0x5597381e01a0 2024-03-11T19:46:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.291+0000 7fa221984640 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5597381e01a0 0x5597381e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa221984640 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5597381e01a0 0x5597381e25c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d8343a0738d704a server_cookie=d6a20c92f77fa0 in_seq=0 out_seq=0 2024-03-11T19:46:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa205ffb640 10 client.16659 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:46:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa205ffb640 1 -- 172.21.15.83:0/982102108 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559737e55890 con 0x5597381e01a0 2024-03-11T19:46:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa205ffb640 10 client.16659 handle_command_reply: tid=0 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa226fc3740 1 client.16659 shutdown 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa226fc3740 1 -- 172.21.15.83:0/982102108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559738169a80 msgr2=0x5597381d94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa226fc3740 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559738169a80 0x5597381d94c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa218011d10 tx=0x7fa21800ff20 comp rx=0 tx=0).stop 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa226fc3740 1 -- 172.21.15.83:0/982102108 shutdown_connections 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa226fc3740 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5597381e01a0 0x5597381e25c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa226fc3740 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55973816f5e0 0x5597381ddfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa226fc3740 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55973816a470 0x5597381dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa205ffb640 0 client.16659 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa226fc3740 1 --2- 172.21.15.83:0/982102108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559738169a80 0x5597381d94c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.292+0000 7fa226fc3740 1 -- 172.21.15.83:0/982102108 >> 172.21.15.83:0/982102108 conn(0x55973815c7e0 msgr2=0x5597381615c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa226fc3740 1 -- 172.21.15.83:0/982102108 shutdown_connections 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa226fc3740 1 -- 172.21.15.83:0/982102108 wait complete. 2024-03-11T19:46:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa226fc3740 20 client.16659 trim_cache size 0 max 16384 2024-03-11T19:46:11.302 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:46:11.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa2243e9640 1 -- 172.21.15.83:0/1638550298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2080a1a30 msgr2=0x7fa2080a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa2243e9640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2080a1a30 0x7fa2080a3ef0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa218009c20 tx=0x7fa218011810 comp rx=0 tx=0).stop 2024-03-11T19:46:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa2243e9640 1 -- 172.21.15.83:0/1638550298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa21c104530 msgr2=0x7fa21c197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa2243e9640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa21c104530 0x7fa21c197560 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa21c068e40 tx=0x7fa20c009e40 comp rx=0 tx=0).stop 2024-03-11T19:46:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa2243e9640 1 -- 172.21.15.83:0/1638550298 shutdown_connections 2024-03-11T19:46:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa2243e9640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2080a1a30 0x7fa2080a3ef0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa2243e9640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c106e60 0x7fa21c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa2243e9640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa21c104530 0x7fa21c197560 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa2243e9640 1 --2- 172.21.15.83:0/1638550298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa21c101c00 0x7fa21c197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa2243e9640 1 -- 172.21.15.83:0/1638550298 >> 172.21.15.83:0/1638550298 conn(0x7fa21c0fb610 msgr2=0x7fa21c104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.293+0000 7fa2243e9640 1 -- 172.21.15.83:0/1638550298 shutdown_connections 2024-03-11T19:46:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.294+0000 7fa2243e9640 1 -- 172.21.15.83:0/1638550298 wait complete. 2024-03-11T19:46:11.310 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:46:11.310 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:46:11.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.459+0000 7f580de57640 1 -- 172.21.15.83:0/3704560264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5808100080 msgr2=0x7f58081004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.459+0000 7f580de57640 1 --2- 172.21.15.83:0/3704560264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5808100080 0x7f58081004a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009900 tx=0x7f57f802bc30 comp rx=0 tx=0).stop 2024-03-11T19:46:11.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.459+0000 7f580de57640 1 -- 172.21.15.83:0/3704560264 shutdown_connections 2024-03-11T19:46:11.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.459+0000 7f580de57640 1 --2- 172.21.15.83:0/3704560264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5808101c30 0x7f58081084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.459+0000 7f580de57640 1 --2- 172.21.15.83:0/3704560264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5808101270 0x7f58081016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.459+0000 7f580de57640 1 --2- 172.21.15.83:0/3704560264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5808100080 0x7f58081004a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.459+0000 7f580de57640 1 -- 172.21.15.83:0/3704560264 >> 172.21.15.83:0/3704560264 conn(0x7f58080fb610 msgr2=0x7f58080fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.459+0000 7f580de57640 1 -- 172.21.15.83:0/3704560264 shutdown_connections 2024-03-11T19:46:11.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.460+0000 7f580de57640 1 -- 172.21.15.83:0/3704560264 wait complete. 2024-03-11T19:46:11.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.460+0000 7f580de57640 1 Processor -- start 2024-03-11T19:46:11.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f580de57640 1 -- start start 2024-03-11T19:46:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f580de57640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5808100080 0x7f58081a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f580de57640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5808101270 0x7f58081a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f580de57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5808101c30 0x7f58081a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f580de57640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58081a53c0 con 0x7f5808101c30 2024-03-11T19:46:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f580de57640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58081a5530 con 0x7f5808100080 2024-03-11T19:46:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f580de57640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58081a56f0 con 0x7f5808101270 2024-03-11T19:46:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f58077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5808100080 0x7f58081a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f58077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5808100080 0x7f58081a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41856/0 (socket says 172.21.15.83:41856) 2024-03-11T19:46:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f58077fe640 1 -- 172.21.15.83:0/611967212 learned_addr learned my addr 172.21.15.83:0/611967212 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f5807fff640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5808101c30 0x7f58081a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f5806ffd640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5808101270 0x7f58081a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f58077fe640 1 -- 172.21.15.83:0/611967212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5808101270 msgr2=0x7f58081a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f58077fe640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5808101270 0x7f58081a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f58077fe640 1 -- 172.21.15.83:0/611967212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5808101c30 msgr2=0x7f58081a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f58077fe640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5808101c30 0x7f58081a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f58077fe640 1 -- 172.21.15.83:0/611967212 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f8009560 con 0x7f5808100080 2024-03-11T19:46:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.461+0000 7f58077fe640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5808100080 0x7f58081a0110 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f57f8000c00 tx=0x7f57f8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.462+0000 7f5804ff9640 1 -- 172.21.15.83:0/611967212 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f80043d0 con 0x7f5808100080 2024-03-11T19:46:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.462+0000 7f5804ff9640 1 -- 172.21.15.83:0/611967212 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57f80052d0 con 0x7f5808100080 2024-03-11T19:46:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.462+0000 7f580de57640 1 -- 172.21.15.83:0/611967212 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58081a5970 con 0x7f5808100080 2024-03-11T19:46:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.462+0000 7f5804ff9640 1 -- 172.21.15.83:0/611967212 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8003ce0 con 0x7f5808100080 2024-03-11T19:46:11.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.462+0000 7f580de57640 1 -- 172.21.15.83:0/611967212 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58081a5e80 con 0x7f5808100080 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.465+0000 7f5804ff9640 1 -- 172.21.15.83:0/611967212 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f57f8004df0 con 0x7f5808100080 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.465+0000 7f5810a1f740 1 Processor -- start 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5810a1f740 1 -- start start 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5804ff9640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57f40a1ab0 0x7f57f40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff04d270 0x5556ff04d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff052760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff052dd0 0x5556ff055260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5810a1f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ff009960 con 0x5556ff052dd0 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5810a1f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fefbd310 con 0x5556ff04dc60 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5810a1f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fefba840 con 0x5556ff04d270 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5804ff9640 1 -- 172.21.15.83:0/611967212 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f57f80efb40 con 0x7f5808100080 2024-03-11T19:46:11.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5807fff640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57f40a1ab0 0x7f57f40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5807fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff052dd0 0x5556ff055260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5807fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff052dd0 0x5556ff055260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43898/0 (socket says 172.21.15.83:43898) 2024-03-11T19:46:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.466+0000 7f5807fff640 1 -- 172.21.15.83:0/4086847242 learned_addr learned my addr 172.21.15.83:0/4086847242 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.467+0000 7f58077fe640 1 --2- 172.21.15.83:0/4086847242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff04d270 0x5556ff04d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.467+0000 7f5806ffd640 1 --2- 172.21.15.83:0/4086847242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff052760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.467+0000 7f5807fff640 1 -- 172.21.15.83:0/4086847242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff04d270 msgr2=0x5556ff04d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.467+0000 7f5807fff640 1 --2- 172.21.15.83:0/4086847242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff04d270 0x5556ff04d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.467+0000 7f5807fff640 1 -- 172.21.15.83:0/4086847242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 msgr2=0x5556ff052760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.467+0000 7f5807fff640 1 --2- 172.21.15.83:0/4086847242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff052760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.467+0000 7f5807fff640 1 -- 172.21.15.83:0/4086847242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556fef78060 con 0x5556ff052dd0 2024-03-11T19:46:11.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.467+0000 7f58077fe640 1 --2- 172.21.15.83:0/4086847242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff04d270 0x5556ff04d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:11.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.467+0000 7f5806ffd640 1 --2- 172.21.15.83:0/4086847242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff052760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:46:11.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.467+0000 7f5807fff640 1 --2- 172.21.15.83:0/4086847242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff052dd0 0x5556ff055260 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f58000118b0 tx=0x7f5800011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8c78d998733d66 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.468+0000 7f57fdffb640 1 -- 172.21.15.83:0/4086847242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580000fac0 con 0x5556ff052dd0 2024-03-11T19:46:11.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.468+0000 7f57fdffb640 1 -- 172.21.15.83:0/4086847242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f580000fc20 con 0x5556ff052dd0 2024-03-11T19:46:11.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.468+0000 7f57fdffb640 1 -- 172.21.15.83:0/4086847242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580000e7c0 con 0x5556ff052dd0 2024-03-11T19:46:11.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5810a1f740 1 -- 172.21.15.83:0/4086847242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff052dd0 msgr2=0x5556ff055260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5810a1f740 1 --2- 172.21.15.83:0/4086847242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff052dd0 0x5556ff055260 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f58000118b0 tx=0x7f5800011d70 comp rx=0 tx=0).stop 2024-03-11T19:46:11.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5807fff640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57f40a1ab0 0x7f57f40a3f70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5800021000 tx=0x7f580000ee30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5810a1f740 1 -- 172.21.15.83:0/4086847242 shutdown_connections 2024-03-11T19:46:11.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5810a1f740 1 --2- 172.21.15.83:0/4086847242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff052dd0 0x5556ff055260 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5810a1f740 1 --2- 172.21.15.83:0/4086847242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff052760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5810a1f740 1 --2- 172.21.15.83:0/4086847242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff04d270 0x5556ff04d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5810a1f740 1 -- 172.21.15.83:0/4086847242 >> 172.21.15.83:0/4086847242 conn(0x5556ff0487e0 msgr2=0x5556ff048be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5810a1f740 1 -- 172.21.15.83:0/4086847242 shutdown_connections 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5810a1f740 1 -- 172.21.15.83:0/4086847242 wait complete. 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5810a1f740 1 -- start start 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.469+0000 7f5810a1f740 1 Processor -- start 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff04d270 0x5556ff057420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04dc60 0x5556ff057960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff052dd0 0x5556ff05e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5810a1f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fefb6020 con 0x5556ff04dc60 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5810a1f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fec89430 con 0x5556ff052dd0 2024-03-11T19:46:11.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5810a1f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fec45e00 con 0x5556ff04d270 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5810a1f740 10 client.0 _ll_register_callbacks cb 0x5556fef5dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5806ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04dc60 0x5556ff057960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5806ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04dc60 0x5556ff057960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43906/0 (socket says 172.21.15.83:43906) 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5806ffd640 1 -- 172.21.15.83:0/4263707216 learned_addr learned my addr 172.21.15.83:0/4263707216 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5807fff640 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff052dd0 0x5556ff05e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f58077fe640 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff04d270 0x5556ff057420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5807fff640 1 -- 172.21.15.83:0/4263707216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff04d270 msgr2=0x5556ff057420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5807fff640 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff04d270 0x5556ff057420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5807fff640 1 -- 172.21.15.83:0/4263707216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04dc60 msgr2=0x5556ff057960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5807fff640 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04dc60 0x5556ff057960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5807fff640 1 -- 172.21.15.83:0/4263707216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5800019040 con 0x5556ff052dd0 2024-03-11T19:46:11.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.470+0000 7f5806ffd640 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04dc60 0x5556ff057960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f5807fff640 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff052dd0 0x5556ff05e9a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5800011db0 tx=0x7f5800009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad4d3e269f3ed36 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f57e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:46:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f57e6ffd640 1 -- 172.21.15.83:0/4263707216 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580000fac0 con 0x5556ff052dd0 2024-03-11T19:46:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f5810a1f740 1 -- 172.21.15.83:0/4263707216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556fecb57e0 con 0x5556ff052dd0 2024-03-11T19:46:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f57e6ffd640 1 -- 172.21.15.83:0/4263707216 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5800004ab0 con 0x5556ff052dd0 2024-03-11T19:46:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f57e6ffd640 1 -- 172.21.15.83:0/4263707216 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5800004c70 con 0x5556ff052dd0 2024-03-11T19:46:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f5810a1f740 10 client.26138 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f5810a1f740 1 -- 172.21.15.83:0/4263707216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5556fefb46a0 con 0x5556ff052dd0 2024-03-11T19:46:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f57e6ffd640 1 -- 172.21.15.83:0/4263707216 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5556fecb57e0 con 0x5556ff052dd0 2024-03-11T19:46:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f57e6ffd640 1 -- 172.21.15.83:0/4263707216 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f580001cd70 con 0x5556ff052dd0 2024-03-11T19:46:11.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f5810a1f740 10 client.26138 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f5810a1f740 10 client.26138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f5810a1f740 20 client.26138 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f5810a1f740 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5556ff063590 0x5556ff0659b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f5810a1f740 4 client.26138 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.471+0000 7f5810a1f740 1 -- 172.21.15.83:0/4263707216 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556fecf2720 con 0x5556ff063590 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.472+0000 7f58077fe640 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5556ff063590 0x5556ff0659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.472+0000 7f57e6ffd640 10 client.26138 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.472+0000 7f58077fe640 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5556ff063590 0x5556ff0659b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=615cae4811fae4ed server_cookie=255f7b29a9a08021 in_seq=0 out_seq=0 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f57e6ffd640 1 -- 172.21.15.83:0/4263707216 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5556fecf2720 con 0x5556ff063590 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f57e6ffd640 10 client.26138 handle_command_reply: tid=0 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 1 client.26138 shutdown 2024-03-11T19:46:11.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 1 -- 172.21.15.83:0/4263707216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff052dd0 msgr2=0x5556ff05e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff052dd0 0x5556ff05e9a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5800011db0 tx=0x7f5800009500 comp rx=0 tx=0).stop 2024-03-11T19:46:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 1 -- 172.21.15.83:0/4263707216 shutdown_connections 2024-03-11T19:46:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5556ff063590 0x5556ff0659b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff052dd0 0x5556ff05e9a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04dc60 0x5556ff057960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 1 --2- 172.21.15.83:0/4263707216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff04d270 0x5556ff057420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 1 -- 172.21.15.83:0/4263707216 >> 172.21.15.83:0/4263707216 conn(0x5556ff0487e0 msgr2=0x5556ff047b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f57e6ffd640 0 client.26138 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 1 -- 172.21.15.83:0/4263707216 shutdown_connections 2024-03-11T19:46:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 1 -- 172.21.15.83:0/4263707216 wait complete. 2024-03-11T19:46:11.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.473+0000 7f5810a1f740 20 client.26138 trim_cache size 0 max 16384 2024-03-11T19:46:11.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f5810a1f740 1 Processor -- start 2024-03-11T19:46:11.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f5810a1f740 1 -- start start 2024-03-11T19:46:11.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 0x5556ff06a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff05e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff052dd0 0x5556ff04f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f5810a1f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556fef3b130 con 0x5556ff04d270 2024-03-11T19:46:11.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f5810a1f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ff043a10 con 0x5556ff04dc60 2024-03-11T19:46:11.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f5810a1f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ff0445c0 con 0x5556ff052dd0 2024-03-11T19:46:11.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f58077fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 0x5556ff06a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f58077fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 0x5556ff06a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43914/0 (socket says 172.21.15.83:43914) 2024-03-11T19:46:11.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f58077fe640 1 -- 172.21.15.83:0/766411789 learned_addr learned my addr 172.21.15.83:0/766411789 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f5806ffd640 1 --2- 172.21.15.83:0/766411789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff05e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f5807fff640 1 --2- 172.21.15.83:0/766411789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff052dd0 0x5556ff04f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f58077fe640 1 -- 172.21.15.83:0/766411789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff052dd0 msgr2=0x5556ff04f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f58077fe640 1 --2- 172.21.15.83:0/766411789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff052dd0 0x5556ff04f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f58077fe640 1 -- 172.21.15.83:0/766411789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 msgr2=0x5556ff05e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f58077fe640 1 --2- 172.21.15.83:0/766411789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff05e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f58077fe640 1 -- 172.21.15.83:0/766411789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556fef50c20 con 0x5556ff04d270 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.490+0000 7f5806ffd640 1 --2- 172.21.15.83:0/766411789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff05e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.491+0000 7f58077fe640 1 --2- 172.21.15.83:0/766411789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 0x5556ff06a330 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005ab0 tx=0x7f57f80ed520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de418163c428842 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.491+0000 7f57fd7fa640 1 -- 172.21.15.83:0/766411789 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f80c0c10 con 0x5556ff04d270 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.491+0000 7f57fd7fa640 1 -- 172.21.15.83:0/766411789 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57f80c5260 con 0x5556ff04d270 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.491+0000 7f57fd7fa640 1 -- 172.21.15.83:0/766411789 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f80b9130 con 0x5556ff04d270 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.491+0000 7f5810a1f740 1 -- 172.21.15.83:0/766411789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 msgr2=0x5556ff06a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.491+0000 7f5810a1f740 1 --2- 172.21.15.83:0/766411789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 0x5556ff06a330 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005ab0 tx=0x7f57f80ed520 comp rx=0 tx=0).stop 2024-03-11T19:46:11.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.492+0000 7f5810a1f740 1 -- 172.21.15.83:0/766411789 shutdown_connections 2024-03-11T19:46:11.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.492+0000 7f5810a1f740 1 --2- 172.21.15.83:0/766411789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff052dd0 0x5556ff04f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.492+0000 7f5810a1f740 1 --2- 172.21.15.83:0/766411789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff05e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.492+0000 7f5810a1f740 1 --2- 172.21.15.83:0/766411789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 0x5556ff06a330 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.492+0000 7f5810a1f740 1 -- 172.21.15.83:0/766411789 >> 172.21.15.83:0/766411789 conn(0x5556ff0487e0 msgr2=0x5556ff056a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.492+0000 7f5810a1f740 1 -- 172.21.15.83:0/766411789 shutdown_connections 2024-03-11T19:46:11.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.492+0000 7f5810a1f740 1 -- 172.21.15.83:0/766411789 wait complete. 2024-03-11T19:46:11.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.492+0000 7f5810a1f740 1 -- start start 2024-03-11T19:46:11.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.492+0000 7f5810a1f740 1 Processor -- start 2024-03-11T19:46:11.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.493+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 0x5556ff05c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.493+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff05ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.493+0000 7f5810a1f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff052dd0 0x5556ff05de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.493+0000 7f5806ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff05ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.493+0000 7f5806ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff05ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41896/0 (socket says 172.21.15.83:41896) 2024-03-11T19:46:11.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.493+0000 7f5806ffd640 1 -- 172.21.15.83:0/3161481560 learned_addr learned my addr 172.21.15.83:0/3161481560 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:11.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.493+0000 7f58077fe640 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 0x5556ff05c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.493+0000 7f5807fff640 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff052dd0 0x5556ff05de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.493+0000 7f5810a1f740 1 -- 172.21.15.83:0/3161481560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ff05e350 con 0x5556ff04d270 2024-03-11T19:46:11.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.494+0000 7f5810a1f740 1 -- 172.21.15.83:0/3161481560 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ff05e4c0 con 0x5556ff04dc60 2024-03-11T19:46:11.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.494+0000 7f5810a1f740 1 -- 172.21.15.83:0/3161481560 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ff05e680 con 0x5556ff052dd0 2024-03-11T19:46:11.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.494+0000 7f5810a1f740 10 client.0 _ll_register_callbacks cb 0x7f5800009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:11.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.494+0000 7f5806ffd640 1 -- 172.21.15.83:0/3161481560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff052dd0 msgr2=0x5556ff05de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.494+0000 7f5806ffd640 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff052dd0 0x5556ff05de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.494+0000 7f5806ffd640 1 -- 172.21.15.83:0/3161481560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 msgr2=0x5556ff05c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:11.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.494+0000 7f5806ffd640 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 0x5556ff05c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.494+0000 7f5806ffd640 1 -- 172.21.15.83:0/3161481560 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f0009ca0 con 0x5556ff04dc60 2024-03-11T19:46:11.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.494+0000 7f5806ffd640 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff05ce00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f57f0012c80 tx=0x7f57f000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b890005c89d23abe server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:11.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.494+0000 7f57e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:46:11.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f57e6ffd640 1 -- 172.21.15.83:0/3161481560 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0017040 con 0x5556ff04dc60 2024-03-11T19:46:11.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f57e6ffd640 1 -- 172.21.15.83:0/3161481560 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f57f0004230 con 0x5556ff04dc60 2024-03-11T19:46:11.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f5810a1f740 1 -- 172.21.15.83:0/3161481560 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f580000fc20 con 0x5556ff04dc60 2024-03-11T19:46:11.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f57e6ffd640 1 -- 172.21.15.83:0/3161481560 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57f00043f0 con 0x5556ff04dc60 2024-03-11T19:46:11.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f5810a1f740 10 client.26144 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:11.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f5810a1f740 1 -- 172.21.15.83:0/3161481560 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f58081a5970 con 0x5556ff04dc60 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f57e6ffd640 1 -- 172.21.15.83:0/3161481560 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f580000fc20 con 0x5556ff04dc60 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f57e6ffd640 1 -- 172.21.15.83:0/3161481560 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f57f0010690 con 0x5556ff04dc60 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f5810a1f740 10 client.26144 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f5810a1f740 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f5810a1f740 20 client.26144 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f5810a1f740 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5556ff0badf0 0x5556ff0bd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f5810a1f740 4 client.26144 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.495+0000 7f5810a1f740 1 -- 172.21.15.83:0/3161481560 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5556fed41890 con 0x5556ff0badf0 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f58077fe640 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5556ff0badf0 0x5556ff0bd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f57e6ffd640 10 client.26144 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f58077fe640 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5556ff0badf0 0x5556ff0bd210 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=517e5d6538530862 server_cookie=ac93aa9ff301f076 in_seq=0 out_seq=0 2024-03-11T19:46:11.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f57e6ffd640 1 -- 172.21.15.83:0/3161481560 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5556fed41890 con 0x5556ff0badf0 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f57e6ffd640 10 client.26144 handle_command_reply: tid=0 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f5810a1f740 1 client.26144 shutdown 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f5810a1f740 1 -- 172.21.15.83:0/3161481560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 msgr2=0x5556ff05ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f5810a1f740 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff05ce00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f57f0012c80 tx=0x7f57f000f790 comp rx=0 tx=0).stop 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f5810a1f740 1 -- 172.21.15.83:0/3161481560 shutdown_connections 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f5810a1f740 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5556ff0badf0 0x5556ff0bd210 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f5810a1f740 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556ff052dd0 0x5556ff05de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f5810a1f740 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556ff04dc60 0x5556ff05ce00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f5810a1f740 1 --2- 172.21.15.83:0/3161481560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ff04d270 0x5556ff05c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.496+0000 7f57e6ffd640 0 client.26144 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f5810a1f740 1 -- 172.21.15.83:0/3161481560 >> 172.21.15.83:0/3161481560 conn(0x5556ff0487e0 msgr2=0x5556ff04f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f5810a1f740 1 -- 172.21.15.83:0/3161481560 shutdown_connections 2024-03-11T19:46:11.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f5810a1f740 1 -- 172.21.15.83:0/3161481560 wait complete. 2024-03-11T19:46:11.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f5810a1f740 20 client.26144 trim_cache size 0 max 16384 2024-03-11T19:46:11.508 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:46:11.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f580de57640 1 -- 172.21.15.83:0/611967212 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57f40a1ab0 msgr2=0x7f57f40a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f580de57640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57f40a1ab0 0x7f57f40a3f70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5800021000 tx=0x7f580000ee30 comp rx=0 tx=0).stop 2024-03-11T19:46:11.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f580de57640 1 -- 172.21.15.83:0/611967212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5808100080 msgr2=0x7f58081a0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:11.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f580de57640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5808100080 0x7f58081a0110 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f57f8000c00 tx=0x7f57f8005e80 comp rx=0 tx=0).stop 2024-03-11T19:46:11.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f580de57640 1 -- 172.21.15.83:0/611967212 shutdown_connections 2024-03-11T19:46:11.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f580de57640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f57f40a1ab0 0x7f57f40a3f70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f580de57640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5808101c30 0x7f58081a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f580de57640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5808101270 0x7f58081a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f580de57640 1 --2- 172.21.15.83:0/611967212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5808100080 0x7f58081a0110 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:11.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.497+0000 7f580de57640 1 -- 172.21.15.83:0/611967212 >> 172.21.15.83:0/611967212 conn(0x7f58080fb610 msgr2=0x7f58080fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:11.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.498+0000 7f580de57640 1 -- 172.21.15.83:0/611967212 shutdown_connections 2024-03-11T19:46:11.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:11.498+0000 7f580de57640 1 -- 172.21.15.83:0/611967212 wait complete. 2024-03-11T19:46:12.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:11 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/814543260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:46:12.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:11 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/814543260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:46:12.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:11 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/814543260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:46:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 415.0 s, 2271.0 tps, lat 43.791 ms stddev 49.586 2024-03-11T19:46:12.517 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:46:12.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:46:12.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:46:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.664+0000 7f7569ee1640 1 -- 172.21.15.83:0/2786205594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564102d40 msgr2=0x7f75641095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.664+0000 7f7569ee1640 1 --2- 172.21.15.83:0/2786205594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564102d40 0x7f75641095d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f755c009ca0 tx=0x7f755c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.666+0000 7f7569ee1640 1 -- 172.21.15.83:0/2786205594 shutdown_connections 2024-03-11T19:46:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.666+0000 7f7569ee1640 1 --2- 172.21.15.83:0/2786205594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564102d40 0x7f75641095d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.666+0000 7f7569ee1640 1 --2- 172.21.15.83:0/2786205594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7564102380 0x7f7564102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.666+0000 7f7569ee1640 1 --2- 172.21.15.83:0/2786205594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7564101190 0x7f75641015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.666+0000 7f7569ee1640 1 -- 172.21.15.83:0/2786205594 >> 172.21.15.83:0/2786205594 conn(0x7f75640fc6e0 msgr2=0x7f75640feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.666+0000 7f7569ee1640 1 -- 172.21.15.83:0/2786205594 shutdown_connections 2024-03-11T19:46:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.666+0000 7f7569ee1640 1 -- 172.21.15.83:0/2786205594 wait complete. 2024-03-11T19:46:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.667+0000 7f7569ee1640 1 Processor -- start 2024-03-11T19:46:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.667+0000 7f7569ee1640 1 -- start start 2024-03-11T19:46:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.667+0000 7f7569ee1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7564101190 0x7f7564199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.667+0000 7f7569ee1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564102380 0x7f75641998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.667+0000 7f7569ee1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7564102d40 0x7f75641a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.667+0000 7f7569ee1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75641a0e60 con 0x7f7564102380 2024-03-11T19:46:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.667+0000 7f7569ee1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75641a0fd0 con 0x7f7564102d40 2024-03-11T19:46:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7569ee1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75641a1190 con 0x7f7564101190 2024-03-11T19:46:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564102380 0x7f75641998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564102380 0x7f75641998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43942/0 (socket says 172.21.15.83:43942) 2024-03-11T19:46:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7562ffd640 1 -- 172.21.15.83:0/47979089 learned_addr learned my addr 172.21.15.83:0/47979089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f75637fe640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7564101190 0x7f7564199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7563fff640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7564102d40 0x7f75641a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7562ffd640 1 -- 172.21.15.83:0/47979089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7564101190 msgr2=0x7f7564199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7562ffd640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7564101190 0x7f7564199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7562ffd640 1 -- 172.21.15.83:0/47979089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7564102d40 msgr2=0x7f75641a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7562ffd640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7564102d40 0x7f75641a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7562ffd640 1 -- 172.21.15.83:0/47979089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c009900 con 0x7f7564102380 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7562ffd640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564102380 0x7f75641998b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f755800c920 tx=0x7f755800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7560ff9640 1 -- 172.21.15.83:0/47979089 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558007c70 con 0x7f7564102380 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7569ee1640 1 -- 172.21.15.83:0/47979089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75641a1470 con 0x7f7564102380 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7569ee1640 1 -- 172.21.15.83:0/47979089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75641a19d0 con 0x7f7564102380 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.668+0000 7f7560ff9640 1 -- 172.21.15.83:0/47979089 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7558007dd0 con 0x7f7564102380 2024-03-11T19:46:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.669+0000 7f7560ff9640 1 -- 172.21.15.83:0/47979089 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558018860 con 0x7f7564102380 2024-03-11T19:46:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.671+0000 7f7560ff9640 1 -- 172.21.15.83:0/47979089 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f75580189c0 con 0x7f7564102380 2024-03-11T19:46:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f756cb8a740 1 Processor -- start 2024-03-11T19:46:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f756cb8a740 1 -- start start 2024-03-11T19:46:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 0x5640b3fb70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 0x5640b3fb8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb8860 0x5640b3fbf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f756cb8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b3f1df90 con 0x5640b3fb8860 2024-03-11T19:46:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f756cb8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b3f1b4c0 con 0x5640b3fb7ea0 2024-03-11T19:46:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f756cb8a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b3f16ca0 con 0x5640b3fb6cd0 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f75637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 0x5640b3fb8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f75637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 0x5640b3fb8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41926/0 (socket says 172.21.15.83:41926) 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f7563fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 0x5640b3fb70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f75637fe640 1 -- 172.21.15.83:0/1406382352 learned_addr learned my addr 172.21.15.83:0/1406382352 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f7562ffd640 1 --2- 172.21.15.83:0/1406382352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb8860 0x5640b3fbf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.672+0000 7f75637fe640 1 -- 172.21.15.83:0/1406382352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 msgr2=0x5640b3fb70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f75637fe640 1 --2- 172.21.15.83:0/1406382352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 0x5640b3fb70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f75637fe640 1 -- 172.21.15.83:0/1406382352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb8860 msgr2=0x5640b3fbf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f75637fe640 1 --2- 172.21.15.83:0/1406382352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb8860 0x5640b3fbf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f75637fe640 1 -- 172.21.15.83:0/1406382352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640b3ed90b0 con 0x5640b3fb7ea0 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f7563fff640 1 --2- 172.21.15.83:0/1406382352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 0x5640b3fb70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f7560ff9640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f754c0a1b30 0x7f754c0a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f75637fe640 1 --2- 172.21.15.83:0/1406382352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 0x5640b3fb8320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f755400bcb0 tx=0x7f75540107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f21dcd233c9a4140 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f7563fff640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f754c0a1b30 0x7f754c0a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f7560ff9640 1 -- 172.21.15.83:0/47979089 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f75580cd7f0 con 0x7f7564102380 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f7549ffb640 1 -- 172.21.15.83:0/1406382352 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554004020 con 0x5640b3fb7ea0 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f7549ffb640 1 -- 172.21.15.83:0/1406382352 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7554015070 con 0x5640b3fb7ea0 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.673+0000 7f7549ffb640 1 -- 172.21.15.83:0/1406382352 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554004c10 con 0x5640b3fb7ea0 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.674+0000 7f756cb8a740 1 -- 172.21.15.83:0/1406382352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 msgr2=0x5640b3fb8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.674+0000 7f7563fff640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f754c0a1b30 0x7f754c0a3ff0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f755c005e30 tx=0x7f755c0326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.674+0000 7f756cb8a740 1 --2- 172.21.15.83:0/1406382352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 0x5640b3fb8320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f755400bcb0 tx=0x7f75540107a0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.674+0000 7f756cb8a740 1 -- 172.21.15.83:0/1406382352 shutdown_connections 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.674+0000 7f756cb8a740 1 --2- 172.21.15.83:0/1406382352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb8860 0x5640b3fbf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.674+0000 7f756cb8a740 1 --2- 172.21.15.83:0/1406382352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 0x5640b3fb8320 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.674+0000 7f756cb8a740 1 --2- 172.21.15.83:0/1406382352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 0x5640b3fb70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.674+0000 7f756cb8a740 1 -- 172.21.15.83:0/1406382352 >> 172.21.15.83:0/1406382352 conn(0x5640b3fb1eb0 msgr2=0x5640b3fb22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.674+0000 7f756cb8a740 1 -- 172.21.15.83:0/1406382352 shutdown_connections 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.674+0000 7f756cb8a740 1 -- 172.21.15.83:0/1406382352 wait complete. 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.674+0000 7f756cb8a740 1 -- start start 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f756cb8a740 1 Processor -- start 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 0x5640b3fc0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb8860 0x5640b3fc83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f756cb8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b3ebbf40 con 0x5640b3fb7ea0 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f756cb8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b3bea4b0 con 0x5640b3fb6cd0 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f756cb8a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b3e9c670 con 0x5640b3fb8860 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f756cb8a740 10 client.0 _ll_register_callbacks cb 0x5640b3f09f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43954/0 (socket says 172.21.15.83:43954) 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f7562ffd640 1 -- 172.21.15.83:0/198849620 learned_addr learned my addr 172.21.15.83:0/198849620 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f75637fe640 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 0x5640b3fc0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f7563fff640 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb8860 0x5640b3fc83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f7562ffd640 1 -- 172.21.15.83:0/198849620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb8860 msgr2=0x5640b3fc83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f7562ffd640 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb8860 0x5640b3fc83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f7562ffd640 1 -- 172.21.15.83:0/198849620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 msgr2=0x5640b3fc0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f7562ffd640 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 0x5640b3fc0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f7562ffd640 1 -- 172.21.15.83:0/198849620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755400b8e0 con 0x5640b3fb7ea0 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f75637fe640 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 0x5640b3fc0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f7562ffd640 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc13a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f755809a700 tx=0x7f755800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25514b0662e46b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.675+0000 7f7542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f7542ffd640 1 -- 172.21.15.83:0/198849620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755809a2b0 con 0x5640b3fb7ea0 2024-03-11T19:46:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f7542ffd640 1 -- 172.21.15.83:0/198849620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f755809ae70 con 0x5640b3fb7ea0 2024-03-11T19:46:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f756cb8a740 1 -- 172.21.15.83:0/198849620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640b3ef0f40 con 0x5640b3fb7ea0 2024-03-11T19:46:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f7542ffd640 1 -- 172.21.15.83:0/198849620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f755809a0d0 con 0x5640b3fb7ea0 2024-03-11T19:46:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f756cb8a740 10 client.16689 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f756cb8a740 1 -- 172.21.15.83:0/198849620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5640b3f15320 con 0x5640b3fb7ea0 2024-03-11T19:46:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f7542ffd640 1 -- 172.21.15.83:0/198849620 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f755809a770 con 0x5640b3fb7ea0 2024-03-11T19:46:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f7542ffd640 1 -- 172.21.15.83:0/198849620 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7558016020 con 0x5640b3fb7ea0 2024-03-11T19:46:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f756cb8a740 10 client.16689 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f756cb8a740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f756cb8a740 20 client.16689 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:12.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f756cb8a740 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5640b3fcd090 0x5640b3fcf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f756cb8a740 4 client.16689 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:12.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.676+0000 7f756cb8a740 1 -- 172.21.15.83:0/198849620 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640b3ef0f40 con 0x5640b3fcd090 2024-03-11T19:46:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.677+0000 7f75637fe640 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5640b3fcd090 0x5640b3fcf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.678+0000 7f7542ffd640 10 client.16689 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.678+0000 7f75637fe640 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5640b3fcd090 0x5640b3fcf530 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805b3b2ac9a828fd server_cookie=fd7439213cd87e4d in_seq=0 out_seq=0 2024-03-11T19:46:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f7542ffd640 1 -- 172.21.15.83:0/198849620 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5640b3ef0f40 con 0x5640b3fcd090 2024-03-11T19:46:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f7542ffd640 10 client.16689 handle_command_reply: tid=0 2024-03-11T19:46:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f756cb8a740 1 client.16689 shutdown 2024-03-11T19:46:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f756cb8a740 1 -- 172.21.15.83:0/198849620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 msgr2=0x5640b3fc13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f756cb8a740 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc13a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f755809a700 tx=0x7f755800a560 comp rx=0 tx=0).stop 2024-03-11T19:46:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f756cb8a740 1 -- 172.21.15.83:0/198849620 shutdown_connections 2024-03-11T19:46:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f756cb8a740 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5640b3fcd090 0x5640b3fcf530 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f756cb8a740 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb8860 0x5640b3fc83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f756cb8a740 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc13a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f756cb8a740 1 --2- 172.21.15.83:0/198849620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 0x5640b3fc0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f756cb8a740 1 -- 172.21.15.83:0/198849620 >> 172.21.15.83:0/198849620 conn(0x5640b3fb1eb0 msgr2=0x5640b3fa88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.679+0000 7f7542ffd640 0 client.16689 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.680+0000 7f756cb8a740 1 -- 172.21.15.83:0/198849620 shutdown_connections 2024-03-11T19:46:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.680+0000 7f756cb8a740 1 -- 172.21.15.83:0/198849620 wait complete. 2024-03-11T19:46:12.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.680+0000 7f756cb8a740 20 client.16689 trim_cache size 0 max 16384 2024-03-11T19:46:12.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.700+0000 7f756cb8a740 1 Processor -- start 2024-03-11T19:46:12.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.700+0000 7f756cb8a740 1 -- start start 2024-03-11T19:46:12.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 0x5640b3fbb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb8860 0x5640b3fbb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f756cb8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b3f9e350 con 0x5640b3fb7ea0 2024-03-11T19:46:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f756cb8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b3fb86e0 con 0x5640b3fb6cd0 2024-03-11T19:46:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f756cb8a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b3fb7920 con 0x5640b3fb8860 2024-03-11T19:46:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f75637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 0x5640b3fbb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f7562ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43968/0 (socket says 172.21.15.83:43968) 2024-03-11T19:46:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f7563fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb8860 0x5640b3fbb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f7562ffd640 1 -- 172.21.15.83:0/881398833 learned_addr learned my addr 172.21.15.83:0/881398833 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f75637fe640 1 -- 172.21.15.83:0/881398833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb8860 msgr2=0x5640b3fbb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f75637fe640 1 --2- 172.21.15.83:0/881398833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb8860 0x5640b3fbb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f75637fe640 1 -- 172.21.15.83:0/881398833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 msgr2=0x5640b3fc83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f75637fe640 1 --2- 172.21.15.83:0/881398833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f75637fe640 1 -- 172.21.15.83:0/881398833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640b3eb1c00 con 0x5640b3fb6cd0 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.701+0000 7f7562ffd640 1 --2- 172.21.15.83:0/881398833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.702+0000 7f75637fe640 1 --2- 172.21.15.83:0/881398833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 0x5640b3fbb0d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f755400bc80 tx=0x7f75540046a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cae24e9df3a8f445 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.702+0000 7f75497fa640 1 -- 172.21.15.83:0/881398833 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554022040 con 0x5640b3fb6cd0 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.702+0000 7f75497fa640 1 -- 172.21.15.83:0/881398833 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5640b3ef0f40 con 0x5640b3fb6cd0 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.702+0000 7f75497fa640 1 -- 172.21.15.83:0/881398833 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7554005670 con 0x5640b3fb6cd0 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 -- 172.21.15.83:0/881398833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 msgr2=0x5640b3fbb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 --2- 172.21.15.83:0/881398833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 0x5640b3fbb0d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f755400bc80 tx=0x7f75540046a0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 -- 172.21.15.83:0/881398833 shutdown_connections 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 --2- 172.21.15.83:0/881398833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb8860 0x5640b3fbb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 --2- 172.21.15.83:0/881398833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 --2- 172.21.15.83:0/881398833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb6cd0 0x5640b3fbb0d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 -- 172.21.15.83:0/881398833 >> 172.21.15.83:0/881398833 conn(0x5640b3fb1eb0 msgr2=0x5640b3fc0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 -- 172.21.15.83:0/881398833 shutdown_connections 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 -- 172.21.15.83:0/881398833 wait complete. 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 -- start start 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 Processor -- start 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.703+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 0x5640b3fc7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f756cb8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb8860 0x5640b4023640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f756cb8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b3fc7d60 con 0x5640b3fb8860 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f756cb8a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b3fc7ed0 con 0x5640b3fb7ea0 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f756cb8a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b4023bd0 con 0x5640b3fb6cd0 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f75637fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 0x5640b3fc7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f75637fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 0x5640b3fc7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47410/0 (socket says 172.21.15.83:47410) 2024-03-11T19:46:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f75637fe640 1 -- 172.21.15.83:0/3406659040 learned_addr learned my addr 172.21.15.83:0/3406659040 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f756cb8a740 10 client.0 _ll_register_callbacks cb 0x7f755400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f7562ffd640 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f7562ffd640 1 -- 172.21.15.83:0/3406659040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 msgr2=0x5640b3fc7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f7562ffd640 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 0x5640b3fc7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f7562ffd640 1 -- 172.21.15.83:0/3406659040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb8860 msgr2=0x5640b4023640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f7562ffd640 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb8860 0x5640b4023640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f7562ffd640 1 -- 172.21.15.83:0/3406659040 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640b3f15320 con 0x5640b3fb7ea0 2024-03-11T19:46:12.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f75637fe640 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 0x5640b3fc7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f7562ffd640 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc7570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f755800c8f0 tx=0x7f75580b3320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e648fa67693f16c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.704+0000 7f7542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.705+0000 7f7542ffd640 1 -- 172.21.15.83:0/3406659040 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755809a2b0 con 0x5640b3fb7ea0 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.705+0000 7f7542ffd640 1 -- 172.21.15.83:0/3406659040 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f755809ae70 con 0x5640b3fb7ea0 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.705+0000 7f756cb8a740 1 -- 172.21.15.83:0/3406659040 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640b3f3c900 con 0x5640b3fb7ea0 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.705+0000 7f7542ffd640 1 -- 172.21.15.83:0/3406659040 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f755809a0d0 con 0x5640b3fb7ea0 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.705+0000 7f756cb8a740 10 client.26159 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.705+0000 7f756cb8a740 1 -- 172.21.15.83:0/3406659040 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5640b3ed90b0 con 0x5640b3fb7ea0 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.705+0000 7f7542ffd640 1 -- 172.21.15.83:0/3406659040 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75580b8080 con 0x5640b3fb7ea0 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.706+0000 7f7542ffd640 1 -- 172.21.15.83:0/3406659040 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7558016040 con 0x5640b3fb7ea0 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.706+0000 7f756cb8a740 10 client.26159 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.706+0000 7f756cb8a740 10 client.26159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.706+0000 7f756cb8a740 20 client.26159 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:12.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.706+0000 7f756cb8a740 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5640b40252a0 0x5640b40276c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.706+0000 7f756cb8a740 4 client.26159 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:46:12.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.706+0000 7f756cb8a740 1 -- 172.21.15.83:0/3406659040 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5640b3f0d310 con 0x5640b40252a0 2024-03-11T19:46:12.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.706+0000 7f75637fe640 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5640b40252a0 0x5640b40276c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.707+0000 7f7542ffd640 10 client.26159 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:12.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.707+0000 7f75637fe640 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5640b40252a0 0x5640b40276c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50029beb768bed3e server_cookie=284e835853e8ac14 in_seq=0 out_seq=0 2024-03-11T19:46:12.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.707+0000 7f7542ffd640 1 -- 172.21.15.83:0/3406659040 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5640b3f0d310 con 0x5640b40252a0 2024-03-11T19:46:12.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.707+0000 7f7542ffd640 10 client.26159 handle_command_reply: tid=0 2024-03-11T19:46:12.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.707+0000 7f756cb8a740 1 client.26159 shutdown 2024-03-11T19:46:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f756cb8a740 1 -- 172.21.15.83:0/3406659040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 msgr2=0x5640b3fc7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f756cb8a740 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc7570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f755800c8f0 tx=0x7f75580b3320 comp rx=0 tx=0).stop 2024-03-11T19:46:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f756cb8a740 1 -- 172.21.15.83:0/3406659040 shutdown_connections 2024-03-11T19:46:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f756cb8a740 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5640b40252a0 0x5640b40276c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f7542ffd640 0 client.26159 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f756cb8a740 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b3fb8860 0x5640b4023640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f756cb8a740 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b3fb7ea0 0x5640b3fc7570 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f756cb8a740 1 --2- 172.21.15.83:0/3406659040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b3fb6cd0 0x5640b3fc7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f756cb8a740 1 -- 172.21.15.83:0/3406659040 >> 172.21.15.83:0/3406659040 conn(0x5640b3fb1eb0 msgr2=0x5640b3fbb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f756cb8a740 1 -- 172.21.15.83:0/3406659040 shutdown_connections 2024-03-11T19:46:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f756cb8a740 1 -- 172.21.15.83:0/3406659040 wait complete. 2024-03-11T19:46:12.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.708+0000 7f756cb8a740 20 client.26159 trim_cache size 0 max 16384 2024-03-11T19:46:12.718 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:46:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:46:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "5a4b217f-b939-41f4-add3-e594ca254617", 2024-03-11T19:46:12.719 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:46:12.719 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:46:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 -- 172.21.15.83:0/47979089 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f754c0a1b30 msgr2=0x7f754c0a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f754c0a1b30 0x7f754c0a3ff0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f755c005e30 tx=0x7f755c0326b0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 -- 172.21.15.83:0/47979089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564102380 msgr2=0x7f75641998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564102380 0x7f75641998b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f755800c920 tx=0x7f755800cde0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 -- 172.21.15.83:0/47979089 shutdown_connections 2024-03-11T19:46:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f754c0a1b30 0x7f754c0a3ff0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7564102d40 0x7f75641a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564102380 0x7f75641998b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 --2- 172.21.15.83:0/47979089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7564101190 0x7f7564199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 -- 172.21.15.83:0/47979089 >> 172.21.15.83:0/47979089 conn(0x7f75640fc6e0 msgr2=0x7f75640733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 -- 172.21.15.83:0/47979089 shutdown_connections 2024-03-11T19:46:12.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.709+0000 7f7569ee1640 1 -- 172.21.15.83:0/47979089 wait complete. 2024-03-11T19:46:12.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:46:12.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:46:12.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:12 smithi083 ceph-mon[31864]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 MiB/s wr, 695 op/s 2024-03-11T19:46:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.893+0000 7fefc281c640 1 -- 172.21.15.83:0/1686791548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc10d830 msgr2=0x7fefbc10fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.893+0000 7fefc281c640 1 --2- 172.21.15.83:0/1686791548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc10d830 0x7fefbc10fcf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009be0 tx=0x7fefb402bef0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.893+0000 7fefc281c640 1 -- 172.21.15.83:0/1686791548 shutdown_connections 2024-03-11T19:46:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.893+0000 7fefc281c640 1 --2- 172.21.15.83:0/1686791548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc10d830 0x7fefbc10fcf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.893+0000 7fefc281c640 1 --2- 172.21.15.83:0/1686791548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefbc10adb0 0x7fefbc10d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.893+0000 7fefc281c640 1 --2- 172.21.15.83:0/1686791548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefbc1083f0 0x7fefbc10a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.893+0000 7fefc281c640 1 -- 172.21.15.83:0/1686791548 >> 172.21.15.83:0/1686791548 conn(0x7fefbc06e360 msgr2=0x7fefbc0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.893+0000 7fefc281c640 1 -- 172.21.15.83:0/1686791548 shutdown_connections 2024-03-11T19:46:12.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.893+0000 7fefc281c640 1 -- 172.21.15.83:0/1686791548 wait complete. 2024-03-11T19:46:12.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.894+0000 7fefc281c640 1 Processor -- start 2024-03-11T19:46:12.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.894+0000 7fefc281c640 1 -- start start 2024-03-11T19:46:12.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.894+0000 7fefc281c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefbc1083f0 0x7fefbc07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.894+0000 7fefc281c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefbc10adb0 0x7fefbc07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.894+0000 7fefc281c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc10d830 0x7fefbc082770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.894+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefbc1083f0 0x7fefbc07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.894+0000 7fefc0db9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc10d830 0x7fefbc082770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefc0db9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc10d830 0x7fefbc082770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43992/0 (socket says 172.21.15.83:43992) 2024-03-11T19:46:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefbb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefbc10adb0 0x7fefbc07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefbc1083f0 0x7fefbc07b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47424/0 (socket says 172.21.15.83:47424) 2024-03-11T19:46:12.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefbb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefbc10adb0 0x7fefbc07b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41972/0 (socket says 172.21.15.83:41972) 2024-03-11T19:46:12.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefc0db9640 1 -- 172.21.15.83:0/3845585538 learned_addr learned my addr 172.21.15.83:0/3845585538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:12.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefc281c640 1 -- 172.21.15.83:0/3845585538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc082cb0 con 0x7fefbc10d830 2024-03-11T19:46:12.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefc281c640 1 -- 172.21.15.83:0/3845585538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc082e20 con 0x7fefbc10adb0 2024-03-11T19:46:12.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefc281c640 1 -- 172.21.15.83:0/3845585538 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc082fe0 con 0x7fefbc1083f0 2024-03-11T19:46:12.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefc0db9640 1 -- 172.21.15.83:0/3845585538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefbc1083f0 msgr2=0x7fefbc07b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefc0db9640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefbc1083f0 0x7fefbc07b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefc0db9640 1 -- 172.21.15.83:0/3845585538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefbc10adb0 msgr2=0x7fefbc07b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefc0db9640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefbc10adb0 0x7fefbc07b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.895+0000 7fefc0db9640 1 -- 172.21.15.83:0/3845585538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4009840 con 0x7fefbc10d830 2024-03-11T19:46:12.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.896+0000 7fefc0db9640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc10d830 0x7fefbc082770 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fefb4033420 tx=0x7fefb4033cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.896+0000 7fefb97fa640 1 -- 172.21.15.83:0/3845585538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb4003b80 con 0x7fefbc10d830 2024-03-11T19:46:12.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.897+0000 7fefb97fa640 1 -- 172.21.15.83:0/3845585538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefb4003ce0 con 0x7fefbc10d830 2024-03-11T19:46:12.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.897+0000 7fefb97fa640 1 -- 172.21.15.83:0/3845585538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40348c0 con 0x7fefbc10d830 2024-03-11T19:46:12.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.897+0000 7fefc281c640 1 -- 172.21.15.83:0/3845585538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc083260 con 0x7fefbc10d830 2024-03-11T19:46:12.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.897+0000 7fefc281c640 1 -- 172.21.15.83:0/3845585538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc083600 con 0x7fefbc10d830 2024-03-11T19:46:12.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.900+0000 7fefb97fa640 1 -- 172.21.15.83:0/3845585538 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fefb4034a20 con 0x7fefbc10d830 2024-03-11T19:46:12.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.901+0000 7fefc53f4740 1 Processor -- start 2024-03-11T19:46:12.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.901+0000 7fefc53f4740 1 -- start start 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.901+0000 7fefb97fa640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefa40a1ab0 0x7fefa40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.901+0000 7fefb97fa640 1 -- 172.21.15.83:0/3845585538 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fefb40efb30 con 0x7fefbc10d830 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.902+0000 7fefbb7fe640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefa40a1ab0 0x7fefa40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.902+0000 7fefc53f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.902+0000 7fefc53f4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a5cc0 0x55aaa31aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.902+0000 7fefc53f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.902+0000 7fefc53f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa3115280 con 0x55aaa31aae30 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.902+0000 7fefc53f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa31127b0 con 0x55aaa31a52d0 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.902+0000 7fefc53f4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa310df90 con 0x55aaa31a5cc0 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.902+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.902+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31a56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41978/0 (socket says 172.21.15.83:41978) 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.902+0000 7fefbbfff640 1 -- 172.21.15.83:0/1163270740 learned_addr learned my addr 172.21.15.83:0/1163270740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.903+0000 7fefc0db9640 1 --2- 172.21.15.83:0/1163270740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.903+0000 7fefbbfff640 1 -- 172.21.15.83:0/1163270740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a5cc0 msgr2=0x55aaa31aa7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.903+0000 7fefbbfff640 1 --2- 172.21.15.83:0/1163270740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a5cc0 0x55aaa31aa7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.903+0000 7fefbbfff640 1 -- 172.21.15.83:0/1163270740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 msgr2=0x55aaa31ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.903+0000 7fefbbfff640 1 --2- 172.21.15.83:0/1163270740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.903+0000 7fefbbfff640 1 -- 172.21.15.83:0/1163270740 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaa30cc570 con 0x55aaa31a52d0 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.903+0000 7fefbbfff640 1 --2- 172.21.15.83:0/1163270740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31a56f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fefac00ad70 tx=0x7fefac00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aed1700534dfb0f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.903+0000 7fefa27fc640 1 -- 172.21.15.83:0/1163270740 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac010840 con 0x55aaa31a52d0 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.903+0000 7fefa27fc640 1 -- 172.21.15.83:0/1163270740 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefac010e80 con 0x55aaa31a52d0 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.904+0000 7fefa27fc640 1 -- 172.21.15.83:0/1163270740 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac016340 con 0x55aaa31a52d0 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.904+0000 7fefc53f4740 1 -- 172.21.15.83:0/1163270740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 msgr2=0x55aaa31a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.904+0000 7fefc53f4740 1 --2- 172.21.15.83:0/1163270740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31a56f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fefac00ad70 tx=0x7fefac00f790 comp rx=0 tx=0).stop 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.904+0000 7fefc53f4740 1 -- 172.21.15.83:0/1163270740 shutdown_connections 2024-03-11T19:46:12.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.904+0000 7fefc53f4740 1 --2- 172.21.15.83:0/1163270740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.904+0000 7fefc53f4740 1 --2- 172.21.15.83:0/1163270740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a5cc0 0x55aaa31aa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.904+0000 7fefc53f4740 1 --2- 172.21.15.83:0/1163270740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31a56f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.904+0000 7fefc53f4740 1 -- 172.21.15.83:0/1163270740 >> 172.21.15.83:0/1163270740 conn(0x55aaa31a07f0 msgr2=0x55aaa31a0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.905+0000 7fefbb7fe640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefa40a1ab0 0x7fefa40a3f70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55aaa31a60c0 tx=0x7fefb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.905+0000 7fefc53f4740 1 -- 172.21.15.83:0/1163270740 shutdown_connections 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.905+0000 7fefc53f4740 1 -- 172.21.15.83:0/1163270740 wait complete. 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.905+0000 7fefc53f4740 1 -- start start 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefc53f4740 1 Processor -- start 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefc53f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefc53f4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a5cc0 0x55aaa31af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefc53f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefc53f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa2de1430 con 0x55aaa31aae30 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefc53f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa2d9de00 con 0x55aaa31a52d0 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefc53f4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa3092fd0 con 0x55aaa31a5cc0 2024-03-11T19:46:12.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefc53f4740 10 client.0 _ll_register_callbacks cb 0x55aaa30e8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefc0db9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:41990/0 (socket says 172.21.15.83:41990) 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefbbfff640 1 -- 172.21.15.83:0/4265109089 learned_addr learned my addr 172.21.15.83:0/4265109089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefbb7fe640 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a5cc0 0x55aaa31af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefbbfff640 1 -- 172.21.15.83:0/4265109089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a5cc0 msgr2=0x55aaa31af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefbbfff640 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a5cc0 0x55aaa31af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefbbfff640 1 -- 172.21.15.83:0/4265109089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 msgr2=0x55aaa31b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.906+0000 7fefbbfff640 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.907+0000 7fefbbfff640 1 -- 172.21.15.83:0/4265109089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefac00a9d0 con 0x55aaa31a52d0 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.907+0000 7fefc0db9640 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.907+0000 7fefbbfff640 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31af270 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fefac000c00 tx=0x7fefac018670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef845e21f87263a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.907+0000 7fef9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.907+0000 7fef9b7fe640 1 -- 172.21.15.83:0/4265109089 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac004440 con 0x55aaa31a52d0 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.907+0000 7fefc53f4740 1 -- 172.21.15.83:0/4265109089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aaa2e4a720 con 0x55aaa31a52d0 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.907+0000 7fef9b7fe640 1 -- 172.21.15.83:0/4265109089 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fefac010640 con 0x55aaa31a52d0 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.907+0000 7fef9b7fe640 1 -- 172.21.15.83:0/4265109089 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefac010800 con 0x55aaa31a52d0 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.907+0000 7fefc53f4740 10 client.26168 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.907+0000 7fefc53f4740 1 -- 172.21.15.83:0/4265109089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55aaa310c610 con 0x55aaa31a52d0 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.907+0000 7fef9b7fe640 1 -- 172.21.15.83:0/4265109089 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aaa2e4a720 con 0x55aaa31a52d0 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.908+0000 7fef9b7fe640 1 -- 172.21.15.83:0/4265109089 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fefac022440 con 0x55aaa31a52d0 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.908+0000 7fefc53f4740 10 client.26168 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.908+0000 7fefc53f4740 10 client.26168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.908+0000 7fefc53f4740 20 client.26168 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.908+0000 7fefc53f4740 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55aaa31bb220 0x55aaa31bd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.908+0000 7fefc53f4740 4 client.26168 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:12.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.908+0000 7fefc53f4740 1 -- 172.21.15.83:0/4265109089 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aaa2e99890 con 0x55aaa31bb220 2024-03-11T19:46:12.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.908+0000 7fefbb7fe640 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55aaa31bb220 0x55aaa31bd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.908+0000 7fef9b7fe640 10 client.26168 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:12.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.908+0000 7fefbb7fe640 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55aaa31bb220 0x55aaa31bd640 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7df851867e8e4b server_cookie=8de0db9d13632f75 in_seq=0 out_seq=0 2024-03-11T19:46:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.910+0000 7fef9b7fe640 1 -- 172.21.15.83:0/4265109089 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aaa2e99890 con 0x55aaa31bb220 2024-03-11T19:46:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.910+0000 7fef9b7fe640 10 client.26168 handle_command_reply: tid=0 2024-03-11T19:46:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.910+0000 7fefc53f4740 1 client.26168 shutdown 2024-03-11T19:46:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fefc53f4740 1 -- 172.21.15.83:0/4265109089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 msgr2=0x55aaa31af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fefc53f4740 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31af270 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fefac000c00 tx=0x7fefac018670 comp rx=0 tx=0).stop 2024-03-11T19:46:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fefc53f4740 1 -- 172.21.15.83:0/4265109089 shutdown_connections 2024-03-11T19:46:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fefc53f4740 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55aaa31bb220 0x55aaa31bd640 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fefc53f4740 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fefc53f4740 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a5cc0 0x55aaa31af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fefc53f4740 1 --2- 172.21.15.83:0/4265109089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a52d0 0x55aaa31af270 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fefac000c00 tx=0x7fefac018670 comp rx=0 tx=0).stop 2024-03-11T19:46:12.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fef9b7fe640 0 client.26168 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:12.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fefc53f4740 1 -- 172.21.15.83:0/4265109089 >> 172.21.15.83:0/4265109089 conn(0x55aaa31a07f0 msgr2=0x55aaa319fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fefc53f4740 1 -- 172.21.15.83:0/4265109089 shutdown_connections 2024-03-11T19:46:12.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fefc53f4740 1 -- 172.21.15.83:0/4265109089 wait complete. 2024-03-11T19:46:12.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.911+0000 7fefc53f4740 20 client.26168 trim_cache size 0 max 16384 2024-03-11T19:46:12.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.939+0000 7fefc53f4740 1 Processor -- start 2024-03-11T19:46:12.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.939+0000 7fefc53f4740 1 -- start start 2024-03-11T19:46:12.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.940+0000 7fefc53f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31a5cc0 0x55aaa31ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.940+0000 7fefc53f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31aae30 0x55aaa31b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.940+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31a5cc0 0x55aaa31ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.940+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31a5cc0 0x55aaa31ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44012/0 (socket says 172.21.15.83:44012) 2024-03-11T19:46:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.940+0000 7fefbbfff640 1 -- 172.21.15.83:0/646659926 learned_addr learned my addr 172.21.15.83:0/646659926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.940+0000 7fefc53f4740 1 --2- 172.21.15.83:0/646659926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a7570 0x55aaa31aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.940+0000 7fefbb7fe640 1 --2- 172.21.15.83:0/646659926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31aae30 0x55aaa31b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.940+0000 7fefc53f4740 1 -- 172.21.15.83:0/646659926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa31a6120 con 0x55aaa31a5cc0 2024-03-11T19:46:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.940+0000 7fefc53f4740 1 -- 172.21.15.83:0/646659926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa319c2e0 con 0x55aaa31aae30 2024-03-11T19:46:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.940+0000 7fefc53f4740 1 -- 172.21.15.83:0/646659926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa31b7d30 con 0x55aaa31a7570 2024-03-11T19:46:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.941+0000 7fefc0db9640 1 --2- 172.21.15.83:0/646659926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a7570 0x55aaa31aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.941+0000 7fefc0db9640 1 -- 172.21.15.83:0/646659926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31aae30 msgr2=0x55aaa31b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.941+0000 7fefc0db9640 1 --2- 172.21.15.83:0/646659926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31aae30 0x55aaa31b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.941+0000 7fefc0db9640 1 -- 172.21.15.83:0/646659926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31a5cc0 msgr2=0x55aaa31ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.941+0000 7fefc0db9640 1 --2- 172.21.15.83:0/646659926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31a5cc0 0x55aaa31ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.941+0000 7fefc0db9640 1 -- 172.21.15.83:0/646659926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaa30a8ac0 con 0x55aaa31a7570 2024-03-11T19:46:12.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.941+0000 7fefc0db9640 1 --2- 172.21.15.83:0/646659926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a7570 0x55aaa31aee00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fefb40bc7b0 tx=0x7fefb40bc520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7624cc957b556525 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.941+0000 7fefa1ffb640 1 -- 172.21.15.83:0/646659926 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40c5250 con 0x55aaa31a7570 2024-03-11T19:46:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.942+0000 7fefa1ffb640 1 -- 172.21.15.83:0/646659926 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fefb40bab50 con 0x55aaa31a7570 2024-03-11T19:46:12.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.942+0000 7fefa1ffb640 1 -- 172.21.15.83:0/646659926 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40bc160 con 0x55aaa31a7570 2024-03-11T19:46:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.943+0000 7fefc53f4740 1 -- 172.21.15.83:0/646659926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a7570 msgr2=0x55aaa31aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.943+0000 7fefc53f4740 1 --2- 172.21.15.83:0/646659926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a7570 0x55aaa31aee00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fefb40bc7b0 tx=0x7fefb40bc520 comp rx=0 tx=0).stop 2024-03-11T19:46:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.943+0000 7fefc53f4740 1 -- 172.21.15.83:0/646659926 shutdown_connections 2024-03-11T19:46:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.943+0000 7fefc53f4740 1 --2- 172.21.15.83:0/646659926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a7570 0x55aaa31aee00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.943+0000 7fefc53f4740 1 --2- 172.21.15.83:0/646659926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31aae30 0x55aaa31b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.943+0000 7fefc53f4740 1 --2- 172.21.15.83:0/646659926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31a5cc0 0x55aaa31ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.943+0000 7fefc53f4740 1 -- 172.21.15.83:0/646659926 >> 172.21.15.83:0/646659926 conn(0x55aaa31a07f0 msgr2=0x55aaa31a6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.943+0000 7fefc53f4740 1 -- 172.21.15.83:0/646659926 shutdown_connections 2024-03-11T19:46:12.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.943+0000 7fefc53f4740 1 -- 172.21.15.83:0/646659926 wait complete. 2024-03-11T19:46:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.944+0000 7fefc53f4740 1 -- start start 2024-03-11T19:46:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.944+0000 7fefc53f4740 1 Processor -- start 2024-03-11T19:46:12.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.944+0000 7fefc53f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a5cc0 0x55aaa31b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.944+0000 7fefc53f4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a7570 0x55aaa31b6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefc53f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a5cc0 0x55aaa31b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefc53f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa31b4ee0 con 0x55aaa31aae30 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefc53f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa31b5050 con 0x55aaa31a5cc0 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefc53f4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa31b5210 con 0x55aaa31a7570 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefbbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a5cc0 0x55aaa31b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:42000/0 (socket says 172.21.15.83:42000) 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefbbfff640 1 -- 172.21.15.83:0/2694325983 learned_addr learned my addr 172.21.15.83:0/2694325983 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefc53f4740 10 client.0 _ll_register_callbacks cb 0x7fefac00a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefbb7fe640 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a7570 0x55aaa31b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefc0db9640 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefbbfff640 1 -- 172.21.15.83:0/2694325983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a7570 msgr2=0x55aaa31b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefbbfff640 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a7570 0x55aaa31b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefbbfff640 1 -- 172.21.15.83:0/2694325983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 msgr2=0x55aaa31b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefbbfff640 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefbbfff640 1 -- 172.21.15.83:0/2694325983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc083600 con 0x55aaa31a5cc0 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fefbbfff640 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a5cc0 0x55aaa31b6110 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fefac012010 tx=0x7fefac022c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c42061cec36c4adb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.945+0000 7fef9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.946+0000 7fef9b7fe640 1 -- 172.21.15.83:0/2694325983 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac0047c0 con 0x55aaa31a5cc0 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.946+0000 7fef9b7fe640 1 -- 172.21.15.83:0/2694325983 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fefac000e40 con 0x55aaa31a5cc0 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.946+0000 7fefc53f4740 1 -- 172.21.15.83:0/2694325983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aaa2e0d7e0 con 0x55aaa31a5cc0 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.946+0000 7fef9b7fe640 1 -- 172.21.15.83:0/2694325983 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefac004920 con 0x55aaa31a5cc0 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.946+0000 7fefc53f4740 10 client.26174 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.946+0000 7fefc53f4740 1 -- 172.21.15.83:0/2694325983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55aaa30ce6a0 con 0x55aaa31a5cc0 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.946+0000 7fef9b7fe640 1 -- 172.21.15.83:0/2694325983 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aaa2e0d7e0 con 0x55aaa31a5cc0 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.946+0000 7fef9b7fe640 1 -- 172.21.15.83:0/2694325983 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fefac005040 con 0x55aaa31a5cc0 2024-03-11T19:46:12.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.946+0000 7fefc53f4740 10 client.26174 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.946+0000 7fefc53f4740 10 client.26174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.946+0000 7fefc53f4740 20 client.26174 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.947+0000 7fefc53f4740 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55aaa3212820 0x55aaa3214c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.947+0000 7fefc53f4740 4 client.26174 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:46:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.947+0000 7fefc53f4740 1 -- 172.21.15.83:0/2694325983 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aaa2dfc4e0 con 0x55aaa3212820 2024-03-11T19:46:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.947+0000 7fefbb7fe640 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55aaa3212820 0x55aaa3214c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.947+0000 7fefbb7fe640 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55aaa3212820 0x55aaa3214c40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb05cde4a81755e server_cookie=835b89d0cffdda0d in_seq=0 out_seq=0 2024-03-11T19:46:12.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.947+0000 7fef9b7fe640 10 client.26174 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.948+0000 7fef9b7fe640 1 -- 172.21.15.83:0/2694325983 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55aaa2dfc4e0 con 0x55aaa3212820 2024-03-11T19:46:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.948+0000 7fef9b7fe640 10 client.26174 handle_command_reply: tid=0 2024-03-11T19:46:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.948+0000 7fefc53f4740 1 client.26174 shutdown 2024-03-11T19:46:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.948+0000 7fefc53f4740 1 -- 172.21.15.83:0/2694325983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a5cc0 msgr2=0x55aaa31b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.948+0000 7fefc53f4740 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a5cc0 0x55aaa31b6110 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fefac012010 tx=0x7fefac022c40 comp rx=0 tx=0).stop 2024-03-11T19:46:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.948+0000 7fefc53f4740 1 -- 172.21.15.83:0/2694325983 shutdown_connections 2024-03-11T19:46:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.948+0000 7fefc53f4740 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55aaa3212820 0x55aaa3214c40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.949+0000 7fefc53f4740 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaa31aae30 0x55aaa31b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.949+0000 7fefc53f4740 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aaa31a7570 0x55aaa31b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.949+0000 7fefc53f4740 1 --2- 172.21.15.83:0/2694325983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aaa31a5cc0 0x55aaa31b6110 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.949+0000 7fefc53f4740 1 -- 172.21.15.83:0/2694325983 >> 172.21.15.83:0/2694325983 conn(0x55aaa31a07f0 msgr2=0x55aaa31a3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.949+0000 7fef9b7fe640 0 client.26174 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:12.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.949+0000 7fefc53f4740 1 -- 172.21.15.83:0/2694325983 shutdown_connections 2024-03-11T19:46:12.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.949+0000 7fefc53f4740 1 -- 172.21.15.83:0/2694325983 wait complete. 2024-03-11T19:46:12.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.949+0000 7fefc53f4740 20 client.26174 trim_cache size 0 max 16384 2024-03-11T19:46:12.957 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:46:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (134 inodes in the stack)", 2024-03-11T19:46:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:46:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "5a4b217f-b939-41f4-add3-e594ca254617": { 2024-03-11T19:46:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:46:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "5a4b217f-b939-41f4-add3-e594ca254617", 2024-03-11T19:46:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:46:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:46:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:46:12.958 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:46:12.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.950+0000 7fefc281c640 1 -- 172.21.15.83:0/3845585538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefa40a1ab0 msgr2=0x7fefa40a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.950+0000 7fefc281c640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefa40a1ab0 0x7fefa40a3f70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55aaa31a60c0 tx=0x7fefb0009260 comp rx=0 tx=0).stop 2024-03-11T19:46:12.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.950+0000 7fefc281c640 1 -- 172.21.15.83:0/3845585538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc10d830 msgr2=0x7fefbc082770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:12.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.950+0000 7fefc281c640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc10d830 0x7fefbc082770 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fefb4033420 tx=0x7fefb4033cb0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.950+0000 7fefc281c640 1 -- 172.21.15.83:0/3845585538 shutdown_connections 2024-03-11T19:46:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.950+0000 7fefc281c640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fefa40a1ab0 0x7fefa40a3f70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.950+0000 7fefc281c640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc10d830 0x7fefbc082770 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.950+0000 7fefc281c640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefbc10adb0 0x7fefbc07b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.950+0000 7fefc281c640 1 --2- 172.21.15.83:0/3845585538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefbc1083f0 0x7fefbc07b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.951+0000 7fefc281c640 1 -- 172.21.15.83:0/3845585538 >> 172.21.15.83:0/3845585538 conn(0x7fefbc06e360 msgr2=0x7fefbc10dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.951+0000 7fefc281c640 1 -- 172.21.15.83:0/3845585538 shutdown_connections 2024-03-11T19:46:12.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:12.951+0000 7fefc281c640 1 -- 172.21.15.83:0/3845585538 wait complete. 2024-03-11T19:46:12.970 INFO:tasks.cephfs.filesystem:scrub status for tag:5a4b217f-b939-41f4-add3-e594ca254617 - {'path': '/', 'tag': '5a4b217f-b939-41f4-add3-e594ca254617', 'options': 'recursive,force'} 2024-03-11T19:46:13.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:12 smithi137 ceph-mon[36220]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 MiB/s wr, 695 op/s 2024-03-11T19:46:13.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:12 smithi204 ceph-mon[36739]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 MiB/s wr, 695 op/s 2024-03-11T19:46:14.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:13 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:46:14.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:13 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:14.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:13 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:46:14.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:13 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:46:14.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:13 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:14.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:13 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:46:14.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:13 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:14.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:13 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:14.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:13 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:46:14.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:13 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:14.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:13 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:46:14.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:13 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:46:14.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:13 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:14.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:13 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:46:14.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:13 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:14.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:13 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:14.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:13 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:46:14.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:13 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:14.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:13 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:46:14.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:13 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:46:14.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:13 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:14.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:13 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:46:14.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:13 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:14.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:13 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:15.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:14 smithi083 ceph-mon[31864]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.3 MiB/s wr, 772 op/s 2024-03-11T19:46:15.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:14 smithi137 ceph-mon[36220]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.3 MiB/s wr, 772 op/s 2024-03-11T19:46:15.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:14 smithi204 ceph-mon[36739]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.3 MiB/s wr, 772 op/s 2024-03-11T19:46:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:16 smithi083 ceph-mon[31864]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 602 op/s 2024-03-11T19:46:17.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:16 smithi137 ceph-mon[36220]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 602 op/s 2024-03-11T19:46:17.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:16 smithi204 ceph-mon[36739]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 602 op/s 2024-03-11T19:46:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 420.0 s, 2717.0 tps, lat 36.982 ms stddev 46.955 2024-03-11T19:46:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:18 smithi083 ceph-mon[31864]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 753 op/s 2024-03-11T19:46:19.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:18 smithi137 ceph-mon[36220]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 753 op/s 2024-03-11T19:46:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:18 smithi204 ceph-mon[36739]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 753 op/s 2024-03-11T19:46:20.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:20 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:46:20.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:20 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:46:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:20 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:46:21.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:21 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:46:21.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:21 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:46:21.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:21 smithi137 ceph-mon[36220]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2024-03-11T19:46:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:21 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:46:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:21 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:46:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:21 smithi204 ceph-mon[36739]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2024-03-11T19:46:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:21 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:46:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:21 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:46:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:21 smithi083 ceph-mon[31864]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.6 MiB/s wr, 642 op/s 2024-03-11T19:46:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 425.0 s, 2437.4 tps, lat 40.919 ms stddev 46.993 2024-03-11T19:46:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:22 smithi083 ceph-mon[31864]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-03-11T19:46:23.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:22 smithi137 ceph-mon[36220]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-03-11T19:46:23.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:22 smithi204 ceph-mon[36739]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 663 op/s 2024-03-11T19:46:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:24 smithi083 ceph-mon[31864]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 753 op/s 2024-03-11T19:46:25.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:24 smithi137 ceph-mon[36220]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 753 op/s 2024-03-11T19:46:25.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:24 smithi204 ceph-mon[36739]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.8 MiB/s wr, 753 op/s 2024-03-11T19:46:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:26 smithi083 ceph-mon[31864]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 662 op/s 2024-03-11T19:46:27.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:26 smithi137 ceph-mon[36220]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 662 op/s 2024-03-11T19:46:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:26 smithi204 ceph-mon[36739]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 662 op/s 2024-03-11T19:46:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 430.0 s, 2755.2 tps, lat 36.213 ms stddev 42.896 2024-03-11T19:46:29.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:28 smithi083 ceph-mon[31864]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 812 op/s 2024-03-11T19:46:29.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:28 smithi137 ceph-mon[36220]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 812 op/s 2024-03-11T19:46:29.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:28 smithi204 ceph-mon[36739]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 812 op/s 2024-03-11T19:46:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:30 smithi083 ceph-mon[31864]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.6 MiB/s wr, 651 op/s 2024-03-11T19:46:31.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:30 smithi137 ceph-mon[36220]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.6 MiB/s wr, 651 op/s 2024-03-11T19:46:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:30 smithi204 ceph-mon[36739]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.6 MiB/s wr, 651 op/s 2024-03-11T19:46:32.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:31 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:32.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:31 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:31 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:46:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 435.0 s, 2342.0 tps, lat 42.680 ms stddev 50.881 2024-03-11T19:46:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi204 to 174.1M 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: Adjusting osd_memory_target on smithi137 to 174.1M 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: Unable to set osd_memory_target on smithi204 to 182606950: error parsing value: Value '182606950' is below minimum 939524096 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: Unable to set osd_memory_target on smithi137 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:46:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:32 smithi083 ceph-mon[31864]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 MiB/s wr, 672 op/s 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi204 to 174.1M 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: Adjusting osd_memory_target on smithi137 to 174.1M 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: Unable to set osd_memory_target on smithi204 to 182606950: error parsing value: Value '182606950' is below minimum 939524096 2024-03-11T19:46:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: Unable to set osd_memory_target on smithi137 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-03-11T19:46:33.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:33.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:33.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:46:33.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:32 smithi137 ceph-mon[36220]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 MiB/s wr, 672 op/s 2024-03-11T19:46:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-11T19:46:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi204 to 174.1M 2024-03-11T19:46:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: Adjusting osd_memory_target on smithi137 to 174.1M 2024-03-11T19:46:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: Unable to set osd_memory_target on smithi204 to 182606950: error parsing value: Value '182606950' is below minimum 939524096 2024-03-11T19:46:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: Unable to set osd_memory_target on smithi137 to 182617702: error parsing value: Value '182617702' is below minimum 939524096 2024-03-11T19:46:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:46:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:46:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:46:33.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:32 smithi204 ceph-mon[36739]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 MiB/s wr, 672 op/s 2024-03-11T19:46:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:34 smithi083 ceph-mon[31864]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 774 op/s 2024-03-11T19:46:35.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:34 smithi137 ceph-mon[36220]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 774 op/s 2024-03-11T19:46:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:34 smithi204 ceph-mon[36739]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 774 op/s 2024-03-11T19:46:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:36 smithi083 ceph-mon[31864]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.9 MiB/s wr, 679 op/s 2024-03-11T19:46:37.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:36 smithi137 ceph-mon[36220]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.9 MiB/s wr, 679 op/s 2024-03-11T19:46:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:36 smithi204 ceph-mon[36739]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.9 MiB/s wr, 679 op/s 2024-03-11T19:46:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 440.0 s, 2775.0 tps, lat 36.078 ms stddev 43.786 2024-03-11T19:46:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:38 smithi083 ceph-mon[31864]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 823 op/s 2024-03-11T19:46:39.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:38 smithi137 ceph-mon[36220]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 823 op/s 2024-03-11T19:46:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:38 smithi204 ceph-mon[36739]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 823 op/s 2024-03-11T19:46:41.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:40 smithi083 ceph-mon[31864]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.3 MiB/s wr, 651 op/s 2024-03-11T19:46:41.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:40 smithi204 ceph-mon[36739]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.3 MiB/s wr, 651 op/s 2024-03-11T19:46:41.179 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:40 smithi137 ceph-mon[36220]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.3 MiB/s wr, 651 op/s 2024-03-11T19:46:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 445.0 s, 3123.2 tps, lat 32.131 ms stddev 41.139 2024-03-11T19:46:42.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:46:42.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:46:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:42 smithi083 ceph-mon[31864]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.7 MiB/s wr, 697 op/s 2024-03-11T19:46:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.101+0000 7f227f5c7640 1 -- 172.21.15.83:0/209463871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2278101c00 msgr2=0x7f2278103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.101+0000 7f227f5c7640 1 --2- 172.21.15.83:0/209463871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2278101c00 0x7f2278103ff0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2274009900 tx=0x7f227402bc30 comp rx=0 tx=0).stop 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.102+0000 7f227f5c7640 1 -- 172.21.15.83:0/209463871 shutdown_connections 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.102+0000 7f227f5c7640 1 --2- 172.21.15.83:0/209463871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2278106e60 0x7f22781092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.102+0000 7f227f5c7640 1 --2- 172.21.15.83:0/209463871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2278104530 0x7f2278106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.102+0000 7f227f5c7640 1 --2- 172.21.15.83:0/209463871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2278101c00 0x7f2278103ff0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.102+0000 7f227f5c7640 1 -- 172.21.15.83:0/209463871 >> 172.21.15.83:0/209463871 conn(0x7f22780fb5f0 msgr2=0x7f22780fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.102+0000 7f227f5c7640 1 -- 172.21.15.83:0/209463871 shutdown_connections 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.102+0000 7f227f5c7640 1 -- 172.21.15.83:0/209463871 wait complete. 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.103+0000 7f227f5c7640 1 Processor -- start 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.103+0000 7f227f5c7640 1 -- start start 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.103+0000 7f227f5c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2278101c00 0x7f2278197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.103+0000 7f227f5c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2278104530 0x7f2278197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.103+0000 7f227f5c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2278106e60 0x7f227819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.103+0000 7f227f5c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227819ed30 con 0x7f2278106e60 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.103+0000 7f227f5c7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227819eea0 con 0x7f2278104530 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.103+0000 7f227f5c7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227819f060 con 0x7f2278101c00 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.103+0000 7f227db64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2278106e60 0x7f227819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227d363640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2278101c00 0x7f2278197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227db64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2278106e60 0x7f227819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52302/0 (socket says 172.21.15.83:52302) 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227db64640 1 -- 172.21.15.83:0/926827460 learned_addr learned my addr 172.21.15.83:0/926827460 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227cb62640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2278104530 0x7f2278197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227d363640 1 -- 172.21.15.83:0/926827460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2278104530 msgr2=0x7f2278197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227d363640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2278104530 0x7f2278197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227d363640 1 -- 172.21.15.83:0/926827460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2278106e60 msgr2=0x7f227819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227d363640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2278106e60 0x7f227819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227d363640 1 -- 172.21.15.83:0/926827460 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274009560 con 0x7f2278101c00 2024-03-11T19:46:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227db64640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2278106e60 0x7f227819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227d363640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2278101c00 0x7f2278197240 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f227402c170 tx=0x7f22740042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f226e7fc640 1 -- 172.21.15.83:0/926827460 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740044e0 con 0x7f2278101c00 2024-03-11T19:46:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227f5c7640 1 -- 172.21.15.83:0/926827460 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227819f2e0 con 0x7f2278101c00 2024-03-11T19:46:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f226e7fc640 1 -- 172.21.15.83:0/926827460 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22740054e0 con 0x7f2278101c00 2024-03-11T19:46:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f226e7fc640 1 -- 172.21.15.83:0/926827460 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274003e90 con 0x7f2278101c00 2024-03-11T19:46:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.104+0000 7f227f5c7640 1 -- 172.21.15.83:0/926827460 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227819f7c0 con 0x7f2278101c00 2024-03-11T19:46:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.107+0000 7f226e7fc640 1 -- 172.21.15.83:0/926827460 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f227403e450 con 0x7f2278101c00 2024-03-11T19:46:43.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.107+0000 7f2282199740 1 Processor -- start 2024-03-11T19:46:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.108+0000 7f2282199740 1 -- start start 2024-03-11T19:46:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.108+0000 7f2282199740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad924080 0x5574ad9244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.108+0000 7f2282199740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924a70 0x5574ad929050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.108+0000 7f2282199740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad92bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.108+0000 7f2282199740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad88b280 con 0x5574ad924080 2024-03-11T19:46:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.108+0000 7f2282199740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad8887b0 con 0x5574ad924a70 2024-03-11T19:46:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.108+0000 7f2282199740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad883f90 con 0x5574ad9296c0 2024-03-11T19:46:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.108+0000 7f227cb62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad92bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.108+0000 7f227cb62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad92bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48408/0 (socket says 172.21.15.83:48408) 2024-03-11T19:46:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.108+0000 7f227cb62640 1 -- 172.21.15.83:0/2207076744 learned_addr learned my addr 172.21.15.83:0/2207076744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.108+0000 7f227db64640 1 --2- 172.21.15.83:0/2207076744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad924080 0x5574ad9244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.109+0000 7f227db64640 1 -- 172.21.15.83:0/2207076744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 msgr2=0x5574ad92bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.109+0000 7f227db64640 1 --2- 172.21.15.83:0/2207076744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad92bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.109+0000 7f227db64640 1 -- 172.21.15.83:0/2207076744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924a70 msgr2=0x5574ad929050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.109+0000 7f227d363640 1 --2- 172.21.15.83:0/2207076744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924a70 0x5574ad929050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.109+0000 7f227db64640 1 --2- 172.21.15.83:0/2207076744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924a70 0x5574ad929050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.109+0000 7f227db64640 1 -- 172.21.15.83:0/2207076744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ad842570 con 0x5574ad924080 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.109+0000 7f227cb62640 1 --2- 172.21.15.83:0/2207076744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad92bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.109+0000 7f227d363640 1 --2- 172.21.15.83:0/2207076744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924a70 0x5574ad929050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.109+0000 7f226e7fc640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22640a1a30 0x7f22640a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.109+0000 7f227d363640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22640a1a30 0x7f22640a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.110+0000 7f226e7fc640 1 -- 172.21.15.83:0/926827460 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f22740efb90 con 0x7f2278101c00 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.110+0000 7f227db64640 1 --2- 172.21.15.83:0/2207076744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad924080 0x5574ad9244a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f22700107d0 tx=0x7f2270010c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3899fd93298c04 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.110+0000 7f227d363640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22640a1a30 0x7f22640a3ef0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x5574ad924f60 tx=0x7f2274042000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.110+0000 7f22637fe640 1 -- 172.21.15.83:0/2207076744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270010ee0 con 0x5574ad924080 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.110+0000 7f22637fe640 1 -- 172.21.15.83:0/2207076744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f227000ee10 con 0x5574ad924080 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.110+0000 7f22637fe640 1 -- 172.21.15.83:0/2207076744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270005900 con 0x5574ad924080 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.111+0000 7f2282199740 1 -- 172.21.15.83:0/2207076744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad924080 msgr2=0x5574ad9244a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.111+0000 7f2282199740 1 --2- 172.21.15.83:0/2207076744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad924080 0x5574ad9244a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f22700107d0 tx=0x7f2270010c90 comp rx=0 tx=0).stop 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.111+0000 7f2282199740 1 -- 172.21.15.83:0/2207076744 shutdown_connections 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.111+0000 7f2282199740 1 --2- 172.21.15.83:0/2207076744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad92bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.111+0000 7f2282199740 1 --2- 172.21.15.83:0/2207076744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924a70 0x5574ad929050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.111+0000 7f2282199740 1 --2- 172.21.15.83:0/2207076744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad924080 0x5574ad9244a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.111+0000 7f2282199740 1 -- 172.21.15.83:0/2207076744 >> 172.21.15.83:0/2207076744 conn(0x5574ad916770 msgr2=0x5574ad916b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.111+0000 7f2282199740 1 -- 172.21.15.83:0/2207076744 shutdown_connections 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.111+0000 7f2282199740 1 -- 172.21.15.83:0/2207076744 wait complete. 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.111+0000 7f2282199740 1 -- start start 2024-03-11T19:46:43.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.112+0000 7f2282199740 1 Processor -- start 2024-03-11T19:46:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.112+0000 7f2282199740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924080 0x5574ad93eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.112+0000 7f2282199740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924a70 0x5574ad93f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.113+0000 7f227db64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924080 0x5574ad93eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.113+0000 7f227db64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924080 0x5574ad93eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48412/0 (socket says 172.21.15.83:48412) 2024-03-11T19:46:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.113+0000 7f227db64640 1 -- 172.21.15.83:0/708354049 learned_addr learned my addr 172.21.15.83:0/708354049 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.113+0000 7f2282199740 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 0x5574ad946040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.113+0000 7f2282199740 1 -- 172.21.15.83:0/708354049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad557430 con 0x5574ad9296c0 2024-03-11T19:46:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.113+0000 7f2282199740 1 -- 172.21.15.83:0/708354049 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad513e00 con 0x5574ad924a70 2024-03-11T19:46:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.113+0000 7f2282199740 1 -- 172.21.15.83:0/708354049 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad808fd0 con 0x5574ad924080 2024-03-11T19:46:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.113+0000 7f227db64640 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 0x5574ad946040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.113+0000 7f2282199740 10 client.0 _ll_register_callbacks cb 0x5574ad85efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.113+0000 7f227db64640 1 -- 172.21.15.83:0/708354049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924a70 msgr2=0x5574ad93f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f227db64640 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924a70 0x5574ad93f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f227db64640 1 -- 172.21.15.83:0/708354049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 msgr2=0x5574ad946040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f227db64640 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 0x5574ad946040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f227db64640 1 -- 172.21.15.83:0/708354049 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2270010430 con 0x5574ad924080 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f2282199740 1 -- 172.21.15.83:0/708354049 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574ad5837e0 con 0x5574ad924080 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f227db64640 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924080 0x5574ad93eac0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f22700053a0 tx=0x7f227000ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f66bbf0ddded32f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f2260ff9640 10 client.26296 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f2260ff9640 1 -- 172.21.15.83:0/708354049 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270004e50 con 0x5574ad924080 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f2260ff9640 1 -- 172.21.15.83:0/708354049 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2270004fb0 con 0x5574ad924080 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f2260ff9640 1 -- 172.21.15.83:0/708354049 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22700059f0 con 0x5574ad924080 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f2282199740 10 client.26296 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.114+0000 7f2282199740 1 -- 172.21.15.83:0/708354049 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=31}) v3 -- 0x5574ad882610 con 0x5574ad924080 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.115+0000 7f2260ff9640 1 -- 172.21.15.83:0/708354049 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2270003970 con 0x5574ad924080 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.115+0000 7f2282199740 10 client.26296 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.115+0000 7f2282199740 10 client.26296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.115+0000 7f2282199740 20 client.26296 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.115+0000 7f2282199740 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574ad94aa40 0x5574ad94ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.115+0000 7f2282199740 4 client.26296 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.115+0000 7f2282199740 1 -- 172.21.15.83:0/708354049 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574ad5c0720 con 0x5574ad94aa40 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.115+0000 7f227cb62640 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574ad94aa40 0x5574ad94ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2260ff9640 10 client.26296 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f227cb62640 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574ad94aa40 0x5574ad94ce60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df90776b8ce23f08 server_cookie=c2ff6a4e113dfd0e in_seq=0 out_seq=0 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2260ff9640 1 -- 172.21.15.83:0/708354049 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574ad5c0720 con 0x5574ad94aa40 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2260ff9640 10 client.26296 handle_command_reply: tid=0 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2282199740 1 client.26296 shutdown 2024-03-11T19:46:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2282199740 1 -- 172.21.15.83:0/708354049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924080 msgr2=0x5574ad93eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2282199740 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924080 0x5574ad93eac0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f22700053a0 tx=0x7f227000ef80 comp rx=0 tx=0).stop 2024-03-11T19:46:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2282199740 1 -- 172.21.15.83:0/708354049 shutdown_connections 2024-03-11T19:46:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2282199740 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574ad94aa40 0x5574ad94ce60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2282199740 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 0x5574ad946040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2260ff9640 0 client.26296 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2282199740 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924a70 0x5574ad93f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2282199740 1 --2- 172.21.15.83:0/708354049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924080 0x5574ad93eac0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.116+0000 7f2282199740 1 -- 172.21.15.83:0/708354049 >> 172.21.15.83:0/708354049 conn(0x5574ad916770 msgr2=0x5574ad923710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.117+0000 7f2282199740 1 -- 172.21.15.83:0/708354049 shutdown_connections 2024-03-11T19:46:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.117+0000 7f2282199740 1 -- 172.21.15.83:0/708354049 wait complete. 2024-03-11T19:46:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.117+0000 7f2282199740 20 client.26296 trim_cache size 0 max 16384 2024-03-11T19:46:43.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:42 smithi137 ceph-mon[36220]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.7 MiB/s wr, 697 op/s 2024-03-11T19:46:43.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.143+0000 7f2282199740 1 Processor -- start 2024-03-11T19:46:43.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.143+0000 7f2282199740 1 -- start start 2024-03-11T19:46:43.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.143+0000 7f2282199740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924080 0x5574ad9151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.143+0000 7f2282199740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad924a70 0x5574ad946000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.143+0000 7f2282199740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad915700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.143+0000 7f2282199740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad9248e0 con 0x5574ad924a70 2024-03-11T19:46:43.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.143+0000 7f2282199740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad952090 con 0x5574ad924080 2024-03-11T19:46:43.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.143+0000 7f2282199740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad9171c0 con 0x5574ad9296c0 2024-03-11T19:46:43.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.143+0000 7f227db64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad915700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.143+0000 7f227db64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad915700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48416/0 (socket says 172.21.15.83:48416) 2024-03-11T19:46:43.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.143+0000 7f227db64640 1 -- 172.21.15.83:0/286826724 learned_addr learned my addr 172.21.15.83:0/286826724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:43.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.144+0000 7f227db64640 1 -- 172.21.15.83:0/286826724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924080 msgr2=0x5574ad9151c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.144+0000 7f227db64640 1 --2- 172.21.15.83:0/286826724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924080 0x5574ad9151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.144+0000 7f227db64640 1 -- 172.21.15.83:0/286826724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad924a70 msgr2=0x5574ad946000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:46:43.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.144+0000 7f227db64640 1 --2- 172.21.15.83:0/286826724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad924a70 0x5574ad946000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.144+0000 7f227db64640 1 -- 172.21.15.83:0/286826724 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ad81eac0 con 0x5574ad9296c0 2024-03-11T19:46:43.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.150+0000 7f227db64640 1 --2- 172.21.15.83:0/286826724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad915700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2270010700 tx=0x7f2270009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e95a19348ef4e82c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:42 smithi204 ceph-mon[36739]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 8.7 MiB/s wr, 697 op/s 2024-03-11T19:46:43.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.151+0000 7f2262ffd640 1 -- 172.21.15.83:0/286826724 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270038030 con 0x5574ad9296c0 2024-03-11T19:46:43.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.151+0000 7f2262ffd640 1 -- 172.21.15.83:0/286826724 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5574ad5837e0 con 0x5574ad9296c0 2024-03-11T19:46:43.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.151+0000 7f2262ffd640 1 -- 172.21.15.83:0/286826724 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270003fa0 con 0x5574ad9296c0 2024-03-11T19:46:43.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.152+0000 7f2282199740 1 -- 172.21.15.83:0/286826724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 msgr2=0x5574ad915700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.152+0000 7f2282199740 1 --2- 172.21.15.83:0/286826724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad915700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2270010700 tx=0x7f2270009760 comp rx=0 tx=0).stop 2024-03-11T19:46:43.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.153+0000 7f2282199740 1 -- 172.21.15.83:0/286826724 shutdown_connections 2024-03-11T19:46:43.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.153+0000 7f2282199740 1 --2- 172.21.15.83:0/286826724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad9296c0 0x5574ad915700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.153+0000 7f2282199740 1 --2- 172.21.15.83:0/286826724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad924a70 0x5574ad946000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.153+0000 7f2282199740 1 --2- 172.21.15.83:0/286826724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924080 0x5574ad9151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.153+0000 7f2282199740 1 -- 172.21.15.83:0/286826724 >> 172.21.15.83:0/286826724 conn(0x5574ad916770 msgr2=0x5574ad925810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.161+0000 7f2282199740 1 -- 172.21.15.83:0/286826724 shutdown_connections 2024-03-11T19:46:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.161+0000 7f2282199740 1 -- 172.21.15.83:0/286826724 wait complete. 2024-03-11T19:46:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.161+0000 7f2282199740 1 -- start start 2024-03-11T19:46:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.161+0000 7f2282199740 1 Processor -- start 2024-03-11T19:46:43.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f2282199740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924080 0x5574ad945f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f2282199740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924a70 0x5574ad9445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f2282199740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 0x5574ad944b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f2282199740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad945260 con 0x5574ad9296c0 2024-03-11T19:46:43.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f2282199740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad9453d0 con 0x5574ad924080 2024-03-11T19:46:43.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f2282199740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ad9a0f70 con 0x5574ad924a70 2024-03-11T19:46:43.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f2282199740 10 client.0 _ll_register_callbacks cb 0x7f2270009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:43.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f227db64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 0x5574ad944b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f227db64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 0x5574ad944b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52322/0 (socket says 172.21.15.83:52322) 2024-03-11T19:46:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f227db64640 1 -- 172.21.15.83:0/354032680 learned_addr learned my addr 172.21.15.83:0/354032680 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f227db64640 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924080 0x5574ad945f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f227cb62640 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924a70 0x5574ad9445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f227db64640 1 -- 172.21.15.83:0/354032680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924a70 msgr2=0x5574ad9445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f227db64640 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924a70 0x5574ad9445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.162+0000 7f227db64640 1 -- 172.21.15.83:0/354032680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924080 msgr2=0x5574ad945f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.163+0000 7f227db64640 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924080 0x5574ad945f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.163+0000 7f227db64640 1 -- 172.21.15.83:0/354032680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574ad882610 con 0x5574ad9296c0 2024-03-11T19:46:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.163+0000 7f227db64640 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 0x5574ad944b60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2270005a10 tx=0x7f2270003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c82051fc66090c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.163+0000 7f2260ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:46:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.167+0000 7f2260ff9640 1 -- 172.21.15.83:0/354032680 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270038070 con 0x5574ad9296c0 2024-03-11T19:46:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.167+0000 7f2282199740 1 -- 172.21.15.83:0/354032680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f227000ee10 con 0x5574ad9296c0 2024-03-11T19:46:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.167+0000 7f2260ff9640 1 -- 172.21.15.83:0/354032680 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2270039070 con 0x5574ad9296c0 2024-03-11T19:46:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.168+0000 7f2260ff9640 1 -- 172.21.15.83:0/354032680 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2270003f50 con 0x5574ad9296c0 2024-03-11T19:46:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.170+0000 7f2282199740 10 client.16725 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.170+0000 7f2282199740 1 -- 172.21.15.83:0/354032680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5574ad8446a0 con 0x5574ad9296c0 2024-03-11T19:46:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.171+0000 7f2260ff9640 1 -- 172.21.15.83:0/354032680 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f227000ee10 con 0x5574ad9296c0 2024-03-11T19:46:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.171+0000 7f2260ff9640 1 -- 172.21.15.83:0/354032680 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f2270041890 con 0x5574ad9296c0 2024-03-11T19:46:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.171+0000 7f2282199740 10 client.16725 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.171+0000 7f2282199740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.171+0000 7f2282199740 20 client.16725 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.171+0000 7f2282199740 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574ad94aa40 0x5574ad9a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.171+0000 7f2282199740 4 client.16725 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:46:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.171+0000 7f2282199740 1 -- 172.21.15.83:0/354032680 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5574ad60f890 con 0x5574ad94aa40 2024-03-11T19:46:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.172+0000 7f227cb62640 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574ad94aa40 0x5574ad9a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.181+0000 7f227cb62640 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574ad94aa40 0x5574ad9a4570 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47b03a0fe6e2c0a server_cookie=9a9d44912b8eb0b3 in_seq=0 out_seq=0 2024-03-11T19:46:43.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.182+0000 7f2260ff9640 10 client.16725 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.182+0000 7f2260ff9640 1 -- 172.21.15.83:0/354032680 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5574ad60f890 con 0x5574ad94aa40 2024-03-11T19:46:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.182+0000 7f2260ff9640 10 client.16725 handle_command_reply: tid=0 2024-03-11T19:46:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.182+0000 7f2282199740 1 client.16725 shutdown 2024-03-11T19:46:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.187+0000 7f2282199740 1 -- 172.21.15.83:0/354032680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 msgr2=0x5574ad944b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.187+0000 7f2282199740 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 0x5574ad944b60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2270005a10 tx=0x7f2270003e60 comp rx=0 tx=0).stop 2024-03-11T19:46:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.187+0000 7f2282199740 1 -- 172.21.15.83:0/354032680 shutdown_connections 2024-03-11T19:46:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.187+0000 7f2282199740 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574ad94aa40 0x5574ad9a4570 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.187+0000 7f2282199740 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574ad9296c0 0x5574ad944b60 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.187+0000 7f2282199740 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ad924a70 0x5574ad9445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.187+0000 7f2282199740 1 --2- 172.21.15.83:0/354032680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574ad924080 0x5574ad945f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.187+0000 7f2282199740 1 -- 172.21.15.83:0/354032680 >> 172.21.15.83:0/354032680 conn(0x5574ad916770 msgr2=0x5574ad922e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.190+0000 7f2260ff9640 0 client.16725 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:43.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.195+0000 7f2282199740 1 -- 172.21.15.83:0/354032680 shutdown_connections 2024-03-11T19:46:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.197+0000 7f2282199740 1 -- 172.21.15.83:0/354032680 wait complete. 2024-03-11T19:46:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.199+0000 7f2282199740 20 client.16725 trim_cache size 0 max 16384 2024-03-11T19:46:43.204 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:46:43.204 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:46:43.205 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:46:43.205 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:46:43.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.202+0000 7f227f5c7640 1 -- 172.21.15.83:0/926827460 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22640a1a30 msgr2=0x7f22640a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.202+0000 7f227f5c7640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22640a1a30 0x7f22640a3ef0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x5574ad924f60 tx=0x7f2274042000 comp rx=0 tx=0).stop 2024-03-11T19:46:43.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.202+0000 7f227f5c7640 1 -- 172.21.15.83:0/926827460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2278101c00 msgr2=0x7f2278197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.203+0000 7f227f5c7640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2278101c00 0x7f2278197240 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f227402c170 tx=0x7f22740042f0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.203+0000 7f227f5c7640 1 -- 172.21.15.83:0/926827460 shutdown_connections 2024-03-11T19:46:43.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.203+0000 7f227f5c7640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22640a1a30 0x7f22640a3ef0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.203+0000 7f227f5c7640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2278106e60 0x7f227819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.203+0000 7f227f5c7640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2278104530 0x7f2278197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.204+0000 7f227f5c7640 1 --2- 172.21.15.83:0/926827460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2278101c00 0x7f2278197240 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.204+0000 7f227f5c7640 1 -- 172.21.15.83:0/926827460 >> 172.21.15.83:0/926827460 conn(0x7f22780fb5f0 msgr2=0x7f2278102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.204+0000 7f227f5c7640 1 -- 172.21.15.83:0/926827460 shutdown_connections 2024-03-11T19:46:43.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.204+0000 7f227f5c7640 1 -- 172.21.15.83:0/926827460 wait complete. 2024-03-11T19:46:43.224 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:46:43.225 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:46:43.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.365+0000 7ff9701c9640 1 -- 172.21.15.83:0/2389325976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff96810aaf0 msgr2=0x7ff96810cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.365+0000 7ff9701c9640 1 --2- 172.21.15.83:0/2389325976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff96810aaf0 0x7ff96810cf80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff96401a910 tx=0x7ff96403cc80 comp rx=0 tx=0).stop 2024-03-11T19:46:43.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.365+0000 7ff9701c9640 1 -- 172.21.15.83:0/2389325976 shutdown_connections 2024-03-11T19:46:43.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.365+0000 7ff9701c9640 1 --2- 172.21.15.83:0/2389325976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff96810d4c0 0x7ff96810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.365+0000 7ff9701c9640 1 --2- 172.21.15.83:0/2389325976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff96810aaf0 0x7ff96810cf80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.365+0000 7ff9701c9640 1 --2- 172.21.15.83:0/2389325976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff968108180 0x7ff96810a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.365+0000 7ff9701c9640 1 -- 172.21.15.83:0/2389325976 >> 172.21.15.83:0/2389325976 conn(0x7ff96806dff0 msgr2=0x7ff968070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.371+0000 7ff9701c9640 1 -- 172.21.15.83:0/2389325976 shutdown_connections 2024-03-11T19:46:43.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.371+0000 7ff9701c9640 1 -- 172.21.15.83:0/2389325976 wait complete. 2024-03-11T19:46:43.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.374+0000 7ff9701c9640 1 Processor -- start 2024-03-11T19:46:43.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.377+0000 7ff9701c9640 1 -- start start 2024-03-11T19:46:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.377+0000 7ff9701c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff968108180 0x7ff96819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.377+0000 7ff9701c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff96810aaf0 0x7ff96819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.377+0000 7ff9701c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff96810d4c0 0x7ff9681a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.377+0000 7ff9701c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681a53c0 con 0x7ff96810aaf0 2024-03-11T19:46:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.377+0000 7ff9701c9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681a5530 con 0x7ff968108180 2024-03-11T19:46:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.377+0000 7ff9701c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681a56f0 con 0x7ff96810d4c0 2024-03-11T19:46:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.377+0000 7ff96df65640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff968108180 0x7ff96819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.377+0000 7ff96df65640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff968108180 0x7ff96819d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52650/0 (socket says 172.21.15.83:52650) 2024-03-11T19:46:43.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.377+0000 7ff96df65640 1 -- 172.21.15.83:0/4265716724 learned_addr learned my addr 172.21.15.83:0/4265716724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.380+0000 7ff96df65640 1 -- 172.21.15.83:0/4265716724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff96810d4c0 msgr2=0x7ff9681a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.380+0000 7ff96df65640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff96810d4c0 0x7ff9681a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.380+0000 7ff96df65640 1 -- 172.21.15.83:0/4265716724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff96810aaf0 msgr2=0x7ff96819de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.380+0000 7ff96df65640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff96810aaf0 0x7ff96819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.380+0000 7ff96df65640 1 -- 172.21.15.83:0/4265716724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96401a570 con 0x7ff968108180 2024-03-11T19:46:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.382+0000 7ff96df65640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff968108180 0x7ff96819d8d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff95800b470 tx=0x7ff95800b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.383+0000 7ff95effd640 1 -- 172.21.15.83:0/4265716724 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958011890 con 0x7ff968108180 2024-03-11T19:46:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.383+0000 7ff9701c9640 1 -- 172.21.15.83:0/4265716724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9681a59d0 con 0x7ff968108180 2024-03-11T19:46:43.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.383+0000 7ff9701c9640 1 -- 172.21.15.83:0/4265716724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9681a5f60 con 0x7ff968108180 2024-03-11T19:46:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.383+0000 7ff95effd640 1 -- 172.21.15.83:0/4265716724 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9580103e0 con 0x7ff968108180 2024-03-11T19:46:43.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.384+0000 7ff95effd640 1 -- 172.21.15.83:0/4265716724 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95800f580 con 0x7ff968108180 2024-03-11T19:46:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.388+0000 7ff9701c9640 1 -- 172.21.15.83:0/4265716724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94c0052c0 con 0x7ff968108180 2024-03-11T19:46:43.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.388+0000 7ff95effd640 1 -- 172.21.15.83:0/4265716724 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff958010550 con 0x7ff968108180 2024-03-11T19:46:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.389+0000 7ff95effd640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9540a1a50 0x7ff9540a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.389+0000 7ff95effd640 1 -- 172.21.15.83:0/4265716724 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff9580cd1b0 con 0x7ff968108180 2024-03-11T19:46:43.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.390+0000 7ff96d764640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9540a1a50 0x7ff9540a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.391+0000 7ff96d764640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9540a1a50 0x7ff9540a3f10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff9640047e0 tx=0x7ff964000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.403+0000 7ff95effd640 1 -- 172.21.15.83:0/4265716724 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9580963d0 con 0x7ff968108180 2024-03-11T19:46:43.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.713+0000 7ff9701c9640 1 -- 172.21.15.83:0/4265716724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff94c005af0 con 0x7ff968108180 2024-03-11T19:46:43.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.714+0000 7ff95effd640 1 -- 172.21.15.83:0/4265716724 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11350 (secure 0 0 0) 0x7ff958099a90 con 0x7ff968108180 2024-03-11T19:46:43.718 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:46:43.718 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:45:19.152624+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:46:43.719 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 31 2024-03-11T19:46:43.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 -- 172.21.15.83:0/4265716724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9540a1a50 msgr2=0x7ff9540a3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9540a1a50 0x7ff9540a3f10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff9640047e0 tx=0x7ff964000bc0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 -- 172.21.15.83:0/4265716724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff968108180 msgr2=0x7ff96819d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff968108180 0x7ff96819d8d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff95800b470 tx=0x7ff95800b930 comp rx=0 tx=0).stop 2024-03-11T19:46:43.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 -- 172.21.15.83:0/4265716724 shutdown_connections 2024-03-11T19:46:43.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff9540a1a50 0x7ff9540a3f10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff96810d4c0 0x7ff9681a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff96810aaf0 0x7ff96819de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 --2- 172.21.15.83:0/4265716724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff968108180 0x7ff96819d8d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 -- 172.21.15.83:0/4265716724 >> 172.21.15.83:0/4265716724 conn(0x7ff96806dff0 msgr2=0x7ff96810b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 -- 172.21.15.83:0/4265716724 shutdown_connections 2024-03-11T19:46:43.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.716+0000 7ff9701c9640 1 -- 172.21.15.83:0/4265716724 wait complete. 2024-03-11T19:46:43.733 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:46:43.733 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:46:43.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.884+0000 7fa08c75e640 1 -- 172.21.15.83:0/3518656577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa084101270 msgr2=0x7fa0841016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.884+0000 7fa08c75e640 1 --2- 172.21.15.83:0/3518656577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa084101270 0x7fa0841016f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa074009900 tx=0x7fa07402bc30 comp rx=0 tx=0).stop 2024-03-11T19:46:43.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.884+0000 7fa08c75e640 1 -- 172.21.15.83:0/3518656577 shutdown_connections 2024-03-11T19:46:43.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.884+0000 7fa08c75e640 1 --2- 172.21.15.83:0/3518656577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa084101c30 0x7fa0841084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.884+0000 7fa08c75e640 1 --2- 172.21.15.83:0/3518656577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa084101270 0x7fa0841016f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.884+0000 7fa08c75e640 1 --2- 172.21.15.83:0/3518656577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa084100080 0x7fa0841004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.884+0000 7fa08c75e640 1 -- 172.21.15.83:0/3518656577 >> 172.21.15.83:0/3518656577 conn(0x7fa0840fb610 msgr2=0x7fa0840fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.884+0000 7fa08c75e640 1 -- 172.21.15.83:0/3518656577 shutdown_connections 2024-03-11T19:46:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.884+0000 7fa08c75e640 1 -- 172.21.15.83:0/3518656577 wait complete. 2024-03-11T19:46:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08c75e640 1 Processor -- start 2024-03-11T19:46:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08c75e640 1 -- start start 2024-03-11T19:46:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08c75e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa084100080 0x7fa084068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08c75e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa084101270 0x7fa08419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08c75e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa084101c30 0x7fa08419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08c75e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08419ed90 con 0x7fa084100080 2024-03-11T19:46:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08c75e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08419ef00 con 0x7fa084101c30 2024-03-11T19:46:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08c75e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08419f0c0 con 0x7fa084101270 2024-03-11T19:46:43.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08acfb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa084101c30 0x7fa08419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08a4fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa084100080 0x7fa084068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08a4fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa084100080 0x7fa084068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52352/0 (socket says 172.21.15.83:52352) 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa08a4fa640 1 -- 172.21.15.83:0/2102994739 learned_addr learned my addr 172.21.15.83:0/2102994739 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.885+0000 7fa089cf9640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa084101270 0x7fa08419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.886+0000 7fa08a4fa640 1 -- 172.21.15.83:0/2102994739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa084101270 msgr2=0x7fa08419bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.886+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa084101270 0x7fa08419bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.886+0000 7fa08a4fa640 1 -- 172.21.15.83:0/2102994739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa084101c30 msgr2=0x7fa08419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.886+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa084101c30 0x7fa08419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.886+0000 7fa08a4fa640 1 -- 172.21.15.83:0/2102994739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa074009560 con 0x7fa084100080 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.886+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa084100080 0x7fa084068e50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00ea80 tx=0x7fa07c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.887+0000 7fa07b7fe640 1 -- 172.21.15.83:0/2102994739 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c00ce40 con 0x7fa084100080 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.887+0000 7fa07b7fe640 1 -- 172.21.15.83:0/2102994739 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa07c013c50 con 0x7fa084100080 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.887+0000 7fa08c75e640 1 -- 172.21.15.83:0/2102994739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa084195dc0 con 0x7fa084100080 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.887+0000 7fa08c75e640 1 -- 172.21.15.83:0/2102994739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0841962d0 con 0x7fa084100080 2024-03-11T19:46:43.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.887+0000 7fa07b7fe640 1 -- 172.21.15.83:0/2102994739 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c01be90 con 0x7fa084100080 2024-03-11T19:46:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.889+0000 7fa08f40d740 1 Processor -- start 2024-03-11T19:46:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.889+0000 7fa08f40d740 1 -- start start 2024-03-11T19:46:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.889+0000 7fa07b7fe640 1 -- 172.21.15.83:0/2102994739 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa07c0243b0 con 0x7fa084100080 2024-03-11T19:46:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b5266ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b526bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b526ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa08f40d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b521a960 con 0x5640b526c5e0 2024-03-11T19:46:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa08f40d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b51ce310 con 0x5640b5266a80 2024-03-11T19:46:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa08f40d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b51cb840 con 0x5640b5267470 2024-03-11T19:46:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa08a4fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b526bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa08a4fa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b526bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48462/0 (socket says 172.21.15.83:48462) 2024-03-11T19:46:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa08a4fa640 1 -- 172.21.15.83:0/2228577096 learned_addr learned my addr 172.21.15.83:0/2228577096 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa08acfb640 1 --2- 172.21.15.83:0/2228577096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b5266ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa089cf9640 1 --2- 172.21.15.83:0/2228577096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b526ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.890+0000 7fa07b7fe640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0700a1b80 0x7fa0700a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.891+0000 7fa07b7fe640 1 -- 172.21.15.83:0/2102994739 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa07c0cd250 con 0x7fa084100080 2024-03-11T19:46:43.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.891+0000 7fa08a4fa640 1 -- 172.21.15.83:0/2228577096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 msgr2=0x5640b5266ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.891+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/2228577096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b5266ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.891+0000 7fa08a4fa640 1 -- 172.21.15.83:0/2228577096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 msgr2=0x5640b526ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.891+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/2228577096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b526ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.891+0000 7fa08a4fa640 1 -- 172.21.15.83:0/2228577096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640b5189060 con 0x5640b5267470 2024-03-11T19:46:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.892+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/2228577096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b526bf70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0cae00 tx=0x7fa07c01b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2462bf040c4aebc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.892+0000 7fa08acfb640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0700a1b80 0x7fa0700a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.892+0000 7fa078ff9640 1 -- 172.21.15.83:0/2228577096 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c09e270 con 0x5640b5267470 2024-03-11T19:46:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.892+0000 7fa078ff9640 1 -- 172.21.15.83:0/2228577096 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa07c09e090 con 0x5640b5267470 2024-03-11T19:46:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.892+0000 7fa078ff9640 1 -- 172.21.15.83:0/2228577096 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c09ada0 con 0x5640b5267470 2024-03-11T19:46:43.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.893+0000 7fa08f40d740 1 -- 172.21.15.83:0/2228577096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 msgr2=0x5640b526bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.893+0000 7fa08acfb640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0700a1b80 0x7fa0700a4040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5640b5267cd0 tx=0x7fa0800046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.893+0000 7fa08f40d740 1 --2- 172.21.15.83:0/2228577096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b526bf70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0cae00 tx=0x7fa07c01b8b0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.893+0000 7fa08f40d740 1 -- 172.21.15.83:0/2228577096 shutdown_connections 2024-03-11T19:46:43.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.893+0000 7fa08f40d740 1 --2- 172.21.15.83:0/2228577096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b526ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.893+0000 7fa08f40d740 1 --2- 172.21.15.83:0/2228577096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b526bf70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.893+0000 7fa08f40d740 1 --2- 172.21.15.83:0/2228577096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b5266ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.893+0000 7fa08f40d740 1 -- 172.21.15.83:0/2228577096 >> 172.21.15.83:0/2228577096 conn(0x5640b5261ff0 msgr2=0x5640b52623f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.893+0000 7fa08f40d740 1 -- 172.21.15.83:0/2228577096 shutdown_connections 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.893+0000 7fa08f40d740 1 -- 172.21.15.83:0/2228577096 wait complete. 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08f40d740 1 -- start start 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08f40d740 1 Processor -- start 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b5270ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b52711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5278220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08f40d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b51c7020 con 0x5640b526c5e0 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08f40d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b4e9a430 con 0x5640b5266a80 2024-03-11T19:46:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08f40d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b4e56e00 con 0x5640b5267470 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08f40d740 10 client.0 _ll_register_callbacks cb 0x5640b516ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08acfb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5278220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08acfb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5278220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52378/0 (socket says 172.21.15.83:52378) 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08acfb640 1 -- 172.21.15.83:0/3716049008 learned_addr learned my addr 172.21.15.83:0/3716049008 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa089cf9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b52711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b5270ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08acfb640 1 -- 172.21.15.83:0/3716049008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 msgr2=0x5640b52711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08acfb640 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b52711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08acfb640 1 -- 172.21.15.83:0/3716049008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 msgr2=0x5640b5270ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08acfb640 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b5270ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.894+0000 7fa08acfb640 1 -- 172.21.15.83:0/3716049008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0841962d0 con 0x5640b526c5e0 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa08acfb640 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5278220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa080003870 tx=0x7fa080004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d3788b5bd0a240 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa06dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:46:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa06dffb640 1 -- 172.21.15.83:0/3716049008 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080024070 con 0x5640b526c5e0 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa08f40d740 1 -- 172.21.15.83:0/3716049008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640b4ec67e0 con 0x5640b526c5e0 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa06dffb640 1 -- 172.21.15.83:0/3716049008 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa08001f0d0 con 0x5640b526c5e0 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa06dffb640 1 -- 172.21.15.83:0/3716049008 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa080009e60 con 0x5640b526c5e0 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa08f40d740 10 client.16737 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa08f40d740 1 -- 172.21.15.83:0/3716049008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5640b51c56a0 con 0x5640b526c5e0 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa06dffb640 1 -- 172.21.15.83:0/3716049008 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5640b4ec67e0 con 0x5640b526c5e0 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa06dffb640 1 -- 172.21.15.83:0/3716049008 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa08001da00 con 0x5640b526c5e0 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa08f40d740 10 client.16737 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa08f40d740 10 client.16737 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa08f40d740 20 client.16737 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa08f40d740 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5640b527cf00 0x5640b527f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa08f40d740 4 client.16737 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.895+0000 7fa08f40d740 1 -- 172.21.15.83:0/3716049008 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640b4f03720 con 0x5640b527cf00 2024-03-11T19:46:43.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.896+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5640b527cf00 0x5640b527f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.896+0000 7fa06dffb640 10 client.16737 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:46:43.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.896+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5640b527cf00 0x5640b527f320 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11e50a826096253b server_cookie=17dbd8486539f895 in_seq=0 out_seq=0 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa06dffb640 1 -- 172.21.15.83:0/3716049008 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5640b4f03720 con 0x5640b527cf00 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa06dffb640 10 client.16737 handle_command_reply: tid=0 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa08f40d740 1 client.16737 shutdown 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa08f40d740 1 -- 172.21.15.83:0/3716049008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 msgr2=0x5640b5278220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa08f40d740 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5278220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa080003870 tx=0x7fa080004730 comp rx=0 tx=0).stop 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa08f40d740 1 -- 172.21.15.83:0/3716049008 shutdown_connections 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa08f40d740 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5640b527cf00 0x5640b527f320 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa08f40d740 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5278220 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa08f40d740 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b52711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa08f40d740 1 --2- 172.21.15.83:0/3716049008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b5270ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa08f40d740 1 -- 172.21.15.83:0/3716049008 >> 172.21.15.83:0/3716049008 conn(0x5640b5261ff0 msgr2=0x5640b5263240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa06dffb640 0 client.16737 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:46:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.897+0000 7fa08f40d740 1 -- 172.21.15.83:0/3716049008 shutdown_connections 2024-03-11T19:46:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.898+0000 7fa08f40d740 1 -- 172.21.15.83:0/3716049008 wait complete. 2024-03-11T19:46:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.898+0000 7fa08f40d740 20 client.16737 trim_cache size 0 max 16384 2024-03-11T19:46:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08f40d740 1 Processor -- start 2024-03-11T19:46:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08f40d740 1 -- start start 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b5266a80 0x5640b52623d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b52781e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b526c5e0 0x5640b5270c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08f40d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b514c130 con 0x5640b5266a80 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08f40d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b5270ae0 con 0x5640b526c5e0 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08f40d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b4f03720 con 0x5640b5267470 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08a4fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b5266a80 0x5640b52623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08a4fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b5266a80 0x5640b52623d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52380/0 (socket says 172.21.15.83:52380) 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08a4fa640 1 -- 172.21.15.83:0/3904670789 learned_addr learned my addr 172.21.15.83:0/3904670789 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa089cf9640 1 --2- 172.21.15.83:0/3904670789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b52781e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.913+0000 7fa08acfb640 1 --2- 172.21.15.83:0/3904670789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b526c5e0 0x5640b5270c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.914+0000 7fa08a4fa640 1 -- 172.21.15.83:0/3904670789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 msgr2=0x5640b52781e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.914+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/3904670789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b52781e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.914+0000 7fa08a4fa640 1 -- 172.21.15.83:0/3904670789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b526c5e0 msgr2=0x5640b5270c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.914+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/3904670789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b526c5e0 0x5640b5270c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.914+0000 7fa08a4fa640 1 -- 172.21.15.83:0/3904670789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640b5161c20 con 0x5640b5266a80 2024-03-11T19:46:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.914+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/3904670789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b5266a80 0x5640b52623d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa07c09b100 tx=0x7fa07c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f3f12df8f19ac9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.914+0000 7fa06ffff640 1 -- 172.21.15.83:0/3904670789 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0a6e20 con 0x5640b5266a80 2024-03-11T19:46:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.914+0000 7fa06ffff640 1 -- 172.21.15.83:0/3904670789 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa07c0d0e20 con 0x5640b5266a80 2024-03-11T19:46:43.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.914+0000 7fa06ffff640 1 -- 172.21.15.83:0/3904670789 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0a6990 con 0x5640b5266a80 2024-03-11T19:46:43.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.915+0000 7fa08f40d740 1 -- 172.21.15.83:0/3904670789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b5266a80 msgr2=0x5640b52623d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.915+0000 7fa08f40d740 1 --2- 172.21.15.83:0/3904670789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b5266a80 0x5640b52623d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa07c09b100 tx=0x7fa07c015040 comp rx=0 tx=0).stop 2024-03-11T19:46:43.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.915+0000 7fa08f40d740 1 -- 172.21.15.83:0/3904670789 shutdown_connections 2024-03-11T19:46:43.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.915+0000 7fa08f40d740 1 --2- 172.21.15.83:0/3904670789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b526c5e0 0x5640b5270c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.915+0000 7fa08f40d740 1 --2- 172.21.15.83:0/3904670789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b52781e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.915+0000 7fa08f40d740 1 --2- 172.21.15.83:0/3904670789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b5266a80 0x5640b52623d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.915+0000 7fa08f40d740 1 -- 172.21.15.83:0/3904670789 >> 172.21.15.83:0/3904670789 conn(0x5640b5261ff0 msgr2=0x5640b5254a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.915+0000 7fa08f40d740 1 -- 172.21.15.83:0/3904670789 shutdown_connections 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.915+0000 7fa08f40d740 1 -- 172.21.15.83:0/3904670789 wait complete. 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08f40d740 1 -- start start 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08f40d740 1 Processor -- start 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b52760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b5276600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08f40d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5277610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08acfb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5277610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08acfb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5277610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52388/0 (socket says 172.21.15.83:52388) 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa089cf9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b5276600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08a4fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b52760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08acfb640 1 -- 172.21.15.83:0/545030870 learned_addr learned my addr 172.21.15.83:0/545030870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08f40d740 1 -- 172.21.15.83:0/545030870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b5277b50 con 0x5640b526c5e0 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08f40d740 1 -- 172.21.15.83:0/545030870 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b5277cc0 con 0x5640b5266a80 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08f40d740 1 -- 172.21.15.83:0/545030870 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640b5277e80 con 0x5640b5267470 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa089cf9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b5276600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48504/0 (socket says 172.21.15.83:48504) 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.916+0000 7fa08f40d740 10 client.0 _ll_register_callbacks cb 0x7fa07c0d0c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08acfb640 1 -- 172.21.15.83:0/545030870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 msgr2=0x5640b5276600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08acfb640 1 --2- 172.21.15.83:0/545030870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b5276600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08acfb640 1 -- 172.21.15.83:0/545030870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 msgr2=0x5640b52760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08acfb640 1 --2- 172.21.15.83:0/545030870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b52760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08acfb640 1 -- 172.21.15.83:0/545030870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640b5189060 con 0x5640b526c5e0 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08acfb640 1 --2- 172.21.15.83:0/545030870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5277610 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa080010c90 tx=0x7fa080009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634a278225a89fab server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa06dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa06dffb640 1 -- 172.21.15.83:0/545030870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080024040 con 0x5640b526c5e0 2024-03-11T19:46:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa06dffb640 1 -- 172.21.15.83:0/545030870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa08001fa30 con 0x5640b526c5e0 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08f40d740 1 -- 172.21.15.83:0/545030870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640b4f52890 con 0x5640b526c5e0 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa06dffb640 1 -- 172.21.15.83:0/545030870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa08001fbf0 con 0x5640b526c5e0 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08f40d740 10 client.16743 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08f40d740 1 -- 172.21.15.83:0/545030870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5640b51878c0 con 0x5640b526c5e0 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa06dffb640 1 -- 172.21.15.83:0/545030870 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5640b4f52890 con 0x5640b526c5e0 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa06dffb640 1 -- 172.21.15.83:0/545030870 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fa08003d570 con 0x5640b526c5e0 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08f40d740 10 client.16743 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08f40d740 10 client.16743 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.917+0000 7fa08f40d740 20 client.16743 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.918+0000 7fa08f40d740 1 --2- 172.21.15.83:0/545030870 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5640b52d4540 0x5640b52d6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.918+0000 7fa08f40d740 4 client.16743 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.918+0000 7fa08f40d740 1 -- 172.21.15.83:0/545030870 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5640b4e72700 con 0x5640b52d4540 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.918+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/545030870 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5640b52d4540 0x5640b52d6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.918+0000 7fa06dffb640 10 client.16743 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.918+0000 7fa08a4fa640 1 --2- 172.21.15.83:0/545030870 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5640b52d4540 0x5640b52d6960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df5adc7a0f19e082 server_cookie=bce180997e0087ce in_seq=0 out_seq=0 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.918+0000 7fa06dffb640 1 -- 172.21.15.83:0/545030870 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5640b4e72700 con 0x5640b52d4540 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.918+0000 7fa06dffb640 10 client.16743 handle_command_reply: tid=0 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.918+0000 7fa08f40d740 1 client.16743 shutdown 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.918+0000 7fa08f40d740 1 -- 172.21.15.83:0/545030870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 msgr2=0x5640b5277610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.918+0000 7fa08f40d740 1 --2- 172.21.15.83:0/545030870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5277610 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa080010c90 tx=0x7fa080009e60 comp rx=0 tx=0).stop 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08f40d740 1 -- 172.21.15.83:0/545030870 shutdown_connections 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08f40d740 1 --2- 172.21.15.83:0/545030870 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5640b52d4540 0x5640b52d6960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08f40d740 1 --2- 172.21.15.83:0/545030870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640b526c5e0 0x5640b5277610 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08f40d740 1 --2- 172.21.15.83:0/545030870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5640b5267470 0x5640b5276600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa06dffb640 0 client.16743 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08f40d740 1 --2- 172.21.15.83:0/545030870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640b5266a80 0x5640b52760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08f40d740 1 -- 172.21.15.83:0/545030870 >> 172.21.15.83:0/545030870 conn(0x5640b5261ff0 msgr2=0x5640b52623d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08f40d740 1 -- 172.21.15.83:0/545030870 shutdown_connections 2024-03-11T19:46:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08f40d740 1 -- 172.21.15.83:0/545030870 wait complete. 2024-03-11T19:46:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08f40d740 20 client.16743 trim_cache size 0 max 16384 2024-03-11T19:46:43.928 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:46:43.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08c75e640 1 -- 172.21.15.83:0/2102994739 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0700a1b80 msgr2=0x7fa0700a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08c75e640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0700a1b80 0x7fa0700a4040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5640b5267cd0 tx=0x7fa0800046c0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08c75e640 1 -- 172.21.15.83:0/2102994739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa084100080 msgr2=0x7fa084068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:43.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08c75e640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa084100080 0x7fa084068e50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00ea80 tx=0x7fa07c00ef40 comp rx=0 tx=0).stop 2024-03-11T19:46:43.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08c75e640 1 -- 172.21.15.83:0/2102994739 shutdown_connections 2024-03-11T19:46:43.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.919+0000 7fa08c75e640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0700a1b80 0x7fa0700a4040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.920+0000 7fa08c75e640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa084101c30 0x7fa08419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.920+0000 7fa08c75e640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa084101270 0x7fa08419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.920+0000 7fa08c75e640 1 --2- 172.21.15.83:0/2102994739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa084100080 0x7fa084068e50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:43.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.920+0000 7fa08c75e640 1 -- 172.21.15.83:0/2102994739 >> 172.21.15.83:0/2102994739 conn(0x7fa0840fb610 msgr2=0x7fa0841045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:43.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.920+0000 7fa08c75e640 1 -- 172.21.15.83:0/2102994739 shutdown_connections 2024-03-11T19:46:43.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:43.920+0000 7fa08c75e640 1 -- 172.21.15.83:0/2102994739 wait complete. 2024-03-11T19:46:43.937 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:46:43.937 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:46:44.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:43 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4265716724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:46:44.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.075+0000 7f179b365640 1 -- 172.21.15.83:0/859185653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17941012f0 msgr2=0x7f1794101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.075+0000 7f179b365640 1 --2- 172.21.15.83:0/859185653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17941012f0 0x7f1794101770 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1784009900 tx=0x7f178402bc30 comp rx=0 tx=0).stop 2024-03-11T19:46:44.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.075+0000 7f179b365640 1 -- 172.21.15.83:0/859185653 shutdown_connections 2024-03-11T19:46:44.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.076+0000 7f179b365640 1 --2- 172.21.15.83:0/859185653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1794101cb0 0x7f1794108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.076+0000 7f179b365640 1 --2- 172.21.15.83:0/859185653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17941012f0 0x7f1794101770 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.076+0000 7f179b365640 1 --2- 172.21.15.83:0/859185653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1794100100 0x7f1794100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.076+0000 7f179b365640 1 -- 172.21.15.83:0/859185653 >> 172.21.15.83:0/859185653 conn(0x7f17940fb6b0 msgr2=0x7f17940fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.076+0000 7f179b365640 1 -- 172.21.15.83:0/859185653 shutdown_connections 2024-03-11T19:46:44.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.076+0000 7f179b365640 1 -- 172.21.15.83:0/859185653 wait complete. 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f179b365640 1 Processor -- start 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f179b365640 1 -- start start 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f179b365640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1794100100 0x7f179406a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f179b365640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17941012f0 0x7f1794199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f179b365640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1794101cb0 0x7f179419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f179b365640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179419ab10 con 0x7f17941012f0 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f179b365640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179419ac80 con 0x7f1794101cb0 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f179b365640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179419ae40 con 0x7f1794100100 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f1798900640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17941012f0 0x7f1794199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f1799101640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1794100100 0x7f179406a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f1799101640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1794100100 0x7f179406a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48522/0 (socket says 172.21.15.83:48522) 2024-03-11T19:46:44.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f1798900640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17941012f0 0x7f1794199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52414/0 (socket says 172.21.15.83:52414) 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f1799101640 1 -- 172.21.15.83:0/3541639423 learned_addr learned my addr 172.21.15.83:0/3541639423 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.078+0000 7f1799902640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1794101cb0 0x7f179419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f1799101640 1 -- 172.21.15.83:0/3541639423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1794101cb0 msgr2=0x7f179419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f1799101640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1794101cb0 0x7f179419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f1799101640 1 -- 172.21.15.83:0/3541639423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17941012f0 msgr2=0x7f1794199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f1799101640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17941012f0 0x7f1794199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f1799101640 1 -- 172.21.15.83:0/3541639423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1784009560 con 0x7f1794100100 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f1799902640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1794101cb0 0x7f179419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f1798900640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17941012f0 0x7f1794199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f1799101640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1794100100 0x7f179406a2c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f179000e9c0 tx=0x7f179000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f178a7fc640 1 -- 172.21.15.83:0/3541639423 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17900096e0 con 0x7f1794100100 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f178a7fc640 1 -- 172.21.15.83:0/3541639423 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f179000cd80 con 0x7f1794100100 2024-03-11T19:46:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f179b365640 1 -- 172.21.15.83:0/3541639423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179419b120 con 0x7f1794100100 2024-03-11T19:46:44.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f178a7fc640 1 -- 172.21.15.83:0/3541639423 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1790010430 con 0x7f1794100100 2024-03-11T19:46:44.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.079+0000 7f179b365640 1 -- 172.21.15.83:0/3541639423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179419b5b0 con 0x7f1794100100 2024-03-11T19:46:44.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.082+0000 7f178a7fc640 1 -- 172.21.15.83:0/3541639423 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1790003680 con 0x7f1794100100 2024-03-11T19:46:44.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.083+0000 7f179e012740 1 Processor -- start 2024-03-11T19:46:44.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.083+0000 7f179e012740 1 -- start start 2024-03-11T19:46:44.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f179e012740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d09690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f179e012740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d09c60 0x55e322d0e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f179e012740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d0edd0 0x55e322d11260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f179e012740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322cc5960 con 0x55e322d09270 2024-03-11T19:46:44.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f179e012740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322c79310 con 0x55e322d0edd0 2024-03-11T19:46:44.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f179e012740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322c76840 con 0x55e322d09c60 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1799902640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d09690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1799902640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d09690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52418/0 (socket says 172.21.15.83:52418) 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1799101640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d09c60 0x55e322d0e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1799902640 1 -- 172.21.15.83:0/2459157259 learned_addr learned my addr 172.21.15.83:0/2459157259 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f178a7fc640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17800a1a30 0x7f17800a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1798900640 1 --2- 172.21.15.83:0/2459157259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d0edd0 0x55e322d11260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1799902640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17800a1a30 0x7f17800a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1799101640 1 -- 172.21.15.83:0/2459157259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d0edd0 msgr2=0x55e322d11260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1799101640 1 --2- 172.21.15.83:0/2459157259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d0edd0 0x55e322d11260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1799101640 1 -- 172.21.15.83:0/2459157259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 msgr2=0x55e322d09690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1799101640 1 --2- 172.21.15.83:0/2459157259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d09690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1799101640 1 -- 172.21.15.83:0/2459157259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e322c34060 con 0x55e322d09c60 2024-03-11T19:46:44.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.084+0000 7f1799902640 1 --2- 172.21.15.83:0/2459157259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d09690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:44.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.085+0000 7f178a7fc640 1 -- 172.21.15.83:0/3541639423 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1790014070 con 0x7f1794100100 2024-03-11T19:46:44.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.085+0000 7f1799101640 1 --2- 172.21.15.83:0/2459157259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d09c60 0x55e322d0e760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f17900cb260 tx=0x7f1790021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73bcf3532524d64 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.085+0000 7f1799902640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17800a1a30 0x7f17800a3ef0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55e322d0a4c0 tx=0x7f178c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.085+0000 7f177f7fe640 1 -- 172.21.15.83:0/2459157259 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179009a240 con 0x55e322d09c60 2024-03-11T19:46:44.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.085+0000 7f177f7fe640 1 -- 172.21.15.83:0/2459157259 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f179009a060 con 0x55e322d09c60 2024-03-11T19:46:44.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.085+0000 7f177f7fe640 1 -- 172.21.15.83:0/2459157259 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179009a700 con 0x55e322d09c60 2024-03-11T19:46:44.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 -- 172.21.15.83:0/2459157259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d09c60 msgr2=0x55e322d0e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 --2- 172.21.15.83:0/2459157259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d09c60 0x55e322d0e760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f17900cb260 tx=0x7f1790021000 comp rx=0 tx=0).stop 2024-03-11T19:46:44.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 -- 172.21.15.83:0/2459157259 shutdown_connections 2024-03-11T19:46:44.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 --2- 172.21.15.83:0/2459157259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d0edd0 0x55e322d11260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 --2- 172.21.15.83:0/2459157259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d09c60 0x55e322d0e760 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 --2- 172.21.15.83:0/2459157259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d09690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 -- 172.21.15.83:0/2459157259 >> 172.21.15.83:0/2459157259 conn(0x55e322d047e0 msgr2=0x55e322d04be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 -- 172.21.15.83:0/2459157259 shutdown_connections 2024-03-11T19:46:44.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 -- 172.21.15.83:0/2459157259 wait complete. 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 -- start start 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 Processor -- start 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09270 0x55e322d13390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.086+0000 7f179e012740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09c60 0x55e322d138d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f179e012740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d1a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f179e012740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322c72020 con 0x55e322d09c60 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f179e012740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322945430 con 0x55e322d09270 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f179e012740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322901e00 con 0x55e322d0edd0 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f179e012740 10 client.0 _ll_register_callbacks cb 0x55e322c19cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f1798900640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09c60 0x55e322d138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f1798900640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09c60 0x55e322d138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52420/0 (socket says 172.21.15.83:52420) 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f1798900640 1 -- 172.21.15.83:0/1558455780 learned_addr learned my addr 172.21.15.83:0/1558455780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f1799101640 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09270 0x55e322d13390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f1799902640 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d1a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f1798900640 1 -- 172.21.15.83:0/1558455780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 msgr2=0x55e322d1a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f1798900640 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d1a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f1798900640 1 -- 172.21.15.83:0/1558455780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09270 msgr2=0x55e322d13390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f1798900640 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09270 0x55e322d13390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f1798900640 1 -- 172.21.15.83:0/1558455780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179419b5b0 con 0x55e322d09c60 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f1798900640 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09c60 0x55e322d138d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f178402c170 tx=0x7f1784004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee0099211ab2a5c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f177cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f177cff9640 1 -- 172.21.15.83:0/1558455780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784004df0 con 0x55e322d09c60 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f179e012740 1 -- 172.21.15.83:0/1558455780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3229717e0 con 0x55e322d09c60 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.087+0000 7f177cff9640 1 -- 172.21.15.83:0/1558455780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1784004f50 con 0x55e322d09c60 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.088+0000 7f177cff9640 1 -- 172.21.15.83:0/1558455780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1784005110 con 0x55e322d09c60 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.088+0000 7f179e012740 10 client.16755 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:44.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.088+0000 7f179e012740 1 -- 172.21.15.83:0/1558455780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e322c706a0 con 0x55e322d09c60 2024-03-11T19:46:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.088+0000 7f177cff9640 1 -- 172.21.15.83:0/1558455780 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e3229717e0 con 0x55e322d09c60 2024-03-11T19:46:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.088+0000 7f177cff9640 1 -- 172.21.15.83:0/1558455780 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1784053070 con 0x55e322d09c60 2024-03-11T19:46:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.088+0000 7f179e012740 10 client.16755 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.088+0000 7f179e012740 10 client.16755 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:46:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.089+0000 7f179e012740 20 client.16755 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.089+0000 7f179e012740 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e322d1f550 0x55e322d21990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.089+0000 7f179e012740 4 client.16755 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.089+0000 7f179e012740 1 -- 172.21.15.83:0/1558455780 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3229ae720 con 0x55e322d1f550 2024-03-11T19:46:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.089+0000 7f1799101640 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e322d1f550 0x55e322d21990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.089+0000 7f177cff9640 10 client.16755 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:46:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.089+0000 7f1799101640 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e322d1f550 0x55e322d21990 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c03e947f9829b62 server_cookie=e2bd468565d02c28 in_seq=0 out_seq=0 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.090+0000 7f177cff9640 1 -- 172.21.15.83:0/1558455780 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e3229ae720 con 0x55e322d1f550 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.090+0000 7f177cff9640 10 client.16755 handle_command_reply: tid=0 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.090+0000 7f179e012740 1 client.16755 shutdown 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.090+0000 7f179e012740 1 -- 172.21.15.83:0/1558455780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09c60 msgr2=0x55e322d138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.090+0000 7f179e012740 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09c60 0x55e322d138d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f178402c170 tx=0x7f1784004270 comp rx=0 tx=0).stop 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.091+0000 7f179e012740 1 -- 172.21.15.83:0/1558455780 shutdown_connections 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.091+0000 7f179e012740 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e322d1f550 0x55e322d21990 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.091+0000 7f179e012740 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d1a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.091+0000 7f179e012740 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09c60 0x55e322d138d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.091+0000 7f179e012740 1 --2- 172.21.15.83:0/1558455780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09270 0x55e322d13390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.091+0000 7f179e012740 1 -- 172.21.15.83:0/1558455780 >> 172.21.15.83:0/1558455780 conn(0x55e322d047e0 msgr2=0x55e322d03b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.091+0000 7f177cff9640 0 client.16755 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:46:44.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.091+0000 7f179e012740 1 -- 172.21.15.83:0/1558455780 shutdown_connections 2024-03-11T19:46:44.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.091+0000 7f179e012740 1 -- 172.21.15.83:0/1558455780 wait complete. 2024-03-11T19:46:44.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.091+0000 7f179e012740 20 client.16755 trim_cache size 0 max 16384 2024-03-11T19:46:44.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f179e012740 1 Processor -- start 2024-03-11T19:46:44.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f179e012740 1 -- start start 2024-03-11T19:46:44.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f179e012740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322cffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f179e012740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09c60 0x55e322d1a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f179e012740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d0b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f179e012740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322bf7130 con 0x55e322d09270 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f179e012740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322d04650 con 0x55e322d09c60 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f179e012740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322d1be00 con 0x55e322d0edd0 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f1799101640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322cffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f1799101640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322cffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52434/0 (socket says 172.21.15.83:52434) 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f1799101640 1 -- 172.21.15.83:0/4189187917 learned_addr learned my addr 172.21.15.83:0/4189187917 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.108+0000 7f1799902640 1 --2- 172.21.15.83:0/4189187917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d0b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.109+0000 7f1798900640 1 --2- 172.21.15.83:0/4189187917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09c60 0x55e322d1a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.109+0000 7f1799101640 1 -- 172.21.15.83:0/4189187917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 msgr2=0x55e322d0b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.109+0000 7f1799101640 1 --2- 172.21.15.83:0/4189187917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d0b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.109+0000 7f1799101640 1 -- 172.21.15.83:0/4189187917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09c60 msgr2=0x55e322d1a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.109+0000 7f1799101640 1 --2- 172.21.15.83:0/4189187917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09c60 0x55e322d1a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.109+0000 7f1799101640 1 -- 172.21.15.83:0/4189187917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e322c0cc20 con 0x55e322d09270 2024-03-11T19:46:44.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.109+0000 7f1799101640 1 --2- 172.21.15.83:0/4189187917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322cffa10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f179000afd0 tx=0x7f179009af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981832efd911f0ee server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.109+0000 7f177effd640 1 -- 172.21.15.83:0/4189187917 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17900cd7e0 con 0x55e322d09270 2024-03-11T19:46:44.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.109+0000 7f177effd640 1 -- 172.21.15.83:0/4189187917 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f179009e8d0 con 0x55e322d09270 2024-03-11T19:46:44.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.109+0000 7f177effd640 1 -- 172.21.15.83:0/4189187917 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1790096d70 con 0x55e322d09270 2024-03-11T19:46:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.110+0000 7f179e012740 1 -- 172.21.15.83:0/4189187917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 msgr2=0x55e322cffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.110+0000 7f179e012740 1 --2- 172.21.15.83:0/4189187917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322cffa10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f179000afd0 tx=0x7f179009af70 comp rx=0 tx=0).stop 2024-03-11T19:46:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.110+0000 7f179e012740 1 -- 172.21.15.83:0/4189187917 shutdown_connections 2024-03-11T19:46:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.110+0000 7f179e012740 1 --2- 172.21.15.83:0/4189187917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d0b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.110+0000 7f179e012740 1 --2- 172.21.15.83:0/4189187917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09c60 0x55e322d1a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.110+0000 7f179e012740 1 --2- 172.21.15.83:0/4189187917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322cffa10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.110+0000 7f179e012740 1 -- 172.21.15.83:0/4189187917 >> 172.21.15.83:0/4189187917 conn(0x55e322d047e0 msgr2=0x55e322d04bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.110+0000 7f179e012740 1 -- 172.21.15.83:0/4189187917 shutdown_connections 2024-03-11T19:46:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.110+0000 7f179e012740 1 -- 172.21.15.83:0/4189187917 wait complete. 2024-03-11T19:46:44.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.110+0000 7f179e012740 1 -- start start 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.110+0000 7f179e012740 1 Processor -- start 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f179e012740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d1a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f179e012740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09c60 0x55e322d1a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f179e012740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d18b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f179e012740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322d19270 con 0x55e322d09270 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f179e012740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322d193e0 con 0x55e322d09c60 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f179e012740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e322d195a0 con 0x55e322d0edd0 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f179e012740 10 client.0 _ll_register_callbacks cb 0x7f17900d0ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f1799101640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d1a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f1799101640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d1a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52450/0 (socket says 172.21.15.83:52450) 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f1799101640 1 -- 172.21.15.83:0/2182388532 learned_addr learned my addr 172.21.15.83:0/2182388532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f1799902640 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d18b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f1798900640 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09c60 0x55e322d1a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.111+0000 7f1799101640 1 -- 172.21.15.83:0/2182388532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 msgr2=0x55e322d18b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.112+0000 7f1799101640 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d18b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.112+0000 7f1799101640 1 -- 172.21.15.83:0/2182388532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09c60 msgr2=0x55e322d1a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.112+0000 7f1799101640 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09c60 0x55e322d1a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.112+0000 7f1799101640 1 -- 172.21.15.83:0/2182388532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e322c34060 con 0x55e322d09270 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.112+0000 7f1799101640 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d1a470 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1790098c50 tx=0x7f1790005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234d9979e9f83121 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.112+0000 7f177cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.112+0000 7f177cff9640 1 -- 172.21.15.83:0/2182388532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17900a2ea0 con 0x55e322d09270 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.112+0000 7f177cff9640 1 -- 172.21.15.83:0/2182388532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f179009a700 con 0x55e322d09270 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.113+0000 7f179e012740 1 -- 172.21.15.83:0/2182388532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e322d000f0 con 0x55e322d09270 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.113+0000 7f177cff9640 1 -- 172.21.15.83:0/2182388532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17900ae390 con 0x55e322d09270 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.113+0000 7f179e012740 10 client.16761 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.113+0000 7f179e012740 1 -- 172.21.15.83:0/2182388532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e322c328c0 con 0x55e322d09270 2024-03-11T19:46:44.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.114+0000 7f177cff9640 1 -- 172.21.15.83:0/2182388532 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e322d000f0 con 0x55e322d09270 2024-03-11T19:46:44.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f177cff9640 1 -- 172.21.15.83:0/2182388532 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f179001f060 con 0x55e322d09270 2024-03-11T19:46:44.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f179e012740 10 client.16761 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:44.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f179e012740 10 client.16761 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:46:44.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f179e012740 20 client.16761 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:44.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f179e012740 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e322d76b20 0x55e322d78f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f179e012740 4 client.16761 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:46:44.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f179e012740 1 -- 172.21.15.83:0/2182388532 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f179009a060 con 0x55e322d76b20 2024-03-11T19:46:44.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f1798900640 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e322d76b20 0x55e322d78f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f1798900640 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e322d76b20 0x55e322d78f10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9fb36aaa79748424 server_cookie=662aab165de36766 in_seq=0 out_seq=0 2024-03-11T19:46:44.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f177cff9640 10 client.16761 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f177cff9640 1 -- 172.21.15.83:0/2182388532 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f179009a060 con 0x55e322d76b20 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f177cff9640 10 client.16761 handle_command_reply: tid=0 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.115+0000 7f179e012740 1 client.16761 shutdown 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f179e012740 1 -- 172.21.15.83:0/2182388532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 msgr2=0x55e322d1a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f179e012740 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d1a470 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1790098c50 tx=0x7f1790005ab0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f179e012740 1 -- 172.21.15.83:0/2182388532 shutdown_connections 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f179e012740 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e322d76b20 0x55e322d78f10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f179e012740 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e322d0edd0 0x55e322d18b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f179e012740 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e322d09c60 0x55e322d1a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f179e012740 1 --2- 172.21.15.83:0/2182388532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e322d09270 0x55e322d1a470 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f177cff9640 0 client.16761 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f179e012740 1 -- 172.21.15.83:0/2182388532 >> 172.21.15.83:0/2182388532 conn(0x55e322d047e0 msgr2=0x55e322d0b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f179e012740 1 -- 172.21.15.83:0/2182388532 shutdown_connections 2024-03-11T19:46:44.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f179e012740 1 -- 172.21.15.83:0/2182388532 wait complete. 2024-03-11T19:46:44.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.116+0000 7f179e012740 20 client.16761 trim_cache size 0 max 16384 2024-03-11T19:46:44.125 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:46:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 -- 172.21.15.83:0/3541639423 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17800a1a30 msgr2=0x7f17800a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17800a1a30 0x7f17800a3ef0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55e322d0a4c0 tx=0x7f178c004ab0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 -- 172.21.15.83:0/3541639423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1794100100 msgr2=0x7f179406a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1794100100 0x7f179406a2c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f179000e9c0 tx=0x7f179000ee80 comp rx=0 tx=0).stop 2024-03-11T19:46:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 -- 172.21.15.83:0/3541639423 shutdown_connections 2024-03-11T19:46:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f17800a1a30 0x7f17800a3ef0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1794101cb0 0x7f179419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17941012f0 0x7f1794199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 --2- 172.21.15.83:0/3541639423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1794100100 0x7f179406a2c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 -- 172.21.15.83:0/3541639423 >> 172.21.15.83:0/3541639423 conn(0x7f17940fb6b0 msgr2=0x7f1794104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 -- 172.21.15.83:0/3541639423 shutdown_connections 2024-03-11T19:46:44.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.117+0000 7f179b365640 1 -- 172.21.15.83:0/3541639423 wait complete. 2024-03-11T19:46:44.135 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:46:44.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:46:44.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:43 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4265716724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:46:44.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:43 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4265716724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:46:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.288+0000 7f1fb572e640 1 -- 172.21.15.83:0/3232151207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fb01023a0 msgr2=0x7f1fb0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.288+0000 7f1fb572e640 1 --2- 172.21.15.83:0/3232151207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fb01023a0 0x7f1fb0102820 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009860 tx=0x7f1f9802bb70 comp rx=0 tx=0).stop 2024-03-11T19:46:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.288+0000 7f1fb572e640 1 -- 172.21.15.83:0/3232151207 shutdown_connections 2024-03-11T19:46:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.288+0000 7f1fb572e640 1 --2- 172.21.15.83:0/3232151207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb0102d60 0x7f1fb01095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.288+0000 7f1fb572e640 1 --2- 172.21.15.83:0/3232151207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fb01023a0 0x7f1fb0102820 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.288+0000 7f1fb572e640 1 --2- 172.21.15.83:0/3232151207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fb01011b0 0x7f1fb01015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.288+0000 7f1fb572e640 1 -- 172.21.15.83:0/3232151207 >> 172.21.15.83:0/3232151207 conn(0x7f1fb00fc720 msgr2=0x7f1fb00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.288+0000 7f1fb572e640 1 -- 172.21.15.83:0/3232151207 shutdown_connections 2024-03-11T19:46:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.288+0000 7f1fb572e640 1 -- 172.21.15.83:0/3232151207 wait complete. 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.289+0000 7f1fb572e640 1 Processor -- start 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.289+0000 7f1fb572e640 1 -- start start 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.289+0000 7f1fb572e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fb01011b0 0x7f1fb0110360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.289+0000 7f1fb572e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fb01023a0 0x7f1fb01108a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.289+0000 7f1fb572e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb0102d60 0x7f1fb01ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.289+0000 7f1fb572e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb0117910 con 0x7f1fb0102d60 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.289+0000 7f1fb572e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb0117a80 con 0x7f1fb01023a0 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.289+0000 7f1fb572e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb01adf00 con 0x7f1fb01011b0 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.290+0000 7f1faf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb0102d60 0x7f1fb01ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.290+0000 7f1faf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb0102d60 0x7f1fb01ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52472/0 (socket says 172.21.15.83:52472) 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.290+0000 7f1faf7fe640 1 -- 172.21.15.83:0/1748525656 learned_addr learned my addr 172.21.15.83:0/1748525656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.290+0000 7f1faeffd640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fb01011b0 0x7f1fb0110360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.290+0000 7f1fae7fc640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fb01023a0 0x7f1fb01108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.290+0000 7f1faf7fe640 1 -- 172.21.15.83:0/1748525656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fb01011b0 msgr2=0x7f1fb0110360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.290+0000 7f1faf7fe640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fb01011b0 0x7f1fb0110360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.290+0000 7f1faf7fe640 1 -- 172.21.15.83:0/1748525656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fb01023a0 msgr2=0x7f1fb01108a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.290+0000 7f1faf7fe640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fb01023a0 0x7f1fb01108a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.290+0000 7f1faf7fe640 1 -- 172.21.15.83:0/1748525656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98009490 con 0x7f1fb0102d60 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.290+0000 7f1faeffd640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fb01011b0 0x7f1fb0110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.291+0000 7f1faf7fe640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb0102d60 0x7f1fb01ad9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009fd0 tx=0x7f1fa800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.291+0000 7f1fa7fff640 1 -- 172.21.15.83:0/1748525656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8009b50 con 0x7f1fb0102d60 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.291+0000 7f1fa7fff640 1 -- 172.21.15.83:0/1748525656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fa8003970 con 0x7f1fb0102d60 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.292+0000 7f1fa7fff640 1 -- 172.21.15.83:0/1748525656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8004860 con 0x7f1fb0102d60 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.292+0000 7f1fb572e640 1 -- 172.21.15.83:0/1748525656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb01ae190 con 0x7f1fb0102d60 2024-03-11T19:46:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.292+0000 7f1fb572e640 1 -- 172.21.15.83:0/1748525656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb01ae620 con 0x7f1fb0102d60 2024-03-11T19:46:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.295+0000 7f1fb83d6740 1 Processor -- start 2024-03-11T19:46:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.295+0000 7f1fa7fff640 1 -- 172.21.15.83:0/1748525656 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1fa800a040 con 0x7f1fb0102d60 2024-03-11T19:46:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.295+0000 7f1fb83d6740 1 -- start start 2024-03-11T19:46:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93270 0x5634f1f93690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93c60 0x5634f1f98760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1f9b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f1f4f960 con 0x5634f1f98dd0 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f1f03310 con 0x5634f1f93270 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f1f00840 con 0x5634f1f93c60 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fae7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93270 0x5634f1f93690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fae7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93270 0x5634f1f93690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52778/0 (socket says 172.21.15.83:52778) 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fae7fc640 1 -- 172.21.15.83:0/3729587709 learned_addr learned my addr 172.21.15.83:0/3729587709 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fae7fc640 1 --2- 172.21.15.83:0/3729587709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1f9b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fae7fc640 1 -- 172.21.15.83:0/3729587709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93c60 msgr2=0x5634f1f98760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fae7fc640 1 --2- 172.21.15.83:0/3729587709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93c60 0x5634f1f98760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fae7fc640 1 -- 172.21.15.83:0/3729587709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 msgr2=0x5634f1f9b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fae7fc640 1 --2- 172.21.15.83:0/3729587709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1f9b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.296+0000 7f1fae7fc640 1 -- 172.21.15.83:0/3729587709 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634f1ebe060 con 0x5634f1f93270 2024-03-11T19:46:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.297+0000 7f1fae7fc640 1 --2- 172.21.15.83:0/3729587709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93270 0x5634f1f93690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f98000c00 tx=0x7f1f98005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad99fd1a7b0a786c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.297+0000 7f1fa57fa640 1 -- 172.21.15.83:0/3729587709 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98004400 con 0x5634f1f93270 2024-03-11T19:46:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.297+0000 7f1fa57fa640 1 -- 172.21.15.83:0/3729587709 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f98034920 con 0x5634f1f93270 2024-03-11T19:46:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.297+0000 7f1fa57fa640 1 -- 172.21.15.83:0/3729587709 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98003850 con 0x5634f1f93270 2024-03-11T19:46:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.298+0000 7f1fa7fff640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1f9c0a1b80 0x7f1f9c0a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.298+0000 7f1fa7fff640 1 -- 172.21.15.83:0/1748525656 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1fa80cd110 con 0x7f1fb0102d60 2024-03-11T19:46:44.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.298+0000 7f1faeffd640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1f9c0a1b80 0x7f1f9c0a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.298+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3729587709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93270 msgr2=0x5634f1f93690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.298+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/3729587709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93270 0x5634f1f93690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f98000c00 tx=0x7f1f98005af0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3729587709 shutdown_connections 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/3729587709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1f9b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/3729587709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93c60 0x5634f1f98760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/3729587709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93270 0x5634f1f93690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3729587709 >> 172.21.15.83:0/3729587709 conn(0x5634f1f8e7e0 msgr2=0x5634f1f8ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1faeffd640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1f9c0a1b80 0x7f1f9c0a4040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x5634f1f94060 tx=0x7f1fa000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3729587709 shutdown_connections 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3729587709 wait complete. 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 -- start start 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 Processor -- start 2024-03-11T19:46:44.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f93270 0x5634f1f9d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1f9da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f98dd0 0x5634f1fa4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.299+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f1efc020 con 0x5634f1f93270 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f1bcf430 con 0x5634f1f93c60 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f1b8be00 con 0x5634f1f98dd0 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1faeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f93270 0x5634f1f9d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1fb83d6740 10 client.0 _ll_register_callbacks cb 0x5634f1ea3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1faf7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f98dd0 0x5634f1fa4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1faeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f93270 0x5634f1f9d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52486/0 (socket says 172.21.15.83:52486) 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1faeffd640 1 -- 172.21.15.83:0/1006607727 learned_addr learned my addr 172.21.15.83:0/1006607727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1fae7fc640 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1f9da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1faf7fe640 1 -- 172.21.15.83:0/1006607727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 msgr2=0x5634f1f9da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1faf7fe640 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1f9da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1faf7fe640 1 -- 172.21.15.83:0/1006607727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f93270 msgr2=0x5634f1f9d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1faf7fe640 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f93270 0x5634f1f9d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1faf7fe640 1 -- 172.21.15.83:0/1006607727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb0109ef0 con 0x5634f1f98dd0 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1fae7fc640 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1f9da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1faeffd640 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f93270 0x5634f1f9d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1faf7fe640 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f98dd0 0x5634f1fa4ac0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80cad10 tx=0x7f1fa809a970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd737a0b9ad8df1d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1f967fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.300+0000 7f1f967fc640 1 -- 172.21.15.83:0/1006607727 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8003ce0 con 0x5634f1f98dd0 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.301+0000 7f1f967fc640 1 -- 172.21.15.83:0/1006607727 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1fa80a2440 con 0x5634f1f98dd0 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.301+0000 7f1fb83d6740 1 -- 172.21.15.83:0/1006607727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634f1c38720 con 0x5634f1f98dd0 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.301+0000 7f1f967fc640 1 -- 172.21.15.83:0/1006607727 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fa80b8030 con 0x5634f1f98dd0 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.301+0000 7f1fb83d6740 10 client.26344 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:44.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.301+0000 7f1fb83d6740 1 -- 172.21.15.83:0/1006607727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5634f1efa6a0 con 0x5634f1f98dd0 2024-03-11T19:46:44.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.302+0000 7f1f967fc640 1 -- 172.21.15.83:0/1006607727 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fa80a6020 con 0x5634f1f98dd0 2024-03-11T19:46:44.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.302+0000 7f1f967fc640 1 -- 172.21.15.83:0/1006607727 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1fa8095e80 con 0x5634f1f98dd0 2024-03-11T19:46:44.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.302+0000 7f1fb83d6740 10 client.26344 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.302+0000 7f1fb83d6740 10 client.26344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.302+0000 7f1fb83d6740 20 client.26344 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.302+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5634f1fa97f0 0x5634f1fabc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.302+0000 7f1fb83d6740 4 client.26344 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.302+0000 7f1fb83d6740 1 -- 172.21.15.83:0/1006607727 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634f1c87890 con 0x5634f1fa97f0 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.302+0000 7f1faeffd640 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5634f1fa97f0 0x5634f1fabc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.303+0000 7f1f967fc640 10 client.26344 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.303+0000 7f1faeffd640 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5634f1fa97f0 0x5634f1fabc00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607c25db6b44f5d server_cookie=d4ba0f5833a623c7 in_seq=0 out_seq=0 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1f967fc640 1 -- 172.21.15.83:0/1006607727 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634f1c87890 con 0x5634f1fa97f0 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1f967fc640 10 client.26344 handle_command_reply: tid=0 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 1 client.26344 shutdown 2024-03-11T19:46:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 1 -- 172.21.15.83:0/1006607727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f98dd0 msgr2=0x5634f1fa4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f98dd0 0x5634f1fa4ac0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80cad10 tx=0x7f1fa809a970 comp rx=0 tx=0).stop 2024-03-11T19:46:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 1 -- 172.21.15.83:0/1006607727 shutdown_connections 2024-03-11T19:46:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5634f1fa97f0 0x5634f1fabc00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f98dd0 0x5634f1fa4ac0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1f967fc640 0 client.26344 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1f9da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/1006607727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f93270 0x5634f1f9d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 1 -- 172.21.15.83:0/1006607727 >> 172.21.15.83:0/1006607727 conn(0x5634f1f8e7e0 msgr2=0x5634f1f8db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 1 -- 172.21.15.83:0/1006607727 shutdown_connections 2024-03-11T19:46:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 1 -- 172.21.15.83:0/1006607727 wait complete. 2024-03-11T19:46:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.304+0000 7f1fb83d6740 20 client.26344 trim_cache size 0 max 16384 2024-03-11T19:46:44.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.320+0000 7f1fb83d6740 1 Processor -- start 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.320+0000 7f1fb83d6740 1 -- start start 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.320+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 0x5634f1f9d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.320+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1fa4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.320+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1f9da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.320+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f1e81130 con 0x5634f1f98dd0 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.320+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f1fa7d70 con 0x5634f1f93c60 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.320+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f1f8a5c0 con 0x5634f1f93270 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1faeffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 0x5634f1f9d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1faeffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 0x5634f1f9d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48632/0 (socket says 172.21.15.83:48632) 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1faeffd640 1 -- 172.21.15.83:0/2899560598 learned_addr learned my addr 172.21.15.83:0/2899560598 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1faf7fe640 1 --2- 172.21.15.83:0/2899560598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1f9da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1fae7fc640 1 --2- 172.21.15.83:0/2899560598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1fa4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1faeffd640 1 -- 172.21.15.83:0/2899560598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 msgr2=0x5634f1fa4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1faeffd640 1 --2- 172.21.15.83:0/2899560598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1fa4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1faeffd640 1 -- 172.21.15.83:0/2899560598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 msgr2=0x5634f1f9da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1faeffd640 1 --2- 172.21.15.83:0/2899560598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1f9da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1faeffd640 1 -- 172.21.15.83:0/2899560598 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634f1e96c20 con 0x5634f1f93270 2024-03-11T19:46:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1faf7fe640 1 --2- 172.21.15.83:0/2899560598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1f9da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1faeffd640 1 --2- 172.21.15.83:0/2899560598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 0x5634f1f9d500 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0011bd0 tx=0x7f1fa0013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2919ea9f248ddc6b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1fa4ff9640 1 -- 172.21.15.83:0/2899560598 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0026dc0 con 0x5634f1f93270 2024-03-11T19:46:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1fa4ff9640 1 -- 172.21.15.83:0/2899560598 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fa0025040 con 0x5634f1f93270 2024-03-11T19:46:44.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.321+0000 7f1fa4ff9640 1 -- 172.21.15.83:0/2899560598 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa001d3b0 con 0x5634f1f93270 2024-03-11T19:46:44.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.322+0000 7f1fb83d6740 1 -- 172.21.15.83:0/2899560598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 msgr2=0x5634f1f9d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.322+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/2899560598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 0x5634f1f9d500 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0011bd0 tx=0x7f1fa0013000 comp rx=0 tx=0).stop 2024-03-11T19:46:44.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.322+0000 7f1fb83d6740 1 -- 172.21.15.83:0/2899560598 shutdown_connections 2024-03-11T19:46:44.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.322+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/2899560598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1f9da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.322+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/2899560598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1fa4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.322+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/2899560598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 0x5634f1f9d500 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.322+0000 7f1fb83d6740 1 -- 172.21.15.83:0/2899560598 >> 172.21.15.83:0/2899560598 conn(0x5634f1f8e7e0 msgr2=0x5634f1fb05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.322+0000 7f1fb83d6740 1 -- 172.21.15.83:0/2899560598 shutdown_connections 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.322+0000 7f1fb83d6740 1 -- 172.21.15.83:0/2899560598 wait complete. 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1fb83d6740 1 -- start start 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1fb83d6740 1 Processor -- start 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 0x5634f1fa33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1fa38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1fb83d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1fa3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f1fffd10 con 0x5634f1f98dd0 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f1fffe50 con 0x5634f1f93c60 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1fb83d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634f2000010 con 0x5634f1f93270 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1fb83d6740 10 client.0 _ll_register_callbacks cb 0x7f1fa8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1faf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1fa3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1faf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1fa3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52500/0 (socket says 172.21.15.83:52500) 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1faeffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 0x5634f1fa33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1faf7fe640 1 -- 172.21.15.83:0/3310847241 learned_addr learned my addr 172.21.15.83:0/3310847241 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1fae7fc640 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1fa38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1faf7fe640 1 -- 172.21.15.83:0/3310847241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 msgr2=0x5634f1fa33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1faf7fe640 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 0x5634f1fa33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1faf7fe640 1 -- 172.21.15.83:0/3310847241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 msgr2=0x5634f1fa38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1faf7fe640 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1fa38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.323+0000 7f1faf7fe640 1 -- 172.21.15.83:0/3310847241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634f1efa6a0 con 0x5634f1f98dd0 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.324+0000 7f1faf7fe640 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1fa3e30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8099e30 tx=0x7f1fa80041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13336274d83caa2b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.324+0000 7f1f967fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.324+0000 7f1f967fc640 1 -- 172.21.15.83:0/3310847241 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80b8040 con 0x5634f1f98dd0 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.325+0000 7f1f967fc640 1 -- 172.21.15.83:0/3310847241 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1fa80a7080 con 0x5634f1f98dd0 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.325+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3310847241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634f1bfb7e0 con 0x5634f1f98dd0 2024-03-11T19:46:44.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.325+0000 7f1f967fc640 1 -- 172.21.15.83:0/3310847241 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fa8099c70 con 0x5634f1f98dd0 2024-03-11T19:46:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.325+0000 7f1fb83d6740 10 client.16779 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.325+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3310847241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f1fa0011900 con 0x5634f1f98dd0 2024-03-11T19:46:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.326+0000 7f1f967fc640 1 -- 172.21.15.83:0/3310847241 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5634f1bfb7e0 con 0x5634f1f98dd0 2024-03-11T19:46:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.326+0000 7f1f967fc640 1 -- 172.21.15.83:0/3310847241 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1fa807b030 con 0x5634f1f98dd0 2024-03-11T19:46:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.326+0000 7f1fb83d6740 10 client.16779 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.326+0000 7f1fb83d6740 10 client.16779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.326+0000 7f1fb83d6740 20 client.16779 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.326+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5634f2001680 0x5634f2003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.326+0000 7f1fb83d6740 4 client.16779 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:46:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.326+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3310847241 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5634f1ba7700 con 0x5634f2001680 2024-03-11T19:46:44.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.326+0000 7f1faeffd640 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5634f2001680 0x5634f2003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1f967fc640 10 client.16779 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1faeffd640 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5634f2001680 0x5634f2003aa0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adfd5ff173590ff8 server_cookie=7fb29c55cfa2fe17 in_seq=0 out_seq=0 2024-03-11T19:46:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1f967fc640 1 -- 172.21.15.83:0/3310847241 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5634f1ba7700 con 0x5634f2001680 2024-03-11T19:46:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1f967fc640 10 client.16779 handle_command_reply: tid=0 2024-03-11T19:46:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 1 client.16779 shutdown 2024-03-11T19:46:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3310847241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 msgr2=0x5634f1fa3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1fa3e30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8099e30 tx=0x7f1fa80041a0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3310847241 shutdown_connections 2024-03-11T19:46:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5634f2001680 0x5634f2003aa0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634f1f98dd0 0x5634f1fa3e30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634f1f93c60 0x5634f1fa38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 1 --2- 172.21.15.83:0/3310847241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5634f1f93270 0x5634f1fa33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3310847241 >> 172.21.15.83:0/3310847241 conn(0x5634f1f8e7e0 msgr2=0x5634f1f95260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1f967fc640 0 client.16779 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:44.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3310847241 shutdown_connections 2024-03-11T19:46:44.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 1 -- 172.21.15.83:0/3310847241 wait complete. 2024-03-11T19:46:44.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.327+0000 7f1fb83d6740 20 client.16779 trim_cache size 0 max 16384 2024-03-11T19:46:44.336 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:46:44.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 -- 172.21.15.83:0/1748525656 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1f9c0a1b80 msgr2=0x7f1f9c0a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1f9c0a1b80 0x7f1f9c0a4040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x5634f1f94060 tx=0x7f1fa000b410 comp rx=0 tx=0).stop 2024-03-11T19:46:44.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 -- 172.21.15.83:0/1748525656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb0102d60 msgr2=0x7f1fb01ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:44.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb0102d60 0x7f1fb01ad9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009fd0 tx=0x7f1fa800c800 comp rx=0 tx=0).stop 2024-03-11T19:46:44.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 -- 172.21.15.83:0/1748525656 shutdown_connections 2024-03-11T19:46:44.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1f9c0a1b80 0x7f1f9c0a4040 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fb0102d60 0x7f1fb01ad9c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1fb01023a0 0x7f1fb01108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 --2- 172.21.15.83:0/1748525656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fb01011b0 0x7f1fb0110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:44.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 -- 172.21.15.83:0/1748525656 >> 172.21.15.83:0/1748525656 conn(0x7f1fb00fc720 msgr2=0x7f1fb01056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:44.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 -- 172.21.15.83:0/1748525656 shutdown_connections 2024-03-11T19:46:44.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:44.328+0000 7f1fb572e640 1 -- 172.21.15.83:0/1748525656 wait complete. 2024-03-11T19:46:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:44 smithi083 ceph-mon[31864]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.5 MiB/s wr, 775 op/s 2024-03-11T19:46:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:44 smithi137 ceph-mon[36220]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.5 MiB/s wr, 775 op/s 2024-03-11T19:46:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:44 smithi204 ceph-mon[36739]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.5 MiB/s wr, 775 op/s 2024-03-11T19:46:45.346 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:46:45.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:46:45.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:46:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.484+0000 7fdb99278640 1 -- 172.21.15.83:0/2895189954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb941045f0 msgr2=0x7fdb941069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.484+0000 7fdb99278640 1 --2- 172.21.15.83:0/2895189954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb941045f0 0x7fdb941069e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009900 tx=0x7fdb8802bc30 comp rx=0 tx=0).stop 2024-03-11T19:46:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.484+0000 7fdb99278640 1 -- 172.21.15.83:0/2895189954 shutdown_connections 2024-03-11T19:46:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.484+0000 7fdb99278640 1 --2- 172.21.15.83:0/2895189954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb94106f20 0x7fdb94109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.484+0000 7fdb99278640 1 --2- 172.21.15.83:0/2895189954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb941045f0 0x7fdb941069e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.484+0000 7fdb99278640 1 --2- 172.21.15.83:0/2895189954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb94101cc0 0x7fdb941040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.484+0000 7fdb99278640 1 -- 172.21.15.83:0/2895189954 >> 172.21.15.83:0/2895189954 conn(0x7fdb940fb610 msgr2=0x7fdb940fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.484+0000 7fdb99278640 1 -- 172.21.15.83:0/2895189954 shutdown_connections 2024-03-11T19:46:45.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.484+0000 7fdb99278640 1 -- 172.21.15.83:0/2895189954 wait complete. 2024-03-11T19:46:45.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.485+0000 7fdb99278640 1 Processor -- start 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.485+0000 7fdb99278640 1 -- start start 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.485+0000 7fdb99278640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb94101cc0 0x7fdb941015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.485+0000 7fdb99278640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb941045f0 0x7fdb940ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.485+0000 7fdb99278640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb94106f20 0x7fdb941001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.485+0000 7fdb99278640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb94100940 con 0x7fdb94101cc0 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.485+0000 7fdb99278640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb94100ab0 con 0x7fdb941045f0 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.485+0000 7fdb99278640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941a1360 con 0x7fdb94106f20 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.485+0000 7fdb92ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb94101cc0 0x7fdb941015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.485+0000 7fdb92ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb94101cc0 0x7fdb941015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52514/0 (socket says 172.21.15.83:52514) 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.485+0000 7fdb92ffd640 1 -- 172.21.15.83:0/3594702084 learned_addr learned my addr 172.21.15.83:0/3594702084 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb927fc640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb941045f0 0x7fdb940ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb937fe640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb94106f20 0x7fdb941001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb92ffd640 1 -- 172.21.15.83:0/3594702084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb94106f20 msgr2=0x7fdb941001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb92ffd640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb94106f20 0x7fdb941001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb92ffd640 1 -- 172.21.15.83:0/3594702084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb941045f0 msgr2=0x7fdb940ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb92ffd640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb941045f0 0x7fdb940ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb92ffd640 1 -- 172.21.15.83:0/3594702084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb88009560 con 0x7fdb94101cc0 2024-03-11T19:46:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb92ffd640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb94101cc0 0x7fdb941015e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400ea20 tx=0x7fdb8400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb7bfff640 1 -- 172.21.15.83:0/3594702084 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84009700 con 0x7fdb94101cc0 2024-03-11T19:46:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb7bfff640 1 -- 172.21.15.83:0/3594702084 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb8400cd80 con 0x7fdb94101cc0 2024-03-11T19:46:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb7bfff640 1 -- 172.21.15.83:0/3594702084 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84018840 con 0x7fdb94101cc0 2024-03-11T19:46:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb99278640 1 -- 172.21.15.83:0/3594702084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb941a15c0 con 0x7fdb94101cc0 2024-03-11T19:46:45.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.486+0000 7fdb99278640 1 -- 172.21.15.83:0/3594702084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb941a1aa0 con 0x7fdb94101cc0 2024-03-11T19:46:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.489+0000 7fdb9be45740 1 Processor -- start 2024-03-11T19:46:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb9be45740 1 -- start start 2024-03-11T19:46:45.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb7bfff640 1 -- 172.21.15.83:0/3594702084 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdb84010a80 con 0x7fdb94101cc0 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74180cd0 0x55dc741810f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74181ea0 0x55dc74182320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 0x55dc741890f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb9be45740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc740e7f90 con 0x55dc74180cd0 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb9be45740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc740e54c0 con 0x55dc74181ea0 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb9be45740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc740e0ca0 con 0x55dc74182860 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74180cd0 0x55dc741810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb927fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 0x55dc741890f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb92ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74181ea0 0x55dc74182320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb927fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 0x55dc741890f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48656/0 (socket says 172.21.15.83:48656) 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb927fc640 1 -- 172.21.15.83:0/2417810724 learned_addr learned my addr 172.21.15.83:0/2417810724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb927fc640 1 -- 172.21.15.83:0/2417810724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74181ea0 msgr2=0x55dc74182320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb927fc640 1 --2- 172.21.15.83:0/2417810724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74181ea0 0x55dc74182320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb927fc640 1 -- 172.21.15.83:0/2417810724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74180cd0 msgr2=0x55dc741810f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb927fc640 1 --2- 172.21.15.83:0/2417810724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74180cd0 0x55dc741810f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb927fc640 1 -- 172.21.15.83:0/2417810724 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc740a30b0 con 0x55dc74182860 2024-03-11T19:46:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.490+0000 7fdb937fe640 1 --2- 172.21.15.83:0/2417810724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74180cd0 0x55dc741810f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.491+0000 7fdb7bfff640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb7c0a1a30 0x7fdb7c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.491+0000 7fdb937fe640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb7c0a1a30 0x7fdb7c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.491+0000 7fdb927fc640 1 --2- 172.21.15.83:0/2417810724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 0x55dc741890f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb94068d80 tx=0x7fdb88005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb63ad128c64426f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.491+0000 7fdb7bfff640 1 -- 172.21.15.83:0/3594702084 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdb84014070 con 0x7fdb94101cc0 2024-03-11T19:46:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.491+0000 7fdb797fa640 1 -- 172.21.15.83:0/2417810724 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8802caa0 con 0x55dc74182860 2024-03-11T19:46:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.491+0000 7fdb797fa640 1 -- 172.21.15.83:0/2417810724 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb8802cc00 con 0x55dc74182860 2024-03-11T19:46:45.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.491+0000 7fdb797fa640 1 -- 172.21.15.83:0/2417810724 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88005250 con 0x55dc74182860 2024-03-11T19:46:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.492+0000 7fdb9be45740 1 -- 172.21.15.83:0/2417810724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 msgr2=0x55dc741890f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.492+0000 7fdb9be45740 1 --2- 172.21.15.83:0/2417810724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 0x55dc741890f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdb94068d80 tx=0x7fdb88005e00 comp rx=0 tx=0).stop 2024-03-11T19:46:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.492+0000 7fdb937fe640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb7c0a1a30 0x7fdb7c0a3ef0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55dc74181da0 tx=0x7fdb8c0096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.492+0000 7fdb9be45740 1 -- 172.21.15.83:0/2417810724 shutdown_connections 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.492+0000 7fdb9be45740 1 --2- 172.21.15.83:0/2417810724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 0x55dc741890f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.492+0000 7fdb9be45740 1 --2- 172.21.15.83:0/2417810724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74181ea0 0x55dc74182320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.492+0000 7fdb9be45740 1 --2- 172.21.15.83:0/2417810724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74180cd0 0x55dc741810f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.492+0000 7fdb9be45740 1 -- 172.21.15.83:0/2417810724 >> 172.21.15.83:0/2417810724 conn(0x55dc7417beb0 msgr2=0x55dc7417c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.492+0000 7fdb9be45740 1 -- 172.21.15.83:0/2417810724 shutdown_connections 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.492+0000 7fdb9be45740 1 -- 172.21.15.83:0/2417810724 wait complete. 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.492+0000 7fdb9be45740 1 -- start start 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb9be45740 1 Processor -- start 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 0x55dc7419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 0x55dc741a3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb9be45740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc74085f40 con 0x55dc74182860 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb9be45740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc73db44b0 con 0x55dc74180cd0 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb9be45740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc74066670 con 0x55dc74181ea0 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 0x55dc741a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb92ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb927fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 0x55dc7419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb927fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 0x55dc7419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48660/0 (socket says 172.21.15.83:48660) 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb927fc640 1 -- 172.21.15.83:0/1724360690 learned_addr learned my addr 172.21.15.83:0/1724360690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb9be45740 10 client.0 _ll_register_callbacks cb 0x55dc740d3f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb927fc640 1 -- 172.21.15.83:0/1724360690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 msgr2=0x55dc7419bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb927fc640 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb927fc640 1 -- 172.21.15.83:0/1724360690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 msgr2=0x55dc741a3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb927fc640 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 0x55dc741a3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb927fc640 1 -- 172.21.15.83:0/1724360690 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb94109ea0 con 0x55dc74181ea0 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb92ffd640 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb927fc640 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 0x55dc7419c4c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb88036040 tx=0x7fdb88004a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=943fc0cc7a18c439 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.493+0000 7fdb76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb76ff5640 1 -- 172.21.15.83:0/1724360690 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb880053a0 con 0x55dc74181ea0 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb76ff5640 1 -- 172.21.15.83:0/1724360690 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdb8804d070 con 0x55dc74181ea0 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb9be45740 1 -- 172.21.15.83:0/1724360690 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc740baf40 con 0x55dc74181ea0 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb76ff5640 1 -- 172.21.15.83:0/1724360690 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb88057030 con 0x55dc74181ea0 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb9be45740 10 client.26362 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb9be45740 1 -- 172.21.15.83:0/1724360690 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55dc740df320 con 0x55dc74181ea0 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb76ff5640 1 -- 172.21.15.83:0/1724360690 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dc740baf40 con 0x55dc74181ea0 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb76ff5640 1 -- 172.21.15.83:0/1724360690 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fdb8803c050 con 0x55dc74181ea0 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb9be45740 10 client.26362 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:45.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb9be45740 10 client.26362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:45.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb9be45740 20 client.26362 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:45.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.494+0000 7fdb9be45740 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dc741a7cb0 0x55dc741aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.495+0000 7fdb9be45740 4 client.26362 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:45.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.495+0000 7fdb9be45740 1 -- 172.21.15.83:0/1724360690 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc740d7310 con 0x55dc741a7cb0 2024-03-11T19:46:45.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.495+0000 7fdb92ffd640 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dc741a7cb0 0x55dc741aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.496+0000 7fdb76ff5640 10 client.26362 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:45.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.496+0000 7fdb92ffd640 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dc741a7cb0 0x55dc741aa0d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec39bd5954994cd1 server_cookie=b96fa4edbbae4233 in_seq=0 out_seq=0 2024-03-11T19:46:45.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb76ff5640 1 -- 172.21.15.83:0/1724360690 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc740d7310 con 0x55dc741a7cb0 2024-03-11T19:46:45.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb76ff5640 10 client.26362 handle_command_reply: tid=0 2024-03-11T19:46:45.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 1 client.26362 shutdown 2024-03-11T19:46:45.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 1 -- 172.21.15.83:0/1724360690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 msgr2=0x55dc7419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 0x55dc7419c4c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb88036040 tx=0x7fdb88004a10 comp rx=0 tx=0).stop 2024-03-11T19:46:45.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 1 -- 172.21.15.83:0/1724360690 shutdown_connections 2024-03-11T19:46:45.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dc741a7cb0 0x55dc741aa0d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 0x55dc741a3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 0x55dc7419c4c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 1 --2- 172.21.15.83:0/1724360690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb76ff5640 0 client.26362 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:45.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 1 -- 172.21.15.83:0/1724360690 >> 172.21.15.83:0/1724360690 conn(0x55dc7417beb0 msgr2=0x55dc741728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 1 -- 172.21.15.83:0/1724360690 shutdown_connections 2024-03-11T19:46:45.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 1 -- 172.21.15.83:0/1724360690 wait complete. 2024-03-11T19:46:45.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.497+0000 7fdb9be45740 20 client.26362 trim_cache size 0 max 16384 2024-03-11T19:46:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb9be45740 1 Processor -- start 2024-03-11T19:46:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb9be45740 1 -- start start 2024-03-11T19:46:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 0x55dc7419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 0x55dc741a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb9be45740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7417c9a0 con 0x55dc74182860 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb9be45740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7419b0b0 con 0x55dc74180cd0 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb9be45740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc740baf40 con 0x55dc74181ea0 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 0x55dc741a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb92ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 0x55dc741a33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52536/0 (socket says 172.21.15.83:52536) 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb927fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 0x55dc7419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb92ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52846/0 (socket says 172.21.15.83:52846) 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb937fe640 1 -- 172.21.15.83:0/3036155296 learned_addr learned my addr 172.21.15.83:0/3036155296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb937fe640 1 -- 172.21.15.83:0/3036155296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 msgr2=0x55dc7419c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb937fe640 1 --2- 172.21.15.83:0/3036155296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 0x55dc7419c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb937fe640 1 -- 172.21.15.83:0/3036155296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 msgr2=0x55dc7419bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb937fe640 1 --2- 172.21.15.83:0/3036155296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb937fe640 1 -- 172.21.15.83:0/3036155296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc7407bc00 con 0x55dc74182860 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb92ffd640 1 --2- 172.21.15.83:0/3036155296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.517+0000 7fdb937fe640 1 --2- 172.21.15.83:0/3036155296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 0x55dc741a33e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c020540 tx=0x7fdb8c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0692f1cd1863787 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb78ff9640 1 -- 172.21.15.83:0/3036155296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c025070 con 0x55dc74182860 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb78ff9640 1 -- 172.21.15.83:0/3036155296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb8c02a900 con 0x55dc74182860 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb78ff9640 1 -- 172.21.15.83:0/3036155296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c016070 con 0x55dc74182860 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb9be45740 1 -- 172.21.15.83:0/3036155296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 msgr2=0x55dc741a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb9be45740 1 --2- 172.21.15.83:0/3036155296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 0x55dc741a33e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c020540 tx=0x7fdb8c016000 comp rx=0 tx=0).stop 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb9be45740 1 -- 172.21.15.83:0/3036155296 shutdown_connections 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb9be45740 1 --2- 172.21.15.83:0/3036155296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74182860 0x55dc741a33e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb9be45740 1 --2- 172.21.15.83:0/3036155296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74181ea0 0x55dc7419c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb9be45740 1 --2- 172.21.15.83:0/3036155296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb9be45740 1 -- 172.21.15.83:0/3036155296 >> 172.21.15.83:0/3036155296 conn(0x55dc7417beb0 msgr2=0x55dc7417c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb9be45740 1 -- 172.21.15.83:0/3036155296 shutdown_connections 2024-03-11T19:46:45.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.518+0000 7fdb9be45740 1 -- 172.21.15.83:0/3036155296 wait complete. 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb9be45740 1 -- start start 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb9be45740 1 Processor -- start 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74181ea0 0x55dc741a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb9be45740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 0x55dc741fe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb9be45740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc741a3490 con 0x55dc74181ea0 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb9be45740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc741a3600 con 0x55dc74180cd0 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb9be45740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc741a37c0 con 0x55dc74182860 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb9be45740 10 client.0 _ll_register_callbacks cb 0x7fdb88009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb927fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74181ea0 0x55dc741a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb927fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74181ea0 0x55dc741a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52540/0 (socket says 172.21.15.83:52540) 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb927fc640 1 -- 172.21.15.83:0/717595944 learned_addr learned my addr 172.21.15.83:0/717595944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb92ffd640 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb937fe640 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 0x55dc741fe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb927fc640 1 -- 172.21.15.83:0/717595944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 msgr2=0x55dc741fe1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb927fc640 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 0x55dc741fe1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb927fc640 1 -- 172.21.15.83:0/717595944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 msgr2=0x55dc7419ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb927fc640 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb927fc640 1 -- 172.21.15.83:0/717595944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb941a1aa0 con 0x55dc74181ea0 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb927fc640 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74181ea0 0x55dc741a04e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009900 tx=0x7fdb88035ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3a23beff1a65f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.519+0000 7fdb76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb76ff5640 1 -- 172.21.15.83:0/717595944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88057070 con 0x55dc74181ea0 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb76ff5640 1 -- 172.21.15.83:0/717595944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdb88000e40 con 0x55dc74181ea0 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb9be45740 1 -- 172.21.15.83:0/717595944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc74106900 con 0x55dc74181ea0 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb76ff5640 1 -- 172.21.15.83:0/717595944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb88005d50 con 0x55dc74181ea0 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb9be45740 10 client.16797 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb9be45740 1 -- 172.21.15.83:0/717595944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fdb8c009b90 con 0x55dc74181ea0 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb76ff5640 1 -- 172.21.15.83:0/717595944 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dc74106900 con 0x55dc74181ea0 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb76ff5640 1 -- 172.21.15.83:0/717595944 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fdb8803c040 con 0x55dc74181ea0 2024-03-11T19:46:45.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb9be45740 10 client.16797 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb9be45740 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb9be45740 20 client.16797 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb9be45740 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dc741ffc80 0x55dc742020a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb9be45740 4 client.16797 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:46:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.520+0000 7fdb9be45740 1 -- 172.21.15.83:0/717595944 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dc73de0860 con 0x55dc741ffc80 2024-03-11T19:46:45.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.521+0000 7fdb92ffd640 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dc741ffc80 0x55dc742020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.522+0000 7fdb92ffd640 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dc741ffc80 0x55dc742020a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a7aa66fea9fc930 server_cookie=f6b81715050a3a47 in_seq=0 out_seq=0 2024-03-11T19:46:45.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.522+0000 7fdb76ff5640 10 client.16797 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:45.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb76ff5640 1 -- 172.21.15.83:0/717595944 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dc73de0860 con 0x55dc741ffc80 2024-03-11T19:46:45.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb76ff5640 10 client.16797 handle_command_reply: tid=0 2024-03-11T19:46:45.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 1 client.16797 shutdown 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 1 -- 172.21.15.83:0/717595944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74181ea0 msgr2=0x55dc741a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74181ea0 0x55dc741a04e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009900 tx=0x7fdb88035ca0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 1 -- 172.21.15.83:0/717595944 shutdown_connections 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dc741ffc80 0x55dc742020a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74182860 0x55dc741fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb76ff5640 0 client.16797 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc74181ea0 0x55dc741a04e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 1 --2- 172.21.15.83:0/717595944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc74180cd0 0x55dc7419ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 1 -- 172.21.15.83:0/717595944 >> 172.21.15.83:0/717595944 conn(0x55dc7417beb0 msgr2=0x55dc741850d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 1 -- 172.21.15.83:0/717595944 shutdown_connections 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 1 -- 172.21.15.83:0/717595944 wait complete. 2024-03-11T19:46:45.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.523+0000 7fdb9be45740 20 client.16797 trim_cache size 0 max 16384 2024-03-11T19:46:45.533 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:46:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:46:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "1a9c02c8-9e2b-4202-acfa-2daf9701b0b2", 2024-03-11T19:46:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:46:45.533 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 -- 172.21.15.83:0/3594702084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb7c0a1a30 msgr2=0x7fdb7c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb7c0a1a30 0x7fdb7c0a3ef0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55dc74181da0 tx=0x7fdb8c0096a0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 -- 172.21.15.83:0/3594702084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb94101cc0 msgr2=0x7fdb941015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb94101cc0 0x7fdb941015e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400ea20 tx=0x7fdb8400eee0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 -- 172.21.15.83:0/3594702084 shutdown_connections 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb7c0a1a30 0x7fdb7c0a3ef0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb94106f20 0x7fdb941001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb941045f0 0x7fdb940ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 --2- 172.21.15.83:0/3594702084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb94101cc0 0x7fdb941015e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 -- 172.21.15.83:0/3594702084 >> 172.21.15.83:0/3594702084 conn(0x7fdb940fb610 msgr2=0x7fdb94102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 -- 172.21.15.83:0/3594702084 shutdown_connections 2024-03-11T19:46:45.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.524+0000 7fdb99278640 1 -- 172.21.15.83:0/3594702084 wait complete. 2024-03-11T19:46:45.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:46:45.542 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:46:45.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.701+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2659138102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bec10adb0 msgr2=0x7f1bec10d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.701+0000 7f1bf3f65640 1 --2- 172.21.15.83:0/2659138102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bec10adb0 0x7f1bec10d1a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1be800af00 tx=0x7f1be802fe50 comp rx=0 tx=0).stop 2024-03-11T19:46:45.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.703+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2659138102 shutdown_connections 2024-03-11T19:46:45.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.703+0000 7f1bf3f65640 1 --2- 172.21.15.83:0/2659138102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec10d830 0x7f1bec10fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.703+0000 7f1bf3f65640 1 --2- 172.21.15.83:0/2659138102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bec10adb0 0x7f1bec10d1a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.703+0000 7f1bf3f65640 1 --2- 172.21.15.83:0/2659138102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bec1083f0 0x7f1bec10a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.703+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2659138102 >> 172.21.15.83:0/2659138102 conn(0x7f1bec06e360 msgr2=0x7f1bec0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.705+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2659138102 shutdown_connections 2024-03-11T19:46:45.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.705+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2659138102 wait complete. 2024-03-11T19:46:45.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.706+0000 7f1bf3f65640 1 Processor -- start 2024-03-11T19:46:45.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.707+0000 7f1bf3f65640 1 -- start start 2024-03-11T19:46:45.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.707+0000 7f1bf3f65640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bec1083f0 0x7f1bec19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.707+0000 7f1bf3f65640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bec10adb0 0x7f1bec19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.707+0000 7f1bf3f65640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec10d830 0x7f1bec1a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.708+0000 7f1bf3f65640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1a5370 con 0x7f1bec10adb0 2024-03-11T19:46:45.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.708+0000 7f1bf3f65640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1a54e0 con 0x7f1bec1083f0 2024-03-11T19:46:45.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.708+0000 7f1bf3f65640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1a56a0 con 0x7f1bec10d830 2024-03-11T19:46:45.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.708+0000 7f1bf2502640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec10d830 0x7f1bec1a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.708+0000 7f1bf2502640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec10d830 0x7f1bec1a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:48694/0 (socket says 172.21.15.83:48694) 2024-03-11T19:46:45.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.708+0000 7f1bf2502640 1 -- 172.21.15.83:0/2267706585 learned_addr learned my addr 172.21.15.83:0/2267706585 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:45.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.708+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bec1083f0 0x7f1bec19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.708+0000 7f1bf2502640 1 -- 172.21.15.83:0/2267706585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bec1083f0 msgr2=0x7f1bec19d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.708+0000 7f1bf2502640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bec1083f0 0x7f1bec19d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.709+0000 7f1bf2502640 1 -- 172.21.15.83:0/2267706585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bec10adb0 msgr2=0x7f1bec19ddc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:46:45.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.709+0000 7f1bf2502640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bec10adb0 0x7f1bec19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.709+0000 7f1bf2502640 1 -- 172.21.15.83:0/2267706585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be80081d0 con 0x7f1bec10d830 2024-03-11T19:46:45.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.709+0000 7f1bf2502640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec10d830 0x7f1bec1a4e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1be400b970 tx=0x7f1be400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.710+0000 7f1be2ffd640 1 -- 172.21.15.83:0/2267706585 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be400cb40 con 0x7f1bec10d830 2024-03-11T19:46:45.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.710+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2267706585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec1a5980 con 0x7f1bec10d830 2024-03-11T19:46:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.710+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2267706585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec1a5eb0 con 0x7f1bec10d830 2024-03-11T19:46:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.710+0000 7f1be2ffd640 1 -- 172.21.15.83:0/2267706585 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1be400cca0 con 0x7f1bec10d830 2024-03-11T19:46:45.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.710+0000 7f1be2ffd640 1 -- 172.21.15.83:0/2267706585 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be40077d0 con 0x7f1bec10d830 2024-03-11T19:46:45.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.715+0000 7f1be2ffd640 1 -- 172.21.15.83:0/2267706585 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1be4007a30 con 0x7f1bec10d830 2024-03-11T19:46:45.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.715+0000 7f1bf6c13740 1 Processor -- start 2024-03-11T19:46:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf6c13740 1 -- start start 2024-03-11T19:46:45.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf6c13740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f45f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf6c13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 0x556ff7f4afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf6c13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7f4dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf6c13740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7ead280 con 0x556ff7f464d0 2024-03-11T19:46:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf6c13740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7eaa7b0 con 0x556ff7f45ae0 2024-03-11T19:46:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf6c13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7ea5f90 con 0x556ff7f4b640 2024-03-11T19:46:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf1500640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f45f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf1500640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f45f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52900/0 (socket says 172.21.15.83:52900) 2024-03-11T19:46:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf1500640 1 -- 172.21.15.83:0/2272570833 learned_addr learned my addr 172.21.15.83:0/2272570833 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/2272570833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 0x556ff7f4afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.716+0000 7f1bf1500640 1 --2- 172.21.15.83:0/2272570833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7f4dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.717+0000 7f1bf1500640 1 -- 172.21.15.83:0/2272570833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 msgr2=0x556ff7f4dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.717+0000 7f1bf1500640 1 --2- 172.21.15.83:0/2272570833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7f4dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.717+0000 7f1bf1500640 1 -- 172.21.15.83:0/2272570833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 msgr2=0x556ff7f4afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.717+0000 7f1bf1500640 1 --2- 172.21.15.83:0/2272570833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 0x556ff7f4afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.717+0000 7f1bf1500640 1 -- 172.21.15.83:0/2272570833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ff7e64570 con 0x556ff7f45ae0 2024-03-11T19:46:45.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.717+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/2272570833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 0x556ff7f4afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:46:45.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.717+0000 7f1be2ffd640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bd80a1c30 0x7f1bd80a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.717+0000 7f1be2ffd640 1 -- 172.21.15.83:0/2267706585 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1be40ccb80 con 0x7f1bec10d830 2024-03-11T19:46:45.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.717+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bd80a1c30 0x7f1bd80a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.717+0000 7f1bf1500640 1 --2- 172.21.15.83:0/2272570833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f45f00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1be8005ab0 tx=0x7f1be8030370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603d7e33c95c8059 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.718+0000 7f1bd7fff640 1 -- 172.21.15.83:0/2272570833 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8038530 con 0x556ff7f45ae0 2024-03-11T19:46:45.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.718+0000 7f1bd7fff640 1 -- 172.21.15.83:0/2272570833 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1be8038690 con 0x556ff7f45ae0 2024-03-11T19:46:45.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.718+0000 7f1bd7fff640 1 -- 172.21.15.83:0/2272570833 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be800a330 con 0x556ff7f45ae0 2024-03-11T19:46:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.719+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2272570833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 msgr2=0x556ff7f45f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.719+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/2272570833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f45f00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1be8005ab0 tx=0x7f1be8030370 comp rx=0 tx=0).stop 2024-03-11T19:46:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.719+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bd80a1c30 0x7f1bd80a40f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x556ff7f468d0 tx=0x7f1bdc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.719+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2272570833 shutdown_connections 2024-03-11T19:46:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.719+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/2272570833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7f4dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.719+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/2272570833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 0x556ff7f4afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.719+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/2272570833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f45f00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.719+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2272570833 >> 172.21.15.83:0/2272570833 conn(0x556ff7f387f0 msgr2=0x556ff7f38bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.719+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2272570833 shutdown_connections 2024-03-11T19:46:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.720+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2272570833 wait complete. 2024-03-11T19:46:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.720+0000 7f1bf6c13740 1 -- start start 2024-03-11T19:46:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.721+0000 7f1bf6c13740 1 Processor -- start 2024-03-11T19:46:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.721+0000 7f1bf6c13740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f3e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.722+0000 7f1bf6c13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 0x556ff7f3ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.722+0000 7f1bf1d01640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f3e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.722+0000 7f1bf1d01640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f3e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52908/0 (socket says 172.21.15.83:52908) 2024-03-11T19:46:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.722+0000 7f1bf1500640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 0x556ff7f3ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.722+0000 7f1bf1500640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 0x556ff7f3ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52564/0 (socket says 172.21.15.83:52564) 2024-03-11T19:46:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.722+0000 7f1bf1d01640 1 -- 172.21.15.83:0/2460844940 learned_addr learned my addr 172.21.15.83:0/2460844940 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.722+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7f43ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.722+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2460844940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7b79430 con 0x556ff7f464d0 2024-03-11T19:46:45.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.722+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2460844940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7b35e00 con 0x556ff7f45ae0 2024-03-11T19:46:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.722+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2460844940 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7e2afd0 con 0x556ff7f4b640 2024-03-11T19:46:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.723+0000 7f1bf2502640 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7f43ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.723+0000 7f1bf6c13740 10 client.0 _ll_register_callbacks cb 0x556ff7e80fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.723+0000 7f1bf1d01640 1 -- 172.21.15.83:0/2460844940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 msgr2=0x556ff7f43ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.723+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7f43ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.723+0000 7f1bf1d01640 1 -- 172.21.15.83:0/2460844940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 msgr2=0x556ff7f3ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.723+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 0x556ff7f3ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.723+0000 7f1bf1d01640 1 -- 172.21.15.83:0/2460844940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec1108d0 con 0x556ff7f45ae0 2024-03-11T19:46:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.723+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f3e770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0167d0 tx=0x7f1bdc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=713418b0e1b37287 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.723+0000 7f1bd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:46:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.724+0000 7f1bd57fa640 1 -- 172.21.15.83:0/2460844940 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc014740 con 0x556ff7f45ae0 2024-03-11T19:46:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.724+0000 7f1bd57fa640 1 -- 172.21.15.83:0/2460844940 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1bdc0148a0 con 0x556ff7f45ae0 2024-03-11T19:46:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.724+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2460844940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ff7ba57e0 con 0x556ff7f45ae0 2024-03-11T19:46:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.724+0000 7f1bd57fa640 1 -- 172.21.15.83:0/2460844940 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bdc014a60 con 0x556ff7f45ae0 2024-03-11T19:46:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.725+0000 7f1bf6c13740 10 client.26267 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.725+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2460844940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556ff7ea4610 con 0x556ff7f45ae0 2024-03-11T19:46:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.726+0000 7f1bd57fa640 1 -- 172.21.15.83:0/2460844940 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1bdc014bc0 con 0x556ff7f45ae0 2024-03-11T19:46:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.726+0000 7f1bd57fa640 1 -- 172.21.15.83:0/2460844940 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1bdc02a460 con 0x556ff7f45ae0 2024-03-11T19:46:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.726+0000 7f1bf6c13740 10 client.26267 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.726+0000 7f1bf6c13740 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.726+0000 7f1bf6c13740 20 client.26267 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.726+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556ff7f642c0 0x556ff7f66780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.726+0000 7f1bf6c13740 4 client.26267 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:46:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.726+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2460844940 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ff7ba57e0 con 0x556ff7f642c0 2024-03-11T19:46:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.726+0000 7f1bf1500640 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556ff7f642c0 0x556ff7f66780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.727+0000 7f1bd57fa640 10 client.26267 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:45.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.727+0000 7f1bf1500640 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556ff7f642c0 0x556ff7f66780 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7216a5e491719ed2 server_cookie=7feefd22c69fe097 in_seq=0 out_seq=0 2024-03-11T19:46:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.727+0000 7f1bd57fa640 1 -- 172.21.15.83:0/2460844940 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ff7ba57e0 con 0x556ff7f642c0 2024-03-11T19:46:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.727+0000 7f1bd57fa640 10 client.26267 handle_command_reply: tid=0 2024-03-11T19:46:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.728+0000 7f1bf6c13740 1 client.26267 shutdown 2024-03-11T19:46:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.728+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2460844940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 msgr2=0x556ff7f3e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.728+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f3e770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc0167d0 tx=0x7f1bdc0046c0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.728+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2460844940 shutdown_connections 2024-03-11T19:46:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.728+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556ff7f642c0 0x556ff7f66780 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.728+0000 7f1bd57fa640 0 client.26267 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.728+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7f43ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.728+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f464d0 0x556ff7f3ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.728+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/2460844940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f45ae0 0x556ff7f3e770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.728+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2460844940 >> 172.21.15.83:0/2460844940 conn(0x556ff7f387f0 msgr2=0x556ff7f37a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.729+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2460844940 shutdown_connections 2024-03-11T19:46:45.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.730+0000 7f1bf6c13740 1 -- 172.21.15.83:0/2460844940 wait complete. 2024-03-11T19:46:45.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.730+0000 7f1bf6c13740 20 client.26267 trim_cache size 0 max 16384 2024-03-11T19:46:45.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf6c13740 1 Processor -- start 2024-03-11T19:46:45.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf6c13740 1 -- start start 2024-03-11T19:46:45.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf6c13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f45ae0 0x556ff7f6b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf6c13740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f464d0 0x556ff7f74660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf6c13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f4b640 0x556ff7f74ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf6c13740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7ba57e0 con 0x556ff7f4b640 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf6c13740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7f34320 con 0x556ff7f464d0 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf6c13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7f46900 con 0x556ff7f45ae0 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf2502640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f4b640 0x556ff7f74ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf2502640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f4b640 0x556ff7f74ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52568/0 (socket says 172.21.15.83:52568) 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf2502640 1 -- 172.21.15.83:0/391363222 learned_addr learned my addr 172.21.15.83:0/391363222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf1500640 1 --2- 172.21.15.83:0/391363222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f464d0 0x556ff7f74660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.750+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/391363222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f45ae0 0x556ff7f6b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.751+0000 7f1bf2502640 1 -- 172.21.15.83:0/391363222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f45ae0 msgr2=0x556ff7f6b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.751+0000 7f1bf2502640 1 --2- 172.21.15.83:0/391363222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f45ae0 0x556ff7f6b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.751+0000 7f1bf2502640 1 -- 172.21.15.83:0/391363222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f464d0 msgr2=0x556ff7f74660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.751+0000 7f1bf2502640 1 --2- 172.21.15.83:0/391363222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f464d0 0x556ff7f74660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.751+0000 7f1bf2502640 1 -- 172.21.15.83:0/391363222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ff7e40ac0 con 0x556ff7f4b640 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.751+0000 7f1bf2502640 1 --2- 172.21.15.83:0/391363222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f4b640 0x556ff7f74ba0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1be40ca690 tx=0x7f1be4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2886a32a9c6a96a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.751+0000 7f1bd77fe640 1 -- 172.21.15.83:0/391363222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be40a22a0 con 0x556ff7f4b640 2024-03-11T19:46:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.751+0000 7f1bd77fe640 1 -- 172.21.15.83:0/391363222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1be400ce10 con 0x556ff7f4b640 2024-03-11T19:46:45.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.751+0000 7f1bd77fe640 1 -- 172.21.15.83:0/391363222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be40a1e10 con 0x556ff7f4b640 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.752+0000 7f1bf6c13740 1 -- 172.21.15.83:0/391363222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f4b640 msgr2=0x556ff7f74ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.752+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/391363222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f4b640 0x556ff7f74ba0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1be40ca690 tx=0x7f1be4004a40 comp rx=0 tx=0).stop 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.752+0000 7f1bf6c13740 1 -- 172.21.15.83:0/391363222 shutdown_connections 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.753+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/391363222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f4b640 0x556ff7f74ba0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.753+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/391363222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f464d0 0x556ff7f74660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.753+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/391363222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f45ae0 0x556ff7f6b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.753+0000 7f1bf6c13740 1 -- 172.21.15.83:0/391363222 >> 172.21.15.83:0/391363222 conn(0x556ff7f387f0 msgr2=0x556ff7f33840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.753+0000 7f1bf6c13740 1 -- 172.21.15.83:0/391363222 shutdown_connections 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.753+0000 7f1bf6c13740 1 -- 172.21.15.83:0/391363222 wait complete. 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf6c13740 1 -- start start 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf6c13740 1 Processor -- start 2024-03-11T19:46:45.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf6c13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f45ae0 0x556ff7fb3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf6c13740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f464d0 0x556ff7fb3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf6c13740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7fba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf1d01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f45ae0 0x556ff7fb3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf6c13740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7fba9a0 con 0x556ff7f45ae0 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf6c13740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7fbaae0 con 0x556ff7f464d0 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf1d01640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f45ae0 0x556ff7fb3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52584/0 (socket says 172.21.15.83:52584) 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf6c13740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ff7fbac50 con 0x556ff7f4b640 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf1d01640 1 -- 172.21.15.83:0/1452922379 learned_addr learned my addr 172.21.15.83:0/1452922379 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf6c13740 10 client.0 _ll_register_callbacks cb 0x7f1be80083f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf1500640 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f464d0 0x556ff7fb3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.754+0000 7f1bf2502640 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7fba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.755+0000 7f1bf1d01640 1 -- 172.21.15.83:0/1452922379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 msgr2=0x556ff7fba460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.755+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7fba460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.755+0000 7f1bf1d01640 1 -- 172.21.15.83:0/1452922379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f464d0 msgr2=0x556ff7fb3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.755+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f464d0 0x556ff7fb3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.755+0000 7f1bf1d01640 1 -- 172.21.15.83:0/1452922379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec1a5eb0 con 0x556ff7f45ae0 2024-03-11T19:46:45.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.755+0000 7f1bf1d01640 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f45ae0 0x556ff7fb3540 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc02a8c0 tx=0x7f1bdc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d73fec31ee777f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:46:45.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.755+0000 7f1bd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:46:45.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.755+0000 7f1bd57fa640 1 -- 172.21.15.83:0/1452922379 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc014c90 con 0x556ff7f45ae0 2024-03-11T19:46:45.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.755+0000 7f1bd57fa640 1 -- 172.21.15.83:0/1452922379 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1bdc014df0 con 0x556ff7f45ae0 2024-03-11T19:46:45.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.755+0000 7f1bf6c13740 1 -- 172.21.15.83:0/1452922379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1be8038690 con 0x556ff7f45ae0 2024-03-11T19:46:45.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bd57fa640 1 -- 172.21.15.83:0/1452922379 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bdc035440 con 0x556ff7f45ae0 2024-03-11T19:46:45.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bf6c13740 10 client.16812 fetch_fsmap learned FSMap version 31 2024-03-11T19:46:45.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bf6c13740 1 -- 172.21.15.83:0/1452922379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556ff7e666a0 con 0x556ff7f45ae0 2024-03-11T19:46:45.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bd57fa640 1 -- 172.21.15.83:0/1452922379 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1be8038690 con 0x556ff7f45ae0 2024-03-11T19:46:45.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bd57fa640 1 -- 172.21.15.83:0/1452922379 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1bdc021700 con 0x556ff7f45ae0 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bf6c13740 10 client.16812 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bf6c13740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bf6c13740 20 client.16812 populate_metadata read hostname 'smithi083' 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556ff7fbc700 0x556ff7fbeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bf6c13740 4 client.16812 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bf6c13740 1 -- 172.21.15.83:0/1452922379 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556ff7be2720 con 0x556ff7fbc700 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.756+0000 7f1bf1500640 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556ff7fbc700 0x556ff7fbeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bd57fa640 10 client.16812 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bf1500640 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556ff7fbc700 0x556ff7fbeb20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c5aeb2538ae39e server_cookie=aa23191123fbad9f in_seq=0 out_seq=0 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bd57fa640 1 -- 172.21.15.83:0/1452922379 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556ff7be2720 con 0x556ff7fbc700 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bd57fa640 10 client.16812 handle_command_reply: tid=0 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bf6c13740 1 client.16812 shutdown 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bf6c13740 1 -- 172.21.15.83:0/1452922379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f45ae0 msgr2=0x556ff7fb3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f45ae0 0x556ff7fb3540 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc02a8c0 tx=0x7f1bdc02a680 comp rx=0 tx=0).stop 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bf6c13740 1 -- 172.21.15.83:0/1452922379 shutdown_connections 2024-03-11T19:46:45.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x556ff7fbc700 0x556ff7fbeb20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556ff7f4b640 0x556ff7fba460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ff7f464d0 0x556ff7fb3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bf6c13740 1 --2- 172.21.15.83:0/1452922379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ff7f45ae0 0x556ff7fb3540 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bd57fa640 0 client.16812 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bf6c13740 1 -- 172.21.15.83:0/1452922379 >> 172.21.15.83:0/1452922379 conn(0x556ff7f387f0 msgr2=0x556ff7f6b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.757+0000 7f1bf6c13740 1 -- 172.21.15.83:0/1452922379 shutdown_connections 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf6c13740 1 -- 172.21.15.83:0/1452922379 wait complete. 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf6c13740 20 client.16812 trim_cache size 0 max 16384 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:46:45.765 INFO:teuthology.orchestra.run.smithi083.stdout: "1a9c02c8-9e2b-4202-acfa-2daf9701b0b2": { 2024-03-11T19:46:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:46:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "1a9c02c8-9e2b-4202-acfa-2daf9701b0b2", 2024-03-11T19:46:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:46:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:46:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:46:45.766 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2267706585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bd80a1c30 msgr2=0x7f1bd80a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bd80a1c30 0x7f1bd80a40f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x556ff7f468d0 tx=0x7f1bdc008040 comp rx=0 tx=0).stop 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2267706585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec10d830 msgr2=0x7f1bec1a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec10d830 0x7f1bec1a4e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1be400b970 tx=0x7f1be400be30 comp rx=0 tx=0).stop 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2267706585 shutdown_connections 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1bd80a1c30 0x7f1bd80a40f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bec10d830 0x7f1bec1a4e30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bec10adb0 0x7f1bec19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 --2- 172.21.15.83:0/2267706585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1bec1083f0 0x7f1bec19d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2267706585 >> 172.21.15.83:0/2267706585 conn(0x7f1bec06e360 msgr2=0x7f1bec108d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2267706585 shutdown_connections 2024-03-11T19:46:45.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:46:45.758+0000 7f1bf3f65640 1 -- 172.21.15.83:0/2267706585 wait complete. 2024-03-11T19:46:45.776 INFO:tasks.cephfs.filesystem:scrub status for tag:1a9c02c8-9e2b-4202-acfa-2daf9701b0b2 - {'path': '/', 'tag': '1a9c02c8-9e2b-4202-acfa-2daf9701b0b2', 'options': 'recursive,force'} 2024-03-11T19:46:46.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:46 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:46:46.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:46 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:46.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:46 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:46:46.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:46 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:46.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:46 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:46:46.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:46 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:47.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:47 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:46:47.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:47 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:47.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:47 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:46:47.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:47 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:46:47.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:47 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:47.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:47 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:47.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:47 smithi137 ceph-mon[36220]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.6 MiB/s wr, 633 op/s 2024-03-11T19:46:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:47 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:46:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:47 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:47 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:46:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:47 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:46:47.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:47 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:47.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:47 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:47.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:47 smithi204 ceph-mon[36739]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.6 MiB/s wr, 633 op/s 2024-03-11T19:46:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 450.0 s, 2384.4 tps, lat 42.005 ms stddev 49.161 2024-03-11T19:46:47.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:47 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:46:47.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:47 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:47.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:47 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:46:47.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:47 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:46:47.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:47 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:47.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:47 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:46:47.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:47 smithi083 ceph-mon[31864]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.6 MiB/s wr, 633 op/s 2024-03-11T19:46:49.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:48 smithi083 ceph-mon[31864]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.7 MiB/s wr, 757 op/s 2024-03-11T19:46:49.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:48 smithi137 ceph-mon[36220]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.7 MiB/s wr, 757 op/s 2024-03-11T19:46:49.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:48 smithi204 ceph-mon[36739]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.7 MiB/s wr, 757 op/s 2024-03-11T19:46:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:50 smithi083 ceph-mon[31864]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-03-11T19:46:51.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:50 smithi137 ceph-mon[36220]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-03-11T19:46:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:50 smithi204 ceph-mon[36739]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-03-11T19:46:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 455.0 s, 3250.0 tps, lat 30.739 ms stddev 36.423 2024-03-11T19:46:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:52 smithi083 ceph-mon[31864]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.7 MiB/s wr, 650 op/s 2024-03-11T19:46:53.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:52 smithi137 ceph-mon[36220]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.7 MiB/s wr, 650 op/s 2024-03-11T19:46:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:52 smithi204 ceph-mon[36739]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.7 MiB/s wr, 650 op/s 2024-03-11T19:46:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:54 smithi083 ceph-mon[31864]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.3 MiB/s wr, 725 op/s 2024-03-11T19:46:55.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:54 smithi137 ceph-mon[36220]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.3 MiB/s wr, 725 op/s 2024-03-11T19:46:55.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:54 smithi204 ceph-mon[36739]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.3 MiB/s wr, 725 op/s 2024-03-11T19:46:56.041 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:55 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:46:56.057 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:55 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:46:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:55 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:46:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:56 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:46:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:56 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:46:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:56 smithi083 ceph-mon[31864]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 MiB/s wr, 643 op/s 2024-03-11T19:46:57.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:56 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:46:57.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:56 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:46:57.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:56 smithi137 ceph-mon[36220]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 MiB/s wr, 643 op/s 2024-03-11T19:46:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:56 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:46:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:56 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:46:57.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:56 smithi204 ceph-mon[36739]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 MiB/s wr, 643 op/s 2024-03-11T19:46:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 460.0 s, 2646.0 tps, lat 37.437 ms stddev 41.862 2024-03-11T19:46:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:46:58 smithi083 ceph-mon[31864]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 783 op/s 2024-03-11T19:46:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:46:58 smithi137 ceph-mon[36220]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 783 op/s 2024-03-11T19:46:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:46:58 smithi204 ceph-mon[36739]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 783 op/s 2024-03-11T19:47:01.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:00 smithi083 ceph-mon[31864]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 MiB/s wr, 656 op/s 2024-03-11T19:47:01.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:00 smithi137 ceph-mon[36220]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 MiB/s wr, 656 op/s 2024-03-11T19:47:01.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:00 smithi204 ceph-mon[36739]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 MiB/s wr, 656 op/s 2024-03-11T19:47:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 465.0 s, 2516.8 tps, lat 39.854 ms stddev 48.118 2024-03-11T19:47:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:02 smithi083 ceph-mon[31864]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 MiB/s wr, 714 op/s 2024-03-11T19:47:03.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:02 smithi137 ceph-mon[36220]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 MiB/s wr, 714 op/s 2024-03-11T19:47:03.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:02 smithi204 ceph-mon[36739]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 MiB/s wr, 714 op/s 2024-03-11T19:47:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:04 smithi083 ceph-mon[31864]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.5 MiB/s wr, 779 op/s 2024-03-11T19:47:05.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:04 smithi137 ceph-mon[36220]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.5 MiB/s wr, 779 op/s 2024-03-11T19:47:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:04 smithi204 ceph-mon[36739]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.5 MiB/s wr, 779 op/s 2024-03-11T19:47:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:06 smithi083 ceph-mon[31864]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.8 MiB/s wr, 696 op/s 2024-03-11T19:47:07.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:06 smithi137 ceph-mon[36220]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.8 MiB/s wr, 696 op/s 2024-03-11T19:47:07.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:06 smithi204 ceph-mon[36739]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.8 MiB/s wr, 696 op/s 2024-03-11T19:47:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 470.0 s, 1653.0 tps, lat 60.358 ms stddev 64.551 2024-03-11T19:47:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:08 smithi083 ceph-mon[31864]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 MiB/s wr, 789 op/s 2024-03-11T19:47:09.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:08 smithi137 ceph-mon[36220]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 MiB/s wr, 789 op/s 2024-03-11T19:47:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:08 smithi204 ceph-mon[36739]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 MiB/s wr, 789 op/s 2024-03-11T19:47:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:10 smithi083 ceph-mon[31864]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.9 MiB/s wr, 642 op/s 2024-03-11T19:47:11.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:10 smithi137 ceph-mon[36220]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.9 MiB/s wr, 642 op/s 2024-03-11T19:47:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:10 smithi204 ceph-mon[36739]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.9 MiB/s wr, 642 op/s 2024-03-11T19:47:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 475.0 s, 1767.4 tps, lat 56.705 ms stddev 65.782 2024-03-11T19:47:13.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:12 smithi083 ceph-mon[31864]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.4 MiB/s wr, 626 op/s 2024-03-11T19:47:13.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:12 smithi137 ceph-mon[36220]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.4 MiB/s wr, 626 op/s 2024-03-11T19:47:13.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:12 smithi204 ceph-mon[36739]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.4 MiB/s wr, 626 op/s 2024-03-11T19:47:15.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:14 smithi083 ceph-mon[31864]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-03-11T19:47:15.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:14 smithi137 ceph-mon[36220]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-03-11T19:47:15.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:14 smithi204 ceph-mon[36739]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.0 MiB/s wr, 697 op/s 2024-03-11T19:47:15.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:47:15.777 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:47:15.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.918+0000 7fde2798c640 1 -- 172.21.15.83:0/3330700611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde201089b0 msgr2=0x7fde20108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:15.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.918+0000 7fde2798c640 1 --2- 172.21.15.83:0/3330700611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde201089b0 0x7fde20108e30 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fde14009900 tx=0x7fde1402bc30 comp rx=0 tx=0).stop 2024-03-11T19:47:15.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.918+0000 7fde2798c640 1 -- 172.21.15.83:0/3330700611 shutdown_connections 2024-03-11T19:47:15.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.918+0000 7fde2798c640 1 --2- 172.21.15.83:0/3330700611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde20109370 0x7fde2010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.918+0000 7fde2798c640 1 --2- 172.21.15.83:0/3330700611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde201089b0 0x7fde20108e30 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.918+0000 7fde2798c640 1 --2- 172.21.15.83:0/3330700611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde201077c0 0x7fde20107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.918+0000 7fde2798c640 1 -- 172.21.15.83:0/3330700611 >> 172.21.15.83:0/3330700611 conn(0x7fde20076770 msgr2=0x7fde20078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:15.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde2798c640 1 -- 172.21.15.83:0/3330700611 shutdown_connections 2024-03-11T19:47:15.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde2798c640 1 -- 172.21.15.83:0/3330700611 wait complete. 2024-03-11T19:47:15.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde2798c640 1 Processor -- start 2024-03-11T19:47:15.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde2798c640 1 -- start start 2024-03-11T19:47:15.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde2798c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde201077c0 0x7fde2019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde2798c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde201089b0 0x7fde2019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde2798c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde20109370 0x7fde201a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde2798c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde201a5350 con 0x7fde201077c0 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde2798c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde201a54c0 con 0x7fde201089b0 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde2798c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde201a5680 con 0x7fde20109370 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde25728640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde201077c0 0x7fde2019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde25728640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde201077c0 0x7fde2019d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41266/0 (socket says 172.21.15.83:41266) 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde25728640 1 -- 172.21.15.83:0/2191747100 learned_addr learned my addr 172.21.15.83:0/2191747100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.919+0000 7fde24f27640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde201089b0 0x7fde2019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde25f29640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde20109370 0x7fde201a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde24f27640 1 -- 172.21.15.83:0/2191747100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde20109370 msgr2=0x7fde201a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde24f27640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde20109370 0x7fde201a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde24f27640 1 -- 172.21.15.83:0/2191747100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde201077c0 msgr2=0x7fde2019d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:15.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde24f27640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde201077c0 0x7fde2019d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde24f27640 1 -- 172.21.15.83:0/2191747100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde14009560 con 0x7fde201089b0 2024-03-11T19:47:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde25728640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde201077c0 0x7fde2019d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde24f27640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde201089b0 0x7fde2019dda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde1402c170 tx=0x7fde140040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde1e7fc640 1 -- 172.21.15.83:0/2191747100 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde140044c0 con 0x7fde201089b0 2024-03-11T19:47:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde2798c640 1 -- 172.21.15.83:0/2191747100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde201a5900 con 0x7fde201089b0 2024-03-11T19:47:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde2798c640 1 -- 172.21.15.83:0/2191747100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde201a5e30 con 0x7fde201089b0 2024-03-11T19:47:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde1e7fc640 1 -- 172.21.15.83:0/2191747100 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde140052d0 con 0x7fde201089b0 2024-03-11T19:47:15.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.920+0000 7fde1e7fc640 1 -- 172.21.15.83:0/2191747100 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14003e20 con 0x7fde201089b0 2024-03-11T19:47:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.923+0000 7fde1e7fc640 1 -- 172.21.15.83:0/2191747100 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fde1403e450 con 0x7fde201089b0 2024-03-11T19:47:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde2a564740 1 Processor -- start 2024-03-11T19:47:15.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde2a564740 1 -- start start 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde1e7fc640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fde080a1b30 0x7fde080a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde2a564740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 0x555c9cc4d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde2a564740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 0x555c9cc527c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde2a564740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc52e30 0x555c9cc552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde2a564740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9cbbd280 con 0x555c9cc4d2d0 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde2a564740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9cbba7b0 con 0x555c9cc52e30 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde2a564740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9cbb5f90 con 0x555c9cc4dcc0 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde1e7fc640 1 -- 172.21.15.83:0/2191747100 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fde140efb60 con 0x7fde201089b0 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde25728640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 0x555c9cc4d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde25f29640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fde080a1b30 0x7fde080a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde25728640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 0x555c9cc4d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41282/0 (socket says 172.21.15.83:41282) 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde25728640 1 -- 172.21.15.83:0/2570638403 learned_addr learned my addr 172.21.15.83:0/2570638403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:15.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde24f27640 1 --2- 172.21.15.83:0/2570638403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 0x555c9cc527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.924+0000 7fde25f29640 1 --2- 172.21.15.83:0/2570638403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc52e30 0x555c9cc552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.925+0000 7fde24f27640 1 -- 172.21.15.83:0/2570638403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc52e30 msgr2=0x555c9cc552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.925+0000 7fde24f27640 1 --2- 172.21.15.83:0/2570638403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc52e30 0x555c9cc552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.925+0000 7fde24f27640 1 -- 172.21.15.83:0/2570638403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 msgr2=0x555c9cc4d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.925+0000 7fde24f27640 1 --2- 172.21.15.83:0/2570638403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 0x555c9cc4d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.925+0000 7fde24f27640 1 -- 172.21.15.83:0/2570638403 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c9cb74570 con 0x555c9cc4dcc0 2024-03-11T19:47:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.925+0000 7fde25f29640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fde080a1b30 0x7fde080a3ff0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fde18005fd0 tx=0x7fde1800ba90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.925+0000 7fde24f27640 1 --2- 172.21.15.83:0/2570638403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 0x555c9cc527c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde140b94b0 tx=0x7fde140bc6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33186c2ac469fd24 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.925+0000 7fde077fe640 1 -- 172.21.15.83:0/2570638403 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14050e80 con 0x555c9cc4dcc0 2024-03-11T19:47:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.925+0000 7fde077fe640 1 -- 172.21.15.83:0/2570638403 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde1404d580 con 0x555c9cc4dcc0 2024-03-11T19:47:15.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.925+0000 7fde077fe640 1 -- 172.21.15.83:0/2570638403 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14005440 con 0x555c9cc4dcc0 2024-03-11T19:47:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.927+0000 7fde2a564740 1 -- 172.21.15.83:0/2570638403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 msgr2=0x555c9cc527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.927+0000 7fde2a564740 1 --2- 172.21.15.83:0/2570638403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 0x555c9cc527c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde140b94b0 tx=0x7fde140bc6a0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.928+0000 7fde2a564740 1 -- 172.21.15.83:0/2570638403 shutdown_connections 2024-03-11T19:47:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.928+0000 7fde2a564740 1 --2- 172.21.15.83:0/2570638403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc52e30 0x555c9cc552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.928+0000 7fde2a564740 1 --2- 172.21.15.83:0/2570638403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 0x555c9cc527c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.928+0000 7fde2a564740 1 --2- 172.21.15.83:0/2570638403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 0x555c9cc4d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.928+0000 7fde2a564740 1 -- 172.21.15.83:0/2570638403 >> 172.21.15.83:0/2570638403 conn(0x555c9cc487f0 msgr2=0x555c9cc48bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:15.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.928+0000 7fde2a564740 1 -- 172.21.15.83:0/2570638403 shutdown_connections 2024-03-11T19:47:15.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.928+0000 7fde2a564740 1 -- 172.21.15.83:0/2570638403 wait complete. 2024-03-11T19:47:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.928+0000 7fde2a564740 1 -- start start 2024-03-11T19:47:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde2a564740 1 Processor -- start 2024-03-11T19:47:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde2a564740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 0x555c9cc56f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde2a564740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde2a564740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 0x555c9cc5e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde2a564740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9c889430 con 0x555c9cc52e30 2024-03-11T19:47:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde2a564740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9c845e00 con 0x555c9cc4dcc0 2024-03-11T19:47:15.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde2a564740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9cb3afd0 con 0x555c9cc4d2d0 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde2a564740 10 client.0 _ll_register_callbacks cb 0x555c9cb90fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde24f27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde25f29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 0x555c9cc5e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde24f27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50536/0 (socket says 172.21.15.83:50536) 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde25f29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 0x555c9cc5e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41284/0 (socket says 172.21.15.83:41284) 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde24f27640 1 -- 172.21.15.83:0/884034928 learned_addr learned my addr 172.21.15.83:0/884034928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde25728640 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 0x555c9cc56f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde24f27640 1 -- 172.21.15.83:0/884034928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 msgr2=0x555c9cc56f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde24f27640 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 0x555c9cc56f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde24f27640 1 -- 172.21.15.83:0/884034928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 msgr2=0x555c9cc5e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde24f27640 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 0x555c9cc5e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.929+0000 7fde24f27640 1 -- 172.21.15.83:0/884034928 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde201a5e30 con 0x555c9cc4dcc0 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde24f27640 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde140bca70 tx=0x7fde140bcb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db6a0e7d3215da55 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:15.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde04ff9640 1 -- 172.21.15.83:0/884034928 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde14007ce0 con 0x555c9cc4dcc0 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde04ff9640 1 -- 172.21.15.83:0/884034928 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fde14007e40 con 0x555c9cc4dcc0 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde2a564740 1 -- 172.21.15.83:0/884034928 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c9c8b57e0 con 0x555c9cc4dcc0 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde04ff9640 1 -- 172.21.15.83:0/884034928 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde1404d200 con 0x555c9cc4dcc0 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde2a564740 10 client.26285 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde2a564740 1 -- 172.21.15.83:0/884034928 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555c9cbb4610 con 0x555c9cc4dcc0 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde04ff9640 1 -- 172.21.15.83:0/884034928 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555c9c8b57e0 con 0x555c9cc4dcc0 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde04ff9640 1 -- 172.21.15.83:0/884034928 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fde1403c070 con 0x555c9cc4dcc0 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde2a564740 10 client.26285 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde2a564740 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde2a564740 20 client.26285 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde2a564740 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555c9cc62ca0 0x555c9cc650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde2a564740 4 client.26285 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:15.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.930+0000 7fde2a564740 1 -- 172.21.15.83:0/884034928 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c9c8f2720 con 0x555c9cc62ca0 2024-03-11T19:47:15.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.931+0000 7fde25728640 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555c9cc62ca0 0x555c9cc650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.932+0000 7fde25728640 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555c9cc62ca0 0x555c9cc650c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a978823ea77e5c97 server_cookie=b86a7a52cb629f59 in_seq=0 out_seq=0 2024-03-11T19:47:15.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.932+0000 7fde04ff9640 10 client.26285 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:15.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.932+0000 7fde04ff9640 1 -- 172.21.15.83:0/884034928 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c9c8f2720 con 0x555c9cc62ca0 2024-03-11T19:47:15.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.932+0000 7fde04ff9640 10 client.26285 handle_command_reply: tid=0 2024-03-11T19:47:15.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.932+0000 7fde2a564740 1 client.26285 shutdown 2024-03-11T19:47:15.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.932+0000 7fde2a564740 1 -- 172.21.15.83:0/884034928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 msgr2=0x555c9cc57450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:15.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.932+0000 7fde2a564740 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde140bca70 tx=0x7fde140bcb50 comp rx=0 tx=0).stop 2024-03-11T19:47:15.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.933+0000 7fde2a564740 1 -- 172.21.15.83:0/884034928 shutdown_connections 2024-03-11T19:47:15.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.933+0000 7fde2a564740 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555c9cc62ca0 0x555c9cc650c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.933+0000 7fde2a564740 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 0x555c9cc5e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.933+0000 7fde2a564740 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.933+0000 7fde04ff9640 0 client.26285 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:15.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.933+0000 7fde2a564740 1 --2- 172.21.15.83:0/884034928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 0x555c9cc56f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.933+0000 7fde2a564740 1 -- 172.21.15.83:0/884034928 >> 172.21.15.83:0/884034928 conn(0x555c9cc487f0 msgr2=0x555c9cc47a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:15.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.933+0000 7fde2a564740 1 -- 172.21.15.83:0/884034928 shutdown_connections 2024-03-11T19:47:15.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.933+0000 7fde2a564740 1 -- 172.21.15.83:0/884034928 wait complete. 2024-03-11T19:47:15.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.933+0000 7fde2a564740 20 client.26285 trim_cache size 0 max 16384 2024-03-11T19:47:15.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.951+0000 7fde2a564740 1 Processor -- start 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.951+0000 7fde2a564740 1 -- start start 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.951+0000 7fde2a564740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 0x555c9cc56ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.951+0000 7fde2a564740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.951+0000 7fde2a564740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 0x555c9cc5e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.951+0000 7fde2a564740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9c8f2720 con 0x555c9cc52e30 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.951+0000 7fde2a564740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9c8b57e0 con 0x555c9cc4dcc0 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.951+0000 7fde2a564740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9cc4e3b0 con 0x555c9cc4d2d0 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25f29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 0x555c9cc5e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde24f27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25728640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 0x555c9cc56ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25728640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 0x555c9cc56ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59430/0 (socket says 172.21.15.83:59430) 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde24f27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50544/0 (socket says 172.21.15.83:50544) 2024-03-11T19:47:15.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25728640 1 -- 172.21.15.83:0/1675539368 learned_addr learned my addr 172.21.15.83:0/1675539368 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25f29640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 0x555c9cc5e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41290/0 (socket says 172.21.15.83:41290) 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25f29640 1 -- 172.21.15.83:0/1675539368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 msgr2=0x555c9cc56ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25f29640 1 --2- 172.21.15.83:0/1675539368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 0x555c9cc56ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25f29640 1 -- 172.21.15.83:0/1675539368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 msgr2=0x555c9cc57430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25f29640 1 --2- 172.21.15.83:0/1675539368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25f29640 1 -- 172.21.15.83:0/1675539368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c9cb50ac0 con 0x555c9cc52e30 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25728640 1 --2- 172.21.15.83:0/1675539368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 0x555c9cc56ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde24f27640 1 --2- 172.21.15.83:0/1675539368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde25f29640 1 --2- 172.21.15.83:0/1675539368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 0x555c9cc5e450 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fde1800b610 tx=0x7fde18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c870d0b8ab6e1cec server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde06ffd640 1 -- 172.21.15.83:0/1675539368 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1800be20 con 0x555c9cc52e30 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde06ffd640 1 -- 172.21.15.83:0/1675539368 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde1801ee50 con 0x555c9cc52e30 2024-03-11T19:47:15.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.952+0000 7fde06ffd640 1 -- 172.21.15.83:0/1675539368 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1802ac20 con 0x555c9cc52e30 2024-03-11T19:47:15.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.953+0000 7fde2a564740 1 -- 172.21.15.83:0/1675539368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 msgr2=0x555c9cc5e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:15.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.953+0000 7fde2a564740 1 --2- 172.21.15.83:0/1675539368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 0x555c9cc5e450 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fde1800b610 tx=0x7fde18004730 comp rx=0 tx=0).stop 2024-03-11T19:47:15.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.953+0000 7fde2a564740 1 -- 172.21.15.83:0/1675539368 shutdown_connections 2024-03-11T19:47:15.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.953+0000 7fde2a564740 1 --2- 172.21.15.83:0/1675539368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc52e30 0x555c9cc5e450 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.953+0000 7fde2a564740 1 --2- 172.21.15.83:0/1675539368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc4dcc0 0x555c9cc57430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.953+0000 7fde2a564740 1 --2- 172.21.15.83:0/1675539368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4d2d0 0x555c9cc56ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.953+0000 7fde2a564740 1 -- 172.21.15.83:0/1675539368 >> 172.21.15.83:0/1675539368 conn(0x555c9cc487f0 msgr2=0x555c9cc4ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:15.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.953+0000 7fde2a564740 1 -- 172.21.15.83:0/1675539368 shutdown_connections 2024-03-11T19:47:15.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.953+0000 7fde2a564740 1 -- 172.21.15.83:0/1675539368 wait complete. 2024-03-11T19:47:15.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.953+0000 7fde2a564740 1 -- start start 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.954+0000 7fde2a564740 1 Processor -- start 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.954+0000 7fde2a564740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 0x555c9cc5cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.954+0000 7fde2a564740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 0x555c9cc5d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.954+0000 7fde2a564740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc52e30 0x555c9cc5d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.954+0000 7fde2a564740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9cc5e7b0 con 0x555c9cc4d2d0 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.954+0000 7fde2a564740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9ccb9190 con 0x555c9cc52e30 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.954+0000 7fde2a564740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9ccb9300 con 0x555c9cc4dcc0 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.954+0000 7fde2a564740 10 client.0 _ll_register_callbacks cb 0x7fde180123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.954+0000 7fde25728640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 0x555c9cc5cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.954+0000 7fde24f27640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 0x555c9cc5d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.954+0000 7fde25728640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 0x555c9cc5cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41300/0 (socket says 172.21.15.83:41300) 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde25728640 1 -- 172.21.15.83:0/1447452335 learned_addr learned my addr 172.21.15.83:0/1447452335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:15.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde25f29640 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc52e30 0x555c9cc5d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde24f27640 1 -- 172.21.15.83:0/1447452335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc52e30 msgr2=0x555c9cc5d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:15.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde24f27640 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc52e30 0x555c9cc5d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde24f27640 1 -- 172.21.15.83:0/1447452335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 msgr2=0x555c9cc5cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:15.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde24f27640 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 0x555c9cc5cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde24f27640 1 -- 172.21.15.83:0/1447452335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde10012aa0 con 0x555c9cc4dcc0 2024-03-11T19:47:15.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde25728640 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 0x555c9cc5cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:15.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde25f29640 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc52e30 0x555c9cc5d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:15.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde24f27640 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 0x555c9cc5d110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fde14009900 tx=0x7fde140b9380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd726ed0f1cc5ea4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:15.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:47:15.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde04ff9640 1 -- 172.21.15.83:0/1447452335 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde140c4ff0 con 0x555c9cc4dcc0 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde04ff9640 1 -- 172.21.15.83:0/1447452335 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fde140c5280 con 0x555c9cc4dcc0 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde2a564740 1 -- 172.21.15.83:0/1447452335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde1404d580 con 0x555c9cc4dcc0 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde04ff9640 1 -- 172.21.15.83:0/1447452335 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde14007790 con 0x555c9cc4dcc0 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde2a564740 10 client.26401 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.955+0000 7fde2a564740 1 -- 172.21.15.83:0/1447452335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x555c9cbb4610 con 0x555c9cc4dcc0 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde04ff9640 1 -- 172.21.15.83:0/1447452335 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde1404d580 con 0x555c9cc4dcc0 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde04ff9640 1 -- 172.21.15.83:0/1447452335 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fde1403c040 con 0x555c9cc4dcc0 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde2a564740 10 client.26401 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde2a564740 10 client.26401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde2a564740 20 client.26401 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde2a564740 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555c9ccba780 0x555c9ccbcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde2a564740 4 client.26401 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:47:15.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde2a564740 1 -- 172.21.15.83:0/1447452335 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555c9c941890 con 0x555c9ccba780 2024-03-11T19:47:15.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde25728640 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555c9ccba780 0x555c9ccbcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:15.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde04ff9640 10 client.26401 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:15.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde25728640 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555c9ccba780 0x555c9ccbcba0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f0b6f58b349d0c2 server_cookie=3bfda4c222a1e88f in_seq=0 out_seq=0 2024-03-11T19:47:15.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde04ff9640 1 -- 172.21.15.83:0/1447452335 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555c9c941890 con 0x555c9ccba780 2024-03-11T19:47:15.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.956+0000 7fde04ff9640 10 client.26401 handle_command_reply: tid=0 2024-03-11T19:47:15.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 1 client.26401 shutdown 2024-03-11T19:47:15.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 1 -- 172.21.15.83:0/1447452335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 msgr2=0x555c9cc5d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:15.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 0x555c9cc5d110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fde14009900 tx=0x7fde140b9380 comp rx=0 tx=0).stop 2024-03-11T19:47:15.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 1 -- 172.21.15.83:0/1447452335 shutdown_connections 2024-03-11T19:47:15.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555c9ccba780 0x555c9ccbcba0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c9cc52e30 0x555c9cc5d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde04ff9640 0 client.26401 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:15.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555c9cc4dcc0 0x555c9cc5d110 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 1 --2- 172.21.15.83:0/1447452335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c9cc4d2d0 0x555c9cc5cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 1 -- 172.21.15.83:0/1447452335 >> 172.21.15.83:0/1447452335 conn(0x555c9cc487f0 msgr2=0x555c9cc4ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:15.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 1 -- 172.21.15.83:0/1447452335 shutdown_connections 2024-03-11T19:47:15.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 1 -- 172.21.15.83:0/1447452335 wait complete. 2024-03-11T19:47:15.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2a564740 20 client.26401 trim_cache size 0 max 16384 2024-03-11T19:47:15.968 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:47:15.968 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:47:15.969 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:47:15.969 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:47:15.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2798c640 1 -- 172.21.15.83:0/2191747100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fde080a1b30 msgr2=0x7fde080a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2798c640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fde080a1b30 0x7fde080a3ff0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fde18005fd0 tx=0x7fde1800ba90 comp rx=0 tx=0).stop 2024-03-11T19:47:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2798c640 1 -- 172.21.15.83:0/2191747100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde201089b0 msgr2=0x7fde2019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2798c640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde201089b0 0x7fde2019dda0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde1402c170 tx=0x7fde140040c0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2798c640 1 -- 172.21.15.83:0/2191747100 shutdown_connections 2024-03-11T19:47:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2798c640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fde080a1b30 0x7fde080a3ff0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2798c640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde20109370 0x7fde201a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2798c640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde201089b0 0x7fde2019dda0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2798c640 1 --2- 172.21.15.83:0/2191747100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde201077c0 0x7fde2019d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.957+0000 7fde2798c640 1 -- 172.21.15.83:0/2191747100 >> 172.21.15.83:0/2191747100 conn(0x7fde20076770 msgr2=0x7fde2010bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.958+0000 7fde2798c640 1 -- 172.21.15.83:0/2191747100 shutdown_connections 2024-03-11T19:47:15.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:15.958+0000 7fde2798c640 1 -- 172.21.15.83:0/2191747100 wait complete. 2024-03-11T19:47:15.974 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:47:15.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:47:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.124+0000 7f945ec76640 1 -- 172.21.15.83:0/3486608152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945810aaf0 msgr2=0x7f945810cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.124+0000 7f945ec76640 1 --2- 172.21.15.83:0/3486608152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945810aaf0 0x7f945810cf80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9450009ca0 tx=0x7f945002c140 comp rx=0 tx=0).stop 2024-03-11T19:47:16.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.126+0000 7f945ec76640 1 -- 172.21.15.83:0/3486608152 shutdown_connections 2024-03-11T19:47:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.126+0000 7f945ec76640 1 --2- 172.21.15.83:0/3486608152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f945810d4c0 0x7f945810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.126+0000 7f945ec76640 1 --2- 172.21.15.83:0/3486608152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945810aaf0 0x7f945810cf80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.126+0000 7f945ec76640 1 --2- 172.21.15.83:0/3486608152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9458108180 0x7f945810a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.126+0000 7f945ec76640 1 -- 172.21.15.83:0/3486608152 >> 172.21.15.83:0/3486608152 conn(0x7f945806dff0 msgr2=0x7f9458070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.132+0000 7f945ec76640 1 -- 172.21.15.83:0/3486608152 shutdown_connections 2024-03-11T19:47:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.135+0000 7f945ec76640 1 -- 172.21.15.83:0/3486608152 wait complete. 2024-03-11T19:47:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.135+0000 7f945ec76640 1 Processor -- start 2024-03-11T19:47:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.135+0000 7f945ec76640 1 -- start start 2024-03-11T19:47:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.136+0000 7f945ec76640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9458108180 0x7f94581a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.136+0000 7f945ec76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945810aaf0 0x7f94581a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.136+0000 7f945ec76640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f945810d4c0 0x7f94580793e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.136+0000 7f945ec76640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9458079950 con 0x7f945810aaf0 2024-03-11T19:47:16.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.136+0000 7f945ec76640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9458079ac0 con 0x7f945810d4c0 2024-03-11T19:47:16.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.136+0000 7f945ec76640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9458079c80 con 0x7f9458108180 2024-03-11T19:47:16.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.136+0000 7f945d213640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f945810d4c0 0x7f94580793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.136+0000 7f945d213640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f945810d4c0 0x7f94580793e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50562/0 (socket says 172.21.15.83:50562) 2024-03-11T19:47:16.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.136+0000 7f945d213640 1 -- 172.21.15.83:0/752516777 learned_addr learned my addr 172.21.15.83:0/752516777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:16.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.137+0000 7f945ca12640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9458108180 0x7f94581a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.139+0000 7f945ca12640 1 -- 172.21.15.83:0/752516777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f945810d4c0 msgr2=0x7f94580793e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.139+0000 7f945ca12640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f945810d4c0 0x7f94580793e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.139+0000 7f945ca12640 1 -- 172.21.15.83:0/752516777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945810aaf0 msgr2=0x7f94581a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.139+0000 7f945ca12640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945810aaf0 0x7f94581a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.139+0000 7f945ca12640 1 -- 172.21.15.83:0/752516777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9450009900 con 0x7f9458108180 2024-03-11T19:47:16.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.144+0000 7f945ca12640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9458108180 0x7f94581a60b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f944800c940 tx=0x7f944800ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.144+0000 7f9455ffb640 1 -- 172.21.15.83:0/752516777 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448004220 con 0x7f9458108180 2024-03-11T19:47:16.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.144+0000 7f9455ffb640 1 -- 172.21.15.83:0/752516777 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9448007e90 con 0x7f9458108180 2024-03-11T19:47:16.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.144+0000 7f9455ffb640 1 -- 172.21.15.83:0/752516777 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944800f450 con 0x7f9458108180 2024-03-11T19:47:16.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.144+0000 7f945ec76640 1 -- 172.21.15.83:0/752516777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9458079f60 con 0x7f9458108180 2024-03-11T19:47:16.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.145+0000 7f945ec76640 1 -- 172.21.15.83:0/752516777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945807a420 con 0x7f9458108180 2024-03-11T19:47:16.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.146+0000 7f945ec76640 1 -- 172.21.15.83:0/752516777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9458070af0 con 0x7f9458108180 2024-03-11T19:47:16.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.151+0000 7f9455ffb640 1 -- 172.21.15.83:0/752516777 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f94480079b0 con 0x7f9458108180 2024-03-11T19:47:16.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.152+0000 7f9455ffb640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94400a1c40 0x7f94400a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.152+0000 7f9455ffb640 1 -- 172.21.15.83:0/752516777 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f94480ccf20 con 0x7f9458108180 2024-03-11T19:47:16.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.153+0000 7f9457fff640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94400a1c40 0x7f94400a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.154+0000 7f9457fff640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94400a1c40 0x7f94400a4100 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f945002c750 tx=0x7f9450009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.157+0000 7f9455ffb640 1 -- 172.21.15.83:0/752516777 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9448096140 con 0x7f9458108180 2024-03-11T19:47:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.465+0000 7f945ec76640 1 -- 172.21.15.83:0/752516777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f945811ba80 con 0x7f9458108180 2024-03-11T19:47:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.466+0000 7f9455ffb640 1 -- 172.21.15.83:0/752516777 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11350 (secure 0 0 0) 0x7f944801d090 con 0x7f9458108180 2024-03-11T19:47:16.471 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:47:16.471 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:45:19.152624+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:47:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 31 2024-03-11T19:47:16.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 -- 172.21.15.83:0/752516777 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94400a1c40 msgr2=0x7f94400a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94400a1c40 0x7f94400a4100 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f945002c750 tx=0x7f9450009870 comp rx=0 tx=0).stop 2024-03-11T19:47:16.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 -- 172.21.15.83:0/752516777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9458108180 msgr2=0x7f94581a60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9458108180 0x7f94581a60b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f944800c940 tx=0x7f944800ce00 comp rx=0 tx=0).stop 2024-03-11T19:47:16.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 -- 172.21.15.83:0/752516777 shutdown_connections 2024-03-11T19:47:16.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94400a1c40 0x7f94400a4100 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f945810d4c0 0x7f94580793e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f945810aaf0 0x7f94581a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 --2- 172.21.15.83:0/752516777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9458108180 0x7f94581a60b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 -- 172.21.15.83:0/752516777 >> 172.21.15.83:0/752516777 conn(0x7f945806dff0 msgr2=0x7f945810a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 -- 172.21.15.83:0/752516777 shutdown_connections 2024-03-11T19:47:16.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.470+0000 7f945ec76640 1 -- 172.21.15.83:0/752516777 wait complete. 2024-03-11T19:47:16.487 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:47:16.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:47:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.645+0000 7f5eb2adf640 1 -- 172.21.15.83:0/3828286853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac0ff4a0 msgr2=0x7f5eac0ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.645+0000 7f5eb2adf640 1 --2- 172.21.15.83:0/3828286853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac0ff4a0 0x7f5eac0ff8c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009900 tx=0x7f5ea002bc30 comp rx=0 tx=0).stop 2024-03-11T19:47:16.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.645+0000 7f5eb2adf640 1 -- 172.21.15.83:0/3828286853 shutdown_connections 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.645+0000 7f5eb2adf640 1 --2- 172.21.15.83:0/3828286853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5eac102ed0 0x7f5eac109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.645+0000 7f5eb2adf640 1 --2- 172.21.15.83:0/3828286853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eac102510 0x7f5eac102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.645+0000 7f5eb2adf640 1 --2- 172.21.15.83:0/3828286853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac0ff4a0 0x7f5eac0ff8c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.645+0000 7f5eb2adf640 1 -- 172.21.15.83:0/3828286853 >> 172.21.15.83:0/3828286853 conn(0x7f5eac0fadb0 msgr2=0x7f5eac0fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.645+0000 7f5eb2adf640 1 -- 172.21.15.83:0/3828286853 shutdown_connections 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.645+0000 7f5eb2adf640 1 -- 172.21.15.83:0/3828286853 wait complete. 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eb2adf640 1 Processor -- start 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eb2adf640 1 -- start start 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eb2adf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5eac0ff4a0 0x7f5eac195100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eb2adf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac102510 0x7f5eac195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eb2adf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eac102ed0 0x7f5eac19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eb2adf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac19cbf0 con 0x7f5eac102510 2024-03-11T19:47:16.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eb2adf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac19cd60 con 0x7f5eac0ff4a0 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eb2adf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac19cf20 con 0x7f5eac102ed0 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac102510 0x7f5eac195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac102510 0x7f5eac195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41348/0 (socket says 172.21.15.83:41348) 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eabfff640 1 -- 172.21.15.83:0/1512722405 learned_addr learned my addr 172.21.15.83:0/1512722405 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eb107c640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eac102ed0 0x7f5eac19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.646+0000 7f5eb087b640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5eac0ff4a0 0x7f5eac195100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5eabfff640 1 -- 172.21.15.83:0/1512722405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eac102ed0 msgr2=0x7f5eac19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5eabfff640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eac102ed0 0x7f5eac19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5eabfff640 1 -- 172.21.15.83:0/1512722405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5eac0ff4a0 msgr2=0x7f5eac195100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5eabfff640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5eac0ff4a0 0x7f5eac195100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5eabfff640 1 -- 172.21.15.83:0/1512722405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea0009560 con 0x7f5eac102510 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5eb087b640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5eac0ff4a0 0x7f5eac195100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:16.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5eabfff640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac102510 0x7f5eac195640 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800c970 tx=0x7f5e9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5ea9ffb640 1 -- 172.21.15.83:0/1512722405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98007bb0 con 0x7f5eac102510 2024-03-11T19:47:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5ea9ffb640 1 -- 172.21.15.83:0/1512722405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e98007d10 con 0x7f5eac102510 2024-03-11T19:47:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5eb2adf640 1 -- 172.21.15.83:0/1512722405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac19d200 con 0x7f5eac102510 2024-03-11T19:47:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5ea9ffb640 1 -- 172.21.15.83:0/1512722405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9800f450 con 0x7f5eac102510 2024-03-11T19:47:16.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.647+0000 7f5eb2adf640 1 -- 172.21.15.83:0/1512722405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac19d710 con 0x7f5eac102510 2024-03-11T19:47:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.650+0000 7f5eb5794740 1 Processor -- start 2024-03-11T19:47:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb5794740 1 -- start start 2024-03-11T19:47:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5ea9ffb640 1 -- 172.21.15.83:0/1512722405 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5e98004f70 con 0x7f5eac102510 2024-03-11T19:47:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb5794740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494270 0x561238494690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb5794740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238494c60 0x561238499760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb5794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238499dd0 0x56123849c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb5794740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561238450960 con 0x561238499dd0 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb5794740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561238404310 con 0x561238494c60 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb5794740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561238401840 con 0x561238494270 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb107c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494270 0x561238494690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238499dd0 0x56123849c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb107c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494270 0x561238494690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59486/0 (socket says 172.21.15.83:59486) 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb107c640 1 -- 172.21.15.83:0/3053467619 learned_addr learned my addr 172.21.15.83:0/3053467619 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb087b640 1 --2- 172.21.15.83:0/3053467619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238494c60 0x561238499760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb107c640 1 -- 172.21.15.83:0/3053467619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238494c60 msgr2=0x561238499760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb107c640 1 --2- 172.21.15.83:0/3053467619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238494c60 0x561238499760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb107c640 1 -- 172.21.15.83:0/3053467619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238499dd0 msgr2=0x56123849c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb107c640 1 --2- 172.21.15.83:0/3053467619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238499dd0 0x56123849c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5eb107c640 1 -- 172.21.15.83:0/3053467619 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612383bf060 con 0x561238494270 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.651+0000 7f5ea9ffb640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5e9c0a5ec0 0x7f5e9c0a8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.652+0000 7f5eb107c640 1 --2- 172.21.15.83:0/3053467619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494270 0x561238494690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40039f0 tx=0x7f5ea4003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce6b2a3801984069 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.652+0000 7f5eb087b640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5e9c0a5ec0 0x7f5e9c0a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.652+0000 7f5ea9ffb640 1 -- 172.21.15.83:0/1512722405 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5e980ccf90 con 0x7f5eac102510 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.652+0000 7f5e96ffd640 1 -- 172.21.15.83:0/3053467619 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea401a470 con 0x561238494270 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.652+0000 7f5e96ffd640 1 -- 172.21.15.83:0/3053467619 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ea4011920 con 0x561238494270 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.652+0000 7f5e96ffd640 1 -- 172.21.15.83:0/3053467619 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4010a70 con 0x561238494270 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.652+0000 7f5eb5794740 1 -- 172.21.15.83:0/3053467619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494270 msgr2=0x561238494690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.652+0000 7f5eb087b640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5e9c0a5ec0 0x7f5e9c0a8380 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5ea002c240 tx=0x7f5ea002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.652+0000 7f5eb5794740 1 --2- 172.21.15.83:0/3053467619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494270 0x561238494690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40039f0 tx=0x7f5ea4003eb0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 -- 172.21.15.83:0/3053467619 shutdown_connections 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 --2- 172.21.15.83:0/3053467619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238499dd0 0x56123849c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 --2- 172.21.15.83:0/3053467619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238494c60 0x561238499760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 --2- 172.21.15.83:0/3053467619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494270 0x561238494690 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 -- 172.21.15.83:0/3053467619 >> 172.21.15.83:0/3053467619 conn(0x56123848f7e0 msgr2=0x56123848fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 -- 172.21.15.83:0/3053467619 shutdown_connections 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 -- 172.21.15.83:0/3053467619 wait complete. 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 -- start start 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 Processor -- start 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238494270 0x56123849e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 0x56123849ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238499dd0 0x5612384a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612383fd020 con 0x561238499dd0 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612380d0430 con 0x561238494270 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.653+0000 7f5eb5794740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123808ce00 con 0x561238494c60 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.654+0000 7f5eb5794740 10 client.0 _ll_register_callbacks cb 0x5612383a4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.654+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 0x56123849ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.654+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 0x56123849ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59500/0 (socket says 172.21.15.83:59500) 2024-03-11T19:47:16.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.654+0000 7f5eabfff640 1 -- 172.21.15.83:0/1096382187 learned_addr learned my addr 172.21.15.83:0/1096382187 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.654+0000 7f5eabfff640 1 -- 172.21.15.83:0/1096382187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238494270 msgr2=0x56123849e510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:47:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.654+0000 7f5eabfff640 1 --2- 172.21.15.83:0/1096382187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238494270 0x56123849e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.654+0000 7f5eabfff640 1 -- 172.21.15.83:0/1096382187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238499dd0 msgr2=0x5612384a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.654+0000 7f5eabfff640 1 --2- 172.21.15.83:0/1096382187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238499dd0 0x5612384a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.654+0000 7f5eabfff640 1 -- 172.21.15.83:0/1096382187 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea4009c90 con 0x561238494c60 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.655+0000 7f5eabfff640 1 --2- 172.21.15.83:0/1096382187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 0x56123849ea50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800c940 tx=0x7f5e9800a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde17a6984d4d314 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.655+0000 7f5e8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.656+0000 7f5e8bfff640 1 -- 172.21.15.83:0/1096382187 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e980cf2d0 con 0x561238494c60 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.656+0000 7f5e8bfff640 1 -- 172.21.15.83:0/1096382187 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5e98005600 con 0x561238494c60 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.656+0000 7f5eb5794740 1 -- 172.21.15.83:0/1096382187 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612380fc7e0 con 0x561238494c60 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.656+0000 7f5e8bfff640 1 -- 172.21.15.83:0/1096382187 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e98099ba0 con 0x561238494c60 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.656+0000 7f5eb5794740 10 client.26413 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.656+0000 7f5eb5794740 1 -- 172.21.15.83:0/1096382187 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5612383fb6a0 con 0x561238494c60 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.656+0000 7f5e8bfff640 1 -- 172.21.15.83:0/1096382187 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5612380fc7e0 con 0x561238494c60 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.657+0000 7f5e8bfff640 1 -- 172.21.15.83:0/1096382187 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f5e98016030 con 0x561238494c60 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.657+0000 7f5eb5794740 10 client.26413 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.657+0000 7f5eb5794740 10 client.26413 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.657+0000 7f5eb5794740 20 client.26413 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:16.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.657+0000 7f5eb5794740 1 --2- 172.21.15.83:0/1096382187 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5612384aa750 0x5612384acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.657+0000 7f5eb5794740 4 client.26413 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.657+0000 7f5eb5794740 1 -- 172.21.15.83:0/1096382187 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561238139720 con 0x5612384aa750 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.657+0000 7f5eb087b640 1 --2- 172.21.15.83:0/1096382187 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5612384aa750 0x5612384acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.657+0000 7f5e8bfff640 10 client.26413 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.657+0000 7f5eb087b640 1 --2- 172.21.15.83:0/1096382187 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5612384aa750 0x5612384acb70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=494c783751a036ae server_cookie=acde8168548ca038 in_seq=0 out_seq=0 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5e8bfff640 1 -- 172.21.15.83:0/1096382187 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561238139720 con 0x5612384aa750 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5e8bfff640 10 client.26413 handle_command_reply: tid=0 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 1 client.26413 shutdown 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 1 -- 172.21.15.83:0/1096382187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 msgr2=0x56123849ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 1 --2- 172.21.15.83:0/1096382187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 0x56123849ea50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800c940 tx=0x7f5e9800a4d0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 1 -- 172.21.15.83:0/1096382187 shutdown_connections 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 1 --2- 172.21.15.83:0/1096382187 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5612384aa750 0x5612384acb70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 1 --2- 172.21.15.83:0/1096382187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238499dd0 0x5612384a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5e8bfff640 0 client.26413 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:47:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 1 --2- 172.21.15.83:0/1096382187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 0x56123849ea50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 1 --2- 172.21.15.83:0/1096382187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238494270 0x56123849e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 1 -- 172.21.15.83:0/1096382187 >> 172.21.15.83:0/1096382187 conn(0x56123848f7e0 msgr2=0x56123848eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 1 -- 172.21.15.83:0/1096382187 shutdown_connections 2024-03-11T19:47:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 1 -- 172.21.15.83:0/1096382187 wait complete. 2024-03-11T19:47:16.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.658+0000 7f5eb5794740 20 client.26413 trim_cache size 0 max 16384 2024-03-11T19:47:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.673+0000 7f5eb5794740 1 Processor -- start 2024-03-11T19:47:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.673+0000 7f5eb5794740 1 -- start start 2024-03-11T19:47:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.673+0000 7f5eb5794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494270 0x56123849e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.673+0000 7f5eb5794740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 0x5612384a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.673+0000 7f5eb5794740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238499dd0 0x56123849ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.673+0000 7f5eb5794740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561238382130 con 0x561238494270 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.673+0000 7f5eb5794740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561238481a40 con 0x561238499dd0 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.673+0000 7f5eb5794740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123848aa10 con 0x561238494c60 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5eb087b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494270 0x56123849e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5eb087b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494270 0x56123849e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41384/0 (socket says 172.21.15.83:41384) 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5eb087b640 1 -- 172.21.15.83:0/2460628705 learned_addr learned my addr 172.21.15.83:0/2460628705 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5eabfff640 1 --2- 172.21.15.83:0/2460628705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 0x5612384a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5eb107c640 1 --2- 172.21.15.83:0/2460628705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238499dd0 0x56123849ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5eb087b640 1 -- 172.21.15.83:0/2460628705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 msgr2=0x5612384a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5eb087b640 1 --2- 172.21.15.83:0/2460628705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 0x5612384a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5eb087b640 1 -- 172.21.15.83:0/2460628705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238499dd0 msgr2=0x56123849ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5eb087b640 1 --2- 172.21.15.83:0/2460628705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238499dd0 0x56123849ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5eb087b640 1 -- 172.21.15.83:0/2460628705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561238397c20 con 0x561238494270 2024-03-11T19:47:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5eb087b640 1 --2- 172.21.15.83:0/2460628705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494270 0x56123849e510 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005c40 tx=0x7f5ea0042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff811dab6d852e4b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5e967fc640 1 -- 172.21.15.83:0/2460628705 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0051040 con 0x561238494270 2024-03-11T19:47:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5e967fc640 1 -- 172.21.15.83:0/2460628705 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ea0058070 con 0x561238494270 2024-03-11T19:47:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.674+0000 7f5e967fc640 1 -- 172.21.15.83:0/2460628705 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0003850 con 0x561238494270 2024-03-11T19:47:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.675+0000 7f5eb5794740 1 -- 172.21.15.83:0/2460628705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494270 msgr2=0x56123849e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.675+0000 7f5eb5794740 1 --2- 172.21.15.83:0/2460628705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494270 0x56123849e510 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0005c40 tx=0x7f5ea0042000 comp rx=0 tx=0).stop 2024-03-11T19:47:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.675+0000 7f5eb5794740 1 -- 172.21.15.83:0/2460628705 shutdown_connections 2024-03-11T19:47:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.675+0000 7f5eb5794740 1 --2- 172.21.15.83:0/2460628705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238499dd0 0x56123849ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.675+0000 7f5eb5794740 1 --2- 172.21.15.83:0/2460628705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494c60 0x5612384a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.675+0000 7f5eb5794740 1 --2- 172.21.15.83:0/2460628705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494270 0x56123849e510 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.675+0000 7f5eb5794740 1 -- 172.21.15.83:0/2460628705 >> 172.21.15.83:0/2460628705 conn(0x56123848f7e0 msgr2=0x5612384b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.675+0000 7f5eb5794740 1 -- 172.21.15.83:0/2460628705 shutdown_connections 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.675+0000 7f5eb5794740 1 -- 172.21.15.83:0/2460628705 wait complete. 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.675+0000 7f5eb5794740 1 -- start start 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.675+0000 7f5eb5794740 1 Processor -- start 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.676+0000 7f5eb5794740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494270 0x5612384a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.676+0000 7f5eb5794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494c60 0x5612384a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.676+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494c60 0x5612384a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.676+0000 7f5eabfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494c60 0x5612384a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41396/0 (socket says 172.21.15.83:41396) 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.676+0000 7f5eabfff640 1 -- 172.21.15.83:0/2431410672 learned_addr learned my addr 172.21.15.83:0/2431410672 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.676+0000 7f5eb5794740 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238499dd0 0x5612384a4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.676+0000 7f5eb5794740 1 -- 172.21.15.83:0/2431410672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612384a4c50 con 0x561238494c60 2024-03-11T19:47:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.676+0000 7f5eb5794740 1 -- 172.21.15.83:0/2431410672 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612384a4dc0 con 0x561238499dd0 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.677+0000 7f5eb5794740 1 -- 172.21.15.83:0/2431410672 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561238500c60 con 0x561238494270 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.677+0000 7f5eb5794740 10 client.0 _ll_register_callbacks cb 0x7f5ea4009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.677+0000 7f5eabfff640 1 -- 172.21.15.83:0/2431410672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494270 msgr2=0x5612384a59a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.677+0000 7f5eabfff640 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494270 0x5612384a59a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.677+0000 7f5eabfff640 1 -- 172.21.15.83:0/2431410672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238499dd0 msgr2=0x5612384a4550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.677+0000 7f5eabfff640 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238499dd0 0x5612384a4550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.677+0000 7f5eabfff640 1 -- 172.21.15.83:0/2431410672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac109ee0 con 0x561238494c60 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.677+0000 7f5eabfff640 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494c60 0x5612384a3fb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800a600 tx=0x7f5e980a24f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6419dea72c6b8032 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.678+0000 7f5e8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.678+0000 7f5e8bfff640 1 -- 172.21.15.83:0/2431410672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98099ba0 con 0x561238494c60 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.678+0000 7f5e8bfff640 1 -- 172.21.15.83:0/2431410672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5e98005600 con 0x561238494c60 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.678+0000 7f5eb5794740 1 -- 172.21.15.83:0/2431410672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ea4011920 con 0x561238494c60 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.678+0000 7f5e8bfff640 1 -- 172.21.15.83:0/2431410672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e98094bd0 con 0x561238494c60 2024-03-11T19:47:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.678+0000 7f5eb5794740 10 client.16848 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.678+0000 7f5eb5794740 1 -- 172.21.15.83:0/2431410672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5612383bd8c0 con 0x561238494c60 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.679+0000 7f5e8bfff640 1 -- 172.21.15.83:0/2431410672 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e980a2750 con 0x561238494c60 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.679+0000 7f5e8bfff640 1 -- 172.21.15.83:0/2431410672 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f5e98016070 con 0x561238494c60 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.679+0000 7f5eb5794740 10 client.16848 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.679+0000 7f5eb5794740 10 client.16848 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.679+0000 7f5eb5794740 20 client.16848 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.679+0000 7f5eb5794740 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5612385022c0 0x5612385046e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.679+0000 7f5eb5794740 4 client.16848 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.679+0000 7f5eb5794740 1 -- 172.21.15.83:0/2431410672 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f5ea4011920 con 0x5612385022c0 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.679+0000 7f5eb087b640 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5612385022c0 0x5612385046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5e8bfff640 10 client.16848 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb087b640 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5612385022c0 0x5612385046e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6efb90eacfe1df04 server_cookie=cb602257fb60568a in_seq=0 out_seq=0 2024-03-11T19:47:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5e8bfff640 1 -- 172.21.15.83:0/2431410672 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f5ea4011920 con 0x5612385022c0 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5e8bfff640 10 client.16848 handle_command_reply: tid=0 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 1 client.16848 shutdown 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 1 -- 172.21.15.83:0/2431410672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494c60 msgr2=0x5612384a3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494c60 0x5612384a3fb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800a600 tx=0x7f5e980a24f0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 1 -- 172.21.15.83:0/2431410672 shutdown_connections 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5612385022c0 0x5612385046e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561238499dd0 0x5612384a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5e8bfff640 0 client.16848 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561238494c60 0x5612384a3fb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 1 --2- 172.21.15.83:0/2431410672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561238494270 0x5612384a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 1 -- 172.21.15.83:0/2431410672 >> 172.21.15.83:0/2431410672 conn(0x56123848f7e0 msgr2=0x5612384961f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 1 -- 172.21.15.83:0/2431410672 shutdown_connections 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 1 -- 172.21.15.83:0/2431410672 wait complete. 2024-03-11T19:47:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.680+0000 7f5eb5794740 20 client.16848 trim_cache size 0 max 16384 2024-03-11T19:47:16.689 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:47:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 -- 172.21.15.83:0/1512722405 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5e9c0a5ec0 msgr2=0x7f5e9c0a8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5e9c0a5ec0 0x7f5e9c0a8380 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5ea002c240 tx=0x7f5ea002c7e0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 -- 172.21.15.83:0/1512722405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac102510 msgr2=0x7f5eac195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac102510 0x7f5eac195640 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800c970 tx=0x7f5e9800ce30 comp rx=0 tx=0).stop 2024-03-11T19:47:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 -- 172.21.15.83:0/1512722405 shutdown_connections 2024-03-11T19:47:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5e9c0a5ec0 0x7f5e9c0a8380 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5eac102ed0 0x7f5eac19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eac102510 0x7f5eac195640 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 --2- 172.21.15.83:0/1512722405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5eac0ff4a0 0x7f5eac195100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 -- 172.21.15.83:0/1512722405 >> 172.21.15.83:0/1512722405 conn(0x7f5eac0fadb0 msgr2=0x7f5eac105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 -- 172.21.15.83:0/1512722405 shutdown_connections 2024-03-11T19:47:16.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.681+0000 7f5eb2adf640 1 -- 172.21.15.83:0/1512722405 wait complete. 2024-03-11T19:47:16.699 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:47:16.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:47:16.750 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:16 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/752516777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:47:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.851+0000 7fbaafc81640 1 -- 172.21.15.83:0/2618404137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa8101270 msgr2=0x7fbaa81016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.851+0000 7fbaafc81640 1 --2- 172.21.15.83:0/2618404137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa8101270 0x7fbaa81016f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009860 tx=0x7fba9c02bb70 comp rx=0 tx=0).stop 2024-03-11T19:47:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.851+0000 7fbaafc81640 1 -- 172.21.15.83:0/2618404137 shutdown_connections 2024-03-11T19:47:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.851+0000 7fbaafc81640 1 --2- 172.21.15.83:0/2618404137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaa8101c30 0x7fbaa81084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.851+0000 7fbaafc81640 1 --2- 172.21.15.83:0/2618404137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa8101270 0x7fbaa81016f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.851+0000 7fbaafc81640 1 --2- 172.21.15.83:0/2618404137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaa8100080 0x7fbaa81004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.851+0000 7fbaafc81640 1 -- 172.21.15.83:0/2618404137 >> 172.21.15.83:0/2618404137 conn(0x7fbaa80fb610 msgr2=0x7fbaa80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.851+0000 7fbaafc81640 1 -- 172.21.15.83:0/2618404137 shutdown_connections 2024-03-11T19:47:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.851+0000 7fbaafc81640 1 -- 172.21.15.83:0/2618404137 wait complete. 2024-03-11T19:47:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaafc81640 1 Processor -- start 2024-03-11T19:47:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaafc81640 1 -- start start 2024-03-11T19:47:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaafc81640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaa8100080 0x7fbaa8199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaafc81640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaa8101270 0x7fbaa8199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaafc81640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa8101c30 0x7fbaa81a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaafc81640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa81a0f20 con 0x7fbaa8101c30 2024-03-11T19:47:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaafc81640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa81a1090 con 0x7fbaa8100080 2024-03-11T19:47:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaafc81640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa81a1250 con 0x7fbaa8101270 2024-03-11T19:47:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaae21e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa8101c30 0x7fbaa81a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaae21e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa8101c30 0x7fbaa81a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41402/0 (socket says 172.21.15.83:41402) 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaae21e640 1 -- 172.21.15.83:0/661160731 learned_addr learned my addr 172.21.15.83:0/661160731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaad21c640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaa8101270 0x7fbaa8199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.852+0000 7fbaada1d640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaa8100080 0x7fbaa8199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fbaad21c640 1 -- 172.21.15.83:0/661160731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaa8100080 msgr2=0x7fbaa8199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fbaad21c640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaa8100080 0x7fbaa8199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fbaad21c640 1 -- 172.21.15.83:0/661160731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa8101c30 msgr2=0x7fbaa81a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fbaad21c640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa8101c30 0x7fbaa81a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fbaad21c640 1 -- 172.21.15.83:0/661160731 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c009490 con 0x7fbaa8101270 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fbaae21e640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa8101c30 0x7fbaa81a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fbaad21c640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaa8101270 0x7fbaa8199970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fba9c02c0b0 tx=0x7fba9c0045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fba9affd640 1 -- 172.21.15.83:0/661160731 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c02c510 con 0x7fbaa8101270 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fbaafc81640 1 -- 172.21.15.83:0/661160731 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa81a14d0 con 0x7fbaa8101270 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fba9affd640 1 -- 172.21.15.83:0/661160731 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba9c02cb50 con 0x7fbaa8101270 2024-03-11T19:47:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fbaafc81640 1 -- 172.21.15.83:0/661160731 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa81a1840 con 0x7fbaa8101270 2024-03-11T19:47:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.853+0000 7fba9affd640 1 -- 172.21.15.83:0/661160731 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c032600 con 0x7fbaa8101270 2024-03-11T19:47:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbab284f740 1 Processor -- start 2024-03-11T19:47:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbab284f740 1 -- start start 2024-03-11T19:47:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbab284f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc17690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbab284f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc1c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbab284f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc1f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbab284f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2bbd3960 con 0x55ea2bc1cdd0 2024-03-11T19:47:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbab284f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2bb87310 con 0x55ea2bc17270 2024-03-11T19:47:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbab284f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2bb84840 con 0x55ea2bc17c60 2024-03-11T19:47:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fba9affd640 1 -- 172.21.15.83:0/661160731 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fba9c03b040 con 0x7fbaa8101270 2024-03-11T19:47:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbaae21e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc17690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbaada1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc1c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbaada1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc1c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59526/0 (socket says 172.21.15.83:59526) 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbaae21e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc17690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50660/0 (socket says 172.21.15.83:50660) 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.856+0000 7fbaada1d640 1 -- 172.21.15.83:0/3197524357 learned_addr learned my addr 172.21.15.83:0/3197524357 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.857+0000 7fbaad21c640 1 --2- 172.21.15.83:0/3197524357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc1f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.857+0000 7fbaae21e640 1 -- 172.21.15.83:0/3197524357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 msgr2=0x55ea2bc1c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.857+0000 7fbaae21e640 1 --2- 172.21.15.83:0/3197524357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc1c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.857+0000 7fbaae21e640 1 -- 172.21.15.83:0/3197524357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc1cdd0 msgr2=0x55ea2bc1f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.857+0000 7fbaae21e640 1 --2- 172.21.15.83:0/3197524357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc1f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.857+0000 7fbaae21e640 1 -- 172.21.15.83:0/3197524357 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2bb42060 con 0x55ea2bc17270 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.857+0000 7fbaae21e640 1 --2- 172.21.15.83:0/3197524357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc17690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00117e0 tx=0x7fbaa0011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70fb1b7427a407aa server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.857+0000 7fba9affd640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba940a1a30 0x7fba940a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.858+0000 7fba9affd640 1 -- 172.21.15.83:0/661160731 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fba9c0eee40 con 0x7fbaa8101270 2024-03-11T19:47:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.858+0000 7fbaada1d640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba940a1a30 0x7fba940a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.859+0000 7fba93fff640 1 -- 172.21.15.83:0/3197524357 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0011250 con 0x55ea2bc17270 2024-03-11T19:47:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.859+0000 7fbaada1d640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba940a1a30 0x7fba940a3ef0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55ea2bc18060 tx=0x7fbaa4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.859+0000 7fba93fff640 1 -- 172.21.15.83:0/3197524357 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbaa000fe80 con 0x55ea2bc17270 2024-03-11T19:47:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.859+0000 7fba93fff640 1 -- 172.21.15.83:0/3197524357 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0010690 con 0x55ea2bc17270 2024-03-11T19:47:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.860+0000 7fbab284f740 1 -- 172.21.15.83:0/3197524357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 msgr2=0x55ea2bc17690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.860+0000 7fbab284f740 1 --2- 172.21.15.83:0/3197524357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc17690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00117e0 tx=0x7fbaa0011ca0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.860+0000 7fbab284f740 1 -- 172.21.15.83:0/3197524357 shutdown_connections 2024-03-11T19:47:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.860+0000 7fbab284f740 1 --2- 172.21.15.83:0/3197524357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc1f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.860+0000 7fbab284f740 1 --2- 172.21.15.83:0/3197524357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc1c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.860+0000 7fbab284f740 1 --2- 172.21.15.83:0/3197524357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc17690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.860+0000 7fbab284f740 1 -- 172.21.15.83:0/3197524357 >> 172.21.15.83:0/3197524357 conn(0x55ea2bc127e0 msgr2=0x55ea2bc12be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.860+0000 7fbab284f740 1 -- 172.21.15.83:0/3197524357 shutdown_connections 2024-03-11T19:47:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.860+0000 7fbab284f740 1 -- 172.21.15.83:0/3197524357 wait complete. 2024-03-11T19:47:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.860+0000 7fbab284f740 1 -- start start 2024-03-11T19:47:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbab284f740 1 Processor -- start 2024-03-11T19:47:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbab284f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc21460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbab284f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbab284f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc289e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbab284f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2bb80020 con 0x55ea2bc1cdd0 2024-03-11T19:47:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbab284f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2b853430 con 0x55ea2bc17270 2024-03-11T19:47:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbab284f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2b80fe00 con 0x55ea2bc17c60 2024-03-11T19:47:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbaada1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc21460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbaada1d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc21460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50668/0 (socket says 172.21.15.83:50668) 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbaada1d640 1 -- 172.21.15.83:0/1997562632 learned_addr learned my addr 172.21.15.83:0/1997562632 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbaae21e640 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbaad21c640 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.861+0000 7fbab284f740 10 client.0 _ll_register_callbacks cb 0x55ea2bb27cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fbaada1d640 1 -- 172.21.15.83:0/1997562632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 msgr2=0x55ea2bc219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fbaada1d640 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fbaada1d640 1 -- 172.21.15.83:0/1997562632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc1cdd0 msgr2=0x55ea2bc289e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fbaada1d640 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc289e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fbaada1d640 1 -- 172.21.15.83:0/1997562632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa00114c0 con 0x55ea2bc17270 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fbaad21c640 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fbab284f740 1 -- 172.21.15.83:0/1997562632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea2b87f7e0 con 0x55ea2bc17270 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fbaada1d640 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc21460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4013010 tx=0x7fbaa4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7c25ea3749fbf08 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fba917fa640 10 client.26318 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:47:16.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fba917fa640 1 -- 172.21.15.83:0/1997562632 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4022070 con 0x55ea2bc17270 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fba917fa640 1 -- 172.21.15.83:0/1997562632 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fbaa4028420 con 0x55ea2bc17270 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fba917fa640 1 -- 172.21.15.83:0/1997562632 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaa4017070 con 0x55ea2bc17270 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fbab284f740 10 client.26318 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.862+0000 7fbab284f740 1 -- 172.21.15.83:0/1997562632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=31}) v3 -- 0x55ea2bb7e6a0 con 0x55ea2bc17270 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.863+0000 7fba917fa640 1 -- 172.21.15.83:0/1997562632 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fbaa400f3c0 con 0x55ea2bc17270 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.863+0000 7fbab284f740 10 client.26318 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.863+0000 7fbab284f740 10 client.26318 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.863+0000 7fbab284f740 20 client.26318 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.863+0000 7fbab284f740 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55ea2bc2d490 0x55ea2bc2f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.863+0000 7fbab284f740 4 client.26318 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.863+0000 7fbab284f740 1 -- 172.21.15.83:0/1997562632 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea2b8bc720 con 0x55ea2bc2d490 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.864+0000 7fbaad21c640 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55ea2bc2d490 0x55ea2bc2f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.864+0000 7fba917fa640 10 client.26318 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.864+0000 7fbaad21c640 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55ea2bc2d490 0x55ea2bc2f8c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6de836c1994f993 server_cookie=ce30e3591e7ba6b7 in_seq=0 out_seq=0 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fba917fa640 1 -- 172.21.15.83:0/1997562632 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea2b8bc720 con 0x55ea2bc2d490 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fba917fa640 10 client.26318 handle_command_reply: tid=0 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 1 client.26318 shutdown 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 1 -- 172.21.15.83:0/1997562632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 msgr2=0x55ea2bc21460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc21460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4013010 tx=0x7fbaa4004730 comp rx=0 tx=0).stop 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 1 -- 172.21.15.83:0/1997562632 shutdown_connections 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55ea2bc2d490 0x55ea2bc2f8c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fba917fa640 0 client.26318 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 1 --2- 172.21.15.83:0/1997562632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc17270 0x55ea2bc21460 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 1 -- 172.21.15.83:0/1997562632 >> 172.21.15.83:0/1997562632 conn(0x55ea2bc127e0 msgr2=0x55ea2bc11b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 1 -- 172.21.15.83:0/1997562632 shutdown_connections 2024-03-11T19:47:16.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 1 -- 172.21.15.83:0/1997562632 wait complete. 2024-03-11T19:47:16.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.865+0000 7fbab284f740 20 client.26318 trim_cache size 0 max 16384 2024-03-11T19:47:16.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.879+0000 7fbab284f740 1 Processor -- start 2024-03-11T19:47:16.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.879+0000 7fbab284f740 1 -- start start 2024-03-11T19:47:16.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.879+0000 7fbab284f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc17270 0x55ea2bc21070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.879+0000 7fbab284f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbab284f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc215b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbab284f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2bb05130 con 0x55ea2bc17270 2024-03-11T19:47:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbab284f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2bc13230 con 0x55ea2bc1cdd0 2024-03-11T19:47:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbab284f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2bc0e120 con 0x55ea2bc17c60 2024-03-11T19:47:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbaad21c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbaad21c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc289a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59544/0 (socket says 172.21.15.83:59544) 2024-03-11T19:47:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbaae21e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbaad21c640 1 -- 172.21.15.83:0/3963095283 learned_addr learned my addr 172.21.15.83:0/3963095283 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbaad21c640 1 -- 172.21.15.83:0/3963095283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc1cdd0 msgr2=0x55ea2bc215b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbaad21c640 1 --2- 172.21.15.83:0/3963095283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc215b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbaad21c640 1 -- 172.21.15.83:0/3963095283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc17270 msgr2=0x55ea2bc21070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbaad21c640 1 --2- 172.21.15.83:0/3963095283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc17270 0x55ea2bc21070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbaad21c640 1 -- 172.21.15.83:0/3963095283 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2bb1ac20 con 0x55ea2bc17c60 2024-03-11T19:47:16.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.880+0000 7fbaad21c640 1 --2- 172.21.15.83:0/3963095283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc289a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0bbdf0 tx=0x7fba9c02c130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7dd7087969b3b2b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.881+0000 7fba937fe640 1 -- 172.21.15.83:0/3963095283 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0bbc30 con 0x55ea2bc17c60 2024-03-11T19:47:16.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.881+0000 7fba937fe640 1 -- 172.21.15.83:0/3963095283 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba9c0bb9a0 con 0x55ea2bc17c60 2024-03-11T19:47:16.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.881+0000 7fba937fe640 1 -- 172.21.15.83:0/3963095283 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0ba160 con 0x55ea2bc17c60 2024-03-11T19:47:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.882+0000 7fbab284f740 1 -- 172.21.15.83:0/3963095283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 msgr2=0x55ea2bc289a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.882+0000 7fbab284f740 1 --2- 172.21.15.83:0/3963095283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc289a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0bbdf0 tx=0x7fba9c02c130 comp rx=0 tx=0).stop 2024-03-11T19:47:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.882+0000 7fbab284f740 1 -- 172.21.15.83:0/3963095283 shutdown_connections 2024-03-11T19:47:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.882+0000 7fbab284f740 1 --2- 172.21.15.83:0/3963095283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc215b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.882+0000 7fbab284f740 1 --2- 172.21.15.83:0/3963095283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17c60 0x55ea2bc289a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.882+0000 7fbab284f740 1 --2- 172.21.15.83:0/3963095283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc17270 0x55ea2bc21070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.882+0000 7fbab284f740 1 -- 172.21.15.83:0/3963095283 >> 172.21.15.83:0/3963095283 conn(0x55ea2bc127e0 msgr2=0x55ea2bc0da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.882+0000 7fbab284f740 1 -- 172.21.15.83:0/3963095283 shutdown_connections 2024-03-11T19:47:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.882+0000 7fbab284f740 1 -- 172.21.15.83:0/3963095283 wait complete. 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbab284f740 1 -- start start 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbab284f740 1 Processor -- start 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbab284f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17270 0x55ea2bc27a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbab284f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc17c60 0x55ea2bc27f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbab284f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc839a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbab284f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2bc83ee0 con 0x55ea2bc17c60 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbab284f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2bc84020 con 0x55ea2bc1cdd0 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbab284f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2bc84160 con 0x55ea2bc17270 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbaad21c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc17c60 0x55ea2bc27f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbab284f740 10 client.0 _ll_register_callbacks cb 0x7fbaa4009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbaad21c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc17c60 0x55ea2bc27f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41426/0 (socket says 172.21.15.83:41426) 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbaad21c640 1 -- 172.21.15.83:0/98010925 learned_addr learned my addr 172.21.15.83:0/98010925 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbaada1d640 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17270 0x55ea2bc27a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.883+0000 7fbaae21e640 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc839a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fbaada1d640 1 -- 172.21.15.83:0/98010925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc1cdd0 msgr2=0x55ea2bc839a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fbaada1d640 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc839a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fbaada1d640 1 -- 172.21.15.83:0/98010925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc17c60 msgr2=0x55ea2bc27f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fbaada1d640 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc17c60 0x55ea2bc27f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fbaada1d640 1 -- 172.21.15.83:0/98010925 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa81a1840 con 0x55ea2bc17270 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fbaada1d640 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17270 0x55ea2bc27a10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400ffd0 tx=0x7fbaa401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=431d166848e0fd80 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fba917fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fba917fa640 1 -- 172.21.15.83:0/98010925 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4017040 con 0x55ea2bc17270 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fba917fa640 1 -- 172.21.15.83:0/98010925 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fbaa401d7b0 con 0x55ea2bc17270 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fbab284f740 1 -- 172.21.15.83:0/98010925 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbaa000fe80 con 0x55ea2bc17270 2024-03-11T19:47:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fba917fa640 1 -- 172.21.15.83:0/98010925 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaa4005b50 con 0x55ea2bc17270 2024-03-11T19:47:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fbab284f740 10 client.26434 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fbab284f740 1 -- 172.21.15.83:0/98010925 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ea2bb408c0 con 0x55ea2bc17270 2024-03-11T19:47:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fba917fa640 1 -- 172.21.15.83:0/98010925 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbaa4041070 con 0x55ea2bc17270 2024-03-11T19:47:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.884+0000 7fba917fa640 1 -- 172.21.15.83:0/98010925 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fbaa4044070 con 0x55ea2bc17270 2024-03-11T19:47:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.885+0000 7fbab284f740 10 client.26434 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.885+0000 7fbab284f740 10 client.26434 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:47:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.885+0000 7fbab284f740 20 client.26434 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.885+0000 7fbab284f740 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55ea2bc2d490 0x55ea2bc87bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.885+0000 7fbab284f740 4 client.26434 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:47:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.885+0000 7fbab284f740 1 -- 172.21.15.83:0/98010925 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fbaa000fe80 con 0x55ea2bc2d490 2024-03-11T19:47:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.885+0000 7fbaad21c640 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55ea2bc2d490 0x55ea2bc87bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbaad21c640 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55ea2bc2d490 0x55ea2bc87bf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63e5b75eed3e0b87 server_cookie=1233b6d83983f868 in_seq=0 out_seq=0 2024-03-11T19:47:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fba917fa640 10 client.26434 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:47:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fba917fa640 1 -- 172.21.15.83:0/98010925 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fbaa000fe80 con 0x55ea2bc2d490 2024-03-11T19:47:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fba917fa640 10 client.26434 handle_command_reply: tid=0 2024-03-11T19:47:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 1 client.26434 shutdown 2024-03-11T19:47:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 1 -- 172.21.15.83:0/98010925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17270 msgr2=0x55ea2bc27a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17270 0x55ea2bc27a10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400ffd0 tx=0x7fbaa401c000 comp rx=0 tx=0).stop 2024-03-11T19:47:16.895 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:16 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/752516777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:47:16.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 1 -- 172.21.15.83:0/98010925 shutdown_connections 2024-03-11T19:47:16.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55ea2bc2d490 0x55ea2bc87bf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ea2bc1cdd0 0x55ea2bc839a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea2bc17c60 0x55ea2bc27f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 1 --2- 172.21.15.83:0/98010925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea2bc17270 0x55ea2bc27a10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 1 -- 172.21.15.83:0/98010925 >> 172.21.15.83:0/98010925 conn(0x55ea2bc127e0 msgr2=0x55ea2bc20fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fba917fa640 0 client.26434 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:47:16.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 1 -- 172.21.15.83:0/98010925 shutdown_connections 2024-03-11T19:47:16.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 1 -- 172.21.15.83:0/98010925 wait complete. 2024-03-11T19:47:16.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.886+0000 7fbab284f740 20 client.26434 trim_cache size 0 max 16384 2024-03-11T19:47:16.896 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:47:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 -- 172.21.15.83:0/661160731 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba940a1a30 msgr2=0x7fba940a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba940a1a30 0x7fba940a3ef0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55ea2bc18060 tx=0x7fbaa4009260 comp rx=0 tx=0).stop 2024-03-11T19:47:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 -- 172.21.15.83:0/661160731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaa8101270 msgr2=0x7fbaa8199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaa8101270 0x7fbaa8199970 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fba9c02c0b0 tx=0x7fba9c0045a0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 -- 172.21.15.83:0/661160731 shutdown_connections 2024-03-11T19:47:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fba940a1a30 0x7fba940a3ef0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaa8101c30 0x7fbaa81a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaa8101270 0x7fbaa8199970 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 --2- 172.21.15.83:0/661160731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaa8100080 0x7fbaa8199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 -- 172.21.15.83:0/661160731 >> 172.21.15.83:0/661160731 conn(0x7fbaa80fb610 msgr2=0x7fbaa80682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 -- 172.21.15.83:0/661160731 shutdown_connections 2024-03-11T19:47:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:16.887+0000 7fbaafc81640 1 -- 172.21.15.83:0/661160731 wait complete. 2024-03-11T19:47:16.904 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:47:16.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:47:16.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:16 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/752516777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:47:17.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.055+0000 7f1c7089a640 1 -- 172.21.15.83:0/986194400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6c100d70 msgr2=0x7f1c6c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:17.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.055+0000 7f1c7089a640 1 --2- 172.21.15.83:0/986194400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6c100d70 0x7f1c6c1011f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009ca0 tx=0x7f1c6402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.056+0000 7f1c7089a640 1 -- 172.21.15.83:0/986194400 shutdown_connections 2024-03-11T19:47:17.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.056+0000 7f1c7089a640 1 --2- 172.21.15.83:0/986194400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6c100d70 0x7f1c6c1011f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.056+0000 7f1c7089a640 1 --2- 172.21.15.83:0/986194400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c1003d0 0x7f1c6c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.056+0000 7f1c7089a640 1 --2- 172.21.15.83:0/986194400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6c0ffa70 0x7f1c6c0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.056+0000 7f1c7089a640 1 -- 172.21.15.83:0/986194400 >> 172.21.15.83:0/986194400 conn(0x7f1c6c0fb630 msgr2=0x7f1c6c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:17.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.057+0000 7f1c7089a640 1 -- 172.21.15.83:0/986194400 shutdown_connections 2024-03-11T19:47:17.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.057+0000 7f1c7089a640 1 -- 172.21.15.83:0/986194400 wait complete. 2024-03-11T19:47:17.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c7089a640 1 Processor -- start 2024-03-11T19:47:17.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c7089a640 1 -- start start 2024-03-11T19:47:17.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c7089a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6c0ffa70 0x7f1c6c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c7089a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6c1003d0 0x7f1c6c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c7089a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c100d70 0x7f1c6c1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c7089a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c1a0ce0 con 0x7f1c6c1003d0 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c7089a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c1a0e50 con 0x7f1c6c100d70 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c7089a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c1a1010 con 0x7f1c6c0ffa70 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6c1003d0 0x7f1c6c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6c1003d0 0x7f1c6c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41438/0 (socket says 172.21.15.83:41438) 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c69d9b640 1 -- 172.21.15.83:0/1724180821 learned_addr learned my addr 172.21.15.83:0/1724180821 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.058+0000 7f1c6a59c640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6c0ffa70 0x7f1c6c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c100d70 0x7f1c6c1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c6a59c640 1 -- 172.21.15.83:0/1724180821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c100d70 msgr2=0x7f1c6c1a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c6a59c640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c100d70 0x7f1c6c1a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c6a59c640 1 -- 172.21.15.83:0/1724180821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6c1003d0 msgr2=0x7f1c6c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c6a59c640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6c1003d0 0x7f1c6c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c6a59c640 1 -- 172.21.15.83:0/1724180821 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c64009900 con 0x7f1c6c0ffa70 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c6a59c640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6c0ffa70 0x7f1c6c1991f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e9f0 tx=0x7f1c5c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c5b7fe640 1 -- 172.21.15.83:0/1724180821 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00ccc0 con 0x7f1c6c0ffa70 2024-03-11T19:47:17.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c7089a640 1 -- 172.21.15.83:0/1724180821 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c1a12f0 con 0x7f1c6c0ffa70 2024-03-11T19:47:17.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c5b7fe640 1 -- 172.21.15.83:0/1724180821 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c5c00ce20 con 0x7f1c6c0ffa70 2024-03-11T19:47:17.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c5b7fe640 1 -- 172.21.15.83:0/1724180821 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c004a00 con 0x7f1c6c0ffa70 2024-03-11T19:47:17.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.059+0000 7f1c7089a640 1 -- 172.21.15.83:0/1724180821 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c1a1800 con 0x7f1c6c0ffa70 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.062+0000 7f1c5b7fe640 1 -- 172.21.15.83:0/1724180821 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1c5c004b60 con 0x7f1c6c0ffa70 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.062+0000 7f1c7346e740 1 Processor -- start 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.062+0000 7f1c7346e740 1 -- start start 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.062+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2e270 0x55b867b2e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.062+0000 7f1c5b7fe640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c540a1a30 0x7f1c540a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.062+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2ec60 0x55b867b33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.062+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b33dd0 0x55b867b36260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.062+0000 7f1c7346e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b867aea960 con 0x55b867b2e270 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.062+0000 7f1c7346e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b867a9e310 con 0x55b867b33dd0 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.062+0000 7f1c7346e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b867a9b840 con 0x55b867b2ec60 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c5b7fe640 1 -- 172.21.15.83:0/1724180821 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1c5c014070 con 0x7f1c6c0ffa70 2024-03-11T19:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c69d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2ec60 0x55b867b33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c6ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2e270 0x55b867b2e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c6a59c640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c540a1a30 0x7f1c540a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c69d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2ec60 0x55b867b33760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59566/0 (socket says 172.21.15.83:59566) 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c6ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2e270 0x55b867b2e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41450/0 (socket says 172.21.15.83:41450) 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c69d9b640 1 -- 172.21.15.83:0/1077979577 learned_addr learned my addr 172.21.15.83:0/1077979577 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/1077979577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b33dd0 0x55b867b36260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c69d9b640 1 -- 172.21.15.83:0/1077979577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b33dd0 msgr2=0x55b867b36260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c69d9b640 1 --2- 172.21.15.83:0/1077979577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b33dd0 0x55b867b36260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c69d9b640 1 -- 172.21.15.83:0/1077979577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2e270 msgr2=0x55b867b2e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c69d9b640 1 --2- 172.21.15.83:0/1077979577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2e270 0x55b867b2e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c69d9b640 1 -- 172.21.15.83:0/1077979577 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b867a59060 con 0x55b867b2ec60 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c6a59c640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c540a1a30 0x7f1c540a3ef0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0cae80 tx=0x7f1c5c0054b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c69d9b640 1 --2- 172.21.15.83:0/1077979577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2ec60 0x55b867b33760 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c60004630 tx=0x7f1c6000bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46385bab23344f0e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c58ff9640 1 -- 172.21.15.83:0/1077979577 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60004130 con 0x55b867b2ec60 2024-03-11T19:47:17.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c58ff9640 1 -- 172.21.15.83:0/1077979577 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c60003dd0 con 0x55b867b2ec60 2024-03-11T19:47:17.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.063+0000 7f1c58ff9640 1 -- 172.21.15.83:0/1077979577 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c600117a0 con 0x55b867b2ec60 2024-03-11T19:47:17.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.064+0000 7f1c7346e740 1 -- 172.21.15.83:0/1077979577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2ec60 msgr2=0x55b867b33760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:17.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.064+0000 7f1c7346e740 1 --2- 172.21.15.83:0/1077979577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2ec60 0x55b867b33760 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c60004630 tx=0x7f1c6000bc70 comp rx=0 tx=0).stop 2024-03-11T19:47:17.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.064+0000 7f1c7346e740 1 -- 172.21.15.83:0/1077979577 shutdown_connections 2024-03-11T19:47:17.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.064+0000 7f1c7346e740 1 --2- 172.21.15.83:0/1077979577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b33dd0 0x55b867b36260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.064+0000 7f1c7346e740 1 --2- 172.21.15.83:0/1077979577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2ec60 0x55b867b33760 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.064+0000 7f1c7346e740 1 --2- 172.21.15.83:0/1077979577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2e270 0x55b867b2e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.064+0000 7f1c7346e740 1 -- 172.21.15.83:0/1077979577 >> 172.21.15.83:0/1077979577 conn(0x55b867b297e0 msgr2=0x55b867b29be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.064+0000 7f1c7346e740 1 -- 172.21.15.83:0/1077979577 shutdown_connections 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.064+0000 7f1c7346e740 1 -- 172.21.15.83:0/1077979577 wait complete. 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c7346e740 1 -- start start 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c7346e740 1 Processor -- start 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b38450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2ec60 0x55b867b38990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b33dd0 0x55b867b3f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c7346e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b867a97020 con 0x55b867b2ec60 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c7346e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86776a430 con 0x55b867b33dd0 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c7346e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b867726e00 con 0x55b867b2e270 2024-03-11T19:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c7346e740 10 client.0 _ll_register_callbacks cb 0x55b867a3ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2ec60 0x55b867b38990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2ec60 0x55b867b38990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41466/0 (socket says 172.21.15.83:41466) 2024-03-11T19:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c6ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b38450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c69d9b640 1 -- 172.21.15.83:0/2764009218 learned_addr learned my addr 172.21.15.83:0/2764009218 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b33dd0 0x55b867b3f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c69d9b640 1 -- 172.21.15.83:0/2764009218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 msgr2=0x55b867b38450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c69d9b640 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b38450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c69d9b640 1 -- 172.21.15.83:0/2764009218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b33dd0 msgr2=0x55b867b3f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c69d9b640 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b33dd0 0x55b867b3f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c69d9b640 1 -- 172.21.15.83:0/2764009218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c60004290 con 0x55b867b2ec60 2024-03-11T19:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.065+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b38450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:17.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.068+0000 7f1c69d9b640 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2ec60 0x55b867b38990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1c60000c00 tx=0x7f1c6001a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8707dc4e0fe231 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.068+0000 7f1c51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.068+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2764009218 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60004110 con 0x55b867b2ec60 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.068+0000 7f1c7346e740 1 -- 172.21.15.83:0/2764009218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8677967e0 con 0x55b867b2ec60 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.069+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2764009218 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1c6001ab20 con 0x55b867b2ec60 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.069+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2764009218 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c6002d030 con 0x55b867b2ec60 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.069+0000 7f1c7346e740 10 client.16875 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.069+0000 7f1c7346e740 1 -- 172.21.15.83:0/2764009218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b867a956a0 con 0x55b867b2ec60 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.069+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2764009218 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b8677967e0 con 0x55b867b2ec60 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.069+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2764009218 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1c60004c10 con 0x55b867b2ec60 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.069+0000 7f1c7346e740 10 client.16875 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.069+0000 7f1c7346e740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:17.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.069+0000 7f1c7346e740 20 client.16875 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.070+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b867b44630 0x55b867b46a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.070+0000 7f1c7346e740 4 client.16875 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.070+0000 7f1c7346e740 1 -- 172.21.15.83:0/2764009218 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8677d3720 con 0x55b867b44630 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.070+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b867b44630 0x55b867b46a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.070+0000 7f1c51ffb640 10 client.16875 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.070+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b867b44630 0x55b867b46a50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5520fda07d9014 server_cookie=c9c8d4e81903a288 in_seq=0 out_seq=0 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.071+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2764009218 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8677d3720 con 0x55b867b44630 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.071+0000 7f1c51ffb640 10 client.16875 handle_command_reply: tid=0 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.071+0000 7f1c7346e740 1 client.16875 shutdown 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.071+0000 7f1c7346e740 1 -- 172.21.15.83:0/2764009218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2ec60 msgr2=0x55b867b38990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.071+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2ec60 0x55b867b38990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1c60000c00 tx=0x7f1c6001a720 comp rx=0 tx=0).stop 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.071+0000 7f1c7346e740 1 -- 172.21.15.83:0/2764009218 shutdown_connections 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.071+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b867b44630 0x55b867b46a50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.071+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b33dd0 0x55b867b3f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.071+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b2ec60 0x55b867b38990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.071+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2764009218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b38450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.071+0000 7f1c7346e740 1 -- 172.21.15.83:0/2764009218 >> 172.21.15.83:0/2764009218 conn(0x55b867b297e0 msgr2=0x55b867b28b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.072+0000 7f1c51ffb640 0 client.16875 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.072+0000 7f1c7346e740 1 -- 172.21.15.83:0/2764009218 shutdown_connections 2024-03-11T19:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.072+0000 7f1c7346e740 1 -- 172.21.15.83:0/2764009218 wait complete. 2024-03-11T19:47:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.072+0000 7f1c7346e740 20 client.16875 trim_cache size 0 max 16384 2024-03-11T19:47:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.086+0000 7f1c7346e740 1 Processor -- start 2024-03-11T19:47:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c7346e740 1 -- start start 2024-03-11T19:47:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b301b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b2ec60 0x55b867b306f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b33dd0 0x55b867b3f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c7346e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b867a1c130 con 0x55b867b33dd0 2024-03-11T19:47:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c7346e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b867b39380 con 0x55b867b2ec60 2024-03-11T19:47:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c7346e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b867b40ec0 con 0x55b867b2e270 2024-03-11T19:47:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c6ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c6ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b301b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59584/0 (socket says 172.21.15.83:59584) 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c69d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b2ec60 0x55b867b306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c6ad9d640 1 -- 172.21.15.83:0/1395151346 learned_addr learned my addr 172.21.15.83:0/1395151346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/1395151346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b33dd0 0x55b867b3f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c6ad9d640 1 -- 172.21.15.83:0/1395151346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b2ec60 msgr2=0x55b867b306f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/1395151346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b2ec60 0x55b867b306f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c6ad9d640 1 -- 172.21.15.83:0/1395151346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b33dd0 msgr2=0x55b867b3f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/1395151346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b33dd0 0x55b867b3f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c6ad9d640 1 -- 172.21.15.83:0/1395151346 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b867a31c20 con 0x55b867b2e270 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c69d9b640 1 --2- 172.21.15.83:0/1395151346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b2ec60 0x55b867b306f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.087+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/1395151346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b301b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c640098d0 tx=0x7f1c64044040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d0e4ca717819eb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c53fff640 1 -- 172.21.15.83:0/1395151346 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6404a070 con 0x55b867b2e270 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c53fff640 1 -- 172.21.15.83:0/1395151346 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c64053070 con 0x55b867b2e270 2024-03-11T19:47:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c53fff640 1 -- 172.21.15.83:0/1395151346 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6404ea90 con 0x55b867b2e270 2024-03-11T19:47:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c7346e740 1 -- 172.21.15.83:0/1395151346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 msgr2=0x55b867b301b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c7346e740 1 --2- 172.21.15.83:0/1395151346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b301b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c640098d0 tx=0x7f1c64044040 comp rx=0 tx=0).stop 2024-03-11T19:47:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c7346e740 1 -- 172.21.15.83:0/1395151346 shutdown_connections 2024-03-11T19:47:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c7346e740 1 --2- 172.21.15.83:0/1395151346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b33dd0 0x55b867b3f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c7346e740 1 --2- 172.21.15.83:0/1395151346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b2ec60 0x55b867b306f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c7346e740 1 --2- 172.21.15.83:0/1395151346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b301b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c7346e740 1 -- 172.21.15.83:0/1395151346 >> 172.21.15.83:0/1395151346 conn(0x55b867b297e0 msgr2=0x55b867b4b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c7346e740 1 -- 172.21.15.83:0/1395151346 shutdown_connections 2024-03-11T19:47:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.088+0000 7f1c7346e740 1 -- 172.21.15.83:0/1395151346 wait complete. 2024-03-11T19:47:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.089+0000 7f1c7346e740 1 -- start start 2024-03-11T19:47:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.089+0000 7f1c7346e740 1 Processor -- start 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.089+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b3f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.089+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b2ec60 0x55b867b3df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.090+0000 7f1c7346e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b33dd0 0x55b867b3e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.090+0000 7f1c6ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b3f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.090+0000 7f1c6ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b3f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59594/0 (socket says 172.21.15.83:59594) 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.090+0000 7f1c6ad9d640 1 -- 172.21.15.83:0/2260477961 learned_addr learned my addr 172.21.15.83:0/2260477961 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.090+0000 7f1c7346e740 1 -- 172.21.15.83:0/2260477961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b867b3eaa0 con 0x55b867b33dd0 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.090+0000 7f1c7346e740 1 -- 172.21.15.83:0/2260477961 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b867b3ec10 con 0x55b867b2ec60 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.090+0000 7f1c7346e740 1 -- 172.21.15.83:0/2260477961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b867b3edd0 con 0x55b867b2e270 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.090+0000 7f1c7346e740 10 client.0 _ll_register_callbacks cb 0x7f1c60004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.091+0000 7f1c69d9b640 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b2ec60 0x55b867b3df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.091+0000 7f1c6ad9d640 1 -- 172.21.15.83:0/2260477961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b2ec60 msgr2=0x55b867b3df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.091+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b2ec60 0x55b867b3df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.091+0000 7f1c6ad9d640 1 -- 172.21.15.83:0/2260477961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b33dd0 msgr2=0x55b867b3e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.091+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b33dd0 0x55b867b3e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.091+0000 7f1c6ad9d640 1 -- 172.21.15.83:0/2260477961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b867a956a0 con 0x55b867b2e270 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.091+0000 7f1c6ad9d640 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b3f8f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c6402c630 tx=0x7f1c64003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d3329d1ff55046 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.091+0000 7f1c51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.091+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2260477961 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6404a070 con 0x55b867b2e270 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c7346e740 1 -- 172.21.15.83:0/2260477961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c60003dd0 con 0x55b867b2e270 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2260477961 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1c64032720 con 0x55b867b2e270 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2260477961 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c64047040 con 0x55b867b2e270 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c7346e740 10 client.26452 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c7346e740 1 -- 172.21.15.83:0/2260477961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f1c6c10c0c0 con 0x55b867b2e270 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2260477961 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c64068020 con 0x55b867b2e270 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2260477961 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f1c6404f050 con 0x55b867b2e270 2024-03-11T19:47:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c7346e740 10 client.26452 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c7346e740 10 client.26452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c7346e740 20 client.26452 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b867b9c1b0 0x55b867b9e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c7346e740 4 client.26452 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.092+0000 7f1c7346e740 1 -- 172.21.15.83:0/2260477961 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b867822890 con 0x55b867b9c1b0 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c69d9b640 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b867b9c1b0 0x55b867b9e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c51ffb640 10 client.26452 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c69d9b640 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b867b9c1b0 0x55b867b9e5d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06293ddae2c7cc4 server_cookie=a82ee62f1d5b504a in_seq=0 out_seq=0 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c51ffb640 1 -- 172.21.15.83:0/2260477961 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b867822890 con 0x55b867b9c1b0 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c51ffb640 10 client.26452 handle_command_reply: tid=0 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c7346e740 1 client.26452 shutdown 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c7346e740 1 -- 172.21.15.83:0/2260477961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 msgr2=0x55b867b3f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b3f8f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c6402c630 tx=0x7f1c64003970 comp rx=0 tx=0).stop 2024-03-11T19:47:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c7346e740 1 -- 172.21.15.83:0/2260477961 shutdown_connections 2024-03-11T19:47:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b867b9c1b0 0x55b867b9e5d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b867b33dd0 0x55b867b3e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b867b2ec60 0x55b867b3df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c51ffb640 0 client.26452 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c7346e740 1 --2- 172.21.15.83:0/2260477961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b867b2e270 0x55b867b3f8f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c7346e740 1 -- 172.21.15.83:0/2260477961 >> 172.21.15.83:0/2260477961 conn(0x55b867b297e0 msgr2=0x55b867b2ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c7346e740 1 -- 172.21.15.83:0/2260477961 shutdown_connections 2024-03-11T19:47:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.093+0000 7f1c7346e740 1 -- 172.21.15.83:0/2260477961 wait complete. 2024-03-11T19:47:17.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7346e740 20 client.26452 trim_cache size 0 max 16384 2024-03-11T19:47:17.102 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:47:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7089a640 1 -- 172.21.15.83:0/1724180821 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c540a1a30 msgr2=0x7f1c540a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7089a640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c540a1a30 0x7f1c540a3ef0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0cae80 tx=0x7f1c5c0054b0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7089a640 1 -- 172.21.15.83:0/1724180821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6c0ffa70 msgr2=0x7f1c6c1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7089a640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6c0ffa70 0x7f1c6c1991f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e9f0 tx=0x7f1c5c00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7089a640 1 -- 172.21.15.83:0/1724180821 shutdown_connections 2024-03-11T19:47:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7089a640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1c540a1a30 0x7f1c540a3ef0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7089a640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c6c100d70 0x7f1c6c1a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7089a640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c6c1003d0 0x7f1c6c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7089a640 1 --2- 172.21.15.83:0/1724180821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c6c0ffa70 0x7f1c6c1991f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:17.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7089a640 1 -- 172.21.15.83:0/1724180821 >> 172.21.15.83:0/1724180821 conn(0x7f1c6c0fb630 msgr2=0x7f1c6c0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:17.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.094+0000 7f1c7089a640 1 -- 172.21.15.83:0/1724180821 shutdown_connections 2024-03-11T19:47:17.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:17.095+0000 7f1c7089a640 1 -- 172.21.15.83:0/1724180821 wait complete. 2024-03-11T19:47:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 480.0 s, 1721.0 tps, lat 58.359 ms stddev 65.795 2024-03-11T19:47:17.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:17 smithi083 ceph-mon[31864]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.5 MiB/s wr, 574 op/s 2024-03-11T19:47:17.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:17 smithi137 ceph-mon[36220]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.5 MiB/s wr, 574 op/s 2024-03-11T19:47:17.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:17 smithi204 ceph-mon[36739]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.5 MiB/s wr, 574 op/s 2024-03-11T19:47:18.112 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:47:18.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:47:18.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:47:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.254+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/975905972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c810d760 msgr2=0x7fc2c810fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.254+0000 7fc2ce1c7640 1 --2- 172.21.15.83:0/975905972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c810d760 0x7fc2c810fb50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009cd0 tx=0x7fc2c002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.254+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/975905972 shutdown_connections 2024-03-11T19:47:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.254+0000 7fc2ce1c7640 1 --2- 172.21.15.83:0/975905972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c810d760 0x7fc2c810fb50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.254+0000 7fc2ce1c7640 1 --2- 172.21.15.83:0/975905972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c8073be0 0x7fc2c8074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.254+0000 7fc2ce1c7640 1 --2- 172.21.15.83:0/975905972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2c80751c0 0x7fc2c8073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.254+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/975905972 >> 172.21.15.83:0/975905972 conn(0x7fc2c80fc700 msgr2=0x7fc2c80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.254+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/975905972 shutdown_connections 2024-03-11T19:47:18.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.254+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/975905972 wait complete. 2024-03-11T19:47:18.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.255+0000 7fc2ce1c7640 1 Processor -- start 2024-03-11T19:47:18.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.255+0000 7fc2ce1c7640 1 -- start start 2024-03-11T19:47:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.255+0000 7fc2ce1c7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2c8073be0 0x7fc2c819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.256+0000 7fc2ce1c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c80751c0 0x7fc2c819dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.256+0000 7fc2ce1c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c810d760 0x7fc2c81a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.256+0000 7fc2c77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2c8073be0 0x7fc2c819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.256+0000 7fc2c77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2c8073be0 0x7fc2c819d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59610/0 (socket says 172.21.15.83:59610) 2024-03-11T19:47:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.256+0000 7fc2c6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c80751c0 0x7fc2c819dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.256+0000 7fc2c6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c80751c0 0x7fc2c819dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50764/0 (socket says 172.21.15.83:50764) 2024-03-11T19:47:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.256+0000 7fc2c77fe640 1 -- 172.21.15.83:0/3397851477 learned_addr learned my addr 172.21.15.83:0/3397851477 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.256+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/3397851477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81a5320 con 0x7fc2c810d760 2024-03-11T19:47:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.256+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/3397851477 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81a5490 con 0x7fc2c80751c0 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.256+0000 7fc2c7fff640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c810d760 0x7fc2c81a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.256+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/3397851477 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81a5650 con 0x7fc2c8073be0 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.257+0000 7fc2c77fe640 1 -- 172.21.15.83:0/3397851477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c80751c0 msgr2=0x7fc2c819dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.257+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c80751c0 0x7fc2c819dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.257+0000 7fc2c77fe640 1 -- 172.21.15.83:0/3397851477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c810d760 msgr2=0x7fc2c81a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.257+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c810d760 0x7fc2c81a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.257+0000 7fc2c77fe640 1 -- 172.21.15.83:0/3397851477 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c0009900 con 0x7fc2c8073be0 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.257+0000 7fc2c7fff640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c810d760 0x7fc2c81a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.258+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2c8073be0 0x7fc2c819d830 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800e9f0 tx=0x7fc2b800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.258+0000 7fc2c4ff9640 1 -- 172.21.15.83:0/3397851477 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8009720 con 0x7fc2c8073be0 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.258+0000 7fc2c4ff9640 1 -- 172.21.15.83:0/3397851477 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2b800cd80 con 0x7fc2c8073be0 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.258+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/3397851477 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c81a5930 con 0x7fc2c8073be0 2024-03-11T19:47:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.258+0000 7fc2c4ff9640 1 -- 172.21.15.83:0/3397851477 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8010690 con 0x7fc2c8073be0 2024-03-11T19:47:18.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.258+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/3397851477 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c81a5e40 con 0x7fc2c8073be0 2024-03-11T19:47:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.260+0000 7fc2c4ff9640 1 -- 172.21.15.83:0/3397851477 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc2b80046c0 con 0x7fc2c8073be0 2024-03-11T19:47:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.260+0000 7fc2d0d96740 1 Processor -- start 2024-03-11T19:47:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.260+0000 7fc2d0d96740 1 -- start start 2024-03-11T19:47:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.261+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 0x55ed79dcb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.261+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79dccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.261+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 0x55ed79dd3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.261+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79d3af90 con 0x55ed79dcd0b0 2024-03-11T19:47:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.261+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79d384c0 con 0x55ed79dcb520 2024-03-11T19:47:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.261+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79d33ca0 con 0x55ed79dcc6f0 2024-03-11T19:47:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.261+0000 7fc2c4ff9640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc2b00a1ab0 0x7fc2b00a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.261+0000 7fc2c4ff9640 1 -- 172.21.15.83:0/3397851477 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc2b8014070 con 0x7fc2c8073be0 2024-03-11T19:47:18.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.262+0000 7fc2c77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79dccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.262+0000 7fc2c7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 0x55ed79dcb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.262+0000 7fc2c7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 0x55ed79dcb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50768/0 (socket says 172.21.15.83:50768) 2024-03-11T19:47:18.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.262+0000 7fc2c7fff640 1 -- 172.21.15.83:0/3925780456 learned_addr learned my addr 172.21.15.83:0/3925780456 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.262+0000 7fc2c6ffd640 1 --2- 172.21.15.83:0/3925780456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 0x55ed79dd3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.262+0000 7fc2c7fff640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc2b00a1ab0 0x7fc2b00a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.262+0000 7fc2c77fe640 1 -- 172.21.15.83:0/3925780456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 msgr2=0x55ed79dcb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.262+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/3925780456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 0x55ed79dcb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.262+0000 7fc2c77fe640 1 -- 172.21.15.83:0/3925780456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 msgr2=0x55ed79dd3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.262+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/3925780456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 0x55ed79dd3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.262+0000 7fc2c77fe640 1 -- 172.21.15.83:0/3925780456 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed79cf60b0 con 0x55ed79dcc6f0 2024-03-11T19:47:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.263+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/3925780456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79dccb70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80cb250 tx=0x7fc2b809a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0ab1303debde server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.263+0000 7fc2adffb640 1 -- 172.21.15.83:0/3925780456 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b809e7c0 con 0x55ed79dcc6f0 2024-03-11T19:47:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.263+0000 7fc2c7fff640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc2b00a1ab0 0x7fc2b00a3f70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0032540 tx=0x7fc2c002c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.263+0000 7fc2adffb640 1 -- 172.21.15.83:0/3925780456 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2b80a2e10 con 0x55ed79dcc6f0 2024-03-11T19:47:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.263+0000 7fc2adffb640 1 -- 172.21.15.83:0/3925780456 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80ba0d0 con 0x55ed79dcc6f0 2024-03-11T19:47:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.264+0000 7fc2d0d96740 1 -- 172.21.15.83:0/3925780456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 msgr2=0x55ed79dccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.264+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/3925780456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79dccb70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80cb250 tx=0x7fc2b809a270 comp rx=0 tx=0).stop 2024-03-11T19:47:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.264+0000 7fc2d0d96740 1 -- 172.21.15.83:0/3925780456 shutdown_connections 2024-03-11T19:47:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.264+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/3925780456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 0x55ed79dd3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.264+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/3925780456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79dccb70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.264+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/3925780456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 0x55ed79dcb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.264+0000 7fc2d0d96740 1 -- 172.21.15.83:0/3925780456 >> 172.21.15.83:0/3925780456 conn(0x55ed79dc6700 msgr2=0x55ed79dc6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.264+0000 7fc2d0d96740 1 -- 172.21.15.83:0/3925780456 shutdown_connections 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.264+0000 7fc2d0d96740 1 -- 172.21.15.83:0/3925780456 wait complete. 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.264+0000 7fc2d0d96740 1 -- start start 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.264+0000 7fc2d0d96740 1 Processor -- start 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 0x55ed79dd5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcc6f0 0x55ed79dd5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcd0b0 0x55ed79ddccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79cd8f40 con 0x55ed79dcb520 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79a074b0 con 0x55ed79dcc6f0 2024-03-11T19:47:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79cb9670 con 0x55ed79dcd0b0 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2d0d96740 10 client.0 _ll_register_callbacks cb 0x55ed79d26f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2c77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 0x55ed79dd5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2c77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 0x55ed79dd5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41542/0 (socket says 172.21.15.83:41542) 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2c7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcd0b0 0x55ed79ddccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2c77fe640 1 -- 172.21.15.83:0/2267803943 learned_addr learned my addr 172.21.15.83:0/2267803943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2c6ffd640 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcc6f0 0x55ed79dd5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2c77fe640 1 -- 172.21.15.83:0/2267803943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcd0b0 msgr2=0x55ed79ddccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcd0b0 0x55ed79ddccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2c77fe640 1 -- 172.21.15.83:0/2267803943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcc6f0 msgr2=0x55ed79dd5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcc6f0 0x55ed79dd5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2c77fe640 1 -- 172.21.15.83:0/2267803943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c81a5e40 con 0x55ed79dcb520 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 0x55ed79dd5770 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc2b809a600 tx=0x7fc2b8004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f775cb50cf1bcd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:47:18.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.265+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/2267803943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80a2b80 con 0x55ed79dcb520 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2d0d96740 1 -- 172.21.15.83:0/2267803943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed79d0df40 con 0x55ed79dcb520 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/2267803943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc2b8083e40 con 0x55ed79dcb520 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/2267803943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2b80a2e10 con 0x55ed79dcb520 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2d0d96740 10 client.16890 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2d0d96740 1 -- 172.21.15.83:0/2267803943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ed79d32320 con 0x55ed79dcb520 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/2267803943 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ed79d0df40 con 0x55ed79dcb520 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/2267803943 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc2b801f060 con 0x55ed79dcb520 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2d0d96740 10 client.16890 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2d0d96740 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2d0d96740 20 client.16890 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ed79de1660 0x55ed79de3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2d0d96740 4 client.16890 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2d0d96740 1 -- 172.21.15.83:0/2267803943 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed79d2a310 con 0x55ed79de1660 2024-03-11T19:47:18.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.266+0000 7fc2c6ffd640 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ed79de1660 0x55ed79de3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.269+0000 7fc2a6ffd640 10 client.16890 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.269+0000 7fc2c6ffd640 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ed79de1660 0x55ed79de3a80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=770d8a1bd7980275 server_cookie=fff4e803cf88f5d7 in_seq=0 out_seq=0 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/2267803943 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed79d2a310 con 0x55ed79de1660 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2a6ffd640 10 client.16890 handle_command_reply: tid=0 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 1 client.16890 shutdown 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 1 -- 172.21.15.83:0/2267803943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 msgr2=0x55ed79dd5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 0x55ed79dd5770 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc2b809a600 tx=0x7fc2b8004a30 comp rx=0 tx=0).stop 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 1 -- 172.21.15.83:0/2267803943 shutdown_connections 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ed79de1660 0x55ed79de3a80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcd0b0 0x55ed79ddccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcc6f0 0x55ed79dd5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/2267803943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 0x55ed79dd5770 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2a6ffd640 0 client.16890 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 1 -- 172.21.15.83:0/2267803943 >> 172.21.15.83:0/2267803943 conn(0x55ed79dc6700 msgr2=0x55ed79dc5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 1 -- 172.21.15.83:0/2267803943 shutdown_connections 2024-03-11T19:47:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 1 -- 172.21.15.83:0/2267803943 wait complete. 2024-03-11T19:47:18.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.270+0000 7fc2d0d96740 20 client.16890 trim_cache size 0 max 16384 2024-03-11T19:47:18.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.291+0000 7fc2d0d96740 1 Processor -- start 2024-03-11T19:47:18.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.292+0000 7fc2d0d96740 1 -- start start 2024-03-11T19:47:18.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 0x55ed79dc51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79ddcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 0x55ed79dc56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79dc1fd0 con 0x55ed79dcd0b0 2024-03-11T19:47:18.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79ddd970 con 0x55ed79dcb520 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79dc1e90 con 0x55ed79dcc6f0 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 0x55ed79dc56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79ddcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 0x55ed79dc56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41552/0 (socket says 172.21.15.83:41552) 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c7fff640 1 -- 172.21.15.83:0/1485203860 learned_addr learned my addr 172.21.15.83:0/1485203860 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/1485203860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 0x55ed79dc51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c7fff640 1 -- 172.21.15.83:0/1485203860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 msgr2=0x55ed79ddcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c7fff640 1 --2- 172.21.15.83:0/1485203860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79ddcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c7fff640 1 -- 172.21.15.83:0/1485203860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 msgr2=0x55ed79dc51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c7fff640 1 --2- 172.21.15.83:0/1485203860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 0x55ed79dc51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c7fff640 1 -- 172.21.15.83:0/1485203860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed79ccec00 con 0x55ed79dcd0b0 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/1485203860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 0x55ed79dc51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.293+0000 7fc2c7fff640 1 --2- 172.21.15.83:0/1485203860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 0x55ed79dc56f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc2c002c700 tx=0x7fc2c0046070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1d280b35ed9078 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.294+0000 7fc2ad7fa640 1 -- 172.21.15.83:0/1485203860 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0003970 con 0x55ed79dcd0b0 2024-03-11T19:47:18.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.294+0000 7fc2ad7fa640 1 -- 172.21.15.83:0/1485203860 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2c0004040 con 0x55ed79dcd0b0 2024-03-11T19:47:18.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.294+0000 7fc2ad7fa640 1 -- 172.21.15.83:0/1485203860 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00507b0 con 0x55ed79dcd0b0 2024-03-11T19:47:18.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.294+0000 7fc2d0d96740 1 -- 172.21.15.83:0/1485203860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 msgr2=0x55ed79dc56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.294+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/1485203860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 0x55ed79dc56f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc2c002c700 tx=0x7fc2c0046070 comp rx=0 tx=0).stop 2024-03-11T19:47:18.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.294+0000 7fc2d0d96740 1 -- 172.21.15.83:0/1485203860 shutdown_connections 2024-03-11T19:47:18.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.294+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/1485203860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcd0b0 0x55ed79dc56f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.294+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/1485203860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79ddcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.294+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/1485203860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcb520 0x55ed79dc51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.294+0000 7fc2d0d96740 1 -- 172.21.15.83:0/1485203860 >> 172.21.15.83:0/1485203860 conn(0x55ed79dc6700 msgr2=0x55ed79dd01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.295+0000 7fc2d0d96740 1 -- 172.21.15.83:0/1485203860 shutdown_connections 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.295+0000 7fc2d0d96740 1 -- 172.21.15.83:0/1485203860 wait complete. 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.295+0000 7fc2d0d96740 1 -- start start 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.295+0000 7fc2d0d96740 1 Processor -- start 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.295+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 0x55ed79ddc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79ddae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2d0d96740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcd0b0 0x55ed79ddb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79dd3a30 con 0x55ed79dcb520 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79ddd010 con 0x55ed79dcd0b0 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2d0d96740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed79ddb8b0 con 0x55ed79dcc6f0 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2c6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79ddae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2c6ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79ddae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59644/0 (socket says 172.21.15.83:59644) 2024-03-11T19:47:18.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2c6ffd640 1 -- 172.21.15.83:0/4055102275 learned_addr learned my addr 172.21.15.83:0/4055102275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2d0d96740 10 client.0 _ll_register_callbacks cb 0x7fc2b8096250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2c7fff640 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcd0b0 0x55ed79ddb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 0x55ed79ddc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2c6ffd640 1 -- 172.21.15.83:0/4055102275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcd0b0 msgr2=0x55ed79ddb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2c6ffd640 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcd0b0 0x55ed79ddb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2c6ffd640 1 -- 172.21.15.83:0/4055102275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 msgr2=0x55ed79ddc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2c6ffd640 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 0x55ed79ddc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.296+0000 7fc2c6ffd640 1 -- 172.21.15.83:0/4055102275 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed79d32320 con 0x55ed79dcc6f0 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.297+0000 7fc2c6ffd640 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79ddae00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55ed79dc7670 tx=0x7fc2bc01e760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cd3513877a7d03d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.297+0000 7fc2a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.297+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/4055102275 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc017040 con 0x55ed79dcc6f0 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.297+0000 7fc2d0d96740 1 -- 172.21.15.83:0/4055102275 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed79d59900 con 0x55ed79dcc6f0 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.297+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/4055102275 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc2bc010c90 con 0x55ed79dcc6f0 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.297+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/4055102275 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2bc010e70 con 0x55ed79dcc6f0 2024-03-11T19:47:18.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.297+0000 7fc2d0d96740 10 client.26470 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.297+0000 7fc2d0d96740 1 -- 172.21.15.83:0/4055102275 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fc2c81104a0 con 0x55ed79dcc6f0 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.297+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/4055102275 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ed79d59900 con 0x55ed79dcc6f0 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.298+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/4055102275 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fc2bc030070 con 0x55ed79dcc6f0 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.298+0000 7fc2d0d96740 10 client.26470 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.298+0000 7fc2d0d96740 10 client.26470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.298+0000 7fc2d0d96740 20 client.26470 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.298+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ed79e39190 0x55ed79e3b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.298+0000 7fc2d0d96740 4 client.26470 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.298+0000 7fc2d0d96740 1 -- 172.21.15.83:0/4055102275 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ed79a33860 con 0x55ed79e39190 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.298+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ed79e39190 0x55ed79e3b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.298+0000 7fc2c77fe640 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ed79e39190 0x55ed79e3b5b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c8bbc9c81ea62bc server_cookie=b2b089d77d93e346 in_seq=0 out_seq=0 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.298+0000 7fc2a6ffd640 10 client.26470 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.299+0000 7fc2a6ffd640 1 -- 172.21.15.83:0/4055102275 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ed79a33860 con 0x55ed79e39190 2024-03-11T19:47:18.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.299+0000 7fc2a6ffd640 10 client.26470 handle_command_reply: tid=0 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.299+0000 7fc2d0d96740 1 client.26470 shutdown 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.299+0000 7fc2d0d96740 1 -- 172.21.15.83:0/4055102275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 msgr2=0x55ed79ddae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.299+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79ddae00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55ed79dc7670 tx=0x7fc2bc01e760 comp rx=0 tx=0).stop 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2d0d96740 1 -- 172.21.15.83:0/4055102275 shutdown_connections 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ed79e39190 0x55ed79e3b5b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed79dcd0b0 0x55ed79ddb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed79dcc6f0 0x55ed79ddae00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2d0d96740 1 --2- 172.21.15.83:0/4055102275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed79dcb520 0x55ed79ddc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2d0d96740 1 -- 172.21.15.83:0/4055102275 >> 172.21.15.83:0/4055102275 conn(0x55ed79dc6700 msgr2=0x55ed79dc9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2a6ffd640 0 client.26470 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2d0d96740 1 -- 172.21.15.83:0/4055102275 shutdown_connections 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2d0d96740 1 -- 172.21.15.83:0/4055102275 wait complete. 2024-03-11T19:47:18.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2d0d96740 20 client.26470 trim_cache size 0 max 16384 2024-03-11T19:47:18.309 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:47:18.309 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:47:18.309 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "2aa82a16-5273-49f8-a8da-9c8bbbd056a2", 2024-03-11T19:47:18.309 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:47:18.309 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:47:18.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/3397851477 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc2b00a1ab0 msgr2=0x7fc2b00a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2ce1c7640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc2b00a1ab0 0x7fc2b00a3f70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0032540 tx=0x7fc2c002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/3397851477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2c8073be0 msgr2=0x7fc2c819d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2ce1c7640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2c8073be0 0x7fc2c819d830 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800e9f0 tx=0x7fc2b800edf0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/3397851477 shutdown_connections 2024-03-11T19:47:18.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2ce1c7640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc2b00a1ab0 0x7fc2b00a3f70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2ce1c7640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c810d760 0x7fc2c81a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2ce1c7640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c80751c0 0x7fc2c819dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2ce1c7640 1 --2- 172.21.15.83:0/3397851477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc2c8073be0 0x7fc2c819d830 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.300+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/3397851477 >> 172.21.15.83:0/3397851477 conn(0x7fc2c80fc700 msgr2=0x7fc2c81009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.301+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/3397851477 shutdown_connections 2024-03-11T19:47:18.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.301+0000 7fc2ce1c7640 1 -- 172.21.15.83:0/3397851477 wait complete. 2024-03-11T19:47:18.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:47:18.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:47:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.481+0000 7fcaac102640 1 -- 172.21.15.83:0/2372238048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaa410d4a0 msgr2=0x7fcaa410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.481+0000 7fcaac102640 1 --2- 172.21.15.83:0/2372238048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaa410d4a0 0x7fcaa410d920 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009860 tx=0x7fcaa002bb90 comp rx=0 tx=0).stop 2024-03-11T19:47:18.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.481+0000 7fcaac102640 1 -- 172.21.15.83:0/2372238048 shutdown_connections 2024-03-11T19:47:18.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.481+0000 7fcaac102640 1 --2- 172.21.15.83:0/2372238048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaa410d4a0 0x7fcaa410d920 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.481+0000 7fcaac102640 1 --2- 172.21.15.83:0/2372238048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa410aaf0 0x7fcaa410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.481+0000 7fcaac102640 1 --2- 172.21.15.83:0/2372238048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcaa4108180 0x7fcaa410a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.481+0000 7fcaac102640 1 -- 172.21.15.83:0/2372238048 >> 172.21.15.83:0/2372238048 conn(0x7fcaa406e200 msgr2=0x7fcaa4070640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.482+0000 7fcaac102640 1 -- 172.21.15.83:0/2372238048 shutdown_connections 2024-03-11T19:47:18.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.482+0000 7fcaac102640 1 -- 172.21.15.83:0/2372238048 wait complete. 2024-03-11T19:47:18.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.483+0000 7fcaac102640 1 Processor -- start 2024-03-11T19:47:18.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.483+0000 7fcaac102640 1 -- start start 2024-03-11T19:47:18.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.484+0000 7fcaac102640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4108180 0x7fcaa41a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.484+0000 7fcaac102640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaa410aaf0 0x7fcaa41a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.484+0000 7fcaac102640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcaa410d4a0 0x7fcaa41ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.484+0000 7fcaac102640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa41adaa0 con 0x7fcaa410aaf0 2024-03-11T19:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.484+0000 7fcaac102640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa41adc10 con 0x7fcaa4108180 2024-03-11T19:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.484+0000 7fcaaa69f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcaa410d4a0 0x7fcaa41ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.484+0000 7fcaa9e9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4108180 0x7fcaa41a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.484+0000 7fcaa9e9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4108180 0x7fcaa41a5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50812/0 (socket says 172.21.15.83:50812) 2024-03-11T19:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.484+0000 7fcaa9e9e640 1 -- 172.21.15.83:0/2825102223 learned_addr learned my addr 172.21.15.83:0/2825102223 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.484+0000 7fcaac102640 1 -- 172.21.15.83:0/2825102223 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa41addd0 con 0x7fcaa410d4a0 2024-03-11T19:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.485+0000 7fcaa969d640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaa410aaf0 0x7fcaa41a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.485+0000 7fcaa9e9e640 1 -- 172.21.15.83:0/2825102223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcaa410d4a0 msgr2=0x7fcaa41ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.485+0000 7fcaa9e9e640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcaa410d4a0 0x7fcaa41ad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.485+0000 7fcaa9e9e640 1 -- 172.21.15.83:0/2825102223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaa410aaf0 msgr2=0x7fcaa41a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.485+0000 7fcaa9e9e640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaa410aaf0 0x7fcaa41a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.485+0000 7fcaa9e9e640 1 -- 172.21.15.83:0/2825102223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa0009490 con 0x7fcaa4108180 2024-03-11T19:47:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.485+0000 7fcaa9e9e640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4108180 0x7fcaa41a5fb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d950 tx=0x7fca9400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.486+0000 7fca9affd640 1 -- 172.21.15.83:0/2825102223 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94009c30 con 0x7fcaa4108180 2024-03-11T19:47:18.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.486+0000 7fca9affd640 1 -- 172.21.15.83:0/2825102223 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca94009d90 con 0x7fcaa4108180 2024-03-11T19:47:18.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.486+0000 7fcaac102640 1 -- 172.21.15.83:0/2825102223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa41ae0b0 con 0x7fcaa4108180 2024-03-11T19:47:18.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.486+0000 7fcaac102640 1 -- 172.21.15.83:0/2825102223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa41ae5c0 con 0x7fcaa4108180 2024-03-11T19:47:18.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.487+0000 7fca9affd640 1 -- 172.21.15.83:0/2825102223 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94009c30 con 0x7fcaa4108180 2024-03-11T19:47:18.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.489+0000 7fcaaedb4740 1 Processor -- start 2024-03-11T19:47:18.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaedb4740 1 -- start start 2024-03-11T19:47:18.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af12d0 0x5555c1af16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af1cc0 0x5555c1af67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1af92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaedb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c1a61280 con 0x5555c1af1cc0 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaedb4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c1a5e7b0 con 0x5555c1af12d0 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaedb4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c1a59f90 con 0x5555c1af6e30 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaa69f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af12d0 0x5555c1af16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaa969d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1af92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaa69f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af12d0 0x5555c1af16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50828/0 (socket says 172.21.15.83:50828) 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaa69f640 1 -- 172.21.15.83:0/823519735 learned_addr learned my addr 172.21.15.83:0/823519735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaa69f640 1 -- 172.21.15.83:0/823519735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 msgr2=0x5555c1af92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/823519735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1af92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaa69f640 1 -- 172.21.15.83:0/823519735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af1cc0 msgr2=0x5555c1af67c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.490+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/823519735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af1cc0 0x5555c1af67c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.491+0000 7fcaaa69f640 1 -- 172.21.15.83:0/823519735 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555c1a18570 con 0x5555c1af12d0 2024-03-11T19:47:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.491+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/823519735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af12d0 0x5555c1af16f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0034ae0 tx=0x7fcaa0032480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46744394dc31d54a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.491+0000 7fca9affd640 1 -- 172.21.15.83:0/2825102223 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fca940050d0 con 0x7fcaa4108180 2024-03-11T19:47:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.491+0000 7fca8ffff640 1 -- 172.21.15.83:0/823519735 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0034640 con 0x5555c1af12d0 2024-03-11T19:47:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.492+0000 7fca8ffff640 1 -- 172.21.15.83:0/823519735 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcaa0035e80 con 0x5555c1af12d0 2024-03-11T19:47:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.492+0000 7fca8ffff640 1 -- 172.21.15.83:0/823519735 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa002c790 con 0x5555c1af12d0 2024-03-11T19:47:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.492+0000 7fca9affd640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca900a1b30 0x7fca900a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.492+0000 7fcaa9e9e640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca900a1b30 0x7fca900a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.492+0000 7fca9affd640 1 -- 172.21.15.83:0/2825102223 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fca940cc010 con 0x7fcaa4108180 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.492+0000 7fcaaedb4740 1 -- 172.21.15.83:0/823519735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af12d0 msgr2=0x5555c1af16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.492+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/823519735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af12d0 0x5555c1af16f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0034ae0 tx=0x7fcaa0032480 comp rx=0 tx=0).stop 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.493+0000 7fcaaedb4740 1 -- 172.21.15.83:0/823519735 shutdown_connections 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.493+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/823519735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1af92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.493+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/823519735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af1cc0 0x5555c1af67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.493+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/823519735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af12d0 0x5555c1af16f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.493+0000 7fcaaedb4740 1 -- 172.21.15.83:0/823519735 >> 172.21.15.83:0/823519735 conn(0x5555c1aec7f0 msgr2=0x5555c1aecbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.493+0000 7fcaa9e9e640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca900a1b30 0x7fca900a3ff0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fca94010f10 tx=0x7fca9400f3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.493+0000 7fcaaedb4740 1 -- 172.21.15.83:0/823519735 shutdown_connections 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.493+0000 7fcaaedb4740 1 -- 172.21.15.83:0/823519735 wait complete. 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.493+0000 7fcaaedb4740 1 -- start start 2024-03-11T19:47:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.493+0000 7fcaaedb4740 1 Processor -- start 2024-03-11T19:47:18.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1afb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1afb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1b02820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaaedb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c172d430 con 0x5555c1af12d0 2024-03-11T19:47:18.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaaedb4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c16e9e00 con 0x5555c1af1cc0 2024-03-11T19:47:18.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaaedb4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c19defd0 con 0x5555c1af6e30 2024-03-11T19:47:18.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaaedb4740 10 client.0 _ll_register_callbacks cb 0x5555c1a34fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaa969d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1afb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaa969d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1afb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50844/0 (socket says 172.21.15.83:50844) 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaa969d640 1 -- 172.21.15.83:0/402508739 learned_addr learned my addr 172.21.15.83:0/402508739 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1afb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1b02820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaa969d640 1 -- 172.21.15.83:0/402508739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 msgr2=0x5555c1b02820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaa969d640 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1b02820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaa969d640 1 -- 172.21.15.83:0/402508739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 msgr2=0x5555c1afb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaa969d640 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1afb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaa969d640 1 -- 172.21.15.83:0/402508739 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa410e460 con 0x5555c1af1cc0 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fcaa969d640 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1afb7e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fca9c010b70 tx=0x7fca9c00f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671a2ed5338a3a92 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.494+0000 7fca8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.495+0000 7fca8d7fa640 1 -- 172.21.15.83:0/402508739 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c015070 con 0x5555c1af1cc0 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.495+0000 7fca8d7fa640 1 -- 172.21.15.83:0/402508739 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fca9c004b20 con 0x5555c1af1cc0 2024-03-11T19:47:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.495+0000 7fcaaedb4740 1 -- 172.21.15.83:0/402508739 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555c1796720 con 0x5555c1af1cc0 2024-03-11T19:47:18.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.495+0000 7fca8d7fa640 1 -- 172.21.15.83:0/402508739 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca9c004ce0 con 0x5555c1af1cc0 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.496+0000 7fcaaedb4740 10 client.26369 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.496+0000 7fcaaedb4740 1 -- 172.21.15.83:0/402508739 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5555c1a58610 con 0x5555c1af1cc0 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.497+0000 7fca8d7fa640 1 -- 172.21.15.83:0/402508739 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5555c1796720 con 0x5555c1af1cc0 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.497+0000 7fca8d7fa640 1 -- 172.21.15.83:0/402508739 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fca9c0046e0 con 0x5555c1af1cc0 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.497+0000 7fcaaedb4740 10 client.26369 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.497+0000 7fcaaedb4740 10 client.26369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.498+0000 7fcaaedb4740 20 client.26369 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.498+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5555c1b07460 0x5555c1b09880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.498+0000 7fcaaedb4740 4 client.26369 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.498+0000 7fcaaedb4740 1 -- 172.21.15.83:0/402508739 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555c17e5890 con 0x5555c1b07460 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.498+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5555c1b07460 0x5555c1b09880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.498+0000 7fca8d7fa640 10 client.26369 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.499+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5555c1b07460 0x5555c1b09880 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7572b34fa525bc1 server_cookie=3731a624dc5fb19c in_seq=0 out_seq=0 2024-03-11T19:47:18.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fca8d7fa640 1 -- 172.21.15.83:0/402508739 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5555c17e5890 con 0x5555c1b07460 2024-03-11T19:47:18.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fca8d7fa640 10 client.26369 handle_command_reply: tid=0 2024-03-11T19:47:18.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fcaaedb4740 1 client.26369 shutdown 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fcaaedb4740 1 -- 172.21.15.83:0/402508739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 msgr2=0x5555c1afb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1afb7e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fca9c010b70 tx=0x7fca9c00f7f0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fcaaedb4740 1 -- 172.21.15.83:0/402508739 shutdown_connections 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5555c1b07460 0x5555c1b09880 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1b02820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1afb7e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/402508739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1afb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fca8d7fa640 0 client.26369 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fcaaedb4740 1 -- 172.21.15.83:0/402508739 >> 172.21.15.83:0/402508739 conn(0x5555c1aec7f0 msgr2=0x5555c1aeba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fcaaedb4740 1 -- 172.21.15.83:0/402508739 shutdown_connections 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.500+0000 7fcaaedb4740 1 -- 172.21.15.83:0/402508739 wait complete. 2024-03-11T19:47:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.501+0000 7fcaaedb4740 20 client.26369 trim_cache size 0 max 16384 2024-03-11T19:47:18.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaaedb4740 1 Processor -- start 2024-03-11T19:47:18.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaaedb4740 1 -- start start 2024-03-11T19:47:18.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1af9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af1cc0 0x5555c1b027e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaaa69f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1af9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaaa69f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1af9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41598/0 (socket says 172.21.15.83:41598) 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af6e30 0x5555c1afa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaaedb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c1afb0e0 con 0x5555c1af12d0 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3570849163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c1b0e870 con 0x5555c1af6e30 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaaa69f640 1 -- 172.21.15.83:0/3570849163 learned_addr learned my addr 172.21.15.83:0/3570849163 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3570849163 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c1796720 con 0x5555c1af1cc0 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.524+0000 7fcaa969d640 1 --2- 172.21.15.83:0/3570849163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af1cc0 0x5555c1b027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.525+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/3570849163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af6e30 0x5555c1afa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.525+0000 7fcaaa69f640 1 -- 172.21.15.83:0/3570849163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af1cc0 msgr2=0x5555c1b027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.525+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/3570849163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af1cc0 0x5555c1b027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.525+0000 7fcaaa69f640 1 -- 172.21.15.83:0/3570849163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af6e30 msgr2=0x5555c1afa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.525+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/3570849163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af6e30 0x5555c1afa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.525+0000 7fcaaa69f640 1 -- 172.21.15.83:0/3570849163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555c19f4ac0 con 0x5555c1af12d0 2024-03-11T19:47:18.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.525+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/3570849163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1af9fd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0034ae0 tx=0x7fcaa0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f00ba64f5aed4d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.525+0000 7fca8f7fe640 1 -- 172.21.15.83:0/3570849163 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0005520 con 0x5555c1af12d0 2024-03-11T19:47:18.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.525+0000 7fca8f7fe640 1 -- 172.21.15.83:0/3570849163 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcaa005c070 con 0x5555c1af12d0 2024-03-11T19:47:18.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.525+0000 7fca8f7fe640 1 -- 172.21.15.83:0/3570849163 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa002c790 con 0x5555c1af12d0 2024-03-11T19:47:18.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.526+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3570849163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 msgr2=0x5555c1af9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.526+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/3570849163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1af9fd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0034ae0 tx=0x7fcaa0005f90 comp rx=0 tx=0).stop 2024-03-11T19:47:18.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.527+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3570849163 shutdown_connections 2024-03-11T19:47:18.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.527+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/3570849163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af6e30 0x5555c1afa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.527+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/3570849163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af1cc0 0x5555c1b027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.527+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/3570849163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1af9fd0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.527+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3570849163 >> 172.21.15.83:0/3570849163 conn(0x5555c1aec7f0 msgr2=0x5555c1af2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.529+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3570849163 shutdown_connections 2024-03-11T19:47:18.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.529+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3570849163 wait complete. 2024-03-11T19:47:18.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.529+0000 7fcaaedb4740 1 -- start start 2024-03-11T19:47:18.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.529+0000 7fcaaedb4740 1 Processor -- start 2024-03-11T19:47:18.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1b01130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1b01670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaaedb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1b01bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaaa69f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1b01130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaa969d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1b01670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaaa69f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1b01130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41610/0 (socket says 172.21.15.83:41610) 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaa969d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1b01670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50870/0 (socket says 172.21.15.83:50870) 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaaa69f640 1 -- 172.21.15.83:0/3784782092 learned_addr learned my addr 172.21.15.83:0/3784782092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3784782092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c1b5d9c0 con 0x5555c1af12d0 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3784782092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c1b5db30 con 0x5555c1af1cc0 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3784782092 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555c1b5dcf0 con 0x5555c1af6e30 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.530+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1b01bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fcaaedb4740 10 client.0 _ll_register_callbacks cb 0x7fcaa00096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:18.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fcaa969d640 1 -- 172.21.15.83:0/3784782092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 msgr2=0x5555c1b01bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fcaa969d640 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1b01bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fcaa969d640 1 -- 172.21.15.83:0/3784782092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 msgr2=0x5555c1b01130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:18.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fcaa969d640 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1b01130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fcaa969d640 1 -- 172.21.15.83:0/3784782092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555c1a18570 con 0x5555c1af1cc0 2024-03-11T19:47:18.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1b01bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fcaa969d640 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1b01670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00e710 tx=0x7fca9c00a990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=963a1ae32891194 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fca8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fca8d7fa640 1 -- 172.21.15.83:0/3784782092 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c015040 con 0x5555c1af1cc0 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fca8d7fa640 1 -- 172.21.15.83:0/3784782092 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fca9c00e880 con 0x5555c1af1cc0 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.531+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3784782092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555c17597e0 con 0x5555c1af1cc0 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.533+0000 7fca8d7fa640 1 -- 172.21.15.83:0/3784782092 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca9c00ea40 con 0x5555c1af1cc0 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.533+0000 7fcaaedb4740 10 client.26375 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.533+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3784782092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5555c1a1a6a0 con 0x5555c1af1cc0 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.533+0000 7fca8d7fa640 1 -- 172.21.15.83:0/3784782092 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca9c033070 con 0x5555c1af1cc0 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.533+0000 7fca8d7fa640 1 -- 172.21.15.83:0/3784782092 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7fca9c036070 con 0x5555c1af1cc0 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.533+0000 7fcaaedb4740 10 client.26375 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.533+0000 7fcaaedb4740 10 client.26375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.533+0000 7fcaaedb4740 20 client.26375 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.533+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5555c1b5f760 0x5555c1b61b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.533+0000 7fcaaedb4740 4 client.26375 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:47:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.533+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3784782092 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5555c17597e0 con 0x5555c1b5f760 2024-03-11T19:47:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.534+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5555c1b5f760 0x5555c1b61b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.535+0000 7fca8d7fa640 10 client.26375 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.535+0000 7fcaaa69f640 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5555c1b5f760 0x5555c1b61b80 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb928c2e361b1e89 server_cookie=1395ce55d1e3fc9a in_seq=0 out_seq=0 2024-03-11T19:47:18.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fca8d7fa640 1 -- 172.21.15.83:0/3784782092 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5555c17597e0 con 0x5555c1b5f760 2024-03-11T19:47:18.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fca8d7fa640 10 client.26375 handle_command_reply: tid=0 2024-03-11T19:47:18.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fcaaedb4740 1 client.26375 shutdown 2024-03-11T19:47:18.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3784782092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 msgr2=0x5555c1b01670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1b01670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00e710 tx=0x7fca9c00a990 comp rx=0 tx=0).stop 2024-03-11T19:47:18.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3784782092 shutdown_connections 2024-03-11T19:47:18.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5555c1b5f760 0x5555c1b61b80 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555c1af6e30 0x5555c1b01bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fca8d7fa640 0 client.26375 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:18.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555c1af1cc0 0x5555c1b01670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fcaaedb4740 1 --2- 172.21.15.83:0/3784782092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555c1af12d0 0x5555c1b01130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.536+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3784782092 >> 172.21.15.83:0/3784782092 conn(0x5555c1aec7f0 msgr2=0x5555c1af9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3784782092 shutdown_connections 2024-03-11T19:47:18.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaaedb4740 1 -- 172.21.15.83:0/3784782092 wait complete. 2024-03-11T19:47:18.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaaedb4740 20 client.26375 trim_cache size 0 max 16384 2024-03-11T19:47:18.542 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:47:18.542 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:47:18.543 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:47:18.543 INFO:teuthology.orchestra.run.smithi083.stdout: "2aa82a16-5273-49f8-a8da-9c8bbbd056a2": { 2024-03-11T19:47:18.543 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:47:18.543 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "2aa82a16-5273-49f8-a8da-9c8bbbd056a2", 2024-03-11T19:47:18.543 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:47:18.543 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:47:18.543 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:47:18.543 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaac102640 1 -- 172.21.15.83:0/2825102223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca900a1b30 msgr2=0x7fca900a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaac102640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca900a1b30 0x7fca900a3ff0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fca94010f10 tx=0x7fca9400f3c0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaac102640 1 -- 172.21.15.83:0/2825102223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4108180 msgr2=0x7fcaa41a5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaac102640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4108180 0x7fcaa41a5fb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d950 tx=0x7fca9400de10 comp rx=0 tx=0).stop 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaac102640 1 -- 172.21.15.83:0/2825102223 shutdown_connections 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaac102640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca900a1b30 0x7fca900a3ff0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaac102640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcaa410d4a0 0x7fcaa41ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaac102640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcaa410aaf0 0x7fcaa41a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaac102640 1 --2- 172.21.15.83:0/2825102223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa4108180 0x7fcaa41a5fb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaac102640 1 -- 172.21.15.83:0/2825102223 >> 172.21.15.83:0/2825102223 conn(0x7fcaa406e200 msgr2=0x7fcaa410a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.537+0000 7fcaac102640 1 -- 172.21.15.83:0/2825102223 shutdown_connections 2024-03-11T19:47:18.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:18.538+0000 7fcaac102640 1 -- 172.21.15.83:0/2825102223 wait complete. 2024-03-11T19:47:18.556 INFO:tasks.cephfs.filesystem:scrub status for tag:2aa82a16-5273-49f8-a8da-9c8bbbd056a2 - {'path': '/', 'tag': '2aa82a16-5273-49f8-a8da-9c8bbbd056a2', 'options': 'recursive,force'} 2024-03-11T19:47:18.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:18 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:47:18.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:18 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:18.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:18 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:47:18.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:18 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:18.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:18 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:47:18.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:18 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:18.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:18 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:47:18.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:18 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:18 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:47:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:18 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:18 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:47:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:18 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:19 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:47:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:19 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:19 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:47:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:19 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:19.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:19 smithi083 ceph-mon[31864]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.2 MiB/s wr, 673 op/s 2024-03-11T19:47:19.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:19 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:47:19.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:19 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:19.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:19 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:47:19.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:19 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:19.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:19 smithi137 ceph-mon[36220]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.2 MiB/s wr, 673 op/s 2024-03-11T19:47:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:19 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:47:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:19 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:19 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:47:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:19 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:19.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:19 smithi204 ceph-mon[36739]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.2 MiB/s wr, 673 op/s 2024-03-11T19:47:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:20 smithi083 ceph-mon[31864]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.9 MiB/s wr, 587 op/s 2024-03-11T19:47:21.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:20 smithi137 ceph-mon[36220]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.9 MiB/s wr, 587 op/s 2024-03-11T19:47:21.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:20 smithi204 ceph-mon[36739]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.9 MiB/s wr, 587 op/s 2024-03-11T19:47:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 485.0 s, 2804.4 tps, lat 35.389 ms stddev 41.836 2024-03-11T19:47:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:22 smithi083 ceph-mon[31864]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 MiB/s wr, 633 op/s 2024-03-11T19:47:23.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:22 smithi137 ceph-mon[36220]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 MiB/s wr, 633 op/s 2024-03-11T19:47:23.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:22 smithi204 ceph-mon[36739]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.6 MiB/s wr, 633 op/s 2024-03-11T19:47:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:24 smithi083 ceph-mon[31864]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 697 op/s 2024-03-11T19:47:25.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:24 smithi137 ceph-mon[36220]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 697 op/s 2024-03-11T19:47:25.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:24 smithi204 ceph-mon[36739]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 697 op/s 2024-03-11T19:47:26.040 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:25 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:47:26.057 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:25 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:47:26.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:25 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:47:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:26 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:47:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:26 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:47:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:26 smithi083 ceph-mon[31864]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 MiB/s wr, 632 op/s 2024-03-11T19:47:27.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:26 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:47:27.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:26 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:47:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:26 smithi137 ceph-mon[36220]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 MiB/s wr, 632 op/s 2024-03-11T19:47:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:26 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:47:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:26 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:47:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:26 smithi204 ceph-mon[36739]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 MiB/s wr, 632 op/s 2024-03-11T19:47:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 490.0 s, 2258.4 tps, lat 44.572 ms stddev 50.089 2024-03-11T19:47:29.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:28 smithi083 ceph-mon[31864]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 765 op/s 2024-03-11T19:47:29.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:28 smithi137 ceph-mon[36220]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 765 op/s 2024-03-11T19:47:29.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:28 smithi204 ceph-mon[36739]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.2 MiB/s wr, 765 op/s 2024-03-11T19:47:31.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:31 smithi083 ceph-mon[31864]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 650 op/s 2024-03-11T19:47:31.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:31 smithi137 ceph-mon[36220]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 650 op/s 2024-03-11T19:47:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:31 smithi204 ceph-mon[36739]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.3 MiB/s wr, 650 op/s 2024-03-11T19:47:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 495.0 s, 2608.6 tps, lat 38.373 ms stddev 43.108 2024-03-11T19:47:32.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:32 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:32.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:32 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:32.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:32 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:47:33.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:33 smithi083 ceph-mon[31864]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-03-11T19:47:33.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:33.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:33.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:47:33.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:33 smithi137 ceph-mon[36220]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-03-11T19:47:33.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:33.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:33.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:47:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:33 smithi204 ceph-mon[36739]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 608 op/s 2024-03-11T19:47:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:47:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:47:33.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:47:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:34 smithi083 ceph-mon[31864]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 742 op/s 2024-03-11T19:47:35.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:34 smithi137 ceph-mon[36220]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 742 op/s 2024-03-11T19:47:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:34 smithi204 ceph-mon[36739]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 742 op/s 2024-03-11T19:47:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:36 smithi083 ceph-mon[31864]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 662 op/s 2024-03-11T19:47:37.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:36 smithi137 ceph-mon[36220]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 662 op/s 2024-03-11T19:47:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:36 smithi204 ceph-mon[36739]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 662 op/s 2024-03-11T19:47:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 500.0 s, 2593.8 tps, lat 38.464 ms stddev 45.705 2024-03-11T19:47:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:38 smithi083 ceph-mon[31864]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-03-11T19:47:39.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:38 smithi137 ceph-mon[36220]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-03-11T19:47:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:38 smithi204 ceph-mon[36739]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 MiB/s wr, 795 op/s 2024-03-11T19:47:41.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:40 smithi083 ceph-mon[31864]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 662 op/s 2024-03-11T19:47:41.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:40 smithi137 ceph-mon[36220]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 662 op/s 2024-03-11T19:47:41.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:40 smithi204 ceph-mon[36739]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 662 op/s 2024-03-11T19:47:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 505.0 s, 3083.0 tps, lat 32.533 ms stddev 36.478 2024-03-11T19:47:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:42 smithi083 ceph-mon[31864]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 700 op/s 2024-03-11T19:47:43.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:42 smithi137 ceph-mon[36220]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 700 op/s 2024-03-11T19:47:43.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:42 smithi204 ceph-mon[36739]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.5 MiB/s wr, 700 op/s 2024-03-11T19:47:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:44 smithi083 ceph-mon[31864]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 MiB/s wr, 768 op/s 2024-03-11T19:47:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:44 smithi137 ceph-mon[36220]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 MiB/s wr, 768 op/s 2024-03-11T19:47:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:44 smithi204 ceph-mon[36739]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 MiB/s wr, 768 op/s 2024-03-11T19:47:47.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:46 smithi083 ceph-mon[31864]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.2 MiB/s wr, 633 op/s 2024-03-11T19:47:47.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:46 smithi137 ceph-mon[36220]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.2 MiB/s wr, 633 op/s 2024-03-11T19:47:47.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:46 smithi204 ceph-mon[36739]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.2 MiB/s wr, 633 op/s 2024-03-11T19:47:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 510.0 s, 2799.6 tps, lat 35.430 ms stddev 42.660 2024-03-11T19:47:48.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:47:48.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:47:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.672+0000 7f87ef823640 1 -- 172.21.15.83:0/2038154342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e8101c30 msgr2=0x7f87e81084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.672+0000 7f87ef823640 1 --2- 172.21.15.83:0/2038154342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e8101c30 0x7f87e81084c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009ca0 tx=0x7f87d802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.672+0000 7f87ef823640 1 -- 172.21.15.83:0/2038154342 shutdown_connections 2024-03-11T19:47:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.672+0000 7f87ef823640 1 --2- 172.21.15.83:0/2038154342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e8101c30 0x7f87e81084c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.672+0000 7f87ef823640 1 --2- 172.21.15.83:0/2038154342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8101270 0x7f87e81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.672+0000 7f87ef823640 1 --2- 172.21.15.83:0/2038154342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87e8100080 0x7f87e81004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.672+0000 7f87ef823640 1 -- 172.21.15.83:0/2038154342 >> 172.21.15.83:0/2038154342 conn(0x7f87e80fb5f0 msgr2=0x7f87e80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.672+0000 7f87ef823640 1 -- 172.21.15.83:0/2038154342 shutdown_connections 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.672+0000 7f87ef823640 1 -- 172.21.15.83:0/2038154342 wait complete. 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.672+0000 7f87ef823640 1 Processor -- start 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ef823640 1 -- start start 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ef823640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8100080 0x7f87e8194f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ef823640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87e8101270 0x7f87e81954c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ef823640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e8101c30 0x7f87e819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ef823640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e819ca70 con 0x7f87e8101c30 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ef823640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e819cbe0 con 0x7f87e8101270 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ef823640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e819cda0 con 0x7f87e8100080 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87eddc0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e8101c30 0x7f87e819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ed5bf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8100080 0x7f87e8194f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ed5bf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8100080 0x7f87e8194f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:51648/0 (socket says 172.21.15.83:51648) 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ed5bf640 1 -- 172.21.15.83:0/2058271593 learned_addr learned my addr 172.21.15.83:0/2058271593 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:48.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87eddc0640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e8101c30 0x7f87e819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41080/0 (socket says 172.21.15.83:41080) 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87e8101270 0x7f87e81954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87eddc0640 1 -- 172.21.15.83:0/2058271593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8100080 msgr2=0x7f87e8194f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87eddc0640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8100080 0x7f87e8194f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87eddc0640 1 -- 172.21.15.83:0/2058271593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87e8101270 msgr2=0x7f87e81954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87eddc0640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87e8101270 0x7f87e81954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87eddc0640 1 -- 172.21.15.83:0/2058271593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e000d8a0 con 0x7f87e8101c30 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ed5bf640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8100080 0x7f87e8194f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87e8101270 0x7f87e81954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.673+0000 7f87eddc0640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e8101c30 0x7f87e819c530 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f87d802c630 tx=0x7f87d8034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.674+0000 7f87e67fc640 1 -- 172.21.15.83:0/2058271593 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d80329b0 con 0x7f87e8101c30 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.674+0000 7f87ef823640 1 -- 172.21.15.83:0/2058271593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d8009900 con 0x7f87e8101c30 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.674+0000 7f87e67fc640 1 -- 172.21.15.83:0/2058271593 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87d8003e50 con 0x7f87e8101c30 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.674+0000 7f87e67fc640 1 -- 172.21.15.83:0/2058271593 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8040450 con 0x7f87e8101c30 2024-03-11T19:47:48.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.674+0000 7f87ef823640 1 -- 172.21.15.83:0/2058271593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e819d310 con 0x7f87e8101c30 2024-03-11T19:47:48.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.677+0000 7f87f23f9740 1 Processor -- start 2024-03-11T19:47:48.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87e67fc640 1 -- 172.21.15.83:0/2058271593 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f87d800a040 con 0x7f87e8101c30 2024-03-11T19:47:48.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87f23f9740 1 -- start start 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd03df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd042fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd043640 0x564cfd045ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87f23f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfcfa5280 con 0x564cfd043640 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87f23f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfcfa27b0 con 0x564cfd03dae0 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87f23f9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfcf9df90 con 0x564cfd03e4d0 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87ecdbe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd043640 0x564cfd045ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87ecdbe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd043640 0x564cfd045ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41082/0 (socket says 172.21.15.83:41082) 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87ecdbe640 1 -- 172.21.15.83:0/986505307 learned_addr learned my addr 172.21.15.83:0/986505307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87ed5bf640 1 --2- 172.21.15.83:0/986505307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd042fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/986505307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd03df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.679+0000 7f87e67fc640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87d00a1a30 0x7f87d00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87ecdbe640 1 -- 172.21.15.83:0/986505307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 msgr2=0x564cfd042fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:48.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/986505307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd042fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87ecdbe640 1 -- 172.21.15.83:0/986505307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 msgr2=0x564cfd03df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/986505307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd03df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87ecdbe640 1 -- 172.21.15.83:0/986505307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cfcf5c570 con 0x564cfd043640 2024-03-11T19:47:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87eddc0640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87d00a1a30 0x7f87d00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87e67fc640 1 -- 172.21.15.83:0/2058271593 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f87d80f0030 con 0x7f87e8101c30 2024-03-11T19:47:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/986505307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd043640 0x564cfd045ad0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00ac70 tx=0x7f87dc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df84204338b753c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87cf7fe640 1 -- 172.21.15.83:0/986505307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc010040 con 0x564cfd043640 2024-03-11T19:47:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87eddc0640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87d00a1a30 0x7f87d00a3ef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f87d80b9a20 tx=0x7f87d80b2770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87cf7fe640 1 -- 172.21.15.83:0/986505307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87dc019de0 con 0x564cfd043640 2024-03-11T19:47:48.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.680+0000 7f87cf7fe640 1 -- 172.21.15.83:0/986505307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc01b690 con 0x564cfd043640 2024-03-11T19:47:48.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.681+0000 7f87f23f9740 1 -- 172.21.15.83:0/986505307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd043640 msgr2=0x564cfd045ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.681+0000 7f87f23f9740 1 --2- 172.21.15.83:0/986505307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd043640 0x564cfd045ad0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00ac70 tx=0x7f87dc005f90 comp rx=0 tx=0).stop 2024-03-11T19:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.681+0000 7f87f23f9740 1 -- 172.21.15.83:0/986505307 shutdown_connections 2024-03-11T19:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.681+0000 7f87f23f9740 1 --2- 172.21.15.83:0/986505307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd043640 0x564cfd045ad0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.681+0000 7f87f23f9740 1 --2- 172.21.15.83:0/986505307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd042fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.681+0000 7f87f23f9740 1 --2- 172.21.15.83:0/986505307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd03df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.681+0000 7f87f23f9740 1 -- 172.21.15.83:0/986505307 >> 172.21.15.83:0/986505307 conn(0x564cfd039000 msgr2=0x564cfd039400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.682+0000 7f87f23f9740 1 -- 172.21.15.83:0/986505307 shutdown_connections 2024-03-11T19:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.682+0000 7f87f23f9740 1 -- 172.21.15.83:0/986505307 wait complete. 2024-03-11T19:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.682+0000 7f87f23f9740 1 -- start start 2024-03-11T19:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.682+0000 7f87f23f9740 1 Processor -- start 2024-03-11T19:47:48.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd047ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03e4d0 0x564cfd048020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd043640 0x564cfd04f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87f23f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfcc71430 con 0x564cfd03e4d0 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87f23f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfcc2de00 con 0x564cfd03dae0 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87f23f9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfcf22fd0 con 0x564cfd043640 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87f23f9740 10 client.0 _ll_register_callbacks cb 0x564cfcf78fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ed5bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03e4d0 0x564cfd048020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ed5bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03e4d0 0x564cfd048020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41090/0 (socket says 172.21.15.83:41090) 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ed5bf640 1 -- 172.21.15.83:0/1737772203 learned_addr learned my addr 172.21.15.83:0/1737772203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd043640 0x564cfd04f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd047ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ed5bf640 1 -- 172.21.15.83:0/1737772203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd043640 msgr2=0x564cfd04f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:48.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ed5bf640 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd043640 0x564cfd04f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ed5bf640 1 -- 172.21.15.83:0/1737772203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 msgr2=0x564cfd047ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ed5bf640 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd047ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ed5bf640 1 -- 172.21.15.83:0/1737772203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc00a950 con 0x564cfd03e4d0 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ed5bf640 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03e4d0 0x564cfd048020 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f87e0012d50 tx=0x7f87e000c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5b557f232a7a57 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.683+0000 7f87ccff9640 1 -- 172.21.15.83:0/1737772203 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0004020 con 0x564cfd03e4d0 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87ccff9640 1 -- 172.21.15.83:0/1737772203 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f87e0004180 con 0x564cfd03e4d0 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87f23f9740 1 -- 172.21.15.83:0/1737772203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564cfcc9d7e0 con 0x564cfd03e4d0 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87ccff9640 1 -- 172.21.15.83:0/1737772203 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87e0004340 con 0x564cfd03e4d0 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87f23f9740 10 client.16923 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87f23f9740 1 -- 172.21.15.83:0/1737772203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564cfcf9c610 con 0x564cfd03e4d0 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87ccff9640 1 -- 172.21.15.83:0/1737772203 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564cfcc9d7e0 con 0x564cfd03e4d0 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87ccff9640 1 -- 172.21.15.83:0/1737772203 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f87e000b050 con 0x564cfd03e4d0 2024-03-11T19:47:48.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87f23f9740 10 client.16923 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87f23f9740 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87f23f9740 20 client.16923 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87f23f9740 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564cfd053de0 0x564cfd056200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87f23f9740 4 client.16923 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87f23f9740 1 -- 172.21.15.83:0/1737772203 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564cfccda720 con 0x564cfd053de0 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.684+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564cfd053de0 0x564cfd056200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.685+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564cfd053de0 0x564cfd056200 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff5318a153c4f19 server_cookie=3f3ffab105c80ffd in_seq=0 out_seq=0 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.685+0000 7f87ccff9640 10 client.16923 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.685+0000 7f87ccff9640 1 -- 172.21.15.83:0/1737772203 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564cfccda720 con 0x564cfd053de0 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.685+0000 7f87ccff9640 10 client.16923 handle_command_reply: tid=0 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.685+0000 7f87f23f9740 1 client.16923 shutdown 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87f23f9740 1 -- 172.21.15.83:0/1737772203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03e4d0 msgr2=0x564cfd048020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87f23f9740 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03e4d0 0x564cfd048020 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f87e0012d50 tx=0x7f87e000c700 comp rx=0 tx=0).stop 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87f23f9740 1 -- 172.21.15.83:0/1737772203 shutdown_connections 2024-03-11T19:47:48.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87f23f9740 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564cfd053de0 0x564cfd056200 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87ccff9640 0 client.16923 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87f23f9740 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd043640 0x564cfd04f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87f23f9740 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03e4d0 0x564cfd048020 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87f23f9740 1 --2- 172.21.15.83:0/1737772203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd047ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87f23f9740 1 -- 172.21.15.83:0/1737772203 >> 172.21.15.83:0/1737772203 conn(0x564cfd039000 msgr2=0x564cfd02fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87f23f9740 1 -- 172.21.15.83:0/1737772203 shutdown_connections 2024-03-11T19:47:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87f23f9740 1 -- 172.21.15.83:0/1737772203 wait complete. 2024-03-11T19:47:48.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.686+0000 7f87f23f9740 20 client.16923 trim_cache size 0 max 16384 2024-03-11T19:47:48.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.710+0000 7f87f23f9740 1 Processor -- start 2024-03-11T19:47:48.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87f23f9740 1 -- start start 2024-03-11T19:47:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd047ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd048000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd043640 0x564cfd04f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87f23f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfd052310 con 0x564cfd043640 2024-03-11T19:47:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87f23f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfd039ff0 con 0x564cfd03dae0 2024-03-11T19:47:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87f23f9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfd038aa0 con 0x564cfd03e4d0 2024-03-11T19:47:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87ecdbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd047ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87ecdbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd047ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:54164/0 (socket says 172.21.15.83:54164) 2024-03-11T19:47:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87ecdbe640 1 -- 172.21.15.83:0/32873376 learned_addr learned my addr 172.21.15.83:0/32873376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87ecdbe640 1 -- 172.21.15.83:0/32873376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 msgr2=0x564cfd048000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87ed5bf640 1 --2- 172.21.15.83:0/32873376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd048000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/32873376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd048000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87ecdbe640 1 -- 172.21.15.83:0/32873376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd043640 msgr2=0x564cfd04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.711+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/32873376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd043640 0x564cfd04f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.712+0000 7f87ecdbe640 1 -- 172.21.15.83:0/32873376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cfcf38ac0 con 0x564cfd03dae0 2024-03-11T19:47:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.716+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/32873376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd047ac0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00ac70 tx=0x7f87dc01cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3252f91701737c91 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:48.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.722+0000 7f87ceffd640 1 -- 172.21.15.83:0/32873376 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc02fd00 con 0x564cfd03dae0 2024-03-11T19:47:48.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.722+0000 7f87ceffd640 1 -- 172.21.15.83:0/32873376 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87dc005640 con 0x564cfd03dae0 2024-03-11T19:47:48.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.723+0000 7f87f23f9740 1 -- 172.21.15.83:0/32873376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 msgr2=0x564cfd047ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.723+0000 7f87f23f9740 1 --2- 172.21.15.83:0/32873376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd047ac0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00ac70 tx=0x7f87dc01cf90 comp rx=0 tx=0).stop 2024-03-11T19:47:48.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.731+0000 7f87f23f9740 1 -- 172.21.15.83:0/32873376 shutdown_connections 2024-03-11T19:47:48.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.731+0000 7f87f23f9740 1 --2- 172.21.15.83:0/32873376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd043640 0x564cfd04f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.731+0000 7f87f23f9740 1 --2- 172.21.15.83:0/32873376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd048000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.731+0000 7f87f23f9740 1 --2- 172.21.15.83:0/32873376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd03dae0 0x564cfd047ac0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.731+0000 7f87f23f9740 1 -- 172.21.15.83:0/32873376 >> 172.21.15.83:0/32873376 conn(0x564cfd039000 msgr2=0x564cfd05ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.739+0000 7f87f23f9740 1 -- 172.21.15.83:0/32873376 shutdown_connections 2024-03-11T19:47:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.742+0000 7f87f23f9740 1 -- 172.21.15.83:0/32873376 wait complete. 2024-03-11T19:47:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.742+0000 7f87f23f9740 1 -- start start 2024-03-11T19:47:48.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.742+0000 7f87f23f9740 1 Processor -- start 2024-03-11T19:47:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.747+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03dae0 0x564cfd04ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.747+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd04d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.747+0000 7f87f23f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd043640 0x564cfd04daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.747+0000 7f87f23f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfd04e150 con 0x564cfd03dae0 2024-03-11T19:47:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.747+0000 7f87f23f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfd04e2c0 con 0x564cfd043640 2024-03-11T19:47:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.747+0000 7f87f23f9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cfd04e480 con 0x564cfd03e4d0 2024-03-11T19:47:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.747+0000 7f87f23f9740 10 client.0 _ll_register_callbacks cb 0x7f87dc00bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.750+0000 7f87ecdbe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03dae0 0x564cfd04ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.750+0000 7f87ecdbe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03dae0 0x564cfd04ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41116/0 (socket says 172.21.15.83:41116) 2024-03-11T19:47:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.750+0000 7f87ecdbe640 1 -- 172.21.15.83:0/661565101 learned_addr learned my addr 172.21.15.83:0/661565101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.752+0000 7f87ed5bf640 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd04d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.753+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd043640 0x564cfd04daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.754+0000 7f87ed5bf640 1 -- 172.21.15.83:0/661565101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd043640 msgr2=0x564cfd04daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.754+0000 7f87ed5bf640 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd043640 0x564cfd04daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.754+0000 7f87ed5bf640 1 -- 172.21.15.83:0/661565101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03dae0 msgr2=0x564cfd04ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.754+0000 7f87ed5bf640 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03dae0 0x564cfd04ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.754+0000 7f87ed5bf640 1 -- 172.21.15.83:0/661565101 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cfcf5c570 con 0x564cfd03e4d0 2024-03-11T19:47:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.755+0000 7f87ed5bf640 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd04d5b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x564cfd03ed90 tx=0x7f87e000cec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f6e16d0cb9525c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:48.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.757+0000 7f87ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:47:48.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.757+0000 7f87ccff9640 1 -- 172.21.15.83:0/661565101 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0005250 con 0x564cfd03e4d0 2024-03-11T19:47:48.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.758+0000 7f87ccff9640 1 -- 172.21.15.83:0/661565101 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f87e00053b0 con 0x564cfd03e4d0 2024-03-11T19:47:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.760+0000 7f87f23f9740 1 -- 172.21.15.83:0/661565101 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87dc019de0 con 0x564cfd03e4d0 2024-03-11T19:47:48.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.763+0000 7f87ccff9640 1 -- 172.21.15.83:0/661565101 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87e0005570 con 0x564cfd03e4d0 2024-03-11T19:47:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.764+0000 7f87f23f9740 10 client.26500 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:48.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.764+0000 7f87f23f9740 1 -- 172.21.15.83:0/661565101 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564cfcf5e6a0 con 0x564cfd03e4d0 2024-03-11T19:47:48.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.768+0000 7f87ccff9640 1 -- 172.21.15.83:0/661565101 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87dc019de0 con 0x564cfd03e4d0 2024-03-11T19:47:48.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.768+0000 7f87ccff9640 1 -- 172.21.15.83:0/661565101 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f87e0035070 con 0x564cfd03e4d0 2024-03-11T19:47:48.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.770+0000 7f87f23f9740 10 client.26500 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:48.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.771+0000 7f87f23f9740 10 client.26500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:48.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.771+0000 7f87f23f9740 20 client.26500 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:48.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.771+0000 7f87f23f9740 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564cfd031aa0 0x564cfd033e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.771+0000 7f87f23f9740 4 client.26500 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:47:48.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.771+0000 7f87f23f9740 1 -- 172.21.15.83:0/661565101 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564cfcd29890 con 0x564cfd031aa0 2024-03-11T19:47:48.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.774+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564cfd031aa0 0x564cfd033e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.776+0000 7f87ecdbe640 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564cfd031aa0 0x564cfd033e90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a85e3236def9217 server_cookie=2f0fae3520ae2de4 in_seq=0 out_seq=0 2024-03-11T19:47:48.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.776+0000 7f87ccff9640 10 client.26500 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:48.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.776+0000 7f87ccff9640 1 -- 172.21.15.83:0/661565101 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564cfcd29890 con 0x564cfd031aa0 2024-03-11T19:47:48.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.776+0000 7f87ccff9640 10 client.26500 handle_command_reply: tid=0 2024-03-11T19:47:48.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.776+0000 7f87f23f9740 1 client.26500 shutdown 2024-03-11T19:47:48.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.778+0000 7f87f23f9740 1 -- 172.21.15.83:0/661565101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 msgr2=0x564cfd04d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.778+0000 7f87f23f9740 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd04d5b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x564cfd03ed90 tx=0x7f87e000cec0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.778+0000 7f87f23f9740 1 -- 172.21.15.83:0/661565101 shutdown_connections 2024-03-11T19:47:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.778+0000 7f87f23f9740 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564cfd031aa0 0x564cfd033e90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.778+0000 7f87f23f9740 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564cfd043640 0x564cfd04daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.778+0000 7f87f23f9740 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564cfd03e4d0 0x564cfd04d5b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.778+0000 7f87f23f9740 1 --2- 172.21.15.83:0/661565101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564cfd03dae0 0x564cfd04ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.778+0000 7f87f23f9740 1 -- 172.21.15.83:0/661565101 >> 172.21.15.83:0/661565101 conn(0x564cfd039000 msgr2=0x564cfd047a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.778+0000 7f87ccff9640 0 client.26500 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:48.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.782+0000 7f87f23f9740 1 -- 172.21.15.83:0/661565101 shutdown_connections 2024-03-11T19:47:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.782+0000 7f87f23f9740 1 -- 172.21.15.83:0/661565101 wait complete. 2024-03-11T19:47:48.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.782+0000 7f87f23f9740 20 client.26500 trim_cache size 0 max 16384 2024-03-11T19:47:48.791 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:47:48.791 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:47:48.791 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:47:48.791 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.790+0000 7f87ef823640 1 -- 172.21.15.83:0/2058271593 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87d00a1a30 msgr2=0x7f87d00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.790+0000 7f87ef823640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87d00a1a30 0x7f87d00a3ef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f87d80b9a20 tx=0x7f87d80b2770 comp rx=0 tx=0).stop 2024-03-11T19:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.790+0000 7f87ef823640 1 -- 172.21.15.83:0/2058271593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e8101c30 msgr2=0x7f87e819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.790+0000 7f87ef823640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e8101c30 0x7f87e819c530 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f87d802c630 tx=0x7f87d8034a90 comp rx=0 tx=0).stop 2024-03-11T19:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.790+0000 7f87ef823640 1 -- 172.21.15.83:0/2058271593 shutdown_connections 2024-03-11T19:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.790+0000 7f87ef823640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f87d00a1a30 0x7f87d00a3ef0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.790+0000 7f87ef823640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87e8101c30 0x7f87e819c530 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.790+0000 7f87ef823640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87e8101270 0x7f87e81954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.791+0000 7f87ef823640 1 --2- 172.21.15.83:0/2058271593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e8100080 0x7f87e8194f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.791+0000 7f87ef823640 1 -- 172.21.15.83:0/2058271593 >> 172.21.15.83:0/2058271593 conn(0x7f87e80fb5f0 msgr2=0x7f87e8104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:48.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.792+0000 7f87ef823640 1 -- 172.21.15.83:0/2058271593 shutdown_connections 2024-03-11T19:47:48.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.793+0000 7f87ef823640 1 -- 172.21.15.83:0/2058271593 wait complete. 2024-03-11T19:47:48.819 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:47:48.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:47:48.848 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:48 smithi083 ceph-mon[31864]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 MiB/s wr, 819 op/s 2024-03-11T19:47:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.970+0000 7fa517dac640 1 -- 172.21.15.83:0/1930343583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa51010adb0 msgr2=0x7fa51010d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.970+0000 7fa517dac640 1 --2- 172.21.15.83:0/1930343583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa51010adb0 0x7fa51010d1c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa5000074e0 tx=0x7fa50002c8f0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.973+0000 7fa517dac640 1 -- 172.21.15.83:0/1930343583 shutdown_connections 2024-03-11T19:47:48.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.973+0000 7fa517dac640 1 --2- 172.21.15.83:0/1930343583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51010d850 0x7fa51010fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.973+0000 7fa517dac640 1 --2- 172.21.15.83:0/1930343583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa51010adb0 0x7fa51010d1c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.973+0000 7fa517dac640 1 --2- 172.21.15.83:0/1930343583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5101083f0 0x7fa51010a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.973+0000 7fa517dac640 1 -- 172.21.15.83:0/1930343583 >> 172.21.15.83:0/1930343583 conn(0x7fa51006e390 msgr2=0x7fa5100707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:48.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.976+0000 7fa517dac640 1 -- 172.21.15.83:0/1930343583 shutdown_connections 2024-03-11T19:47:48.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.979+0000 7fa517dac640 1 -- 172.21.15.83:0/1930343583 wait complete. 2024-03-11T19:47:48.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.980+0000 7fa517dac640 1 Processor -- start 2024-03-11T19:47:48.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.980+0000 7fa517dac640 1 -- start start 2024-03-11T19:47:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.980+0000 7fa517dac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5101083f0 0x7fa51019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.980+0000 7fa517dac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa51010adb0 0x7fa51019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.980+0000 7fa517dac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51010d850 0x7fa5101ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.980+0000 7fa517dac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51019e5b0 con 0x7fa51010adb0 2024-03-11T19:47:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.980+0000 7fa517dac640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101ade20 con 0x7fa5101083f0 2024-03-11T19:47:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.980+0000 7fa517dac640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101adfb0 con 0x7fa51010d850 2024-03-11T19:47:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.981+0000 7fa516349640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51010d850 0x7fa5101ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.981+0000 7fa516349640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51010d850 0x7fa5101ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:51712/0 (socket says 172.21.15.83:51712) 2024-03-11T19:47:48.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.981+0000 7fa516349640 1 -- 172.21.15.83:0/3678669340 learned_addr learned my addr 172.21.15.83:0/3678669340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:48.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.981+0000 7fa515347640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa51010adb0 0x7fa51019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.981+0000 7fa515b48640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5101083f0 0x7fa51019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:48.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.982+0000 7fa515347640 1 -- 172.21.15.83:0/3678669340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51010d850 msgr2=0x7fa5101ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.982+0000 7fa515347640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51010d850 0x7fa5101ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.982+0000 7fa515347640 1 -- 172.21.15.83:0/3678669340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5101083f0 msgr2=0x7fa51019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:48.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.982+0000 7fa515347640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5101083f0 0x7fa51019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:48.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.982+0000 7fa515347640 1 -- 172.21.15.83:0/3678669340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5000071c0 con 0x7fa51010adb0 2024-03-11T19:47:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.982+0000 7fa515347640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa51010adb0 0x7fa51019deb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa510073380 tx=0x7fa500004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.983+0000 7fa506ffd640 1 -- 172.21.15.83:0/3678669340 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500036040 con 0x7fa51010adb0 2024-03-11T19:47:48.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.983+0000 7fa506ffd640 1 -- 172.21.15.83:0/3678669340 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa500004510 con 0x7fa51010adb0 2024-03-11T19:47:48.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.984+0000 7fa506ffd640 1 -- 172.21.15.83:0/3678669340 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50000a190 con 0x7fa51010adb0 2024-03-11T19:47:48.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.985+0000 7fa517dac640 1 -- 172.21.15.83:0/3678669340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5101ae230 con 0x7fa51010adb0 2024-03-11T19:47:48.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.985+0000 7fa517dac640 1 -- 172.21.15.83:0/3678669340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5101ae740 con 0x7fa51010adb0 2024-03-11T19:47:48.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.986+0000 7fa517dac640 1 -- 172.21.15.83:0/3678669340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa510067f70 con 0x7fa51010adb0 2024-03-11T19:47:48.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.990+0000 7fa506ffd640 1 -- 172.21.15.83:0/3678669340 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa50004a030 con 0x7fa51010adb0 2024-03-11T19:47:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.991+0000 7fa506ffd640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa4fc0a1b20 0x7fa4fc0a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:48.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.991+0000 7fa506ffd640 1 -- 172.21.15.83:0/3678669340 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa5000eeb60 con 0x7fa51010adb0 2024-03-11T19:47:49.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.994+0000 7fa515b48640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa4fc0a1b20 0x7fa4fc0a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:48.999+0000 7fa515b48640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa4fc0a1b20 0x7fa4fc0a3fe0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009890 tx=0x7fa50c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.000+0000 7fa506ffd640 1 -- 172.21.15.83:0/3678669340 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa5000b7e00 con 0x7fa51010adb0 2024-03-11T19:47:49.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:48 smithi137 ceph-mon[36220]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 MiB/s wr, 819 op/s 2024-03-11T19:47:49.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:48 smithi204 ceph-mon[36739]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 MiB/s wr, 819 op/s 2024-03-11T19:47:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.312+0000 7fa517dac640 1 -- 172.21.15.83:0/3678669340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5101156c0 con 0x7fa51010adb0 2024-03-11T19:47:49.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.314+0000 7fa506ffd640 1 -- 172.21.15.83:0/3678669340 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11350 (secure 0 0 0) 0x7fa5000bb4c0 con 0x7fa51010adb0 2024-03-11T19:47:49.319 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:47:49.319 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:45:19.152624+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:47:49.319 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 31 2024-03-11T19:47:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 -- 172.21.15.83:0/3678669340 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa4fc0a1b20 msgr2=0x7fa4fc0a3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa4fc0a1b20 0x7fa4fc0a3fe0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009890 tx=0x7fa50c008040 comp rx=0 tx=0).stop 2024-03-11T19:47:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 -- 172.21.15.83:0/3678669340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa51010adb0 msgr2=0x7fa51019deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa51010adb0 0x7fa51019deb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa510073380 tx=0x7fa500004d80 comp rx=0 tx=0).stop 2024-03-11T19:47:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 -- 172.21.15.83:0/3678669340 shutdown_connections 2024-03-11T19:47:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa4fc0a1b20 0x7fa4fc0a3fe0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa51010d850 0x7fa5101ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa51010adb0 0x7fa51019deb0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 --2- 172.21.15.83:0/3678669340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5101083f0 0x7fa51019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 -- 172.21.15.83:0/3678669340 >> 172.21.15.83:0/3678669340 conn(0x7fa51006e390 msgr2=0x7fa5101129d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 -- 172.21.15.83:0/3678669340 shutdown_connections 2024-03-11T19:47:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.316+0000 7fa517dac640 1 -- 172.21.15.83:0/3678669340 wait complete. 2024-03-11T19:47:49.335 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:47:49.335 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:47:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.527+0000 7efcdd546640 1 -- 172.21.15.83:0/3550687994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0032ad0 con 0x7efcd8102d60 2024-03-11T19:47:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.528+0000 7efce07ac640 1 -- 172.21.15.83:0/3550687994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcd8102d60 msgr2=0x7efcd81095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.528+0000 7efce07ac640 1 --2- 172.21.15.83:0/3550687994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcd8102d60 0x7efcd81095f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009ca0 tx=0x7efcd002c100 comp rx=0 tx=0).stop 2024-03-11T19:47:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.528+0000 7efce07ac640 1 -- 172.21.15.83:0/3550687994 shutdown_connections 2024-03-11T19:47:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.528+0000 7efce07ac640 1 --2- 172.21.15.83:0/3550687994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcd8102d60 0x7efcd81095f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.528+0000 7efce07ac640 1 --2- 172.21.15.83:0/3550687994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd81023a0 0x7efcd8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.528+0000 7efce07ac640 1 --2- 172.21.15.83:0/3550687994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd81011b0 0x7efcd81015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.528+0000 7efce07ac640 1 -- 172.21.15.83:0/3550687994 >> 172.21.15.83:0/3550687994 conn(0x7efcd80fc720 msgr2=0x7efcd80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.528+0000 7efce07ac640 1 -- 172.21.15.83:0/3550687994 shutdown_connections 2024-03-11T19:47:49.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.529+0000 7efce07ac640 1 -- 172.21.15.83:0/3550687994 wait complete. 2024-03-11T19:47:49.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.529+0000 7efce07ac640 1 Processor -- start 2024-03-11T19:47:49.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.529+0000 7efce07ac640 1 -- start start 2024-03-11T19:47:49.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.529+0000 7efce07ac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd81011b0 0x7efcd8195090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efce07ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcd81023a0 0x7efcd81955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efce07ac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd8102d60 0x7efcd819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efce07ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd819cb80 con 0x7efcd81023a0 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efce07ac640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd819ccf0 con 0x7efcd81011b0 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efce07ac640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd819ceb0 con 0x7efcd8102d60 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efcddd47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcd81023a0 0x7efcd81955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efcddd47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcd81023a0 0x7efcd81955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36178/0 (socket says 172.21.15.83:36178) 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efcddd47640 1 -- 172.21.15.83:0/829629575 learned_addr learned my addr 172.21.15.83:0/829629575 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efcded49640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd8102d60 0x7efcd819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efcde548640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd81011b0 0x7efcd8195090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efcded49640 1 -- 172.21.15.83:0/829629575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd81011b0 msgr2=0x7efcd8195090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efcded49640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd81011b0 0x7efcd8195090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efcded49640 1 -- 172.21.15.83:0/829629575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcd81023a0 msgr2=0x7efcd81955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efcded49640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcd81023a0 0x7efcd81955d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efcded49640 1 -- 172.21.15.83:0/829629575 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd0009900 con 0x7efcd8102d60 2024-03-11T19:47:49.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.530+0000 7efcddd47640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcd81023a0 0x7efcd81955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:49.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.531+0000 7efcded49640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd8102d60 0x7efcd819c640 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efcd002c640 tx=0x7efcd0034ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.531+0000 7efccf7fe640 1 -- 172.21.15.83:0/829629575 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0034850 con 0x7efcd8102d60 2024-03-11T19:47:49.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.531+0000 7efccf7fe640 1 -- 172.21.15.83:0/829629575 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcd0003e50 con 0x7efcd8102d60 2024-03-11T19:47:49.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.531+0000 7efccf7fe640 1 -- 172.21.15.83:0/829629575 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0040450 con 0x7efcd8102d60 2024-03-11T19:47:49.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.532+0000 7efce07ac640 1 -- 172.21.15.83:0/829629575 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd819d130 con 0x7efcd8102d60 2024-03-11T19:47:49.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.532+0000 7efce07ac640 1 -- 172.21.15.83:0/829629575 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd819d560 con 0x7efcd8102d60 2024-03-11T19:47:49.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.534+0000 7efccf7fe640 1 -- 172.21.15.83:0/829629575 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7efcd000a040 con 0x7efcd8102d60 2024-03-11T19:47:49.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.534+0000 7efccf7fe640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efcc40a5ec0 0x7efcc40a8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.534+0000 7efce337f740 1 Processor -- start 2024-03-11T19:47:49.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efccf7fe640 1 -- 172.21.15.83:0/829629575 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7efcd00efd90 con 0x7efcd8102d60 2024-03-11T19:47:49.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efcde548640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efcc40a5ec0 0x7efcc40a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efce337f740 1 -- start start 2024-03-11T19:47:49.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efce337f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 0x558d8d759690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efce337f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d759c60 0x558d8d75e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efce337f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d75edd0 0x558d8d761260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efce337f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d715960 con 0x558d8d75edd0 2024-03-11T19:47:49.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efce337f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d6c9310 con 0x558d8d759270 2024-03-11T19:47:49.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efce337f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d6c6840 con 0x558d8d759c60 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efcded49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d75edd0 0x558d8d761260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efcddd47640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d759c60 0x558d8d75e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efcde548640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efcc40a5ec0 0x7efcc40a8380 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efcd8102200 tx=0x7efcd4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efcded49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d75edd0 0x558d8d761260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36194/0 (socket says 172.21.15.83:36194) 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efcddd47640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d759c60 0x558d8d75e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41016/0 (socket says 172.21.15.83:41016) 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efcded49640 1 -- 172.21.15.83:0/3251340182 learned_addr learned my addr 172.21.15.83:0/3251340182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.535+0000 7efcde548640 1 --2- 172.21.15.83:0/3251340182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 0x558d8d759690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.536+0000 7efcded49640 1 -- 172.21.15.83:0/3251340182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d759c60 msgr2=0x558d8d75e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.536+0000 7efcded49640 1 --2- 172.21.15.83:0/3251340182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d759c60 0x558d8d75e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.536+0000 7efcded49640 1 -- 172.21.15.83:0/3251340182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 msgr2=0x558d8d759690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.536+0000 7efcded49640 1 --2- 172.21.15.83:0/3251340182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 0x558d8d759690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.536+0000 7efcded49640 1 -- 172.21.15.83:0/3251340182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d8d684060 con 0x558d8d75edd0 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.536+0000 7efcded49640 1 --2- 172.21.15.83:0/3251340182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d75edd0 0x558d8d761260 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7efcd00b9810 tx=0x7efcd0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809ac05b5d2c7ac3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.536+0000 7efcccff9640 1 -- 172.21.15.83:0/3251340182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd00c0f90 con 0x558d8d75edd0 2024-03-11T19:47:49.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.536+0000 7efcccff9640 1 -- 172.21.15.83:0/3251340182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcd00c5550 con 0x558d8d75edd0 2024-03-11T19:47:49.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.536+0000 7efcccff9640 1 -- 172.21.15.83:0/3251340182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd00040d0 con 0x558d8d75edd0 2024-03-11T19:47:49.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.538+0000 7efce337f740 1 -- 172.21.15.83:0/3251340182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d75edd0 msgr2=0x558d8d761260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.538+0000 7efce337f740 1 --2- 172.21.15.83:0/3251340182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d75edd0 0x558d8d761260 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7efcd00b9810 tx=0x7efcd0004040 comp rx=0 tx=0).stop 2024-03-11T19:47:49.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.538+0000 7efce337f740 1 -- 172.21.15.83:0/3251340182 shutdown_connections 2024-03-11T19:47:49.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.538+0000 7efce337f740 1 --2- 172.21.15.83:0/3251340182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d75edd0 0x558d8d761260 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.538+0000 7efce337f740 1 --2- 172.21.15.83:0/3251340182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d759c60 0x558d8d75e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.538+0000 7efce337f740 1 --2- 172.21.15.83:0/3251340182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 0x558d8d759690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.538+0000 7efce337f740 1 -- 172.21.15.83:0/3251340182 >> 172.21.15.83:0/3251340182 conn(0x558d8d7547e0 msgr2=0x558d8d754be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.538+0000 7efce337f740 1 -- 172.21.15.83:0/3251340182 shutdown_connections 2024-03-11T19:47:49.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.538+0000 7efce337f740 1 -- 172.21.15.83:0/3251340182 wait complete. 2024-03-11T19:47:49.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.539+0000 7efce337f740 1 -- start start 2024-03-11T19:47:49.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.539+0000 7efce337f740 1 Processor -- start 2024-03-11T19:47:49.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.539+0000 7efce337f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 0x558d8d763160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.539+0000 7efce337f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d759c60 0x558d8d7636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.539+0000 7efce337f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d75edd0 0x558d8d76a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.540+0000 7efce337f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d6c2020 con 0x558d8d759270 2024-03-11T19:47:49.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.540+0000 7efce337f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d395430 con 0x558d8d75edd0 2024-03-11T19:47:49.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.540+0000 7efce337f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d351e00 con 0x558d8d759c60 2024-03-11T19:47:49.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.540+0000 7efce337f740 10 client.0 _ll_register_callbacks cb 0x558d8d669cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:49.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.542+0000 7efcded49640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d75edd0 0x558d8d76a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.542+0000 7efcded49640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d75edd0 0x558d8d76a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:35988/0 (socket says 172.21.15.83:35988) 2024-03-11T19:47:49.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.542+0000 7efcded49640 1 -- 172.21.15.83:0/3117476550 learned_addr learned my addr 172.21.15.83:0/3117476550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:49.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.545+0000 7efcde548640 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 0x558d8d763160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.547+0000 7efcde548640 1 -- 172.21.15.83:0/3117476550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d759c60 msgr2=0x558d8d7636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.547+0000 7efcde548640 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d759c60 0x558d8d7636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.547+0000 7efcde548640 1 -- 172.21.15.83:0/3117476550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d75edd0 msgr2=0x558d8d76a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.547+0000 7efcde548640 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d75edd0 0x558d8d76a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.547+0000 7efcde548640 1 -- 172.21.15.83:0/3117476550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd819d560 con 0x558d8d759270 2024-03-11T19:47:49.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.548+0000 7efcde548640 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 0x558d8d763160 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009fd0 tx=0x7efcd40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85c60f1105222f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.548+0000 7efcc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:47:49.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.549+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3117476550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4013720 con 0x558d8d759270 2024-03-11T19:47:49.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.549+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3117476550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7efcd4013880 con 0x558d8d759270 2024-03-11T19:47:49.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.551+0000 7efce337f740 1 -- 172.21.15.83:0/3117476550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d8d3c17e0 con 0x558d8d759270 2024-03-11T19:47:49.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.554+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3117476550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcd4013a40 con 0x558d8d759270 2024-03-11T19:47:49.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.554+0000 7efce337f740 10 client.16944 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:49.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.554+0000 7efce337f740 1 -- 172.21.15.83:0/3117476550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558d8d6c06a0 con 0x558d8d759270 2024-03-11T19:47:49.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.556+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3117476550 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcd4013ba0 con 0x558d8d759270 2024-03-11T19:47:49.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.556+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3117476550 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7efcd4011070 con 0x558d8d759270 2024-03-11T19:47:49.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.557+0000 7efce337f740 10 client.16944 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:49.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.557+0000 7efce337f740 10 client.16944 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:47:49.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.557+0000 7efce337f740 20 client.16944 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:49.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.557+0000 7efce337f740 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x558d8d76f380 0x558d8d771820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.557+0000 7efce337f740 4 client.16944 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:49.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.557+0000 7efce337f740 1 -- 172.21.15.83:0/3117476550 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d8d3c17e0 con 0x558d8d76f380 2024-03-11T19:47:49.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.561+0000 7efcddd47640 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x558d8d76f380 0x558d8d771820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.570+0000 7efcddd47640 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x558d8d76f380 0x558d8d771820 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3074dd85cf4bcda server_cookie=85ba8caa9dea3d80 in_seq=0 out_seq=0 2024-03-11T19:47:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.571+0000 7efcc1ffb640 10 client.16944 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:47:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.572+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3117476550 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d8d3c17e0 con 0x558d8d76f380 2024-03-11T19:47:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.572+0000 7efcc1ffb640 10 client.16944 handle_command_reply: tid=0 2024-03-11T19:47:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.572+0000 7efce337f740 1 client.16944 shutdown 2024-03-11T19:47:49.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.573+0000 7efce337f740 1 -- 172.21.15.83:0/3117476550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 msgr2=0x558d8d763160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.573+0000 7efce337f740 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 0x558d8d763160 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009fd0 tx=0x7efcd40047d0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.574+0000 7efce337f740 1 -- 172.21.15.83:0/3117476550 shutdown_connections 2024-03-11T19:47:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.574+0000 7efce337f740 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x558d8d76f380 0x558d8d771820 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.574+0000 7efce337f740 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d75edd0 0x558d8d76a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.574+0000 7efcc1ffb640 0 client.16944 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:47:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.574+0000 7efce337f740 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d759c60 0x558d8d7636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.574+0000 7efce337f740 1 --2- 172.21.15.83:0/3117476550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 0x558d8d763160 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.574+0000 7efce337f740 1 -- 172.21.15.83:0/3117476550 >> 172.21.15.83:0/3117476550 conn(0x558d8d7547e0 msgr2=0x558d8d753b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.574+0000 7efce337f740 1 -- 172.21.15.83:0/3117476550 shutdown_connections 2024-03-11T19:47:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.574+0000 7efce337f740 1 -- 172.21.15.83:0/3117476550 wait complete. 2024-03-11T19:47:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.575+0000 7efce337f740 20 client.16944 trim_cache size 0 max 16384 2024-03-11T19:47:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.595+0000 7efce337f740 1 Processor -- start 2024-03-11T19:47:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efce337f740 1 -- start start 2024-03-11T19:47:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efce337f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 0x558d8d762d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efce337f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759c60 0x558d8d76a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efce337f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d75edd0 0x558d8d7632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efce337f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d647130 con 0x558d8d759c60 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efce337f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d3c17e0 con 0x558d8d759270 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efce337f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d7641b0 con 0x558d8d75edd0 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efcddd47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759c60 0x558d8d76a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efcde548640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 0x558d8d762d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efcde548640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 0x558d8d762d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:35994/0 (socket says 172.21.15.83:35994) 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efcde548640 1 -- 172.21.15.83:0/3097009426 learned_addr learned my addr 172.21.15.83:0/3097009426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efcddd47640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759c60 0x558d8d76a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36204/0 (socket says 172.21.15.83:36204) 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.596+0000 7efcded49640 1 --2- 172.21.15.83:0/3097009426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d75edd0 0x558d8d7632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.597+0000 7efcde548640 1 -- 172.21.15.83:0/3097009426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d75edd0 msgr2=0x558d8d7632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.597+0000 7efcde548640 1 --2- 172.21.15.83:0/3097009426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d75edd0 0x558d8d7632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.597+0000 7efcde548640 1 -- 172.21.15.83:0/3097009426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759c60 msgr2=0x558d8d76a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.597+0000 7efcde548640 1 --2- 172.21.15.83:0/3097009426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759c60 0x558d8d76a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.597+0000 7efcde548640 1 -- 172.21.15.83:0/3097009426 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d8d65cc20 con 0x558d8d759270 2024-03-11T19:47:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.597+0000 7efcde548640 1 --2- 172.21.15.83:0/3097009426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 0x558d8d762d70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efcd4021440 tx=0x7efcd4006b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd87608bafc3d09 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.598+0000 7efcc3fff640 1 -- 172.21.15.83:0/3097009426 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4021ce0 con 0x558d8d759270 2024-03-11T19:47:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.598+0000 7efcc3fff640 1 -- 172.21.15.83:0/3097009426 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efcd400ede0 con 0x558d8d759270 2024-03-11T19:47:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.598+0000 7efcc3fff640 1 -- 172.21.15.83:0/3097009426 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4034980 con 0x558d8d759270 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 -- 172.21.15.83:0/3097009426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 msgr2=0x558d8d762d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 --2- 172.21.15.83:0/3097009426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 0x558d8d762d70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efcd4021440 tx=0x7efcd4006b20 comp rx=0 tx=0).stop 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 -- 172.21.15.83:0/3097009426 shutdown_connections 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 --2- 172.21.15.83:0/3097009426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d75edd0 0x558d8d7632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 --2- 172.21.15.83:0/3097009426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759c60 0x558d8d76a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 --2- 172.21.15.83:0/3097009426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759270 0x558d8d762d70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 -- 172.21.15.83:0/3097009426 >> 172.21.15.83:0/3097009426 conn(0x558d8d7547e0 msgr2=0x558d8d754bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 -- 172.21.15.83:0/3097009426 shutdown_connections 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 -- 172.21.15.83:0/3097009426 wait complete. 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 -- start start 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 Processor -- start 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 0x558d8d7695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759c60 0x558d8d769ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d75edd0 0x558d8d7c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d76a0f0 con 0x558d8d759270 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d7c5e70 con 0x558d8d759c60 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8d7c5fb0 con 0x558d8d75edd0 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efce337f740 10 client.0 _ll_register_callbacks cb 0x7efcd400e850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efcde548640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 0x558d8d7695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efcde548640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 0x558d8d7695a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36214/0 (socket says 172.21.15.83:36214) 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efcde548640 1 -- 172.21.15.83:0/3425903020 learned_addr learned my addr 172.21.15.83:0/3425903020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.599+0000 7efcddd47640 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759c60 0x558d8d769ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcded49640 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d75edd0 0x558d8d7c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcde548640 1 -- 172.21.15.83:0/3425903020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d75edd0 msgr2=0x558d8d7c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcde548640 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d75edd0 0x558d8d7c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcde548640 1 -- 172.21.15.83:0/3425903020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759c60 msgr2=0x558d8d769ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcde548640 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759c60 0x558d8d769ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcde548640 1 -- 172.21.15.83:0/3425903020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d8d6c06a0 con 0x558d8d759270 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcde548640 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 0x558d8d7695a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7efcd40214c0 tx=0x7efcd4013950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd42fb8b3c2dd52 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:47:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3425903020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4013cd0 con 0x558d8d759270 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3425903020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7efcd4009700 con 0x558d8d759270 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efce337f740 1 -- 172.21.15.83:0/3425903020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d8d7500f0 con 0x558d8d759270 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3425903020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcd4013e70 con 0x558d8d759270 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efce337f740 10 client.16950 fetch_fsmap learned FSMap version 31 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efce337f740 1 -- 172.21.15.83:0/3425903020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558d8d6828c0 con 0x558d8d759270 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.600+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3425903020 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558d8d7500f0 con 0x558d8d759270 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.601+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3425903020 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 31) v1 ==== 7886+0+0 (secure 0 0 0) 0x7efcd4006020 con 0x558d8d759270 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.601+0000 7efce337f740 10 client.16950 fetch_fsmap finished waiting for FSMap version 31 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.601+0000 7efce337f740 10 client.16950 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.601+0000 7efce337f740 20 client.16950 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.601+0000 7efce337f740 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x558d8d76f380 0x558d8d7c9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.601+0000 7efce337f740 4 client.16950 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.601+0000 7efce337f740 1 -- 172.21.15.83:0/3425903020 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7efcd00c5550 con 0x558d8d76f380 2024-03-11T19:47:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.601+0000 7efcddd47640 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x558d8d76f380 0x558d8d7c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.601+0000 7efcc1ffb640 10 client.16950 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.601+0000 7efcddd47640 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x558d8d76f380 0x558d8d7c9520 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=adfe6e04fddb1c1c server_cookie=6fd8de37ce689f29 in_seq=0 out_seq=0 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efcc1ffb640 1 -- 172.21.15.83:0/3425903020 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7efcd00c5550 con 0x558d8d76f380 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efcc1ffb640 10 client.16950 handle_command_reply: tid=0 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 1 client.16950 shutdown 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 1 -- 172.21.15.83:0/3425903020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 msgr2=0x558d8d7695a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 0x558d8d7695a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7efcd40214c0 tx=0x7efcd4013950 comp rx=0 tx=0).stop 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 1 -- 172.21.15.83:0/3425903020 shutdown_connections 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x558d8d76f380 0x558d8d7c9520 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d8d75edd0 0x558d8d7c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efcc1ffb640 0 client.16950 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558d8d759c60 0x558d8d769ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 1 --2- 172.21.15.83:0/3425903020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d8d759270 0x558d8d7695a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 1 -- 172.21.15.83:0/3425903020 >> 172.21.15.83:0/3425903020 conn(0x558d8d7547e0 msgr2=0x558d8d762ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 1 -- 172.21.15.83:0/3425903020 shutdown_connections 2024-03-11T19:47:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 1 -- 172.21.15.83:0/3425903020 wait complete. 2024-03-11T19:47:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.602+0000 7efce337f740 20 client.16950 trim_cache size 0 max 16384 2024-03-11T19:47:49.612 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 -- 172.21.15.83:0/829629575 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efcc40a5ec0 msgr2=0x7efcc40a8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efcc40a5ec0 0x7efcc40a8380 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efcd8102200 tx=0x7efcd4009110 comp rx=0 tx=0).stop 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 -- 172.21.15.83:0/829629575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd8102d60 msgr2=0x7efcd819c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd8102d60 0x7efcd819c640 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efcd002c640 tx=0x7efcd0034ea0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 -- 172.21.15.83:0/829629575 shutdown_connections 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7efcc40a5ec0 0x7efcc40a8380 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd8102d60 0x7efcd819c640 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efcd81023a0 0x7efcd81955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 --2- 172.21.15.83:0/829629575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd81011b0 0x7efcd8195090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 -- 172.21.15.83:0/829629575 >> 172.21.15.83:0/829629575 conn(0x7efcd80fc720 msgr2=0x7efcd8074e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 -- 172.21.15.83:0/829629575 shutdown_connections 2024-03-11T19:47:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.604+0000 7efce07ac640 1 -- 172.21.15.83:0/829629575 wait complete. 2024-03-11T19:47:49.623 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:47:49.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:47:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.781+0000 7fe546327640 1 -- 172.21.15.83:0/2807096914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe54010aaf0 msgr2=0x7fe54010cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.781+0000 7fe546327640 1 --2- 172.21.15.83:0/2807096914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe54010aaf0 0x7fe54010cf60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe534009900 tx=0x7fe53402bc60 comp rx=0 tx=0).stop 2024-03-11T19:47:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.781+0000 7fe546327640 1 -- 172.21.15.83:0/2807096914 shutdown_connections 2024-03-11T19:47:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.781+0000 7fe546327640 1 --2- 172.21.15.83:0/2807096914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54010d4a0 0x7fe54010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.781+0000 7fe546327640 1 --2- 172.21.15.83:0/2807096914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe54010aaf0 0x7fe54010cf60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.781+0000 7fe546327640 1 --2- 172.21.15.83:0/2807096914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe540108180 0x7fe54010a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.781+0000 7fe546327640 1 -- 172.21.15.83:0/2807096914 >> 172.21.15.83:0/2807096914 conn(0x7fe54006dfe0 msgr2=0x7fe540070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.786+0000 7fe546327640 1 -- 172.21.15.83:0/2807096914 shutdown_connections 2024-03-11T19:47:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.786+0000 7fe546327640 1 -- 172.21.15.83:0/2807096914 wait complete. 2024-03-11T19:47:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.786+0000 7fe546327640 1 Processor -- start 2024-03-11T19:47:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.786+0000 7fe546327640 1 -- start start 2024-03-11T19:47:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.787+0000 7fe546327640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe540108180 0x7fe5401a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.787+0000 7fe546327640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54010aaf0 0x7fe5401a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.787+0000 7fe546327640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54010d4a0 0x7fe5400793e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.787+0000 7fe546327640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe540079920 con 0x7fe54010d4a0 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.787+0000 7fe546327640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe540079a90 con 0x7fe54010aaf0 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.787+0000 7fe546327640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe540079c50 con 0x7fe540108180 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.787+0000 7fe5448c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54010d4a0 0x7fe5400793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.788+0000 7fe53ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe540108180 0x7fe5401a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.788+0000 7fe5448c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54010d4a0 0x7fe5400793e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36234/0 (socket says 172.21.15.83:36234) 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.788+0000 7fe5448c4640 1 -- 172.21.15.83:0/98716236 learned_addr learned my addr 172.21.15.83:0/98716236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.788+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54010aaf0 0x7fe5401a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.788+0000 7fe5448c4640 1 -- 172.21.15.83:0/98716236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe540108180 msgr2=0x7fe5401a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.788+0000 7fe5448c4640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe540108180 0x7fe5401a60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.788+0000 7fe5448c4640 1 -- 172.21.15.83:0/98716236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54010aaf0 msgr2=0x7fe5401a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.788+0000 7fe5448c4640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54010aaf0 0x7fe5401a65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.788+0000 7fe5448c4640 1 -- 172.21.15.83:0/98716236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe534009560 con 0x7fe54010d4a0 2024-03-11T19:47:49.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.788+0000 7fe5448c4640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54010d4a0 0x7fe5400793e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe53800eb40 tx=0x7fe53800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.792+0000 7fe53d7fa640 1 -- 172.21.15.83:0/98716236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538004b20 con 0x7fe54010d4a0 2024-03-11T19:47:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.792+0000 7fe546327640 1 -- 172.21.15.83:0/98716236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe540079f30 con 0x7fe54010d4a0 2024-03-11T19:47:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.792+0000 7fe546327640 1 -- 172.21.15.83:0/98716236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54007a440 con 0x7fe54010d4a0 2024-03-11T19:47:49.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.792+0000 7fe53d7fa640 1 -- 172.21.15.83:0/98716236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe538010910 con 0x7fe54010d4a0 2024-03-11T19:47:49.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.792+0000 7fe53d7fa640 1 -- 172.21.15.83:0/98716236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53800f920 con 0x7fe54010d4a0 2024-03-11T19:47:49.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.800+0000 7fe53d7fa640 1 -- 172.21.15.83:0/98716236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe538010430 con 0x7fe54010d4a0 2024-03-11T19:47:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.802+0000 7fe548fce740 1 Processor -- start 2024-03-11T19:47:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.802+0000 7fe548fce740 1 -- start start 2024-03-11T19:47:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.802+0000 7fe548fce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c1b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.802+0000 7fe548fce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.802+0000 7fe548fce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c20dd0 0x5585a8c23260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.802+0000 7fe548fce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8bd7960 con 0x5585a8c20dd0 2024-03-11T19:47:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.802+0000 7fe548fce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8b8b310 con 0x5585a8c1bc60 2024-03-11T19:47:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.802+0000 7fe548fce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8b88840 con 0x5585a8c1b270 2024-03-11T19:47:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.804+0000 7fe53d7fa640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe5280a1b80 0x7fe5280a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.804+0000 7fe53d7fa640 1 -- 172.21.15.83:0/98716236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe538014070 con 0x7fe54010d4a0 2024-03-11T19:47:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.806+0000 7fe53f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c20dd0 0x5585a8c23260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.806+0000 7fe53f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c20dd0 0x5585a8c23260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36246/0 (socket says 172.21.15.83:36246) 2024-03-11T19:47:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.806+0000 7fe53f7fe640 1 -- 172.21.15.83:0/1936154955 learned_addr learned my addr 172.21.15.83:0/1936154955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:49.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.806+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/1936154955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c1b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.807+0000 7fe5448c4640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe5280a1b80 0x7fe5280a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.808+0000 7fe53f7fe640 1 -- 172.21.15.83:0/1936154955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 msgr2=0x5585a8c20760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.808+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/1936154955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.808+0000 7fe53f7fe640 1 -- 172.21.15.83:0/1936154955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c20dd0 msgr2=0x5585a8c23260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.808+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/1936154955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c20dd0 0x5585a8c23260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.808+0000 7fe53f7fe640 1 -- 172.21.15.83:0/1936154955 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585a8b46060 con 0x5585a8c1b270 2024-03-11T19:47:49.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.808+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/1936154955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c1b690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe53402c270 tx=0x7fe534005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2b812fd1b0a01a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.808+0000 7fe5267fc640 1 -- 172.21.15.83:0/1936154955 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5340048c0 con 0x5585a8c1b270 2024-03-11T19:47:49.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.808+0000 7fe5267fc640 1 -- 172.21.15.83:0/1936154955 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe534004510 con 0x5585a8c1b270 2024-03-11T19:47:49.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.808+0000 7fe5267fc640 1 -- 172.21.15.83:0/1936154955 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534003ca0 con 0x5585a8c1b270 2024-03-11T19:47:49.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.810+0000 7fe548fce740 1 -- 172.21.15.83:0/1936154955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 msgr2=0x5585a8c1b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.810+0000 7fe548fce740 1 --2- 172.21.15.83:0/1936154955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c1b690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe53402c270 tx=0x7fe534005fd0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.810+0000 7fe548fce740 1 -- 172.21.15.83:0/1936154955 shutdown_connections 2024-03-11T19:47:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.810+0000 7fe548fce740 1 --2- 172.21.15.83:0/1936154955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c20dd0 0x5585a8c23260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.810+0000 7fe548fce740 1 --2- 172.21.15.83:0/1936154955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c20760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.810+0000 7fe548fce740 1 --2- 172.21.15.83:0/1936154955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c1b690 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.810+0000 7fe548fce740 1 -- 172.21.15.83:0/1936154955 >> 172.21.15.83:0/1936154955 conn(0x5585a8c167e0 msgr2=0x5585a8c16be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.811+0000 7fe5448c4640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe5280a1b80 0x7fe5280a4040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe538004af0 tx=0x7fe538004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.814+0000 7fe548fce740 1 -- 172.21.15.83:0/1936154955 shutdown_connections 2024-03-11T19:47:49.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.814+0000 7fe548fce740 1 -- 172.21.15.83:0/1936154955 wait complete. 2024-03-11T19:47:49.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.814+0000 7fe548fce740 1 -- start start 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.815+0000 7fe548fce740 1 Processor -- start 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.815+0000 7fe548fce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c1b270 0x5585a8c255a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.815+0000 7fe548fce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c25ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.815+0000 7fe548fce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c20dd0 0x5585a8c2cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.815+0000 7fe548fce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8b84020 con 0x5585a8c1b270 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.815+0000 7fe548fce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8857430 con 0x5585a8c1bc60 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.815+0000 7fe548fce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8813e00 con 0x5585a8c20dd0 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.815+0000 7fe548fce740 10 client.0 _ll_register_callbacks cb 0x5585a8b2bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.816+0000 7fe53ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c25ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.816+0000 7fe53ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c25ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:36044/0 (socket says 172.21.15.83:36044) 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.816+0000 7fe53ffff640 1 -- 172.21.15.83:0/2964879050 learned_addr learned my addr 172.21.15.83:0/2964879050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:49.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.816+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c20dd0 0x5585a8c2cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.816+0000 7fe53ffff640 1 -- 172.21.15.83:0/2964879050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c20dd0 msgr2=0x5585a8c2cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.816+0000 7fe53ffff640 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c20dd0 0x5585a8c2cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.816+0000 7fe53ffff640 1 -- 172.21.15.83:0/2964879050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c1b270 msgr2=0x5585a8c255a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.816+0000 7fe53ffff640 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c1b270 0x5585a8c255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.817+0000 7fe53ffff640 1 -- 172.21.15.83:0/2964879050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54010e4b0 con 0x5585a8c1bc60 2024-03-11T19:47:49.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.817+0000 7fe53ffff640 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c25ae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe5300105f0 tx=0x7fe530010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1476df10beb81d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.818+0000 7fe51f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:47:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.819+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2964879050 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530004220 con 0x5585a8c1bc60 2024-03-11T19:47:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.819+0000 7fe548fce740 1 -- 172.21.15.83:0/2964879050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585a88837e0 con 0x5585a8c1bc60 2024-03-11T19:47:49.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.819+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2964879050 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe530016070 con 0x5585a8c1bc60 2024-03-11T19:47:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.822+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2964879050 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe530020030 con 0x5585a8c1bc60 2024-03-11T19:47:49.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.822+0000 7fe548fce740 10 client.26417 fetch_fsmap learned FSMap version 32 2024-03-11T19:47:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.822+0000 7fe548fce740 1 -- 172.21.15.83:0/2964879050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5585a8b826a0 con 0x5585a8c1bc60 2024-03-11T19:47:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.823+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2964879050 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5585a88837e0 con 0x5585a8c1bc60 2024-03-11T19:47:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.823+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2964879050 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe5300038f0 con 0x5585a8c1bc60 2024-03-11T19:47:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.823+0000 7fe548fce740 10 client.26417 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:47:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.823+0000 7fe548fce740 10 client.26417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:47:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.823+0000 7fe548fce740 20 client.26417 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.823+0000 7fe548fce740 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5585a8c31750 0x5585a8c33b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.823+0000 7fe548fce740 4 client.26417 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.823+0000 7fe548fce740 1 -- 172.21.15.83:0/2964879050 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585a88c0720 con 0x5585a8c31750 2024-03-11T19:47:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.823+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5585a8c31750 0x5585a8c33b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.825+0000 7fe51f7fe640 10 client.26417 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:47:49.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.825+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5585a8c31750 0x5585a8c33b70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62453a5054db782b server_cookie=f8fb1970769ec43 in_seq=0 out_seq=0 2024-03-11T19:47:49.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.825+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2964879050 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585a88c0720 con 0x5585a8c31750 2024-03-11T19:47:49.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.825+0000 7fe51f7fe640 10 client.26417 handle_command_reply: tid=0 2024-03-11T19:47:49.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.825+0000 7fe548fce740 1 client.26417 shutdown 2024-03-11T19:47:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.826+0000 7fe548fce740 1 -- 172.21.15.83:0/2964879050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 msgr2=0x5585a8c25ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.826+0000 7fe548fce740 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c25ae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe5300105f0 tx=0x7fe530010ab0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.826+0000 7fe548fce740 1 -- 172.21.15.83:0/2964879050 shutdown_connections 2024-03-11T19:47:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.826+0000 7fe548fce740 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5585a8c31750 0x5585a8c33b70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.826+0000 7fe548fce740 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c20dd0 0x5585a8c2cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.826+0000 7fe548fce740 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c25ae0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.826+0000 7fe548fce740 1 --2- 172.21.15.83:0/2964879050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c1b270 0x5585a8c255a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.826+0000 7fe548fce740 1 -- 172.21.15.83:0/2964879050 >> 172.21.15.83:0/2964879050 conn(0x5585a8c167e0 msgr2=0x5585a8c15b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.826+0000 7fe51f7fe640 0 client.26417 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:47:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.826+0000 7fe548fce740 1 -- 172.21.15.83:0/2964879050 shutdown_connections 2024-03-11T19:47:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.827+0000 7fe548fce740 1 -- 172.21.15.83:0/2964879050 wait complete. 2024-03-11T19:47:49.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.827+0000 7fe548fce740 20 client.26417 trim_cache size 0 max 16384 2024-03-11T19:47:49.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.849+0000 7fe548fce740 1 Processor -- start 2024-03-11T19:47:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.851+0000 7fe548fce740 1 -- start start 2024-03-11T19:47:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.851+0000 7fe548fce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.851+0000 7fe548fce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c1bc60 0x5585a8c2ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.851+0000 7fe548fce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c20dd0 0x5585a8c255a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.851+0000 7fe548fce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8b09130 con 0x5585a8c1bc60 2024-03-11T19:47:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.851+0000 7fe548fce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8c1cde0 con 0x5585a8c20dd0 2024-03-11T19:47:49.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.851+0000 7fe548fce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8c12320 con 0x5585a8c1b270 2024-03-11T19:47:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.851+0000 7fe53f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.851+0000 7fe53f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c384f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41074/0 (socket says 172.21.15.83:41074) 2024-03-11T19:47:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.852+0000 7fe53f7fe640 1 -- 172.21.15.83:0/571401417 learned_addr learned my addr 172.21.15.83:0/571401417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.852+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/571401417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c20dd0 0x5585a8c255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.852+0000 7fe53f7fe640 1 -- 172.21.15.83:0/571401417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c20dd0 msgr2=0x5585a8c255a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.852+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/571401417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c20dd0 0x5585a8c255a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.852+0000 7fe53f7fe640 1 -- 172.21.15.83:0/571401417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c1bc60 msgr2=0x5585a8c2ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.852+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/571401417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c1bc60 0x5585a8c2ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.852+0000 7fe53f7fe640 1 -- 172.21.15.83:0/571401417 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585a8b1ec20 con 0x5585a8c1b270 2024-03-11T19:47:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.852+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/571401417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c384f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe534005ab0 tx=0x7fe53402c180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43e41ffadc4fba25 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.853+0000 7fe525ffb640 1 -- 172.21.15.83:0/571401417 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534004800 con 0x5585a8c1b270 2024-03-11T19:47:49.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.853+0000 7fe525ffb640 1 -- 172.21.15.83:0/571401417 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe534004960 con 0x5585a8c1b270 2024-03-11T19:47:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.854+0000 7fe548fce740 1 -- 172.21.15.83:0/571401417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 msgr2=0x5585a8c384f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.854+0000 7fe548fce740 1 --2- 172.21.15.83:0/571401417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c384f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe534005ab0 tx=0x7fe53402c180 comp rx=0 tx=0).stop 2024-03-11T19:47:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.854+0000 7fe548fce740 1 -- 172.21.15.83:0/571401417 shutdown_connections 2024-03-11T19:47:49.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.854+0000 7fe548fce740 1 --2- 172.21.15.83:0/571401417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c20dd0 0x5585a8c255a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.854+0000 7fe548fce740 1 --2- 172.21.15.83:0/571401417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c1bc60 0x5585a8c2ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.854+0000 7fe548fce740 1 --2- 172.21.15.83:0/571401417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c384f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.854+0000 7fe548fce740 1 -- 172.21.15.83:0/571401417 >> 172.21.15.83:0/571401417 conn(0x5585a8c167e0 msgr2=0x5585a8c1c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.854+0000 7fe548fce740 1 -- 172.21.15.83:0/571401417 shutdown_connections 2024-03-11T19:47:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.855+0000 7fe548fce740 1 -- 172.21.15.83:0/571401417 wait complete. 2024-03-11T19:47:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.855+0000 7fe548fce740 1 -- start start 2024-03-11T19:47:49.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.855+0000 7fe548fce740 1 Processor -- start 2024-03-11T19:47:49.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe548fce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c2a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe548fce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c2aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe548fce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c20dd0 0x5585a8c2bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe548fce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8c2c430 con 0x5585a8c20dd0 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe548fce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8c2c5a0 con 0x5585a8c1bc60 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe548fce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585a8c2c6e0 con 0x5585a8c1b270 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe548fce740 10 client.0 _ll_register_callbacks cb 0x7fe530005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe53ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c2aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe53ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c2aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:36068/0 (socket says 172.21.15.83:36068) 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe53ffff640 1 -- 172.21.15.83:0/2756881299 learned_addr learned my addr 172.21.15.83:0/2756881299 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c20dd0 0x5585a8c2bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c2a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe53ffff640 1 -- 172.21.15.83:0/2756881299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 msgr2=0x5585a8c2a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe53ffff640 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c2a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe53ffff640 1 -- 172.21.15.83:0/2756881299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c20dd0 msgr2=0x5585a8c2bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe53ffff640 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c20dd0 0x5585a8c2bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe53ffff640 1 -- 172.21.15.83:0/2756881299 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe534057040 con 0x5585a8c1bc60 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe53ffff640 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c2aee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe5300039e0 tx=0x7fe530003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd08394459b270fc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.856+0000 7fe51f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2756881299 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530020040 con 0x5585a8c1bc60 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2756881299 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe530029070 con 0x5585a8c1bc60 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe548fce740 1 -- 172.21.15.83:0/2756881299 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe534004510 con 0x5585a8c1bc60 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2756881299 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5300040b0 con 0x5585a8c1bc60 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe548fce740 10 client.26423 fetch_fsmap learned FSMap version 32 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe548fce740 1 -- 172.21.15.83:0/2756881299 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5585a8b448c0 con 0x5585a8c1bc60 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2756881299 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe534004510 con 0x5585a8c1bc60 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2756881299 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe530037070 con 0x5585a8c1bc60 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe548fce740 10 client.26423 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:47:49.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe548fce740 10 client.26423 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:47:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe548fce740 20 client.26423 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe548fce740 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5585a8c99e90 0x5585a8c9c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe548fce740 4 client.26423 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:47:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe548fce740 1 -- 172.21.15.83:0/2756881299 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5585a890f890 con 0x5585a8c99e90 2024-03-11T19:47:49.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.857+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5585a8c99e90 0x5585a8c9c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.858+0000 7fe51f7fe640 10 client.26423 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.858+0000 7fe53f7fe640 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5585a8c99e90 0x5585a8c9c2b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba3e8bf9d7b8307f server_cookie=8b2ded0c3b9603c6 in_seq=0 out_seq=0 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.858+0000 7fe51f7fe640 1 -- 172.21.15.83:0/2756881299 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5585a890f890 con 0x5585a8c99e90 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.858+0000 7fe51f7fe640 10 client.26423 handle_command_reply: tid=0 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.858+0000 7fe548fce740 1 client.26423 shutdown 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.858+0000 7fe548fce740 1 -- 172.21.15.83:0/2756881299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 msgr2=0x5585a8c2aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.858+0000 7fe548fce740 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c2aee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe5300039e0 tx=0x7fe530003a10 comp rx=0 tx=0).stop 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.859+0000 7fe548fce740 1 -- 172.21.15.83:0/2756881299 shutdown_connections 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.859+0000 7fe548fce740 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5585a8c99e90 0x5585a8c9c2b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.859+0000 7fe548fce740 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585a8c20dd0 0x5585a8c2bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.859+0000 7fe548fce740 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5585a8c1bc60 0x5585a8c2aee0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.859+0000 7fe548fce740 1 --2- 172.21.15.83:0/2756881299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5585a8c1b270 0x5585a8c2a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.859+0000 7fe51f7fe640 0 client.26423 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:47:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.859+0000 7fe548fce740 1 -- 172.21.15.83:0/2756881299 >> 172.21.15.83:0/2756881299 conn(0x5585a8c167e0 msgr2=0x5585a8c384f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.859+0000 7fe548fce740 1 -- 172.21.15.83:0/2756881299 shutdown_connections 2024-03-11T19:47:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.860+0000 7fe548fce740 1 -- 172.21.15.83:0/2756881299 wait complete. 2024-03-11T19:47:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.860+0000 7fe548fce740 20 client.26423 trim_cache size 0 max 16384 2024-03-11T19:47:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:47:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.860+0000 7fe546327640 1 -- 172.21.15.83:0/98716236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe5280a1b80 msgr2=0x7fe5280a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.860+0000 7fe546327640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe5280a1b80 0x7fe5280a4040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe538004af0 tx=0x7fe538004a40 comp rx=0 tx=0).stop 2024-03-11T19:47:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.861+0000 7fe546327640 1 -- 172.21.15.83:0/98716236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54010d4a0 msgr2=0x7fe5400793e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.861+0000 7fe546327640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54010d4a0 0x7fe5400793e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe53800eb40 tx=0x7fe53800c2d0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.861+0000 7fe546327640 1 -- 172.21.15.83:0/98716236 shutdown_connections 2024-03-11T19:47:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.861+0000 7fe546327640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe5280a1b80 0x7fe5280a4040 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.861+0000 7fe546327640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54010d4a0 0x7fe5400793e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.861+0000 7fe546327640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54010aaf0 0x7fe5401a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.861+0000 7fe546327640 1 --2- 172.21.15.83:0/98716236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe540108180 0x7fe5401a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.861+0000 7fe546327640 1 -- 172.21.15.83:0/98716236 >> 172.21.15.83:0/98716236 conn(0x7fe54006dfe0 msgr2=0x7fe5401097d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:49.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.861+0000 7fe546327640 1 -- 172.21.15.83:0/98716236 shutdown_connections 2024-03-11T19:47:49.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:49.861+0000 7fe546327640 1 -- 172.21.15.83:0/98716236 wait complete. 2024-03-11T19:47:49.879 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:49 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3678669340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:47:49.880 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:47:49.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:47:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.028+0000 7f0b677fe640 1 -- 172.21.15.83:0/1901190266 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c02cbe0 con 0x7f0b680ff440 2024-03-11T19:47:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.029+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1901190266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b680ff440 msgr2=0x7f0b680ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.029+0000 7f0b6efe2640 1 --2- 172.21.15.83:0/1901190266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b680ff440 0x7f0b680ff860 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009900 tx=0x7f0b5c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:47:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.029+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1901190266 shutdown_connections 2024-03-11T19:47:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.029+0000 7f0b6efe2640 1 --2- 172.21.15.83:0/1901190266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b68100b60 0x7f0b68105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.029+0000 7f0b6efe2640 1 --2- 172.21.15.83:0/1901190266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b681001a0 0x7f0b68100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.029+0000 7f0b6efe2640 1 --2- 172.21.15.83:0/1901190266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b680ff440 0x7f0b680ff860 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.029+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1901190266 >> 172.21.15.83:0/1901190266 conn(0x7f0b680fad70 msgr2=0x7f0b680fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.029+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1901190266 shutdown_connections 2024-03-11T19:47:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.029+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1901190266 wait complete. 2024-03-11T19:47:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.030+0000 7f0b6efe2640 1 Processor -- start 2024-03-11T19:47:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.030+0000 7f0b6efe2640 1 -- start start 2024-03-11T19:47:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.030+0000 7f0b6efe2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b680ff440 0x7f0b6819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.030+0000 7f0b6efe2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b681001a0 0x7f0b6819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.030+0000 7f0b6efe2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b68100b60 0x7f0b681a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.030+0000 7f0b6efe2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b681a53a0 con 0x7f0b680ff440 2024-03-11T19:47:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.030+0000 7f0b6efe2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b681a5510 con 0x7f0b681001a0 2024-03-11T19:47:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.030+0000 7f0b6efe2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b681a56d0 con 0x7f0b68100b60 2024-03-11T19:47:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.030+0000 7f0b67fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b681001a0 0x7f0b6819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b67fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b681001a0 0x7f0b6819ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:36074/0 (socket says 172.21.15.83:36074) 2024-03-11T19:47:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b67fff640 1 -- 172.21.15.83:0/1926713775 learned_addr learned my addr 172.21.15.83:0/1926713775 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b68100b60 0x7f0b681a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b680ff440 0x7f0b6819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b67fff640 1 -- 172.21.15.83:0/1926713775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b68100b60 msgr2=0x7f0b681a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b67fff640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b68100b60 0x7f0b681a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b67fff640 1 -- 172.21.15.83:0/1926713775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b680ff440 msgr2=0x7f0b6819d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b67fff640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b680ff440 0x7f0b6819d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b67fff640 1 -- 172.21.15.83:0/1926713775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c009560 con 0x7f0b681001a0 2024-03-11T19:47:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b68100b60 0x7f0b681a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b67fff640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b681001a0 0x7f0b6819ddf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400d950 tx=0x7f0b5400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b65ffb640 1 -- 172.21.15.83:0/1926713775 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54009c30 con 0x7f0b681001a0 2024-03-11T19:47:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.031+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1926713775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b681a59b0 con 0x7f0b681001a0 2024-03-11T19:47:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.032+0000 7f0b65ffb640 1 -- 172.21.15.83:0/1926713775 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b54009d90 con 0x7f0b681001a0 2024-03-11T19:47:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.032+0000 7f0b65ffb640 1 -- 172.21.15.83:0/1926713775 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5400f720 con 0x7f0b681001a0 2024-03-11T19:47:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.032+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1926713775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b681a5f40 con 0x7f0b681001a0 2024-03-11T19:47:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.035+0000 7f0b71bbc740 1 Processor -- start 2024-03-11T19:47:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.035+0000 7f0b71bbc740 1 -- start start 2024-03-11T19:47:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.035+0000 7f0b65ffb640 1 -- 172.21.15.83:0/1926713775 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0b54010a60 con 0x7f0b681001a0 2024-03-11T19:47:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.035+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5021690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.035+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5021c60 0x55eea5026760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.035+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5026dd0 0x55eea5029260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.035+0000 7f0b71bbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea4fdd960 con 0x55eea5026dd0 2024-03-11T19:47:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.035+0000 7f0b71bbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea4f91310 con 0x55eea5021270 2024-03-11T19:47:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.035+0000 7f0b71bbc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea4f8e840 con 0x55eea5021c60 2024-03-11T19:47:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b67fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5026dd0 0x55eea5029260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6d57f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5021690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6d57f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5021690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:36084/0 (socket says 172.21.15.83:36084) 2024-03-11T19:47:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6d57f640 1 -- 172.21.15.83:0/811824590 learned_addr learned my addr 172.21.15.83:0/811824590 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b67fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5026dd0 0x55eea5029260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36298/0 (socket says 172.21.15.83:36298) 2024-03-11T19:47:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/811824590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5021c60 0x55eea5026760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6d57f640 1 -- 172.21.15.83:0/811824590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5021c60 msgr2=0x55eea5026760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/811824590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5021c60 0x55eea5026760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6d57f640 1 -- 172.21.15.83:0/811824590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5026dd0 msgr2=0x55eea5029260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/811824590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5026dd0 0x55eea5029260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6d57f640 1 -- 172.21.15.83:0/811824590 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eea4f4c060 con 0x55eea5021270 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/811824590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5021c60 0x55eea5026760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b65ffb640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b580a1ab0 0x7f0b580a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/811824590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5021690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000a870 tx=0x7f0b60009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1130f1f305c47c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b580a1ab0 0x7f0b580a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b65ffb640 1 -- 172.21.15.83:0/1926713775 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0b540cd070 con 0x7f0b681001a0 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b52ffd640 1 -- 172.21.15.83:0/811824590 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60010040 con 0x55eea5021270 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b52ffd640 1 -- 172.21.15.83:0/811824590 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b60014400 con 0x55eea5021270 2024-03-11T19:47:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.036+0000 7f0b52ffd640 1 -- 172.21.15.83:0/811824590 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b600135c0 con 0x55eea5021270 2024-03-11T19:47:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.037+0000 7f0b71bbc740 1 -- 172.21.15.83:0/811824590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 msgr2=0x55eea5021690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.037+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/811824590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5021690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000a870 tx=0x7f0b60009650 comp rx=0 tx=0).stop 2024-03-11T19:47:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.037+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b580a1ab0 0x7f0b580a3f70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c005940 tx=0x7f0b5c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.037+0000 7f0b71bbc740 1 -- 172.21.15.83:0/811824590 shutdown_connections 2024-03-11T19:47:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.037+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/811824590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5026dd0 0x55eea5029260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.037+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/811824590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5021c60 0x55eea5026760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.037+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/811824590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5021690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.037+0000 7f0b71bbc740 1 -- 172.21.15.83:0/811824590 >> 172.21.15.83:0/811824590 conn(0x55eea501c7e0 msgr2=0x55eea501cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.038+0000 7f0b71bbc740 1 -- 172.21.15.83:0/811824590 shutdown_connections 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.038+0000 7f0b71bbc740 1 -- 172.21.15.83:0/811824590 wait complete. 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.038+0000 7f0b71bbc740 1 -- start start 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.038+0000 7f0b71bbc740 1 Processor -- start 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea502b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 0x55eea502b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea50329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b71bbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea4f8a020 con 0x55eea5021c60 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b71bbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea4c5d430 con 0x55eea5021270 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b71bbc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea4c19e00 con 0x55eea5026dd0 2024-03-11T19:47:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b71bbc740 10 client.0 _ll_register_callbacks cb 0x55eea4f31cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b67fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 0x55eea502b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b67fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 0x55eea502b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36312/0 (socket says 172.21.15.83:36312) 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b67fff640 1 -- 172.21.15.83:0/150176893 learned_addr learned my addr 172.21.15.83:0/150176893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea50329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea502b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b6d57f640 1 -- 172.21.15.83:0/150176893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 msgr2=0x55eea502b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea502b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b6d57f640 1 -- 172.21.15.83:0/150176893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 msgr2=0x55eea502b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 0x55eea502b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b6d57f640 1 -- 172.21.15.83:0/150176893 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6000a4a0 con 0x55eea5026dd0 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.039+0000 7f0b67fff640 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 0x55eea502b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea50329e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0b60004a80 tx=0x7f0b60014be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=983be947a9b3ae83 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b47fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:47:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b47fff640 1 -- 172.21.15.83:0/150176893 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6000ed40 con 0x55eea5026dd0 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b47fff640 1 -- 172.21.15.83:0/150176893 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0b6001d420 con 0x55eea5026dd0 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b71bbc740 1 -- 172.21.15.83:0/150176893 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eea4c897e0 con 0x55eea5026dd0 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b47fff640 1 -- 172.21.15.83:0/150176893 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b6000eea0 con 0x55eea5026dd0 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b71bbc740 10 client.26542 fetch_fsmap learned FSMap version 32 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b71bbc740 1 -- 172.21.15.83:0/150176893 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55eea4f886a0 con 0x55eea5026dd0 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b47fff640 1 -- 172.21.15.83:0/150176893 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55eea4c897e0 con 0x55eea5026dd0 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b47fff640 1 -- 172.21.15.83:0/150176893 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0b60008030 con 0x55eea5026dd0 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b71bbc740 10 client.26542 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b71bbc740 10 client.26542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b71bbc740 20 client.26542 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55eea50374e0 0x55eea50398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b71bbc740 4 client.26542 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.040+0000 7f0b71bbc740 1 -- 172.21.15.83:0/150176893 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eea4cc6720 con 0x55eea50374e0 2024-03-11T19:47:50.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.041+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55eea50374e0 0x55eea50398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.042+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55eea50374e0 0x55eea50398d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d9de7eedd5f7d6 server_cookie=99803e61ff009753 in_seq=0 out_seq=0 2024-03-11T19:47:50.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.042+0000 7f0b47fff640 10 client.26542 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:50.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b47fff640 1 -- 172.21.15.83:0/150176893 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eea4cc6720 con 0x55eea50374e0 2024-03-11T19:47:50.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b47fff640 10 client.26542 handle_command_reply: tid=0 2024-03-11T19:47:50.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 1 client.26542 shutdown 2024-03-11T19:47:50.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 1 -- 172.21.15.83:0/150176893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 msgr2=0x55eea50329e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea50329e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0b60004a80 tx=0x7f0b60014be0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 1 -- 172.21.15.83:0/150176893 shutdown_connections 2024-03-11T19:47:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55eea50374e0 0x55eea50398d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea50329e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 0x55eea502b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/150176893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea502b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 1 -- 172.21.15.83:0/150176893 >> 172.21.15.83:0/150176893 conn(0x55eea501c7e0 msgr2=0x55eea501bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b47fff640 0 client.26542 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 1 -- 172.21.15.83:0/150176893 shutdown_connections 2024-03-11T19:47:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 1 -- 172.21.15.83:0/150176893 wait complete. 2024-03-11T19:47:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.043+0000 7f0b71bbc740 20 client.26542 trim_cache size 0 max 16384 2024-03-11T19:47:50.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b71bbc740 1 Processor -- start 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b71bbc740 1 -- start start 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021270 0x55eea502b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021c60 0x55eea50329a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea502b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b71bbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea4f0f130 con 0x55eea5021270 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b71bbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea500ea40 con 0x55eea5021c60 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b6cd7e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021270 0x55eea502b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b6cd7e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021270 0x55eea502b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36320/0 (socket says 172.21.15.83:36320) 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b6cd7e640 1 -- 172.21.15.83:0/1313933142 learned_addr learned my addr 172.21.15.83:0/1313933142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b71bbc740 1 -- 172.21.15.83:0/1313933142 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea501c650 con 0x55eea5026dd0 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b67fff640 1 --2- 172.21.15.83:0/1313933142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021c60 0x55eea50329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/1313933142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea502b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b67fff640 1 -- 172.21.15.83:0/1313933142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 msgr2=0x55eea502b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b67fff640 1 --2- 172.21.15.83:0/1313933142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea502b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b67fff640 1 -- 172.21.15.83:0/1313933142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021270 msgr2=0x55eea502b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b67fff640 1 --2- 172.21.15.83:0/1313933142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021270 0x55eea502b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b67fff640 1 -- 172.21.15.83:0/1313933142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eea4f24c20 con 0x55eea5021c60 2024-03-11T19:47:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/1313933142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021270 0x55eea502b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.060+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/1313933142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea502b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:47:50.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.061+0000 7f0b67fff640 1 --2- 172.21.15.83:0/1313933142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021c60 0x55eea50329a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0b54096a60 tx=0x7f0b54009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7911b87d10890f90 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.061+0000 7f0b527fc640 1 -- 172.21.15.83:0/1313933142 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54099e60 con 0x55eea5021c60 2024-03-11T19:47:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.061+0000 7f0b527fc640 1 -- 172.21.15.83:0/1313933142 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b54099bd0 con 0x55eea5021c60 2024-03-11T19:47:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.061+0000 7f0b527fc640 1 -- 172.21.15.83:0/1313933142 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b54096700 con 0x55eea5021c60 2024-03-11T19:47:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.062+0000 7f0b71bbc740 1 -- 172.21.15.83:0/1313933142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021c60 msgr2=0x55eea50329a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.062+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/1313933142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021c60 0x55eea50329a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0b54096a60 tx=0x7f0b54009f00 comp rx=0 tx=0).stop 2024-03-11T19:47:50.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.062+0000 7f0b71bbc740 1 -- 172.21.15.83:0/1313933142 shutdown_connections 2024-03-11T19:47:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.062+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/1313933142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea502b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.062+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/1313933142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021c60 0x55eea50329a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.062+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/1313933142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021270 0x55eea502b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.062+0000 7f0b71bbc740 1 -- 172.21.15.83:0/1313933142 >> 172.21.15.83:0/1313933142 conn(0x55eea501c7e0 msgr2=0x55eea5023120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.063+0000 7f0b71bbc740 1 -- 172.21.15.83:0/1313933142 shutdown_connections 2024-03-11T19:47:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.063+0000 7f0b71bbc740 1 -- 172.21.15.83:0/1313933142 wait complete. 2024-03-11T19:47:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.063+0000 7f0b71bbc740 1 -- start start 2024-03-11T19:47:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.063+0000 7f0b71bbc740 1 Processor -- start 2024-03-11T19:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5031120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 0x55eea5031660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b71bbc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea5031ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b71bbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea5032d70 con 0x55eea5021c60 2024-03-11T19:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b71bbc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea508d970 con 0x55eea5021270 2024-03-11T19:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b71bbc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea508db30 con 0x55eea5026dd0 2024-03-11T19:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b71bbc740 10 client.0 _ll_register_callbacks cb 0x7f0b60009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b6cd7e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5031120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b6cd7e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5031120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:36106/0 (socket says 172.21.15.83:36106) 2024-03-11T19:47:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b6cd7e640 1 -- 172.21.15.83:0/549219067 learned_addr learned my addr 172.21.15.83:0/549219067 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b6d57f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea5031ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b67fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 0x55eea5031660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b6cd7e640 1 -- 172.21.15.83:0/549219067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 msgr2=0x55eea5031ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea5031ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b6cd7e640 1 -- 172.21.15.83:0/549219067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 msgr2=0x55eea5031660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 0x55eea5031660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b6cd7e640 1 -- 172.21.15.83:0/549219067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c030070 con 0x55eea5021270 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.064+0000 7f0b6d57f640 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea5031ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b6cd7e640 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5031120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c036040 tx=0x7f0b5c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751506911a71d941 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b47fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b47fff640 1 -- 172.21.15.83:0/549219067 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c004a40 con 0x55eea5021270 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b47fff640 1 -- 172.21.15.83:0/549219067 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0b5c051040 con 0x55eea5021270 2024-03-11T19:47:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b71bbc740 1 -- 172.21.15.83:0/549219067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b60014400 con 0x55eea5021270 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b47fff640 1 -- 172.21.15.83:0/549219067 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b5c004040 con 0x55eea5021270 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b71bbc740 10 client.26441 fetch_fsmap learned FSMap version 32 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b71bbc740 1 -- 172.21.15.83:0/549219067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f0b681a5f40 con 0x55eea5021270 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b47fff640 1 -- 172.21.15.83:0/549219067 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b60014400 con 0x55eea5021270 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b47fff640 1 -- 172.21.15.83:0/549219067 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0b5c066020 con 0x55eea5021270 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b71bbc740 10 client.26441 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b71bbc740 10 client.26441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b71bbc740 20 client.26441 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55eea508f6b0 0x55eea5091ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b71bbc740 4 client.26441 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.065+0000 7f0b71bbc740 1 -- 172.21.15.83:0/549219067 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eea4d15890 con 0x55eea508f6b0 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.066+0000 7f0b67fff640 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55eea508f6b0 0x55eea5091ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.066+0000 7f0b67fff640 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55eea508f6b0 0x55eea5091ad0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55c4c9b07e18fd18 server_cookie=a62c097311934be2 in_seq=0 out_seq=0 2024-03-11T19:47:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.066+0000 7f0b47fff640 10 client.26441 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.066+0000 7f0b47fff640 1 -- 172.21.15.83:0/549219067 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eea4d15890 con 0x55eea508f6b0 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.066+0000 7f0b47fff640 10 client.26441 handle_command_reply: tid=0 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.066+0000 7f0b71bbc740 1 client.26441 shutdown 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b71bbc740 1 -- 172.21.15.83:0/549219067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 msgr2=0x55eea5031120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5031120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c036040 tx=0x7f0b5c009260 comp rx=0 tx=0).stop 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b71bbc740 1 -- 172.21.15.83:0/549219067 shutdown_connections 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55eea508f6b0 0x55eea5091ad0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eea5026dd0 0x55eea5031ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eea5021c60 0x55eea5031660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b71bbc740 1 --2- 172.21.15.83:0/549219067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eea5021270 0x55eea5031120 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b47fff640 0 client.26441 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:50.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b71bbc740 1 -- 172.21.15.83:0/549219067 >> 172.21.15.83:0/549219067 conn(0x55eea501c7e0 msgr2=0x55eea5023120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:50.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b71bbc740 1 -- 172.21.15.83:0/549219067 shutdown_connections 2024-03-11T19:47:50.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b71bbc740 1 -- 172.21.15.83:0/549219067 wait complete. 2024-03-11T19:47:50.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.067+0000 7f0b71bbc740 20 client.26441 trim_cache size 0 max 16384 2024-03-11T19:47:50.076 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:47:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1926713775 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b580a1ab0 msgr2=0x7f0b580a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b580a1ab0 0x7f0b580a3f70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c005940 tx=0x7f0b5c02c7e0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1926713775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b681001a0 msgr2=0x7f0b6819ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b681001a0 0x7f0b6819ddf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400d950 tx=0x7f0b5400de10 comp rx=0 tx=0).stop 2024-03-11T19:47:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1926713775 shutdown_connections 2024-03-11T19:47:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0b580a1ab0 0x7f0b580a3f70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b68100b60 0x7f0b681a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b681001a0 0x7f0b6819ddf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 --2- 172.21.15.83:0/1926713775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b680ff440 0x7f0b6819d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1926713775 >> 172.21.15.83:0/1926713775 conn(0x7f0b680fad70 msgr2=0x7f0b681013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1926713775 shutdown_connections 2024-03-11T19:47:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:50.068+0000 7f0b6efe2640 1 -- 172.21.15.83:0/1926713775 wait complete. 2024-03-11T19:47:50.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:49 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3678669340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:47:50.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:49 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3678669340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:47:51.089 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:47:51.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:47:51.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:47:51.118 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:50 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:47:51.118 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:50 smithi083 ceph-mon[31864]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.0 MiB/s wr, 623 op/s 2024-03-11T19:47:51.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:50 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:47:51.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:50 smithi137 ceph-mon[36220]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.0 MiB/s wr, 623 op/s 2024-03-11T19:47:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:50 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:47:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:50 smithi204 ceph-mon[36739]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.0 MiB/s wr, 623 op/s 2024-03-11T19:47:51.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.245+0000 7f24d789d640 1 -- 172.21.15.83:0/810590953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24d010bbc0 msgr2=0x7f24d010dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.245+0000 7f24d789d640 1 --2- 172.21.15.83:0/810590953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24d010bbc0 0x7f24d010dfb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009900 tx=0x7f24c002bc80 comp rx=0 tx=0).stop 2024-03-11T19:47:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.252+0000 7f24d789d640 1 -- 172.21.15.83:0/810590953 shutdown_connections 2024-03-11T19:47:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.252+0000 7f24d789d640 1 --2- 172.21.15.83:0/810590953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d0108180 0x7f24d0108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.252+0000 7f24d789d640 1 --2- 172.21.15.83:0/810590953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24d010bbc0 0x7f24d010dfb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.252+0000 7f24d789d640 1 --2- 172.21.15.83:0/810590953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d0109290 0x7f24d010b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.252+0000 7f24d789d640 1 -- 172.21.15.83:0/810590953 >> 172.21.15.83:0/810590953 conn(0x7f24d006e380 msgr2=0x7f24d00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.252+0000 7f24d789d640 1 -- 172.21.15.83:0/810590953 shutdown_connections 2024-03-11T19:47:51.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.252+0000 7f24d789d640 1 -- 172.21.15.83:0/810590953 wait complete. 2024-03-11T19:47:51.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.253+0000 7f24d789d640 1 Processor -- start 2024-03-11T19:47:51.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.257+0000 7f24d789d640 1 -- start start 2024-03-11T19:47:51.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.257+0000 7f24d789d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d0108180 0x7f24d019d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.257+0000 7f24d789d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d0109290 0x7f24d019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.257+0000 7f24d789d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24d010bbc0 0x7f24d01a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.257+0000 7f24d789d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01a53a0 con 0x7f24d0109290 2024-03-11T19:47:51.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.257+0000 7f24d789d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01a5510 con 0x7f24d0108180 2024-03-11T19:47:51.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.257+0000 7f24d789d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01b6700 con 0x7f24d010bbc0 2024-03-11T19:47:51.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.261+0000 7f24d4e38640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d0109290 0x7f24d019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.262+0000 7f24d4e38640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d0109290 0x7f24d019ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36350/0 (socket says 172.21.15.83:36350) 2024-03-11T19:47:51.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.262+0000 7f24d4e38640 1 -- 172.21.15.83:0/2949429492 learned_addr learned my addr 172.21.15.83:0/2949429492 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:51.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.262+0000 7f24d4e38640 1 -- 172.21.15.83:0/2949429492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24d010bbc0 msgr2=0x7f24d01a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.262+0000 7f24d4e38640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24d010bbc0 0x7f24d01a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.262+0000 7f24d4e38640 1 -- 172.21.15.83:0/2949429492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d0108180 msgr2=0x7f24d019d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.262+0000 7f24d4e38640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d0108180 0x7f24d019d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.262+0000 7f24d4e38640 1 -- 172.21.15.83:0/2949429492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c0009560 con 0x7f24d0109290 2024-03-11T19:47:51.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.263+0000 7f24d4e38640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d0109290 0x7f24d019ddf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f24c002c790 tx=0x7f24c00045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.263+0000 7f24c67fc640 1 -- 172.21.15.83:0/2949429492 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c00052c0 con 0x7f24d0109290 2024-03-11T19:47:51.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.263+0000 7f24d789d640 1 -- 172.21.15.83:0/2949429492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d01b6980 con 0x7f24d0109290 2024-03-11T19:47:51.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.263+0000 7f24d789d640 1 -- 172.21.15.83:0/2949429492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d01b6e30 con 0x7f24d0109290 2024-03-11T19:47:51.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.264+0000 7f24c67fc640 1 -- 172.21.15.83:0/2949429492 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24c0005420 con 0x7f24d0109290 2024-03-11T19:47:51.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.264+0000 7f24c67fc640 1 -- 172.21.15.83:0/2949429492 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c003e8f0 con 0x7f24d0109290 2024-03-11T19:47:51.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.267+0000 7f24c67fc640 1 -- 172.21.15.83:0/2949429492 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f24c003ea50 con 0x7f24d0109290 2024-03-11T19:47:51.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.269+0000 7f24da475740 1 Processor -- start 2024-03-11T19:47:51.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.269+0000 7f24da475740 1 -- start start 2024-03-11T19:47:51.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.269+0000 7f24c67fc640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24bc0a1bc0 0x7f24bc0a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.269+0000 7f24da475740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a768e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.270+0000 7f24d5639640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24bc0a1bc0 0x7f24bc0a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.270+0000 7f24d5e3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.270+0000 7f24d5e3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a768e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:36142/0 (socket says 172.21.15.83:36142) 2024-03-11T19:47:51.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.270+0000 7f24d5e3a640 1 -- 172.21.15.83:0/3237076828 learned_addr learned my addr 172.21.15.83:0/3237076828 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:51.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.270+0000 7f24da475740 1 --2- 172.21.15.83:0/3237076828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a77b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.270+0000 7f24da475740 1 --2- 172.21.15.83:0/3237076828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239a7e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.270+0000 7f24d4e38640 1 --2- 172.21.15.83:0/3237076828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a77b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.271+0000 7f24d5639640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24bc0a1bc0 0x7f24bc0a4080 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f24d006a2b0 tx=0x7f24c8009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.271+0000 7f24da475740 1 -- 172.21.15.83:0/3237076828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2399e5f90 con 0x55c239a78050 2024-03-11T19:47:51.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.271+0000 7f24da475740 1 -- 172.21.15.83:0/3237076828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2399e34c0 con 0x55c239a764c0 2024-03-11T19:47:51.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.271+0000 7f24da475740 1 -- 172.21.15.83:0/3237076828 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2399deca0 con 0x55c239a77690 2024-03-11T19:47:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.271+0000 7f24d5e3a640 1 --2- 172.21.15.83:0/3237076828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239a7e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.271+0000 7f24d5e3a640 1 -- 172.21.15.83:0/3237076828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 msgr2=0x55c239a77b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.271+0000 7f24d5e3a640 1 --2- 172.21.15.83:0/3237076828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a77b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.271+0000 7f24d5e3a640 1 -- 172.21.15.83:0/3237076828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 msgr2=0x55c239a7e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.271+0000 7f24d5e3a640 1 --2- 172.21.15.83:0/3237076828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239a7e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.271+0000 7f24d5e3a640 1 -- 172.21.15.83:0/3237076828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2399a10b0 con 0x55c239a764c0 2024-03-11T19:47:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.271+0000 7f24c67fc640 1 -- 172.21.15.83:0/2949429492 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f24c00efbd0 con 0x7f24d0109290 2024-03-11T19:47:51.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.272+0000 7f24d5e3a640 1 --2- 172.21.15.83:0/3237076828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a768e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0147e0 tx=0x7f24cc00b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29181ec738b967f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.274+0000 7f24bb7fe640 1 -- 172.21.15.83:0/3237076828 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc014e20 con 0x55c239a764c0 2024-03-11T19:47:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.274+0000 7f24bb7fe640 1 -- 172.21.15.83:0/3237076828 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24cc005500 con 0x55c239a764c0 2024-03-11T19:47:51.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.274+0000 7f24bb7fe640 1 -- 172.21.15.83:0/3237076828 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc004da0 con 0x55c239a764c0 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.275+0000 7f24da475740 1 -- 172.21.15.83:0/3237076828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 msgr2=0x55c239a768e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.275+0000 7f24da475740 1 --2- 172.21.15.83:0/3237076828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a768e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0147e0 tx=0x7f24cc00b480 comp rx=0 tx=0).stop 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.275+0000 7f24da475740 1 -- 172.21.15.83:0/3237076828 shutdown_connections 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.275+0000 7f24da475740 1 --2- 172.21.15.83:0/3237076828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239a7e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.275+0000 7f24da475740 1 --2- 172.21.15.83:0/3237076828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a77b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.275+0000 7f24da475740 1 --2- 172.21.15.83:0/3237076828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a768e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.276+0000 7f24da475740 1 -- 172.21.15.83:0/3237076828 >> 172.21.15.83:0/3237076828 conn(0x55c239a716a0 msgr2=0x55c239a71aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.276+0000 7f24da475740 1 -- 172.21.15.83:0/3237076828 shutdown_connections 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.277+0000 7f24da475740 1 -- 172.21.15.83:0/3237076828 wait complete. 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.277+0000 7f24da475740 1 -- start start 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.278+0000 7f24da475740 1 Processor -- start 2024-03-11T19:47:51.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.278+0000 7f24da475740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.278+0000 7f24da475740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a77690 0x55c239a80b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.279+0000 7f24d5639640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.279+0000 7f24d5639640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:36156/0 (socket says 172.21.15.83:36156) 2024-03-11T19:47:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.279+0000 7f24d5639640 1 -- 172.21.15.83:0/2174416641 learned_addr learned my addr 172.21.15.83:0/2174416641 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.279+0000 7f24da475740 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a78050 0x55c239a87b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.279+0000 7f24da475740 1 -- 172.21.15.83:0/2174416641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239983f40 con 0x55c239a77690 2024-03-11T19:47:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.279+0000 7f24da475740 1 -- 172.21.15.83:0/2174416641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2396b24b0 con 0x55c239a764c0 2024-03-11T19:47:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.279+0000 7f24da475740 1 -- 172.21.15.83:0/2174416641 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239964670 con 0x55c239a78050 2024-03-11T19:47:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.279+0000 7f24d5e3a640 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a78050 0x55c239a87b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.279+0000 7f24da475740 10 client.0 _ll_register_callbacks cb 0x55c2399d1f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:51.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.284+0000 7f24d5639640 1 -- 172.21.15.83:0/2174416641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a78050 msgr2=0x55c239a87b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.284+0000 7f24d5639640 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a78050 0x55c239a87b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.284+0000 7f24d5639640 1 -- 172.21.15.83:0/2174416641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a77690 msgr2=0x55c239a80b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.284+0000 7f24d5639640 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a77690 0x55c239a80b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.284+0000 7f24d5639640 1 -- 172.21.15.83:0/2174416641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24cc014490 con 0x55c239a764c0 2024-03-11T19:47:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.286+0000 7f24d5639640 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80640 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009fc0 tx=0x7f24c8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5df5b6da659a11a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.287+0000 7f24b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:47:51.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.289+0000 7f24b8ff9640 1 -- 172.21.15.83:0/2174416641 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8017e90 con 0x55c239a764c0 2024-03-11T19:47:51.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.289+0000 7f24da475740 1 -- 172.21.15.83:0/2174416641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2399b8f40 con 0x55c239a764c0 2024-03-11T19:47:51.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.289+0000 7f24b8ff9640 1 -- 172.21.15.83:0/2174416641 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f24c801d070 con 0x55c239a764c0 2024-03-11T19:47:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.294+0000 7f24b8ff9640 1 -- 172.21.15.83:0/2174416641 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24c8022480 con 0x55c239a764c0 2024-03-11T19:47:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.294+0000 7f24da475740 10 client.26447 fetch_fsmap learned FSMap version 32 2024-03-11T19:47:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.294+0000 7f24da475740 1 -- 172.21.15.83:0/2174416641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c2399dd320 con 0x55c239a764c0 2024-03-11T19:47:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.295+0000 7f24b8ff9640 1 -- 172.21.15.83:0/2174416641 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c2399b8f40 con 0x55c239a764c0 2024-03-11T19:47:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.296+0000 7f24b8ff9640 1 -- 172.21.15.83:0/2174416641 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f24c802c990 con 0x55c239a764c0 2024-03-11T19:47:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.296+0000 7f24da475740 10 client.26447 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:47:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.296+0000 7f24da475740 10 client.26447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.296+0000 7f24da475740 20 client.26447 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.296+0000 7f24da475740 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c239a8c880 0x55c239a8eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.296+0000 7f24da475740 4 client.26447 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.296+0000 7f24da475740 1 -- 172.21.15.83:0/2174416641 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2399d5310 con 0x55c239a8c880 2024-03-11T19:47:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.301+0000 7f24d4e38640 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c239a8c880 0x55c239a8eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.302+0000 7f24d4e38640 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c239a8c880 0x55c239a8eca0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e42c2aba8ec137a server_cookie=53507450c0c86e79 in_seq=0 out_seq=0 2024-03-11T19:47:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.302+0000 7f24b8ff9640 10 client.26447 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.303+0000 7f24b8ff9640 1 -- 172.21.15.83:0/2174416641 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2399d5310 con 0x55c239a8c880 2024-03-11T19:47:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.303+0000 7f24b8ff9640 10 client.26447 handle_command_reply: tid=0 2024-03-11T19:47:51.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.303+0000 7f24da475740 1 client.26447 shutdown 2024-03-11T19:47:51.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.304+0000 7f24da475740 1 -- 172.21.15.83:0/2174416641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 msgr2=0x55c239a80640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.304+0000 7f24da475740 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80640 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009fc0 tx=0x7f24c8004b40 comp rx=0 tx=0).stop 2024-03-11T19:47:51.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.304+0000 7f24da475740 1 -- 172.21.15.83:0/2174416641 shutdown_connections 2024-03-11T19:47:51.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.304+0000 7f24da475740 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c239a8c880 0x55c239a8eca0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.304+0000 7f24da475740 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a78050 0x55c239a87b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.304+0000 7f24da475740 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a77690 0x55c239a80b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.304+0000 7f24da475740 1 --2- 172.21.15.83:0/2174416641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80640 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.304+0000 7f24da475740 1 -- 172.21.15.83:0/2174416641 >> 172.21.15.83:0/2174416641 conn(0x55c239a716a0 msgr2=0x55c239a708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.304+0000 7f24b8ff9640 0 client.26447 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:51.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.304+0000 7f24da475740 1 -- 172.21.15.83:0/2174416641 shutdown_connections 2024-03-11T19:47:51.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.304+0000 7f24da475740 1 -- 172.21.15.83:0/2174416641 wait complete. 2024-03-11T19:47:51.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.305+0000 7f24da475740 20 client.26447 trim_cache size 0 max 16384 2024-03-11T19:47:51.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24da475740 1 Processor -- start 2024-03-11T19:47:51.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24da475740 1 -- start start 2024-03-11T19:47:51.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24da475740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24da475740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a87b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24da475740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239a70050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24da475740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239a66350 con 0x55c239a78050 2024-03-11T19:47:51.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24da475740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239a93c70 con 0x55c239a764c0 2024-03-11T19:47:51.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24da475740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2399d5310 con 0x55c239a77690 2024-03-11T19:47:51.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24d5639640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24d5639640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:36160/0 (socket says 172.21.15.83:36160) 2024-03-11T19:47:51.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24d5639640 1 -- 172.21.15.83:0/590487280 learned_addr learned my addr 172.21.15.83:0/590487280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:51.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24d4e38640 1 --2- 172.21.15.83:0/590487280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a87b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.336+0000 7f24d5e3a640 1 --2- 172.21.15.83:0/590487280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239a70050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.337+0000 7f24d5639640 1 -- 172.21.15.83:0/590487280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 msgr2=0x55c239a87b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.337+0000 7f24d5639640 1 --2- 172.21.15.83:0/590487280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a87b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.337+0000 7f24d5639640 1 -- 172.21.15.83:0/590487280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 msgr2=0x55c239a70050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.337+0000 7f24d5639640 1 --2- 172.21.15.83:0/590487280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239a70050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.337+0000 7f24d5639640 1 -- 172.21.15.83:0/590487280 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c239979c00 con 0x55c239a764c0 2024-03-11T19:47:51.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.337+0000 7f24d4e38640 1 --2- 172.21.15.83:0/590487280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a87b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:51.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.337+0000 7f24d5639640 1 --2- 172.21.15.83:0/590487280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f24c802cd70 tx=0x7f24c800aa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a16c695d2343bea7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.337+0000 7f24baffd640 1 -- 172.21.15.83:0/590487280 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8031050 con 0x55c239a764c0 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.337+0000 7f24baffd640 1 -- 172.21.15.83:0/590487280 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24c8017e90 con 0x55c239a764c0 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.337+0000 7f24baffd640 1 -- 172.21.15.83:0/590487280 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c802c420 con 0x55c239a764c0 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.338+0000 7f24da475740 1 -- 172.21.15.83:0/590487280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 msgr2=0x55c239a80960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.338+0000 7f24da475740 1 --2- 172.21.15.83:0/590487280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f24c802cd70 tx=0x7f24c800aa30 comp rx=0 tx=0).stop 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.338+0000 7f24da475740 1 -- 172.21.15.83:0/590487280 shutdown_connections 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.338+0000 7f24da475740 1 --2- 172.21.15.83:0/590487280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239a70050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.338+0000 7f24da475740 1 --2- 172.21.15.83:0/590487280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a87b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.338+0000 7f24da475740 1 --2- 172.21.15.83:0/590487280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a80960 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.338+0000 7f24da475740 1 -- 172.21.15.83:0/590487280 >> 172.21.15.83:0/590487280 conn(0x55c239a716a0 msgr2=0x55c239a80250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.338+0000 7f24da475740 1 -- 172.21.15.83:0/590487280 shutdown_connections 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.338+0000 7f24da475740 1 -- 172.21.15.83:0/590487280 wait complete. 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24da475740 1 -- start start 2024-03-11T19:47:51.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24da475740 1 Processor -- start 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24da475740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a87110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24da475740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a87650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24da475740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239ae3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24da475740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239ae3890 con 0x55c239a78050 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24da475740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239ae39d0 con 0x55c239a764c0 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24da475740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239ae3b60 con 0x55c239a77690 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24da475740 10 client.0 _ll_register_callbacks cb 0x7f24cc00b360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24d5639640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a87110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24d5639640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a87110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:36170/0 (socket says 172.21.15.83:36170) 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24d5639640 1 -- 172.21.15.83:0/359127018 learned_addr learned my addr 172.21.15.83:0/359127018 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24d5e3a640 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239ae3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.339+0000 7f24d4e38640 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a87650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24d5639640 1 -- 172.21.15.83:0/359127018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 msgr2=0x55c239a87650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24d5639640 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a87650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24d5639640 1 -- 172.21.15.83:0/359127018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 msgr2=0x55c239ae3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24d5639640 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239ae3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24d5639640 1 -- 172.21.15.83:0/359127018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d01b6e30 con 0x55c239a764c0 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24d5e3a640 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239ae3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24d5639640 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a87110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f24c8019ae0 tx=0x7f24c8009fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c09fa694606738da server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24b8ff9640 1 -- 172.21.15.83:0/359127018 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c802c420 con 0x55c239a764c0 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24b8ff9640 1 -- 172.21.15.83:0/359127018 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f24c801d040 con 0x55c239a764c0 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24da475740 1 -- 172.21.15.83:0/359127018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c239a04900 con 0x55c239a764c0 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24b8ff9640 1 -- 172.21.15.83:0/359127018 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24c802c580 con 0x55c239a764c0 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24da475740 10 client.26453 fetch_fsmap learned FSMap version 32 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24da475740 1 -- 172.21.15.83:0/359127018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c2399dd320 con 0x55c239a764c0 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.340+0000 7f24b8ff9640 1 -- 172.21.15.83:0/359127018 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c239a04900 con 0x55c239a764c0 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.341+0000 7f24b8ff9640 1 -- 172.21.15.83:0/359127018 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f24c8004630 con 0x55c239a764c0 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.341+0000 7f24da475740 10 client.26453 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.341+0000 7f24da475740 10 client.26453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:51.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.341+0000 7f24da475740 20 client.26453 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.341+0000 7f24da475740 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c239ae52d0 0x55c239ae76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.341+0000 7f24da475740 4 client.26453 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:47:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.341+0000 7f24da475740 1 -- 172.21.15.83:0/359127018 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c2396de860 con 0x55c239ae52d0 2024-03-11T19:47:51.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.341+0000 7f24d4e38640 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c239ae52d0 0x55c239ae76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.343+0000 7f24b8ff9640 10 client.26453 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.343+0000 7f24d4e38640 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c239ae52d0 0x55c239ae76f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a182546f60e3e7f1 server_cookie=b3a3431bf88b7f8c in_seq=0 out_seq=0 2024-03-11T19:47:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.344+0000 7f24b8ff9640 1 -- 172.21.15.83:0/359127018 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c2396de860 con 0x55c239ae52d0 2024-03-11T19:47:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.344+0000 7f24b8ff9640 10 client.26453 handle_command_reply: tid=0 2024-03-11T19:47:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.344+0000 7f24da475740 1 client.26453 shutdown 2024-03-11T19:47:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.344+0000 7f24da475740 1 -- 172.21.15.83:0/359127018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 msgr2=0x55c239a87110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.344+0000 7f24da475740 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a87110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f24c8019ae0 tx=0x7f24c8009fa0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.345+0000 7f24da475740 1 -- 172.21.15.83:0/359127018 shutdown_connections 2024-03-11T19:47:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.345+0000 7f24da475740 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c239ae52d0 0x55c239ae76f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.345+0000 7f24b8ff9640 0 client.26453 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.345+0000 7f24da475740 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c239a78050 0x55c239ae3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.345+0000 7f24da475740 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239a77690 0x55c239a87650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.345+0000 7f24da475740 1 --2- 172.21.15.83:0/359127018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c239a764c0 0x55c239a87110 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.345+0000 7f24da475740 1 -- 172.21.15.83:0/359127018 >> 172.21.15.83:0/359127018 conn(0x55c239a716a0 msgr2=0x55c239a74900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.345+0000 7f24da475740 1 -- 172.21.15.83:0/359127018 shutdown_connections 2024-03-11T19:47:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.345+0000 7f24da475740 1 -- 172.21.15.83:0/359127018 wait complete. 2024-03-11T19:47:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.345+0000 7f24da475740 20 client.26453 trim_cache size 0 max 16384 2024-03-11T19:47:51.351 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:47:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:47:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "f53ff92e-75ec-47da-9396-efbdc2a288fd", 2024-03-11T19:47:51.352 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:47:51.352 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:47:51.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.346+0000 7f24d789d640 1 -- 172.21.15.83:0/2949429492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24bc0a1bc0 msgr2=0x7f24bc0a4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.346+0000 7f24d789d640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24bc0a1bc0 0x7f24bc0a4080 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f24d006a2b0 tx=0x7f24c8009250 comp rx=0 tx=0).stop 2024-03-11T19:47:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.346+0000 7f24d789d640 1 -- 172.21.15.83:0/2949429492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d0109290 msgr2=0x7f24d019ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.346+0000 7f24d789d640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d0109290 0x7f24d019ddf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f24c002c790 tx=0x7f24c00045c0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.346+0000 7f24d789d640 1 -- 172.21.15.83:0/2949429492 shutdown_connections 2024-03-11T19:47:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.346+0000 7f24d789d640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24bc0a1bc0 0x7f24bc0a4080 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.346+0000 7f24d789d640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24d010bbc0 0x7f24d01a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.346+0000 7f24d789d640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24d0109290 0x7f24d019ddf0 secure :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f24c002c790 tx=0x7f24c00045c0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.346+0000 7f24d789d640 1 --2- 172.21.15.83:0/2949429492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d0108180 0x7f24d019d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.346+0000 7f24d789d640 1 -- 172.21.15.83:0/2949429492 >> 172.21.15.83:0/2949429492 conn(0x7f24d006e380 msgr2=0x7f24d010ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.347+0000 7f24d789d640 1 -- 172.21.15.83:0/2949429492 shutdown_connections 2024-03-11T19:47:51.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.347+0000 7f24d789d640 1 -- 172.21.15.83:0/2949429492 wait complete. 2024-03-11T19:47:51.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:47:51.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:47:51.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.552+0000 7f247624e640 1 -- 172.21.15.83:0/2423869433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2470102d60 msgr2=0x7f24701095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.552+0000 7f247624e640 1 --2- 172.21.15.83:0/2423869433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2470102d60 0x7f24701095f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2468009ca0 tx=0x7f246802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.552+0000 7f247624e640 1 -- 172.21.15.83:0/2423869433 shutdown_connections 2024-03-11T19:47:51.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.552+0000 7f247624e640 1 --2- 172.21.15.83:0/2423869433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2470102d60 0x7f24701095f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.552+0000 7f247624e640 1 --2- 172.21.15.83:0/2423869433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24701023a0 0x7f2470102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.552+0000 7f247624e640 1 --2- 172.21.15.83:0/2423869433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24701011b0 0x7f24701015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.552+0000 7f247624e640 1 -- 172.21.15.83:0/2423869433 >> 172.21.15.83:0/2423869433 conn(0x7f24700fc740 msgr2=0x7f24700feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.552+0000 7f247624e640 1 -- 172.21.15.83:0/2423869433 shutdown_connections 2024-03-11T19:47:51.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.552+0000 7f247624e640 1 -- 172.21.15.83:0/2423869433 wait complete. 2024-03-11T19:47:51.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f247624e640 1 Processor -- start 2024-03-11T19:47:51.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f247624e640 1 -- start start 2024-03-11T19:47:51.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f247624e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24701011b0 0x7f247019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f247624e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701023a0 0x7f247019dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f247624e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2470102d60 0x7f24701a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f246effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701023a0 0x7f247019dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f247624e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24701a52c0 con 0x7f24701023a0 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f247624e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24701a5430 con 0x7f2470102d60 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f246effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701023a0 0x7f247019dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36390/0 (socket says 172.21.15.83:36390) 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f246effd640 1 -- 172.21.15.83:0/3369399619 learned_addr learned my addr 172.21.15.83:0/3369399619 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f247624e640 1 -- 172.21.15.83:0/3369399619 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24701a55f0 con 0x7f24701011b0 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f246f7fe640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24701011b0 0x7f247019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f246ffff640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2470102d60 0x7f24701a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f246effd640 1 -- 172.21.15.83:0/3369399619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24701011b0 msgr2=0x7f247019d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f246effd640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24701011b0 0x7f247019d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f246effd640 1 -- 172.21.15.83:0/3369399619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2470102d60 msgr2=0x7f24701a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f246effd640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2470102d60 0x7f24701a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.553+0000 7f246effd640 1 -- 172.21.15.83:0/3369399619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2468009900 con 0x7f24701023a0 2024-03-11T19:47:51.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.554+0000 7f246effd640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701023a0 0x7f247019dd10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f246400e9f0 tx=0x7f246400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.554+0000 7f246cff9640 1 -- 172.21.15.83:0/3369399619 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464009700 con 0x7f24701023a0 2024-03-11T19:47:51.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.554+0000 7f247624e640 1 -- 172.21.15.83:0/3369399619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24701a58d0 con 0x7f24701023a0 2024-03-11T19:47:51.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.554+0000 7f247624e640 1 -- 172.21.15.83:0/3369399619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24701a5d70 con 0x7f24701023a0 2024-03-11T19:47:51.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.554+0000 7f246cff9640 1 -- 172.21.15.83:0/3369399619 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f246400cd00 con 0x7f24701023a0 2024-03-11T19:47:51.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.554+0000 7f246cff9640 1 -- 172.21.15.83:0/3369399619 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464018800 con 0x7f24701023a0 2024-03-11T19:47:51.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.557+0000 7f246cff9640 1 -- 172.21.15.83:0/3369399619 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2464010430 con 0x7f24701023a0 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.557+0000 7f2478e1d740 1 Processor -- start 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.557+0000 7f2478e1d740 1 -- start start 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.557+0000 7f246cff9640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24580a1b00 0x7f24580a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.557+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.557+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c8dcc0 0x555ff4c927c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.557+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c92e30 0x555ff4c952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.557+0000 7f2478e1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff4bfd280 con 0x555ff4c8d2d0 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.557+0000 7f2478e1d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff4bfa7b0 con 0x555ff4c8dcc0 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.557+0000 7f2478e1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff4bf5f90 con 0x555ff4c92e30 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246cff9640 1 -- 172.21.15.83:0/3369399619 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2464014070 con 0x7f24701023a0 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246ffff640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24580a1b00 0x7f24580a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c8dcc0 0x555ff4c927c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36396/0 (socket says 172.21.15.83:36396) 2024-03-11T19:47:51.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246f7fe640 1 -- 172.21.15.83:0/3825180503 learned_addr learned my addr 172.21.15.83:0/3825180503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:51.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246ffff640 1 --2- 172.21.15.83:0/3825180503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c92e30 0x555ff4c952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246effd640 1 -- 172.21.15.83:0/3825180503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c92e30 msgr2=0x555ff4c952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246effd640 1 --2- 172.21.15.83:0/3825180503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c92e30 0x555ff4c952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246effd640 1 -- 172.21.15.83:0/3825180503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 msgr2=0x555ff4c8d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246effd640 1 --2- 172.21.15.83:0/3825180503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246effd640 1 -- 172.21.15.83:0/3825180503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ff4bb4570 con 0x555ff4c8dcc0 2024-03-11T19:47:51.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246f7fe640 1 --2- 172.21.15.83:0/3825180503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:47:51.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.558+0000 7f246effd640 1 --2- 172.21.15.83:0/3825180503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c8dcc0 0x555ff4c927c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24640cb0a0 tx=0x7f2464049930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e03abefae269fb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.559+0000 7f246ffff640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24580a1b00 0x7f24580a3fc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2468032710 tx=0x7f2468034520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.559+0000 7f2455ffb640 1 -- 172.21.15.83:0/3825180503 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24640ba110 con 0x555ff4c8dcc0 2024-03-11T19:47:51.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.559+0000 7f2455ffb640 1 -- 172.21.15.83:0/3825180503 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f246409e610 con 0x555ff4c8dcc0 2024-03-11T19:47:51.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.559+0000 7f2455ffb640 1 -- 172.21.15.83:0/3825180503 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24640851d0 con 0x555ff4c8dcc0 2024-03-11T19:47:51.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.560+0000 7f2478e1d740 1 -- 172.21.15.83:0/3825180503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c8dcc0 msgr2=0x555ff4c927c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.560+0000 7f2478e1d740 1 --2- 172.21.15.83:0/3825180503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c8dcc0 0x555ff4c927c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24640cb0a0 tx=0x7f2464049930 comp rx=0 tx=0).stop 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.560+0000 7f2478e1d740 1 -- 172.21.15.83:0/3825180503 shutdown_connections 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.560+0000 7f2478e1d740 1 --2- 172.21.15.83:0/3825180503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c92e30 0x555ff4c952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.560+0000 7f2478e1d740 1 --2- 172.21.15.83:0/3825180503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c8dcc0 0x555ff4c927c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.560+0000 7f2478e1d740 1 --2- 172.21.15.83:0/3825180503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.560+0000 7f2478e1d740 1 -- 172.21.15.83:0/3825180503 >> 172.21.15.83:0/3825180503 conn(0x555ff4c887f0 msgr2=0x555ff4c88bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.560+0000 7f2478e1d740 1 -- 172.21.15.83:0/3825180503 shutdown_connections 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.560+0000 7f2478e1d740 1 -- 172.21.15.83:0/3825180503 wait complete. 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.561+0000 7f2478e1d740 1 -- start start 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.561+0000 7f2478e1d740 1 Processor -- start 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.561+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c97340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.561+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c97880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.561+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4c9e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.561+0000 7f2478e1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff48c9430 con 0x555ff4c8d2d0 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.561+0000 7f2478e1d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff4885e00 con 0x555ff4c92e30 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.561+0000 7f2478e1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff4b7afd0 con 0x555ff4c8dcc0 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.561+0000 7f2478e1d740 10 client.0 _ll_register_callbacks cb 0x555ff4bd0fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c97880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c97340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c97880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41246/0 (socket says 172.21.15.83:41246) 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246effd640 1 -- 172.21.15.83:0/288271794 learned_addr learned my addr 172.21.15.83:0/288271794 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246ffff640 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4c9e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246effd640 1 -- 172.21.15.83:0/288271794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 msgr2=0x555ff4c9e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246effd640 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4c9e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246effd640 1 -- 172.21.15.83:0/288271794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 msgr2=0x555ff4c97340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246effd640 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c97340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246effd640 1 -- 172.21.15.83:0/288271794 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24701a5d70 con 0x555ff4c8dcc0 2024-03-11T19:47:51.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246ffff640 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4c9e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:47:51.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f246effd640 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c97880 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2464018490 tx=0x7f246409a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2463d5a79b5cfb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f244effd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:47:51.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f244effd640 1 -- 172.21.15.83:0/288271794 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2464050e80 con 0x555ff4c8dcc0 2024-03-11T19:47:51.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f244effd640 1 -- 172.21.15.83:0/288271794 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f24640a2c60 con 0x555ff4c8dcc0 2024-03-11T19:47:51.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.562+0000 7f2478e1d740 1 -- 172.21.15.83:0/288271794 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ff48f57e0 con 0x555ff4c8dcc0 2024-03-11T19:47:51.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.563+0000 7f244effd640 1 -- 172.21.15.83:0/288271794 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24640cd320 con 0x555ff4c8dcc0 2024-03-11T19:47:51.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.563+0000 7f2478e1d740 10 client.26575 fetch_fsmap learned FSMap version 32 2024-03-11T19:47:51.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.563+0000 7f2478e1d740 1 -- 172.21.15.83:0/288271794 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555ff4bf4610 con 0x555ff4c8dcc0 2024-03-11T19:47:51.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.563+0000 7f244effd640 1 -- 172.21.15.83:0/288271794 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555ff48f57e0 con 0x555ff4c8dcc0 2024-03-11T19:47:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.564+0000 7f244effd640 1 -- 172.21.15.83:0/288271794 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f246409e210 con 0x555ff4c8dcc0 2024-03-11T19:47:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.564+0000 7f2478e1d740 10 client.26575 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:47:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.564+0000 7f2478e1d740 10 client.26575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.564+0000 7f2478e1d740 20 client.26575 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.564+0000 7f2478e1d740 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555ff4ca3150 0x555ff4ca5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.564+0000 7f2478e1d740 4 client.26575 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:47:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.564+0000 7f2478e1d740 1 -- 172.21.15.83:0/288271794 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ff4932720 con 0x555ff4ca3150 2024-03-11T19:47:51.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.564+0000 7f246f7fe640 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555ff4ca3150 0x555ff4ca5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.566+0000 7f246f7fe640 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555ff4ca3150 0x555ff4ca5570 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c244c2755a1bdd server_cookie=7a199e73677ead60 in_seq=0 out_seq=0 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.566+0000 7f244effd640 10 client.26575 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.566+0000 7f244effd640 1 -- 172.21.15.83:0/288271794 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ff4932720 con 0x555ff4ca3150 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.566+0000 7f244effd640 10 client.26575 handle_command_reply: tid=0 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 1 client.26575 shutdown 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 1 -- 172.21.15.83:0/288271794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 msgr2=0x555ff4c97880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c97880 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2464018490 tx=0x7f246409a6f0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 1 -- 172.21.15.83:0/288271794 shutdown_connections 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555ff4ca3150 0x555ff4ca5570 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4c9e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c97880 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 1 --2- 172.21.15.83:0/288271794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c97340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f244effd640 0 client.26575 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:51.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 1 -- 172.21.15.83:0/288271794 >> 172.21.15.83:0/288271794 conn(0x555ff4c887f0 msgr2=0x555ff4c87a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 1 -- 172.21.15.83:0/288271794 shutdown_connections 2024-03-11T19:47:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 1 -- 172.21.15.83:0/288271794 wait complete. 2024-03-11T19:47:51.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.567+0000 7f2478e1d740 20 client.26575 trim_cache size 0 max 16384 2024-03-11T19:47:51.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.589+0000 7f2478e1d740 1 Processor -- start 2024-03-11T19:47:51.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.589+0000 7f2478e1d740 1 -- start start 2024-03-11T19:47:51.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c9e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4c8f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f2478e1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff4c95980 con 0x555ff4c8d2d0 2024-03-11T19:47:51.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f2478e1d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff4932720 con 0x555ff4c92e30 2024-03-11T19:47:51.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f2478e1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff48f57e0 con 0x555ff4c8dcc0 2024-03-11T19:47:51.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f246f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f246f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36418/0 (socket says 172.21.15.83:36418) 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f246f7fe640 1 -- 172.21.15.83:0/2066396946 learned_addr learned my addr 172.21.15.83:0/2066396946 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f246ffff640 1 --2- 172.21.15.83:0/2066396946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4c8f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f246effd640 1 --2- 172.21.15.83:0/2066396946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c9e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f246f7fe640 1 -- 172.21.15.83:0/2066396946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 msgr2=0x555ff4c9e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f246f7fe640 1 --2- 172.21.15.83:0/2066396946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c9e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f246f7fe640 1 -- 172.21.15.83:0/2066396946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 msgr2=0x555ff4c8f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f246f7fe640 1 --2- 172.21.15.83:0/2066396946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4c8f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f246f7fe640 1 -- 172.21.15.83:0/2066396946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ff4b90ac0 con 0x555ff4c8d2d0 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f246f7fe640 1 --2- 172.21.15.83:0/2066396946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8efa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x555ff4ca5e50 tx=0x7f246000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43979e782017979 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.590+0000 7f24557fa640 1 -- 172.21.15.83:0/2066396946 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460016030 con 0x555ff4c8d2d0 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.591+0000 7f24557fa640 1 -- 172.21.15.83:0/2066396946 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2460017040 con 0x555ff4c8d2d0 2024-03-11T19:47:51.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.591+0000 7f24557fa640 1 -- 172.21.15.83:0/2066396946 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460004d70 con 0x555ff4c8d2d0 2024-03-11T19:47:51.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.591+0000 7f2478e1d740 1 -- 172.21.15.83:0/2066396946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 msgr2=0x555ff4c8efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.591+0000 7f2478e1d740 1 --2- 172.21.15.83:0/2066396946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8efa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x555ff4ca5e50 tx=0x7f246000b390 comp rx=0 tx=0).stop 2024-03-11T19:47:51.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.591+0000 7f2478e1d740 1 -- 172.21.15.83:0/2066396946 shutdown_connections 2024-03-11T19:47:51.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.591+0000 7f2478e1d740 1 --2- 172.21.15.83:0/2066396946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4c8f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.591+0000 7f2478e1d740 1 --2- 172.21.15.83:0/2066396946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c9e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.591+0000 7f2478e1d740 1 --2- 172.21.15.83:0/2066396946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c8efa0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.591+0000 7f2478e1d740 1 -- 172.21.15.83:0/2066396946 >> 172.21.15.83:0/2066396946 conn(0x555ff4c887f0 msgr2=0x555ff4c97340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.591+0000 7f2478e1d740 1 -- 172.21.15.83:0/2066396946 shutdown_connections 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.591+0000 7f2478e1d740 1 -- 172.21.15.83:0/2066396946 wait complete. 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.592+0000 7f2478e1d740 1 -- start start 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.592+0000 7f2478e1d740 1 Processor -- start 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.592+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c9d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.592+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c9dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.592+0000 7f2478e1d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4cf9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.592+0000 7f2478e1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff4cf9bc0 con 0x555ff4c8d2d0 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.592+0000 7f2478e1d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff4cf9d00 con 0x555ff4c92e30 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.592+0000 7f2478e1d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ff4cf9e70 con 0x555ff4c8dcc0 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.592+0000 7f2478e1d740 10 client.0 _ll_register_callbacks cb 0x7f2464096810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c9d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c9d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36434/0 (socket says 172.21.15.83:36434) 2024-03-11T19:47:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246f7fe640 1 -- 172.21.15.83:0/1308052609 learned_addr learned my addr 172.21.15.83:0/1308052609 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246ffff640 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4cf9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246effd640 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c9dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246f7fe640 1 -- 172.21.15.83:0/1308052609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 msgr2=0x555ff4c9dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246f7fe640 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c9dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246f7fe640 1 -- 172.21.15.83:0/1308052609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 msgr2=0x555ff4cf9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246f7fe640 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4cf9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246f7fe640 1 -- 172.21.15.83:0/1308052609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246000b690 con 0x555ff4c8d2d0 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246effd640 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c9dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.593+0000 7f246f7fe640 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c9d840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f246000bfd0 tx=0x7f2460003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b50460b392f809 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f244effd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f244effd640 1 -- 172.21.15.83:0/1308052609 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460016040 con 0x555ff4c8d2d0 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f244effd640 1 -- 172.21.15.83:0/1308052609 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2460004af0 con 0x555ff4c8d2d0 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f2478e1d740 1 -- 172.21.15.83:0/1308052609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f246409e610 con 0x555ff4c8d2d0 2024-03-11T19:47:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f244effd640 1 -- 172.21.15.83:0/1308052609 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2460004cb0 con 0x555ff4c8d2d0 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f2478e1d740 10 client.17013 fetch_fsmap learned FSMap version 32 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f2478e1d740 1 -- 172.21.15.83:0/1308052609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555ff4bb66a0 con 0x555ff4c8d2d0 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f244effd640 1 -- 172.21.15.83:0/1308052609 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f246409e610 con 0x555ff4c8d2d0 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f244effd640 1 -- 172.21.15.83:0/1308052609 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2460005200 con 0x555ff4c8d2d0 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f2478e1d740 10 client.17013 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f2478e1d740 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f2478e1d740 20 client.17013 populate_metadata read hostname 'smithi083' 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f2478e1d740 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555ff4cfb8b0 0x555ff4cfdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f2478e1d740 4 client.17013 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f2478e1d740 1 -- 172.21.15.83:0/1308052609 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555ff4981890 con 0x555ff4cfb8b0 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.594+0000 7f246effd640 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555ff4cfb8b0 0x555ff4cfdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:47:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.595+0000 7f246effd640 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555ff4cfb8b0 0x555ff4cfdcd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24078249c1d7ee00 server_cookie=d8cb065a8ef39486 in_seq=0 out_seq=0 2024-03-11T19:47:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.595+0000 7f244effd640 10 client.17013 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.595+0000 7f244effd640 1 -- 172.21.15.83:0/1308052609 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555ff4981890 con 0x555ff4cfb8b0 2024-03-11T19:47:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.595+0000 7f244effd640 10 client.17013 handle_command_reply: tid=0 2024-03-11T19:47:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.595+0000 7f2478e1d740 1 client.17013 shutdown 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.595+0000 7f2478e1d740 1 -- 172.21.15.83:0/1308052609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 msgr2=0x555ff4c9d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.595+0000 7f2478e1d740 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c9d840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f246000bfd0 tx=0x7f2460003cb0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.596+0000 7f2478e1d740 1 -- 172.21.15.83:0/1308052609 shutdown_connections 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.596+0000 7f2478e1d740 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x555ff4cfb8b0 0x555ff4cfdcd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.596+0000 7f2478e1d740 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ff4c92e30 0x555ff4cf9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.596+0000 7f2478e1d740 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ff4c8dcc0 0x555ff4c9dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.596+0000 7f2478e1d740 1 --2- 172.21.15.83:0/1308052609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ff4c8d2d0 0x555ff4c9d840 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.596+0000 7f2478e1d740 1 -- 172.21.15.83:0/1308052609 >> 172.21.15.83:0/1308052609 conn(0x555ff4c887f0 msgr2=0x555ff4c8efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.596+0000 7f244effd640 0 client.17013 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.596+0000 7f2478e1d740 1 -- 172.21.15.83:0/1308052609 shutdown_connections 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.596+0000 7f2478e1d740 1 -- 172.21.15.83:0/1308052609 wait complete. 2024-03-11T19:47:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.596+0000 7f2478e1d740 20 client.17013 trim_cache size 0 max 16384 2024-03-11T19:47:51.605 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:47:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:47:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:47:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "f53ff92e-75ec-47da-9396-efbdc2a288fd": { 2024-03-11T19:47:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:47:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "f53ff92e-75ec-47da-9396-efbdc2a288fd", 2024-03-11T19:47:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:47:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:47:51.605 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:47:51.605 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:47:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 -- 172.21.15.83:0/3369399619 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24580a1b00 msgr2=0x7f24580a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24580a1b00 0x7f24580a3fc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2468032710 tx=0x7f2468034520 comp rx=0 tx=0).stop 2024-03-11T19:47:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 -- 172.21.15.83:0/3369399619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701023a0 msgr2=0x7f247019dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:47:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701023a0 0x7f247019dd10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f246400e9f0 tx=0x7f246400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 -- 172.21.15.83:0/3369399619 shutdown_connections 2024-03-11T19:47:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f24580a1b00 0x7f24580a3fc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2470102d60 0x7f24701a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24701023a0 0x7f247019dd10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 --2- 172.21.15.83:0/3369399619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24701011b0 0x7f247019d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:47:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 -- 172.21.15.83:0/3369399619 >> 172.21.15.83:0/3369399619 conn(0x7f24700fc740 msgr2=0x7f24700fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:47:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 -- 172.21.15.83:0/3369399619 shutdown_connections 2024-03-11T19:47:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:47:51.597+0000 7f247624e640 1 -- 172.21.15.83:0/3369399619 wait complete. 2024-03-11T19:47:51.615 INFO:tasks.cephfs.filesystem:scrub status for tag:f53ff92e-75ec-47da-9396-efbdc2a288fd - {'path': '/', 'tag': 'f53ff92e-75ec-47da-9396-efbdc2a288fd', 'options': 'recursive,force'} 2024-03-11T19:47:52.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:51 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:47:52.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:51 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:52.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:51 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:47:52.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:51 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:47:52.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:51 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:52.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:51 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:47:52.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:51 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:47:52.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:51 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:52.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:51 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:47:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 515.0 s, 2220.2 tps, lat 45.312 ms stddev 53.362 2024-03-11T19:47:53.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:52 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:47:53.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:52 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:53.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:52 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:47:53.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:52 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:53.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:52 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:53.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:52 smithi137 ceph-mon[36220]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.2 MiB/s wr, 641 op/s 2024-03-11T19:47:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:52 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:47:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:52 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:52 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:47:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:52 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:52 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:52 smithi204 ceph-mon[36739]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.2 MiB/s wr, 641 op/s 2024-03-11T19:47:53.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:52 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:47:53.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:52 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:53.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:52 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:47:53.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:52 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:53.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:52 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:47:53.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:52 smithi083 ceph-mon[31864]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.2 MiB/s wr, 641 op/s 2024-03-11T19:47:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:54 smithi083 ceph-mon[31864]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-03-11T19:47:55.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:54 smithi137 ceph-mon[36220]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-03-11T19:47:55.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:54 smithi204 ceph-mon[36739]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-03-11T19:47:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:56 smithi083 ceph-mon[31864]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.0 MiB/s wr, 694 op/s 2024-03-11T19:47:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:56 smithi137 ceph-mon[36220]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.0 MiB/s wr, 694 op/s 2024-03-11T19:47:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:56 smithi204 ceph-mon[36739]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 7.0 MiB/s wr, 694 op/s 2024-03-11T19:47:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 520.0 s, 3758.9 tps, lat 26.695 ms stddev 31.902 2024-03-11T19:47:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:47:58 smithi083 ceph-mon[31864]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 MiB/s wr, 811 op/s 2024-03-11T19:47:59.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:47:58 smithi137 ceph-mon[36220]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 MiB/s wr, 811 op/s 2024-03-11T19:47:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:47:58 smithi204 ceph-mon[36739]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 MiB/s wr, 811 op/s 2024-03-11T19:48:00.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:00 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:48:00.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:00 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:48:01.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:00 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:48:01.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:01 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:48:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:01 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:48:01.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:01 smithi137 ceph-mon[36220]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 MiB/s wr, 624 op/s 2024-03-11T19:48:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:01 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:48:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:01 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:48:01.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:01 smithi204 ceph-mon[36739]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 MiB/s wr, 624 op/s 2024-03-11T19:48:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:01 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:48:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:01 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:48:02.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:01 smithi083 ceph-mon[31864]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 MiB/s wr, 624 op/s 2024-03-11T19:48:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 525.0 s, 2613.6 tps, lat 37.821 ms stddev 45.145 2024-03-11T19:48:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:02 smithi083 ceph-mon[31864]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.5 MiB/s wr, 785 op/s 2024-03-11T19:48:03.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:02 smithi137 ceph-mon[36220]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.5 MiB/s wr, 785 op/s 2024-03-11T19:48:03.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:02 smithi204 ceph-mon[36739]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.5 MiB/s wr, 785 op/s 2024-03-11T19:48:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:04 smithi083 ceph-mon[31864]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-03-11T19:48:05.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:04 smithi137 ceph-mon[36220]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-03-11T19:48:05.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:04 smithi204 ceph-mon[36739]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.9 MiB/s wr, 826 op/s 2024-03-11T19:48:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:06 smithi083 ceph-mon[31864]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.3 MiB/s wr, 658 op/s 2024-03-11T19:48:07.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:06 smithi137 ceph-mon[36220]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.3 MiB/s wr, 658 op/s 2024-03-11T19:48:07.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:06 smithi204 ceph-mon[36739]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.3 MiB/s wr, 658 op/s 2024-03-11T19:48:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 530.0 s, 1824.6 tps, lat 55.211 ms stddev 66.201 2024-03-11T19:48:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:08 smithi083 ceph-mon[31864]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.8 MiB/s wr, 807 op/s 2024-03-11T19:48:09.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:08 smithi137 ceph-mon[36220]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.8 MiB/s wr, 807 op/s 2024-03-11T19:48:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:08 smithi204 ceph-mon[36739]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.8 MiB/s wr, 807 op/s 2024-03-11T19:48:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:10 smithi083 ceph-mon[31864]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.9 MiB/s wr, 647 op/s 2024-03-11T19:48:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:10 smithi137 ceph-mon[36220]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.9 MiB/s wr, 647 op/s 2024-03-11T19:48:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:10 smithi204 ceph-mon[36739]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.9 MiB/s wr, 647 op/s 2024-03-11T19:48:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 535.0 s, 3062.5 tps, lat 32.649 ms stddev 38.829 2024-03-11T19:48:13.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:12 smithi083 ceph-mon[31864]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.0 MiB/s wr, 740 op/s 2024-03-11T19:48:13.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:12 smithi137 ceph-mon[36220]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.0 MiB/s wr, 740 op/s 2024-03-11T19:48:13.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:12 smithi204 ceph-mon[36739]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.0 MiB/s wr, 740 op/s 2024-03-11T19:48:15.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:14 smithi083 ceph-mon[31864]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 643 op/s 2024-03-11T19:48:15.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:14 smithi137 ceph-mon[36220]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 643 op/s 2024-03-11T19:48:15.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:14 smithi204 ceph-mon[36739]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 643 op/s 2024-03-11T19:48:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:16 smithi083 ceph-mon[31864]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-03-11T19:48:17.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:16 smithi137 ceph-mon[36220]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-03-11T19:48:17.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:16 smithi204 ceph-mon[36739]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.6 MiB/s wr, 638 op/s 2024-03-11T19:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 540.0 s, 2493.6 tps, lat 39.986 ms stddev 43.380 2024-03-11T19:48:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:18 smithi083 ceph-mon[31864]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 788 op/s 2024-03-11T19:48:19.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:18 smithi137 ceph-mon[36220]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 788 op/s 2024-03-11T19:48:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:18 smithi204 ceph-mon[36739]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 MiB/s wr, 788 op/s 2024-03-11T19:48:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:20 smithi083 ceph-mon[31864]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.8 MiB/s wr, 604 op/s 2024-03-11T19:48:21.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:20 smithi137 ceph-mon[36220]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.8 MiB/s wr, 604 op/s 2024-03-11T19:48:21.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:20 smithi204 ceph-mon[36739]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.8 MiB/s wr, 604 op/s 2024-03-11T19:48:21.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:48:21.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:48:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.742+0000 7f0bea423640 1 -- 172.21.15.83:0/1544187352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4106ec0 msgr2=0x7f0be41092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.742+0000 7f0bea423640 1 --2- 172.21.15.83:0/1544187352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4106ec0 0x7f0be41092b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009ca0 tx=0x7f0bdc02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.743+0000 7f0bea423640 1 -- 172.21.15.83:0/1544187352 shutdown_connections 2024-03-11T19:48:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.743+0000 7f0bea423640 1 --2- 172.21.15.83:0/1544187352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4106ec0 0x7f0be41092b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.743+0000 7f0bea423640 1 --2- 172.21.15.83:0/1544187352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be4104590 0x7f0be4106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.743+0000 7f0bea423640 1 --2- 172.21.15.83:0/1544187352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0be4101c60 0x7f0be4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.743+0000 7f0bea423640 1 -- 172.21.15.83:0/1544187352 >> 172.21.15.83:0/1544187352 conn(0x7f0be40fb5f0 msgr2=0x7f0be40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.744+0000 7f0bea423640 1 -- 172.21.15.83:0/1544187352 shutdown_connections 2024-03-11T19:48:21.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.744+0000 7f0bea423640 1 -- 172.21.15.83:0/1544187352 wait complete. 2024-03-11T19:48:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.744+0000 7f0bea423640 1 Processor -- start 2024-03-11T19:48:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.744+0000 7f0bea423640 1 -- start start 2024-03-11T19:48:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0bea423640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4101c60 0x7f0be4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0bea423640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be4104590 0x7f0be4197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0bea423640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0be4106ec0 0x7f0be419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0bea423640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be419ecf0 con 0x7f0be4101c60 2024-03-11T19:48:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0bea423640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be419ee60 con 0x7f0be4106ec0 2024-03-11T19:48:21.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0bea423640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be419f020 con 0x7f0be4104590 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4101c60 0x7f0be4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4101c60 0x7f0be4197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36746/0 (socket says 172.21.15.83:36746) 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0be3fff640 1 -- 172.21.15.83:0/2847126624 learned_addr learned my addr 172.21.15.83:0/2847126624 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0be37fe640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be4104590 0x7f0be4197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0be89c0640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0be4106ec0 0x7f0be419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0be3fff640 1 -- 172.21.15.83:0/2847126624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be4104590 msgr2=0x7f0be4197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0be3fff640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be4104590 0x7f0be4197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0be3fff640 1 -- 172.21.15.83:0/2847126624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0be4106ec0 msgr2=0x7f0be419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0be3fff640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0be4106ec0 0x7f0be419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0be3fff640 1 -- 172.21.15.83:0/2847126624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc009900 con 0x7f0be4101c60 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.745+0000 7f0be3fff640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4101c60 0x7f0be4197200 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800e9f0 tx=0x7f0bd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.746+0000 7f0be17fa640 1 -- 172.21.15.83:0/2847126624 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8009700 con 0x7f0be4101c60 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.746+0000 7f0be17fa640 1 -- 172.21.15.83:0/2847126624 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bd800cd80 con 0x7f0be4101c60 2024-03-11T19:48:21.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.746+0000 7f0bea423640 1 -- 172.21.15.83:0/2847126624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be419f300 con 0x7f0be4101c60 2024-03-11T19:48:21.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.746+0000 7f0be17fa640 1 -- 172.21.15.83:0/2847126624 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8018920 con 0x7f0be4101c60 2024-03-11T19:48:21.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.746+0000 7f0bea423640 1 -- 172.21.15.83:0/2847126624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be419f810 con 0x7f0be4101c60 2024-03-11T19:48:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.748+0000 7f0be17fa640 1 -- 172.21.15.83:0/2847126624 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0bd8018a80 con 0x7f0be4101c60 2024-03-11T19:48:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.748+0000 7f0becff6740 1 Processor -- start 2024-03-11T19:48:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0becff6740 1 -- start start 2024-03-11T19:48:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0becff6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 0x55c15d5d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0becff6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5d87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0becff6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 0x55c15d5db2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0becff6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d543280 con 0x55c15d5d8e30 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0becff6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d5407b0 con 0x55c15d5d3cc0 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0becff6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d53bf90 con 0x55c15d5d32d0 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5d87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 0x55c15d5db2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be89c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 0x55c15d5d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 0x55c15d5db2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36750/0 (socket says 172.21.15.83:36750) 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5d87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47412/0 (socket says 172.21.15.83:47412) 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be37fe640 1 -- 172.21.15.83:0/4068091308 learned_addr learned my addr 172.21.15.83:0/4068091308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be17fa640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0bcc0a1a30 0x7f0bcc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be89c0640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0bcc0a1a30 0x7f0bcc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be37fe640 1 -- 172.21.15.83:0/4068091308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 msgr2=0x55c15d5d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be37fe640 1 --2- 172.21.15.83:0/4068091308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 0x55c15d5d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be37fe640 1 -- 172.21.15.83:0/4068091308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 msgr2=0x55c15d5d87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be37fe640 1 --2- 172.21.15.83:0/4068091308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5d87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be37fe640 1 -- 172.21.15.83:0/4068091308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c15d4fa570 con 0x55c15d5d8e30 2024-03-11T19:48:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be17fa640 1 -- 172.21.15.83:0/2847126624 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0bd8014070 con 0x7f0be4101c60 2024-03-11T19:48:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.749+0000 7f0be37fe640 1 --2- 172.21.15.83:0/4068091308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 0x55c15d5db2c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009c60 tx=0x7f0bd4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2b4463a6825920 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0be89c0640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0bcc0a1a30 0x7f0bcc0a3ef0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc005e30 tx=0x7f0bdc02c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0bca7fc640 1 -- 172.21.15.83:0/4068091308 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4004230 con 0x55c15d5d8e30 2024-03-11T19:48:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0bca7fc640 1 -- 172.21.15.83:0/4068091308 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bd4004390 con 0x55c15d5d8e30 2024-03-11T19:48:21.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0bca7fc640 1 -- 172.21.15.83:0/4068091308 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40197a0 con 0x55c15d5d8e30 2024-03-11T19:48:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0becff6740 1 -- 172.21.15.83:0/4068091308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 msgr2=0x55c15d5db2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0becff6740 1 --2- 172.21.15.83:0/4068091308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 0x55c15d5db2c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009c60 tx=0x7f0bd4011810 comp rx=0 tx=0).stop 2024-03-11T19:48:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0becff6740 1 -- 172.21.15.83:0/4068091308 shutdown_connections 2024-03-11T19:48:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0becff6740 1 --2- 172.21.15.83:0/4068091308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 0x55c15d5db2c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0becff6740 1 --2- 172.21.15.83:0/4068091308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5d87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0becff6740 1 --2- 172.21.15.83:0/4068091308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 0x55c15d5d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0becff6740 1 -- 172.21.15.83:0/4068091308 >> 172.21.15.83:0/4068091308 conn(0x55c15d5ce7f0 msgr2=0x55c15d5cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0becff6740 1 -- 172.21.15.83:0/4068091308 shutdown_connections 2024-03-11T19:48:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.750+0000 7f0becff6740 1 -- 172.21.15.83:0/4068091308 wait complete. 2024-03-11T19:48:21.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.751+0000 7f0becff6740 1 -- start start 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.751+0000 7f0becff6740 1 Processor -- start 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0becff6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 0x55c15d5dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0becff6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5dd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0becff6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 0x55c15d5e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0becff6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d20f430 con 0x55c15d5d8e30 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0becff6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d1cbe00 con 0x55c15d5d3cc0 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0becff6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d4c0fd0 con 0x55c15d5d32d0 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 0x55c15d5dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 0x55c15d5dd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:51848/0 (socket says 172.21.15.83:51848) 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0be3fff640 1 -- 172.21.15.83:0/2532618137 learned_addr learned my addr 172.21.15.83:0/2532618137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0be89c0640 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 0x55c15d5e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0becff6740 10 client.0 _ll_register_callbacks cb 0x55c15d516fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:21.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0be37fe640 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0be3fff640 1 -- 172.21.15.83:0/2532618137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 msgr2=0x55c15d5dd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0be3fff640 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5dd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0be3fff640 1 -- 172.21.15.83:0/2532618137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 msgr2=0x55c15d5e47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0be3fff640 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 0x55c15d5e47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0be3fff640 1 -- 172.21.15.83:0/2532618137 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd4009890 con 0x55c15d5d32d0 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.752+0000 7f0be37fe640 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.753+0000 7f0be3fff640 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 0x55c15d5dd240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800e9c0 tx=0x7f0bd801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e8dd16b87293ea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.753+0000 7f0bc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.753+0000 7f0bc37fe640 1 -- 172.21.15.83:0/2532618137 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80a2e80 con 0x55c15d5d32d0 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.753+0000 7f0bc37fe640 1 -- 172.21.15.83:0/2532618137 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0bd809a790 con 0x55c15d5d32d0 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.753+0000 7f0becff6740 1 -- 172.21.15.83:0/2532618137 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c15d23b7e0 con 0x55c15d5d32d0 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.753+0000 7f0bc37fe640 1 -- 172.21.15.83:0/2532618137 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bd809e830 con 0x55c15d5d32d0 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.753+0000 7f0becff6740 10 client.26593 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:21.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.753+0000 7f0becff6740 1 -- 172.21.15.83:0/2532618137 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c15d53a610 con 0x55c15d5d32d0 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.753+0000 7f0bc37fe640 1 -- 172.21.15.83:0/2532618137 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c15d23b7e0 con 0x55c15d5d32d0 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.753+0000 7f0bc37fe640 1 -- 172.21.15.83:0/2532618137 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0bd80964e0 con 0x55c15d5d32d0 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.754+0000 7f0becff6740 10 client.26593 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.754+0000 7f0becff6740 10 client.26593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.754+0000 7f0becff6740 20 client.26593 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.754+0000 7f0becff6740 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c15d5e9440 0x55c15d5eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.754+0000 7f0becff6740 4 client.26593 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.754+0000 7f0becff6740 1 -- 172.21.15.83:0/2532618137 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c15d278720 con 0x55c15d5e9440 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.754+0000 7f0be37fe640 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c15d5e9440 0x55c15d5eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.754+0000 7f0bc37fe640 10 client.26593 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.754+0000 7f0be37fe640 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c15d5e9440 0x55c15d5eb880 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262ecffdfdf7c3e3 server_cookie=826f10e509c26288 in_seq=0 out_seq=0 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0bc37fe640 1 -- 172.21.15.83:0/2532618137 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c15d278720 con 0x55c15d5e9440 2024-03-11T19:48:21.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0bc37fe640 10 client.26593 handle_command_reply: tid=0 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0becff6740 1 client.26593 shutdown 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0becff6740 1 -- 172.21.15.83:0/2532618137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 msgr2=0x55c15d5dd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0becff6740 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 0x55c15d5dd240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800e9c0 tx=0x7f0bd801f000 comp rx=0 tx=0).stop 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0becff6740 1 -- 172.21.15.83:0/2532618137 shutdown_connections 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0becff6740 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c15d5e9440 0x55c15d5eb880 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0becff6740 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d8e30 0x55c15d5e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0bc37fe640 0 client.26593 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0becff6740 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5dd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0becff6740 1 --2- 172.21.15.83:0/2532618137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d32d0 0x55c15d5dd240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.755+0000 7f0becff6740 1 -- 172.21.15.83:0/2532618137 >> 172.21.15.83:0/2532618137 conn(0x55c15d5ce7f0 msgr2=0x55c15d5cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.756+0000 7f0becff6740 1 -- 172.21.15.83:0/2532618137 shutdown_connections 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.756+0000 7f0becff6740 1 -- 172.21.15.83:0/2532618137 wait complete. 2024-03-11T19:48:21.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.756+0000 7f0becff6740 20 client.26593 trim_cache size 0 max 16384 2024-03-11T19:48:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.786+0000 7f0becff6740 1 Processor -- start 2024-03-11T19:48:21.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.787+0000 7f0becff6740 1 -- start start 2024-03-11T19:48:21.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.787+0000 7f0becff6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d32d0 0x55c15d5dc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.787+0000 7f0becff6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5e4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.787+0000 7f0becff6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 0x55c15d5dce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.788+0000 7f0becff6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d5dc330 con 0x55c15d5d32d0 2024-03-11T19:48:21.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.788+0000 7f0becff6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d5cf7e0 con 0x55c15d5d3cc0 2024-03-11T19:48:21.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.788+0000 7f0be89c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 0x55c15d5dce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.788+0000 7f0be89c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 0x55c15d5dce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:51862/0 (socket says 172.21.15.83:51862) 2024-03-11T19:48:21.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.788+0000 7f0be89c0640 1 -- 172.21.15.83:0/296468358 learned_addr learned my addr 172.21.15.83:0/296468358 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:21.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.788+0000 7f0becff6740 1 -- 172.21.15.83:0/296468358 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d5f0ad0 con 0x55c15d5d8e30 2024-03-11T19:48:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.791+0000 7f0be37fe640 1 --2- 172.21.15.83:0/296468358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5e4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.791+0000 7f0be3fff640 1 --2- 172.21.15.83:0/296468358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d32d0 0x55c15d5dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.793+0000 7f0be37fe640 1 -- 172.21.15.83:0/296468358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 msgr2=0x55c15d5dce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.793+0000 7f0be37fe640 1 --2- 172.21.15.83:0/296468358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 0x55c15d5dce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.793+0000 7f0be37fe640 1 -- 172.21.15.83:0/296468358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d32d0 msgr2=0x55c15d5dc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.793+0000 7f0be37fe640 1 --2- 172.21.15.83:0/296468358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d32d0 0x55c15d5dc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.793+0000 7f0be37fe640 1 -- 172.21.15.83:0/296468358 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c15d4d6ac0 con 0x55c15d5d3cc0 2024-03-11T19:48:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.795+0000 7f0be37fe640 1 --2- 172.21.15.83:0/296468358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5e4780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009c30 tx=0x7f0bd4004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2360fe545c2f27de server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.796+0000 7f0bc9ffb640 1 -- 172.21.15.83:0/296468358 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4005510 con 0x55c15d5d3cc0 2024-03-11T19:48:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.796+0000 7f0bc9ffb640 1 -- 172.21.15.83:0/296468358 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bd4009ed0 con 0x55c15d5d3cc0 2024-03-11T19:48:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.796+0000 7f0bc9ffb640 1 -- 172.21.15.83:0/296468358 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4003a10 con 0x55c15d5d3cc0 2024-03-11T19:48:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.797+0000 7f0becff6740 1 -- 172.21.15.83:0/296468358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 msgr2=0x55c15d5e4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.797+0000 7f0becff6740 1 --2- 172.21.15.83:0/296468358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5e4780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009c30 tx=0x7f0bd4004320 comp rx=0 tx=0).stop 2024-03-11T19:48:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.797+0000 7f0becff6740 1 -- 172.21.15.83:0/296468358 shutdown_connections 2024-03-11T19:48:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.797+0000 7f0becff6740 1 --2- 172.21.15.83:0/296468358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 0x55c15d5dce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.797+0000 7f0becff6740 1 --2- 172.21.15.83:0/296468358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5e4780 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.797+0000 7f0becff6740 1 --2- 172.21.15.83:0/296468358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d32d0 0x55c15d5dc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.797+0000 7f0becff6740 1 -- 172.21.15.83:0/296468358 >> 172.21.15.83:0/296468358 conn(0x55c15d5ce7f0 msgr2=0x55c15d5cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.797+0000 7f0becff6740 1 -- 172.21.15.83:0/296468358 shutdown_connections 2024-03-11T19:48:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.797+0000 7f0becff6740 1 -- 172.21.15.83:0/296468358 wait complete. 2024-03-11T19:48:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.798+0000 7f0becff6740 1 -- start start 2024-03-11T19:48:21.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.798+0000 7f0becff6740 1 Processor -- start 2024-03-11T19:48:21.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0becff6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d32d0 0x55c15d5e4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0becff6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5e2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0becff6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 0x55c15d5e2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0becff6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d5e34e0 con 0x55c15d5d32d0 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0becff6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d5e3650 con 0x55c15d5d3cc0 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0becff6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15d5e3810 con 0x55c15d5d8e30 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0be89c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 0x55c15d5e2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0be89c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 0x55c15d5e2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:51866/0 (socket says 172.21.15.83:51866) 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0be89c0640 1 -- 172.21.15.83:0/4081091867 learned_addr learned my addr 172.21.15.83:0/4081091867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0be37fe640 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5e2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0be3fff640 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d32d0 0x55c15d5e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0becff6740 10 client.0 _ll_register_callbacks cb 0x7f0bd4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0be89c0640 1 -- 172.21.15.83:0/4081091867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 msgr2=0x55c15d5e2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0be89c0640 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5e2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0be89c0640 1 -- 172.21.15.83:0/4081091867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d32d0 msgr2=0x55c15d5e4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:21.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0be89c0640 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d32d0 0x55c15d5e4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0be89c0640 1 -- 172.21.15.83:0/4081091867 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c15d4fa570 con 0x55c15d5d8e30 2024-03-11T19:48:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.799+0000 7f0be3fff640 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d32d0 0x55c15d5e4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:48:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.800+0000 7f0becff6740 1 -- 172.21.15.83:0/4081091867 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bd4004390 con 0x55c15d5d8e30 2024-03-11T19:48:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.800+0000 7f0be89c0640 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 0x55c15d5e2fa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009430 tx=0x7f0bdc0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feddf0c8be624cab server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.800+0000 7f0bc37fe640 10 client.26599 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:48:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.800+0000 7f0bc37fe640 1 -- 172.21.15.83:0/4081091867 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc003d70 con 0x55c15d5d8e30 2024-03-11T19:48:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.800+0000 7f0bc37fe640 1 -- 172.21.15.83:0/4081091867 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0bdc004040 con 0x55c15d5d8e30 2024-03-11T19:48:21.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.801+0000 7f0bc37fe640 1 -- 172.21.15.83:0/4081091867 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bdc004200 con 0x55c15d5d8e30 2024-03-11T19:48:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.801+0000 7f0becff6740 10 client.26599 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.801+0000 7f0becff6740 1 -- 172.21.15.83:0/4081091867 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x55c15d4fc6a0 con 0x55c15d5d8e30 2024-03-11T19:48:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.802+0000 7f0bc37fe640 1 -- 172.21.15.83:0/4081091867 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0bdc03e040 con 0x55c15d5d8e30 2024-03-11T19:48:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.802+0000 7f0becff6740 10 client.26599 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.802+0000 7f0becff6740 10 client.26599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.802+0000 7f0becff6740 20 client.26599 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.802+0000 7f0becff6740 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c15d649800 0x55c15d64bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.802+0000 7f0becff6740 4 client.26599 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:48:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.802+0000 7f0becff6740 1 -- 172.21.15.83:0/4081091867 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f0bd4004390 con 0x55c15d649800 2024-03-11T19:48:21.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.803+0000 7f0be3fff640 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c15d649800 0x55c15d64bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.804+0000 7f0be3fff640 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c15d649800 0x55c15d64bc20 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46a815a66fabc4b4 server_cookie=d84382a98f63efd1 in_seq=0 out_seq=0 2024-03-11T19:48:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.804+0000 7f0bc37fe640 10 client.26599 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.805+0000 7f0bc37fe640 1 -- 172.21.15.83:0/4081091867 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f0bd4004390 con 0x55c15d649800 2024-03-11T19:48:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.805+0000 7f0bc37fe640 10 client.26599 handle_command_reply: tid=0 2024-03-11T19:48:21.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.805+0000 7f0becff6740 1 client.26599 shutdown 2024-03-11T19:48:21.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.806+0000 7f0becff6740 1 -- 172.21.15.83:0/4081091867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 msgr2=0x55c15d5e2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.806+0000 7f0becff6740 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 0x55c15d5e2fa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009430 tx=0x7f0bdc0095c0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.806+0000 7f0becff6740 1 -- 172.21.15.83:0/4081091867 shutdown_connections 2024-03-11T19:48:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.807+0000 7f0becff6740 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c15d649800 0x55c15d64bc20 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.807+0000 7f0becff6740 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c15d5d8e30 0x55c15d5e2fa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.807+0000 7f0becff6740 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c15d5d3cc0 0x55c15d5e2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.807+0000 7f0becff6740 1 --2- 172.21.15.83:0/4081091867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c15d5d32d0 0x55c15d5e4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.807+0000 7f0becff6740 1 -- 172.21.15.83:0/4081091867 >> 172.21.15.83:0/4081091867 conn(0x55c15d5ce7f0 msgr2=0x55c15d5dce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:21.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.807+0000 7f0bc37fe640 0 client.26599 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.808+0000 7f0becff6740 1 -- 172.21.15.83:0/4081091867 shutdown_connections 2024-03-11T19:48:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.808+0000 7f0becff6740 1 -- 172.21.15.83:0/4081091867 wait complete. 2024-03-11T19:48:21.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.808+0000 7f0becff6740 20 client.26599 trim_cache size 0 max 16384 2024-03-11T19:48:21.812 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:48:21.813 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:48:21.813 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:48:21.813 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:48:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.809+0000 7f0bea423640 1 -- 172.21.15.83:0/2847126624 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0bcc0a1a30 msgr2=0x7f0bcc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.809+0000 7f0bea423640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0bcc0a1a30 0x7f0bcc0a3ef0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc005e30 tx=0x7f0bdc02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.809+0000 7f0bea423640 1 -- 172.21.15.83:0/2847126624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4101c60 msgr2=0x7f0be4197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.809+0000 7f0bea423640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4101c60 0x7f0be4197200 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800e9f0 tx=0x7f0bd800eeb0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.809+0000 7f0bea423640 1 -- 172.21.15.83:0/2847126624 shutdown_connections 2024-03-11T19:48:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.809+0000 7f0bea423640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0bcc0a1a30 0x7f0bcc0a3ef0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.809+0000 7f0bea423640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0be4106ec0 0x7f0be419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.809+0000 7f0bea423640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0be4104590 0x7f0be4197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.809+0000 7f0bea423640 1 --2- 172.21.15.83:0/2847126624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0be4101c60 0x7f0be4197200 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.809+0000 7f0bea423640 1 -- 172.21.15.83:0/2847126624 >> 172.21.15.83:0/2847126624 conn(0x7f0be40fb5f0 msgr2=0x7f0be41025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:21.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.809+0000 7f0bea423640 1 -- 172.21.15.83:0/2847126624 shutdown_connections 2024-03-11T19:48:21.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.810+0000 7f0bea423640 1 -- 172.21.15.83:0/2847126624 wait complete. 2024-03-11T19:48:21.827 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:48:21.828 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:48:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 -- 172.21.15.83:0/1168182612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a58102d50 msgr2=0x7f0a581095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 --2- 172.21.15.83:0/1168182612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a58102d50 0x7f0a581095e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009ca0 tx=0x7f0a5002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 -- 172.21.15.83:0/1168182612 shutdown_connections 2024-03-11T19:48:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 --2- 172.21.15.83:0/1168182612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a58102d50 0x7f0a581095e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 --2- 172.21.15.83:0/1168182612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a58102390 0x7f0a58102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 --2- 172.21.15.83:0/1168182612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a581011a0 0x7f0a581015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 -- 172.21.15.83:0/1168182612 >> 172.21.15.83:0/1168182612 conn(0x7f0a580fc730 msgr2=0x7f0a580feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 -- 172.21.15.83:0/1168182612 shutdown_connections 2024-03-11T19:48:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 -- 172.21.15.83:0/1168182612 wait complete. 2024-03-11T19:48:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 Processor -- start 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 -- start start 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a581011a0 0x7f0a58197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a58102390 0x7f0a58197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a58102d50 0x7f0a5819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5819ecf0 con 0x7f0a58102390 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5819ee60 con 0x7f0a58102d50 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a5d815640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5819f020 con 0x7f0a581011a0 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a58102390 0x7f0a58197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a58102390 0x7f0a58197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36796/0 (socket says 172.21.15.83:36796) 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.946+0000 7f0a4ffff640 1 -- 172.21.15.83:0/3031570132 learned_addr learned my addr 172.21.15.83:0/3031570132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a56ffd640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a581011a0 0x7f0a58197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a577fe640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a58102d50 0x7f0a5819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a4ffff640 1 -- 172.21.15.83:0/3031570132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a581011a0 msgr2=0x7f0a58197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a4ffff640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a581011a0 0x7f0a58197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a4ffff640 1 -- 172.21.15.83:0/3031570132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a58102d50 msgr2=0x7f0a5819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a4ffff640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a58102d50 0x7f0a5819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a4ffff640 1 -- 172.21.15.83:0/3031570132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a50009900 con 0x7f0a58102390 2024-03-11T19:48:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a4ffff640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a58102390 0x7f0a58197740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000d770 tx=0x7f0a4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a5c813640 1 -- 172.21.15.83:0/3031570132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40009c30 con 0x7f0a58102390 2024-03-11T19:48:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a5d815640 1 -- 172.21.15.83:0/3031570132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5819f300 con 0x7f0a58102390 2024-03-11T19:48:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a5d815640 1 -- 172.21.15.83:0/3031570132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5819f810 con 0x7f0a58102390 2024-03-11T19:48:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a5c813640 1 -- 172.21.15.83:0/3031570132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a4000de80 con 0x7f0a58102390 2024-03-11T19:48:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.947+0000 7f0a5c813640 1 -- 172.21.15.83:0/3031570132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40004e40 con 0x7f0a58102390 2024-03-11T19:48:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.949+0000 7f0a5d815640 1 -- 172.21.15.83:0/3031570132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a58108460 con 0x7f0a58102390 2024-03-11T19:48:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.950+0000 7f0a5c813640 1 -- 172.21.15.83:0/3031570132 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0a40004fe0 con 0x7f0a58102390 2024-03-11T19:48:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.951+0000 7f0a5c813640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a440a1ab0 0x7f0a440a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.951+0000 7f0a5c813640 1 -- 172.21.15.83:0/3031570132 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0a400ccc30 con 0x7f0a58102390 2024-03-11T19:48:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.951+0000 7f0a56ffd640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a440a1ab0 0x7f0a440a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:21.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.951+0000 7f0a56ffd640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a440a1ab0 0x7f0a440a3f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0a48005fd0 tx=0x7f0a4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:21.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:21.956+0000 7f0a5c813640 1 -- 172.21.15.83:0/3031570132 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0a40095fa0 con 0x7f0a58102390 2024-03-11T19:48:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.283+0000 7f0a5d815640 1 -- 172.21.15.83:0/3031570132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a5810cd60 con 0x7f0a58102390 2024-03-11T19:48:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.284+0000 7f0a5c813640 1 -- 172.21.15.83:0/3031570132 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f0a40020090 con 0x7f0a58102390 2024-03-11T19:48:22.288 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:48:22.288 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:48:22.288 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:48:22.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.287+0000 7f0a5d815640 1 -- 172.21.15.83:0/3031570132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a440a1ab0 msgr2=0x7f0a440a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.287+0000 7f0a5d815640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a440a1ab0 0x7f0a440a3f70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0a48005fd0 tx=0x7f0a4800b410 comp rx=0 tx=0).stop 2024-03-11T19:48:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.287+0000 7f0a5d815640 1 -- 172.21.15.83:0/3031570132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a58102390 msgr2=0x7f0a58197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.287+0000 7f0a5d815640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a58102390 0x7f0a58197740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000d770 tx=0x7f0a4000dc30 comp rx=0 tx=0).stop 2024-03-11T19:48:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.287+0000 7f0a5d815640 1 -- 172.21.15.83:0/3031570132 shutdown_connections 2024-03-11T19:48:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.287+0000 7f0a5d815640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0a440a1ab0 0x7f0a440a3f70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.287+0000 7f0a5d815640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a58102d50 0x7f0a5819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.287+0000 7f0a5d815640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a58102390 0x7f0a58197740 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.287+0000 7f0a5d815640 1 --2- 172.21.15.83:0/3031570132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a581011a0 0x7f0a58197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.287+0000 7f0a5d815640 1 -- 172.21.15.83:0/3031570132 >> 172.21.15.83:0/3031570132 conn(0x7f0a580fc730 msgr2=0x7f0a58105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.287+0000 7f0a5d815640 1 -- 172.21.15.83:0/3031570132 shutdown_connections 2024-03-11T19:48:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.288+0000 7f0a5d815640 1 -- 172.21.15.83:0/3031570132 wait complete. 2024-03-11T19:48:22.305 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:48:22.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:48:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.454+0000 7ff2b660f640 1 -- 172.21.15.83:0/2670218179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b010d4a0 msgr2=0x7ff2b010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.454+0000 7ff2b660f640 1 --2- 172.21.15.83:0/2670218179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b010d4a0 0x7ff2b010d920 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009860 tx=0x7ff2a002bbb0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.457+0000 7ff2b660f640 1 -- 172.21.15.83:0/2670218179 shutdown_connections 2024-03-11T19:48:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.457+0000 7ff2b660f640 1 --2- 172.21.15.83:0/2670218179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b010d4a0 0x7ff2b010d920 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.457+0000 7ff2b660f640 1 --2- 172.21.15.83:0/2670218179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b010aaf0 0x7ff2b010cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.457+0000 7ff2b660f640 1 --2- 172.21.15.83:0/2670218179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2b0108180 0x7ff2b010a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.457+0000 7ff2b660f640 1 -- 172.21.15.83:0/2670218179 >> 172.21.15.83:0/2670218179 conn(0x7ff2b006dfe0 msgr2=0x7ff2b0070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.464+0000 7ff2b660f640 1 -- 172.21.15.83:0/2670218179 shutdown_connections 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.467+0000 7ff2b660f640 1 -- 172.21.15.83:0/2670218179 wait complete. 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.468+0000 7ff2b660f640 1 Processor -- start 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.468+0000 7ff2b660f640 1 -- start start 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.468+0000 7ff2b660f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b0108180 0x7ff2b01146f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.468+0000 7ff2b660f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2b010aaf0 0x7ff2b0114c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.468+0000 7ff2b660f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b010d4a0 0x7ff2b01a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.468+0000 7ff2b660f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0117cf0 con 0x7ff2b0108180 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.468+0000 7ff2b660f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0117e60 con 0x7ff2b010aaf0 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.468+0000 7ff2b660f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b01a5710 con 0x7ff2b010d4a0 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.469+0000 7ff2b4bac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b010d4a0 0x7ff2b01a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.469+0000 7ff2b4bac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b010d4a0 0x7ff2b01a51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:51912/0 (socket says 172.21.15.83:51912) 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.469+0000 7ff2b4bac640 1 -- 172.21.15.83:0/397483451 learned_addr learned my addr 172.21.15.83:0/397483451 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.469+0000 7ff2affff640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b0108180 0x7ff2b01146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.469+0000 7ff2af7fe640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2b010aaf0 0x7ff2b0114c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.470+0000 7ff2affff640 1 -- 172.21.15.83:0/397483451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b010d4a0 msgr2=0x7ff2b01a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.470+0000 7ff2affff640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b010d4a0 0x7ff2b01a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.470+0000 7ff2affff640 1 -- 172.21.15.83:0/397483451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2b010aaf0 msgr2=0x7ff2b0114c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.470+0000 7ff2affff640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2b010aaf0 0x7ff2b0114c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.470+0000 7ff2affff640 1 -- 172.21.15.83:0/397483451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a0009490 con 0x7ff2b0108180 2024-03-11T19:48:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.471+0000 7ff2affff640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b0108180 0x7ff2b01146f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800ef60 tx=0x7ff2a800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.471+0000 7ff2ad7fa640 1 -- 172.21.15.83:0/397483451 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a800b570 con 0x7ff2b0108180 2024-03-11T19:48:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.471+0000 7ff2ad7fa640 1 -- 172.21.15.83:0/397483451 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2a800bbb0 con 0x7ff2b0108180 2024-03-11T19:48:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.471+0000 7ff2ad7fa640 1 -- 172.21.15.83:0/397483451 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8016cd0 con 0x7ff2b0108180 2024-03-11T19:48:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.472+0000 7ff2b660f640 1 -- 172.21.15.83:0/397483451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b01a59f0 con 0x7ff2b0108180 2024-03-11T19:48:22.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.472+0000 7ff2b660f640 1 -- 172.21.15.83:0/397483451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b01a5eb0 con 0x7ff2b0108180 2024-03-11T19:48:22.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.475+0000 7ff2b91db740 1 Processor -- start 2024-03-11T19:48:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.475+0000 7ff2b91db740 1 -- start start 2024-03-11T19:48:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.475+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9270 0x564a53fb9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.475+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fb9c60 0x564a53fbe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.475+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fbedd0 0x564a53fc1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.475+0000 7ff2b91db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a53f75960 con 0x564a53fbedd0 2024-03-11T19:48:22.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.475+0000 7ff2b91db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a53f29310 con 0x564a53fb9c60 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.475+0000 7ff2b91db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a53f26840 con 0x564a53fb9270 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.476+0000 7ff2b4bac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9270 0x564a53fb9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.476+0000 7ff2b4bac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9270 0x564a53fb9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:51928/0 (socket says 172.21.15.83:51928) 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.476+0000 7ff2b4bac640 1 -- 172.21.15.83:0/1967306462 learned_addr learned my addr 172.21.15.83:0/1967306462 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.476+0000 7ff2b4bac640 1 -- 172.21.15.83:0/1967306462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fb9c60 msgr2=0x564a53fbe760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.476+0000 7ff2b4bac640 1 --2- 172.21.15.83:0/1967306462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fb9c60 0x564a53fbe760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.476+0000 7ff2b4bac640 1 -- 172.21.15.83:0/1967306462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fbedd0 msgr2=0x564a53fc1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.476+0000 7ff2b4bac640 1 --2- 172.21.15.83:0/1967306462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fbedd0 0x564a53fc1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.476+0000 7ff2b4bac640 1 -- 172.21.15.83:0/1967306462 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a53ee4060 con 0x564a53fb9270 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.476+0000 7ff2b4bac640 1 --2- 172.21.15.83:0/1967306462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9270 0x564a53fb9690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0034b20 tx=0x7ff2a00323d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a63b2d74d56078e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.477+0000 7ff2967fc640 1 -- 172.21.15.83:0/1967306462 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0004030 con 0x564a53fb9270 2024-03-11T19:48:22.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.477+0000 7ff2967fc640 1 -- 172.21.15.83:0/1967306462 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2a0034440 con 0x564a53fb9270 2024-03-11T19:48:22.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.477+0000 7ff2967fc640 1 -- 172.21.15.83:0/1967306462 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a002c7b0 con 0x564a53fb9270 2024-03-11T19:48:22.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.477+0000 7ff2ad7fa640 1 -- 172.21.15.83:0/397483451 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff2a800a040 con 0x7ff2b0108180 2024-03-11T19:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2b91db740 1 -- 172.21.15.83:0/1967306462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9270 msgr2=0x564a53fb9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2b91db740 1 --2- 172.21.15.83:0/1967306462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9270 0x564a53fb9690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0034b20 tx=0x7ff2a00323d0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2b91db740 1 -- 172.21.15.83:0/1967306462 shutdown_connections 2024-03-11T19:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2b91db740 1 --2- 172.21.15.83:0/1967306462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fbedd0 0x564a53fc1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2b91db740 1 --2- 172.21.15.83:0/1967306462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fb9c60 0x564a53fbe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2b91db740 1 --2- 172.21.15.83:0/1967306462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9270 0x564a53fb9690 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2b91db740 1 -- 172.21.15.83:0/1967306462 >> 172.21.15.83:0/1967306462 conn(0x564a53fb47e0 msgr2=0x564a53fb4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2b91db740 1 -- 172.21.15.83:0/1967306462 shutdown_connections 2024-03-11T19:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2b91db740 1 -- 172.21.15.83:0/1967306462 wait complete. 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2ad7fa640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2980a1ac0 0x7ff2980a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2af7fe640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2980a1ac0 0x7ff2980a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2ad7fa640 1 -- 172.21.15.83:0/397483451 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff2a80cd230 con 0x7ff2b0108180 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2b91db740 1 -- start start 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2b91db740 1 Processor -- start 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.478+0000 7ff2af7fe640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2980a1ac0 0x7ff2980a3f80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x564a53fb5750 tx=0x7ff2a40046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 0x564a53fc3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9c60 0x564a53fc39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 0x564a53fca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b91db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a53f22020 con 0x564a53fb9270 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b91db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a53bf5430 con 0x564a53fbedd0 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b91db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a53bb1e00 con 0x564a53fb9c60 2024-03-11T19:48:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b91db740 10 client.0 _ll_register_callbacks cb 0x564a53ec9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 0x564a53fc3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 0x564a53fc3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36830/0 (socket says 172.21.15.83:36830) 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2affff640 1 -- 172.21.15.83:0/3788638650 learned_addr learned my addr 172.21.15.83:0/3788638650 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2af7fe640 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9c60 0x564a53fc39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b4bac640 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 0x564a53fca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b4bac640 1 -- 172.21.15.83:0/3788638650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9c60 msgr2=0x564a53fc39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b4bac640 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9c60 0x564a53fc39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b4bac640 1 -- 172.21.15.83:0/3788638650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 msgr2=0x564a53fc3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b4bac640 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 0x564a53fc3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2b4bac640 1 -- 172.21.15.83:0/3788638650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b010e4b0 con 0x564a53fbedd0 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.479+0000 7ff2affff640 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 0x564a53fc3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff2b4bac640 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 0x564a53fca9f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff2a002c1c0 tx=0x7ff2a00051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262cfbfc0df3754d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff28f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff28f7fe640 1 -- 172.21.15.83:0/3788638650 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a004b040 con 0x564a53fbedd0 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff28f7fe640 1 -- 172.21.15.83:0/3788638650 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff2a0004450 con 0x564a53fbedd0 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff2b91db740 1 -- 172.21.15.83:0/3788638650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a53c217e0 con 0x564a53fbedd0 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff28f7fe640 1 -- 172.21.15.83:0/3788638650 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2a002c550 con 0x564a53fbedd0 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff2b91db740 10 client.26504 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff2b91db740 1 -- 172.21.15.83:0/3788638650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564a53f206a0 con 0x564a53fbedd0 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff28f7fe640 1 -- 172.21.15.83:0/3788638650 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564a53c217e0 con 0x564a53fbedd0 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff28f7fe640 1 -- 172.21.15.83:0/3788638650 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff2a0045020 con 0x564a53fbedd0 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff2b91db740 10 client.26504 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.480+0000 7ff2b91db740 10 client.26504 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.481+0000 7ff2b91db740 20 client.26504 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.481+0000 7ff2b91db740 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x564a53fcf3a0 0x564a53fd17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.481+0000 7ff2b91db740 4 client.26504 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.481+0000 7ff2b91db740 1 -- 172.21.15.83:0/3788638650 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a53c5e720 con 0x564a53fcf3a0 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.481+0000 7ff2affff640 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x564a53fcf3a0 0x564a53fd17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.481+0000 7ff28f7fe640 10 client.26504 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.481+0000 7ff2affff640 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x564a53fcf3a0 0x564a53fd17c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae1f43625487a1c5 server_cookie=e47853529d6f09d6 in_seq=0 out_seq=0 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff28f7fe640 1 -- 172.21.15.83:0/3788638650 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a53c5e720 con 0x564a53fcf3a0 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff28f7fe640 10 client.26504 handle_command_reply: tid=0 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 1 client.26504 shutdown 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 1 -- 172.21.15.83:0/3788638650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 msgr2=0x564a53fca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 0x564a53fca9f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff2a002c1c0 tx=0x7ff2a00051f0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 1 -- 172.21.15.83:0/3788638650 shutdown_connections 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x564a53fcf3a0 0x564a53fd17c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 0x564a53fca9f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff28f7fe640 0 client.26504 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9c60 0x564a53fc39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 1 --2- 172.21.15.83:0/3788638650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 0x564a53fc3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 1 -- 172.21.15.83:0/3788638650 >> 172.21.15.83:0/3788638650 conn(0x564a53fb47e0 msgr2=0x564a53fb3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 1 -- 172.21.15.83:0/3788638650 shutdown_connections 2024-03-11T19:48:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 1 -- 172.21.15.83:0/3788638650 wait complete. 2024-03-11T19:48:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.482+0000 7ff2b91db740 20 client.26504 trim_cache size 0 max 16384 2024-03-11T19:48:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 545.0 s, 2542.4 tps, lat 39.137 ms stddev 48.783 2024-03-11T19:48:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.499+0000 7ff2b91db740 1 Processor -- start 2024-03-11T19:48:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.499+0000 7ff2b91db740 1 -- start start 2024-03-11T19:48:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.499+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fb9270 0x564a53fc3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.499+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9c60 0x564a53fca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fbedd0 0x564a53fc39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2b91db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a53ea7130 con 0x564a53fb9c60 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2b91db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a53c5e720 con 0x564a53fb9270 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2b91db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a53c217e0 con 0x564a53fbedd0 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2affff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fb9270 0x564a53fc3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2af7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9c60 0x564a53fca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2affff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fb9270 0x564a53fc3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47488/0 (socket says 172.21.15.83:47488) 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2affff640 1 -- 172.21.15.83:0/1651335790 learned_addr learned my addr 172.21.15.83:0/1651335790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2b4bac640 1 --2- 172.21.15.83:0/1651335790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fbedd0 0x564a53fc39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2affff640 1 -- 172.21.15.83:0/1651335790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fbedd0 msgr2=0x564a53fc39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2affff640 1 --2- 172.21.15.83:0/1651335790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fbedd0 0x564a53fc39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2affff640 1 -- 172.21.15.83:0/1651335790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9c60 msgr2=0x564a53fca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2affff640 1 --2- 172.21.15.83:0/1651335790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9c60 0x564a53fca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.500+0000 7ff2affff640 1 -- 172.21.15.83:0/1651335790 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a53ebcc20 con 0x564a53fb9270 2024-03-11T19:48:22.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.501+0000 7ff2affff640 1 --2- 172.21.15.83:0/1651335790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fb9270 0x564a53fc3470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80966c0 tx=0x7ff2a8022040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18591ae6228614c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.501+0000 7ff295ffb640 1 -- 172.21.15.83:0/1651335790 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80cae50 con 0x564a53fb9270 2024-03-11T19:48:22.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.501+0000 7ff295ffb640 1 -- 172.21.15.83:0/1651335790 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2a8098560 con 0x564a53fb9270 2024-03-11T19:48:22.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.501+0000 7ff295ffb640 1 -- 172.21.15.83:0/1651335790 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80a2ba0 con 0x564a53fb9270 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.502+0000 7ff2b91db740 1 -- 172.21.15.83:0/1651335790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fb9270 msgr2=0x564a53fc3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.502+0000 7ff2b91db740 1 --2- 172.21.15.83:0/1651335790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fb9270 0x564a53fc3470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80966c0 tx=0x7ff2a8022040 comp rx=0 tx=0).stop 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.502+0000 7ff2b91db740 1 -- 172.21.15.83:0/1651335790 shutdown_connections 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.502+0000 7ff2b91db740 1 --2- 172.21.15.83:0/1651335790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fbedd0 0x564a53fc39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.502+0000 7ff2b91db740 1 --2- 172.21.15.83:0/1651335790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9c60 0x564a53fca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.502+0000 7ff2b91db740 1 --2- 172.21.15.83:0/1651335790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fb9270 0x564a53fc3470 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.502+0000 7ff2b91db740 1 -- 172.21.15.83:0/1651335790 >> 172.21.15.83:0/1651335790 conn(0x564a53fb47e0 msgr2=0x564a53fbb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.502+0000 7ff2b91db740 1 -- 172.21.15.83:0/1651335790 shutdown_connections 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.503+0000 7ff2b91db740 1 -- 172.21.15.83:0/1651335790 wait complete. 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.503+0000 7ff2b91db740 1 -- start start 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.503+0000 7ff2b91db740 1 Processor -- start 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.503+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 0x564a53fc9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.503+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9c60 0x564a53fc98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.503+0000 7ff2b91db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 0x564a53fc9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.503+0000 7ff2b91db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a54025980 con 0x564a53fb9270 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.503+0000 7ff2b91db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a54025af0 con 0x564a53fbedd0 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.503+0000 7ff2b91db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a54025cb0 con 0x564a53fb9c60 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.503+0000 7ff2b91db740 10 client.0 _ll_register_callbacks cb 0x7ff2a8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2af7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9c60 0x564a53fc98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2af7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9c60 0x564a53fc98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:51962/0 (socket says 172.21.15.83:51962) 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2af7fe640 1 -- 172.21.15.83:0/2885103880 learned_addr learned my addr 172.21.15.83:0/2885103880 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2b4bac640 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 0x564a53fc9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2affff640 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 0x564a53fc9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2affff640 1 -- 172.21.15.83:0/2885103880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9c60 msgr2=0x564a53fc98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2affff640 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9c60 0x564a53fc98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2affff640 1 -- 172.21.15.83:0/2885103880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 msgr2=0x564a53fc9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2affff640 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 0x564a53fc9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2affff640 1 -- 172.21.15.83:0/2885103880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b01a5eb0 con 0x564a53fb9270 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2b4bac640 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 0x564a53fc9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2affff640 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 0x564a53fc9380 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8096670 tx=0x7ff2a80166b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914486210e740917 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff28f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff28f7fe640 1 -- 172.21.15.83:0/2885103880 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80a2da0 con 0x564a53fb9270 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff2b91db740 1 -- 172.21.15.83:0/2885103880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2a0034440 con 0x564a53fb9270 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.504+0000 7ff28f7fe640 1 -- 172.21.15.83:0/2885103880 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff2a80996b0 con 0x564a53fb9270 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff28f7fe640 1 -- 172.21.15.83:0/2885103880 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2a8099870 con 0x564a53fb9270 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff2b91db740 10 client.17052 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff2b91db740 1 -- 172.21.15.83:0/2885103880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564a53ee28c0 con 0x564a53fb9270 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff28f7fe640 1 -- 172.21.15.83:0/2885103880 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2a0034440 con 0x564a53fb9270 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff28f7fe640 1 -- 172.21.15.83:0/2885103880 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff2a80aa070 con 0x564a53fb9270 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff2b91db740 10 client.17052 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff2b91db740 10 client.17052 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff2b91db740 20 client.17052 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff2b91db740 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x564a540273e0 0x564a54029800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff2b91db740 4 client.17052 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff2b91db740 1 -- 172.21.15.83:0/2885103880 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a53cad890 con 0x564a540273e0 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.505+0000 7ff2af7fe640 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x564a540273e0 0x564a54029800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff28f7fe640 10 client.17052 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2af7fe640 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x564a540273e0 0x564a54029800 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d79f3bbccf9c694c server_cookie=8287dca3412f818d in_seq=0 out_seq=0 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff28f7fe640 1 -- 172.21.15.83:0/2885103880 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a53cad890 con 0x564a540273e0 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff28f7fe640 10 client.17052 handle_command_reply: tid=0 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 1 client.17052 shutdown 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 1 -- 172.21.15.83:0/2885103880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 msgr2=0x564a53fc9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 0x564a53fc9380 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8096670 tx=0x7ff2a80166b0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 1 -- 172.21.15.83:0/2885103880 shutdown_connections 2024-03-11T19:48:22.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x564a540273e0 0x564a54029800 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a53fbedd0 0x564a53fc9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff28f7fe640 0 client.17052 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:48:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564a53fb9c60 0x564a53fc98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 1 --2- 172.21.15.83:0/2885103880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a53fb9270 0x564a53fc9380 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 1 -- 172.21.15.83:0/2885103880 >> 172.21.15.83:0/2885103880 conn(0x564a53fb47e0 msgr2=0x564a53fc3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 1 -- 172.21.15.83:0/2885103880 shutdown_connections 2024-03-11T19:48:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 1 -- 172.21.15.83:0/2885103880 wait complete. 2024-03-11T19:48:22.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.506+0000 7ff2b91db740 20 client.17052 trim_cache size 0 max 16384 2024-03-11T19:48:22.515 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:48:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 -- 172.21.15.83:0/397483451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2980a1ac0 msgr2=0x7ff2980a3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2980a1ac0 0x7ff2980a3f80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x564a53fb5750 tx=0x7ff2a40046c0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 -- 172.21.15.83:0/397483451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b0108180 msgr2=0x7ff2b01146f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b0108180 0x7ff2b01146f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800ef60 tx=0x7ff2a800ef90 comp rx=0 tx=0).stop 2024-03-11T19:48:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 -- 172.21.15.83:0/397483451 shutdown_connections 2024-03-11T19:48:22.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff2980a1ac0 0x7ff2980a3f80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2b010d4a0 0x7ff2b01a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2b010aaf0 0x7ff2b0114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 --2- 172.21.15.83:0/397483451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2b0108180 0x7ff2b01146f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 -- 172.21.15.83:0/397483451 >> 172.21.15.83:0/397483451 conn(0x7ff2b006dfe0 msgr2=0x7ff2b01085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 -- 172.21.15.83:0/397483451 shutdown_connections 2024-03-11T19:48:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.507+0000 7ff2b660f640 1 -- 172.21.15.83:0/397483451 wait complete. 2024-03-11T19:48:22.524 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:48:22.524 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:48:22.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:22 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3031570132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:48:22.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:22 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3031570132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:48:22.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:22 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3031570132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:48:22.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.678+0000 7fc355476640 1 -- 172.21.15.83:0/34145072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc35010d830 msgr2=0x7fc35010fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.678+0000 7fc355476640 1 --2- 172.21.15.83:0/34145072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc35010d830 0x7fc35010fcf0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc340005fd0 tx=0x7fc340030130 comp rx=0 tx=0).stop 2024-03-11T19:48:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.680+0000 7fc355476640 1 -- 172.21.15.83:0/34145072 shutdown_connections 2024-03-11T19:48:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.680+0000 7fc355476640 1 --2- 172.21.15.83:0/34145072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc35010d830 0x7fc35010fcf0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.680+0000 7fc355476640 1 --2- 172.21.15.83:0/34145072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35010adb0 0x7fc35010d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.680+0000 7fc355476640 1 --2- 172.21.15.83:0/34145072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3501083f0 0x7fc35010a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.680+0000 7fc355476640 1 -- 172.21.15.83:0/34145072 >> 172.21.15.83:0/34145072 conn(0x7fc35006e380 msgr2=0x7fc3500707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.685+0000 7fc355476640 1 -- 172.21.15.83:0/34145072 shutdown_connections 2024-03-11T19:48:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.687+0000 7fc355476640 1 -- 172.21.15.83:0/34145072 wait complete. 2024-03-11T19:48:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.688+0000 7fc355476640 1 Processor -- start 2024-03-11T19:48:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.688+0000 7fc355476640 1 -- start start 2024-03-11T19:48:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.688+0000 7fc355476640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3501083f0 0x7fc35011a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.688+0000 7fc355476640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc35010adb0 0x7fc350116be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc355476640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35010d830 0x7fc350117240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc355476640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3501b6270 con 0x7fc3501083f0 2024-03-11T19:48:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc355476640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3501b63b0 con 0x7fc35010adb0 2024-03-11T19:48:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc355476640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3501b6570 con 0x7fc35010d830 2024-03-11T19:48:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3501083f0 0x7fc35011a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35010d830 0x7fc350117240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35010d830 0x7fc350117240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:51970/0 (socket says 172.21.15.83:51970) 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3501083f0 0x7fc35011a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36864/0 (socket says 172.21.15.83:36864) 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34f7fe640 1 -- 172.21.15.83:0/755712383 learned_addr learned my addr 172.21.15.83:0/755712383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34e7fc640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc35010adb0 0x7fc350116be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34f7fe640 1 -- 172.21.15.83:0/755712383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc35010adb0 msgr2=0x7fc350116be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc35010adb0 0x7fc350116be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34f7fe640 1 -- 172.21.15.83:0/755712383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3501083f0 msgr2=0x7fc35011a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3501083f0 0x7fc35011a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34f7fe640 1 -- 172.21.15.83:0/755712383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3400083c0 con 0x7fc35010d830 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34effd640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3501083f0 0x7fc35011a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34e7fc640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc35010adb0 0x7fc350116be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.689+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35010d830 0x7fc350117240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc34003a040 tx=0x7fc340030bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.690+0000 7fc347fff640 1 -- 172.21.15.83:0/755712383 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340012320 con 0x7fc35010d830 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.690+0000 7fc355476640 1 -- 172.21.15.83:0/755712383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3500733b0 con 0x7fc35010d830 2024-03-11T19:48:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.690+0000 7fc347fff640 1 -- 172.21.15.83:0/755712383 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc340012480 con 0x7fc35010d830 2024-03-11T19:48:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.690+0000 7fc347fff640 1 -- 172.21.15.83:0/755712383 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340011a20 con 0x7fc35010d830 2024-03-11T19:48:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.690+0000 7fc355476640 1 -- 172.21.15.83:0/755712383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc350073910 con 0x7fc35010d830 2024-03-11T19:48:22.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.693+0000 7fc347fff640 1 -- 172.21.15.83:0/755712383 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc340011b80 con 0x7fc35010d830 2024-03-11T19:48:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.695+0000 7fc347fff640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc33c0a1b70 0x7fc33c0a4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.695+0000 7fc347fff640 1 -- 172.21.15.83:0/755712383 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc340111660 con 0x7fc35010d830 2024-03-11T19:48:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.696+0000 7fc34effd640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc33c0a1b70 0x7fc33c0a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.696+0000 7fc34effd640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc33c0a1b70 0x7fc33c0a4030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc33800bfd0 tx=0x7fc33800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.696+0000 7fc358047740 1 Processor -- start 2024-03-11T19:48:22.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.696+0000 7fc358047740 1 -- start start 2024-03-11T19:48:22.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc358047740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2b270 0x55e59cf2b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc358047740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2bc60 0x55e59cf30760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc358047740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf30dd0 0x55e59cf33260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc358047740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59cee7960 con 0x55e59cf2b270 2024-03-11T19:48:22.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc358047740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ce9b310 con 0x55e59cf30dd0 2024-03-11T19:48:22.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc358047740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ce98840 con 0x55e59cf2bc60 2024-03-11T19:48:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc34effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2b270 0x55e59cf2b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc34effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2b270 0x55e59cf2b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36874/0 (socket says 172.21.15.83:36874) 2024-03-11T19:48:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc34effd640 1 -- 172.21.15.83:0/149770620 learned_addr learned my addr 172.21.15.83:0/149770620 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc34e7fc640 1 --2- 172.21.15.83:0/149770620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2bc60 0x55e59cf30760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/149770620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf30dd0 0x55e59cf33260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc34e7fc640 1 -- 172.21.15.83:0/149770620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf30dd0 msgr2=0x55e59cf33260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc34e7fc640 1 --2- 172.21.15.83:0/149770620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf30dd0 0x55e59cf33260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.697+0000 7fc34e7fc640 1 -- 172.21.15.83:0/149770620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2b270 msgr2=0x55e59cf2b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.698+0000 7fc34e7fc640 1 --2- 172.21.15.83:0/149770620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2b270 0x55e59cf2b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.698+0000 7fc34e7fc640 1 -- 172.21.15.83:0/149770620 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e59ce56060 con 0x55e59cf2bc60 2024-03-11T19:48:22.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.698+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/149770620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf30dd0 0x55e59cf33260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.699+0000 7fc34e7fc640 1 --2- 172.21.15.83:0/149770620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2bc60 0x55e59cf30760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc3500732c0 tx=0x7fc34800ad50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab4ba3aeb8a258d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.699+0000 7fc3457fa640 1 -- 172.21.15.83:0/149770620 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34800cb40 con 0x55e59cf2bc60 2024-03-11T19:48:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.699+0000 7fc3457fa640 1 -- 172.21.15.83:0/149770620 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc34800cca0 con 0x55e59cf2bc60 2024-03-11T19:48:22.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.699+0000 7fc3457fa640 1 -- 172.21.15.83:0/149770620 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348018690 con 0x55e59cf2bc60 2024-03-11T19:48:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.700+0000 7fc358047740 1 -- 172.21.15.83:0/149770620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2bc60 msgr2=0x55e59cf30760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.700+0000 7fc358047740 1 --2- 172.21.15.83:0/149770620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2bc60 0x55e59cf30760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc3500732c0 tx=0x7fc34800ad50 comp rx=0 tx=0).stop 2024-03-11T19:48:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.700+0000 7fc358047740 1 -- 172.21.15.83:0/149770620 shutdown_connections 2024-03-11T19:48:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.700+0000 7fc358047740 1 --2- 172.21.15.83:0/149770620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf30dd0 0x55e59cf33260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.700+0000 7fc358047740 1 --2- 172.21.15.83:0/149770620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2bc60 0x55e59cf30760 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.700+0000 7fc358047740 1 --2- 172.21.15.83:0/149770620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2b270 0x55e59cf2b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.700+0000 7fc358047740 1 -- 172.21.15.83:0/149770620 >> 172.21.15.83:0/149770620 conn(0x55e59cf267e0 msgr2=0x55e59cf26be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.700+0000 7fc358047740 1 -- 172.21.15.83:0/149770620 shutdown_connections 2024-03-11T19:48:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.700+0000 7fc358047740 1 -- 172.21.15.83:0/149770620 wait complete. 2024-03-11T19:48:22.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc358047740 1 -- start start 2024-03-11T19:48:22.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc358047740 1 Processor -- start 2024-03-11T19:48:22.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc358047740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2b270 0x55e59cf354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc358047740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2bc60 0x55e59cf359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc358047740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 0x55e59cf3ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc358047740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ce94020 con 0x55e59cf2bc60 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc358047740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59cb67430 con 0x55e59cf2b270 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc358047740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59cb23e00 con 0x55e59cf30dd0 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc358047740 10 client.0 _ll_register_callbacks cb 0x55e59ce3bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc34f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 0x55e59cf3ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc34f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 0x55e59cf3ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:51982/0 (socket says 172.21.15.83:51982) 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc34f7fe640 1 -- 172.21.15.83:0/1771340229 learned_addr learned my addr 172.21.15.83:0/1771340229 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc34effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2b270 0x55e59cf354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc34e7fc640 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2bc60 0x55e59cf359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc34f7fe640 1 -- 172.21.15.83:0/1771340229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2b270 msgr2=0x55e59cf354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2b270 0x55e59cf354b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc34f7fe640 1 -- 172.21.15.83:0/1771340229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2bc60 msgr2=0x55e59cf359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2bc60 0x55e59cf359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.701+0000 7fc34f7fe640 1 -- 172.21.15.83:0/1771340229 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34800a520 con 0x55e59cf30dd0 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.702+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 0x55e59cf3ca30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc3400dafb0 tx=0x7fc34000aea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c2bd62293427c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.702+0000 7fc3367fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:48:22.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.702+0000 7fc3367fc640 1 -- 172.21.15.83:0/1771340229 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3400de5e0 con 0x55e59cf30dd0 2024-03-11T19:48:22.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.702+0000 7fc358047740 1 -- 172.21.15.83:0/1771340229 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e59cb937e0 con 0x55e59cf30dd0 2024-03-11T19:48:22.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.702+0000 7fc3367fc640 1 -- 172.21.15.83:0/1771340229 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc3400e2730 con 0x55e59cf30dd0 2024-03-11T19:48:22.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.702+0000 7fc3367fc640 1 -- 172.21.15.83:0/1771340229 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3400e6d80 con 0x55e59cf30dd0 2024-03-11T19:48:22.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.702+0000 7fc358047740 10 client.26632 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:22.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.702+0000 7fc358047740 1 -- 172.21.15.83:0/1771340229 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e59ce926a0 con 0x55e59cf30dd0 2024-03-11T19:48:22.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.702+0000 7fc3367fc640 1 -- 172.21.15.83:0/1771340229 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc340038560 con 0x55e59cf30dd0 2024-03-11T19:48:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.704+0000 7fc3367fc640 1 -- 172.21.15.83:0/1771340229 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc3400de840 con 0x55e59cf30dd0 2024-03-11T19:48:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.705+0000 7fc358047740 10 client.26632 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.705+0000 7fc358047740 10 client.26632 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:48:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.705+0000 7fc358047740 20 client.26632 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.705+0000 7fc358047740 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e59cf413f0 0x55e59cf43820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.705+0000 7fc358047740 4 client.26632 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.705+0000 7fc358047740 1 -- 172.21.15.83:0/1771340229 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e59cb937e0 con 0x55e59cf413f0 2024-03-11T19:48:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.705+0000 7fc34effd640 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e59cf413f0 0x55e59cf43820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.706+0000 7fc34effd640 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e59cf413f0 0x55e59cf43820 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=638d3f0d0e198f1e server_cookie=8473058625157e23 in_seq=0 out_seq=0 2024-03-11T19:48:22.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.706+0000 7fc3367fc640 10 client.26632 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:48:22.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.706+0000 7fc3367fc640 1 -- 172.21.15.83:0/1771340229 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e59cb937e0 con 0x55e59cf413f0 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.706+0000 7fc3367fc640 10 client.26632 handle_command_reply: tid=0 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.706+0000 7fc358047740 1 client.26632 shutdown 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.706+0000 7fc358047740 1 -- 172.21.15.83:0/1771340229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 msgr2=0x55e59cf3ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.706+0000 7fc358047740 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 0x55e59cf3ca30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc3400dafb0 tx=0x7fc34000aea0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.706+0000 7fc358047740 1 -- 172.21.15.83:0/1771340229 shutdown_connections 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.706+0000 7fc358047740 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e59cf413f0 0x55e59cf43820 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.707+0000 7fc358047740 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 0x55e59cf3ca30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.707+0000 7fc358047740 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2bc60 0x55e59cf359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.707+0000 7fc3367fc640 0 client.26632 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.707+0000 7fc358047740 1 --2- 172.21.15.83:0/1771340229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2b270 0x55e59cf354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.707+0000 7fc358047740 1 -- 172.21.15.83:0/1771340229 >> 172.21.15.83:0/1771340229 conn(0x55e59cf267e0 msgr2=0x55e59cf25b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.707+0000 7fc358047740 1 -- 172.21.15.83:0/1771340229 shutdown_connections 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.707+0000 7fc358047740 1 -- 172.21.15.83:0/1771340229 wait complete. 2024-03-11T19:48:22.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.707+0000 7fc358047740 20 client.26632 trim_cache size 0 max 16384 2024-03-11T19:48:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.721+0000 7fc358047740 1 Processor -- start 2024-03-11T19:48:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.721+0000 7fc358047740 1 -- start start 2024-03-11T19:48:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.722+0000 7fc358047740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2b270 0x55e59cf26bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.722+0000 7fc358047740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2bc60 0x55e59cf3c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.722+0000 7fc34effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2b270 0x55e59cf26bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.722+0000 7fc34e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2bc60 0x55e59cf3c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.722+0000 7fc34effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2b270 0x55e59cf26bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47552/0 (socket says 172.21.15.83:47552) 2024-03-11T19:48:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.722+0000 7fc34effd640 1 -- 172.21.15.83:0/3999962896 learned_addr learned my addr 172.21.15.83:0/3999962896 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.722+0000 7fc358047740 1 --2- 172.21.15.83:0/3999962896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 0x55e59cf2d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.722+0000 7fc358047740 1 -- 172.21.15.83:0/3999962896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59ce19130 con 0x55e59cf2bc60 2024-03-11T19:48:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.722+0000 7fc358047740 1 -- 172.21.15.83:0/3999962896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59cf333f0 con 0x55e59cf2b270 2024-03-11T19:48:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.722+0000 7fc358047740 1 -- 172.21.15.83:0/3999962896 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59cf22120 con 0x55e59cf30dd0 2024-03-11T19:48:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.723+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/3999962896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 0x55e59cf2d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.723+0000 7fc34e7fc640 1 -- 172.21.15.83:0/3999962896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 msgr2=0x55e59cf2d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.723+0000 7fc34e7fc640 1 --2- 172.21.15.83:0/3999962896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 0x55e59cf2d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.723+0000 7fc34e7fc640 1 -- 172.21.15.83:0/3999962896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2b270 msgr2=0x55e59cf26bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.723+0000 7fc34e7fc640 1 --2- 172.21.15.83:0/3999962896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2b270 0x55e59cf26bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.723+0000 7fc34e7fc640 1 -- 172.21.15.83:0/3999962896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e59ce2ec20 con 0x55e59cf2bc60 2024-03-11T19:48:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.723+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/3999962896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 0x55e59cf2d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.723+0000 7fc34e7fc640 1 --2- 172.21.15.83:0/3999962896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2bc60 0x55e59cf3c9f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55e59cf27750 tx=0x7fc348011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba35fd517476600 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.723+0000 7fc344ff9640 1 -- 172.21.15.83:0/3999962896 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348020040 con 0x55e59cf2bc60 2024-03-11T19:48:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.723+0000 7fc344ff9640 1 -- 172.21.15.83:0/3999962896 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc34800ce90 con 0x55e59cf2bc60 2024-03-11T19:48:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.723+0000 7fc344ff9640 1 -- 172.21.15.83:0/3999962896 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348003da0 con 0x55e59cf2bc60 2024-03-11T19:48:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.724+0000 7fc358047740 1 -- 172.21.15.83:0/3999962896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2bc60 msgr2=0x55e59cf3c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.724+0000 7fc358047740 1 --2- 172.21.15.83:0/3999962896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2bc60 0x55e59cf3c9f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55e59cf27750 tx=0x7fc348011f90 comp rx=0 tx=0).stop 2024-03-11T19:48:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.724+0000 7fc358047740 1 -- 172.21.15.83:0/3999962896 shutdown_connections 2024-03-11T19:48:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.724+0000 7fc358047740 1 --2- 172.21.15.83:0/3999962896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf30dd0 0x55e59cf2d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.724+0000 7fc358047740 1 --2- 172.21.15.83:0/3999962896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf2bc60 0x55e59cf3c9f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.724+0000 7fc358047740 1 --2- 172.21.15.83:0/3999962896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2b270 0x55e59cf26bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.724+0000 7fc358047740 1 -- 172.21.15.83:0/3999962896 >> 172.21.15.83:0/3999962896 conn(0x55e59cf267e0 msgr2=0x55e59cf21a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.724+0000 7fc358047740 1 -- 172.21.15.83:0/3999962896 shutdown_connections 2024-03-11T19:48:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.724+0000 7fc358047740 1 -- 172.21.15.83:0/3999962896 wait complete. 2024-03-11T19:48:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc358047740 1 -- start start 2024-03-11T19:48:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc358047740 1 Processor -- start 2024-03-11T19:48:22.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc358047740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2b270 0x55e59cf3b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc358047740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2bc60 0x55e59cf3b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc358047740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf30dd0 0x55e59cf3bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc358047740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59cf2cd70 con 0x55e59cf30dd0 2024-03-11T19:48:22.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc358047740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59cf34230 con 0x55e59cf2bc60 2024-03-11T19:48:22.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc358047740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59cf97aa0 con 0x55e59cf2b270 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc34f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf30dd0 0x55e59cf3bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc34effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2b270 0x55e59cf3b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc34f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf30dd0 0x55e59cf3bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36886/0 (socket says 172.21.15.83:36886) 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc34effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2b270 0x55e59cf3b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:51998/0 (socket says 172.21.15.83:51998) 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc34f7fe640 1 -- 172.21.15.83:0/3293042090 learned_addr learned my addr 172.21.15.83:0/3293042090 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc34e7fc640 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2bc60 0x55e59cf3b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.725+0000 7fc358047740 10 client.0 _ll_register_callbacks cb 0x7fc34800a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc34f7fe640 1 -- 172.21.15.83:0/3293042090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2b270 msgr2=0x55e59cf3b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2b270 0x55e59cf3b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc34f7fe640 1 -- 172.21.15.83:0/3293042090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2bc60 msgr2=0x55e59cf3b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2bc60 0x55e59cf3b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc34f7fe640 1 -- 172.21.15.83:0/3293042090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e59ce926a0 con 0x55e59cf30dd0 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc34f7fe640 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf30dd0 0x55e59cf3bed0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc34000ab40 tx=0x7fc3400fe040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb6ca87007d0386 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc3367fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc3367fc640 1 -- 172.21.15.83:0/3293042090 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3400ddf40 con 0x55e59cf30dd0 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc358047740 1 -- 172.21.15.83:0/3293042090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc34800cca0 con 0x55e59cf30dd0 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc3367fc640 1 -- 172.21.15.83:0/3293042090 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc3400de5e0 con 0x55e59cf30dd0 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc3367fc640 1 -- 172.21.15.83:0/3293042090 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3400e2730 con 0x55e59cf30dd0 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc358047740 10 client.17070 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc358047740 1 -- 172.21.15.83:0/3293042090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e59ce56060 con 0x55e59cf30dd0 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc3367fc640 1 -- 172.21.15.83:0/3293042090 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc34800cca0 con 0x55e59cf30dd0 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc3367fc640 1 -- 172.21.15.83:0/3293042090 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc3400dda30 con 0x55e59cf30dd0 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.726+0000 7fc358047740 10 client.17070 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 10 client.17070 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 20 client.17070 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e59cf990b0 0x55e59cf9b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 4 client.17070 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:48:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 1 -- 172.21.15.83:0/3293042090 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e59cbd0720 con 0x55e59cf990b0 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc34effd640 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e59cf990b0 0x55e59cf9b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc3367fc640 10 client.17070 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc34effd640 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e59cf990b0 0x55e59cf9b4a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6086eb98f1c49be server_cookie=3696e700ffd50495 in_seq=0 out_seq=0 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc3367fc640 1 -- 172.21.15.83:0/3293042090 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e59cbd0720 con 0x55e59cf990b0 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc3367fc640 10 client.17070 handle_command_reply: tid=0 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 1 client.17070 shutdown 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 1 -- 172.21.15.83:0/3293042090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf30dd0 msgr2=0x55e59cf3bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf30dd0 0x55e59cf3bed0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc34000ab40 tx=0x7fc3400fe040 comp rx=0 tx=0).stop 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 1 -- 172.21.15.83:0/3293042090 shutdown_connections 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e59cf990b0 0x55e59cf9b4a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc3367fc640 0 client.17070 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e59cf30dd0 0x55e59cf3bed0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.727+0000 7fc358047740 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e59cf2bc60 0x55e59cf3b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc358047740 1 --2- 172.21.15.83:0/3293042090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e59cf2b270 0x55e59cf3b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc358047740 1 -- 172.21.15.83:0/3293042090 >> 172.21.15.83:0/3293042090 conn(0x55e59cf267e0 msgr2=0x55e59cf26bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc358047740 1 -- 172.21.15.83:0/3293042090 shutdown_connections 2024-03-11T19:48:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc358047740 1 -- 172.21.15.83:0/3293042090 wait complete. 2024-03-11T19:48:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc358047740 20 client.17070 trim_cache size 0 max 16384 2024-03-11T19:48:22.738 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:48:22.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 -- 172.21.15.83:0/755712383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc33c0a1b70 msgr2=0x7fc33c0a4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc33c0a1b70 0x7fc33c0a4030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc33800bfd0 tx=0x7fc33800b390 comp rx=0 tx=0).stop 2024-03-11T19:48:22.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 -- 172.21.15.83:0/755712383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35010d830 msgr2=0x7fc350117240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35010d830 0x7fc350117240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc34003a040 tx=0x7fc340030bb0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 -- 172.21.15.83:0/755712383 shutdown_connections 2024-03-11T19:48:22.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc33c0a1b70 0x7fc33c0a4030 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35010d830 0x7fc350117240 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc35010adb0 0x7fc350116be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 --2- 172.21.15.83:0/755712383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3501083f0 0x7fc35011a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 -- 172.21.15.83:0/755712383 >> 172.21.15.83:0/755712383 conn(0x7fc35006e380 msgr2=0x7fc350108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 -- 172.21.15.83:0/755712383 shutdown_connections 2024-03-11T19:48:22.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.728+0000 7fc355476640 1 -- 172.21.15.83:0/755712383 wait complete. 2024-03-11T19:48:22.746 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:48:22.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:48:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 -- 172.21.15.83:0/2383323395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb500100080 msgr2=0x7fb5001004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 --2- 172.21.15.83:0/2383323395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb500100080 0x7fb5001004a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009900 tx=0x7fb4f402bc30 comp rx=0 tx=0).stop 2024-03-11T19:48:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 -- 172.21.15.83:0/2383323395 shutdown_connections 2024-03-11T19:48:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 --2- 172.21.15.83:0/2383323395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb500101c30 0x7fb5001084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 --2- 172.21.15.83:0/2383323395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb500101270 0x7fb5001016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 --2- 172.21.15.83:0/2383323395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb500100080 0x7fb5001004a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 -- 172.21.15.83:0/2383323395 >> 172.21.15.83:0/2383323395 conn(0x7fb5000fb5f0 msgr2=0x7fb5000fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 -- 172.21.15.83:0/2383323395 shutdown_connections 2024-03-11T19:48:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 -- 172.21.15.83:0/2383323395 wait complete. 2024-03-11T19:48:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 Processor -- start 2024-03-11T19:48:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 -- start start 2024-03-11T19:48:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb500100080 0x7fb50006a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb500101270 0x7fb50019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb500101c30 0x7fb50019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50019ede0 con 0x7fb500101c30 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50019ef50 con 0x7fb500100080 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb506f0c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50019f110 con 0x7fb500101270 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb5054a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb500101c30 0x7fb50019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb5054a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb500101c30 0x7fb50019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36898/0 (socket says 172.21.15.83:36898) 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb5054a9640 1 -- 172.21.15.83:0/419749157 learned_addr learned my addr 172.21.15.83:0/419749157 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.883+0000 7fb504ca8640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb500100080 0x7fb50006a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb4fffff640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb500101270 0x7fb50019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb5054a9640 1 -- 172.21.15.83:0/419749157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb500101270 msgr2=0x7fb50019bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb5054a9640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb500101270 0x7fb50019bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb5054a9640 1 -- 172.21.15.83:0/419749157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb500100080 msgr2=0x7fb50006a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb5054a9640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb500100080 0x7fb50006a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb5054a9640 1 -- 172.21.15.83:0/419749157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec009490 con 0x7fb500101c30 2024-03-11T19:48:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb5054a9640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb500101c30 0x7fb50019e600 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009fd0 tx=0x7fb4f800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb4fdffb640 1 -- 172.21.15.83:0/419749157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f800bea0 con 0x7fb500101c30 2024-03-11T19:48:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb506f0c640 1 -- 172.21.15.83:0/419749157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f4009560 con 0x7fb500101c30 2024-03-11T19:48:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb4fdffb640 1 -- 172.21.15.83:0/419749157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4f8009940 con 0x7fb500101c30 2024-03-11T19:48:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb4fdffb640 1 -- 172.21.15.83:0/419749157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f80106c0 con 0x7fb500101c30 2024-03-11T19:48:22.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.884+0000 7fb506f0c640 1 -- 172.21.15.83:0/419749157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50019f710 con 0x7fb500101c30 2024-03-11T19:48:22.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.888+0000 7fb4fdffb640 1 -- 172.21.15.83:0/419749157 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fb4f8004240 con 0x7fb500101c30 2024-03-11T19:48:22.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.888+0000 7fb509bc4740 1 Processor -- start 2024-03-11T19:48:22.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.888+0000 7fb509bc4740 1 -- start start 2024-03-11T19:48:22.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd3680 0x55f91cbd3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 0x55f91cbd8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd91e0 0x55f91cbdb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb509bc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91cb8b960 con 0x55f91cbd4070 2024-03-11T19:48:22.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb509bc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91cb3f310 con 0x55f91cbd91e0 2024-03-11T19:48:22.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb509bc4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91cb3c840 con 0x55f91cbd3680 2024-03-11T19:48:22.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb4fdffb640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4f00a1a30 0x7fb4f00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb504ca8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 0x55f91cbd8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb4fffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd3680 0x55f91cbd3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb504ca8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 0x55f91cbd8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36914/0 (socket says 172.21.15.83:36914) 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb4fffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd3680 0x55f91cbd3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52014/0 (socket says 172.21.15.83:52014) 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb504ca8640 1 -- 172.21.15.83:0/2613783539 learned_addr learned my addr 172.21.15.83:0/2613783539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb4fffff640 1 --2- 172.21.15.83:0/2613783539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd91e0 0x55f91cbdb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb5054a9640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4f00a1a30 0x7fb4f00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb4fdffb640 1 -- 172.21.15.83:0/419749157 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fb4f80ccfc0 con 0x7fb500101c30 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb504ca8640 1 -- 172.21.15.83:0/2613783539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd3680 msgr2=0x55f91cbd3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb504ca8640 1 --2- 172.21.15.83:0/2613783539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd3680 0x55f91cbd3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb504ca8640 1 -- 172.21.15.83:0/2613783539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd91e0 msgr2=0x55f91cbdb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb504ca8640 1 --2- 172.21.15.83:0/2613783539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd91e0 0x55f91cbdb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb504ca8640 1 -- 172.21.15.83:0/2613783539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91cafa060 con 0x55f91cbd4070 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.889+0000 7fb4fffff640 1 --2- 172.21.15.83:0/2613783539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd91e0 0x55f91cbdb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.890+0000 7fb504ca8640 1 --2- 172.21.15.83:0/2613783539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 0x55f91cbd8b70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009900 tx=0x7fb4f4005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d83d7341f3c8304 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.890+0000 7fb5054a9640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4f00a1a30 0x7fb4f00a3ef0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800dfd0 tx=0x7fb4f8004640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.890+0000 7fb4eaffd640 1 -- 172.21.15.83:0/2613783539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f402ca20 con 0x55f91cbd4070 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.890+0000 7fb4eaffd640 1 -- 172.21.15.83:0/2613783539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4f402cb80 con 0x55f91cbd4070 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.890+0000 7fb4eaffd640 1 -- 172.21.15.83:0/2613783539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4005150 con 0x55f91cbd4070 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.891+0000 7fb509bc4740 1 -- 172.21.15.83:0/2613783539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 msgr2=0x55f91cbd8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.891+0000 7fb509bc4740 1 --2- 172.21.15.83:0/2613783539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 0x55f91cbd8b70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009900 tx=0x7fb4f4005e00 comp rx=0 tx=0).stop 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.891+0000 7fb509bc4740 1 -- 172.21.15.83:0/2613783539 shutdown_connections 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.891+0000 7fb509bc4740 1 --2- 172.21.15.83:0/2613783539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd91e0 0x55f91cbdb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.891+0000 7fb509bc4740 1 --2- 172.21.15.83:0/2613783539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 0x55f91cbd8b70 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.891+0000 7fb509bc4740 1 --2- 172.21.15.83:0/2613783539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd3680 0x55f91cbd3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.891+0000 7fb509bc4740 1 -- 172.21.15.83:0/2613783539 >> 172.21.15.83:0/2613783539 conn(0x55f91cbca7e0 msgr2=0x55f91cbcabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.891+0000 7fb509bc4740 1 -- 172.21.15.83:0/2613783539 shutdown_connections 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.891+0000 7fb509bc4740 1 -- 172.21.15.83:0/2613783539 wait complete. 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.891+0000 7fb509bc4740 1 -- start start 2024-03-11T19:48:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.891+0000 7fb509bc4740 1 Processor -- start 2024-03-11T19:48:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd3680 0x55f91cbdd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd4070 0x55f91cbddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd91e0 0x55f91cbe4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb509bc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91cb38020 con 0x55f91cbd91e0 2024-03-11T19:48:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb509bc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91c80b430 con 0x55f91cbd4070 2024-03-11T19:48:22.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb509bc4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91c7c7e00 con 0x55f91cbd3680 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb509bc4740 10 client.0 _ll_register_callbacks cb 0x55f91cadfcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd91e0 0x55f91cbe4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd91e0 0x55f91cbe4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36920/0 (socket says 172.21.15.83:36920) 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4fffff640 1 -- 172.21.15.83:0/3913811835 learned_addr learned my addr 172.21.15.83:0/3913811835 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb504ca8640 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd4070 0x55f91cbddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4fffff640 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd3680 0x55f91cbdd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4fffff640 1 -- 172.21.15.83:0/3913811835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd3680 msgr2=0x55f91cbdd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4fffff640 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd3680 0x55f91cbdd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4fffff640 1 -- 172.21.15.83:0/3913811835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd4070 msgr2=0x55f91cbddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4fffff640 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd4070 0x55f91cbddd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4fffff640 1 -- 172.21.15.83:0/3913811835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb500108dc0 con 0x55f91cbd91e0 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4fffff640 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd91e0 0x55f91cbe4d50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec011e20 tx=0x7fb4ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a739b29cdd166f75 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4dffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:48:22.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4dffff640 1 -- 172.21.15.83:0/3913811835 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec016030 con 0x55f91cbd91e0 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4dffff640 1 -- 172.21.15.83:0/3913811835 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fb4ec00fe60 con 0x55f91cbd91e0 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb509bc4740 1 -- 172.21.15.83:0/3913811835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f91c8377e0 con 0x55f91cbd91e0 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.892+0000 7fb4dffff640 1 -- 172.21.15.83:0/3913811835 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4ec004220 con 0x55f91cbd91e0 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.893+0000 7fb509bc4740 10 client.17082 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.893+0000 7fb509bc4740 1 -- 172.21.15.83:0/3913811835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f91cb366a0 con 0x55f91cbd91e0 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.893+0000 7fb4dffff640 1 -- 172.21.15.83:0/3913811835 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4ec004380 con 0x55f91cbd91e0 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.893+0000 7fb4dffff640 1 -- 172.21.15.83:0/3913811835 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb4ec0053b0 con 0x55f91cbd91e0 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.893+0000 7fb509bc4740 10 client.17082 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.893+0000 7fb509bc4740 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.893+0000 7fb509bc4740 20 client.17082 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.893+0000 7fb509bc4740 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f91cbe9990 0x55f91cbebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.893+0000 7fb509bc4740 4 client.17082 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.893+0000 7fb509bc4740 1 -- 172.21.15.83:0/3913811835 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f91c8377e0 con 0x55f91cbe9990 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.894+0000 7fb504ca8640 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f91cbe9990 0x55f91cbebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.894+0000 7fb4dffff640 10 client.17082 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.894+0000 7fb504ca8640 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f91cbe9990 0x55f91cbebe10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b8373745815b3b4 server_cookie=ab2d7a77c0d349d1 in_seq=0 out_seq=0 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.895+0000 7fb4dffff640 1 -- 172.21.15.83:0/3913811835 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f91c8377e0 con 0x55f91cbe9990 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.895+0000 7fb4dffff640 10 client.17082 handle_command_reply: tid=0 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.895+0000 7fb509bc4740 1 client.17082 shutdown 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb509bc4740 1 -- 172.21.15.83:0/3913811835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd91e0 msgr2=0x55f91cbe4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb509bc4740 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd91e0 0x55f91cbe4d50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec011e20 tx=0x7fb4ec0046c0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb509bc4740 1 -- 172.21.15.83:0/3913811835 shutdown_connections 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb509bc4740 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f91cbe9990 0x55f91cbebe10 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb509bc4740 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd91e0 0x55f91cbe4d50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb509bc4740 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd4070 0x55f91cbddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb509bc4740 1 --2- 172.21.15.83:0/3913811835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd3680 0x55f91cbdd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb509bc4740 1 -- 172.21.15.83:0/3913811835 >> 172.21.15.83:0/3913811835 conn(0x55f91cbca7e0 msgr2=0x55f91cbc9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb4dffff640 0 client.17082 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb509bc4740 1 -- 172.21.15.83:0/3913811835 shutdown_connections 2024-03-11T19:48:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb509bc4740 1 -- 172.21.15.83:0/3913811835 wait complete. 2024-03-11T19:48:22.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.896+0000 7fb509bc4740 20 client.17082 trim_cache size 0 max 16384 2024-03-11T19:48:22.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.909+0000 7fb509bc4740 1 Processor -- start 2024-03-11T19:48:22.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.910+0000 7fb509bc4740 1 -- start start 2024-03-11T19:48:22.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.910+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd3680 0x55f91cbd55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.910+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd4070 0x55f91cbd5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.910+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd91e0 0x55f91cbe4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.910+0000 7fb509bc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91cabd130 con 0x55f91cbd3680 2024-03-11T19:48:22.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.910+0000 7fb509bc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91cadfcb0 con 0x55f91cbd91e0 2024-03-11T19:48:22.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.910+0000 7fb509bc4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91cbd3340 con 0x55f91cbd4070 2024-03-11T19:48:22.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.911+0000 7fb504ca8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd4070 0x55f91cbd5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.911+0000 7fb4fffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd91e0 0x55f91cbe4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.911+0000 7fb4fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd3680 0x55f91cbd55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.911+0000 7fb4fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd3680 0x55f91cbd55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36926/0 (socket says 172.21.15.83:36926) 2024-03-11T19:48:22.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.911+0000 7fb4fffff640 1 -- 172.21.15.83:0/1483472740 learned_addr learned my addr 172.21.15.83:0/1483472740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.911+0000 7fb504ca8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd4070 0x55f91cbd5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52040/0 (socket says 172.21.15.83:52040) 2024-03-11T19:48:22.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.912+0000 7fb4fffff640 1 -- 172.21.15.83:0/1483472740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd4070 msgr2=0x55f91cbd5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.912+0000 7fb4fffff640 1 --2- 172.21.15.83:0/1483472740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd4070 0x55f91cbd5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.912+0000 7fb4fffff640 1 -- 172.21.15.83:0/1483472740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd91e0 msgr2=0x55f91cbe4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.912+0000 7fb4fffff640 1 --2- 172.21.15.83:0/1483472740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd91e0 0x55f91cbe4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.912+0000 7fb4fffff640 1 -- 172.21.15.83:0/1483472740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91cad2c20 con 0x55f91cbd3680 2024-03-11T19:48:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.912+0000 7fb4fffff640 1 --2- 172.21.15.83:0/1483472740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd3680 0x55f91cbd55c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0056e0 tx=0x7fb4ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d730cb982d8c96 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.912+0000 7fb4ea7fc640 1 -- 172.21.15.83:0/1483472740 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec016070 con 0x55f91cbd3680 2024-03-11T19:48:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.912+0000 7fb4ea7fc640 1 -- 172.21.15.83:0/1483472740 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4ec005500 con 0x55f91cbd3680 2024-03-11T19:48:22.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.912+0000 7fb4ea7fc640 1 -- 172.21.15.83:0/1483472740 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec004b10 con 0x55f91cbd3680 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.913+0000 7fb509bc4740 1 -- 172.21.15.83:0/1483472740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd3680 msgr2=0x55f91cbd55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.913+0000 7fb509bc4740 1 --2- 172.21.15.83:0/1483472740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd3680 0x55f91cbd55c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0056e0 tx=0x7fb4ec000bc0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.913+0000 7fb509bc4740 1 -- 172.21.15.83:0/1483472740 shutdown_connections 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.913+0000 7fb509bc4740 1 --2- 172.21.15.83:0/1483472740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd91e0 0x55f91cbe4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.913+0000 7fb509bc4740 1 --2- 172.21.15.83:0/1483472740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd4070 0x55f91cbd5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.913+0000 7fb509bc4740 1 --2- 172.21.15.83:0/1483472740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd3680 0x55f91cbd55c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.913+0000 7fb509bc4740 1 -- 172.21.15.83:0/1483472740 >> 172.21.15.83:0/1483472740 conn(0x55f91cbca7e0 msgr2=0x55f91cbdd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.913+0000 7fb509bc4740 1 -- 172.21.15.83:0/1483472740 shutdown_connections 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.913+0000 7fb509bc4740 1 -- 172.21.15.83:0/1483472740 wait complete. 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.913+0000 7fb509bc4740 1 -- start start 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.913+0000 7fb509bc4740 1 Processor -- start 2024-03-11T19:48:22.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd3680 0x55f91cbe3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 0x55f91cbe3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb509bc4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd91e0 0x55f91cbe40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb509bc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91cbe51e0 con 0x55f91cbd4070 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb509bc4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91cc3ff50 con 0x55f91cbd3680 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb509bc4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91cc40110 con 0x55f91cbd91e0 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb509bc4740 10 client.0 _ll_register_callbacks cb 0x7fb4f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb4fffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd3680 0x55f91cbe3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb504ca8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 0x55f91cbe3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb4fffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd3680 0x55f91cbe3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47630/0 (socket says 172.21.15.83:47630) 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb4fffff640 1 -- 172.21.15.83:0/2355304766 learned_addr learned my addr 172.21.15.83:0/2355304766 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb4fffff640 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd91e0 0x55f91cbe40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb504ca8640 1 -- 172.21.15.83:0/2355304766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd91e0 msgr2=0x55f91cbe40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb504ca8640 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd91e0 0x55f91cbe40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb504ca8640 1 -- 172.21.15.83:0/2355304766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd3680 msgr2=0x55f91cbe3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb504ca8640 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd3680 0x55f91cbe3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.914+0000 7fb504ca8640 1 -- 172.21.15.83:0/2355304766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91cb366a0 con 0x55f91cbd4070 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb504ca8640 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 0x55f91cbe3bb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55f91cbcb750 tx=0x7fb4f4005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f10839bae20a9ad server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb4dffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb4dffff640 1 -- 172.21.15.83:0/2355304766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f402c8e0 con 0x55f91cbd4070 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb4dffff640 1 -- 172.21.15.83:0/2355304766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fb4f404e050 con 0x55f91cbd4070 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb509bc4740 1 -- 172.21.15.83:0/2355304766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4f402cb80 con 0x55f91cbd4070 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb4dffff640 1 -- 172.21.15.83:0/2355304766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4f4005890 con 0x55f91cbd4070 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb509bc4740 10 client.17088 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb509bc4740 1 -- 172.21.15.83:0/2355304766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f91caf88c0 con 0x55f91cbd4070 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb4dffff640 1 -- 172.21.15.83:0/2355304766 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4f402cb80 con 0x55f91cbd4070 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb4dffff640 1 -- 172.21.15.83:0/2355304766 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb4f4056070 con 0x55f91cbd4070 2024-03-11T19:48:22.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb509bc4740 10 client.17088 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb509bc4740 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb509bc4740 20 client.17088 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb509bc4740 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f91cc41800 0x55f91cc43c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb509bc4740 4 client.17088 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb509bc4740 1 -- 172.21.15.83:0/2355304766 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f91c874720 con 0x55f91cc41800 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.915+0000 7fb4fffff640 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f91cc41800 0x55f91cc43c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb4dffff640 10 client.17088 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb4fffff640 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f91cc41800 0x55f91cc43c20 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce41571f0e7b7004 server_cookie=87780ab516300dd2 in_seq=0 out_seq=0 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb4dffff640 1 -- 172.21.15.83:0/2355304766 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f91c874720 con 0x55f91cc41800 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb4dffff640 10 client.17088 handle_command_reply: tid=0 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 1 client.17088 shutdown 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 1 -- 172.21.15.83:0/2355304766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 msgr2=0x55f91cbe3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 0x55f91cbe3bb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x55f91cbcb750 tx=0x7fb4f4005e00 comp rx=0 tx=0).stop 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 1 -- 172.21.15.83:0/2355304766 shutdown_connections 2024-03-11T19:48:22.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f91cc41800 0x55f91cc43c20 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f91cbd91e0 0x55f91cbe40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb4dffff640 0 client.17088 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f91cbd4070 0x55f91cbe3bb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 1 --2- 172.21.15.83:0/2355304766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91cbd3680 0x55f91cbe3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 1 -- 172.21.15.83:0/2355304766 >> 172.21.15.83:0/2355304766 conn(0x55f91cbca7e0 msgr2=0x55f91cbdd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 1 -- 172.21.15.83:0/2355304766 shutdown_connections 2024-03-11T19:48:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 1 -- 172.21.15.83:0/2355304766 wait complete. 2024-03-11T19:48:22.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.916+0000 7fb509bc4740 20 client.17088 trim_cache size 0 max 16384 2024-03-11T19:48:22.925 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:48:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 -- 172.21.15.83:0/419749157 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4f00a1a30 msgr2=0x7fb4f00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4f00a1a30 0x7fb4f00a3ef0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800dfd0 tx=0x7fb4f8004640 comp rx=0 tx=0).stop 2024-03-11T19:48:22.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 -- 172.21.15.83:0/419749157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb500101c30 msgr2=0x7fb50019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb500101c30 0x7fb50019e600 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009fd0 tx=0x7fb4f800b260 comp rx=0 tx=0).stop 2024-03-11T19:48:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 -- 172.21.15.83:0/419749157 shutdown_connections 2024-03-11T19:48:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb4f00a1a30 0x7fb4f00a3ef0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb500101c30 0x7fb50019e600 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb500101270 0x7fb50019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 --2- 172.21.15.83:0/419749157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb500100080 0x7fb50006a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 -- 172.21.15.83:0/419749157 >> 172.21.15.83:0/419749157 conn(0x7fb5000fb5f0 msgr2=0x7fb500104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 -- 172.21.15.83:0/419749157 shutdown_connections 2024-03-11T19:48:22.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:22.917+0000 7fb506f0c640 1 -- 172.21.15.83:0/419749157 wait complete. 2024-03-11T19:48:23.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:23 smithi137 ceph-mon[36220]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 722 op/s 2024-03-11T19:48:23.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:23 smithi204 ceph-mon[36739]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 722 op/s 2024-03-11T19:48:23.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:23 smithi083 ceph-mon[31864]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 722 op/s 2024-03-11T19:48:23.935 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:48:23.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:48:23.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:48:24.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.064+0000 7f3db6bd0640 1 -- 172.21.15.83:0/1450868999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3db0101c90 msgr2=0x7f3db0104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.064+0000 7f3db6bd0640 1 --2- 172.21.15.83:0/1450868999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3db0101c90 0x7f3db0104080 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009900 tx=0x7f3dac02bc30 comp rx=0 tx=0).stop 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.064+0000 7f3db6bd0640 1 -- 172.21.15.83:0/1450868999 shutdown_connections 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.064+0000 7f3db6bd0640 1 --2- 172.21.15.83:0/1450868999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db0106ef0 0x7f3db01092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.064+0000 7f3db6bd0640 1 --2- 172.21.15.83:0/1450868999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3db01045c0 0x7f3db01069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.064+0000 7f3db6bd0640 1 --2- 172.21.15.83:0/1450868999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3db0101c90 0x7f3db0104080 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.064+0000 7f3db6bd0640 1 -- 172.21.15.83:0/1450868999 >> 172.21.15.83:0/1450868999 conn(0x7f3db00fb620 msgr2=0x7f3db00fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.064+0000 7f3db6bd0640 1 -- 172.21.15.83:0/1450868999 shutdown_connections 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.064+0000 7f3db6bd0640 1 -- 172.21.15.83:0/1450868999 wait complete. 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.065+0000 7f3db6bd0640 1 Processor -- start 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.065+0000 7f3db6bd0640 1 -- start start 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.065+0000 7f3db6bd0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db0101c90 0x7f3db0197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.065+0000 7f3db6bd0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3db01045c0 0x7f3db01977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.065+0000 7f3db6bd0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3db0106ef0 0x7f3db019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.065+0000 7f3db6bd0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db019ed70 con 0x7f3db01045c0 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.065+0000 7f3db6bd0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db019eee0 con 0x7f3db0106ef0 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.065+0000 7f3db6bd0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db019f0a0 con 0x7f3db0101c90 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.065+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3db01045c0 0x7f3db01977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.065+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3db01045c0 0x7f3db01977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36954/0 (socket says 172.21.15.83:36954) 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.065+0000 7f3da7fff640 1 -- 172.21.15.83:0/4168200699 learned_addr learned my addr 172.21.15.83:0/4168200699 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3db516d640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3db0106ef0 0x7f3db019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3db496c640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db0101c90 0x7f3db0197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3da7fff640 1 -- 172.21.15.83:0/4168200699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db0101c90 msgr2=0x7f3db0197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3da7fff640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db0101c90 0x7f3db0197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3da7fff640 1 -- 172.21.15.83:0/4168200699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3db0106ef0 msgr2=0x7f3db019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3da7fff640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3db0106ef0 0x7f3db019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3da7fff640 1 -- 172.21.15.83:0/4168200699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac009560 con 0x7f3db01045c0 2024-03-11T19:48:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3da7fff640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3db01045c0 0x7f3db01977c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3da000d770 tx=0x7f3da000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3da5ffb640 1 -- 172.21.15.83:0/4168200699 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0009c30 con 0x7f3db01045c0 2024-03-11T19:48:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3db6bd0640 1 -- 172.21.15.83:0/4168200699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db00ffae0 con 0x7f3db01045c0 2024-03-11T19:48:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3da5ffb640 1 -- 172.21.15.83:0/4168200699 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3da000de80 con 0x7f3db01045c0 2024-03-11T19:48:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.066+0000 7f3da5ffb640 1 -- 172.21.15.83:0/4168200699 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da000f6b0 con 0x7f3db01045c0 2024-03-11T19:48:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.067+0000 7f3db6bd0640 1 -- 172.21.15.83:0/4168200699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db00ffff0 con 0x7f3db01045c0 2024-03-11T19:48:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.068+0000 7f3da5ffb640 1 -- 172.21.15.83:0/4168200699 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f3da0010460 con 0x7f3db01045c0 2024-03-11T19:48:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.069+0000 7f3db97a1740 1 Processor -- start 2024-03-11T19:48:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.069+0000 7f3db97a1740 1 -- start start 2024-03-11T19:48:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.069+0000 7f3da5ffb640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3d980a1a30 0x7f3d980a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.069+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393d4c0 0x55756393d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.069+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x55756393eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.069+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 0x5575639458e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.069+0000 7f3db97a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575638acf90 con 0x55756393f050 2024-03-11T19:48:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.069+0000 7f3db97a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575638aa4c0 con 0x55756393d4c0 2024-03-11T19:48:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.069+0000 7f3db97a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575638a5ca0 con 0x55756393e690 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.069+0000 7f3da5ffb640 1 -- 172.21.15.83:0/4168200699 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3da00cd050 con 0x7f3db01045c0 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.069+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x55756393eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.070+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x55756393eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52100/0 (socket says 172.21.15.83:52100) 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.070+0000 7f3da7fff640 1 -- 172.21.15.83:0/1116942800 learned_addr learned my addr 172.21.15.83:0/1116942800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.070+0000 7f3db516d640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3d980a1a30 0x7f3d980a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.070+0000 7f3db496c640 1 --2- 172.21.15.83:0/1116942800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393d4c0 0x55756393d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.070+0000 7f3db516d640 1 --2- 172.21.15.83:0/1116942800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 0x5575639458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.070+0000 7f3da7fff640 1 -- 172.21.15.83:0/1116942800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393d4c0 msgr2=0x55756393d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.070+0000 7f3da7fff640 1 --2- 172.21.15.83:0/1116942800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393d4c0 0x55756393d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.070+0000 7f3da7fff640 1 -- 172.21.15.83:0/1116942800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 msgr2=0x5575639458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.070+0000 7f3da7fff640 1 --2- 172.21.15.83:0/1116942800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 0x5575639458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.070+0000 7f3da7fff640 1 -- 172.21.15.83:0/1116942800 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575638680b0 con 0x55756393e690 2024-03-11T19:48:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.070+0000 7f3db516d640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3d980a1a30 0x7f3d980a3ef0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3da8005fd0 tx=0x7f3da8009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.071+0000 7f3da7fff640 1 --2- 172.21.15.83:0/1116942800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x55756393eb10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3da00a22e0 tx=0x7f3da0099a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b09ff49ce91385 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.071+0000 7f3d96ffd640 1 -- 172.21.15.83:0/1116942800 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0011f30 con 0x55756393e690 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.071+0000 7f3d96ffd640 1 -- 172.21.15.83:0/1116942800 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3da00124d0 con 0x55756393e690 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.071+0000 7f3d96ffd640 1 -- 172.21.15.83:0/1116942800 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0098420 con 0x55756393e690 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.072+0000 7f3db97a1740 1 -- 172.21.15.83:0/1116942800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 msgr2=0x55756393eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.072+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1116942800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x55756393eb10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3da00a22e0 tx=0x7f3da0099a80 comp rx=0 tx=0).stop 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.072+0000 7f3db97a1740 1 -- 172.21.15.83:0/1116942800 shutdown_connections 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.072+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1116942800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 0x5575639458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.072+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1116942800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x55756393eb10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.072+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1116942800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393d4c0 0x55756393d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.072+0000 7f3db97a1740 1 -- 172.21.15.83:0/1116942800 >> 172.21.15.83:0/1116942800 conn(0x5575639386a0 msgr2=0x557563938aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.072+0000 7f3db97a1740 1 -- 172.21.15.83:0/1116942800 shutdown_connections 2024-03-11T19:48:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.072+0000 7f3db97a1740 1 -- 172.21.15.83:0/1116942800 wait complete. 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.072+0000 7f3db97a1740 1 -- start start 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db97a1740 1 Processor -- start 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393d4c0 0x557563947790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x557563947cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393f050 0x55756394ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db97a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756384af40 con 0x55756393d4c0 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db97a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575635794b0 con 0x55756393f050 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db97a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756382b670 con 0x55756393e690 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db97a1740 10 client.0 _ll_register_callbacks cb 0x557563898f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db496c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393d4c0 0x557563947790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x557563947cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db496c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393d4c0 0x557563947790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36980/0 (socket says 172.21.15.83:36980) 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db496c640 1 -- 172.21.15.83:0/1623952727 learned_addr learned my addr 172.21.15.83:0/1623952727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:24.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db516d640 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393f050 0x55756394ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db496c640 1 -- 172.21.15.83:0/1623952727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 msgr2=0x557563947cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db496c640 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x557563947cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db496c640 1 -- 172.21.15.83:0/1623952727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393f050 msgr2=0x55756394ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db496c640 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393f050 0x55756394ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3db496c640 1 -- 172.21.15.83:0/1623952727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db00ffff0 con 0x55756393d4c0 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.073+0000 7f3da7fff640 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x557563947cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3db496c640 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393d4c0 0x557563947790 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3dac02c170 tx=0x7f3dac0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feda5acd67993f74 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3d8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1623952727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac004240 con 0x55756393d4c0 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1623952727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3dac042070 con 0x55756393d4c0 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3db97a1740 1 -- 172.21.15.83:0/1623952727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55756387ff40 con 0x55756393d4c0 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1623952727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dac04c030 con 0x55756393d4c0 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3db97a1740 10 client.17100 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:24.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3db97a1740 1 -- 172.21.15.83:0/1623952727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5575638a4320 con 0x55756393d4c0 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1623952727 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55756387ff40 con 0x55756393d4c0 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1623952727 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3dac03c040 con 0x55756393d4c0 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3db97a1740 10 client.17100 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3db97a1740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3db97a1740 20 client.17100 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557563953990 0x557563955dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3db97a1740 4 client.17100 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3db97a1740 1 -- 172.21.15.83:0/1623952727 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55756389c310 con 0x557563953990 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.074+0000 7f3da7fff640 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557563953990 0x557563955dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.075+0000 7f3d8ffff640 10 client.17100 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.075+0000 7f3da7fff640 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557563953990 0x557563955dd0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0288fbdad4b4dc1 server_cookie=3c34f8fc82b9ef8b in_seq=0 out_seq=0 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.075+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1623952727 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55756389c310 con 0x557563953990 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.075+0000 7f3d8ffff640 10 client.17100 handle_command_reply: tid=0 2024-03-11T19:48:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 1 client.17100 shutdown 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 1 -- 172.21.15.83:0/1623952727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393d4c0 msgr2=0x557563947790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393d4c0 0x557563947790 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3dac02c170 tx=0x7f3dac0048f0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 1 -- 172.21.15.83:0/1623952727 shutdown_connections 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557563953990 0x557563955dd0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393f050 0x55756394ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3d8ffff640 0 client.17100 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x557563947cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1623952727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393d4c0 0x557563947790 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 1 -- 172.21.15.83:0/1623952727 >> 172.21.15.83:0/1623952727 conn(0x5575639386a0 msgr2=0x5575639378e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 1 -- 172.21.15.83:0/1623952727 shutdown_connections 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 1 -- 172.21.15.83:0/1623952727 wait complete. 2024-03-11T19:48:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.076+0000 7f3db97a1740 20 client.17100 trim_cache size 0 max 16384 2024-03-11T19:48:24.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.095+0000 7f3db97a1740 1 Processor -- start 2024-03-11T19:48:24.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.095+0000 7f3db97a1740 1 -- start start 2024-03-11T19:48:24.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.095+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393d4c0 0x5575639422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.095+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x557563942820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.095+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393f050 0x55756394ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.095+0000 7f3db97a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756389c310 con 0x55756393d4c0 2024-03-11T19:48:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.095+0000 7f3db97a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756387ff40 con 0x55756393f050 2024-03-11T19:48:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.095+0000 7f3db97a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756393e160 con 0x55756393e690 2024-03-11T19:48:24.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.096+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x557563942820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.096+0000 7f3da7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x557563942820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52124/0 (socket says 172.21.15.83:52124) 2024-03-11T19:48:24.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.096+0000 7f3da7fff640 1 -- 172.21.15.83:0/692301422 learned_addr learned my addr 172.21.15.83:0/692301422 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.103+0000 7f3da7fff640 1 -- 172.21.15.83:0/692301422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393f050 msgr2=0x55756394ecd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:48:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.103+0000 7f3da7fff640 1 --2- 172.21.15.83:0/692301422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393f050 0x55756394ecd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.103+0000 7f3da7fff640 1 -- 172.21.15.83:0/692301422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393d4c0 msgr2=0x5575639422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.103+0000 7f3da7fff640 1 --2- 172.21.15.83:0/692301422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393d4c0 0x5575639422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.103+0000 7f3da7fff640 1 -- 172.21.15.83:0/692301422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557563840c00 con 0x55756393e690 2024-03-11T19:48:24.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.108+0000 7f3da7fff640 1 --2- 172.21.15.83:0/692301422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x557563942820 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3da0096580 tx=0x7f3da0009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11441a2d57152d04 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.111+0000 7f3d967fc640 1 -- 172.21.15.83:0/692301422 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da00b0070 con 0x55756393e690 2024-03-11T19:48:24.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.111+0000 7f3d967fc640 1 -- 172.21.15.83:0/692301422 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3da00b2040 con 0x55756393e690 2024-03-11T19:48:24.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.111+0000 7f3d967fc640 1 -- 172.21.15.83:0/692301422 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0009d90 con 0x55756393e690 2024-03-11T19:48:24.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.112+0000 7f3db97a1740 1 -- 172.21.15.83:0/692301422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 msgr2=0x557563942820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.112+0000 7f3db97a1740 1 --2- 172.21.15.83:0/692301422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x557563942820 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3da0096580 tx=0x7f3da0009740 comp rx=0 tx=0).stop 2024-03-11T19:48:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.122+0000 7f3db97a1740 1 -- 172.21.15.83:0/692301422 shutdown_connections 2024-03-11T19:48:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.122+0000 7f3db97a1740 1 --2- 172.21.15.83:0/692301422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393f050 0x55756394ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.122+0000 7f3db97a1740 1 --2- 172.21.15.83:0/692301422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393e690 0x557563942820 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.122+0000 7f3db97a1740 1 --2- 172.21.15.83:0/692301422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393d4c0 0x5575639422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.122+0000 7f3db97a1740 1 -- 172.21.15.83:0/692301422 >> 172.21.15.83:0/692301422 conn(0x5575639386a0 msgr2=0x55756395a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.124+0000 7f3db97a1740 1 -- 172.21.15.83:0/692301422 shutdown_connections 2024-03-11T19:48:24.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.125+0000 7f3db97a1740 1 -- 172.21.15.83:0/692301422 wait complete. 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.125+0000 7f3db97a1740 1 -- start start 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.125+0000 7f3db97a1740 1 Processor -- start 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.126+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393d4c0 0x55756394aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.126+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393e690 0x55756394b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.126+0000 7f3db97a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 0x55756394e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.126+0000 7f3db97a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756394e990 con 0x55756393f050 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.126+0000 7f3db97a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756394eb00 con 0x55756393e690 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.126+0000 7f3db97a1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756394ecc0 con 0x55756393d4c0 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.126+0000 7f3db97a1740 10 client.0 _ll_register_callbacks cb 0x7f3da0010cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.127+0000 7f3db516d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 0x55756394e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.127+0000 7f3db516d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 0x55756394e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36998/0 (socket says 172.21.15.83:36998) 2024-03-11T19:48:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.127+0000 7f3db516d640 1 -- 172.21.15.83:0/1010039834 learned_addr learned my addr 172.21.15.83:0/1010039834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.127+0000 7f3db516d640 1 -- 172.21.15.83:0/1010039834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393d4c0 msgr2=0x55756394aef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:48:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.127+0000 7f3db516d640 1 --2- 172.21.15.83:0/1010039834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393d4c0 0x55756394aef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.127+0000 7f3db516d640 1 -- 172.21.15.83:0/1010039834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393e690 msgr2=0x55756394b430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:48:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.127+0000 7f3db516d640 1 --2- 172.21.15.83:0/1010039834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393e690 0x55756394b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.127+0000 7f3db516d640 1 -- 172.21.15.83:0/1010039834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575638680b0 con 0x55756393f050 2024-03-11T19:48:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.128+0000 7f3db516d640 1 --2- 172.21.15.83:0/1010039834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 0x55756394e450 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3da8005e10 tx=0x7f3da8025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181718d048603822 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.128+0000 7f3d8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:48:24.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.128+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1010039834 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8016050 con 0x55756393f050 2024-03-11T19:48:24.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.128+0000 7f3db97a1740 1 -- 172.21.15.83:0/1010039834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575638cb900 con 0x55756393f050 2024-03-11T19:48:24.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.128+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1010039834 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f3da8004d20 con 0x55756393f050 2024-03-11T19:48:24.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.129+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1010039834 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3da801e740 con 0x55756393f050 2024-03-11T19:48:24.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.129+0000 7f3db97a1740 10 client.17103 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:24.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.129+0000 7f3db97a1740 1 -- 172.21.15.83:0/1010039834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557563866910 con 0x55756393f050 2024-03-11T19:48:24.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.129+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1010039834 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5575638cb900 con 0x55756393f050 2024-03-11T19:48:24.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.129+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1010039834 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3da801d5e0 con 0x55756393f050 2024-03-11T19:48:24.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.130+0000 7f3db97a1740 10 client.17103 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:24.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.130+0000 7f3db97a1740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:24.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.130+0000 7f3db97a1740 20 client.17103 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:24.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.130+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1010039834 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5575639ab3a0 0x5575639ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.130+0000 7f3db97a1740 4 client.17103 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:48:24.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.130+0000 7f3db97a1740 1 -- 172.21.15.83:0/1010039834 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5575635a5860 con 0x5575639ab3a0 2024-03-11T19:48:24.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.131+0000 7f3db496c640 1 --2- 172.21.15.83:0/1010039834 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5575639ab3a0 0x5575639ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.133+0000 7f3db496c640 1 --2- 172.21.15.83:0/1010039834 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5575639ab3a0 0x5575639ad7c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8983394966418458 server_cookie=dae600a278d2c77f in_seq=0 out_seq=0 2024-03-11T19:48:24.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.133+0000 7f3d8ffff640 10 client.17103 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:24.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.134+0000 7f3d8ffff640 1 -- 172.21.15.83:0/1010039834 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5575635a5860 con 0x5575639ab3a0 2024-03-11T19:48:24.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.134+0000 7f3d8ffff640 10 client.17103 handle_command_reply: tid=0 2024-03-11T19:48:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.134+0000 7f3db97a1740 1 client.17103 shutdown 2024-03-11T19:48:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.136+0000 7f3db97a1740 1 -- 172.21.15.83:0/1010039834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 msgr2=0x55756394e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.136+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1010039834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 0x55756394e450 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3da8005e10 tx=0x7f3da8025000 comp rx=0 tx=0).stop 2024-03-11T19:48:24.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.137+0000 7f3db97a1740 1 -- 172.21.15.83:0/1010039834 shutdown_connections 2024-03-11T19:48:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.137+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1010039834 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5575639ab3a0 0x5575639ad7c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.137+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1010039834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55756393f050 0x55756394e450 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.137+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1010039834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55756393e690 0x55756394b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.137+0000 7f3db97a1740 1 --2- 172.21.15.83:0/1010039834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55756393d4c0 0x55756394aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.137+0000 7f3db97a1740 1 -- 172.21.15.83:0/1010039834 >> 172.21.15.83:0/1010039834 conn(0x5575639386a0 msgr2=0x5575639418c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.137+0000 7f3d8ffff640 0 client.17103 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.137+0000 7f3db97a1740 1 -- 172.21.15.83:0/1010039834 shutdown_connections 2024-03-11T19:48:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.137+0000 7f3db97a1740 1 -- 172.21.15.83:0/1010039834 wait complete. 2024-03-11T19:48:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.137+0000 7f3db97a1740 20 client.17103 trim_cache size 0 max 16384 2024-03-11T19:48:24.142 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:48:24.142 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:48:24.143 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "afca1676-17b5-4322-a467-bac539244f89", 2024-03-11T19:48:24.143 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:48:24.143 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:48:24.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.139+0000 7f3db6bd0640 1 -- 172.21.15.83:0/4168200699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3d980a1a30 msgr2=0x7f3d980a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.139+0000 7f3db6bd0640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3d980a1a30 0x7f3d980a3ef0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3da8005fd0 tx=0x7f3da8009f10 comp rx=0 tx=0).stop 2024-03-11T19:48:24.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.139+0000 7f3db6bd0640 1 -- 172.21.15.83:0/4168200699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3db01045c0 msgr2=0x7f3db01977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.139+0000 7f3db6bd0640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3db01045c0 0x7f3db01977c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3da000d770 tx=0x7f3da000dc30 comp rx=0 tx=0).stop 2024-03-11T19:48:24.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.139+0000 7f3db6bd0640 1 -- 172.21.15.83:0/4168200699 shutdown_connections 2024-03-11T19:48:24.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.139+0000 7f3db6bd0640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f3d980a1a30 0x7f3d980a3ef0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.139+0000 7f3db6bd0640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3db0106ef0 0x7f3db019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.139+0000 7f3db6bd0640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3db01045c0 0x7f3db01977c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.140+0000 7f3db6bd0640 1 --2- 172.21.15.83:0/4168200699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db0101c90 0x7f3db0197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.140+0000 7f3db6bd0640 1 -- 172.21.15.83:0/4168200699 >> 172.21.15.83:0/4168200699 conn(0x7f3db00fb620 msgr2=0x7f3db0102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.141+0000 7f3db6bd0640 1 -- 172.21.15.83:0/4168200699 shutdown_connections 2024-03-11T19:48:24.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.141+0000 7f3db6bd0640 1 -- 172.21.15.83:0/4168200699 wait complete. 2024-03-11T19:48:24.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:48:24.160 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:48:24.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.289+0000 7f9cf85d7640 1 -- 172.21.15.83:0/1451558274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf0101ca0 msgr2=0x7f9cf0104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.289+0000 7f9cf85d7640 1 --2- 172.21.15.83:0/1451558274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf0101ca0 0x7f9cf0104090 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009900 tx=0x7f9cec02bc30 comp rx=0 tx=0).stop 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.289+0000 7f9cf85d7640 1 -- 172.21.15.83:0/1451558274 shutdown_connections 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.289+0000 7f9cf85d7640 1 --2- 172.21.15.83:0/1451558274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf0106f00 0x7f9cf01092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.289+0000 7f9cf85d7640 1 --2- 172.21.15.83:0/1451558274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cf01045d0 0x7f9cf01069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.289+0000 7f9cf85d7640 1 --2- 172.21.15.83:0/1451558274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf0101ca0 0x7f9cf0104090 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.289+0000 7f9cf85d7640 1 -- 172.21.15.83:0/1451558274 >> 172.21.15.83:0/1451558274 conn(0x7f9cf00fb610 msgr2=0x7f9cf00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.289+0000 7f9cf85d7640 1 -- 172.21.15.83:0/1451558274 shutdown_connections 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.289+0000 7f9cf85d7640 1 -- 172.21.15.83:0/1451558274 wait complete. 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf85d7640 1 Processor -- start 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf85d7640 1 -- start start 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf85d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf0101ca0 0x7f9cf0101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf85d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf01045d0 0x7f9cf00ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf85d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cf0106f00 0x7f9cf0100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf85d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf01008f0 con 0x7f9cf0101ca0 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf85d7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0100a60 con 0x7f9cf0106f00 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf85d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf01a1290 con 0x7f9cf01045d0 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf6373640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf0101ca0 0x7f9cf0101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf6373640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf0101ca0 0x7f9cf0101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37010/0 (socket says 172.21.15.83:37010) 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf6373640 1 -- 172.21.15.83:0/2145288458 learned_addr learned my addr 172.21.15.83:0/2145288458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf5b72640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf01045d0 0x7f9cf00ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.290+0000 7f9cf6b74640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cf0106f00 0x7f9cf0100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9cf6373640 1 -- 172.21.15.83:0/2145288458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf01045d0 msgr2=0x7f9cf00ffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9cf6373640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf01045d0 0x7f9cf00ffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9cf6373640 1 -- 172.21.15.83:0/2145288458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cf0106f00 msgr2=0x7f9cf0100190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9cf6373640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cf0106f00 0x7f9cf0100190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9cf6373640 1 -- 172.21.15.83:0/2145288458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec009560 con 0x7f9cf0101ca0 2024-03-11T19:48:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9cf5b72640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf01045d0 0x7f9cf00ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9cf6373640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf0101ca0 0x7f9cf0101610 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9cec036040 tx=0x7f9cec005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9ce77fe640 1 -- 172.21.15.83:0/2145288458 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0043d0 con 0x7f9cf0101ca0 2024-03-11T19:48:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9ce77fe640 1 -- 172.21.15.83:0/2145288458 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cec0052d0 con 0x7f9cf0101ca0 2024-03-11T19:48:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9ce77fe640 1 -- 172.21.15.83:0/2145288458 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec003cc0 con 0x7f9cf0101ca0 2024-03-11T19:48:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9cf85d7640 1 -- 172.21.15.83:0/2145288458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf01a14e0 con 0x7f9cf0101ca0 2024-03-11T19:48:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.291+0000 7f9cf85d7640 1 -- 172.21.15.83:0/2145288458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf01a19a0 con 0x7f9cf0101ca0 2024-03-11T19:48:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.294+0000 7f9ce77fe640 1 -- 172.21.15.83:0/2145288458 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9cec005440 con 0x7f9cf0101ca0 2024-03-11T19:48:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cfb1a9740 1 Processor -- start 2024-03-11T19:48:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cfb1a9740 1 -- start start 2024-03-11T19:48:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9ce77fe640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9cdc0a1b80 0x7f9cdc0a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cfb1a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cfb1a9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 0x55a5058cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cfb1a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cfe30 0x55a5058d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cfb1a9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a50583a280 con 0x55a5058cacc0 2024-03-11T19:48:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cfb1a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5058377b0 con 0x55a5058cfe30 2024-03-11T19:48:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cfb1a9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a505832f90 con 0x55a5058ca2d0 2024-03-11T19:48:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cf5b72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 0x55a5058cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cf6b74640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9cdc0a1b80 0x7f9cdc0a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cf5b72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 0x55a5058cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37014/0 (socket says 172.21.15.83:37014) 2024-03-11T19:48:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cf5b72640 1 -- 172.21.15.83:0/3036617275 learned_addr learned my addr 172.21.15.83:0/3036617275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cf6373640 1 --2- 172.21.15.83:0/3036617275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9cf6b74640 1 --2- 172.21.15.83:0/3036617275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cfe30 0x55a5058d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.295+0000 7f9ce77fe640 1 -- 172.21.15.83:0/2145288458 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9cec0ef170 con 0x7f9cf0101ca0 2024-03-11T19:48:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.296+0000 7f9cf5b72640 1 -- 172.21.15.83:0/3036617275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 msgr2=0x55a5058ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.296+0000 7f9cf5b72640 1 --2- 172.21.15.83:0/3036617275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.296+0000 7f9cf5b72640 1 -- 172.21.15.83:0/3036617275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cfe30 msgr2=0x55a5058d22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.296+0000 7f9cf5b72640 1 --2- 172.21.15.83:0/3036617275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cfe30 0x55a5058d22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.296+0000 7f9cf5b72640 1 -- 172.21.15.83:0/3036617275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5057f1570 con 0x55a5058cacc0 2024-03-11T19:48:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.296+0000 7f9cf5b72640 1 --2- 172.21.15.83:0/3036617275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 0x55a5058cf7c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00114f0 tx=0x7f9ce00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c195757ef057091f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.296+0000 7f9cf6b74640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9cdc0a1b80 0x7f9cdc0a4040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8005fd0 tx=0x7f9ce8009e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.296+0000 7f9ce4bfe640 1 -- 172.21.15.83:0/3036617275 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0011310 con 0x55a5058cacc0 2024-03-11T19:48:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.296+0000 7f9ce4bfe640 1 -- 172.21.15.83:0/3036617275 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ce001ae50 con 0x55a5058cacc0 2024-03-11T19:48:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.296+0000 7f9ce4bfe640 1 -- 172.21.15.83:0/3036617275 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0004a00 con 0x55a5058cacc0 2024-03-11T19:48:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.297+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/3036617275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 msgr2=0x55a5058cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.297+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/3036617275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 0x55a5058cf7c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00114f0 tx=0x7f9ce00119b0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.297+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/3036617275 shutdown_connections 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.297+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/3036617275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cfe30 0x55a5058d22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.297+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/3036617275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 0x55a5058cf7c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.297+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/3036617275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.297+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/3036617275 >> 172.21.15.83:0/3036617275 conn(0x55a5058c57f0 msgr2=0x55a5058c5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.298+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/3036617275 shutdown_connections 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.298+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/3036617275 wait complete. 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.298+0000 7f9cfb1a9740 1 -- start start 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.298+0000 7f9cfb1a9740 1 Processor -- start 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.299+0000 7f9cfb1a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.299+0000 7f9cfb1a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cacc0 0x55a5058d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.299+0000 7f9cfb1a9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cfe30 0x55a5058db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.299+0000 7f9cf5b72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cacc0 0x55a5058d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.299+0000 7f9cf5b72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cacc0 0x55a5058d47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47728/0 (socket says 172.21.15.83:47728) 2024-03-11T19:48:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.299+0000 7f9cf5b72640 1 -- 172.21.15.83:0/413525582 learned_addr learned my addr 172.21.15.83:0/413525582 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.299+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/413525582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a505506430 con 0x55a5058cfe30 2024-03-11T19:48:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.299+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/413525582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5054c2e00 con 0x55a5058cacc0 2024-03-11T19:48:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.299+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/413525582 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5057b7fd0 con 0x55a5058ca2d0 2024-03-11T19:48:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.300+0000 7f9cfb1a9740 10 client.0 _ll_register_callbacks cb 0x55a50580dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.304+0000 7f9cf5b72640 1 -- 172.21.15.83:0/413525582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 msgr2=0x55a5058d4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.304+0000 7f9cf5b72640 1 --2- 172.21.15.83:0/413525582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.304+0000 7f9cf5b72640 1 -- 172.21.15.83:0/413525582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cfe30 msgr2=0x55a5058db7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.304+0000 7f9cf5b72640 1 --2- 172.21.15.83:0/413525582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cfe30 0x55a5058db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.304+0000 7f9cf5b72640 1 -- 172.21.15.83:0/413525582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce0009c80 con 0x55a5058cacc0 2024-03-11T19:48:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.305+0000 7f9cf5b72640 1 --2- 172.21.15.83:0/413525582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cacc0 0x55a5058d47b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0011eb0 tx=0x7f9ce001ac10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e278876824251680 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.305+0000 7f9cd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:48:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.310+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/413525582 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000f970 con 0x55a5058cacc0 2024-03-11T19:48:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.310+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/413525582 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9ce000fad0 con 0x55a5058cacc0 2024-03-11T19:48:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.310+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/413525582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5055327e0 con 0x55a5058cacc0 2024-03-11T19:48:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.310+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/413525582 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ce000fc90 con 0x55a5058cacc0 2024-03-11T19:48:24.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.315+0000 7f9cfb1a9740 10 client.26564 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:24.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.315+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/413525582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a505831610 con 0x55a5058cacc0 2024-03-11T19:48:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.316+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/413525582 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ce000fdf0 con 0x55a5058cacc0 2024-03-11T19:48:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.317+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/413525582 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9ce002d620 con 0x55a5058cacc0 2024-03-11T19:48:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.318+0000 7f9cfb1a9740 10 client.26564 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.318+0000 7f9cfb1a9740 10 client.26564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.318+0000 7f9cfb1a9740 20 client.26564 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.318+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/413525582 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a5058e0370 0x55a5058e2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.318+0000 7f9cfb1a9740 4 client.26564 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:24.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.318+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/413525582 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5055327e0 con 0x55a5058e0370 2024-03-11T19:48:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.322+0000 7f9cf6373640 1 --2- 172.21.15.83:0/413525582 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a5058e0370 0x55a5058e2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.329+0000 7f9cd9ffb640 10 client.26564 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.329+0000 7f9cf6373640 1 --2- 172.21.15.83:0/413525582 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a5058e0370 0x55a5058e2830 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=215cb14d13864cc7 server_cookie=428a34e57505c40b in_seq=0 out_seq=0 2024-03-11T19:48:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.334+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/413525582 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5055327e0 con 0x55a5058e0370 2024-03-11T19:48:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.334+0000 7f9cd9ffb640 10 client.26564 handle_command_reply: tid=0 2024-03-11T19:48:24.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.334+0000 7f9cfb1a9740 1 client.26564 shutdown 2024-03-11T19:48:24.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.340+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/413525582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cacc0 msgr2=0x55a5058d47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.340+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/413525582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cacc0 0x55a5058d47b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0011eb0 tx=0x7f9ce001ac10 comp rx=0 tx=0).stop 2024-03-11T19:48:24.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.343+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/413525582 shutdown_connections 2024-03-11T19:48:24.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.343+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/413525582 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a5058e0370 0x55a5058e2830 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.343+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/413525582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cfe30 0x55a5058db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.343+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/413525582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058cacc0 0x55a5058d47b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.343+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/413525582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058d4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.343+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/413525582 >> 172.21.15.83:0/413525582 conn(0x55a5058c57f0 msgr2=0x55a5058c4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.346+0000 7f9cd9ffb640 0 client.26564 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:24.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.347+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/413525582 shutdown_connections 2024-03-11T19:48:24.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.351+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/413525582 wait complete. 2024-03-11T19:48:24.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.351+0000 7f9cfb1a9740 20 client.26564 trim_cache size 0 max 16384 2024-03-11T19:48:24.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.383+0000 7f9cfb1a9740 1 Processor -- start 2024-03-11T19:48:24.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.383+0000 7f9cfb1a9740 1 -- start start 2024-03-11T19:48:24.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.384+0000 7f9cfb1a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058c57f0 0x55a5058d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.384+0000 7f9cfb1a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.384+0000 7f9cf5b72640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.384+0000 7f9cf5b72640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058db7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52160/0 (socket says 172.21.15.83:52160) 2024-03-11T19:48:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.384+0000 7f9cf6373640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058c57f0 0x55a5058d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.384+0000 7f9cf6373640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058c57f0 0x55a5058d4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:47744/0 (socket says 172.21.15.83:47744) 2024-03-11T19:48:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.384+0000 7f9cf5b72640 1 -- 172.21.15.83:0/143764189 learned_addr learned my addr 172.21.15.83:0/143764189 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.385+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/143764189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 0x55a5058d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.385+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/143764189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5058d5460 con 0x55a5058cacc0 2024-03-11T19:48:24.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.385+0000 7f9cf6b74640 1 --2- 172.21.15.83:0/143764189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 0x55a5058d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.385+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/143764189 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5058c6820 con 0x55a5058c57f0 2024-03-11T19:48:24.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.385+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/143764189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5058cb0c0 con 0x55a5058ca2d0 2024-03-11T19:48:24.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.385+0000 7f9cf6b74640 1 -- 172.21.15.83:0/143764189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 msgr2=0x55a5058db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.385+0000 7f9cf6b74640 1 --2- 172.21.15.83:0/143764189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058db7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.385+0000 7f9cf6b74640 1 -- 172.21.15.83:0/143764189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058c57f0 msgr2=0x55a5058d4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.385+0000 7f9cf6b74640 1 --2- 172.21.15.83:0/143764189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058c57f0 0x55a5058d4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.385+0000 7f9cf6b74640 1 -- 172.21.15.83:0/143764189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5057cdac0 con 0x55a5058cacc0 2024-03-11T19:48:24.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.386+0000 7f9cf6b74640 1 --2- 172.21.15.83:0/143764189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 0x55a5058d47b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009270 tx=0x7f9ce801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbe9b44628fbf1f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.386+0000 7f9cdbfff640 1 -- 172.21.15.83:0/143764189 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8013720 con 0x55a5058cacc0 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.386+0000 7f9cdbfff640 1 -- 172.21.15.83:0/143764189 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ce8012e00 con 0x55a5058cacc0 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.386+0000 7f9cdbfff640 1 -- 172.21.15.83:0/143764189 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce801f3f0 con 0x55a5058cacc0 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.387+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/143764189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 msgr2=0x55a5058d47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.387+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/143764189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 0x55a5058d47b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009270 tx=0x7f9ce801d000 comp rx=0 tx=0).stop 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.387+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/143764189 shutdown_connections 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.387+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/143764189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058cacc0 0x55a5058d47b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.387+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/143764189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058ca2d0 0x55a5058db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.387+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/143764189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058c57f0 0x55a5058d4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.387+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/143764189 >> 172.21.15.83:0/143764189 conn(0x55a5058cf900 msgr2=0x55a5058c5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.387+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/143764189 shutdown_connections 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.387+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/143764189 wait complete. 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.387+0000 7f9cfb1a9740 1 -- start start 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.387+0000 7f9cfb1a9740 1 Processor -- start 2024-03-11T19:48:24.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cfb1a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058c57f0 0x55a5058da2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cfb1a9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058ca2d0 0x55a5058da7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cfb1a9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058cacc0 0x55a505936940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cfb1a9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5058dafe0 con 0x55a5058ca2d0 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cfb1a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5058db150 con 0x55a5058c57f0 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cfb1a9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a505936ed0 con 0x55a5058cacc0 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cf5b72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058ca2d0 0x55a5058da7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cfb1a9740 10 client.0 _ll_register_callbacks cb 0x7f9ce0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cf5b72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058ca2d0 0x55a5058da7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37030/0 (socket says 172.21.15.83:37030) 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cf5b72640 1 -- 172.21.15.83:0/1655662955 learned_addr learned my addr 172.21.15.83:0/1655662955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cf6b74640 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058cacc0 0x55a505936940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cf6373640 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058c57f0 0x55a5058da2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cf5b72640 1 -- 172.21.15.83:0/1655662955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058cacc0 msgr2=0x55a505936940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cf5b72640 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058cacc0 0x55a505936940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cf5b72640 1 -- 172.21.15.83:0/1655662955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058c57f0 msgr2=0x55a5058da2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:24.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cf5b72640 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058c57f0 0x55a5058da2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cf5b72640 1 -- 172.21.15.83:0/1655662955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce801b040 con 0x55a5058ca2d0 2024-03-11T19:48:24.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cf5b72640 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058ca2d0 0x55a5058da7f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009250 tx=0x7f9ce002d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47f64aafd59f47a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:24.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:48:24.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/1655662955 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0011eb0 con 0x55a5058ca2d0 2024-03-11T19:48:24.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/1655662955 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9ce0000e40 con 0x55a5058ca2d0 2024-03-11T19:48:24.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.388+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/1655662955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ce001ae50 con 0x55a5058ca2d0 2024-03-11T19:48:24.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.389+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/1655662955 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ce0009ea0 con 0x55a5058ca2d0 2024-03-11T19:48:24.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.390+0000 7f9cfb1a9740 10 client.17118 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:24.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.390+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/1655662955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a5057f36a0 con 0x55a5058ca2d0 2024-03-11T19:48:24.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.392+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/1655662955 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ce001ae50 con 0x55a5058ca2d0 2024-03-11T19:48:24.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.392+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/1655662955 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9ce00035f0 con 0x55a5058ca2d0 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.392+0000 7f9cfb1a9740 10 client.17118 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.392+0000 7f9cfb1a9740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.392+0000 7f9cfb1a9740 20 client.17118 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.392+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a5058cfe30 0x55a50593a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.392+0000 7f9cfb1a9740 4 client.17118 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.392+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/1655662955 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a50556f720 con 0x55a5058cfe30 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.392+0000 7f9cf6373640 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a5058cfe30 0x55a50593a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cd9ffb640 10 client.17118 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cf6373640 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a5058cfe30 0x55a50593a690 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=178362d4078d670d server_cookie=a61a2ee96995c391 in_seq=0 out_seq=0 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cd9ffb640 1 -- 172.21.15.83:0/1655662955 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a50556f720 con 0x55a5058cfe30 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cd9ffb640 10 client.17118 handle_command_reply: tid=0 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cfb1a9740 1 client.17118 shutdown 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/1655662955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058ca2d0 msgr2=0x55a5058da7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058ca2d0 0x55a5058da7f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009250 tx=0x7f9ce002d650 comp rx=0 tx=0).stop 2024-03-11T19:48:24.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/1655662955 shutdown_connections 2024-03-11T19:48:24.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55a5058cfe30 0x55a50593a690 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5058cacc0 0x55a505936940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5058ca2d0 0x55a5058da7f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cfb1a9740 1 --2- 172.21.15.83:0/1655662955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5058c57f0 0x55a5058da2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/1655662955 >> 172.21.15.83:0/1655662955 conn(0x55a5058cf900 msgr2=0x55a5058cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.393+0000 7f9cd9ffb640 0 client.17118 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:24.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.394+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/1655662955 shutdown_connections 2024-03-11T19:48:24.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.394+0000 7f9cfb1a9740 1 -- 172.21.15.83:0/1655662955 wait complete. 2024-03-11T19:48:24.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.394+0000 7f9cfb1a9740 20 client.17118 trim_cache size 0 max 16384 2024-03-11T19:48:24.400 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:48:24.401 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:48:24.401 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:48:24.401 INFO:teuthology.orchestra.run.smithi083.stdout: "afca1676-17b5-4322-a467-bac539244f89": { 2024-03-11T19:48:24.401 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:48:24.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "afca1676-17b5-4322-a467-bac539244f89", 2024-03-11T19:48:24.401 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:48:24.401 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:48:24.401 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:48:24.401 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:48:24.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.394+0000 7f9cf85d7640 1 -- 172.21.15.83:0/2145288458 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9cdc0a1b80 msgr2=0x7f9cdc0a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.394+0000 7f9cf85d7640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9cdc0a1b80 0x7f9cdc0a4040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8005fd0 tx=0x7f9ce8009e60 comp rx=0 tx=0).stop 2024-03-11T19:48:24.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.394+0000 7f9cf85d7640 1 -- 172.21.15.83:0/2145288458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf0101ca0 msgr2=0x7f9cf0101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:24.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.394+0000 7f9cf85d7640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf0101ca0 0x7f9cf0101610 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9cec036040 tx=0x7f9cec005e80 comp rx=0 tx=0).stop 2024-03-11T19:48:24.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.395+0000 7f9cf85d7640 1 -- 172.21.15.83:0/2145288458 shutdown_connections 2024-03-11T19:48:24.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.395+0000 7f9cf85d7640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9cdc0a1b80 0x7f9cdc0a4040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.395+0000 7f9cf85d7640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cf0106f00 0x7f9cf0100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.395+0000 7f9cf85d7640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9cf01045d0 0x7f9cf00ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.395+0000 7f9cf85d7640 1 --2- 172.21.15.83:0/2145288458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cf0101ca0 0x7f9cf0101610 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:24.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.395+0000 7f9cf85d7640 1 -- 172.21.15.83:0/2145288458 >> 172.21.15.83:0/2145288458 conn(0x7f9cf00fb610 msgr2=0x7f9cf0102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:24.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.395+0000 7f9cf85d7640 1 -- 172.21.15.83:0/2145288458 shutdown_connections 2024-03-11T19:48:24.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:24.396+0000 7f9cf85d7640 1 -- 172.21.15.83:0/2145288458 wait complete. 2024-03-11T19:48:24.412 INFO:tasks.cephfs.filesystem:scrub status for tag:afca1676-17b5-4322-a467-bac539244f89 - {'path': '/', 'tag': 'afca1676-17b5-4322-a467-bac539244f89', 'options': 'recursive,force'} 2024-03-11T19:48:24.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:24 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:48:24.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:24 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:24.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:24 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:48:24.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:24 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:24 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:48:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:24 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:25.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:25 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:48:25.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:25 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:25.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:25 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:48:25.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:25 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:48:25.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:25 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:25.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:25 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:25.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:25 smithi137 ceph-mon[36220]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 682 op/s 2024-03-11T19:48:25.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:25 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:48:25.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:25 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:25.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:25 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:48:25.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:25 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:48:25.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:25 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:25.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:25 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:25.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:25 smithi204 ceph-mon[36739]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 682 op/s 2024-03-11T19:48:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:25 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:48:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:25 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:25 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:48:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:25 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:48:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:25 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:25.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:25 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:25.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:25 smithi083 ceph-mon[31864]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 682 op/s 2024-03-11T19:48:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:26 smithi083 ceph-mon[31864]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 619 op/s 2024-03-11T19:48:27.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:26 smithi137 ceph-mon[36220]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 619 op/s 2024-03-11T19:48:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:26 smithi204 ceph-mon[36739]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 MiB/s wr, 619 op/s 2024-03-11T19:48:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 550.0 s, 1902.8 tps, lat 52.602 ms stddev 60.778 2024-03-11T19:48:29.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:28 smithi083 ceph-mon[31864]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 790 op/s 2024-03-11T19:48:29.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:28 smithi137 ceph-mon[36220]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 790 op/s 2024-03-11T19:48:29.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:28 smithi204 ceph-mon[36739]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 790 op/s 2024-03-11T19:48:30.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:30 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:48:30.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:30 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:48:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:30 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:48:31.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:31 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:48:31.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:31 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:48:31.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:31 smithi137 ceph-mon[36220]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 684 op/s 2024-03-11T19:48:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:31 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:48:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:31 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:48:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:31 smithi204 ceph-mon[36739]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 684 op/s 2024-03-11T19:48:32.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:31 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:48:32.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:31 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:48:32.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:31 smithi083 ceph-mon[31864]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 684 op/s 2024-03-11T19:48:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 555.0 s, 1751.8 tps, lat 57.040 ms stddev 69.598 2024-03-11T19:48:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:32 smithi083 ceph-mon[31864]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 806 op/s 2024-03-11T19:48:33.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:32 smithi137 ceph-mon[36220]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 806 op/s 2024-03-11T19:48:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:32 smithi204 ceph-mon[36739]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 MiB/s wr, 806 op/s 2024-03-11T19:48:34.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:33 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:34.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:33 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:34.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:33 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:48:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:34 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:34 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:34 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:48:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:34 smithi083 ceph-mon[31864]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 16 MiB/s wr, 782 op/s 2024-03-11T19:48:35.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:34 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:35.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:34 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:35.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:34 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:48:35.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:34 smithi137 ceph-mon[36220]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 16 MiB/s wr, 782 op/s 2024-03-11T19:48:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:34 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:48:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:34 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:48:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:34 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:48:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:34 smithi204 ceph-mon[36739]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 16 MiB/s wr, 782 op/s 2024-03-11T19:48:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:36 smithi083 ceph-mon[31864]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 750 op/s 2024-03-11T19:48:37.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:36 smithi137 ceph-mon[36220]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 750 op/s 2024-03-11T19:48:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:36 smithi204 ceph-mon[36739]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 16 MiB/s wr, 750 op/s 2024-03-11T19:48:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 560.0 s, 2052.2 tps, lat 48.719 ms stddev 53.335 2024-03-11T19:48:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:38 smithi083 ceph-mon[31864]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 23 MiB/s wr, 916 op/s 2024-03-11T19:48:39.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:38 smithi137 ceph-mon[36220]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 23 MiB/s wr, 916 op/s 2024-03-11T19:48:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:38 smithi204 ceph-mon[36739]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 23 MiB/s wr, 916 op/s 2024-03-11T19:48:41.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:40 smithi083 ceph-mon[31864]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 20 MiB/s wr, 723 op/s 2024-03-11T19:48:41.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:40 smithi137 ceph-mon[36220]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 20 MiB/s wr, 723 op/s 2024-03-11T19:48:41.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:40 smithi204 ceph-mon[36739]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 20 MiB/s wr, 723 op/s 2024-03-11T19:48:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 565.0 s, 1875.0 tps, lat 53.276 ms stddev 59.353 2024-03-11T19:48:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:42 smithi083 ceph-mon[31864]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 19 MiB/s wr, 699 op/s 2024-03-11T19:48:43.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:42 smithi137 ceph-mon[36220]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 19 MiB/s wr, 699 op/s 2024-03-11T19:48:43.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:42 smithi204 ceph-mon[36739]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 19 MiB/s wr, 699 op/s 2024-03-11T19:48:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:44 smithi083 ceph-mon[31864]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 17 MiB/s wr, 699 op/s 2024-03-11T19:48:45.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:44 smithi137 ceph-mon[36220]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 17 MiB/s wr, 699 op/s 2024-03-11T19:48:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:44 smithi204 ceph-mon[36739]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 17 MiB/s wr, 699 op/s 2024-03-11T19:48:47.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:46 smithi083 ceph-mon[31864]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 17 MiB/s wr, 631 op/s 2024-03-11T19:48:47.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:46 smithi137 ceph-mon[36220]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 17 MiB/s wr, 631 op/s 2024-03-11T19:48:47.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:46 smithi204 ceph-mon[36739]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 17 MiB/s wr, 631 op/s 2024-03-11T19:48:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 570.0 s, 2438.4 tps, lat 41.453 ms stddev 48.597 2024-03-11T19:48:49.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:48 smithi083 ceph-mon[31864]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 20 MiB/s wr, 758 op/s 2024-03-11T19:48:49.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:48 smithi137 ceph-mon[36220]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 20 MiB/s wr, 758 op/s 2024-03-11T19:48:49.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:48 smithi204 ceph-mon[36739]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 20 MiB/s wr, 758 op/s 2024-03-11T19:48:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:50 smithi083 ceph-mon[31864]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 620 op/s 2024-03-11T19:48:51.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:50 smithi137 ceph-mon[36220]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 620 op/s 2024-03-11T19:48:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:50 smithi204 ceph-mon[36739]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 620 op/s 2024-03-11T19:48:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 575.0 s, 1973.8 tps, lat 50.723 ms stddev 56.063 2024-03-11T19:48:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:52 smithi083 ceph-mon[31864]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 14 MiB/s wr, 632 op/s 2024-03-11T19:48:53.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:52 smithi137 ceph-mon[36220]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 14 MiB/s wr, 632 op/s 2024-03-11T19:48:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:52 smithi204 ceph-mon[36739]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 14 MiB/s wr, 632 op/s 2024-03-11T19:48:54.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:48:54.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:48:54.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.568+0000 7f7c5a42e640 1 -- 172.21.15.83:0/3075933164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c54108180 msgr2=0x7f7c5410a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.568+0000 7f7c5a42e640 1 --2- 172.21.15.83:0/3075933164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c54108180 0x7f7c5410a5b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009900 tx=0x7f7c4802bc80 comp rx=0 tx=0).stop 2024-03-11T19:48:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.568+0000 7f7c5a42e640 1 -- 172.21.15.83:0/3075933164 shutdown_connections 2024-03-11T19:48:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.568+0000 7f7c5a42e640 1 --2- 172.21.15.83:0/3075933164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c5410d4a0 0x7f7c5410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.568+0000 7f7c5a42e640 1 --2- 172.21.15.83:0/3075933164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c5410aaf0 0x7f7c5410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.568+0000 7f7c5a42e640 1 --2- 172.21.15.83:0/3075933164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c54108180 0x7f7c5410a5b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.568+0000 7f7c5a42e640 1 -- 172.21.15.83:0/3075933164 >> 172.21.15.83:0/3075933164 conn(0x7f7c5406e200 msgr2=0x7f7c54070640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.568+0000 7f7c5a42e640 1 -- 172.21.15.83:0/3075933164 shutdown_connections 2024-03-11T19:48:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.568+0000 7f7c5a42e640 1 -- 172.21.15.83:0/3075933164 wait complete. 2024-03-11T19:48:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.568+0000 7f7c5a42e640 1 Processor -- start 2024-03-11T19:48:54.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.568+0000 7f7c5a42e640 1 -- start start 2024-03-11T19:48:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c5a42e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c54108180 0x7f7c541a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c5a42e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c5410aaf0 0x7f7c541a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c5a42e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c5410d4a0 0x7f7c541ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c5a42e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c541adc80 con 0x7f7c5410d4a0 2024-03-11T19:48:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c5a42e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c541addf0 con 0x7f7c5410aaf0 2024-03-11T19:48:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c5a42e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c541adfb0 con 0x7f7c54108180 2024-03-11T19:48:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c53fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c54108180 0x7f7c541a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c53fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c54108180 0x7f7c541a6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59886/0 (socket says 172.21.15.83:59886) 2024-03-11T19:48:54.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c53fff640 1 -- 172.21.15.83:0/4125488898 learned_addr learned my addr 172.21.15.83:0/4125488898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c589cb640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c5410d4a0 0x7f7c541ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c537fe640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c5410aaf0 0x7f7c541a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c53fff640 1 -- 172.21.15.83:0/4125488898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c5410aaf0 msgr2=0x7f7c541a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.569+0000 7f7c53fff640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c5410aaf0 0x7f7c541a66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.570+0000 7f7c53fff640 1 -- 172.21.15.83:0/4125488898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c5410d4a0 msgr2=0x7f7c541ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.570+0000 7f7c53fff640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c5410d4a0 0x7f7c541ad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.570+0000 7f7c53fff640 1 -- 172.21.15.83:0/4125488898 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c48009560 con 0x7f7c54108180 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.570+0000 7f7c537fe640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c5410aaf0 0x7f7c541a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.570+0000 7f7c53fff640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c54108180 0x7f7c541a6190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009230 tx=0x7f7c480042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.570+0000 7f7c517fa640 1 -- 172.21.15.83:0/4125488898 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c480044e0 con 0x7f7c54108180 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.570+0000 7f7c5a42e640 1 -- 172.21.15.83:0/4125488898 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c541ae230 con 0x7f7c54108180 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.570+0000 7f7c5a42e640 1 -- 172.21.15.83:0/4125488898 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c541ae810 con 0x7f7c54108180 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.570+0000 7f7c517fa640 1 -- 172.21.15.83:0/4125488898 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c480054e0 con 0x7f7c54108180 2024-03-11T19:48:54.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.570+0000 7f7c517fa640 1 -- 172.21.15.83:0/4125488898 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4803e450 con 0x7f7c54108180 2024-03-11T19:48:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.572+0000 7f7c517fa640 1 -- 172.21.15.83:0/4125488898 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7c48005000 con 0x7f7c54108180 2024-03-11T19:48:54.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.573+0000 7f7c517fa640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7c3c0a1ab0 0x7f7c3c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.573+0000 7f7c517fa640 1 -- 172.21.15.83:0/4125488898 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7c480efc00 con 0x7f7c54108180 2024-03-11T19:48:54.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.573+0000 7f7c5d003740 1 Processor -- start 2024-03-11T19:48:54.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.573+0000 7f7c5d003740 1 -- start start 2024-03-11T19:48:54.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.574+0000 7f7c537fe640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7c3c0a1ab0 0x7f7c3c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.574+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89a2d0 0x55cb2a89a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.574+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a89f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.574+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89fe30 0x55cb2a8a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.574+0000 7f7c5d003740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a80a280 con 0x55cb2a89a2d0 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.574+0000 7f7c5d003740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a8077b0 con 0x55cb2a89acc0 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.574+0000 7f7c5d003740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a802f90 con 0x55cb2a89fe30 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.574+0000 7f7c537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a89f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.574+0000 7f7c537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a89f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51924/0 (socket says 172.21.15.83:51924) 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.574+0000 7f7c537fe640 1 -- 172.21.15.83:0/4262532120 learned_addr learned my addr 172.21.15.83:0/4262532120 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.575+0000 7f7c537fe640 1 -- 172.21.15.83:0/4262532120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89fe30 msgr2=0x55cb2a8a22c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.575+0000 7f7c537fe640 1 --2- 172.21.15.83:0/4262532120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89fe30 0x55cb2a8a22c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.575+0000 7f7c537fe640 1 -- 172.21.15.83:0/4262532120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89a2d0 msgr2=0x55cb2a89a6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:48:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.575+0000 7f7c537fe640 1 --2- 172.21.15.83:0/4262532120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89a2d0 0x55cb2a89a6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.575+0000 7f7c537fe640 1 -- 172.21.15.83:0/4262532120 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb2a7c1570 con 0x55cb2a89acc0 2024-03-11T19:48:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.576+0000 7f7c537fe640 1 --2- 172.21.15.83:0/4262532120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a89f7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00afd0 tx=0x7f7c4c017910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5071f4e593051e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:54.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.576+0000 7f7c537fe640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7c3c0a1ab0 0x7f7c3c0a3f70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c017e10 tx=0x7f7c4c011000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.577+0000 7f7c3a7fc640 1 -- 172.21.15.83:0/4262532120 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c010050 con 0x55cb2a89acc0 2024-03-11T19:48:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.577+0000 7f7c3a7fc640 1 -- 172.21.15.83:0/4262532120 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c4c02ae30 con 0x55cb2a89acc0 2024-03-11T19:48:54.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.577+0000 7f7c3a7fc640 1 -- 172.21.15.83:0/4262532120 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c02b640 con 0x55cb2a89acc0 2024-03-11T19:48:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.577+0000 7f7c5d003740 1 -- 172.21.15.83:0/4262532120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 msgr2=0x55cb2a89f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.578+0000 7f7c5d003740 1 --2- 172.21.15.83:0/4262532120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a89f7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00afd0 tx=0x7f7c4c017910 comp rx=0 tx=0).stop 2024-03-11T19:48:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.578+0000 7f7c5d003740 1 -- 172.21.15.83:0/4262532120 shutdown_connections 2024-03-11T19:48:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.578+0000 7f7c5d003740 1 --2- 172.21.15.83:0/4262532120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89fe30 0x55cb2a8a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.578+0000 7f7c5d003740 1 --2- 172.21.15.83:0/4262532120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a89f7c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.578+0000 7f7c5d003740 1 --2- 172.21.15.83:0/4262532120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89a2d0 0x55cb2a89a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.578+0000 7f7c5d003740 1 -- 172.21.15.83:0/4262532120 >> 172.21.15.83:0/4262532120 conn(0x55cb2a8957f0 msgr2=0x55cb2a895bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.578+0000 7f7c5d003740 1 -- 172.21.15.83:0/4262532120 shutdown_connections 2024-03-11T19:48:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.578+0000 7f7c5d003740 1 -- 172.21.15.83:0/4262532120 wait complete. 2024-03-11T19:48:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.578+0000 7f7c5d003740 1 -- start start 2024-03-11T19:48:54.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.578+0000 7f7c5d003740 1 Processor -- start 2024-03-11T19:48:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89a2d0 0x55cb2a8a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89acc0 0x55cb2a8a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89fe30 0x55cb2a8ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c5d003740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a4d6430 con 0x55cb2a89a2d0 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c5d003740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a492e00 con 0x55cb2a89fe30 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89a2d0 0x55cb2a8a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c5d003740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a787fd0 con 0x55cb2a89acc0 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89a2d0 0x55cb2a8a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39244/0 (socket says 172.21.15.83:39244) 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c53fff640 1 -- 172.21.15.83:0/1310052613 learned_addr learned my addr 172.21.15.83:0/1310052613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c5d003740 10 client.0 _ll_register_callbacks cb 0x55cb2a7ddfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c537fe640 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89acc0 0x55cb2a8a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c53fff640 1 -- 172.21.15.83:0/1310052613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89acc0 msgr2=0x55cb2a8a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c53fff640 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89acc0 0x55cb2a8a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c53fff640 1 -- 172.21.15.83:0/1310052613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89fe30 msgr2=0x55cb2a8ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c53fff640 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89fe30 0x55cb2a8ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c53fff640 1 -- 172.21.15.83:0/1310052613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4c013040 con 0x55cb2a89a2d0 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c537fe640 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89acc0 0x55cb2a8a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.579+0000 7f7c53fff640 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89a2d0 0x55cb2a8a4350 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009430 tx=0x7f7c48009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39db579ff779ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c337fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c337fe640 1 -- 172.21.15.83:0/1310052613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c480c53c0 con 0x55cb2a89a2d0 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c337fe640 1 -- 172.21.15.83:0/1310052613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7c480baf20 con 0x55cb2a89a2d0 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c5d003740 1 -- 172.21.15.83:0/1310052613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb2a5027e0 con 0x55cb2a89a2d0 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c337fe640 1 -- 172.21.15.83:0/1310052613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c480bce80 con 0x55cb2a89a2d0 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c5d003740 10 client.17124 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c5d003740 1 -- 172.21.15.83:0/1310052613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cb2a801610 con 0x55cb2a89a2d0 2024-03-11T19:48:54.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c337fe640 1 -- 172.21.15.83:0/1310052613 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c480d9080 con 0x55cb2a89a2d0 2024-03-11T19:48:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c337fe640 1 -- 172.21.15.83:0/1310052613 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7c4803c070 con 0x55cb2a89a2d0 2024-03-11T19:48:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c5d003740 10 client.17124 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c5d003740 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c5d003740 20 client.17124 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c5d003740 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cb2a8b0380 0x55cb2a8b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c5d003740 4 client.17124 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.580+0000 7f7c5d003740 1 -- 172.21.15.83:0/1310052613 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb2a53f720 con 0x55cb2a8b0380 2024-03-11T19:48:54.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.581+0000 7f7c537fe640 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cb2a8b0380 0x55cb2a8b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.582+0000 7f7c337fe640 10 client.17124 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:54.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.582+0000 7f7c537fe640 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cb2a8b0380 0x55cb2a8b2790 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=830448d490841cdc server_cookie=3acb88302586ef21 in_seq=0 out_seq=0 2024-03-11T19:48:54.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.583+0000 7f7c337fe640 1 -- 172.21.15.83:0/1310052613 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f7c4c040a90 con 0x55cb2a8b0380 2024-03-11T19:48:54.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.583+0000 7f7c337fe640 10 client.17124 handle_command_reply: tid=0 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.583+0000 7f7c5d003740 1 client.17124 shutdown 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c5d003740 1 -- 172.21.15.83:0/1310052613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89a2d0 msgr2=0x55cb2a8a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c5d003740 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89a2d0 0x55cb2a8a4350 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009430 tx=0x7f7c48009260 comp rx=0 tx=0).stop 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c5d003740 1 -- 172.21.15.83:0/1310052613 shutdown_connections 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c5d003740 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cb2a8b0380 0x55cb2a8b2790 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c5d003740 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89fe30 0x55cb2a8ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c5d003740 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89acc0 0x55cb2a8a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c5d003740 1 --2- 172.21.15.83:0/1310052613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89a2d0 0x55cb2a8a4350 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c5d003740 1 -- 172.21.15.83:0/1310052613 >> 172.21.15.83:0/1310052613 conn(0x55cb2a8957f0 msgr2=0x55cb2a894a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c337fe640 0 client.17124 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c5d003740 1 -- 172.21.15.83:0/1310052613 shutdown_connections 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c5d003740 1 -- 172.21.15.83:0/1310052613 wait complete. 2024-03-11T19:48:54.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.584+0000 7f7c5d003740 20 client.17124 trim_cache size 0 max 16384 2024-03-11T19:48:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.606+0000 7f7c5d003740 1 Processor -- start 2024-03-11T19:48:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c5d003740 1 -- start start 2024-03-11T19:48:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 0x55cb2a89c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a89c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89fe30 0x55cb2a8ab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c5d003740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a8acc50 con 0x55cb2a89fe30 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c5d003740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a896620 con 0x55cb2a89acc0 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c5d003740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a8b2a00 con 0x55cb2a89a2d0 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c53fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 0x55cb2a89c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c53fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 0x55cb2a89c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59918/0 (socket says 172.21.15.83:59918) 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c53fff640 1 -- 172.21.15.83:0/343244484 learned_addr learned my addr 172.21.15.83:0/343244484 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c589cb640 1 --2- 172.21.15.83:0/343244484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89fe30 0x55cb2a8ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.607+0000 7f7c537fe640 1 --2- 172.21.15.83:0/343244484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a89c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.608+0000 7f7c537fe640 1 -- 172.21.15.83:0/343244484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 msgr2=0x55cb2a89c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.608+0000 7f7c537fe640 1 --2- 172.21.15.83:0/343244484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 0x55cb2a89c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.608+0000 7f7c537fe640 1 -- 172.21.15.83:0/343244484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89fe30 msgr2=0x55cb2a8ab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.608+0000 7f7c537fe640 1 --2- 172.21.15.83:0/343244484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89fe30 0x55cb2a8ab860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.608+0000 7f7c537fe640 1 -- 172.21.15.83:0/343244484 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb2a79dac0 con 0x55cb2a89acc0 2024-03-11T19:48:54.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.608+0000 7f7c537fe640 1 --2- 172.21.15.83:0/343244484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a89c580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c02ad40 tx=0x7f7c4c017ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f320cccb904b20d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.608+0000 7f7c39ffb640 1 -- 172.21.15.83:0/343244484 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c03a050 con 0x55cb2a89acc0 2024-03-11T19:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.608+0000 7f7c39ffb640 1 -- 172.21.15.83:0/343244484 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c4c03c030 con 0x55cb2a89acc0 2024-03-11T19:48:54.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.608+0000 7f7c39ffb640 1 -- 172.21.15.83:0/343244484 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c030070 con 0x55cb2a89acc0 2024-03-11T19:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.609+0000 7f7c5d003740 1 -- 172.21.15.83:0/343244484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 msgr2=0x55cb2a89c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.609+0000 7f7c5d003740 1 --2- 172.21.15.83:0/343244484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a89c580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c02ad40 tx=0x7f7c4c017ec0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.609+0000 7f7c5d003740 1 -- 172.21.15.83:0/343244484 shutdown_connections 2024-03-11T19:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.609+0000 7f7c5d003740 1 --2- 172.21.15.83:0/343244484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89fe30 0x55cb2a8ab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.609+0000 7f7c5d003740 1 --2- 172.21.15.83:0/343244484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a89c580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.609+0000 7f7c5d003740 1 --2- 172.21.15.83:0/343244484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 0x55cb2a89c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.609+0000 7f7c5d003740 1 -- 172.21.15.83:0/343244484 >> 172.21.15.83:0/343244484 conn(0x55cb2a8957f0 msgr2=0x55cb2a8b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:54.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.609+0000 7f7c5d003740 1 -- 172.21.15.83:0/343244484 shutdown_connections 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.609+0000 7f7c5d003740 1 -- 172.21.15.83:0/343244484 wait complete. 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.609+0000 7f7c5d003740 1 -- start start 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c5d003740 1 Processor -- start 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 0x55cb2a8ab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a8a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c5d003740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89fe30 0x55cb2a8aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c5d003740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a8aaae0 con 0x55cb2a89fe30 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c5d003740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a8aac50 con 0x55cb2a89acc0 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c5d003740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb2a906800 con 0x55cb2a89a2d0 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c589cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89fe30 0x55cb2a8aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c53fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 0x55cb2a8ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c589cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89fe30 0x55cb2a8aa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39270/0 (socket says 172.21.15.83:39270) 2024-03-11T19:48:54.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c589cb640 1 -- 172.21.15.83:0/2413857593 learned_addr learned my addr 172.21.15.83:0/2413857593 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a8a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c5d003740 10 client.0 _ll_register_callbacks cb 0x7f7c4c00ab50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c537fe640 1 -- 172.21.15.83:0/2413857593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 msgr2=0x55cb2a8ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c537fe640 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 0x55cb2a8ab890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c537fe640 1 -- 172.21.15.83:0/2413857593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89fe30 msgr2=0x55cb2a8aa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c537fe640 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89fe30 0x55cb2a8aa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c537fe640 1 -- 172.21.15.83:0/2413857593 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb2a7c1570 con 0x55cb2a89acc0 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c53fff640 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 0x55cb2a8ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c537fe640 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a8a9ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c017450 tx=0x7f7c4c00ad90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c0a5a38a734425 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.610+0000 7f7c337fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.611+0000 7f7c337fe640 1 -- 172.21.15.83:0/2413857593 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c030040 con 0x55cb2a89acc0 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.611+0000 7f7c337fe640 1 -- 172.21.15.83:0/2413857593 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7c4c036710 con 0x55cb2a89acc0 2024-03-11T19:48:54.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.611+0000 7f7c5d003740 1 -- 172.21.15.83:0/2413857593 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb2a8b2b60 con 0x55cb2a89acc0 2024-03-11T19:48:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.611+0000 7f7c337fe640 1 -- 172.21.15.83:0/2413857593 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c4c03a020 con 0x55cb2a89acc0 2024-03-11T19:48:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.611+0000 7f7c5d003740 10 client.26585 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.611+0000 7f7c5d003740 1 -- 172.21.15.83:0/2413857593 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55cb2a7c36a0 con 0x55cb2a89acc0 2024-03-11T19:48:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.611+0000 7f7c337fe640 1 -- 172.21.15.83:0/2413857593 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cb2a8b2b60 con 0x55cb2a89acc0 2024-03-11T19:48:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.611+0000 7f7c337fe640 1 -- 172.21.15.83:0/2413857593 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7c4c034020 con 0x55cb2a89acc0 2024-03-11T19:48:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.611+0000 7f7c5d003740 10 client.26585 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.611+0000 7f7c5d003740 10 client.26585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.612+0000 7f7c5d003740 20 client.26585 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:54.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.612+0000 7f7c5d003740 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cb2a907e00 0x55cb2a90a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.612+0000 7f7c5d003740 4 client.26585 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:48:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.612+0000 7f7c5d003740 1 -- 172.21.15.83:0/2413857593 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f7c4c02ae30 con 0x55cb2a907e00 2024-03-11T19:48:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.612+0000 7f7c53fff640 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cb2a907e00 0x55cb2a90a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.612+0000 7f7c337fe640 10 client.26585 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.612+0000 7f7c53fff640 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cb2a907e00 0x55cb2a90a220 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe3dd03b1f707aa0 server_cookie=edd73875621e3891 in_seq=0 out_seq=0 2024-03-11T19:48:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.612+0000 7f7c337fe640 1 -- 172.21.15.83:0/2413857593 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f7c4c02ae30 con 0x55cb2a907e00 2024-03-11T19:48:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.612+0000 7f7c337fe640 10 client.26585 handle_command_reply: tid=0 2024-03-11T19:48:54.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.612+0000 7f7c5d003740 1 client.26585 shutdown 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c5d003740 1 -- 172.21.15.83:0/2413857593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 msgr2=0x55cb2a8a9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c5d003740 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a8a9ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c017450 tx=0x7f7c4c00ad90 comp rx=0 tx=0).stop 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c5d003740 1 -- 172.21.15.83:0/2413857593 shutdown_connections 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c5d003740 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55cb2a907e00 0x55cb2a90a220 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c5d003740 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb2a89fe30 0x55cb2a8aa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c337fe640 0 client.26585 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c5d003740 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cb2a89acc0 0x55cb2a8a9ea0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c5d003740 1 --2- 172.21.15.83:0/2413857593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cb2a89a2d0 0x55cb2a8ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c5d003740 1 -- 172.21.15.83:0/2413857593 >> 172.21.15.83:0/2413857593 conn(0x55cb2a8957f0 msgr2=0x55cb2a89c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c5d003740 1 -- 172.21.15.83:0/2413857593 shutdown_connections 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c5d003740 1 -- 172.21.15.83:0/2413857593 wait complete. 2024-03-11T19:48:54.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.615+0000 7f7c5d003740 20 client.26585 trim_cache size 0 max 16384 2024-03-11T19:48:54.625 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:48:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:48:54.625 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:48:54.625 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 -- 172.21.15.83:0/4125488898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7c3c0a1ab0 msgr2=0x7f7c3c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7c3c0a1ab0 0x7f7c3c0a3f70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c017e10 tx=0x7f7c4c011000 comp rx=0 tx=0).stop 2024-03-11T19:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 -- 172.21.15.83:0/4125488898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c54108180 msgr2=0x7f7c541a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c54108180 0x7f7c541a6190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009230 tx=0x7f7c480042f0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 -- 172.21.15.83:0/4125488898 shutdown_connections 2024-03-11T19:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7c3c0a1ab0 0x7f7c3c0a3f70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c5410d4a0 0x7f7c541ad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c5410aaf0 0x7f7c541a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 --2- 172.21.15.83:0/4125488898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c54108180 0x7f7c541a6190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 -- 172.21.15.83:0/4125488898 >> 172.21.15.83:0/4125488898 conn(0x7f7c5406e200 msgr2=0x7f7c5410af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:54.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 -- 172.21.15.83:0/4125488898 shutdown_connections 2024-03-11T19:48:54.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.616+0000 7f7c5a42e640 1 -- 172.21.15.83:0/4125488898 wait complete. 2024-03-11T19:48:54.634 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:48:54.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:48:54.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.796+0000 7ff485318640 1 -- 172.21.15.83:0/1313477253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4800ffa00 msgr2=0x7ff4801095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.796+0000 7ff485318640 1 --2- 172.21.15.83:0/1313477253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4800ffa00 0x7ff4801095b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff478009ca0 tx=0x7ff47802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.797+0000 7ff485318640 1 -- 172.21.15.83:0/1313477253 shutdown_connections 2024-03-11T19:48:54.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.797+0000 7ff485318640 1 --2- 172.21.15.83:0/1313477253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4800ffa00 0x7ff4801095b0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.797+0000 7ff485318640 1 --2- 172.21.15.83:0/1313477253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4800ff040 0x7ff4800ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.797+0000 7ff485318640 1 --2- 172.21.15.83:0/1313477253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4801025d0 0x7ff4801029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.797+0000 7ff485318640 1 -- 172.21.15.83:0/1313477253 >> 172.21.15.83:0/1313477253 conn(0x7ff4800fabe0 msgr2=0x7ff4800fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:54.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.797+0000 7ff485318640 1 -- 172.21.15.83:0/1313477253 shutdown_connections 2024-03-11T19:48:54.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.797+0000 7ff485318640 1 -- 172.21.15.83:0/1313477253 wait complete. 2024-03-11T19:48:54.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.797+0000 7ff485318640 1 Processor -- start 2024-03-11T19:48:54.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff485318640 1 -- start start 2024-03-11T19:48:54.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff485318640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4800ff040 0x7ff480196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff485318640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4800ffa00 0x7ff480197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff485318640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4801025d0 0x7ff48019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff485318640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48019e930 con 0x7ff4801025d0 2024-03-11T19:48:54.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff485318640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48019eaa0 con 0x7ff4800ffa00 2024-03-11T19:48:54.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff485318640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff480100150 con 0x7ff4800ff040 2024-03-11T19:48:54.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff47ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4800ff040 0x7ff480196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff484b17640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4801025d0 0x7ff48019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff47ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4800ff040 0x7ff480196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:59956/0 (socket says 172.21.15.83:59956) 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff47ffff640 1 -- 172.21.15.83:0/2991141829 learned_addr learned my addr 172.21.15.83:0/2991141829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff47f7fe640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4800ffa00 0x7ff480197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff484b17640 1 -- 172.21.15.83:0/2991141829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4800ff040 msgr2=0x7ff480196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff484b17640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4800ff040 0x7ff480196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff484b17640 1 -- 172.21.15.83:0/2991141829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4800ffa00 msgr2=0x7ff480197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff484b17640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4800ffa00 0x7ff480197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.798+0000 7ff484b17640 1 -- 172.21.15.83:0/2991141829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff478009900 con 0x7ff4801025d0 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.799+0000 7ff484b17640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4801025d0 0x7ff48019e3f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff478009bf0 tx=0x7ff478034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.799+0000 7ff47d7fa640 1 -- 172.21.15.83:0/2991141829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478032b30 con 0x7ff4801025d0 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.799+0000 7ff485318640 1 -- 172.21.15.83:0/2991141829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4801003d0 con 0x7ff4801025d0 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.799+0000 7ff47d7fa640 1 -- 172.21.15.83:0/2991141829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff478003e50 con 0x7ff4801025d0 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.799+0000 7ff485318640 1 -- 172.21.15.83:0/2991141829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4801008a0 con 0x7ff4801025d0 2024-03-11T19:48:54.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.799+0000 7ff47d7fa640 1 -- 172.21.15.83:0/2991141829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478040650 con 0x7ff4801025d0 2024-03-11T19:48:54.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.800+0000 7ff485318640 1 -- 172.21.15.83:0/2991141829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4801021d0 con 0x7ff4801025d0 2024-03-11T19:48:54.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.801+0000 7ff47d7fa640 1 -- 172.21.15.83:0/2991141829 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff47800a040 con 0x7ff4801025d0 2024-03-11T19:48:54.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.802+0000 7ff47d7fa640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4680a1a30 0x7ff4680a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:54.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.802+0000 7ff47d7fa640 1 -- 172.21.15.83:0/2991141829 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff4780f0330 con 0x7ff4801025d0 2024-03-11T19:48:54.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.804+0000 7ff47ffff640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4680a1a30 0x7ff4680a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:54.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.805+0000 7ff47ffff640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4680a1a30 0x7ff4680a3ef0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff4700058c0 tx=0x7ff470005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:54.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:54.808+0000 7ff47d7fa640 1 -- 172.21.15.83:0/2991141829 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4780b9550 con 0x7ff4801025d0 2024-03-11T19:48:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:54 smithi083 ceph-mon[31864]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 715 op/s 2024-03-11T19:48:55.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.121+0000 7ff485318640 1 -- 172.21.15.83:0/2991141829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff48002cea0 con 0x7ff4801025d0 2024-03-11T19:48:55.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.127+0000 7ff47d7fa640 1 -- 172.21.15.83:0/2991141829 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7ff4780bcc10 con 0x7ff4801025d0 2024-03-11T19:48:55.131 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:48:55.131 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:48:55.131 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:48:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.131+0000 7ff485318640 1 -- 172.21.15.83:0/2991141829 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4680a1a30 msgr2=0x7ff4680a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.131+0000 7ff485318640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4680a1a30 0x7ff4680a3ef0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff4700058c0 tx=0x7ff470005f90 comp rx=0 tx=0).stop 2024-03-11T19:48:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.131+0000 7ff485318640 1 -- 172.21.15.83:0/2991141829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4801025d0 msgr2=0x7ff48019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.131+0000 7ff485318640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4801025d0 0x7ff48019e3f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff478009bf0 tx=0x7ff478034cc0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.132+0000 7ff485318640 1 -- 172.21.15.83:0/2991141829 shutdown_connections 2024-03-11T19:48:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.132+0000 7ff485318640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff4680a1a30 0x7ff4680a3ef0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.132+0000 7ff485318640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4801025d0 0x7ff48019e3f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.132+0000 7ff485318640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4800ffa00 0x7ff480197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.132+0000 7ff485318640 1 --2- 172.21.15.83:0/2991141829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4800ff040 0x7ff480196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.132+0000 7ff485318640 1 -- 172.21.15.83:0/2991141829 >> 172.21.15.83:0/2991141829 conn(0x7ff4800fabe0 msgr2=0x7ff480105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.132+0000 7ff485318640 1 -- 172.21.15.83:0/2991141829 shutdown_connections 2024-03-11T19:48:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.133+0000 7ff485318640 1 -- 172.21.15.83:0/2991141829 wait complete. 2024-03-11T19:48:55.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:54 smithi137 ceph-mon[36220]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 715 op/s 2024-03-11T19:48:55.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:54 smithi204 ceph-mon[36739]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 715 op/s 2024-03-11T19:48:55.159 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:48:55.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:48:55.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.317+0000 7f1697be9640 1 -- 172.21.15.83:0/3909996745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169010d4a0 msgr2=0x7f169010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.317+0000 7f1697be9640 1 --2- 172.21.15.83:0/3909996745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169010d4a0 0x7f169010d920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1688009be0 tx=0x7f168802bf30 comp rx=0 tx=0).stop 2024-03-11T19:48:55.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.319+0000 7f1697be9640 1 -- 172.21.15.83:0/3909996745 shutdown_connections 2024-03-11T19:48:55.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.319+0000 7f1697be9640 1 --2- 172.21.15.83:0/3909996745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169010d4a0 0x7f169010d920 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.319+0000 7f1697be9640 1 --2- 172.21.15.83:0/3909996745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f169010aaf0 0x7f169010cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.319+0000 7f1697be9640 1 --2- 172.21.15.83:0/3909996745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1690108180 0x7f169010a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.319+0000 7f1697be9640 1 -- 172.21.15.83:0/3909996745 >> 172.21.15.83:0/3909996745 conn(0x7f169006dfe0 msgr2=0x7f1690070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.322+0000 7f1697be9640 1 -- 172.21.15.83:0/3909996745 shutdown_connections 2024-03-11T19:48:55.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.322+0000 7f1697be9640 1 -- 172.21.15.83:0/3909996745 wait complete. 2024-03-11T19:48:55.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.323+0000 7f1697be9640 1 Processor -- start 2024-03-11T19:48:55.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.323+0000 7f1697be9640 1 -- start start 2024-03-11T19:48:55.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.324+0000 7f1697be9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1690108180 0x7f16901a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.324+0000 7f1697be9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169010aaf0 0x7f16901a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.324+0000 7f1697be9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f169010d4a0 0x7f16901ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.324+0000 7f1697be9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16901adbb0 con 0x7f1690108180 2024-03-11T19:48:55.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.324+0000 7f1697be9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16901add20 con 0x7f169010aaf0 2024-03-11T19:48:55.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.324+0000 7f1697be9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16901adee0 con 0x7f169010d4a0 2024-03-11T19:48:55.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.324+0000 7f1695985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1690108180 0x7f16901a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.324+0000 7f1695985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1690108180 0x7f16901a6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39310/0 (socket says 172.21.15.83:39310) 2024-03-11T19:48:55.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.324+0000 7f1695985640 1 -- 172.21.15.83:0/2668062890 learned_addr learned my addr 172.21.15.83:0/2668062890 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.325+0000 7f1696186640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f169010d4a0 0x7f16901ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.325+0000 7f1695985640 1 -- 172.21.15.83:0/2668062890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f169010d4a0 msgr2=0x7f16901ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.325+0000 7f1695985640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f169010d4a0 0x7f16901ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.325+0000 7f1695985640 1 -- 172.21.15.83:0/2668062890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169010aaf0 msgr2=0x7f16901a6650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:48:55.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.325+0000 7f1695985640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169010aaf0 0x7f16901a6650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.325+0000 7f1695985640 1 -- 172.21.15.83:0/2668062890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688009840 con 0x7f1690108180 2024-03-11T19:48:55.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.326+0000 7f1695985640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1690108180 0x7f16901a6110 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f168000e9c0 tx=0x7f168000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.326+0000 7f1686ffd640 1 -- 172.21.15.83:0/2668062890 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168000ccc0 con 0x7f1690108180 2024-03-11T19:48:55.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.327+0000 7f1686ffd640 1 -- 172.21.15.83:0/2668062890 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f168000ce20 con 0x7f1690108180 2024-03-11T19:48:55.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.327+0000 7f1686ffd640 1 -- 172.21.15.83:0/2668062890 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680010430 con 0x7f1690108180 2024-03-11T19:48:55.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.328+0000 7f1697be9640 1 -- 172.21.15.83:0/2668062890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16901ae1c0 con 0x7f1690108180 2024-03-11T19:48:55.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.328+0000 7f1697be9640 1 -- 172.21.15.83:0/2668062890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16901ae5f0 con 0x7f1690108180 2024-03-11T19:48:55.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.332+0000 7f169a7c0740 1 Processor -- start 2024-03-11T19:48:55.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.333+0000 7f1686ffd640 1 -- 172.21.15.83:0/2668062890 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1680005020 con 0x7f1690108180 2024-03-11T19:48:55.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.333+0000 7f1686ffd640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f167c0a1c00 0x7f167c0a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.334+0000 7f1686ffd640 1 -- 172.21.15.83:0/2668062890 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1680014070 con 0x7f1690108180 2024-03-11T19:48:55.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.334+0000 7f169a7c0740 1 -- start start 2024-03-11T19:48:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.334+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983270 0x55c3b1983690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.334+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983c60 0x55c3b1988760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.334+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1988dd0 0x55c3b198b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.334+0000 7f169a7c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b193f960 con 0x55c3b1983270 2024-03-11T19:48:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.334+0000 7f169a7c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b18f3310 con 0x55c3b1988dd0 2024-03-11T19:48:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.334+0000 7f169a7c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b18f0840 con 0x55c3b1983c60 2024-03-11T19:48:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.335+0000 7f1695985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983270 0x55c3b1983690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.335+0000 7f1695985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983270 0x55c3b1983690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39320/0 (socket says 172.21.15.83:39320) 2024-03-11T19:48:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.335+0000 7f1695985640 1 -- 172.21.15.83:0/2900423968 learned_addr learned my addr 172.21.15.83:0/2900423968 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.335+0000 7f1696186640 1 --2- 172.21.15.83:0/2900423968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1988dd0 0x55c3b198b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.335+0000 7f1695184640 1 --2- 172.21.15.83:0/2900423968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983c60 0x55c3b1988760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.335+0000 7f1695184640 1 -- 172.21.15.83:0/2900423968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1988dd0 msgr2=0x55c3b198b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.335+0000 7f1695184640 1 --2- 172.21.15.83:0/2900423968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1988dd0 0x55c3b198b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.335+0000 7f1695184640 1 -- 172.21.15.83:0/2900423968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983270 msgr2=0x55c3b1983690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.335+0000 7f1695184640 1 --2- 172.21.15.83:0/2900423968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983270 0x55c3b1983690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.335+0000 7f1695184640 1 -- 172.21.15.83:0/2900423968 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b18ae060 con 0x55c3b1983c60 2024-03-11T19:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.335+0000 7f1696186640 1 --2- 172.21.15.83:0/2900423968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1988dd0 0x55c3b198b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.336+0000 7f1695184640 1 --2- 172.21.15.83:0/2900423968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983c60 0x55c3b1988760 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f16901a75d0 tx=0x7f168c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1a1656e0f91f2e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.336+0000 7f1696186640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f167c0a1c00 0x7f167c0a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.336+0000 7f167bfff640 1 -- 172.21.15.83:0/2900423968 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c00b8c0 con 0x55c3b1983c60 2024-03-11T19:48:55.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.336+0000 7f167bfff640 1 -- 172.21.15.83:0/2900423968 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f168c011420 con 0x55c3b1983c60 2024-03-11T19:48:55.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.337+0000 7f167bfff640 1 -- 172.21.15.83:0/2900423968 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c0105c0 con 0x55c3b1983c60 2024-03-11T19:48:55.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.337+0000 7f169a7c0740 1 -- 172.21.15.83:0/2900423968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983c60 msgr2=0x55c3b1988760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.337+0000 7f169a7c0740 1 --2- 172.21.15.83:0/2900423968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983c60 0x55c3b1988760 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f16901a75d0 tx=0x7f168c009b20 comp rx=0 tx=0).stop 2024-03-11T19:48:55.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.337+0000 7f1696186640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f167c0a1c00 0x7f167c0a40c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1688034f50 tx=0x7f168802bf30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.338+0000 7f169a7c0740 1 -- 172.21.15.83:0/2900423968 shutdown_connections 2024-03-11T19:48:55.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.338+0000 7f169a7c0740 1 --2- 172.21.15.83:0/2900423968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1988dd0 0x55c3b198b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.338+0000 7f169a7c0740 1 --2- 172.21.15.83:0/2900423968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983c60 0x55c3b1988760 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.338+0000 7f169a7c0740 1 --2- 172.21.15.83:0/2900423968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983270 0x55c3b1983690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.338+0000 7f169a7c0740 1 -- 172.21.15.83:0/2900423968 >> 172.21.15.83:0/2900423968 conn(0x55c3b197e7e0 msgr2=0x55c3b197ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.338+0000 7f169a7c0740 1 -- 172.21.15.83:0/2900423968 shutdown_connections 2024-03-11T19:48:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.338+0000 7f169a7c0740 1 -- 172.21.15.83:0/2900423968 wait complete. 2024-03-11T19:48:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.338+0000 7f169a7c0740 1 -- start start 2024-03-11T19:48:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.338+0000 7f169a7c0740 1 Processor -- start 2024-03-11T19:48:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 0x55c3b198d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 0x55c3b198d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1988dd0 0x55c3b19949c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f169a7c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b18ec020 con 0x55c3b1983c60 2024-03-11T19:48:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f169a7c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b15bf430 con 0x55c3b1983270 2024-03-11T19:48:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f169a7c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b157be00 con 0x55c3b1988dd0 2024-03-11T19:48:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f169a7c0740 10 client.0 _ll_register_callbacks cb 0x55c3b1893cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f1695184640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 0x55c3b198d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f1695184640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 0x55c3b198d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39330/0 (socket says 172.21.15.83:39330) 2024-03-11T19:48:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f1695184640 1 -- 172.21.15.83:0/1236995199 learned_addr learned my addr 172.21.15.83:0/1236995199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f1696186640 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1988dd0 0x55c3b19949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f1695985640 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 0x55c3b198d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.339+0000 7f1695184640 1 -- 172.21.15.83:0/1236995199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1988dd0 msgr2=0x55c3b19949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.340+0000 7f1695184640 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1988dd0 0x55c3b19949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.340+0000 7f1695184640 1 -- 172.21.15.83:0/1236995199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 msgr2=0x55c3b198d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.340+0000 7f1695184640 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 0x55c3b198d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.340+0000 7f1695184640 1 -- 172.21.15.83:0/1236995199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168c0092f0 con 0x55c3b1983c60 2024-03-11T19:48:55.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.340+0000 7f1695985640 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 0x55c3b198d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:55.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.340+0000 7f1695184640 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 0x55c3b198d980 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f168c00b4b0 tx=0x7f168c0117c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5592dc8ce3514622 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.340+0000 7f16797fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:48:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.340+0000 7f16797fa640 1 -- 172.21.15.83:0/1236995199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c004450 con 0x55c3b1983c60 2024-03-11T19:48:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.340+0000 7f16797fa640 1 -- 172.21.15.83:0/1236995199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f168c0045b0 con 0x55c3b1983c60 2024-03-11T19:48:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.341+0000 7f169a7c0740 1 -- 172.21.15.83:0/1236995199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3b15eb7e0 con 0x55c3b1983c60 2024-03-11T19:48:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.341+0000 7f16797fa640 1 -- 172.21.15.83:0/1236995199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f168c0036b0 con 0x55c3b1983c60 2024-03-11T19:48:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.341+0000 7f169a7c0740 10 client.17148 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.341+0000 7f169a7c0740 1 -- 172.21.15.83:0/1236995199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c3b18ea6a0 con 0x55c3b1983c60 2024-03-11T19:48:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.342+0000 7f16797fa640 1 -- 172.21.15.83:0/1236995199 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c3b15eb7e0 con 0x55c3b1983c60 2024-03-11T19:48:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.342+0000 7f16797fa640 1 -- 172.21.15.83:0/1236995199 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f168c022030 con 0x55c3b1983c60 2024-03-11T19:48:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.342+0000 7f169a7c0740 10 client.17148 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:55.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.342+0000 7f169a7c0740 10 client.17148 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:48:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.342+0000 7f169a7c0740 20 client.17148 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.342+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c3b1999520 0x55c3b199b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.342+0000 7f169a7c0740 4 client.17148 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.342+0000 7f169a7c0740 1 -- 172.21.15.83:0/1236995199 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3b1628720 con 0x55c3b1999520 2024-03-11T19:48:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.342+0000 7f1695985640 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c3b1999520 0x55c3b199b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.343+0000 7f1695985640 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c3b1999520 0x55c3b199b940 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a99132d451199a7 server_cookie=7b109b1e122ef726 in_seq=0 out_seq=0 2024-03-11T19:48:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.343+0000 7f16797fa640 10 client.17148 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:48:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.343+0000 7f16797fa640 1 -- 172.21.15.83:0/1236995199 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3b1628720 con 0x55c3b1999520 2024-03-11T19:48:55.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.343+0000 7f16797fa640 10 client.17148 handle_command_reply: tid=0 2024-03-11T19:48:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.343+0000 7f169a7c0740 1 client.17148 shutdown 2024-03-11T19:48:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.343+0000 7f169a7c0740 1 -- 172.21.15.83:0/1236995199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 msgr2=0x55c3b198d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.343+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 0x55c3b198d980 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f168c00b4b0 tx=0x7f168c0117c0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.344+0000 7f169a7c0740 1 -- 172.21.15.83:0/1236995199 shutdown_connections 2024-03-11T19:48:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.344+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c3b1999520 0x55c3b199b940 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.344+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1988dd0 0x55c3b19949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.344+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 0x55c3b198d980 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.344+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1236995199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 0x55c3b198d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.344+0000 7f169a7c0740 1 -- 172.21.15.83:0/1236995199 >> 172.21.15.83:0/1236995199 conn(0x55c3b197e7e0 msgr2=0x55c3b197db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.344+0000 7f16797fa640 0 client.17148 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:48:55.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.344+0000 7f169a7c0740 1 -- 172.21.15.83:0/1236995199 shutdown_connections 2024-03-11T19:48:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.344+0000 7f169a7c0740 1 -- 172.21.15.83:0/1236995199 wait complete. 2024-03-11T19:48:55.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.344+0000 7f169a7c0740 20 client.17148 trim_cache size 0 max 16384 2024-03-11T19:48:55.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.358+0000 7f169a7c0740 1 Processor -- start 2024-03-11T19:48:55.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.358+0000 7f169a7c0740 1 -- start start 2024-03-11T19:48:55.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.358+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b197e7e0 0x55c3b1985120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.358+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983270 0x55c3b1985660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.358+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 0x55c3b1994980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.358+0000 7f169a7c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b1871130 con 0x55c3b1983c60 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.358+0000 7f169a7c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b15eb7e0 con 0x55c3b197e7e0 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.358+0000 7f169a7c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b198be90 con 0x55c3b1983270 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1696186640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 0x55c3b1994980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1695985640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b197e7e0 0x55c3b1985120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1695985640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b197e7e0 0x55c3b1985120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52008/0 (socket says 172.21.15.83:52008) 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1696186640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 0x55c3b1994980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39344/0 (socket says 172.21.15.83:39344) 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1695985640 1 -- 172.21.15.83:0/2969741397 learned_addr learned my addr 172.21.15.83:0/2969741397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1695184640 1 --2- 172.21.15.83:0/2969741397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983270 0x55c3b1985660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1695985640 1 -- 172.21.15.83:0/2969741397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983270 msgr2=0x55c3b1985660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1695985640 1 --2- 172.21.15.83:0/2969741397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983270 0x55c3b1985660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1695985640 1 -- 172.21.15.83:0/2969741397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 msgr2=0x55c3b1994980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1695985640 1 --2- 172.21.15.83:0/2969741397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 0x55c3b1994980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1695985640 1 -- 172.21.15.83:0/2969741397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b1886c20 con 0x55c3b197e7e0 2024-03-11T19:48:55.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f1695985640 1 --2- 172.21.15.83:0/2969741397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b197e7e0 0x55c3b1985120 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f168009a6d0 tx=0x7f168009d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afe5ea667f6af6bf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f167b7fe640 1 -- 172.21.15.83:0/2969741397 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16800a4070 con 0x55c3b197e7e0 2024-03-11T19:48:55.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f167b7fe640 1 -- 172.21.15.83:0/2969741397 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f168009e890 con 0x55c3b197e7e0 2024-03-11T19:48:55.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.359+0000 7f167b7fe640 1 -- 172.21.15.83:0/2969741397 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16800a8a90 con 0x55c3b197e7e0 2024-03-11T19:48:55.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.361+0000 7f169a7c0740 1 -- 172.21.15.83:0/2969741397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b197e7e0 msgr2=0x55c3b1985120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.361+0000 7f169a7c0740 1 --2- 172.21.15.83:0/2969741397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b197e7e0 0x55c3b1985120 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f168009a6d0 tx=0x7f168009d040 comp rx=0 tx=0).stop 2024-03-11T19:48:55.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.361+0000 7f169a7c0740 1 -- 172.21.15.83:0/2969741397 shutdown_connections 2024-03-11T19:48:55.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.361+0000 7f169a7c0740 1 --2- 172.21.15.83:0/2969741397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b1983c60 0x55c3b1994980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.361+0000 7f169a7c0740 1 --2- 172.21.15.83:0/2969741397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983270 0x55c3b1985660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.361+0000 7f169a7c0740 1 --2- 172.21.15.83:0/2969741397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b197e7e0 0x55c3b1985120 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.361+0000 7f169a7c0740 1 -- 172.21.15.83:0/2969741397 >> 172.21.15.83:0/2969741397 conn(0x55c3b1984a40 msgr2=0x55c3b197ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.361+0000 7f169a7c0740 1 -- 172.21.15.83:0/2969741397 shutdown_connections 2024-03-11T19:48:55.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.361+0000 7f169a7c0740 1 -- 172.21.15.83:0/2969741397 wait complete. 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.361+0000 7f169a7c0740 1 -- start start 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.362+0000 7f169a7c0740 1 Processor -- start 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.362+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b197e7e0 0x55c3b19948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.362+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 0x55c3b1992f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.362+0000 7f169a7c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983c60 0x55c3b1993530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.362+0000 7f169a7c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b1993c30 con 0x55c3b197e7e0 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.362+0000 7f169a7c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b1993da0 con 0x55c3b1983270 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.362+0000 7f169a7c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b19efa30 con 0x55c3b1983c60 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.362+0000 7f169a7c0740 10 client.0 _ll_register_callbacks cb 0x7f168c006e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.362+0000 7f1695985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b197e7e0 0x55c3b19948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f1695184640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 0x55c3b1992f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f1695184640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 0x55c3b1992f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52024/0 (socket says 172.21.15.83:52024) 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f1695184640 1 -- 172.21.15.83:0/1775702323 learned_addr learned my addr 172.21.15.83:0/1775702323 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f1695985640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b197e7e0 0x55c3b19948f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39352/0 (socket says 172.21.15.83:39352) 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f1696186640 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983c60 0x55c3b1993530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f1695184640 1 -- 172.21.15.83:0/1775702323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983c60 msgr2=0x55c3b1993530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f1695184640 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983c60 0x55c3b1993530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f1695184640 1 -- 172.21.15.83:0/1775702323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b197e7e0 msgr2=0x55c3b19948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f1695184640 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b197e7e0 0x55c3b19948f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f1695184640 1 -- 172.21.15.83:0/1775702323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16901ae5f0 con 0x55c3b1983270 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f1695184640 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 0x55c3b1992f70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f168c009b60 tx=0x7f168c011b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca4cb1afb92dc05 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f16797fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f16797fa640 1 -- 172.21.15.83:0/1775702323 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c004520 con 0x55c3b1983270 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f16797fa640 1 -- 172.21.15.83:0/1775702323 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f168c00b6c0 con 0x55c3b1983270 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f169a7c0740 1 -- 172.21.15.83:0/1775702323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f168c011420 con 0x55c3b1983270 2024-03-11T19:48:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.363+0000 7f16797fa640 1 -- 172.21.15.83:0/1775702323 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f168c004680 con 0x55c3b1983270 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f169a7c0740 10 client.26606 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f169a7c0740 1 -- 172.21.15.83:0/1775702323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c3b18ac8c0 con 0x55c3b1983270 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f16797fa640 1 -- 172.21.15.83:0/1775702323 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f168c011420 con 0x55c3b1983270 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f16797fa640 1 -- 172.21.15.83:0/1775702323 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f168c022040 con 0x55c3b1983270 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f169a7c0740 10 client.26606 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f169a7c0740 10 client.26606 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f169a7c0740 20 client.26606 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c3b1988dd0 0x55c3b19f3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f169a7c0740 4 client.26606 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f169a7c0740 1 -- 172.21.15.83:0/1775702323 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c3b1677890 con 0x55c3b1988dd0 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f1695985640 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c3b1988dd0 0x55c3b19f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f16797fa640 10 client.26606 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:48:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.364+0000 7f1695985640 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c3b1988dd0 0x55c3b19f3310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b0a0cb9556bedb1 server_cookie=aa85e0cd4cbe9e5c in_seq=0 out_seq=0 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f16797fa640 1 -- 172.21.15.83:0/1775702323 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c3b1677890 con 0x55c3b1988dd0 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f16797fa640 10 client.26606 handle_command_reply: tid=0 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 1 client.26606 shutdown 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 1 -- 172.21.15.83:0/1775702323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 msgr2=0x55c3b1992f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 0x55c3b1992f70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f168c009b60 tx=0x7f168c011b40 comp rx=0 tx=0).stop 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 1 -- 172.21.15.83:0/1775702323 shutdown_connections 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55c3b1988dd0 0x55c3b19f3310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c3b1983c60 0x55c3b1993530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f16797fa640 0 client.26606 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3b1983270 0x55c3b1992f70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 1 --2- 172.21.15.83:0/1775702323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3b197e7e0 0x55c3b19948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 1 -- 172.21.15.83:0/1775702323 >> 172.21.15.83:0/1775702323 conn(0x55c3b1984a40 msgr2=0x55c3b1985120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 1 -- 172.21.15.83:0/1775702323 shutdown_connections 2024-03-11T19:48:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 1 -- 172.21.15.83:0/1775702323 wait complete. 2024-03-11T19:48:55.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f169a7c0740 20 client.26606 trim_cache size 0 max 16384 2024-03-11T19:48:55.374 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:48:55.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f1697be9640 1 -- 172.21.15.83:0/2668062890 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f167c0a1c00 msgr2=0x7f167c0a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.365+0000 7f1697be9640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f167c0a1c00 0x7f167c0a40c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1688034f50 tx=0x7f168802bf30 comp rx=0 tx=0).stop 2024-03-11T19:48:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.366+0000 7f1697be9640 1 -- 172.21.15.83:0/2668062890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1690108180 msgr2=0x7f16901a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.366+0000 7f1697be9640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1690108180 0x7f16901a6110 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f168000e9c0 tx=0x7f168000ee80 comp rx=0 tx=0).stop 2024-03-11T19:48:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.366+0000 7f1697be9640 1 -- 172.21.15.83:0/2668062890 shutdown_connections 2024-03-11T19:48:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.366+0000 7f1697be9640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f167c0a1c00 0x7f167c0a40c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.366+0000 7f1697be9640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f169010d4a0 0x7f16901ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.366+0000 7f1697be9640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f169010aaf0 0x7f16901a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.366+0000 7f1697be9640 1 --2- 172.21.15.83:0/2668062890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1690108180 0x7f16901a6110 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.366+0000 7f1697be9640 1 -- 172.21.15.83:0/2668062890 >> 172.21.15.83:0/2668062890 conn(0x7f169006dfe0 msgr2=0x7f169010c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.366+0000 7f1697be9640 1 -- 172.21.15.83:0/2668062890 shutdown_connections 2024-03-11T19:48:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.366+0000 7f1697be9640 1 -- 172.21.15.83:0/2668062890 wait complete. 2024-03-11T19:48:55.384 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:48:55.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:48:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.538+0000 7f7ef8193640 1 -- 172.21.15.83:0/3360580101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef010d4a0 msgr2=0x7f7ef010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.538+0000 7f7ef8193640 1 --2- 172.21.15.83:0/3360580101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef010d4a0 0x7f7ef010d920 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009860 tx=0x7f7eec02bb70 comp rx=0 tx=0).stop 2024-03-11T19:48:55.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.539+0000 7f7ef8193640 1 -- 172.21.15.83:0/3360580101 shutdown_connections 2024-03-11T19:48:55.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.539+0000 7f7ef8193640 1 --2- 172.21.15.83:0/3360580101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef010d4a0 0x7f7ef010d920 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.539+0000 7f7ef8193640 1 --2- 172.21.15.83:0/3360580101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef010aaf0 0x7f7ef010cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.539+0000 7f7ef8193640 1 --2- 172.21.15.83:0/3360580101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef0108180 0x7f7ef010a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.539+0000 7f7ef8193640 1 -- 172.21.15.83:0/3360580101 >> 172.21.15.83:0/3360580101 conn(0x7f7ef006dfe0 msgr2=0x7f7ef0070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.539+0000 7f7ef8193640 1 -- 172.21.15.83:0/3360580101 shutdown_connections 2024-03-11T19:48:55.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.540+0000 7f7ef8193640 1 -- 172.21.15.83:0/3360580101 wait complete. 2024-03-11T19:48:55.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.541+0000 7f7ef8193640 1 Processor -- start 2024-03-11T19:48:55.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.541+0000 7f7ef8193640 1 -- start start 2024-03-11T19:48:55.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.541+0000 7f7ef8193640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef0108180 0x7f7ef01a6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.541+0000 7f7ef8193640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef010aaf0 0x7f7ef01a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.541+0000 7f7ef8193640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef010d4a0 0x7f7ef01ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.541+0000 7f7ef8193640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef01adb30 con 0x7f7ef010aaf0 2024-03-11T19:48:55.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.541+0000 7f7ef8193640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef01adca0 con 0x7f7ef010d4a0 2024-03-11T19:48:55.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.541+0000 7f7ef8193640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef01ade60 con 0x7f7ef0108180 2024-03-11T19:48:55.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.542+0000 7f7ef6730640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef010d4a0 0x7f7ef01ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.542+0000 7f7ef6730640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef010d4a0 0x7f7ef01ad5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52050/0 (socket says 172.21.15.83:52050) 2024-03-11T19:48:55.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.542+0000 7f7ef6730640 1 -- 172.21.15.83:0/3530983835 learned_addr learned my addr 172.21.15.83:0/3530983835 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.542+0000 7f7ef5f2f640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef0108180 0x7f7ef01a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.543+0000 7f7ef572e640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef010aaf0 0x7f7ef01a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.543+0000 7f7ef6730640 1 -- 172.21.15.83:0/3530983835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef0108180 msgr2=0x7f7ef01a6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.543+0000 7f7ef6730640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef0108180 0x7f7ef01a6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.543+0000 7f7ef6730640 1 -- 172.21.15.83:0/3530983835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef010aaf0 msgr2=0x7f7ef01a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.543+0000 7f7ef6730640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef010aaf0 0x7f7ef01a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.543+0000 7f7ef6730640 1 -- 172.21.15.83:0/3530983835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee0009560 con 0x7f7ef010d4a0 2024-03-11T19:48:55.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.543+0000 7f7ef5f2f640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef0108180 0x7f7ef01a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:55.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.543+0000 7f7ef6730640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef010d4a0 0x7f7ef01ad5f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009780 tx=0x7f7eec004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.543+0000 7f7ee6ffd640 1 -- 172.21.15.83:0/3530983835 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec034640 con 0x7f7ef010d4a0 2024-03-11T19:48:55.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.544+0000 7f7ef8193640 1 -- 172.21.15.83:0/3530983835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec009490 con 0x7f7ef010d4a0 2024-03-11T19:48:55.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.544+0000 7f7ee6ffd640 1 -- 172.21.15.83:0/3530983835 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7eec0347a0 con 0x7f7ef010d4a0 2024-03-11T19:48:55.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.544+0000 7f7ef8193640 1 -- 172.21.15.83:0/3530983835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef01ae2c0 con 0x7f7ef010d4a0 2024-03-11T19:48:55.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.544+0000 7f7ee6ffd640 1 -- 172.21.15.83:0/3530983835 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec045640 con 0x7f7ef010d4a0 2024-03-11T19:48:55.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.547+0000 7f7efad6f740 1 Processor -- start 2024-03-11T19:48:55.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.548+0000 7f7efad6f740 1 -- start start 2024-03-11T19:48:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.548+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a98270 0x55e350a98690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.548+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 0x55e350a9d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.548+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a9ddd0 0x55e350aa0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.548+0000 7f7efad6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e350a54960 con 0x55e350a98c60 2024-03-11T19:48:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.548+0000 7f7efad6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e350a08310 con 0x55e350a9ddd0 2024-03-11T19:48:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.548+0000 7f7efad6f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e350a05840 con 0x55e350a98270 2024-03-11T19:48:55.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.548+0000 7f7ee6ffd640 1 -- 172.21.15.83:0/3530983835 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7eec03b040 con 0x7f7ef010d4a0 2024-03-11T19:48:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.549+0000 7f7ef5f2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 0x55e350a9d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.549+0000 7f7ef5f2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 0x55e350a9d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39390/0 (socket says 172.21.15.83:39390) 2024-03-11T19:48:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.549+0000 7f7ef5f2f640 1 -- 172.21.15.83:0/1011501422 learned_addr learned my addr 172.21.15.83:0/1011501422 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.549+0000 7f7ef572e640 1 --2- 172.21.15.83:0/1011501422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a98270 0x55e350a98690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.549+0000 7f7ef572e640 1 --2- 172.21.15.83:0/1011501422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a9ddd0 0x55e350aa0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.549+0000 7f7ee6ffd640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7edc0a1b00 0x7f7edc0a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.550+0000 7f7ee6ffd640 1 -- 172.21.15.83:0/3530983835 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7eec0ef830 con 0x7f7ef010d4a0 2024-03-11T19:48:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.550+0000 7f7ef5f2f640 1 -- 172.21.15.83:0/1011501422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a98270 msgr2=0x55e350a98690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.550+0000 7f7ef5f2f640 1 --2- 172.21.15.83:0/1011501422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a98270 0x55e350a98690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.550+0000 7f7ef5f2f640 1 -- 172.21.15.83:0/1011501422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a9ddd0 msgr2=0x55e350aa0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.550+0000 7f7ef5f2f640 1 --2- 172.21.15.83:0/1011501422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a9ddd0 0x55e350aa0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.550+0000 7f7ef5f2f640 1 -- 172.21.15.83:0/1011501422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3509c3060 con 0x55e350a98c60 2024-03-11T19:48:55.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.550+0000 7f7ef5f2f640 1 --2- 172.21.15.83:0/1011501422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 0x55e350a9d760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8011860 tx=0x7f7ee8011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6a812cd6e40ede server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.550+0000 7f7ef6730640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7edc0a1b00 0x7f7edc0a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.551+0000 7f7edbfff640 1 -- 172.21.15.83:0/1011501422 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80112f0 con 0x55e350a98c60 2024-03-11T19:48:55.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.551+0000 7f7edbfff640 1 -- 172.21.15.83:0/1011501422 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ee8003e50 con 0x55e350a98c60 2024-03-11T19:48:55.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.551+0000 7f7edbfff640 1 -- 172.21.15.83:0/1011501422 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8004590 con 0x55e350a98c60 2024-03-11T19:48:55.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.552+0000 7f7efad6f740 1 -- 172.21.15.83:0/1011501422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 msgr2=0x55e350a9d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.552+0000 7f7efad6f740 1 --2- 172.21.15.83:0/1011501422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 0x55e350a9d760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8011860 tx=0x7f7ee8011d20 comp rx=0 tx=0).stop 2024-03-11T19:48:55.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.552+0000 7f7efad6f740 1 -- 172.21.15.83:0/1011501422 shutdown_connections 2024-03-11T19:48:55.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.552+0000 7f7efad6f740 1 --2- 172.21.15.83:0/1011501422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a9ddd0 0x55e350aa0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.552+0000 7f7efad6f740 1 --2- 172.21.15.83:0/1011501422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 0x55e350a9d760 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.552+0000 7f7efad6f740 1 --2- 172.21.15.83:0/1011501422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a98270 0x55e350a98690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.552+0000 7f7efad6f740 1 -- 172.21.15.83:0/1011501422 >> 172.21.15.83:0/1011501422 conn(0x55e350a937e0 msgr2=0x55e350a93be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.552+0000 7f7efad6f740 1 -- 172.21.15.83:0/1011501422 shutdown_connections 2024-03-11T19:48:55.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.553+0000 7f7ef6730640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7edc0a1b00 0x7f7edc0a3fc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0b9180 tx=0x7f7eec0dc040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.553+0000 7f7efad6f740 1 -- 172.21.15.83:0/1011501422 wait complete. 2024-03-11T19:48:55.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.553+0000 7f7efad6f740 1 -- start start 2024-03-11T19:48:55.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.553+0000 7f7efad6f740 1 Processor -- start 2024-03-11T19:48:55.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.554+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a98270 0x55e350aa24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.554+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 0x55e350aa2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.554+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a9ddd0 0x55e350aa9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.554+0000 7f7efad6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e350a01020 con 0x55e350a9ddd0 2024-03-11T19:48:55.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.554+0000 7f7efad6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3506d4430 con 0x55e350a98c60 2024-03-11T19:48:55.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.554+0000 7f7efad6f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e350690e00 con 0x55e350a98270 2024-03-11T19:48:55.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.554+0000 7f7efad6f740 10 client.0 _ll_register_callbacks cb 0x55e3509a8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.554+0000 7f7ef5f2f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 0x55e350aa2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.554+0000 7f7ef5f2f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 0x55e350aa2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52070/0 (socket says 172.21.15.83:52070) 2024-03-11T19:48:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.555+0000 7f7ef5f2f640 1 -- 172.21.15.83:0/103455202 learned_addr learned my addr 172.21.15.83:0/103455202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.555+0000 7f7ef572e640 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a98270 0x55e350aa24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.555+0000 7f7ef572e640 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a9ddd0 0x55e350aa9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.556+0000 7f7ef572e640 1 -- 172.21.15.83:0/103455202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 msgr2=0x55e350aa2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.556+0000 7f7ef572e640 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 0x55e350aa2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.556+0000 7f7ef572e640 1 -- 172.21.15.83:0/103455202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a9ddd0 msgr2=0x55e350aa9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.556+0000 7f7ef572e640 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a9ddd0 0x55e350aa9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.556+0000 7f7ef572e640 1 -- 172.21.15.83:0/103455202 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee80114c0 con 0x55e350a98270 2024-03-11T19:48:55.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.557+0000 7f7ef572e640 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a98270 0x55e350aa24f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ef01a7550 tx=0x7f7ee00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0dd25d8aa4e72df server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.557+0000 7f7ed97fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:48:55.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.557+0000 7f7ed97fa640 1 -- 172.21.15.83:0/103455202 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0010e40 con 0x55e350a98270 2024-03-11T19:48:55.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.557+0000 7f7efad6f740 1 -- 172.21.15.83:0/103455202 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3507007e0 con 0x55e350a98270 2024-03-11T19:48:55.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.558+0000 7f7ed97fa640 1 -- 172.21.15.83:0/103455202 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7ee0004030 con 0x55e350a98270 2024-03-11T19:48:55.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.558+0000 7f7ed97fa640 1 -- 172.21.15.83:0/103455202 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ee0010e40 con 0x55e350a98270 2024-03-11T19:48:55.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.558+0000 7f7efad6f740 10 client.26728 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:55.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.558+0000 7f7efad6f740 1 -- 172.21.15.83:0/103455202 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e3509ff6a0 con 0x55e350a98270 2024-03-11T19:48:55.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.560+0000 7f7ed97fa640 1 -- 172.21.15.83:0/103455202 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e3507007e0 con 0x55e350a98270 2024-03-11T19:48:55.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.560+0000 7f7ed97fa640 1 -- 172.21.15.83:0/103455202 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7ee001f070 con 0x55e350a98270 2024-03-11T19:48:55.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.560+0000 7f7efad6f740 10 client.26728 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.560+0000 7f7efad6f740 10 client.26728 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:48:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.560+0000 7f7efad6f740 20 client.26728 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.560+0000 7f7efad6f740 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e350aae740 0x55e350ab0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.560+0000 7f7efad6f740 4 client.26728 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.560+0000 7f7efad6f740 1 -- 172.21.15.83:0/103455202 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e35073d720 con 0x55e350aae740 2024-03-11T19:48:55.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.562+0000 7f7ef5f2f640 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e350aae740 0x55e350ab0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.563+0000 7f7ed97fa640 10 client.26728 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:48:55.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.563+0000 7f7ef5f2f640 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e350aae740 0x55e350ab0b60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f083d26a7acb1b21 server_cookie=74ccab65fee07d89 in_seq=0 out_seq=0 2024-03-11T19:48:55.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.564+0000 7f7ed97fa640 1 -- 172.21.15.83:0/103455202 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e35073d720 con 0x55e350aae740 2024-03-11T19:48:55.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.564+0000 7f7ed97fa640 10 client.26728 handle_command_reply: tid=0 2024-03-11T19:48:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.564+0000 7f7efad6f740 1 client.26728 shutdown 2024-03-11T19:48:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.564+0000 7f7efad6f740 1 -- 172.21.15.83:0/103455202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a98270 msgr2=0x55e350aa24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.564+0000 7f7efad6f740 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a98270 0x55e350aa24f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ef01a7550 tx=0x7f7ee00046c0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.565+0000 7f7efad6f740 1 -- 172.21.15.83:0/103455202 shutdown_connections 2024-03-11T19:48:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.565+0000 7f7efad6f740 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e350aae740 0x55e350ab0b60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.565+0000 7f7efad6f740 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a9ddd0 0x55e350aa9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.565+0000 7f7efad6f740 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 0x55e350aa2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.565+0000 7f7efad6f740 1 --2- 172.21.15.83:0/103455202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a98270 0x55e350aa24f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.565+0000 7f7efad6f740 1 -- 172.21.15.83:0/103455202 >> 172.21.15.83:0/103455202 conn(0x55e350a937e0 msgr2=0x55e350a92b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.565+0000 7f7ed97fa640 0 client.26728 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:48:55.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.565+0000 7f7efad6f740 1 -- 172.21.15.83:0/103455202 shutdown_connections 2024-03-11T19:48:55.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.566+0000 7f7efad6f740 1 -- 172.21.15.83:0/103455202 wait complete. 2024-03-11T19:48:55.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.566+0000 7f7efad6f740 20 client.26728 trim_cache size 0 max 16384 2024-03-11T19:48:55.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7efad6f740 1 Processor -- start 2024-03-11T19:48:55.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7efad6f740 1 -- start start 2024-03-11T19:48:55.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98270 0x55e350aa2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 0x55e350aa9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a9ddd0 0x55e350aa2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7efad6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e350986130 con 0x55e350a98270 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7efad6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e350a99040 con 0x55e350a98c60 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7efad6f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e350aaaf60 con 0x55e350a9ddd0 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7ef572e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98270 0x55e350aa2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7ef572e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98270 0x55e350aa2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39396/0 (socket says 172.21.15.83:39396) 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7ef572e640 1 -- 172.21.15.83:0/3187632236 learned_addr learned my addr 172.21.15.83:0/3187632236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.585+0000 7f7ef5f2f640 1 --2- 172.21.15.83:0/3187632236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 0x55e350aa9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.586+0000 7f7ef572e640 1 --2- 172.21.15.83:0/3187632236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a9ddd0 0x55e350aa2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.586+0000 7f7ef572e640 1 -- 172.21.15.83:0/3187632236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a9ddd0 msgr2=0x55e350aa2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.586+0000 7f7ef572e640 1 --2- 172.21.15.83:0/3187632236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a9ddd0 0x55e350aa2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.586+0000 7f7ef572e640 1 -- 172.21.15.83:0/3187632236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 msgr2=0x55e350aa9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.586+0000 7f7ef572e640 1 --2- 172.21.15.83:0/3187632236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 0x55e350aa9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.586+0000 7f7ef572e640 1 -- 172.21.15.83:0/3187632236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e350a99c90 con 0x55e350a98270 2024-03-11T19:48:55.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.586+0000 7f7ef5f2f640 1 --2- 172.21.15.83:0/3187632236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 0x55e350aa9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:48:55.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.586+0000 7f7ef572e640 1 --2- 172.21.15.83:0/3187632236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98270 0x55e350aa2100 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00093d0 tx=0x7f7ee0013660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfd52e46ca7a336 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.586+0000 7f7edb7fe640 1 -- 172.21.15.83:0/3187632236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0010e40 con 0x55e350a98270 2024-03-11T19:48:55.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.586+0000 7f7edb7fe640 1 -- 172.21.15.83:0/3187632236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ee0021580 con 0x55e350a98270 2024-03-11T19:48:55.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.586+0000 7f7edb7fe640 1 -- 172.21.15.83:0/3187632236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00217e0 con 0x55e350a98270 2024-03-11T19:48:55.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.587+0000 7f7efad6f740 1 -- 172.21.15.83:0/3187632236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98270 msgr2=0x55e350aa2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.587+0000 7f7efad6f740 1 --2- 172.21.15.83:0/3187632236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98270 0x55e350aa2100 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00093d0 tx=0x7f7ee0013660 comp rx=0 tx=0).stop 2024-03-11T19:48:55.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.587+0000 7f7efad6f740 1 -- 172.21.15.83:0/3187632236 shutdown_connections 2024-03-11T19:48:55.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.587+0000 7f7efad6f740 1 --2- 172.21.15.83:0/3187632236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a9ddd0 0x55e350aa2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.587+0000 7f7efad6f740 1 --2- 172.21.15.83:0/3187632236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98c60 0x55e350aa9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.587+0000 7f7efad6f740 1 --2- 172.21.15.83:0/3187632236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98270 0x55e350aa2100 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.587+0000 7f7efad6f740 1 -- 172.21.15.83:0/3187632236 >> 172.21.15.83:0/3187632236 conn(0x55e350a937e0 msgr2=0x55e350a9a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.587+0000 7f7efad6f740 1 -- 172.21.15.83:0/3187632236 shutdown_connections 2024-03-11T19:48:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.587+0000 7f7efad6f740 1 -- 172.21.15.83:0/3187632236 wait complete. 2024-03-11T19:48:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.588+0000 7f7efad6f740 1 -- start start 2024-03-11T19:48:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.588+0000 7f7efad6f740 1 Processor -- start 2024-03-11T19:48:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.588+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98270 0x55e350aa9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.588+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 0x55e350aa8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.588+0000 7f7efad6f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a9ddd0 0x55e350aa85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.588+0000 7f7efad6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e350aa8cf0 con 0x55e350a98c60 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.588+0000 7f7efad6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e350aa8e60 con 0x55e350a98270 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.588+0000 7f7efad6f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e350b04c70 con 0x55e350a9ddd0 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.588+0000 7f7ef5f2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 0x55e350aa8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.588+0000 7f7ef5f2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 0x55e350aa8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39404/0 (socket says 172.21.15.83:39404) 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.588+0000 7f7ef5f2f640 1 -- 172.21.15.83:0/3505485820 learned_addr learned my addr 172.21.15.83:0/3505485820 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.589+0000 7f7efad6f740 10 client.0 _ll_register_callbacks cb 0x7f7ee8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.589+0000 7f7ef572e640 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98270 0x55e350aa9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.589+0000 7f7ef572e640 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a9ddd0 0x55e350aa85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.589+0000 7f7ef572e640 1 -- 172.21.15.83:0/3505485820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a9ddd0 msgr2=0x55e350aa85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.589+0000 7f7ef572e640 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a9ddd0 0x55e350aa85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.589+0000 7f7ef572e640 1 -- 172.21.15.83:0/3505485820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 msgr2=0x55e350aa8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.589+0000 7f7ef572e640 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 0x55e350aa8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.589+0000 7f7ef572e640 1 -- 172.21.15.83:0/3505485820 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3509ff6a0 con 0x55e350a98270 2024-03-11T19:48:55.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.589+0000 7f7ef572e640 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98270 0x55e350aa9a60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0003e00 tx=0x7f7ee00136d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f67a5377d8179ff server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.589+0000 7f7ed97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:48:55.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.590+0000 7f7ed97fa640 1 -- 172.21.15.83:0/3505485820 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0010e60 con 0x55e350a98270 2024-03-11T19:48:55.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.590+0000 7f7efad6f740 1 -- 172.21.15.83:0/3505485820 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ee8003e50 con 0x55e350a98270 2024-03-11T19:48:55.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.590+0000 7f7ed97fa640 1 -- 172.21.15.83:0/3505485820 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7ee0000e40 con 0x55e350a98270 2024-03-11T19:48:55.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.591+0000 7f7ed97fa640 1 -- 172.21.15.83:0/3505485820 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ee0055020 con 0x55e350a98270 2024-03-11T19:48:55.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.591+0000 7f7efad6f740 10 client.26624 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:55.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.591+0000 7f7efad6f740 1 -- 172.21.15.83:0/3505485820 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e35099bc20 con 0x55e350a98270 2024-03-11T19:48:55.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.592+0000 7f7ed97fa640 1 -- 172.21.15.83:0/3505485820 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ee0009bf0 con 0x55e350a98270 2024-03-11T19:48:55.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.592+0000 7f7ed97fa640 1 -- 172.21.15.83:0/3505485820 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7ee001f070 con 0x55e350a98270 2024-03-11T19:48:55.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.592+0000 7f7efad6f740 10 client.26624 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.592+0000 7f7efad6f740 10 client.26624 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.592+0000 7f7efad6f740 20 client.26624 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.592+0000 7f7efad6f740 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e350b061e0 0x55e350b08600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.592+0000 7f7efad6f740 4 client.26624 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.592+0000 7f7efad6f740 1 -- 172.21.15.83:0/3505485820 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f7ee8003e50 con 0x55e350b061e0 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.592+0000 7f7ef5f2f640 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e350b061e0 0x55e350b08600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7ed97fa640 10 client.26624 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7ef5f2f640 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e350b061e0 0x55e350b08600 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54eca002a248e5ee server_cookie=6db1576a75007d0c in_seq=0 out_seq=0 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7ed97fa640 1 -- 172.21.15.83:0/3505485820 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f7ee8003e50 con 0x55e350b061e0 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7ed97fa640 10 client.26624 handle_command_reply: tid=0 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7efad6f740 1 client.26624 shutdown 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7efad6f740 1 -- 172.21.15.83:0/3505485820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98270 msgr2=0x55e350aa9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7efad6f740 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98270 0x55e350aa9a60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0003e00 tx=0x7f7ee00136d0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7efad6f740 1 -- 172.21.15.83:0/3505485820 shutdown_connections 2024-03-11T19:48:55.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7efad6f740 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55e350b061e0 0x55e350b08600 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7ed97fa640 0 client.26624 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:48:55.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7efad6f740 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e350a9ddd0 0x55e350aa85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7efad6f740 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e350a98c60 0x55e350aa8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7efad6f740 1 --2- 172.21.15.83:0/3505485820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e350a98270 0x55e350aa9a60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.593+0000 7f7efad6f740 1 -- 172.21.15.83:0/3505485820 >> 172.21.15.83:0/3505485820 conn(0x55e350a937e0 msgr2=0x55e350aa2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7efad6f740 1 -- 172.21.15.83:0/3505485820 shutdown_connections 2024-03-11T19:48:55.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7efad6f740 1 -- 172.21.15.83:0/3505485820 wait complete. 2024-03-11T19:48:55.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7efad6f740 20 client.26624 trim_cache size 0 max 16384 2024-03-11T19:48:55.600 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:48:55.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 -- 172.21.15.83:0/3530983835 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7edc0a1b00 msgr2=0x7f7edc0a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7edc0a1b00 0x7f7edc0a3fc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0b9180 tx=0x7f7eec0dc040 comp rx=0 tx=0).stop 2024-03-11T19:48:55.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 -- 172.21.15.83:0/3530983835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef010d4a0 msgr2=0x7f7ef01ad5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef010d4a0 0x7f7ef01ad5f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009780 tx=0x7f7eec004270 comp rx=0 tx=0).stop 2024-03-11T19:48:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 -- 172.21.15.83:0/3530983835 shutdown_connections 2024-03-11T19:48:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7edc0a1b00 0x7f7edc0a3fc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef010d4a0 0x7f7ef01ad5f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef010aaf0 0x7f7ef01a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 --2- 172.21.15.83:0/3530983835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef0108180 0x7f7ef01a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 -- 172.21.15.83:0/3530983835 >> 172.21.15.83:0/3530983835 conn(0x7f7ef006dfe0 msgr2=0x7f7ef01085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 -- 172.21.15.83:0/3530983835 shutdown_connections 2024-03-11T19:48:55.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.594+0000 7f7ef8193640 1 -- 172.21.15.83:0/3530983835 wait complete. 2024-03-11T19:48:55.611 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:48:55.612 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:48:55.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.750+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2175041668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc81013b0 msgr2=0x7f5bc8107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.750+0000 7f5bcf93e640 1 --2- 172.21.15.83:0/2175041668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc81013b0 0x7f5bc8107c40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000acf0 tx=0x7f5bc002d230 comp rx=0 tx=0).stop 2024-03-11T19:48:55.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.751+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2175041668 shutdown_connections 2024-03-11T19:48:55.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.751+0000 7f5bcf93e640 1 --2- 172.21.15.83:0/2175041668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc81013b0 0x7f5bc8107c40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.751+0000 7f5bcf93e640 1 --2- 172.21.15.83:0/2175041668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bc81009f0 0x7f5bc8100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.751+0000 7f5bcf93e640 1 --2- 172.21.15.83:0/2175041668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bc80ff800 0x7f5bc80ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.751+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2175041668 >> 172.21.15.83:0/2175041668 conn(0x7f5bc80fad70 msgr2=0x7f5bc80fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.751+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2175041668 shutdown_connections 2024-03-11T19:48:55.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.751+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2175041668 wait complete. 2024-03-11T19:48:55.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcf93e640 1 Processor -- start 2024-03-11T19:48:55.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcf93e640 1 -- start start 2024-03-11T19:48:55.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcf93e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bc80ff800 0x7f5bc8194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcf93e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc81009f0 0x7f5bc8195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcf93e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bc81013b0 0x7f5bc819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcf93e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc819c8c0 con 0x7f5bc80ff800 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcd6da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bc80ff800 0x7f5bc8194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcd6da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bc80ff800 0x7f5bc8194dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39418/0 (socket says 172.21.15.83:39418) 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcd6da640 1 -- 172.21.15.83:0/2297692681 learned_addr learned my addr 172.21.15.83:0/2297692681 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcdedb640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bc81013b0 0x7f5bc819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2297692681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc819ca30 con 0x7f5bc81013b0 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.752+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2297692681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc819cbf0 con 0x7f5bc81009f0 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.753+0000 7f5bcced9640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc81009f0 0x7f5bc8195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.753+0000 7f5bcdedb640 1 -- 172.21.15.83:0/2297692681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc81009f0 msgr2=0x7f5bc8195310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.753+0000 7f5bcdedb640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc81009f0 0x7f5bc8195310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.753+0000 7f5bcdedb640 1 -- 172.21.15.83:0/2297692681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bc80ff800 msgr2=0x7f5bc8194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.753+0000 7f5bcdedb640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bc80ff800 0x7f5bc8194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.753+0000 7f5bcdedb640 1 -- 172.21.15.83:0/2297692681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb8009560 con 0x7f5bc81013b0 2024-03-11T19:48:55.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.753+0000 7f5bcdedb640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bc81013b0 0x7f5bc819c380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0037040 tx=0x7f5bc0034bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.753+0000 7f5bbe7fc640 1 -- 172.21.15.83:0/2297692681 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0035640 con 0x7f5bc81013b0 2024-03-11T19:48:55.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.753+0000 7f5bbe7fc640 1 -- 172.21.15.83:0/2297692681 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bc0003e50 con 0x7f5bc81013b0 2024-03-11T19:48:55.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.753+0000 7f5bbe7fc640 1 -- 172.21.15.83:0/2297692681 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0004860 con 0x7f5bc81013b0 2024-03-11T19:48:55.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.754+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2297692681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc000a950 con 0x7f5bc81013b0 2024-03-11T19:48:55.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.754+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2297692681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc819d0e0 con 0x7f5bc81013b0 2024-03-11T19:48:55.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.757+0000 7f5bbe7fc640 1 -- 172.21.15.83:0/2297692681 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5bc0003970 con 0x7f5bc81013b0 2024-03-11T19:48:55.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.759+0000 7f5bd25eb740 1 Processor -- start 2024-03-11T19:48:55.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.759+0000 7f5bbe7fc640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5bb40a1a30 0x7f5bb40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.759+0000 7f5bd25eb740 1 -- start start 2024-03-11T19:48:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.759+0000 7f5bbe7fc640 1 -- 172.21.15.83:0/2297692681 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5bc00f1880 con 0x7f5bc81013b0 2024-03-11T19:48:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.759+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789ca40 0x5577e789ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.759+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e789d430 0x5577e78a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.759+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e78a2190 0x5577e78a4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.759+0000 7f5bd25eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e7854960 con 0x5577e78a2190 2024-03-11T19:48:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.759+0000 7f5bd25eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e7808310 con 0x5577e789ca40 2024-03-11T19:48:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.759+0000 7f5bd25eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e7805840 con 0x5577e789d430 2024-03-11T19:48:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.759+0000 7f5bcced9640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5bb40a1a30 0x7f5bb40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.760+0000 7f5bcdedb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e78a2190 0x5577e78a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.760+0000 7f5bcdedb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e78a2190 0x5577e78a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39428/0 (socket says 172.21.15.83:39428) 2024-03-11T19:48:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.760+0000 7f5bcdedb640 1 -- 172.21.15.83:0/1925890154 learned_addr learned my addr 172.21.15.83:0/1925890154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.760+0000 7f5bcced9640 1 --2- 172.21.15.83:0/1925890154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e789d430 0x5577e78a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.760+0000 7f5bcd6da640 1 --2- 172.21.15.83:0/1925890154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789ca40 0x5577e789ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.760+0000 7f5bcdedb640 1 -- 172.21.15.83:0/1925890154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e789d430 msgr2=0x5577e78a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.760+0000 7f5bcdedb640 1 --2- 172.21.15.83:0/1925890154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e789d430 0x5577e78a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.760+0000 7f5bcdedb640 1 -- 172.21.15.83:0/1925890154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789ca40 msgr2=0x5577e789ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.760+0000 7f5bcdedb640 1 --2- 172.21.15.83:0/1925890154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789ca40 0x5577e789ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.760+0000 7f5bcdedb640 1 -- 172.21.15.83:0/1925890154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577e77c3060 con 0x5577e78a2190 2024-03-11T19:48:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.760+0000 7f5bcced9640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5bb40a1a30 0x7f5bb40a3ef0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5bc81962e0 tx=0x7f5bb8009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.761+0000 7f5bcdedb640 1 --2- 172.21.15.83:0/1925890154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e78a2190 0x5577e78a4650 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00bb250 tx=0x7f5bc00bac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f61fa2f1559b69c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.761+0000 7f5bb37fe640 1 -- 172.21.15.83:0/1925890154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00c38f0 con 0x5577e78a2190 2024-03-11T19:48:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.761+0000 7f5bb37fe640 1 -- 172.21.15.83:0/1925890154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bc0034da0 con 0x5577e78a2190 2024-03-11T19:48:55.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.761+0000 7f5bb37fe640 1 -- 172.21.15.83:0/1925890154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00bcb80 con 0x5577e78a2190 2024-03-11T19:48:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.762+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1925890154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e78a2190 msgr2=0x5577e78a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.762+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/1925890154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e78a2190 0x5577e78a4650 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00bb250 tx=0x7f5bc00bac50 comp rx=0 tx=0).stop 2024-03-11T19:48:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.762+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1925890154 shutdown_connections 2024-03-11T19:48:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.762+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/1925890154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e78a2190 0x5577e78a4650 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.762+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/1925890154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e789d430 0x5577e78a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.762+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/1925890154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789ca40 0x5577e789ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.762+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1925890154 >> 172.21.15.83:0/1925890154 conn(0x5577e7894e20 msgr2=0x5577e7893370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.762+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1925890154 shutdown_connections 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.762+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1925890154 wait complete. 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bd25eb740 1 -- start start 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bd25eb740 1 Processor -- start 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78a6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 0x5577e78a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e78a2190 0x5577e78adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bd25eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e7801020 con 0x5577e789ca40 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bd25eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e74d4430 con 0x5577e789d430 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bd25eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e7490e00 con 0x5577e78a2190 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bd25eb740 10 client.0 _ll_register_callbacks cb 0x5577e77a8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bcd6da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bcd6da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39440/0 (socket says 172.21.15.83:39440) 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.763+0000 7f5bcd6da640 1 -- 172.21.15.83:0/1127352175 learned_addr learned my addr 172.21.15.83:0/1127352175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bcced9640 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 0x5577e78a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bcdedb640 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e78a2190 0x5577e78adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bcd6da640 1 -- 172.21.15.83:0/1127352175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e78a2190 msgr2=0x5577e78adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bcd6da640 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e78a2190 0x5577e78adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bcd6da640 1 -- 172.21.15.83:0/1127352175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 msgr2=0x5577e78a6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bcd6da640 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 0x5577e78a6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bcd6da640 1 -- 172.21.15.83:0/1127352175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc819d0e0 con 0x5577e789ca40 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bcced9640 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 0x5577e78a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bcd6da640 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78a6640 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40127d0 tx=0x7f5bc4012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eaa13cac54cb5ea server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/1127352175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4017070 con 0x5577e789ca40 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/1127352175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5bc4004030 con 0x5577e789ca40 2024-03-11T19:48:55.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1127352175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577e75007e0 con 0x5577e789ca40 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/1127352175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bc4022030 con 0x5577e789ca40 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bd25eb740 10 client.17184 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.764+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1127352175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5577e77ff6a0 con 0x5577e789ca40 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.765+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/1127352175 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5577e75007e0 con 0x5577e789ca40 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.765+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/1127352175 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5bc4003800 con 0x5577e789ca40 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.765+0000 7f5bd25eb740 10 client.17184 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.765+0000 7f5bd25eb740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.765+0000 7f5bd25eb740 20 client.17184 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.765+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5577e78b2910 0x5577e78b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.765+0000 7f5bd25eb740 4 client.17184 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.765+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1127352175 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577e753d720 con 0x5577e78b2910 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.765+0000 7f5bcced9640 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5577e78b2910 0x5577e78b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.766+0000 7f5bb0ff9640 10 client.17184 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:55.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.766+0000 7f5bcced9640 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5577e78b2910 0x5577e78b4d30 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d34231a7f0a0af3 server_cookie=e905397887ef077d in_seq=0 out_seq=0 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.767+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/1127352175 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5577e753d720 con 0x5577e78b2910 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.767+0000 7f5bb0ff9640 10 client.17184 handle_command_reply: tid=0 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.767+0000 7f5bd25eb740 1 client.17184 shutdown 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.767+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1127352175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 msgr2=0x5577e78a6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.767+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78a6640 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40127d0 tx=0x7f5bc4012bd0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.767+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1127352175 shutdown_connections 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.767+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5577e78b2910 0x5577e78b4d30 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.767+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e78a2190 0x5577e78adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.767+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 0x5577e78a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.767+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/1127352175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78a6640 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.767+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1127352175 >> 172.21.15.83:0/1127352175 conn(0x5577e7894e20 msgr2=0x5577e7892b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.768+0000 7f5bb0ff9640 0 client.17184 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.768+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1127352175 shutdown_connections 2024-03-11T19:48:55.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.768+0000 7f5bd25eb740 1 -- 172.21.15.83:0/1127352175 wait complete. 2024-03-11T19:48:55.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.768+0000 7f5bd25eb740 20 client.17184 trim_cache size 0 max 16384 2024-03-11T19:48:55.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bd25eb740 1 Processor -- start 2024-03-11T19:48:55.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bd25eb740 1 -- start start 2024-03-11T19:48:55.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e7893300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e789d430 0x5577e78adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e78a2190 0x5577e78a5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcd6da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e7893300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcd6da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e7893300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39442/0 (socket says 172.21.15.83:39442) 2024-03-11T19:48:55.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcd6da640 1 -- 172.21.15.83:0/2376164756 learned_addr learned my addr 172.21.15.83:0/2376164756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bd25eb740 1 -- 172.21.15.83:0/2376164756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e7786130 con 0x5577e789ca40 2024-03-11T19:48:55.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bd25eb740 1 -- 172.21.15.83:0/2376164756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e789eb30 con 0x5577e78a2190 2024-03-11T19:48:55.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bd25eb740 1 -- 172.21.15.83:0/2376164756 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e753d720 con 0x5577e789d430 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcdedb640 1 --2- 172.21.15.83:0/2376164756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e78a2190 0x5577e78a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcced9640 1 --2- 172.21.15.83:0/2376164756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e789d430 0x5577e78adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcd6da640 1 -- 172.21.15.83:0/2376164756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e789d430 msgr2=0x5577e78adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcd6da640 1 --2- 172.21.15.83:0/2376164756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e789d430 0x5577e78adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcd6da640 1 -- 172.21.15.83:0/2376164756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e78a2190 msgr2=0x5577e78a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcd6da640 1 --2- 172.21.15.83:0/2376164756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e78a2190 0x5577e78a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcd6da640 1 -- 172.21.15.83:0/2376164756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577e779bc20 con 0x5577e789ca40 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcced9640 1 --2- 172.21.15.83:0/2376164756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e789d430 0x5577e78adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.783+0000 7f5bcdedb640 1 --2- 172.21.15.83:0/2376164756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e78a2190 0x5577e78a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.784+0000 7f5bcd6da640 1 --2- 172.21.15.83:0/2376164756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e7893300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009ed0 tx=0x7f5bc4012420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cdbd13c93b54db server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.784+0000 7f5bb2ffd640 1 -- 172.21.15.83:0/2376164756 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4017040 con 0x5577e789ca40 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.784+0000 7f5bb2ffd640 1 -- 172.21.15.83:0/2376164756 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bc4003a00 con 0x5577e789ca40 2024-03-11T19:48:55.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.784+0000 7f5bb2ffd640 1 -- 172.21.15.83:0/2376164756 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4022040 con 0x5577e789ca40 2024-03-11T19:48:55.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.785+0000 7f5bd25eb740 1 -- 172.21.15.83:0/2376164756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 msgr2=0x5577e7893300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.785+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/2376164756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e7893300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009ed0 tx=0x7f5bc4012420 comp rx=0 tx=0).stop 2024-03-11T19:48:55.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.785+0000 7f5bd25eb740 1 -- 172.21.15.83:0/2376164756 shutdown_connections 2024-03-11T19:48:55.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.785+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/2376164756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e78a2190 0x5577e78a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.785+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/2376164756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e789d430 0x5577e78adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.785+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/2376164756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e7893300 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.785+0000 7f5bd25eb740 1 -- 172.21.15.83:0/2376164756 >> 172.21.15.83:0/2376164756 conn(0x5577e7894e20 msgr2=0x5577e789c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.785+0000 7f5bd25eb740 1 -- 172.21.15.83:0/2376164756 shutdown_connections 2024-03-11T19:48:55.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.785+0000 7f5bd25eb740 1 -- 172.21.15.83:0/2376164756 wait complete. 2024-03-11T19:48:55.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.785+0000 7f5bd25eb740 1 -- start start 2024-03-11T19:48:55.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.785+0000 7f5bd25eb740 1 Processor -- start 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 0x5577e78ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bd25eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e78a2190 0x5577e7909220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bd25eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e7909760 con 0x5577e789ca40 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bd25eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e79098a0 con 0x5577e789d430 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bd25eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e7909a30 con 0x5577e78a2190 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bd25eb740 10 client.0 _ll_register_callbacks cb 0x7f5bc0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bcd6da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bcd6da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78acf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39448/0 (socket says 172.21.15.83:39448) 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bcced9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 0x5577e78ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bcd6da640 1 -- 172.21.15.83:0/3489116912 learned_addr learned my addr 172.21.15.83:0/3489116912 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bcdedb640 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e78a2190 0x5577e7909220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bcced9640 1 -- 172.21.15.83:0/3489116912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e78a2190 msgr2=0x5577e7909220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bcced9640 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e78a2190 0x5577e7909220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bcced9640 1 -- 172.21.15.83:0/3489116912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 msgr2=0x5577e78acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bcced9640 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bcced9640 1 -- 172.21.15.83:0/3489116912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577e77c3060 con 0x5577e789d430 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.786+0000 7f5bcd6da640 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bcced9640 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 0x5577e78ad4d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5577e789d830 tx=0x7f5bb801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8343d51fa7e806f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/3489116912 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8018040 con 0x5577e789d430 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/3489116912 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5bb8010980 con 0x5577e789d430 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bd25eb740 1 -- 172.21.15.83:0/3489116912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bc0034da0 con 0x5577e789d430 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/3489116912 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb8010b40 con 0x5577e789d430 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bd25eb740 10 client.26642 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bd25eb740 1 -- 172.21.15.83:0/3489116912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5577e77ff6a0 con 0x5577e789d430 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/3489116912 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5bc0034da0 con 0x5577e789d430 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/3489116912 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5bb803e070 con 0x5577e789d430 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bd25eb740 10 client.26642 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:55.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bd25eb740 10 client.26642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bd25eb740 20 client.26642 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5577e790b600 0x5577e790da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bd25eb740 4 client.26642 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.787+0000 7f5bd25eb740 1 -- 172.21.15.83:0/3489116912 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5577e758c890 con 0x5577e790b600 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bcd6da640 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5577e790b600 0x5577e790da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bb0ff9640 10 client.26642 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bcd6da640 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5577e790b600 0x5577e790da20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ff059cf997b425 server_cookie=9132af0b683bdc4e in_seq=0 out_seq=0 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bb0ff9640 1 -- 172.21.15.83:0/3489116912 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5577e758c890 con 0x5577e790b600 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bb0ff9640 10 client.26642 handle_command_reply: tid=0 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bd25eb740 1 client.26642 shutdown 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bd25eb740 1 -- 172.21.15.83:0/3489116912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 msgr2=0x5577e78ad4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 0x5577e78ad4d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5577e789d830 tx=0x7f5bb801b000 comp rx=0 tx=0).stop 2024-03-11T19:48:55.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bd25eb740 1 -- 172.21.15.83:0/3489116912 shutdown_connections 2024-03-11T19:48:55.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5577e790b600 0x5577e790da20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577e78a2190 0x5577e7909220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577e789d430 0x5577e78ad4d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bd25eb740 1 --2- 172.21.15.83:0/3489116912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577e789ca40 0x5577e78acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bd25eb740 1 -- 172.21.15.83:0/3489116912 >> 172.21.15.83:0/3489116912 conn(0x5577e7894e20 msgr2=0x5577e78922f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.788+0000 7f5bb0ff9640 0 client.26642 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:55.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bd25eb740 1 -- 172.21.15.83:0/3489116912 shutdown_connections 2024-03-11T19:48:55.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bd25eb740 1 -- 172.21.15.83:0/3489116912 wait complete. 2024-03-11T19:48:55.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bd25eb740 20 client.26642 trim_cache size 0 max 16384 2024-03-11T19:48:55.798 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:48:55.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2297692681 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5bb40a1a30 msgr2=0x7f5bb40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bcf93e640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5bb40a1a30 0x7f5bb40a3ef0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5bc81962e0 tx=0x7f5bb8009f90 comp rx=0 tx=0).stop 2024-03-11T19:48:55.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2297692681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bc81013b0 msgr2=0x7f5bc819c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:55.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bcf93e640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bc81013b0 0x7f5bc819c380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0037040 tx=0x7f5bc0034bb0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2297692681 shutdown_connections 2024-03-11T19:48:55.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bcf93e640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5bb40a1a30 0x7f5bb40a3ef0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bcf93e640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bc81013b0 0x7f5bc819c380 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bcf93e640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bc81009f0 0x7f5bc8195310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bcf93e640 1 --2- 172.21.15.83:0/2297692681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bc80ff800 0x7f5bc8194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:55.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.789+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2297692681 >> 172.21.15.83:0/2297692681 conn(0x7f5bc80fad70 msgr2=0x7f5bc8103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:55.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.790+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2297692681 shutdown_connections 2024-03-11T19:48:55.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:55.790+0000 7f5bcf93e640 1 -- 172.21.15.83:0/2297692681 wait complete. 2024-03-11T19:48:56.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:55 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2991141829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:48:56.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:55 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2991141829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:48:56.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:55 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2991141829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:48:56.809 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:48:56.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:48:56.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:48:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.943+0000 7fcaba4db640 1 -- 172.21.15.83:0/1020562730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab4102380 msgr2=0x7fcab4102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.943+0000 7fcaba4db640 1 --2- 172.21.15.83:0/1020562730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab4102380 0x7fcab4102800 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009900 tx=0x7fcaa802bc50 comp rx=0 tx=0).stop 2024-03-11T19:48:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.943+0000 7fcaba4db640 1 -- 172.21.15.83:0/1020562730 shutdown_connections 2024-03-11T19:48:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.943+0000 7fcaba4db640 1 --2- 172.21.15.83:0/1020562730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcab4102d40 0x7fcab41095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.943+0000 7fcaba4db640 1 --2- 172.21.15.83:0/1020562730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab4102380 0x7fcab4102800 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.943+0000 7fcaba4db640 1 --2- 172.21.15.83:0/1020562730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab4101190 0x7fcab41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.943+0000 7fcaba4db640 1 -- 172.21.15.83:0/1020562730 >> 172.21.15.83:0/1020562730 conn(0x7fcab40fc6e0 msgr2=0x7fcab40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:56.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.943+0000 7fcaba4db640 1 -- 172.21.15.83:0/1020562730 shutdown_connections 2024-03-11T19:48:56.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.943+0000 7fcaba4db640 1 -- 172.21.15.83:0/1020562730 wait complete. 2024-03-11T19:48:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcaba4db640 1 Processor -- start 2024-03-11T19:48:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcaba4db640 1 -- start start 2024-03-11T19:48:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcaba4db640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab4101190 0x7fcab4195070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcaba4db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab4102380 0x7fcab41955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcaba4db640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcab4102d40 0x7fcab419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcaba4db640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab419cb60 con 0x7fcab4102380 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcaba4db640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab419ccd0 con 0x7fcab4101190 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcaba4db640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab419ce90 con 0x7fcab4102d40 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcab37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab4102380 0x7fcab41955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcab37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab4102380 0x7fcab41955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39454/0 (socket says 172.21.15.83:39454) 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcab37fe640 1 -- 172.21.15.83:0/1382691908 learned_addr learned my addr 172.21.15.83:0/1382691908 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcab8a78640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcab4102d40 0x7fcab419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.944+0000 7fcab3fff640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab4101190 0x7fcab4195070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.945+0000 7fcab37fe640 1 -- 172.21.15.83:0/1382691908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcab4102d40 msgr2=0x7fcab419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.945+0000 7fcab37fe640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcab4102d40 0x7fcab419c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.945+0000 7fcab37fe640 1 -- 172.21.15.83:0/1382691908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab4101190 msgr2=0x7fcab4195070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.945+0000 7fcab37fe640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab4101190 0x7fcab4195070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.945+0000 7fcab37fe640 1 -- 172.21.15.83:0/1382691908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa8009560 con 0x7fcab4102380 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.945+0000 7fcab37fe640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab4102380 0x7fcab41955b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fcaa802c890 tx=0x7fcaa8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.945+0000 7fcab17fa640 1 -- 172.21.15.83:0/1382691908 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8004aa0 con 0x7fcab4102380 2024-03-11T19:48:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.945+0000 7fcaba4db640 1 -- 172.21.15.83:0/1382691908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab419d110 con 0x7fcab4102380 2024-03-11T19:48:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.945+0000 7fcab17fa640 1 -- 172.21.15.83:0/1382691908 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcaa80052d0 con 0x7fcab4102380 2024-03-11T19:48:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.945+0000 7fcab17fa640 1 -- 172.21.15.83:0/1382691908 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8003d90 con 0x7fcab4102380 2024-03-11T19:48:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.945+0000 7fcaba4db640 1 -- 172.21.15.83:0/1382691908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab419d670 con 0x7fcab4102380 2024-03-11T19:48:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.948+0000 7fcabd183740 1 Processor -- start 2024-03-11T19:48:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.948+0000 7fcab17fa640 1 -- 172.21.15.83:0/1382691908 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fcaa80035f0 con 0x7fcab4102380 2024-03-11T19:48:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.948+0000 7fcabd183740 1 -- start start 2024-03-11T19:48:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.948+0000 7fcabd183740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.948+0000 7fcabd183740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6317690 0x564ed6317b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.948+0000 7fcabd183740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6318050 0x564ed631e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.948+0000 7fcabd183740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed6285f90 con 0x564ed6318050 2024-03-11T19:48:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.948+0000 7fcabd183740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed62834c0 con 0x564ed63164c0 2024-03-11T19:48:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.948+0000 7fcabd183740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed627eca0 con 0x564ed6317690 2024-03-11T19:48:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.948+0000 7fcab37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6318050 0x564ed631e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.949+0000 7fcab8a78640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.949+0000 7fcab8a78640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63168e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52160/0 (socket says 172.21.15.83:52160) 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.949+0000 7fcab8a78640 1 -- 172.21.15.83:0/2462242027 learned_addr learned my addr 172.21.15.83:0/2462242027 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.949+0000 7fcab17fa640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca9c0a5e90 0x7fca9c0a8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.949+0000 7fcab8a78640 1 -- 172.21.15.83:0/2462242027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6317690 msgr2=0x564ed6317b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.949+0000 7fcab8a78640 1 --2- 172.21.15.83:0/2462242027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6317690 0x564ed6317b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.949+0000 7fcab8a78640 1 -- 172.21.15.83:0/2462242027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6318050 msgr2=0x564ed631e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.949+0000 7fcab8a78640 1 --2- 172.21.15.83:0/2462242027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6318050 0x564ed631e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.949+0000 7fcab8a78640 1 -- 172.21.15.83:0/2462242027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ed62410b0 con 0x564ed63164c0 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.950+0000 7fcab17fa640 1 -- 172.21.15.83:0/1382691908 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fcaa80efb40 con 0x7fcab4102380 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.950+0000 7fcab8a78640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca9c0a5e90 0x7fca9c0a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.951+0000 7fcab8a78640 1 --2- 172.21.15.83:0/2462242027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63168e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcaac011590 tx=0x7fcaac011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9204e5d5b7091189 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.951+0000 7fca9a7fc640 1 -- 172.21.15.83:0/2462242027 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac021030 con 0x564ed63164c0 2024-03-11T19:48:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.951+0000 7fca9a7fc640 1 -- 172.21.15.83:0/2462242027 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcaac003d30 con 0x564ed63164c0 2024-03-11T19:48:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.951+0000 7fca9a7fc640 1 -- 172.21.15.83:0/2462242027 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00fd50 con 0x564ed63164c0 2024-03-11T19:48:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.952+0000 7fcabd183740 1 -- 172.21.15.83:0/2462242027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 msgr2=0x564ed63168e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.952+0000 7fcab8a78640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca9c0a5e90 0x7fca9c0a8350 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcaac021000 tx=0x7fcaac00ff90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.952+0000 7fcabd183740 1 --2- 172.21.15.83:0/2462242027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63168e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcaac011590 tx=0x7fcaac011a50 comp rx=0 tx=0).stop 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.952+0000 7fcabd183740 1 -- 172.21.15.83:0/2462242027 shutdown_connections 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.952+0000 7fcabd183740 1 --2- 172.21.15.83:0/2462242027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6318050 0x564ed631e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.952+0000 7fcabd183740 1 --2- 172.21.15.83:0/2462242027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6317690 0x564ed6317b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.952+0000 7fcabd183740 1 --2- 172.21.15.83:0/2462242027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63168e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.952+0000 7fcabd183740 1 -- 172.21.15.83:0/2462242027 >> 172.21.15.83:0/2462242027 conn(0x564ed63116a0 msgr2=0x564ed6311aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.952+0000 7fcabd183740 1 -- 172.21.15.83:0/2462242027 shutdown_connections 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.952+0000 7fcabd183740 1 -- 172.21.15.83:0/2462242027 wait complete. 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.952+0000 7fcabd183740 1 -- start start 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcabd183740 1 Processor -- start 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcabd183740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed6320730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcabd183740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed6320c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcabd183740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6318050 0x564ed6327cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcabd183740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed6223f40 con 0x564ed6317690 2024-03-11T19:48:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcabd183740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed5f524b0 con 0x564ed63164c0 2024-03-11T19:48:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcabd183740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed6204670 con 0x564ed6318050 2024-03-11T19:48:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcabd183740 10 client.0 _ll_register_callbacks cb 0x564ed6271f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcab3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed6320730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcab37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed6320c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcab3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed6320730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52170/0 (socket says 172.21.15.83:52170) 2024-03-11T19:48:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcab3fff640 1 -- 172.21.15.83:0/3960495532 learned_addr learned my addr 172.21.15.83:0/3960495532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcab8a78640 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6318050 0x564ed6327cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcab3fff640 1 -- 172.21.15.83:0/3960495532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6318050 msgr2=0x564ed6327cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcab3fff640 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6318050 0x564ed6327cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcab3fff640 1 -- 172.21.15.83:0/3960495532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 msgr2=0x564ed6320c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcab3fff640 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed6320c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.953+0000 7fcab3fff640 1 -- 172.21.15.83:0/3960495532 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaac009cd0 con 0x564ed63164c0 2024-03-11T19:48:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.954+0000 7fcab8a78640 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6318050 0x564ed6327cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:48:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.954+0000 7fcab37fe640 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed6320c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.954+0000 7fcab3fff640 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed6320730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40116f0 tx=0x7fcaa4011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b29b4d091964977 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.954+0000 7fca937fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.954+0000 7fca937fe640 1 -- 172.21.15.83:0/3960495532 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4004030 con 0x564ed63164c0 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.954+0000 7fcabd183740 1 -- 172.21.15.83:0/3960495532 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ed6258f40 con 0x564ed63164c0 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.954+0000 7fca937fe640 1 -- 172.21.15.83:0/3960495532 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fcaa4004190 con 0x564ed63164c0 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.954+0000 7fca937fe640 1 -- 172.21.15.83:0/3960495532 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaa4004350 con 0x564ed63164c0 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.954+0000 7fcabd183740 10 client.26654 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.954+0000 7fcabd183740 1 -- 172.21.15.83:0/3960495532 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564ed627d320 con 0x564ed63164c0 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.954+0000 7fca937fe640 1 -- 172.21.15.83:0/3960495532 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564ed6258f40 con 0x564ed63164c0 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.955+0000 7fca937fe640 1 -- 172.21.15.83:0/3960495532 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcaa4003a30 con 0x564ed63164c0 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.955+0000 7fcabd183740 10 client.26654 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.955+0000 7fcabd183740 10 client.26654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.955+0000 7fcabd183740 20 client.26654 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.955+0000 7fcabd183740 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564ed632c980 0x564ed632eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.955+0000 7fcabd183740 4 client.26654 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.955+0000 7fcabd183740 1 -- 172.21.15.83:0/3960495532 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ed6275310 con 0x564ed632c980 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.955+0000 7fcab37fe640 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564ed632c980 0x564ed632eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.955+0000 7fca937fe640 10 client.26654 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.955+0000 7fcab37fe640 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564ed632c980 0x564ed632eda0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d320376f03b13c23 server_cookie=1303d58af5623056 in_seq=0 out_seq=0 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fca937fe640 1 -- 172.21.15.83:0/3960495532 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564ed6275310 con 0x564ed632c980 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fca937fe640 10 client.26654 handle_command_reply: tid=0 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 1 client.26654 shutdown 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 1 -- 172.21.15.83:0/3960495532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 msgr2=0x564ed6320730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed6320730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40116f0 tx=0x7fcaa4011af0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 1 -- 172.21.15.83:0/3960495532 shutdown_connections 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564ed632c980 0x564ed632eda0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6318050 0x564ed6327cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fca937fe640 0 client.26654 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed6320c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 1 --2- 172.21.15.83:0/3960495532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed6320730 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 1 -- 172.21.15.83:0/3960495532 >> 172.21.15.83:0/3960495532 conn(0x564ed63116a0 msgr2=0x564ed63108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:56.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 1 -- 172.21.15.83:0/3960495532 shutdown_connections 2024-03-11T19:48:56.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 1 -- 172.21.15.83:0/3960495532 wait complete. 2024-03-11T19:48:56.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.956+0000 7fcabd183740 20 client.26654 trim_cache size 0 max 16384 2024-03-11T19:48:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.996+0000 7fcabd183740 1 Processor -- start 2024-03-11T19:48:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.998+0000 7fcabd183740 1 -- start start 2024-03-11T19:48:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.998+0000 7fcabd183740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed63164c0 0x564ed6333740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.998+0000 7fcabd183740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed6327c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.998+0000 7fcabd183740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed6318050 0x564ed631b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.998+0000 7fcabd183740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed631ae40 con 0x564ed6317690 2024-03-11T19:48:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.998+0000 7fcabd183740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed6275340 con 0x564ed6318050 2024-03-11T19:48:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:56.998+0000 7fcabd183740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed63218b0 con 0x564ed63164c0 2024-03-11T19:48:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.002+0000 7fcab8a78640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed6318050 0x564ed631b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.002+0000 7fcab8a78640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed6318050 0x564ed631b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52174/0 (socket says 172.21.15.83:52174) 2024-03-11T19:48:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.002+0000 7fcab8a78640 1 -- 172.21.15.83:0/3797991757 learned_addr learned my addr 172.21.15.83:0/3797991757 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.002+0000 7fcab37fe640 1 --2- 172.21.15.83:0/3797991757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed6327c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.003+0000 7fcab37fe640 1 -- 172.21.15.83:0/3797991757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed63164c0 msgr2=0x564ed6333740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:48:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.003+0000 7fcab37fe640 1 --2- 172.21.15.83:0/3797991757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed63164c0 0x564ed6333740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.003+0000 7fcab37fe640 1 -- 172.21.15.83:0/3797991757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed6318050 msgr2=0x564ed631b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.003+0000 7fcab37fe640 1 --2- 172.21.15.83:0/3797991757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed6318050 0x564ed631b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.003+0000 7fcab37fe640 1 -- 172.21.15.83:0/3797991757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ed6219c00 con 0x564ed6317690 2024-03-11T19:48:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.004+0000 7fcab37fe640 1 --2- 172.21.15.83:0/3797991757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed6327c70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80ed740 tx=0x7fcaa80ef040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b56751c84edca35 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.004+0000 7fca99ffb640 1 -- 172.21.15.83:0/3797991757 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80c0dd0 con 0x564ed6317690 2024-03-11T19:48:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.004+0000 7fca99ffb640 1 -- 172.21.15.83:0/3797991757 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcaa80c5420 con 0x564ed6317690 2024-03-11T19:48:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.004+0000 7fca99ffb640 1 -- 172.21.15.83:0/3797991757 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80bc0d0 con 0x564ed6317690 2024-03-11T19:48:57.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.005+0000 7fcabd183740 1 -- 172.21.15.83:0/3797991757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 msgr2=0x564ed6327c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.005+0000 7fcabd183740 1 --2- 172.21.15.83:0/3797991757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed6327c70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80ed740 tx=0x7fcaa80ef040 comp rx=0 tx=0).stop 2024-03-11T19:48:57.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.008+0000 7fcabd183740 1 -- 172.21.15.83:0/3797991757 shutdown_connections 2024-03-11T19:48:57.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.008+0000 7fcabd183740 1 --2- 172.21.15.83:0/3797991757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed6318050 0x564ed631b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.008+0000 7fcabd183740 1 --2- 172.21.15.83:0/3797991757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed6327c70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.008+0000 7fcabd183740 1 --2- 172.21.15.83:0/3797991757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed63164c0 0x564ed6333740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.008+0000 7fcabd183740 1 -- 172.21.15.83:0/3797991757 >> 172.21.15.83:0/3797991757 conn(0x564ed63116a0 msgr2=0x564ed631a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:57.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.015+0000 7fcabd183740 1 -- 172.21.15.83:0/3797991757 shutdown_connections 2024-03-11T19:48:57.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.019+0000 7fcabd183740 1 -- 172.21.15.83:0/3797991757 wait complete. 2024-03-11T19:48:57.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.019+0000 7fcabd183740 1 -- start start 2024-03-11T19:48:57.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.020+0000 7fcabd183740 1 Processor -- start 2024-03-11T19:48:57.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.020+0000 7fcabd183740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.020+0000 7fcabd183740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed63243d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.020+0000 7fcabd183740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6318050 0x564ed6324910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.020+0000 7fcabd183740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed6324e50 con 0x564ed6317690 2024-03-11T19:48:57.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.020+0000 7fcabd183740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed6324fc0 con 0x564ed63164c0 2024-03-11T19:48:57.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.020+0000 7fcabd183740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ed6382eb0 con 0x564ed6318050 2024-03-11T19:48:57.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.021+0000 7fcabd183740 10 client.0 _ll_register_callbacks cb 0x7fcaac009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.024+0000 7fcab3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.024+0000 7fcab3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63278e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52188/0 (socket says 172.21.15.83:52188) 2024-03-11T19:48:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.024+0000 7fcab3fff640 1 -- 172.21.15.83:0/2046182739 learned_addr learned my addr 172.21.15.83:0/2046182739 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.024+0000 7fcab8a78640 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6318050 0x564ed6324910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.025+0000 7fcab3fff640 1 -- 172.21.15.83:0/2046182739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6318050 msgr2=0x564ed6324910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:57.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.025+0000 7fcab3fff640 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6318050 0x564ed6324910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.025+0000 7fcab3fff640 1 -- 172.21.15.83:0/2046182739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 msgr2=0x564ed63243d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:57.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.025+0000 7fcab3fff640 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed63243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.025+0000 7fcab3fff640 1 -- 172.21.15.83:0/2046182739 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab419d670 con 0x564ed63164c0 2024-03-11T19:48:57.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.025+0000 7fcab3fff640 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63278e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40116c0 tx=0x7fcaa4009e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cff4ab6850736b82 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:57.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.025+0000 7fca937fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:48:57.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.026+0000 7fca937fe640 1 -- 172.21.15.83:0/2046182739 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4005270 con 0x564ed63164c0 2024-03-11T19:48:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.026+0000 7fca937fe640 1 -- 172.21.15.83:0/2046182739 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fcaa4000e40 con 0x564ed63164c0 2024-03-11T19:48:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.027+0000 7fcabd183740 1 -- 172.21.15.83:0/2046182739 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ed62a4900 con 0x564ed63164c0 2024-03-11T19:48:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.027+0000 7fca937fe640 1 -- 172.21.15.83:0/2046182739 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaa40245d0 con 0x564ed63164c0 2024-03-11T19:48:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.027+0000 7fcabd183740 10 client.26660 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.027+0000 7fcabd183740 1 -- 172.21.15.83:0/2046182739 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564ed623f910 con 0x564ed63164c0 2024-03-11T19:48:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.028+0000 7fca937fe640 1 -- 172.21.15.83:0/2046182739 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564ed62a4900 con 0x564ed63164c0 2024-03-11T19:48:57.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.029+0000 7fca937fe640 1 -- 172.21.15.83:0/2046182739 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcaa4024940 con 0x564ed63164c0 2024-03-11T19:48:57.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.029+0000 7fcabd183740 10 client.26660 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:57.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.029+0000 7fcabd183740 10 client.26660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:57.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.029+0000 7fcabd183740 20 client.26660 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:57.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.029+0000 7fcabd183740 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564ed632c980 0x564ed6386950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.029+0000 7fcabd183740 4 client.26660 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:48:57.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.029+0000 7fcabd183740 1 -- 172.21.15.83:0/2046182739 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564ed5f7e860 con 0x564ed632c980 2024-03-11T19:48:57.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.031+0000 7fcab37fe640 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564ed632c980 0x564ed6386950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.038+0000 7fca937fe640 10 client.26660 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:57.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.038+0000 7fcab37fe640 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564ed632c980 0x564ed6386950 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5618ad5a83d8ca92 server_cookie=36d0668f7cb9cf81 in_seq=0 out_seq=0 2024-03-11T19:48:57.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.039+0000 7fca937fe640 1 -- 172.21.15.83:0/2046182739 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564ed5f7e860 con 0x564ed632c980 2024-03-11T19:48:57.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.039+0000 7fca937fe640 10 client.26660 handle_command_reply: tid=0 2024-03-11T19:48:57.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.039+0000 7fcabd183740 1 client.26660 shutdown 2024-03-11T19:48:57.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.041+0000 7fcabd183740 1 -- 172.21.15.83:0/2046182739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 msgr2=0x564ed63278e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.041+0000 7fcabd183740 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63278e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40116c0 tx=0x7fcaa4009e40 comp rx=0 tx=0).stop 2024-03-11T19:48:57.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.041+0000 7fcabd183740 1 -- 172.21.15.83:0/2046182739 shutdown_connections 2024-03-11T19:48:57.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.041+0000 7fcabd183740 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564ed632c980 0x564ed6386950 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.041+0000 7fcabd183740 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ed6318050 0x564ed6324910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.041+0000 7fcabd183740 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed6317690 0x564ed63243d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.041+0000 7fcabd183740 1 --2- 172.21.15.83:0/2046182739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed63164c0 0x564ed63278e0 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40116c0 tx=0x7fcaa4009e40 comp rx=0 tx=0).stop 2024-03-11T19:48:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.041+0000 7fcabd183740 1 -- 172.21.15.83:0/2046182739 >> 172.21.15.83:0/2046182739 conn(0x564ed63116a0 msgr2=0x564ed631b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:57.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.041+0000 7fca937fe640 0 client.26660 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:57.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.043+0000 7fcabd183740 1 -- 172.21.15.83:0/2046182739 shutdown_connections 2024-03-11T19:48:57.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.043+0000 7fcabd183740 1 -- 172.21.15.83:0/2046182739 wait complete. 2024-03-11T19:48:57.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.043+0000 7fcabd183740 20 client.26660 trim_cache size 0 max 16384 2024-03-11T19:48:57.048 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:48:57.048 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:48:57.048 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "babb0012-6d3b-47c2-9723-ae6e71f43ed0", 2024-03-11T19:48:57.048 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:48:57.048 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:48:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.051+0000 7fcaba4db640 1 -- 172.21.15.83:0/1382691908 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca9c0a5e90 msgr2=0x7fca9c0a8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.051+0000 7fcaba4db640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca9c0a5e90 0x7fca9c0a8350 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcaac021000 tx=0x7fcaac00ff90 comp rx=0 tx=0).stop 2024-03-11T19:48:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.051+0000 7fcaba4db640 1 -- 172.21.15.83:0/1382691908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab4102380 msgr2=0x7fcab41955b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.051+0000 7fcaba4db640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab4102380 0x7fcab41955b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fcaa802c890 tx=0x7fcaa8004270 comp rx=0 tx=0).stop 2024-03-11T19:48:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.051+0000 7fcaba4db640 1 -- 172.21.15.83:0/1382691908 shutdown_connections 2024-03-11T19:48:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.051+0000 7fcaba4db640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fca9c0a5e90 0x7fca9c0a8350 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.051+0000 7fcaba4db640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcab4102d40 0x7fcab419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.051+0000 7fcaba4db640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcab4102380 0x7fcab41955b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.051+0000 7fcaba4db640 1 --2- 172.21.15.83:0/1382691908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcab4101190 0x7fcab4195070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.051+0000 7fcaba4db640 1 -- 172.21.15.83:0/1382691908 >> 172.21.15.83:0/1382691908 conn(0x7fcab40fc6e0 msgr2=0x7fcab4105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.051+0000 7fcaba4db640 1 -- 172.21.15.83:0/1382691908 shutdown_connections 2024-03-11T19:48:57.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.052+0000 7fcaba4db640 1 -- 172.21.15.83:0/1382691908 wait complete. 2024-03-11T19:48:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:56 smithi083 ceph-mon[31864]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 608 op/s 2024-03-11T19:48:57.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:48:57.071 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:48:57.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:56 smithi137 ceph-mon[36220]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 608 op/s 2024-03-11T19:48:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:56 smithi204 ceph-mon[36739]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 12 MiB/s wr, 608 op/s 2024-03-11T19:48:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.221+0000 7f21bec9d640 1 -- 172.21.15.83:0/1739794442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b8101270 msgr2=0x7f21b81016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.222+0000 7f21bec9d640 1 --2- 172.21.15.83:0/1739794442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b8101270 0x7f21b81016f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009900 tx=0x7f21a402bc30 comp rx=0 tx=0).stop 2024-03-11T19:48:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.222+0000 7f21bec9d640 1 -- 172.21.15.83:0/1739794442 shutdown_connections 2024-03-11T19:48:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.222+0000 7f21bec9d640 1 --2- 172.21.15.83:0/1739794442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21b8101c30 0x7f21b81084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.222+0000 7f21bec9d640 1 --2- 172.21.15.83:0/1739794442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b8101270 0x7f21b81016f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.222+0000 7f21bec9d640 1 --2- 172.21.15.83:0/1739794442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b8100080 0x7f21b81004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.222+0000 7f21bec9d640 1 -- 172.21.15.83:0/1739794442 >> 172.21.15.83:0/1739794442 conn(0x7f21b80fb5f0 msgr2=0x7f21b80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.222+0000 7f21bec9d640 1 -- 172.21.15.83:0/1739794442 shutdown_connections 2024-03-11T19:48:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.222+0000 7f21bec9d640 1 -- 172.21.15.83:0/1739794442 wait complete. 2024-03-11T19:48:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.222+0000 7f21bec9d640 1 Processor -- start 2024-03-11T19:48:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.222+0000 7f21bec9d640 1 -- start start 2024-03-11T19:48:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21bec9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21b8100080 0x7f21b8199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21bec9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b8101270 0x7f21b81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21bec9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b8101c30 0x7f21b81a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21b7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b8101270 0x7f21b81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21b7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b8101270 0x7f21b81999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:60146/0 (socket says 172.21.15.83:60146) 2024-03-11T19:48:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21b7fff640 1 -- 172.21.15.83:0/2983557892 learned_addr learned my addr 172.21.15.83:0/2983557892 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21bec9d640 1 -- 172.21.15.83:0/2983557892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b81a0f70 con 0x7f21b8101c30 2024-03-11T19:48:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21bec9d640 1 -- 172.21.15.83:0/2983557892 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b81a10e0 con 0x7f21b8100080 2024-03-11T19:48:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21bec9d640 1 -- 172.21.15.83:0/2983557892 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b81a12a0 con 0x7f21b8101270 2024-03-11T19:48:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21bca39640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21b8100080 0x7f21b8199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21b7fff640 1 -- 172.21.15.83:0/2983557892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21b8100080 msgr2=0x7f21b8199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21b7fff640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21b8100080 0x7f21b8199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21b7fff640 1 -- 172.21.15.83:0/2983557892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b8101c30 msgr2=0x7f21b81a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21b7fff640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b8101c30 0x7f21b81a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.223+0000 7f21b7fff640 1 -- 172.21.15.83:0/2983557892 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a4009560 con 0x7f21b8101270 2024-03-11T19:48:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.224+0000 7f21b7fff640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b8101270 0x7f21b81999c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f21a402c840 tx=0x7f21a40040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:57.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.224+0000 7f21b5ffb640 1 -- 172.21.15.83:0/2983557892 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a40044c0 con 0x7f21b8101270 2024-03-11T19:48:57.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.224+0000 7f21b5ffb640 1 -- 172.21.15.83:0/2983557892 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21a40052d0 con 0x7f21b8101270 2024-03-11T19:48:57.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.225+0000 7f21b5ffb640 1 -- 172.21.15.83:0/2983557892 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4003e80 con 0x7f21b8101270 2024-03-11T19:48:57.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.225+0000 7f21bec9d640 1 -- 172.21.15.83:0/2983557892 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b81a1520 con 0x7f21b8101270 2024-03-11T19:48:57.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.226+0000 7f21bec9d640 1 -- 172.21.15.83:0/2983557892 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b81a19b0 con 0x7f21b8101270 2024-03-11T19:48:57.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.228+0000 7f21b5ffb640 1 -- 172.21.15.83:0/2983557892 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f21a4005440 con 0x7f21b8101270 2024-03-11T19:48:57.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.228+0000 7f21b5ffb640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21a80a1910 0x7f21a80a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.228+0000 7f21c187b740 1 Processor -- start 2024-03-11T19:48:57.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.228+0000 7f21c187b740 1 -- start start 2024-03-11T19:48:57.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.228+0000 7f21bca39640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21a80a1910 0x7f21a80a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.228+0000 7f21b5ffb640 1 -- 172.21.15.83:0/2983557892 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f21a40ef4f0 con 0x7f21b8101270 2024-03-11T19:48:57.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.228+0000 7f21c187b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 0x55db4c2986f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.228+0000 7f21c187b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c298cc0 0x55db4c29d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.228+0000 7f21c187b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c29de30 0x55db4c2a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21c187b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4c208280 con 0x55db4c29de30 2024-03-11T19:48:57.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21c187b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4c2057b0 con 0x55db4c2982d0 2024-03-11T19:48:57.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21c187b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4c200f90 con 0x55db4c298cc0 2024-03-11T19:48:57.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21bd23a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c29de30 0x55db4c2a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21b7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c298cc0 0x55db4c29d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21b7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c298cc0 0x55db4c29d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:60152/0 (socket says 172.21.15.83:60152) 2024-03-11T19:48:57.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21b7fff640 1 -- 172.21.15.83:0/2007254929 learned_addr learned my addr 172.21.15.83:0/2007254929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:57.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21bca39640 1 --2- 172.21.15.83:0/2007254929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 0x55db4c2986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21b7fff640 1 -- 172.21.15.83:0/2007254929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 msgr2=0x55db4c2986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21b7fff640 1 --2- 172.21.15.83:0/2007254929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 0x55db4c2986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21b7fff640 1 -- 172.21.15.83:0/2007254929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c29de30 msgr2=0x55db4c2a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:57.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21b7fff640 1 --2- 172.21.15.83:0/2007254929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c29de30 0x55db4c2a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21b7fff640 1 -- 172.21.15.83:0/2007254929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db4c1bf570 con 0x55db4c298cc0 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21bca39640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21a80a1910 0x7f21a80a3dd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f21b81010d0 tx=0x7f21ac014950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21b7fff640 1 --2- 172.21.15.83:0/2007254929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c298cc0 0x55db4c29d7c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f21a40b8ec0 tx=0x7f21a402c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3123cc8d0605fd76 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.229+0000 7f21a2ffd640 1 -- 172.21.15.83:0/2007254929 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a40bc4f0 con 0x55db4c298cc0 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.230+0000 7f21a2ffd640 1 -- 172.21.15.83:0/2007254929 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21a40c0640 con 0x55db4c298cc0 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.230+0000 7f21a2ffd640 1 -- 172.21.15.83:0/2007254929 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a40b8b60 con 0x55db4c298cc0 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.230+0000 7f21c187b740 1 -- 172.21.15.83:0/2007254929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c298cc0 msgr2=0x55db4c29d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.230+0000 7f21c187b740 1 --2- 172.21.15.83:0/2007254929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c298cc0 0x55db4c29d7c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f21a40b8ec0 tx=0x7f21a402c6d0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.230+0000 7f21c187b740 1 -- 172.21.15.83:0/2007254929 shutdown_connections 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.230+0000 7f21c187b740 1 --2- 172.21.15.83:0/2007254929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c29de30 0x55db4c2a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.230+0000 7f21c187b740 1 --2- 172.21.15.83:0/2007254929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c298cc0 0x55db4c29d7c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.230+0000 7f21c187b740 1 --2- 172.21.15.83:0/2007254929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 0x55db4c2986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.230+0000 7f21c187b740 1 -- 172.21.15.83:0/2007254929 >> 172.21.15.83:0/2007254929 conn(0x55db4c2937f0 msgr2=0x55db4c293bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.231+0000 7f21c187b740 1 -- 172.21.15.83:0/2007254929 shutdown_connections 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.231+0000 7f21c187b740 1 -- 172.21.15.83:0/2007254929 wait complete. 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.231+0000 7f21c187b740 1 -- start start 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.231+0000 7f21c187b740 1 Processor -- start 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.231+0000 7f21c187b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 0x55db4c2a2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.231+0000 7f21c187b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c2a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21c187b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c29de30 0x55db4c2a9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21c187b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4bed4430 con 0x55db4c298cc0 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21c187b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4be90e00 con 0x55db4c2982d0 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21c187b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4c185fd0 con 0x55db4c29de30 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21bca39640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 0x55db4c2a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c2a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21bca39640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 0x55db4c2a2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:52218/0 (socket says 172.21.15.83:52218) 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21bca39640 1 -- 172.21.15.83:0/3958677591 learned_addr learned my addr 172.21.15.83:0/3958677591 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:57.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21bd23a640 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c29de30 0x55db4c2a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21c187b740 10 client.0 _ll_register_callbacks cb 0x55db4c1dbfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21bca39640 1 -- 172.21.15.83:0/3958677591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c29de30 msgr2=0x55db4c2a9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21bca39640 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c29de30 0x55db4c2a9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21bca39640 1 -- 172.21.15.83:0/3958677591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 msgr2=0x55db4c2a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21bca39640 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c2a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21bca39640 1 -- 172.21.15.83:0/3958677591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b81a19b0 con 0x55db4c2982d0 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21bd23a640 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c29de30 0x55db4c2a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21b7fff640 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c2a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f21bca39640 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 0x55db4c2a2000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f21ac014150 tx=0x7f21ac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f85693781698b2e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f2197fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.232+0000 7f2197fff640 1 -- 172.21.15.83:0/3958677591 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac020850 con 0x55db4c2982d0 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f21c187b740 1 -- 172.21.15.83:0/3958677591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db4bf007e0 con 0x55db4c2982d0 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f2197fff640 1 -- 172.21.15.83:0/3958677591 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f21ac0209b0 con 0x55db4c2982d0 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f2197fff640 1 -- 172.21.15.83:0/3958677591 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21ac014300 con 0x55db4c2982d0 2024-03-11T19:48:57.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f21c187b740 10 client.26672 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f21c187b740 1 -- 172.21.15.83:0/3958677591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55db4c1ff610 con 0x55db4c2982d0 2024-03-11T19:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f2197fff640 1 -- 172.21.15.83:0/3958677591 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55db4bf007e0 con 0x55db4c2982d0 2024-03-11T19:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f2197fff640 1 -- 172.21.15.83:0/3958677591 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f21ac026070 con 0x55db4c2982d0 2024-03-11T19:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f21c187b740 10 client.26672 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f21c187b740 10 client.26672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f21c187b740 20 client.26672 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f21c187b740 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55db4c2ae150 0x55db4c2b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f21c187b740 4 client.26672 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.233+0000 7f21c187b740 1 -- 172.21.15.83:0/3958677591 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db4bf3d720 con 0x55db4c2ae150 2024-03-11T19:48:57.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.234+0000 7f21b7fff640 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55db4c2ae150 0x55db4c2b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.235+0000 7f21b7fff640 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55db4c2ae150 0x55db4c2b0580 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e3f3158d48591e0 server_cookie=a6467b5ba18232b2 in_seq=0 out_seq=0 2024-03-11T19:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.235+0000 7f2197fff640 10 client.26672 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f2197fff640 1 -- 172.21.15.83:0/3958677591 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db4bf3d720 con 0x55db4c2ae150 2024-03-11T19:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f2197fff640 10 client.26672 handle_command_reply: tid=0 2024-03-11T19:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f21c187b740 1 client.26672 shutdown 2024-03-11T19:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f21c187b740 1 -- 172.21.15.83:0/3958677591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 msgr2=0x55db4c2a2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f21c187b740 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 0x55db4c2a2000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f21ac014150 tx=0x7f21ac004730 comp rx=0 tx=0).stop 2024-03-11T19:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f21c187b740 1 -- 172.21.15.83:0/3958677591 shutdown_connections 2024-03-11T19:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f21c187b740 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55db4c2ae150 0x55db4c2b0580 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f21c187b740 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c29de30 0x55db4c2a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f2197fff640 0 client.26672 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f21c187b740 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c2a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f21c187b740 1 --2- 172.21.15.83:0/3958677591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c2982d0 0x55db4c2a2000 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.236+0000 7f21c187b740 1 -- 172.21.15.83:0/3958677591 >> 172.21.15.83:0/3958677591 conn(0x55db4c2937f0 msgr2=0x55db4c292a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.237+0000 7f21c187b740 1 -- 172.21.15.83:0/3958677591 shutdown_connections 2024-03-11T19:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.237+0000 7f21c187b740 1 -- 172.21.15.83:0/3958677591 wait complete. 2024-03-11T19:48:57.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.237+0000 7f21c187b740 20 client.26672 trim_cache size 0 max 16384 2024-03-11T19:48:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.254+0000 7f21c187b740 1 Processor -- start 2024-03-11T19:48:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.254+0000 7f21c187b740 1 -- start start 2024-03-11T19:48:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21c187b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c2982d0 0x55db4c299fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21c187b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c29a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21c187b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c29de30 0x55db4c2a9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21c187b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4c29d880 con 0x55db4c298cc0 2024-03-11T19:48:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21c187b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4c294240 con 0x55db4c29de30 2024-03-11T19:48:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21c187b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4c2ac7c0 con 0x55db4c2982d0 2024-03-11T19:48:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c29a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c29a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39514/0 (socket says 172.21.15.83:39514) 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21b7fff640 1 -- 172.21.15.83:0/3369338267 learned_addr learned my addr 172.21.15.83:0/3369338267 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21bca39640 1 --2- 172.21.15.83:0/3369338267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c2982d0 0x55db4c299fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21bd23a640 1 --2- 172.21.15.83:0/3369338267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c29de30 0x55db4c2a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21bca39640 1 -- 172.21.15.83:0/3369338267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c29de30 msgr2=0x55db4c2a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21bca39640 1 --2- 172.21.15.83:0/3369338267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c29de30 0x55db4c2a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21bca39640 1 -- 172.21.15.83:0/3369338267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 msgr2=0x55db4c29a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21bca39640 1 --2- 172.21.15.83:0/3369338267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c29a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21bca39640 1 -- 172.21.15.83:0/3369338267 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db4c19bac0 con 0x55db4c2982d0 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21b7fff640 1 --2- 172.21.15.83:0/3369338267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c29a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.255+0000 7f21bca39640 1 --2- 172.21.15.83:0/3369338267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c2982d0 0x55db4c299fe0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00b3d0 tx=0x7f21ac00b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=239cc132022d1648 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.256+0000 7f21a27fc640 1 -- 172.21.15.83:0/3369338267 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac037070 con 0x55db4c2982d0 2024-03-11T19:48:57.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.258+0000 7f21a27fc640 1 -- 172.21.15.83:0/3369338267 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21ac01a070 con 0x55db4c2982d0 2024-03-11T19:48:57.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.258+0000 7f21a27fc640 1 -- 172.21.15.83:0/3369338267 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac011440 con 0x55db4c2982d0 2024-03-11T19:48:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.259+0000 7f21c187b740 1 -- 172.21.15.83:0/3369338267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c2982d0 msgr2=0x55db4c299fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.259+0000 7f21c187b740 1 --2- 172.21.15.83:0/3369338267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c2982d0 0x55db4c299fe0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00b3d0 tx=0x7f21ac00b570 comp rx=0 tx=0).stop 2024-03-11T19:48:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.259+0000 7f21c187b740 1 -- 172.21.15.83:0/3369338267 shutdown_connections 2024-03-11T19:48:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.259+0000 7f21c187b740 1 --2- 172.21.15.83:0/3369338267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c29de30 0x55db4c2a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.259+0000 7f21c187b740 1 --2- 172.21.15.83:0/3369338267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c29a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.259+0000 7f21c187b740 1 --2- 172.21.15.83:0/3369338267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c2982d0 0x55db4c299fe0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.259+0000 7f21c187b740 1 -- 172.21.15.83:0/3369338267 >> 172.21.15.83:0/3369338267 conn(0x55db4c2937f0 msgr2=0x55db4c2b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.259+0000 7f21c187b740 1 -- 172.21.15.83:0/3369338267 shutdown_connections 2024-03-11T19:48:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.259+0000 7f21c187b740 1 -- 172.21.15.83:0/3369338267 wait complete. 2024-03-11T19:48:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.259+0000 7f21c187b740 1 -- start start 2024-03-11T19:48:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.259+0000 7f21c187b740 1 Processor -- start 2024-03-11T19:48:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21c187b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c2982d0 0x55db4c2a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21c187b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c2a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21c187b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c29de30 0x55db4c2a7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21c187b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4c2a7b30 con 0x55db4c298cc0 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21c187b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4c2a7ca0 con 0x55db4c29de30 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21c187b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4c2a7e60 con 0x55db4c2982d0 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21c187b740 10 client.0 _ll_register_callbacks cb 0x7f21a4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c2a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21b7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c2a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39520/0 (socket says 172.21.15.83:39520) 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21b7fff640 1 -- 172.21.15.83:0/3240405401 learned_addr learned my addr 172.21.15.83:0/3240405401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21bd23a640 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c29de30 0x55db4c2a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21bca39640 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c2982d0 0x55db4c2a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21b7fff640 1 -- 172.21.15.83:0/3240405401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c2982d0 msgr2=0x55db4c2a8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21b7fff640 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c2982d0 0x55db4c2a8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21b7fff640 1 -- 172.21.15.83:0/3240405401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c29de30 msgr2=0x55db4c2a7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:48:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21b7fff640 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c29de30 0x55db4c2a7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21b7fff640 1 -- 172.21.15.83:0/3240405401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db4c1bf570 con 0x55db4c298cc0 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21b7fff640 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c2a9450 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f21a4005ab0 tx=0x7f21a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235028a939fae35f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f2197fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f2197fff640 1 -- 172.21.15.83:0/3240405401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a40a1070 con 0x55db4c298cc0 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f2197fff640 1 -- 172.21.15.83:0/3240405401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f21a40bcbf0 con 0x55db4c298cc0 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.260+0000 7f21c187b740 1 -- 172.21.15.83:0/3240405401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21a40c0640 con 0x55db4c298cc0 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f2197fff640 1 -- 172.21.15.83:0/3240405401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21a40bc4f0 con 0x55db4c298cc0 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f21c187b740 10 client.17217 fetch_fsmap learned FSMap version 32 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f21c187b740 1 -- 172.21.15.83:0/3240405401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55db4c1ff610 con 0x55db4c298cc0 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f2197fff640 1 -- 172.21.15.83:0/3240405401 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21a40c0640 con 0x55db4c298cc0 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f2197fff640 1 -- 172.21.15.83:0/3240405401 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f21a40a4030 con 0x55db4c298cc0 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f21c187b740 10 client.17217 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f21c187b740 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f21c187b740 20 client.17217 populate_metadata read hostname 'smithi083' 2024-03-11T19:48:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f21c187b740 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55db4c2ae150 0x55db4c307720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f21c187b740 4 client.17217 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f21c187b740 1 -- 172.21.15.83:0/3240405401 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55db4bf8c890 con 0x55db4c2ae150 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.261+0000 7f21bca39640 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55db4c2ae150 0x55db4c307720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f2197fff640 10 client.17217 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21bca39640 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55db4c2ae150 0x55db4c307720 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d877e2cdebded23 server_cookie=cb8ab3c2d017defe in_seq=0 out_seq=0 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f2197fff640 1 -- 172.21.15.83:0/3240405401 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55db4bf8c890 con 0x55db4c2ae150 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f2197fff640 10 client.17217 handle_command_reply: tid=0 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21c187b740 1 client.17217 shutdown 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21c187b740 1 -- 172.21.15.83:0/3240405401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 msgr2=0x55db4c2a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21c187b740 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c2a9450 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f21a4005ab0 tx=0x7f21a4009260 comp rx=0 tx=0).stop 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21c187b740 1 -- 172.21.15.83:0/3240405401 shutdown_connections 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21c187b740 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55db4c2ae150 0x55db4c307720 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21c187b740 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db4c29de30 0x55db4c2a7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21c187b740 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db4c298cc0 0x55db4c2a9450 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21c187b740 1 --2- 172.21.15.83:0/3240405401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55db4c2982d0 0x55db4c2a8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21c187b740 1 -- 172.21.15.83:0/3240405401 >> 172.21.15.83:0/3240405401 conn(0x55db4c2937f0 msgr2=0x55db4c299fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f2197fff640 0 client.17217 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:48:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21c187b740 1 -- 172.21.15.83:0/3240405401 shutdown_connections 2024-03-11T19:48:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.262+0000 7f21c187b740 1 -- 172.21.15.83:0/3240405401 wait complete. 2024-03-11T19:48:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.263+0000 7f21c187b740 20 client.17217 trim_cache size 0 max 16384 2024-03-11T19:48:57.271 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:48:57.271 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:48:57.271 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:48:57.271 INFO:teuthology.orchestra.run.smithi083.stdout: "babb0012-6d3b-47c2-9723-ae6e71f43ed0": { 2024-03-11T19:48:57.272 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:48:57.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "babb0012-6d3b-47c2-9723-ae6e71f43ed0", 2024-03-11T19:48:57.272 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:48:57.272 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:48:57.272 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:48:57.272 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.263+0000 7f21bec9d640 1 -- 172.21.15.83:0/2983557892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21a80a1910 msgr2=0x7f21a80a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.263+0000 7f21bec9d640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21a80a1910 0x7f21a80a3dd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f21b81010d0 tx=0x7f21ac014950 comp rx=0 tx=0).stop 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.263+0000 7f21bec9d640 1 -- 172.21.15.83:0/2983557892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b8101270 msgr2=0x7f21b81999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.263+0000 7f21bec9d640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b8101270 0x7f21b81999c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f21a402c840 tx=0x7f21a40040c0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.263+0000 7f21bec9d640 1 -- 172.21.15.83:0/2983557892 shutdown_connections 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.263+0000 7f21bec9d640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21a80a1910 0x7f21a80a3dd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.263+0000 7f21bec9d640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21b8101c30 0x7f21b81a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.263+0000 7f21bec9d640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b8101270 0x7f21b81999c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.263+0000 7f21bec9d640 1 --2- 172.21.15.83:0/2983557892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21b8100080 0x7f21b8199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.263+0000 7f21bec9d640 1 -- 172.21.15.83:0/2983557892 >> 172.21.15.83:0/2983557892 conn(0x7f21b80fb5f0 msgr2=0x7f21b80682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.264+0000 7f21bec9d640 1 -- 172.21.15.83:0/2983557892 shutdown_connections 2024-03-11T19:48:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:48:57.264+0000 7f21bec9d640 1 -- 172.21.15.83:0/2983557892 wait complete. 2024-03-11T19:48:57.282 INFO:tasks.cephfs.filesystem:scrub status for tag:babb0012-6d3b-47c2-9723-ae6e71f43ed0 - {'path': '/', 'tag': 'babb0012-6d3b-47c2-9723-ae6e71f43ed0', 'options': 'recursive,force'} 2024-03-11T19:48:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 580.0 s, 1624.8 tps, lat 60.537 ms stddev 73.619 2024-03-11T19:48:58.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:57 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:48:58.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:57 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:58.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:57 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:48:58.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:57 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:48:58.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:57 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:58.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:57 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:48:58.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:57 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:58.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:57 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:57 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:48:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:57 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:57 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:48:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:57 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:48:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:57 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:57 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:48:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:57 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:58.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:57 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:57 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:48:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:57 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:57 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:48:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:57 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:48:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:57 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:57 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:48:58.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:57 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:58.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:57 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:48:59.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:48:58 smithi137 ceph-mon[36220]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 732 op/s 2024-03-11T19:48:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:48:58 smithi204 ceph-mon[36739]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 732 op/s 2024-03-11T19:48:59.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:48:58 smithi083 ceph-mon[31864]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 732 op/s 2024-03-11T19:49:01.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:00 smithi083 ceph-mon[31864]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 823 op/s 2024-03-11T19:49:01.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:00 smithi137 ceph-mon[36220]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 823 op/s 2024-03-11T19:49:01.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:00 smithi204 ceph-mon[36739]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 823 op/s 2024-03-11T19:49:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 585.0 s, 1584.0 tps, lat 64.057 ms stddev 101.380 2024-03-11T19:49:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:02 smithi083 ceph-mon[31864]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-03-11T19:49:03.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:02 smithi137 ceph-mon[36220]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-03-11T19:49:03.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:02 smithi204 ceph-mon[36739]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-03-11T19:49:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:04 smithi083 ceph-mon[31864]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 1.69k op/s 2024-03-11T19:49:05.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:04 smithi137 ceph-mon[36220]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 1.69k op/s 2024-03-11T19:49:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:04 smithi204 ceph-mon[36739]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 1.69k op/s 2024-03-11T19:49:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:06 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:49:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:06 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:49:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:06 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:49:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:06 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:49:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:06 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:49:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:06 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:49:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:06 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:49:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:06 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:49:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:06 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:49:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 590.0 s, 2303.0 tps, lat 43.152 ms stddev 55.778 2024-03-11T19:49:07.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:07 smithi083 ceph-mon[31864]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 MiB/s wr, 1.62k op/s 2024-03-11T19:49:07.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:07 smithi137 ceph-mon[36220]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 MiB/s wr, 1.62k op/s 2024-03-11T19:49:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:07 smithi204 ceph-mon[36739]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 MiB/s wr, 1.62k op/s 2024-03-11T19:49:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:08 smithi083 ceph-mon[31864]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 MiB/s wr, 1.78k op/s 2024-03-11T19:49:09.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:08 smithi137 ceph-mon[36220]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 MiB/s wr, 1.78k op/s 2024-03-11T19:49:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:08 smithi204 ceph-mon[36739]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 MiB/s wr, 1.78k op/s 2024-03-11T19:49:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:10 smithi083 ceph-mon[31864]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 1.65k op/s 2024-03-11T19:49:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:10 smithi137 ceph-mon[36220]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 1.65k op/s 2024-03-11T19:49:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:10 smithi204 ceph-mon[36739]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 MiB/s wr, 1.65k op/s 2024-03-11T19:49:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 595.0 s, 2029.2 tps, lat 49.053 ms stddev 58.779 2024-03-11T19:49:13.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:12 smithi083 ceph-mon[31864]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 1.45k op/s 2024-03-11T19:49:13.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:12 smithi137 ceph-mon[36220]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 1.45k op/s 2024-03-11T19:49:13.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:12 smithi204 ceph-mon[36739]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 1.45k op/s 2024-03-11T19:49:15.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:14 smithi083 ceph-mon[31864]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 MiB/s wr, 1.29k op/s 2024-03-11T19:49:15.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:14 smithi137 ceph-mon[36220]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 MiB/s wr, 1.29k op/s 2024-03-11T19:49:15.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:14 smithi204 ceph-mon[36739]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 MiB/s wr, 1.29k op/s 2024-03-11T19:49:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:16 smithi083 ceph-mon[31864]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.8 MiB/s wr, 631 op/s 2024-03-11T19:49:17.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:16 smithi137 ceph-mon[36220]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.8 MiB/s wr, 631 op/s 2024-03-11T19:49:17.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:16 smithi204 ceph-mon[36739]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.8 MiB/s wr, 631 op/s 2024-03-11T19:49:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 600.0 s, 2534.6 tps, lat 39.832 ms stddev 48.748 2024-03-11T19:49:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:18 smithi083 ceph-mon[31864]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 768 op/s 2024-03-11T19:49:19.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:18 smithi137 ceph-mon[36220]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 768 op/s 2024-03-11T19:49:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:18 smithi204 ceph-mon[36739]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 768 op/s 2024-03-11T19:49:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:20 smithi083 ceph-mon[31864]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 608 op/s 2024-03-11T19:49:21.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:20 smithi137 ceph-mon[36220]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 608 op/s 2024-03-11T19:49:21.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:20 smithi204 ceph-mon[36739]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 608 op/s 2024-03-11T19:49:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 605.0 s, 2084.0 tps, lat 47.851 ms stddev 58.893 2024-03-11T19:49:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:22 smithi083 ceph-mon[31864]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 MiB/s wr, 647 op/s 2024-03-11T19:49:23.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:22 smithi137 ceph-mon[36220]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 MiB/s wr, 647 op/s 2024-03-11T19:49:23.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:22 smithi204 ceph-mon[36739]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 MiB/s wr, 647 op/s 2024-03-11T19:49:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:24 smithi083 ceph-mon[31864]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 690 op/s 2024-03-11T19:49:25.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:24 smithi137 ceph-mon[36220]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 690 op/s 2024-03-11T19:49:25.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:24 smithi204 ceph-mon[36739]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 690 op/s 2024-03-11T19:49:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:26 smithi083 ceph-mon[31864]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 597 op/s 2024-03-11T19:49:27.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:26 smithi137 ceph-mon[36220]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 597 op/s 2024-03-11T19:49:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:26 smithi204 ceph-mon[36739]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 MiB/s wr, 597 op/s 2024-03-11T19:49:27.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:49:27.284 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:49:27.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.458+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1590747560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8101c00 msgr2=0x7f5fd8103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.458+0000 7f5fdc9d0640 1 --2- 172.21.15.83:0/1590747560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8101c00 0x7f5fd8103ff0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009900 tx=0x7f5fcc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:27.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.459+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1590747560 shutdown_connections 2024-03-11T19:49:27.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.459+0000 7f5fdc9d0640 1 --2- 172.21.15.83:0/1590747560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fd8106e60 0x7f5fd81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.459+0000 7f5fdc9d0640 1 --2- 172.21.15.83:0/1590747560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd8104530 0x7f5fd8106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.459+0000 7f5fdc9d0640 1 --2- 172.21.15.83:0/1590747560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8101c00 0x7f5fd8103ff0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.459+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1590747560 >> 172.21.15.83:0/1590747560 conn(0x7f5fd80fb5f0 msgr2=0x7f5fd80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.459+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1590747560 shutdown_connections 2024-03-11T19:49:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.459+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1590747560 wait complete. 2024-03-11T19:49:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fdc9d0640 1 Processor -- start 2024-03-11T19:49:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fdc9d0640 1 -- start start 2024-03-11T19:49:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fdc9d0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8101c00 0x7f5fd8199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fdc9d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fd8104530 0x7f5fd81999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fdc9d0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd8106e60 0x7f5fd81a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fdc9d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81a0f60 con 0x7f5fd8104530 2024-03-11T19:49:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fdc9d0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81a10d0 con 0x7f5fd8101c00 2024-03-11T19:49:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fdc9d0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81a1290 con 0x7f5fd8106e60 2024-03-11T19:49:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fd659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8101c00 0x7f5fd8199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fd6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd8106e60 0x7f5fd81a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fd659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8101c00 0x7f5fd8199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:49840/0 (socket says 172.21.15.83:49840) 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.460+0000 7f5fd659c640 1 -- 172.21.15.83:0/1679104078 learned_addr learned my addr 172.21.15.83:0/1679104078 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fd8104530 0x7f5fd81999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fd6d9d640 1 -- 172.21.15.83:0/1679104078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8101c00 msgr2=0x7f5fd8199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fd6d9d640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8101c00 0x7f5fd8199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fd6d9d640 1 -- 172.21.15.83:0/1679104078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fd8104530 msgr2=0x7f5fd81999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fd6d9d640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fd8104530 0x7f5fd81999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fd6d9d640 1 -- 172.21.15.83:0/1679104078 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd0009840 con 0x7f5fd8106e60 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fd6d9d640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd8106e60 0x7f5fd81a0a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000ec50 tx=0x7f5fd000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fc77fe640 1 -- 172.21.15.83:0/1679104078 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0003b70 con 0x7f5fd8106e60 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fc77fe640 1 -- 172.21.15.83:0/1679104078 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fd000ce70 con 0x7f5fd8106e60 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1679104078 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc009560 con 0x7f5fd8106e60 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fc77fe640 1 -- 172.21.15.83:0/1679104078 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00107f0 con 0x7f5fd8106e60 2024-03-11T19:49:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.461+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1679104078 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd81a18f0 con 0x7f5fd8106e60 2024-03-11T19:49:27.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.465+0000 7f5fdf5a7740 1 Processor -- start 2024-03-11T19:49:27.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.466+0000 7f5fc77fe640 1 -- 172.21.15.83:0/1679104078 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5fd0004000 con 0x7f5fd8106e60 2024-03-11T19:49:27.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.466+0000 7f5fdf5a7740 1 -- start start 2024-03-11T19:49:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.467+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd03226f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.467+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0322cc0 0x55dfd03277c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.467+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0327e30 0x55dfd032a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.467+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd0292280 con 0x55dfd0322cc0 2024-03-11T19:49:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.467+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd028f7b0 con 0x55dfd03222d0 2024-03-11T19:49:27.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.467+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd028af90 con 0x55dfd0327e30 2024-03-11T19:49:27.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.467+0000 7f5fd659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0322cc0 0x55dfd03277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.467+0000 7f5fd5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd03226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.467+0000 7f5fd5d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0327e30 0x55dfd032a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.467+0000 7f5fd659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0322cc0 0x55dfd03277c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57588/0 (socket says 172.21.15.83:57588) 2024-03-11T19:49:27.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.467+0000 7f5fd659c640 1 -- 172.21.15.83:0/406680958 learned_addr learned my addr 172.21.15.83:0/406680958 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:27.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/406680958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0327e30 msgr2=0x55dfd032a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:27.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/406680958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0327e30 0x55dfd032a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/406680958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0322cc0 msgr2=0x55dfd03277c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/406680958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0322cc0 0x55dfd03277c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fc77fe640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fc00a1b80 0x7f5fc00a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/406680958 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfd0249570 con 0x55dfd03222d0 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fd659c640 1 --2- 172.21.15.83:0/406680958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0322cc0 0x55dfd03277c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fd6d9d640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fc00a1b80 0x7f5fc00a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/406680958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd03226f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5fc80058c0 tx=0x7f5fc8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94e77abc7eba5445 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fc77fe640 1 -- 172.21.15.83:0/1679104078 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5fd0014070 con 0x7f5fd8106e60 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fc4ff9640 1 -- 172.21.15.83:0/406680958 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8014270 con 0x55dfd03222d0 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fc4ff9640 1 -- 172.21.15.83:0/406680958 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fc801a020 con 0x55dfd03222d0 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.468+0000 7f5fc4ff9640 1 -- 172.21.15.83:0/406680958 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8011df0 con 0x55dfd03222d0 2024-03-11T19:49:27.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.469+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/406680958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 msgr2=0x55dfd03226f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.469+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/406680958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd03226f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5fc80058c0 tx=0x7f5fc8005850 comp rx=0 tx=0).stop 2024-03-11T19:49:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.469+0000 7f5fd6d9d640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fc00a1b80 0x7f5fc00a4040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0097010 tx=0x7f5fd0065e90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.469+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/406680958 shutdown_connections 2024-03-11T19:49:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.469+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/406680958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0327e30 0x55dfd032a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.469+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/406680958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0322cc0 0x55dfd03277c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.469+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/406680958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd03226f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.469+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/406680958 >> 172.21.15.83:0/406680958 conn(0x55dfd031d7f0 msgr2=0x55dfd031dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.469+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/406680958 shutdown_connections 2024-03-11T19:49:27.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.469+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/406680958 wait complete. 2024-03-11T19:49:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.470+0000 7f5fdf5a7740 1 -- start start 2024-03-11T19:49:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.470+0000 7f5fdf5a7740 1 Processor -- start 2024-03-11T19:49:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.470+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd0334ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.470+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0322cc0 0x55dfd0334ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.470+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd033c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.470+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcff5e430 con 0x55dfd0327e30 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.470+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcff1ae00 con 0x55dfd03222d0 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.470+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd020ffd0 con 0x55dfd0322cc0 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.470+0000 7f5fdf5a7740 10 client.0 _ll_register_callbacks cb 0x55dfd0265fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.472+0000 7f5fd659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0322cc0 0x55dfd0334ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.472+0000 7f5fd5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd0334ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.472+0000 7f5fd5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd0334ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:49858/0 (socket says 172.21.15.83:49858) 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.472+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/1073155307 learned_addr learned my addr 172.21.15.83:0/1073155307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.472+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd033c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.472+0000 7f5fd659c640 1 -- 172.21.15.83:0/1073155307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 msgr2=0x55dfd0334ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.472+0000 7f5fd659c640 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd0334ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.472+0000 7f5fd659c640 1 -- 172.21.15.83:0/1073155307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 msgr2=0x55dfd033c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.472+0000 7f5fd659c640 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd033c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.472+0000 7f5fd659c640 1 -- 172.21.15.83:0/1073155307 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc80145c0 con 0x55dfd0322cc0 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.472+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd033c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.473+0000 7f5fd659c640 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0322cc0 0x55dfd0334ff0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55dfd0323730 tx=0x7f5fcc0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d0ab06b5d7740f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.473+0000 7f5fbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.473+0000 7f5fbdffb640 1 -- 172.21.15.83:0/1073155307 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc004240 con 0x55dfd0322cc0 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.473+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/1073155307 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfcff8a7e0 con 0x55dfd0322cc0 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.473+0000 7f5fbdffb640 1 -- 172.21.15.83:0/1073155307 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5fcc045070 con 0x55dfd0322cc0 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.473+0000 7f5fbdffb640 1 -- 172.21.15.83:0/1073155307 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fcc0043a0 con 0x55dfd0322cc0 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.473+0000 7f5fdf5a7740 10 client.26794 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.473+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/1073155307 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dfd0289610 con 0x55dfd0322cc0 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.473+0000 7f5fbdffb640 1 -- 172.21.15.83:0/1073155307 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dfcff8a7e0 con 0x55dfd0322cc0 2024-03-11T19:49:27.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.473+0000 7f5fbdffb640 1 -- 172.21.15.83:0/1073155307 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fcc03c040 con 0x55dfd0322cc0 2024-03-11T19:49:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.474+0000 7f5fdf5a7740 10 client.26794 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.474+0000 7f5fdf5a7740 10 client.26794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:49:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.474+0000 7f5fdf5a7740 20 client.26794 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.474+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dfd0340d00 0x55dfd03430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.474+0000 7f5fdf5a7740 4 client.26794 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:49:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.474+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/1073155307 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfcffc7720 con 0x55dfd0340d00 2024-03-11T19:49:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.474+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dfd0340d00 0x55dfd03430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.475+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dfd0340d00 0x55dfd03430f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47a02319a66bd1f server_cookie=a6c50815881a0ef8 in_seq=0 out_seq=0 2024-03-11T19:49:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.475+0000 7f5fbdffb640 10 client.26794 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.475+0000 7f5fbdffb640 1 -- 172.21.15.83:0/1073155307 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dfcffc7720 con 0x55dfd0340d00 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.475+0000 7f5fbdffb640 10 client.26794 handle_command_reply: tid=0 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.475+0000 7f5fdf5a7740 1 client.26794 shutdown 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.476+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/1073155307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0322cc0 msgr2=0x55dfd0334ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.476+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0322cc0 0x55dfd0334ff0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55dfd0323730 tx=0x7f5fcc0048f0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.476+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/1073155307 shutdown_connections 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.476+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dfd0340d00 0x55dfd03430f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.476+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd033c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.476+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0322cc0 0x55dfd0334ff0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.476+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/1073155307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd0334ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.477+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/1073155307 >> 172.21.15.83:0/1073155307 conn(0x55dfd031d7f0 msgr2=0x55dfd031ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.477+0000 7f5fbdffb640 0 client.26794 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.477+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/1073155307 shutdown_connections 2024-03-11T19:49:27.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.477+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/1073155307 wait complete. 2024-03-11T19:49:27.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.477+0000 7f5fdf5a7740 20 client.26794 trim_cache size 0 max 16384 2024-03-11T19:49:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 610.0 s, 2648.4 tps, lat 37.955 ms stddev 43.547 2024-03-11T19:49:27.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.503+0000 7f5fdf5a7740 1 Processor -- start 2024-03-11T19:49:27.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.503+0000 7f5fdf5a7740 1 -- start start 2024-03-11T19:49:27.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.503+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd03222d0 0x55dfd0334a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.503+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd0322cc0 0x55dfd033bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.503+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd0334fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.503+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcffc7720 con 0x55dfd0327e30 2024-03-11T19:49:27.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.503+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcff8a7e0 con 0x55dfd0322cc0 2024-03-11T19:49:27.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.503+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd03359b0 con 0x55dfd03222d0 2024-03-11T19:49:27.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.504+0000 7f5fd5d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd03222d0 0x55dfd0334a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.505+0000 7f5fd5d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd03222d0 0x55dfd0334a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57402/0 (socket says 172.21.15.83:57402) 2024-03-11T19:49:27.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.505+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/3625329514 learned_addr learned my addr 172.21.15.83:0/3625329514 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:27.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.505+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/3625329514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd0334fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.505+0000 7f5fd659c640 1 --2- 172.21.15.83:0/3625329514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd0322cc0 0x55dfd033bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.507+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/3625329514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd03222d0 msgr2=0x55dfd0334a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.507+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/3625329514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd03222d0 0x55dfd0334a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.507+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/3625329514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd0322cc0 msgr2=0x55dfd033bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.507+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/3625329514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd0322cc0 0x55dfd033bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.507+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/3625329514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfd0225ac0 con 0x55dfd0327e30 2024-03-11T19:49:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.507+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/3625329514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd0334fd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800ed50 tx=0x7f5fc8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f8a3744186a2f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.508+0000 7f5fbffff640 1 -- 172.21.15.83:0/3625329514 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8025040 con 0x55dfd0327e30 2024-03-11T19:49:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.508+0000 7f5fbffff640 1 -- 172.21.15.83:0/3625329514 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fc8011e50 con 0x55dfd0327e30 2024-03-11T19:49:27.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.508+0000 7f5fbffff640 1 -- 172.21.15.83:0/3625329514 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8004500 con 0x55dfd0327e30 2024-03-11T19:49:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.509+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/3625329514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 msgr2=0x55dfd0334fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.509+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/3625329514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd0334fd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800ed50 tx=0x7f5fc8005c50 comp rx=0 tx=0).stop 2024-03-11T19:49:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.509+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/3625329514 shutdown_connections 2024-03-11T19:49:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.509+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/3625329514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd0334fd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.509+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/3625329514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd0322cc0 0x55dfd033bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.509+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/3625329514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd03222d0 0x55dfd0334a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.509+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/3625329514 >> 172.21.15.83:0/3625329514 conn(0x55dfd031d7f0 msgr2=0x55dfd0347a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.509+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/3625329514 shutdown_connections 2024-03-11T19:49:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.509+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/3625329514 wait complete. 2024-03-11T19:49:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.509+0000 7f5fdf5a7740 1 -- start start 2024-03-11T19:49:27.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.510+0000 7f5fdf5a7740 1 Processor -- start 2024-03-11T19:49:27.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.510+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd033bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.510+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0322cc0 0x55dfd03386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.510+0000 7f5fdf5a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd0338bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.510+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd0339130 con 0x55dfd0327e30 2024-03-11T19:49:27.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.510+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd03392a0 con 0x55dfd03222d0 2024-03-11T19:49:27.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.510+0000 7f5fdf5a7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd032f020 con 0x55dfd0322cc0 2024-03-11T19:49:27.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.510+0000 7f5fdf5a7740 10 client.0 _ll_register_callbacks cb 0x7f5fc800ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:27.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.510+0000 7f5fd5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd033bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.510+0000 7f5fd5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd033bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:49882/0 (socket says 172.21.15.83:49882) 2024-03-11T19:49:27.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.510+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/4247556408 learned_addr learned my addr 172.21.15.83:0/4247556408 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:27.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.511+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd0338bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.511+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/4247556408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0322cc0 msgr2=0x55dfd03386b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:27.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.511+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0322cc0 0x55dfd03386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.511+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/4247556408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 msgr2=0x55dfd033bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:27.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.511+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd033bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.511+0000 7f5fd5d9b640 1 -- 172.21.15.83:0/4247556408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc8028040 con 0x55dfd0327e30 2024-03-11T19:49:27.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.511+0000 7f5fd5d9b640 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd0338bf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800ee70 tx=0x7f5fc800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7faf5a2723d41123 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:27.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.511+0000 7f5fbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:49:27.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.512+0000 7f5fbdffb640 1 -- 172.21.15.83:0/4247556408 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8025030 con 0x55dfd0327e30 2024-03-11T19:49:27.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.512+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/4247556408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfd0016890 con 0x55dfd0327e30 2024-03-11T19:49:27.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.512+0000 7f5fbdffb640 1 -- 172.21.15.83:0/4247556408 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5fc801b030 con 0x55dfd0327e30 2024-03-11T19:49:27.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.512+0000 7f5fbdffb640 1 -- 172.21.15.83:0/4247556408 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fc8014110 con 0x55dfd0327e30 2024-03-11T19:49:27.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.512+0000 7f5fdf5a7740 10 client.17235 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:27.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.512+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/4247556408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dfd024b6a0 con 0x55dfd0327e30 2024-03-11T19:49:27.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.513+0000 7f5fbdffb640 1 -- 172.21.15.83:0/4247556408 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dfd0016890 con 0x55dfd0327e30 2024-03-11T19:49:27.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.514+0000 7f5fbdffb640 1 -- 172.21.15.83:0/4247556408 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5fc8045070 con 0x55dfd0327e30 2024-03-11T19:49:27.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.514+0000 7f5fdf5a7740 10 client.17235 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:27.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.514+0000 7f5fdf5a7740 10 client.17235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:49:27.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.514+0000 7f5fdf5a7740 20 client.17235 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:27.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.514+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dfd0330610 0x55dfd0332a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.514+0000 7f5fdf5a7740 4 client.17235 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:49:27.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.514+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/4247556408 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dfcff794e0 con 0x55dfd0330610 2024-03-11T19:49:27.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.516+0000 7f5fd659c640 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dfd0330610 0x55dfd0332a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.526+0000 7f5fd659c640 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dfd0330610 0x55dfd0332a30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b531b042cd92036b server_cookie=56b29493f1a80e4f in_seq=0 out_seq=0 2024-03-11T19:49:27.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.526+0000 7f5fbdffb640 10 client.17235 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:27.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.530+0000 7f5fbdffb640 1 -- 172.21.15.83:0/4247556408 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55dfcff794e0 con 0x55dfd0330610 2024-03-11T19:49:27.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.530+0000 7f5fbdffb640 10 client.17235 handle_command_reply: tid=0 2024-03-11T19:49:27.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.530+0000 7f5fdf5a7740 1 client.17235 shutdown 2024-03-11T19:49:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.532+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/4247556408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 msgr2=0x55dfd0338bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.532+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd0338bf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800ee70 tx=0x7f5fc800ed80 comp rx=0 tx=0).stop 2024-03-11T19:49:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.533+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/4247556408 shutdown_connections 2024-03-11T19:49:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.533+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55dfd0330610 0x55dfd0332a30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.533+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dfd0327e30 0x55dfd0338bf0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.533+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfd0322cc0 0x55dfd03386b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.533+0000 7f5fdf5a7740 1 --2- 172.21.15.83:0/4247556408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfd03222d0 0x55dfd033bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.533+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/4247556408 >> 172.21.15.83:0/4247556408 conn(0x55dfd031d7f0 msgr2=0x55dfd0334a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:27.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.534+0000 7f5fbdffb640 0 client.17235 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:27.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.534+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/4247556408 shutdown_connections 2024-03-11T19:49:27.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.537+0000 7f5fdf5a7740 1 -- 172.21.15.83:0/4247556408 wait complete. 2024-03-11T19:49:27.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.537+0000 7f5fdf5a7740 20 client.17235 trim_cache size 0 max 16384 2024-03-11T19:49:27.542 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:49:27.542 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:49:27.542 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:49:27.542 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:49:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.539+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1679104078 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fc00a1b80 msgr2=0x7f5fc00a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.539+0000 7f5fdc9d0640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fc00a1b80 0x7f5fc00a4040 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0097010 tx=0x7f5fd0065e90 comp rx=0 tx=0).stop 2024-03-11T19:49:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.539+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1679104078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd8106e60 msgr2=0x7f5fd81a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.539+0000 7f5fdc9d0640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd8106e60 0x7f5fd81a0a20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000ec50 tx=0x7f5fd000c6a0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.541+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1679104078 shutdown_connections 2024-03-11T19:49:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.541+0000 7f5fdc9d0640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5fc00a1b80 0x7f5fc00a4040 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.541+0000 7f5fdc9d0640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd8106e60 0x7f5fd81a0a20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.541+0000 7f5fdc9d0640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fd8104530 0x7f5fd81999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.541+0000 7f5fdc9d0640 1 --2- 172.21.15.83:0/1679104078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8101c00 0x7f5fd8199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.541+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1679104078 >> 172.21.15.83:0/1679104078 conn(0x7f5fd80fb5f0 msgr2=0x7f5fd81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:27.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.542+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1679104078 shutdown_connections 2024-03-11T19:49:27.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.542+0000 7f5fdc9d0640 1 -- 172.21.15.83:0/1679104078 wait complete. 2024-03-11T19:49:27.570 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:49:27.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:49:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.728+0000 7f56dc9a0640 1 -- 172.21.15.83:0/4066526961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d810d240 msgr2=0x7f56d81180c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.728+0000 7f56dc9a0640 1 --2- 172.21.15.83:0/4066526961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d810d240 0x7f56d81180c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f56d0009ca0 tx=0x7f56d002c140 comp rx=0 tx=0).stop 2024-03-11T19:49:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.730+0000 7f56dc9a0640 1 -- 172.21.15.83:0/4066526961 shutdown_connections 2024-03-11T19:49:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.730+0000 7f56dc9a0640 1 --2- 172.21.15.83:0/4066526961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d810d240 0x7f56d81180c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.730+0000 7f56dc9a0640 1 --2- 172.21.15.83:0/4066526961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56d810a760 0x7f56d810cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.730+0000 7f56dc9a0640 1 --2- 172.21.15.83:0/4066526961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d8072f80 0x7f56d810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.730+0000 7f56dc9a0640 1 -- 172.21.15.83:0/4066526961 >> 172.21.15.83:0/4066526961 conn(0x7f56d806deb0 msgr2=0x7f56d80702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.736+0000 7f56dc9a0640 1 -- 172.21.15.83:0/4066526961 shutdown_connections 2024-03-11T19:49:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.736+0000 7f56dc9a0640 1 -- 172.21.15.83:0/4066526961 wait complete. 2024-03-11T19:49:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.736+0000 7f56dc9a0640 1 Processor -- start 2024-03-11T19:49:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.737+0000 7f56dc9a0640 1 -- start start 2024-03-11T19:49:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.737+0000 7f56dc9a0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d8072f80 0x7f56d8114d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.737+0000 7f56dc9a0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56d810a760 0x7f56d81152c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.737+0000 7f56dc9a0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d810d240 0x7f56d81b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.737+0000 7f56dc9a0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d81158c0 con 0x7f56d810a760 2024-03-11T19:49:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.737+0000 7f56dc9a0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8115a30 con 0x7f56d8072f80 2024-03-11T19:49:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.737+0000 7f56dc9a0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d81b66c0 con 0x7f56d810d240 2024-03-11T19:49:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.738+0000 7f56d6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56d810a760 0x7f56d81152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.738+0000 7f56d6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56d810a760 0x7f56d81152c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57648/0 (socket says 172.21.15.83:57648) 2024-03-11T19:49:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.738+0000 7f56d6ffd640 1 -- 172.21.15.83:0/626973030 learned_addr learned my addr 172.21.15.83:0/626973030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.738+0000 7f56d77fe640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d8072f80 0x7f56d8114d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.738+0000 7f56d7fff640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d810d240 0x7f56d81b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.738+0000 7f56d77fe640 1 -- 172.21.15.83:0/626973030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d810d240 msgr2=0x7f56d81b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.738+0000 7f56d77fe640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d810d240 0x7f56d81b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.738+0000 7f56d77fe640 1 -- 172.21.15.83:0/626973030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56d810a760 msgr2=0x7f56d81152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.738+0000 7f56d77fe640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56d810a760 0x7f56d81152c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.739+0000 7f56d77fe640 1 -- 172.21.15.83:0/626973030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d0009900 con 0x7f56d8072f80 2024-03-11T19:49:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.739+0000 7f56d77fe640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d8072f80 0x7f56d8114d80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f56c000b3c0 tx=0x7f56c000b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.741+0000 7f56d4ff9640 1 -- 172.21.15.83:0/626973030 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0011830 con 0x7f56d8072f80 2024-03-11T19:49:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.741+0000 7f56d4ff9640 1 -- 172.21.15.83:0/626973030 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56c0011e70 con 0x7f56d8072f80 2024-03-11T19:49:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.741+0000 7f56d4ff9640 1 -- 172.21.15.83:0/626973030 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c000f2e0 con 0x7f56d8072f80 2024-03-11T19:49:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.741+0000 7f56dc9a0640 1 -- 172.21.15.83:0/626973030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d81b69a0 con 0x7f56d8072f80 2024-03-11T19:49:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.741+0000 7f56dc9a0640 1 -- 172.21.15.83:0/626973030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d81b6e60 con 0x7f56d8072f80 2024-03-11T19:49:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.742+0000 7f56dc9a0640 1 -- 172.21.15.83:0/626973030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d80710c0 con 0x7f56d8072f80 2024-03-11T19:49:27.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.748+0000 7f56d4ff9640 1 -- 172.21.15.83:0/626973030 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f56c001b460 con 0x7f56d8072f80 2024-03-11T19:49:27.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.749+0000 7f56d4ff9640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56c40a1c30 0x7f56c40a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:27.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.749+0000 7f56d4ff9640 1 -- 172.21.15.83:0/626973030 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f56c00ce650 con 0x7f56d8072f80 2024-03-11T19:49:27.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.752+0000 7f56d6ffd640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56c40a1c30 0x7f56c40a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:27.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.755+0000 7f56d6ffd640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56c40a1c30 0x7f56c40a40f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f56d8116290 tx=0x7f56c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:27.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:27.757+0000 7f56d4ff9640 1 -- 172.21.15.83:0/626973030 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56c00978f0 con 0x7f56d8072f80 2024-03-11T19:49:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.092+0000 7f56dc9a0640 1 -- 172.21.15.83:0/626973030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56d811baf0 con 0x7f56d8072f80 2024-03-11T19:49:28.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.092+0000 7f56d4ff9640 1 -- 172.21.15.83:0/626973030 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f56c009afb0 con 0x7f56d8072f80 2024-03-11T19:49:28.097 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:49:28.097 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:49:28.097 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:49:28.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.095+0000 7f56dc9a0640 1 -- 172.21.15.83:0/626973030 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56c40a1c30 msgr2=0x7f56c40a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.095+0000 7f56dc9a0640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56c40a1c30 0x7f56c40a40f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f56d8116290 tx=0x7f56c8008040 comp rx=0 tx=0).stop 2024-03-11T19:49:28.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.095+0000 7f56dc9a0640 1 -- 172.21.15.83:0/626973030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d8072f80 msgr2=0x7f56d8114d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.095+0000 7f56dc9a0640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d8072f80 0x7f56d8114d80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f56c000b3c0 tx=0x7f56c000b880 comp rx=0 tx=0).stop 2024-03-11T19:49:28.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.096+0000 7f56dc9a0640 1 -- 172.21.15.83:0/626973030 shutdown_connections 2024-03-11T19:49:28.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.096+0000 7f56dc9a0640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56c40a1c30 0x7f56c40a40f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.096+0000 7f56dc9a0640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d810d240 0x7f56d81b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.096+0000 7f56dc9a0640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56d810a760 0x7f56d81152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.096+0000 7f56dc9a0640 1 --2- 172.21.15.83:0/626973030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56d8072f80 0x7f56d8114d80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.096+0000 7f56dc9a0640 1 -- 172.21.15.83:0/626973030 >> 172.21.15.83:0/626973030 conn(0x7f56d806deb0 msgr2=0x7f56d806e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.096+0000 7f56dc9a0640 1 -- 172.21.15.83:0/626973030 shutdown_connections 2024-03-11T19:49:28.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.096+0000 7f56dc9a0640 1 -- 172.21.15.83:0/626973030 wait complete. 2024-03-11T19:49:28.114 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:49:28.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:49:28.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.267+0000 7fa24f793640 1 -- 172.21.15.83:0/3491336624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa248101190 msgr2=0x7fa2481015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.267+0000 7fa24f793640 1 --2- 172.21.15.83:0/3491336624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa248101190 0x7fa2481015b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa240009ca0 tx=0x7fa24002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 -- 172.21.15.83:0/3491336624 shutdown_connections 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 --2- 172.21.15.83:0/3491336624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa248102d40 0x7fa2481095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 --2- 172.21.15.83:0/3491336624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa248102380 0x7fa248102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 --2- 172.21.15.83:0/3491336624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa248101190 0x7fa2481015b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 -- 172.21.15.83:0/3491336624 >> 172.21.15.83:0/3491336624 conn(0x7fa2480fc6e0 msgr2=0x7fa2480feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 -- 172.21.15.83:0/3491336624 shutdown_connections 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 -- 172.21.15.83:0/3491336624 wait complete. 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 Processor -- start 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 -- start start 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa248101190 0x7fa248197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa248102380 0x7fa2481977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa248102d40 0x7fa24819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24819ed60 con 0x7fa248101190 2024-03-11T19:49:28.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24819eed0 con 0x7fa248102380 2024-03-11T19:49:28.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.268+0000 7fa24f793640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24819f090 con 0x7fa248102d40 2024-03-11T19:49:28.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.269+0000 7fa24dd30640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa248102d40 0x7fa24819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.269+0000 7fa24cd2e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa248102380 0x7fa2481977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.269+0000 7fa24dd30640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa248102d40 0x7fa24819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57458/0 (socket says 172.21.15.83:57458) 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.269+0000 7fa24dd30640 1 -- 172.21.15.83:0/718891082 learned_addr learned my addr 172.21.15.83:0/718891082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.269+0000 7fa24d52f640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa248101190 0x7fa248197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.269+0000 7fa24dd30640 1 -- 172.21.15.83:0/718891082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa248102380 msgr2=0x7fa2481977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.269+0000 7fa24dd30640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa248102380 0x7fa2481977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.269+0000 7fa24dd30640 1 -- 172.21.15.83:0/718891082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa248101190 msgr2=0x7fa248197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.269+0000 7fa24dd30640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa248101190 0x7fa248197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.269+0000 7fa24dd30640 1 -- 172.21.15.83:0/718891082 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa240009900 con 0x7fa248102d40 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.270+0000 7fa24dd30640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa248102d40 0x7fa24819e820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa23800bfd0 tx=0x7fa23800dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.270+0000 7fa23e7fc640 1 -- 172.21.15.83:0/718891082 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238009a30 con 0x7fa248102d40 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.270+0000 7fa23e7fc640 1 -- 172.21.15.83:0/718891082 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa238016070 con 0x7fa248102d40 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.270+0000 7fa24f793640 1 -- 172.21.15.83:0/718891082 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa248073410 con 0x7fa248102d40 2024-03-11T19:49:28.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.270+0000 7fa23e7fc640 1 -- 172.21.15.83:0/718891082 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238004d00 con 0x7fa248102d40 2024-03-11T19:49:28.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.270+0000 7fa24f793640 1 -- 172.21.15.83:0/718891082 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2480738d0 con 0x7fa248102d40 2024-03-11T19:49:28.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.272+0000 7fa23e7fc640 1 -- 172.21.15.83:0/718891082 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa238004e60 con 0x7fa248102d40 2024-03-11T19:49:28.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.273+0000 7fa23e7fc640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2340a1a30 0x7fa2340a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.273+0000 7fa25236f740 1 Processor -- start 2024-03-11T19:49:28.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.273+0000 7fa25236f740 1 -- start start 2024-03-11T19:49:28.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.273+0000 7fa24d52f640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2340a1a30 0x7fa2340a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.273+0000 7fa23e7fc640 1 -- 172.21.15.83:0/718891082 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa2380ccbe0 con 0x7fa248102d40 2024-03-11T19:49:28.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.273+0000 7fa25236f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3ac9f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.273+0000 7fa25236f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3ac9fc60 0x555e3aca4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.273+0000 7fa25236f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3aca4dd0 0x555e3aca7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.273+0000 7fa25236f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3ac5b960 con 0x555e3ac9fc60 2024-03-11T19:49:28.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.273+0000 7fa25236f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3ac0f310 con 0x555e3ac9f270 2024-03-11T19:49:28.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.273+0000 7fa25236f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3ac0c840 con 0x555e3aca4dd0 2024-03-11T19:49:28.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24cd2e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3ac9fc60 0x555e3aca4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24cd2e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3ac9fc60 0x555e3aca4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57666/0 (socket says 172.21.15.83:57666) 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24cd2e640 1 -- 172.21.15.83:0/3324870155 learned_addr learned my addr 172.21.15.83:0/3324870155 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24d52f640 1 --2- 172.21.15.83:0/3324870155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3ac9f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24dd30640 1 --2- 172.21.15.83:0/3324870155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3aca4dd0 0x555e3aca7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24d52f640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2340a1a30 0x7fa2340a3ef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa24002c700 tx=0x7fa240009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24cd2e640 1 -- 172.21.15.83:0/3324870155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3aca4dd0 msgr2=0x555e3aca7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24cd2e640 1 --2- 172.21.15.83:0/3324870155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3aca4dd0 0x555e3aca7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24cd2e640 1 -- 172.21.15.83:0/3324870155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 msgr2=0x555e3ac9f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24cd2e640 1 --2- 172.21.15.83:0/3324870155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3ac9f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24cd2e640 1 -- 172.21.15.83:0/3324870155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e3abca060 con 0x555e3ac9fc60 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24d52f640 1 --2- 172.21.15.83:0/3324870155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3ac9f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa24cd2e640 1 --2- 172.21.15.83:0/3324870155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3ac9fc60 0x555e3aca4760 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa244009c00 tx=0x7fa244010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a9d66abfd21c73 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa2337fe640 1 -- 172.21.15.83:0/3324870155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244009700 con 0x555e3ac9fc60 2024-03-11T19:49:28.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa2337fe640 1 -- 172.21.15.83:0/3324870155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa24402a8d0 con 0x555e3ac9fc60 2024-03-11T19:49:28.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.274+0000 7fa2337fe640 1 -- 172.21.15.83:0/3324870155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244022980 con 0x555e3ac9fc60 2024-03-11T19:49:28.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.275+0000 7fa25236f740 1 -- 172.21.15.83:0/3324870155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3ac9fc60 msgr2=0x555e3aca4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.275+0000 7fa25236f740 1 --2- 172.21.15.83:0/3324870155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3ac9fc60 0x555e3aca4760 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa244009c00 tx=0x7fa244010740 comp rx=0 tx=0).stop 2024-03-11T19:49:28.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.275+0000 7fa25236f740 1 -- 172.21.15.83:0/3324870155 shutdown_connections 2024-03-11T19:49:28.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.275+0000 7fa25236f740 1 --2- 172.21.15.83:0/3324870155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3aca4dd0 0x555e3aca7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.276+0000 7fa25236f740 1 --2- 172.21.15.83:0/3324870155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3ac9fc60 0x555e3aca4760 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.276+0000 7fa25236f740 1 --2- 172.21.15.83:0/3324870155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3ac9f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.276+0000 7fa25236f740 1 -- 172.21.15.83:0/3324870155 >> 172.21.15.83:0/3324870155 conn(0x555e3ac9a7e0 msgr2=0x555e3ac9abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.276+0000 7fa25236f740 1 -- 172.21.15.83:0/3324870155 shutdown_connections 2024-03-11T19:49:28.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.276+0000 7fa25236f740 1 -- 172.21.15.83:0/3324870155 wait complete. 2024-03-11T19:49:28.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.276+0000 7fa25236f740 1 -- start start 2024-03-11T19:49:28.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa25236f740 1 Processor -- start 2024-03-11T19:49:28.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa25236f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acba490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa25236f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9fc60 0x555e3acba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa25236f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acc19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa25236f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3ac08020 con 0x555e3aca4dd0 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa25236f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3a8db430 con 0x555e3ac9f270 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa25236f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3a897e00 con 0x555e3ac9fc60 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa24d52f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa24d52f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acba490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:49926/0 (socket says 172.21.15.83:49926) 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa24d52f640 1 -- 172.21.15.83:0/2511206916 learned_addr learned my addr 172.21.15.83:0/2511206916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa25236f740 10 client.0 _ll_register_callbacks cb 0x555e3abafcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa24dd30640 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acc19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.277+0000 7fa24cd2e640 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9fc60 0x555e3acba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa24d52f640 1 -- 172.21.15.83:0/2511206916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9fc60 msgr2=0x555e3acba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa24d52f640 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9fc60 0x555e3acba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa24d52f640 1 -- 172.21.15.83:0/2511206916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 msgr2=0x555e3acc19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa24d52f640 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acc19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa24d52f640 1 -- 172.21.15.83:0/2511206916 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa244009860 con 0x555e3ac9f270 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa24d52f640 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acba490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa240032710 tx=0x7fa24002c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04a258ac6a76b24 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa230ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa230ff9640 1 -- 172.21.15.83:0/2511206916 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240003d80 con 0x555e3ac9f270 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa230ff9640 1 -- 172.21.15.83:0/2511206916 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa240004040 con 0x555e3ac9f270 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa25236f740 1 -- 172.21.15.83:0/2511206916 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e3a9077e0 con 0x555e3ac9f270 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa230ff9640 1 -- 172.21.15.83:0/2511206916 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa240032b60 con 0x555e3ac9f270 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa25236f740 10 client.26705 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa25236f740 1 -- 172.21.15.83:0/2511206916 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555e3ac066a0 con 0x555e3ac9f270 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa230ff9640 1 -- 172.21.15.83:0/2511206916 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555e3a9077e0 con 0x555e3ac9f270 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa230ff9640 1 -- 172.21.15.83:0/2511206916 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa2400409c0 con 0x555e3ac9f270 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa25236f740 10 client.26705 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.278+0000 7fa25236f740 10 client.26705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:49:28.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.279+0000 7fa25236f740 20 client.26705 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.279+0000 7fa25236f740 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555e3acc6650 0x555e3acc8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.279+0000 7fa25236f740 4 client.26705 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.279+0000 7fa25236f740 1 -- 172.21.15.83:0/2511206916 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e3a944720 con 0x555e3acc6650 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.279+0000 7fa24cd2e640 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555e3acc6650 0x555e3acc8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.279+0000 7fa230ff9640 10 client.26705 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.279+0000 7fa24cd2e640 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555e3acc6650 0x555e3acc8a90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95302a63c59fd17b server_cookie=d4c8176c65d6b59d in_seq=0 out_seq=0 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa230ff9640 1 -- 172.21.15.83:0/2511206916 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e3a944720 con 0x555e3acc6650 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa230ff9640 10 client.26705 handle_command_reply: tid=0 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa25236f740 1 client.26705 shutdown 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa25236f740 1 -- 172.21.15.83:0/2511206916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 msgr2=0x555e3acba490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa25236f740 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acba490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa240032710 tx=0x7fa24002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa25236f740 1 -- 172.21.15.83:0/2511206916 shutdown_connections 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa25236f740 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555e3acc6650 0x555e3acc8a90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa25236f740 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acc19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa230ff9640 0 client.26705 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:49:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa25236f740 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9fc60 0x555e3acba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa25236f740 1 --2- 172.21.15.83:0/2511206916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acba490 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa25236f740 1 -- 172.21.15.83:0/2511206916 >> 172.21.15.83:0/2511206916 conn(0x555e3ac9a7e0 msgr2=0x555e3ac99b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa25236f740 1 -- 172.21.15.83:0/2511206916 shutdown_connections 2024-03-11T19:49:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.280+0000 7fa25236f740 1 -- 172.21.15.83:0/2511206916 wait complete. 2024-03-11T19:49:28.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.281+0000 7fa25236f740 20 client.26705 trim_cache size 0 max 16384 2024-03-11T19:49:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa25236f740 1 Processor -- start 2024-03-11T19:49:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa25236f740 1 -- start start 2024-03-11T19:49:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa25236f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9f270 0x555e3ac9abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa25236f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9fc60 0x555e3acc1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa25236f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa25236f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3ab8d130 con 0x555e3aca4dd0 2024-03-11T19:49:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa25236f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3a944720 con 0x555e3ac9fc60 2024-03-11T19:49:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa25236f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3ac8ca40 con 0x555e3ac9f270 2024-03-11T19:49:28.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa24d52f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9f270 0x555e3ac9abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa24d52f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9f270 0x555e3ac9abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57484/0 (socket says 172.21.15.83:57484) 2024-03-11T19:49:28.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa24dd30640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa24d52f640 1 -- 172.21.15.83:0/3456056202 learned_addr learned my addr 172.21.15.83:0/3456056202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.299+0000 7fa24cd2e640 1 --2- 172.21.15.83:0/3456056202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9fc60 0x555e3acc1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.300+0000 7fa24d52f640 1 -- 172.21.15.83:0/3456056202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9fc60 msgr2=0x555e3acc1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.300+0000 7fa24d52f640 1 --2- 172.21.15.83:0/3456056202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9fc60 0x555e3acc1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.300+0000 7fa24d52f640 1 -- 172.21.15.83:0/3456056202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 msgr2=0x555e3acba0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.300+0000 7fa24d52f640 1 --2- 172.21.15.83:0/3456056202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acba0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.300+0000 7fa24d52f640 1 -- 172.21.15.83:0/3456056202 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e3aba2c20 con 0x555e3ac9f270 2024-03-11T19:49:28.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.300+0000 7fa24cd2e640 1 --2- 172.21.15.83:0/3456056202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9fc60 0x555e3acc1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:49:28.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.300+0000 7fa24dd30640 1 --2- 172.21.15.83:0/3456056202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acba0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:28.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.300+0000 7fa24d52f640 1 --2- 172.21.15.83:0/3456056202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9f270 0x555e3ac9abc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa240009ca0 tx=0x7fa240009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38587dfc920d5d95 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.301+0000 7fa232ffd640 1 -- 172.21.15.83:0/3456056202 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240040630 con 0x555e3ac9f270 2024-03-11T19:49:28.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.301+0000 7fa232ffd640 1 -- 172.21.15.83:0/3456056202 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa240032b60 con 0x555e3ac9f270 2024-03-11T19:49:28.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.301+0000 7fa232ffd640 1 -- 172.21.15.83:0/3456056202 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240054e30 con 0x555e3ac9f270 2024-03-11T19:49:28.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.302+0000 7fa25236f740 1 -- 172.21.15.83:0/3456056202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9f270 msgr2=0x555e3ac9abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.302+0000 7fa25236f740 1 --2- 172.21.15.83:0/3456056202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9f270 0x555e3ac9abc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa240009ca0 tx=0x7fa240009ba0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.302+0000 7fa25236f740 1 -- 172.21.15.83:0/3456056202 shutdown_connections 2024-03-11T19:49:28.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.302+0000 7fa25236f740 1 --2- 172.21.15.83:0/3456056202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acba0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.302+0000 7fa25236f740 1 --2- 172.21.15.83:0/3456056202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9fc60 0x555e3acc1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.302+0000 7fa25236f740 1 --2- 172.21.15.83:0/3456056202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9f270 0x555e3ac9abc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.302+0000 7fa25236f740 1 -- 172.21.15.83:0/3456056202 >> 172.21.15.83:0/3456056202 conn(0x555e3ac9a7e0 msgr2=0x555e3ac95a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.302+0000 7fa25236f740 1 -- 172.21.15.83:0/3456056202 shutdown_connections 2024-03-11T19:49:28.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.302+0000 7fa25236f740 1 -- 172.21.15.83:0/3456056202 wait complete. 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa25236f740 1 -- start start 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa25236f740 1 Processor -- start 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa25236f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acc17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa25236f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9fc60 0x555e3acbfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa25236f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acc0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa25236f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3acc0a40 con 0x555e3aca4dd0 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa25236f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3acc0bb0 con 0x555e3ac9f270 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa25236f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e3ad1cba0 con 0x555e3ac9fc60 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa25236f740 10 client.0 _ll_register_callbacks cb 0x7fa244009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa24d52f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acc17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa24dd30640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acc0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa24d52f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acc17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:49934/0 (socket says 172.21.15.83:49934) 2024-03-11T19:49:28.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa24d52f640 1 -- 172.21.15.83:0/2129252077 learned_addr learned my addr 172.21.15.83:0/2129252077 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.303+0000 7fa24cd2e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9fc60 0x555e3acbfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa24d52f640 1 -- 172.21.15.83:0/2129252077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9fc60 msgr2=0x555e3acbfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa24d52f640 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9fc60 0x555e3acbfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa24d52f640 1 -- 172.21.15.83:0/2129252077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 msgr2=0x555e3acc0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa24d52f640 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acc0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa24d52f640 1 -- 172.21.15.83:0/2129252077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2480738d0 con 0x555e3ac9f270 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa24dd30640 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acc0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa24d52f640 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acc17f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa240042010 tx=0x7fa240054eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dab3ad09297f5212 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa230ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa230ff9640 1 -- 172.21.15.83:0/2129252077 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240040630 con 0x555e3ac9f270 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa230ff9640 1 -- 172.21.15.83:0/2129252077 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa240054790 con 0x555e3ac9f270 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa25236f740 1 -- 172.21.15.83:0/2129252077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa24402a8d0 con 0x555e3ac9f270 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa230ff9640 1 -- 172.21.15.83:0/2129252077 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa240040790 con 0x555e3ac9f270 2024-03-11T19:49:28.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa25236f740 10 client.26711 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa25236f740 1 -- 172.21.15.83:0/2129252077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555e3ac066a0 con 0x555e3ac9f270 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.304+0000 7fa230ff9640 1 -- 172.21.15.83:0/2129252077 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa24402a8d0 con 0x555e3ac9f270 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa230ff9640 1 -- 172.21.15.83:0/2129252077 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa2400673f0 con 0x555e3ac9f270 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa25236f740 10 client.26711 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa25236f740 10 client.26711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa25236f740 20 client.26711 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa25236f740 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555e3acc6650 0x555e3ad20120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa25236f740 4 client.26711 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa25236f740 1 -- 172.21.15.83:0/2129252077 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555e3a993890 con 0x555e3acc6650 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa24cd2e640 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555e3acc6650 0x555e3ad20120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa230ff9640 10 client.26711 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa24cd2e640 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555e3acc6650 0x555e3ad20120 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82d4ca5fb513afd9 server_cookie=c046082cea707813 in_seq=0 out_seq=0 2024-03-11T19:49:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa230ff9640 1 -- 172.21.15.83:0/2129252077 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555e3a993890 con 0x555e3acc6650 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa230ff9640 10 client.26711 handle_command_reply: tid=0 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.305+0000 7fa25236f740 1 client.26711 shutdown 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.306+0000 7fa25236f740 1 -- 172.21.15.83:0/2129252077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 msgr2=0x555e3acc17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.306+0000 7fa25236f740 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acc17f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa240042010 tx=0x7fa240054eb0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.306+0000 7fa25236f740 1 -- 172.21.15.83:0/2129252077 shutdown_connections 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.306+0000 7fa25236f740 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555e3acc6650 0x555e3ad20120 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.306+0000 7fa230ff9640 0 client.26711 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.306+0000 7fa25236f740 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e3aca4dd0 0x555e3acc0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.306+0000 7fa25236f740 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555e3ac9fc60 0x555e3acbfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.306+0000 7fa25236f740 1 --2- 172.21.15.83:0/2129252077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555e3ac9f270 0x555e3acc17f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.306+0000 7fa25236f740 1 -- 172.21.15.83:0/2129252077 >> 172.21.15.83:0/2129252077 conn(0x555e3ac9a7e0 msgr2=0x555e3ac9abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.307+0000 7fa25236f740 1 -- 172.21.15.83:0/2129252077 shutdown_connections 2024-03-11T19:49:28.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.307+0000 7fa25236f740 1 -- 172.21.15.83:0/2129252077 wait complete. 2024-03-11T19:49:28.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.307+0000 7fa25236f740 20 client.26711 trim_cache size 0 max 16384 2024-03-11T19:49:28.316 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.307+0000 7fa24f793640 1 -- 172.21.15.83:0/718891082 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2340a1a30 msgr2=0x7fa2340a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.307+0000 7fa24f793640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2340a1a30 0x7fa2340a3ef0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa24002c700 tx=0x7fa240009870 comp rx=0 tx=0).stop 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.307+0000 7fa24f793640 1 -- 172.21.15.83:0/718891082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa248102d40 msgr2=0x7fa24819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.307+0000 7fa24f793640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa248102d40 0x7fa24819e820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa23800bfd0 tx=0x7fa23800dbb0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.308+0000 7fa24f793640 1 -- 172.21.15.83:0/718891082 shutdown_connections 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.308+0000 7fa24f793640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2340a1a30 0x7fa2340a3ef0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.308+0000 7fa24f793640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa248102d40 0x7fa24819e820 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.308+0000 7fa24f793640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa248102380 0x7fa2481977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.308+0000 7fa24f793640 1 --2- 172.21.15.83:0/718891082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa248101190 0x7fa248197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.308+0000 7fa24f793640 1 -- 172.21.15.83:0/718891082 >> 172.21.15.83:0/718891082 conn(0x7fa2480fc6e0 msgr2=0x7fa248105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.308+0000 7fa24f793640 1 -- 172.21.15.83:0/718891082 shutdown_connections 2024-03-11T19:49:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.308+0000 7fa24f793640 1 -- 172.21.15.83:0/718891082 wait complete. 2024-03-11T19:49:28.325 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:49:28.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:49:28.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:28 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/626973030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:28.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:28 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/626973030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.475+0000 7f8755255640 1 -- 172.21.15.83:0/1943450493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8750100d20 msgr2=0x7f87501085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.475+0000 7f8755255640 1 --2- 172.21.15.83:0/1943450493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8750100d20 0x7f87501085c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8748009c10 tx=0x7f874802bef0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.475+0000 7f8755255640 1 -- 172.21.15.83:0/1943450493 shutdown_connections 2024-03-11T19:49:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.475+0000 7f8755255640 1 --2- 172.21.15.83:0/1943450493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8750100d20 0x7f87501085c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.475+0000 7f8755255640 1 --2- 172.21.15.83:0/1943450493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8750100360 0x7f87501007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.475+0000 7f8755255640 1 --2- 172.21.15.83:0/1943450493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8750101630 0x7f8750101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.475+0000 7f8755255640 1 -- 172.21.15.83:0/1943450493 >> 172.21.15.83:0/1943450493 conn(0x7f87500fadb0 msgr2=0x7f87500fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.475+0000 7f8755255640 1 -- 172.21.15.83:0/1943450493 shutdown_connections 2024-03-11T19:49:28.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.475+0000 7f8755255640 1 -- 172.21.15.83:0/1943450493 wait complete. 2024-03-11T19:49:28.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8755255640 1 Processor -- start 2024-03-11T19:49:28.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8755255640 1 -- start start 2024-03-11T19:49:28.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8755255640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8750100360 0x7f87500ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8755255640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8750100d20 0x7f875019bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8755255640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8750101630 0x7f875019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8755255640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875019ecd0 con 0x7f8750100d20 2024-03-11T19:49:28.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8755255640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875019ee40 con 0x7f8750100360 2024-03-11T19:49:28.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8755255640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875019f000 con 0x7f8750101630 2024-03-11T19:49:28.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8747fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8750100d20 0x7f875019bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8747fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8750100d20 0x7f875019bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57714/0 (socket says 172.21.15.83:57714) 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8747fff640 1 -- 172.21.15.83:0/762113307 learned_addr learned my addr 172.21.15.83:0/762113307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f874ed9d640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8750100360 0x7f87500ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f874f59e640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8750101630 0x7f875019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8747fff640 1 -- 172.21.15.83:0/762113307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8750101630 msgr2=0x7f875019e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8747fff640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8750101630 0x7f875019e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8747fff640 1 -- 172.21.15.83:0/762113307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8750100360 msgr2=0x7f87500ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8747fff640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8750100360 0x7f87500ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.476+0000 7f8747fff640 1 -- 172.21.15.83:0/762113307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8740009490 con 0x7f8750100d20 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.477+0000 7f874ed9d640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8750100360 0x7f87500ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.477+0000 7f8747fff640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8750100d20 0x7f875019bf00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f873800e9c0 tx=0x7f873800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.477+0000 7f874cd99640 1 -- 172.21.15.83:0/762113307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873800ccc0 con 0x7f8750100d20 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.478+0000 7f874cd99640 1 -- 172.21.15.83:0/762113307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f873800ce20 con 0x7f8750100d20 2024-03-11T19:49:28.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.478+0000 7f874cd99640 1 -- 172.21.15.83:0/762113307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738010430 con 0x7f8750100d20 2024-03-11T19:49:28.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.478+0000 7f8755255640 1 -- 172.21.15.83:0/762113307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8748009840 con 0x7f8750100d20 2024-03-11T19:49:28.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.478+0000 7f8755255640 1 -- 172.21.15.83:0/762113307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8750196060 con 0x7f8750100d20 2024-03-11T19:49:28.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.480+0000 7f874cd99640 1 -- 172.21.15.83:0/762113307 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8738010590 con 0x7f8750100d20 2024-03-11T19:49:28.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.481+0000 7f8757e2d740 1 Processor -- start 2024-03-11T19:49:28.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.481+0000 7f8757e2d740 1 -- start start 2024-03-11T19:49:28.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.481+0000 7f8757e2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 0x55c5e3f86690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.481+0000 7f8757e2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f8b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.481+0000 7f874cd99640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f873c0a1ab0 0x7f873c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.481+0000 7f874cd99640 1 -- 172.21.15.83:0/762113307 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8738014070 con 0x7f8750100d20 2024-03-11T19:49:28.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.482+0000 7f874ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f8b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.482+0000 7f874f59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 0x55c5e3f86690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.482+0000 7f874ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f8b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57726/0 (socket says 172.21.15.83:57726) 2024-03-11T19:49:28.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.482+0000 7f874ed9d640 1 -- 172.21.15.83:0/1299413590 learned_addr learned my addr 172.21.15.83:0/1299413590 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.482+0000 7f8757e2d740 1 --2- 172.21.15.83:0/1299413590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f8e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.482+0000 7f8757e2d740 1 -- 172.21.15.83:0/1299413590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3f42960 con 0x55c5e3f86c60 2024-03-11T19:49:28.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.482+0000 7f8757e2d740 1 -- 172.21.15.83:0/1299413590 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3ef6310 con 0x55c5e3f8bdd0 2024-03-11T19:49:28.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.482+0000 7f8757e2d740 1 -- 172.21.15.83:0/1299413590 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3ef3840 con 0x55c5e3f86270 2024-03-11T19:49:28.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.483+0000 7f874f59e640 1 -- 172.21.15.83:0/1299413590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f8bdd0 msgr2=0x55c5e3f8e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.483+0000 7f874f59e640 1 --2- 172.21.15.83:0/1299413590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f8e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.483+0000 7f874f59e640 1 -- 172.21.15.83:0/1299413590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 msgr2=0x55c5e3f8b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.483+0000 7f874f59e640 1 --2- 172.21.15.83:0/1299413590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f8b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.483+0000 7f874f59e640 1 -- 172.21.15.83:0/1299413590 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5e3eb1060 con 0x55c5e3f86270 2024-03-11T19:49:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.483+0000 7f874f59e640 1 --2- 172.21.15.83:0/1299413590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 0x55c5e3f86690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8748036040 tx=0x7f874802cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d3cc070ca0a35c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.483+0000 7f8747fff640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f873c0a1ab0 0x7f873c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.483+0000 7f87457fa640 1 -- 172.21.15.83:0/1299413590 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874802ca90 con 0x55c5e3f86270 2024-03-11T19:49:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.483+0000 7f87457fa640 1 -- 172.21.15.83:0/1299413590 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87480334c0 con 0x55c5e3f86270 2024-03-11T19:49:28.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.483+0000 7f87457fa640 1 -- 172.21.15.83:0/1299413590 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748004290 con 0x55c5e3f86270 2024-03-11T19:49:28.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.484+0000 7f8757e2d740 1 -- 172.21.15.83:0/1299413590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 msgr2=0x55c5e3f86690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.484+0000 7f8747fff640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f873c0a1ab0 0x7f873c0a3f70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8738096a10 tx=0x7f873800cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.484+0000 7f8757e2d740 1 --2- 172.21.15.83:0/1299413590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 0x55c5e3f86690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8748036040 tx=0x7f874802cd60 comp rx=0 tx=0).stop 2024-03-11T19:49:28.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.484+0000 7f8757e2d740 1 -- 172.21.15.83:0/1299413590 shutdown_connections 2024-03-11T19:49:28.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.484+0000 7f8757e2d740 1 --2- 172.21.15.83:0/1299413590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f8e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.484+0000 7f8757e2d740 1 --2- 172.21.15.83:0/1299413590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f8b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.484+0000 7f8757e2d740 1 --2- 172.21.15.83:0/1299413590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 0x55c5e3f86690 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.484+0000 7f8757e2d740 1 -- 172.21.15.83:0/1299413590 >> 172.21.15.83:0/1299413590 conn(0x55c5e3f817e0 msgr2=0x55c5e3f81be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.484+0000 7f8757e2d740 1 -- 172.21.15.83:0/1299413590 shutdown_connections 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f8757e2d740 1 -- 172.21.15.83:0/1299413590 wait complete. 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f8757e2d740 1 -- start start 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f8757e2d740 1 Processor -- start 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f8757e2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 0x55c5e3f902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f8757e2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f8757e2d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f97820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f8757e2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3eef020 con 0x55c5e3f86c60 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f8757e2d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3bc2430 con 0x55c5e3f8bdd0 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f8757e2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3b7ee00 con 0x55c5e3f86270 2024-03-11T19:49:28.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f874ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f8757e2d740 10 client.0 _ll_register_callbacks cb 0x55c5e3e96cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f874ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f907e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57742/0 (socket says 172.21.15.83:57742) 2024-03-11T19:49:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f874ed9d640 1 -- 172.21.15.83:0/1772413104 learned_addr learned my addr 172.21.15.83:0/1772413104 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.485+0000 7f874f59e640 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 0x55c5e3f902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.486+0000 7f874f59e640 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f97820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.486+0000 7f874ed9d640 1 -- 172.21.15.83:0/1772413104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 msgr2=0x55c5e3f902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.486+0000 7f874ed9d640 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 0x55c5e3f902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.486+0000 7f874ed9d640 1 -- 172.21.15.83:0/1772413104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f8bdd0 msgr2=0x55c5e3f97820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.486+0000 7f874ed9d640 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f97820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.486+0000 7f874ed9d640 1 -- 172.21.15.83:0/1772413104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8750108dd0 con 0x55c5e3f86c60 2024-03-11T19:49:28.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.486+0000 7f874f59e640 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 0x55c5e3f902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:28.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.486+0000 7f874ed9d640 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f907e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8740011f40 tx=0x7f874000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f8444ab2b89fc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.486+0000 7f87367fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.487+0000 7f87367fc640 1 -- 172.21.15.83:0/1772413104 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740017040 con 0x55c5e3f86c60 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.487+0000 7f87367fc640 1 -- 172.21.15.83:0/1772413104 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8740004260 con 0x55c5e3f86c60 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.487+0000 7f8757e2d740 1 -- 172.21.15.83:0/1772413104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5e3bee7e0 con 0x55c5e3f86c60 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.487+0000 7f87367fc640 1 -- 172.21.15.83:0/1772413104 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8740004460 con 0x55c5e3f86c60 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.487+0000 7f8757e2d740 10 client.17268 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.487+0000 7f8757e2d740 1 -- 172.21.15.83:0/1772413104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c5e3eed6a0 con 0x55c5e3f86c60 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.487+0000 7f87367fc640 1 -- 172.21.15.83:0/1772413104 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8740003ad0 con 0x55c5e3f86c60 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.488+0000 7f87367fc640 1 -- 172.21.15.83:0/1772413104 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f874001a070 con 0x55c5e3f86c60 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.488+0000 7f8757e2d740 10 client.17268 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.488+0000 7f8757e2d740 10 client.17268 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.488+0000 7f8757e2d740 20 client.17268 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.488+0000 7f8757e2d740 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55c5e3f9c4d0 0x55c5e3f9e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.488+0000 7f8757e2d740 4 client.17268 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:49:28.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.488+0000 7f8757e2d740 1 -- 172.21.15.83:0/1772413104 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5e3bee7e0 con 0x55c5e3f9c4d0 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.488+0000 7f874f59e640 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55c5e3f9c4d0 0x55c5e3f9e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.488+0000 7f87367fc640 10 client.17268 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.488+0000 7f874f59e640 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55c5e3f9c4d0 0x55c5e3f9e930 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b6bfeac308a4125 server_cookie=93ddc09db7aceae4 in_seq=0 out_seq=0 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f87367fc640 1 -- 172.21.15.83:0/1772413104 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5e3bee7e0 con 0x55c5e3f9c4d0 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f87367fc640 10 client.17268 handle_command_reply: tid=0 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 1 client.17268 shutdown 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 1 -- 172.21.15.83:0/1772413104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 msgr2=0x55c5e3f907e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f907e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8740011f40 tx=0x7f874000b690 comp rx=0 tx=0).stop 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 1 -- 172.21.15.83:0/1772413104 shutdown_connections 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55c5e3f9c4d0 0x55c5e3f9e930 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f97820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f907e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 1 --2- 172.21.15.83:0/1772413104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f86270 0x55c5e3f902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 1 -- 172.21.15.83:0/1772413104 >> 172.21.15.83:0/1772413104 conn(0x55c5e3f817e0 msgr2=0x55c5e3f80b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f87367fc640 0 client.17268 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:49:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 1 -- 172.21.15.83:0/1772413104 shutdown_connections 2024-03-11T19:49:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 1 -- 172.21.15.83:0/1772413104 wait complete. 2024-03-11T19:49:28.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.489+0000 7f8757e2d740 20 client.17268 trim_cache size 0 max 16384 2024-03-11T19:49:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.505+0000 7f8757e2d740 1 Processor -- start 2024-03-11T19:49:28.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.505+0000 7f8757e2d740 1 -- start start 2024-03-11T19:49:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.505+0000 7f8757e2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86270 0x55c5e3fa32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.505+0000 7f8757e2d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f86c60 0x55c5e3f977e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.505+0000 7f8757e2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.505+0000 7f8757e2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3e74130 con 0x55c5e3f86270 2024-03-11T19:49:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.505+0000 7f8757e2d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3f900e0 con 0x55c5e3f86c60 2024-03-11T19:49:28.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.505+0000 7f8757e2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3f911a0 con 0x55c5e3f8bdd0 2024-03-11T19:49:28.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.506+0000 7f874f59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86270 0x55c5e3fa32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.506+0000 7f874f59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86270 0x55c5e3fa32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57754/0 (socket says 172.21.15.83:57754) 2024-03-11T19:49:28.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.506+0000 7f874f59e640 1 -- 172.21.15.83:0/4149248562 learned_addr learned my addr 172.21.15.83:0/4149248562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.506+0000 7f874ed9d640 1 --2- 172.21.15.83:0/4149248562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f86c60 0x55c5e3f977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.506+0000 7f874f59e640 1 --2- 172.21.15.83:0/4149248562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.506+0000 7f874f59e640 1 -- 172.21.15.83:0/4149248562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f86c60 msgr2=0x55c5e3f977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.506+0000 7f874f59e640 1 --2- 172.21.15.83:0/4149248562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f86c60 0x55c5e3f977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.506+0000 7f874f59e640 1 -- 172.21.15.83:0/4149248562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86270 msgr2=0x55c5e3fa32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.506+0000 7f874f59e640 1 --2- 172.21.15.83:0/4149248562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86270 0x55c5e3fa32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.506+0000 7f874f59e640 1 -- 172.21.15.83:0/4149248562 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5e3e89c20 con 0x55c5e3f8bdd0 2024-03-11T19:49:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.507+0000 7f874f59e640 1 --2- 172.21.15.83:0/4149248562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f902a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8748002230 tx=0x7f874802bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fdd128776636610 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.507+0000 7f8744ff9640 1 -- 172.21.15.83:0/4149248562 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874804a040 con 0x55c5e3f8bdd0 2024-03-11T19:49:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.507+0000 7f8744ff9640 1 -- 172.21.15.83:0/4149248562 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f874804b040 con 0x55c5e3f8bdd0 2024-03-11T19:49:28.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.507+0000 7f8744ff9640 1 -- 172.21.15.83:0/4149248562 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748034790 con 0x55c5e3f8bdd0 2024-03-11T19:49:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.508+0000 7f8757e2d740 1 -- 172.21.15.83:0/4149248562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f8bdd0 msgr2=0x55c5e3f902a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.508+0000 7f8757e2d740 1 --2- 172.21.15.83:0/4149248562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f902a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8748002230 tx=0x7f874802bef0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.508+0000 7f8757e2d740 1 -- 172.21.15.83:0/4149248562 shutdown_connections 2024-03-11T19:49:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.508+0000 7f8757e2d740 1 --2- 172.21.15.83:0/4149248562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f902a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.508+0000 7f8757e2d740 1 --2- 172.21.15.83:0/4149248562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f86c60 0x55c5e3f977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.508+0000 7f8757e2d740 1 --2- 172.21.15.83:0/4149248562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86270 0x55c5e3fa32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.508+0000 7f8757e2d740 1 -- 172.21.15.83:0/4149248562 >> 172.21.15.83:0/4149248562 conn(0x55c5e3f817e0 msgr2=0x55c5e3f81bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.508+0000 7f8757e2d740 1 -- 172.21.15.83:0/4149248562 shutdown_connections 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.508+0000 7f8757e2d740 1 -- 172.21.15.83:0/4149248562 wait complete. 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.508+0000 7f8757e2d740 1 -- start start 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f8757e2d740 1 Processor -- start 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f8757e2d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f86270 0x55c5e3f97160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f8757e2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f8757e2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f95770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f8757e2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3f95dd0 con 0x55c5e3f86c60 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f8757e2d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3f95f40 con 0x55c5e3f86270 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f8757e2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e3f96100 con 0x55c5e3f8bdd0 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f8757e2d740 10 client.0 _ll_register_callbacks cb 0x7f8740005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57760/0 (socket says 172.21.15.83:57760) 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874f59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f86270 0x55c5e3f97160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874ed9d640 1 -- 172.21.15.83:0/50801674 learned_addr learned my addr 172.21.15.83:0/50801674 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874f59e640 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f95770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874ed9d640 1 -- 172.21.15.83:0/50801674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f8bdd0 msgr2=0x55c5e3f95770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874ed9d640 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f95770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874ed9d640 1 -- 172.21.15.83:0/50801674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f86270 msgr2=0x55c5e3f97160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874ed9d640 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f86270 0x55c5e3f97160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874ed9d640 1 -- 172.21.15.83:0/50801674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5e3eb1060 con 0x55c5e3f86c60 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874f59e640 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f86270 0x55c5e3f97160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.509+0000 7f874ed9d640 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f976a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8740009230 tx=0x7f8740011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133ed099b4e76fe9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.510+0000 7f87367fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.510+0000 7f87367fc640 1 -- 172.21.15.83:0/50801674 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740025920 con 0x55c5e3f86c60 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.510+0000 7f87367fc640 1 -- 172.21.15.83:0/50801674 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f8740000e40 con 0x55c5e3f86c60 2024-03-11T19:49:28.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.510+0000 7f8757e2d740 1 -- 172.21.15.83:0/50801674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87480334c0 con 0x55c5e3f86c60 2024-03-11T19:49:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.510+0000 7f87367fc640 1 -- 172.21.15.83:0/50801674 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8740005040 con 0x55c5e3f86c60 2024-03-11T19:49:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.510+0000 7f8757e2d740 10 client.17274 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.510+0000 7f8757e2d740 1 -- 172.21.15.83:0/50801674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c5e3eaf8c0 con 0x55c5e3f86c60 2024-03-11T19:49:28.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.510+0000 7f87367fc640 1 -- 172.21.15.83:0/50801674 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8740005680 con 0x55c5e3f86c60 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.512+0000 7f87367fc640 1 -- 172.21.15.83:0/50801674 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f874001a070 con 0x55c5e3f86c60 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.512+0000 7f8757e2d740 10 client.17274 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.512+0000 7f8757e2d740 10 client.17274 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.512+0000 7f8757e2d740 20 client.17274 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.512+0000 7f8757e2d740 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55c5e3ff3e30 0x55c5e3ff6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.512+0000 7f8757e2d740 4 client.17274 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.512+0000 7f8757e2d740 1 -- 172.21.15.83:0/50801674 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f87480334c0 con 0x55c5e3ff3e30 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.513+0000 7f874f59e640 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55c5e3ff3e30 0x55c5e3ff6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.513+0000 7f874f59e640 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55c5e3ff3e30 0x55c5e3ff6250 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5297dda016504b91 server_cookie=8791ca114a8d7284 in_seq=0 out_seq=0 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.513+0000 7f87367fc640 10 client.17274 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.513+0000 7f87367fc640 1 -- 172.21.15.83:0/50801674 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f87480334c0 con 0x55c5e3ff3e30 2024-03-11T19:49:28.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.513+0000 7f87367fc640 10 client.17274 handle_command_reply: tid=0 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.513+0000 7f8757e2d740 1 client.17274 shutdown 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.513+0000 7f8757e2d740 1 -- 172.21.15.83:0/50801674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 msgr2=0x55c5e3f976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.513+0000 7f8757e2d740 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f976a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8740009230 tx=0x7f8740011b60 comp rx=0 tx=0).stop 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.513+0000 7f8757e2d740 1 -- 172.21.15.83:0/50801674 shutdown_connections 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.513+0000 7f8757e2d740 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55c5e3ff3e30 0x55c5e3ff6250 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f87367fc640 0 client.17274 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.513+0000 7f8757e2d740 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5e3f8bdd0 0x55c5e3f95770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8757e2d740 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5e3f86c60 0x55c5e3f976a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8757e2d740 1 --2- 172.21.15.83:0/50801674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c5e3f86270 0x55c5e3f97160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8757e2d740 1 -- 172.21.15.83:0/50801674 >> 172.21.15.83:0/50801674 conn(0x55c5e3f817e0 msgr2=0x55c5e3fa32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8757e2d740 1 -- 172.21.15.83:0/50801674 shutdown_connections 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8757e2d740 1 -- 172.21.15.83:0/50801674 wait complete. 2024-03-11T19:49:28.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8757e2d740 20 client.17274 trim_cache size 0 max 16384 2024-03-11T19:49:28.522 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:49:28.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 -- 172.21.15.83:0/762113307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f873c0a1ab0 msgr2=0x7f873c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f873c0a1ab0 0x7f873c0a3f70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8738096a10 tx=0x7f873800cf90 comp rx=0 tx=0).stop 2024-03-11T19:49:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 -- 172.21.15.83:0/762113307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8750100d20 msgr2=0x7f875019bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8750100d20 0x7f875019bf00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f873800e9c0 tx=0x7f873800ee80 comp rx=0 tx=0).stop 2024-03-11T19:49:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 -- 172.21.15.83:0/762113307 shutdown_connections 2024-03-11T19:49:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f873c0a1ab0 0x7f873c0a3f70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8750101630 0x7f875019e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8750100d20 0x7f875019bf00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 --2- 172.21.15.83:0/762113307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8750100360 0x7f87500ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 -- 172.21.15.83:0/762113307 >> 172.21.15.83:0/762113307 conn(0x7f87500fadb0 msgr2=0x7f8750104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 -- 172.21.15.83:0/762113307 shutdown_connections 2024-03-11T19:49:28.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.514+0000 7f8755255640 1 -- 172.21.15.83:0/762113307 wait complete. 2024-03-11T19:49:28.533 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:49:28.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:49:28.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:28 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/626973030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:49:28.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.729+0000 7f94b4a0e640 1 -- 172.21.15.83:0/2428177242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0102410 msgr2=0x7f94b0102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.729+0000 7f94b4a0e640 1 --2- 172.21.15.83:0/2428177242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0102410 0x7f94b0102890 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f94a000fb10 tx=0x7f94a00341b0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.729+0000 7f94b4a0e640 1 -- 172.21.15.83:0/2428177242 shutdown_connections 2024-03-11T19:49:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.729+0000 7f94b4a0e640 1 --2- 172.21.15.83:0/2428177242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94b0102dd0 0x7f94b0109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.729+0000 7f94b4a0e640 1 --2- 172.21.15.83:0/2428177242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0102410 0x7f94b0102890 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.729+0000 7f94b4a0e640 1 --2- 172.21.15.83:0/2428177242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b00ff440 0x7f94b00ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.729+0000 7f94b4a0e640 1 -- 172.21.15.83:0/2428177242 >> 172.21.15.83:0/2428177242 conn(0x7f94b00fad70 msgr2=0x7f94b00fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.730+0000 7f94b4a0e640 1 -- 172.21.15.83:0/2428177242 shutdown_connections 2024-03-11T19:49:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.730+0000 7f94b4a0e640 1 -- 172.21.15.83:0/2428177242 wait complete. 2024-03-11T19:49:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.730+0000 7f94b4a0e640 1 Processor -- start 2024-03-11T19:49:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.730+0000 7f94b4a0e640 1 -- start start 2024-03-11T19:49:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.730+0000 7f94b4a0e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b00ff440 0x7f94b0195020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.730+0000 7f94b4a0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0102410 0x7f94b0195560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.730+0000 7f94b4a0e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94b0102dd0 0x7f94b019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.730+0000 7f94b4a0e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b019cb10 con 0x7f94b0102410 2024-03-11T19:49:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.730+0000 7f94b4a0e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b019cc80 con 0x7f94b0102dd0 2024-03-11T19:49:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.730+0000 7f94b4a0e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b019ce40 con 0x7f94b00ff440 2024-03-11T19:49:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.731+0000 7f94add9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0102410 0x7f94b0195560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.731+0000 7f94add9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0102410 0x7f94b0195560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57778/0 (socket says 172.21.15.83:57778) 2024-03-11T19:49:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.731+0000 7f94add9b640 1 -- 172.21.15.83:0/1668265215 learned_addr learned my addr 172.21.15.83:0/1668265215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.731+0000 7f94aed9d640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94b0102dd0 0x7f94b019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.731+0000 7f94ae59c640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b00ff440 0x7f94b0195020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.731+0000 7f94add9b640 1 -- 172.21.15.83:0/1668265215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b00ff440 msgr2=0x7f94b0195020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.731+0000 7f94add9b640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b00ff440 0x7f94b0195020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.731+0000 7f94add9b640 1 -- 172.21.15.83:0/1668265215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94b0102dd0 msgr2=0x7f94b019c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.731+0000 7f94add9b640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94b0102dd0 0x7f94b019c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.731+0000 7f94add9b640 1 -- 172.21.15.83:0/1668265215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a000f770 con 0x7f94b0102410 2024-03-11T19:49:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.732+0000 7f94add9b640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0102410 0x7f94b0195560 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f94a0000c00 tx=0x7f94a000f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.732+0000 7f949f7fe640 1 -- 172.21.15.83:0/1668265215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a000e970 con 0x7f94b0102410 2024-03-11T19:49:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.732+0000 7f949f7fe640 1 -- 172.21.15.83:0/1668265215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94a0036070 con 0x7f94b0102410 2024-03-11T19:49:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.732+0000 7f949f7fe640 1 -- 172.21.15.83:0/1668265215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a003c680 con 0x7f94b0102410 2024-03-11T19:49:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.732+0000 7f94b4a0e640 1 -- 172.21.15.83:0/1668265215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b019d0c0 con 0x7f94b0102410 2024-03-11T19:49:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.732+0000 7f94b4a0e640 1 -- 172.21.15.83:0/1668265215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b019d520 con 0x7f94b0102410 2024-03-11T19:49:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.735+0000 7f949f7fe640 1 -- 172.21.15.83:0/1668265215 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f94a00361e0 con 0x7f94b0102410 2024-03-11T19:49:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.735+0000 7f94b75e6740 1 Processor -- start 2024-03-11T19:49:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.735+0000 7f94b75e6740 1 -- start start 2024-03-11T19:49:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.735+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a019f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.735+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.735+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 0x55d5a01a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.735+0000 7f94b75e6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a015b960 con 0x55d5a019f270 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.735+0000 7f94b75e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a010f310 con 0x55d5a019fc60 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.735+0000 7f94b75e6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a010c840 con 0x55d5a01a4dd0 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94aed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94aed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57784/0 (socket says 172.21.15.83:57784) 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94aed9d640 1 -- 172.21.15.83:0/3040682192 learned_addr learned my addr 172.21.15.83:0/3040682192 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f949f7fe640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94980a1b80 0x7f94980a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94ae59c640 1 --2- 172.21.15.83:0/3040682192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f949f7fe640 1 -- 172.21.15.83:0/1668265215 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f94a00f7a80 con 0x7f94b0102410 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94aed9d640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94980a1b80 0x7f94980a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94add9b640 1 --2- 172.21.15.83:0/3040682192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 0x55d5a01a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94aed9d640 1 -- 172.21.15.83:0/3040682192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 msgr2=0x55d5a01a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94aed9d640 1 --2- 172.21.15.83:0/3040682192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 0x55d5a01a7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94aed9d640 1 -- 172.21.15.83:0/3040682192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 msgr2=0x55d5a01a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94aed9d640 1 --2- 172.21.15.83:0/3040682192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94aed9d640 1 -- 172.21.15.83:0/3040682192 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5a00ca060 con 0x55d5a019f270 2024-03-11T19:49:28.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.736+0000 7f94add9b640 1 --2- 172.21.15.83:0/3040682192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 0x55d5a01a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:49:28.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.737+0000 7f94aed9d640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94980a1b80 0x7f94980a4040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f94a8012460 tx=0x7f94a801d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.737+0000 7f94aed9d640 1 --2- 172.21.15.83:0/3040682192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a019f690 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f94a8012a70 tx=0x7f94a8012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb6b9d917ef7949 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.737+0000 7f949cff9640 1 -- 172.21.15.83:0/3040682192 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8011c40 con 0x55d5a019f270 2024-03-11T19:49:28.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.738+0000 7f949cff9640 1 -- 172.21.15.83:0/3040682192 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94a8003e50 con 0x55d5a019f270 2024-03-11T19:49:28.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.738+0000 7f949cff9640 1 -- 172.21.15.83:0/3040682192 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8004770 con 0x55d5a019f270 2024-03-11T19:49:28.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.739+0000 7f94b75e6740 1 -- 172.21.15.83:0/3040682192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 msgr2=0x55d5a019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.739+0000 7f94b75e6740 1 --2- 172.21.15.83:0/3040682192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a019f690 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f94a8012a70 tx=0x7f94a8012f30 comp rx=0 tx=0).stop 2024-03-11T19:49:28.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.739+0000 7f94b75e6740 1 -- 172.21.15.83:0/3040682192 shutdown_connections 2024-03-11T19:49:28.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.739+0000 7f94b75e6740 1 --2- 172.21.15.83:0/3040682192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 0x55d5a01a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.739+0000 7f94b75e6740 1 --2- 172.21.15.83:0/3040682192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.739+0000 7f94b75e6740 1 --2- 172.21.15.83:0/3040682192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a019f690 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.739+0000 7f94b75e6740 1 -- 172.21.15.83:0/3040682192 >> 172.21.15.83:0/3040682192 conn(0x55d5a019a7e0 msgr2=0x55d5a019abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.739+0000 7f94b75e6740 1 -- 172.21.15.83:0/3040682192 shutdown_connections 2024-03-11T19:49:28.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.739+0000 7f94b75e6740 1 -- 172.21.15.83:0/3040682192 wait complete. 2024-03-11T19:49:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.739+0000 7f94b75e6740 1 -- start start 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.739+0000 7f94b75e6740 1 Processor -- start 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a01a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 0x55d5a01b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94b75e6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a0108020 con 0x55d5a019f270 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94b75e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59fddb430 con 0x55d5a019fc60 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94b75e6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59fd97e00 con 0x55d5a01a4dd0 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94b75e6740 10 client.0 _ll_register_callbacks cb 0x55d5a00afcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94ae59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a01a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94ae59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a01a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57786/0 (socket says 172.21.15.83:57786) 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94add9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94ae59c640 1 -- 172.21.15.83:0/3962596854 learned_addr learned my addr 172.21.15.83:0/3962596854 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94aed9d640 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 0x55d5a01b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94ae59c640 1 -- 172.21.15.83:0/3962596854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 msgr2=0x55d5a01b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94ae59c640 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 0x55d5a01b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94ae59c640 1 -- 172.21.15.83:0/3962596854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 msgr2=0x55d5a01a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94ae59c640 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94ae59c640 1 -- 172.21.15.83:0/3962596854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a8012650 con 0x55d5a019f270 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94aed9d640 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 0x55d5a01b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94add9b640 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f94ae59c640 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a01a94d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f94a4012700 tx=0x7f94a4012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bdd37a34194a91 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.740+0000 7f9495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.741+0000 7f9495ffb640 1 -- 172.21.15.83:0/3962596854 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4017070 con 0x55d5a019f270 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.741+0000 7f9495ffb640 1 -- 172.21.15.83:0/3962596854 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f94a4004020 con 0x55d5a019f270 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.741+0000 7f94b75e6740 1 -- 172.21.15.83:0/3962596854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d59fe077e0 con 0x55d5a019f270 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.741+0000 7f9495ffb640 1 -- 172.21.15.83:0/3962596854 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94a4022030 con 0x55d5a019f270 2024-03-11T19:49:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.741+0000 7f94b75e6740 10 client.17286 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.741+0000 7f94b75e6740 1 -- 172.21.15.83:0/3962596854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d5a01066a0 con 0x55d5a019f270 2024-03-11T19:49:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.741+0000 7f9495ffb640 1 -- 172.21.15.83:0/3962596854 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d59fe077e0 con 0x55d5a019f270 2024-03-11T19:49:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.741+0000 7f9495ffb640 1 -- 172.21.15.83:0/3962596854 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f94a400f590 con 0x55d5a019f270 2024-03-11T19:49:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.742+0000 7f94b75e6740 10 client.17286 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.742+0000 7f94b75e6740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:49:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.742+0000 7f94b75e6740 20 client.17286 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.742+0000 7f94b75e6740 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d5a01b56f0 0x55d5a01b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.742+0000 7f94b75e6740 4 client.17286 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:49:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.742+0000 7f94b75e6740 1 -- 172.21.15.83:0/3962596854 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d59fe44720 con 0x55d5a01b56f0 2024-03-11T19:49:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.742+0000 7f94add9b640 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d5a01b56f0 0x55d5a01b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.743+0000 7f9495ffb640 10 client.17286 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.743+0000 7f94add9b640 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d5a01b56f0 0x55d5a01b7b10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65fa331382776efc server_cookie=25cd39c2f1e22571 in_seq=0 out_seq=0 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.744+0000 7f9495ffb640 1 -- 172.21.15.83:0/3962596854 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d59fe44720 con 0x55d5a01b56f0 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.744+0000 7f9495ffb640 10 client.17286 handle_command_reply: tid=0 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.744+0000 7f94b75e6740 1 client.17286 shutdown 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.744+0000 7f94b75e6740 1 -- 172.21.15.83:0/3962596854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 msgr2=0x55d5a01a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.745+0000 7f94b75e6740 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a01a94d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f94a4012700 tx=0x7f94a4012b00 comp rx=0 tx=0).stop 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.745+0000 7f94b75e6740 1 -- 172.21.15.83:0/3962596854 shutdown_connections 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.745+0000 7f94b75e6740 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d5a01b56f0 0x55d5a01b7b10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.745+0000 7f94b75e6740 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a01a4dd0 0x55d5a01b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.745+0000 7f94b75e6740 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.745+0000 7f94b75e6740 1 --2- 172.21.15.83:0/3962596854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a019f270 0x55d5a01a94d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.745+0000 7f94b75e6740 1 -- 172.21.15.83:0/3962596854 >> 172.21.15.83:0/3962596854 conn(0x55d5a019a7e0 msgr2=0x55d5a0199b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.745+0000 7f9495ffb640 0 client.17286 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:28.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.745+0000 7f94b75e6740 1 -- 172.21.15.83:0/3962596854 shutdown_connections 2024-03-11T19:49:28.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.745+0000 7f94b75e6740 1 -- 172.21.15.83:0/3962596854 wait complete. 2024-03-11T19:49:28.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.745+0000 7f94b75e6740 20 client.17286 trim_cache size 0 max 16384 2024-03-11T19:49:28.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.769+0000 7f94b75e6740 1 Processor -- start 2024-03-11T19:49:28.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.773+0000 7f94b75e6740 1 -- start start 2024-03-11T19:49:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.773+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 0x55d5a01a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.773+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.773+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 0x55d5a01a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.773+0000 7f94b75e6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a008d130 con 0x55d5a01a4dd0 2024-03-11T19:49:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.773+0000 7f94b75e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a01b3c70 con 0x55d5a019fc60 2024-03-11T19:49:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.773+0000 7f94b75e6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a01b1ed0 con 0x55d5a019f270 2024-03-11T19:49:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.774+0000 7f94aed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 0x55d5a01a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.775+0000 7f94aed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 0x55d5a01a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57796/0 (socket says 172.21.15.83:57796) 2024-03-11T19:49:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.775+0000 7f94aed9d640 1 -- 172.21.15.83:0/706262439 learned_addr learned my addr 172.21.15.83:0/706262439 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.775+0000 7f94add9b640 1 --2- 172.21.15.83:0/706262439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.775+0000 7f94ae59c640 1 --2- 172.21.15.83:0/706262439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 0x55d5a01a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.775+0000 7f94add9b640 1 -- 172.21.15.83:0/706262439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 msgr2=0x55d5a01a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.775+0000 7f94add9b640 1 --2- 172.21.15.83:0/706262439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 0x55d5a01a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.775+0000 7f94add9b640 1 -- 172.21.15.83:0/706262439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 msgr2=0x55d5a01a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.775+0000 7f94add9b640 1 --2- 172.21.15.83:0/706262439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 0x55d5a01a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.775+0000 7f94add9b640 1 -- 172.21.15.83:0/706262439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5a00a2c20 con 0x55d5a019fc60 2024-03-11T19:49:28.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.776+0000 7f94aed9d640 1 --2- 172.21.15.83:0/706262439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 0x55d5a01a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:28.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.776+0000 7f94add9b640 1 --2- 172.21.15.83:0/706262439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01b0a10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f94a00f5680 tx=0x7f94a00f9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95f309b7b63c5c8b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.780+0000 7f9497fff640 1 -- 172.21.15.83:0/706262439 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a00c8bf0 con 0x55d5a019fc60 2024-03-11T19:49:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.780+0000 7f9497fff640 1 -- 172.21.15.83:0/706262439 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94a00cd240 con 0x55d5a019fc60 2024-03-11T19:49:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.781+0000 7f9497fff640 1 -- 172.21.15.83:0/706262439 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a00cd010 con 0x55d5a019fc60 2024-03-11T19:49:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.781+0000 7f94b75e6740 1 -- 172.21.15.83:0/706262439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 msgr2=0x55d5a01b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.781+0000 7f94b75e6740 1 --2- 172.21.15.83:0/706262439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01b0a10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f94a00f5680 tx=0x7f94a00f9040 comp rx=0 tx=0).stop 2024-03-11T19:49:28.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.782+0000 7f94b75e6740 1 -- 172.21.15.83:0/706262439 shutdown_connections 2024-03-11T19:49:28.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.782+0000 7f94b75e6740 1 --2- 172.21.15.83:0/706262439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 0x55d5a01a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.782+0000 7f94b75e6740 1 --2- 172.21.15.83:0/706262439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01b0a10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.782+0000 7f94b75e6740 1 --2- 172.21.15.83:0/706262439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 0x55d5a01a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.782+0000 7f94b75e6740 1 -- 172.21.15.83:0/706262439 >> 172.21.15.83:0/706262439 conn(0x55d5a019a7e0 msgr2=0x55d5a019abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.782+0000 7f94b75e6740 1 -- 172.21.15.83:0/706262439 shutdown_connections 2024-03-11T19:49:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.782+0000 7f94b75e6740 1 -- 172.21.15.83:0/706262439 wait complete. 2024-03-11T19:49:28.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.783+0000 7f94b75e6740 1 -- start start 2024-03-11T19:49:28.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.783+0000 7f94b75e6740 1 Processor -- start 2024-03-11T19:49:28.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.783+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 0x55d5a01ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.783+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.783+0000 7f94b75e6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 0x55d5a01afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.783+0000 7f94b75e6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a01b0110 con 0x55d5a01a4dd0 2024-03-11T19:49:28.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.783+0000 7f94b75e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a01b0280 con 0x55d5a019fc60 2024-03-11T19:49:28.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.783+0000 7f94b75e6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5a01b0440 con 0x55d5a019f270 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.783+0000 7f94b75e6740 10 client.0 _ll_register_callbacks cb 0x7f94a8009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.783+0000 7f94ae59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 0x55d5a01ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f94ae59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 0x55d5a01ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57598/0 (socket says 172.21.15.83:57598) 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f94ae59c640 1 -- 172.21.15.83:0/518001132 learned_addr learned my addr 172.21.15.83:0/518001132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f94aed9d640 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 0x55d5a01afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f94add9b640 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f94ae59c640 1 -- 172.21.15.83:0/518001132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 msgr2=0x55d5a01aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f94ae59c640 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f94ae59c640 1 -- 172.21.15.83:0/518001132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 msgr2=0x55d5a01afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f94ae59c640 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 0x55d5a01afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f94ae59c640 1 -- 172.21.15.83:0/518001132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5a00ca060 con 0x55d5a019f270 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f94add9b640 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:28.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f94ae59c640 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 0x55d5a01ae6d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f94a4000c00 tx=0x7f94a400fdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e6a831d2be3146a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:28.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f9495ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:49:28.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.784+0000 7f9495ffb640 1 -- 172.21.15.83:0/518001132 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4017070 con 0x55d5a019f270 2024-03-11T19:49:28.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.785+0000 7f94b75e6740 1 -- 172.21.15.83:0/518001132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94a8003e50 con 0x55d5a019f270 2024-03-11T19:49:28.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.785+0000 7f9495ffb640 1 -- 172.21.15.83:0/518001132 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f94a4000e40 con 0x55d5a019f270 2024-03-11T19:49:28.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.785+0000 7f9495ffb640 1 -- 172.21.15.83:0/518001132 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94a4022040 con 0x55d5a019f270 2024-03-11T19:49:28.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.785+0000 7f94b75e6740 10 client.26854 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:28.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.785+0000 7f94b75e6740 1 -- 172.21.15.83:0/518001132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f94b019d520 con 0x55d5a019f270 2024-03-11T19:49:28.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.785+0000 7f9495ffb640 1 -- 172.21.15.83:0/518001132 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94a8003e50 con 0x55d5a019f270 2024-03-11T19:49:28.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.786+0000 7f9495ffb640 1 -- 172.21.15.83:0/518001132 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f94a401c470 con 0x55d5a019f270 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.787+0000 7f94b75e6740 10 client.26854 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.787+0000 7f94b75e6740 10 client.26854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.787+0000 7f94b75e6740 20 client.26854 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.787+0000 7f94b75e6740 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d5a020cab0 0x55d5a020eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.787+0000 7f94b75e6740 4 client.26854 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.787+0000 7f94b75e6740 1 -- 172.21.15.83:0/518001132 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d59fe93890 con 0x55d5a020cab0 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.787+0000 7f94add9b640 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d5a020cab0 0x55d5a020eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94add9b640 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d5a020cab0 0x55d5a020eed0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b96377487ccd5fcf server_cookie=9571d0e6f0cefc9c in_seq=0 out_seq=0 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f9495ffb640 10 client.26854 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f9495ffb640 1 -- 172.21.15.83:0/518001132 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d59fe93890 con 0x55d5a020cab0 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f9495ffb640 10 client.26854 handle_command_reply: tid=0 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 1 client.26854 shutdown 2024-03-11T19:49:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 1 -- 172.21.15.83:0/518001132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 msgr2=0x55d5a01ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 0x55d5a01ae6d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f94a4000c00 tx=0x7f94a400fdc0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 1 -- 172.21.15.83:0/518001132 shutdown_connections 2024-03-11T19:49:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d5a020cab0 0x55d5a020eed0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5a01a4dd0 0x55d5a01afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5a019fc60 0x55d5a01aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 1 --2- 172.21.15.83:0/518001132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5a019f270 0x55d5a01ae6d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f9495ffb640 0 client.26854 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 1 -- 172.21.15.83:0/518001132 >> 172.21.15.83:0/518001132 conn(0x55d5a019a7e0 msgr2=0x55d5a01a11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 1 -- 172.21.15.83:0/518001132 shutdown_connections 2024-03-11T19:49:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 1 -- 172.21.15.83:0/518001132 wait complete. 2024-03-11T19:49:28.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.788+0000 7f94b75e6740 20 client.26854 trim_cache size 0 max 16384 2024-03-11T19:49:28.795 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:49:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.789+0000 7f94b4a0e640 1 -- 172.21.15.83:0/1668265215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94980a1b80 msgr2=0x7f94980a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.789+0000 7f94b4a0e640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94980a1b80 0x7f94980a4040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f94a8012460 tx=0x7f94a801d000 comp rx=0 tx=0).stop 2024-03-11T19:49:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.789+0000 7f94b4a0e640 1 -- 172.21.15.83:0/1668265215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0102410 msgr2=0x7f94b0195560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.789+0000 7f94b4a0e640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0102410 0x7f94b0195560 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f94a0000c00 tx=0x7f94a000f440 comp rx=0 tx=0).stop 2024-03-11T19:49:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.790+0000 7f94b4a0e640 1 -- 172.21.15.83:0/1668265215 shutdown_connections 2024-03-11T19:49:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.790+0000 7f94b4a0e640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94980a1b80 0x7f94980a4040 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.790+0000 7f94b4a0e640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94b0102dd0 0x7f94b019c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.790+0000 7f94b4a0e640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94b0102410 0x7f94b0195560 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.790+0000 7f94b4a0e640 1 --2- 172.21.15.83:0/1668265215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b00ff440 0x7f94b0195020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.790+0000 7f94b4a0e640 1 -- 172.21.15.83:0/1668265215 >> 172.21.15.83:0/1668265215 conn(0x7f94b00fad70 msgr2=0x7f94b0105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.790+0000 7f94b4a0e640 1 -- 172.21.15.83:0/1668265215 shutdown_connections 2024-03-11T19:49:28.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:28.790+0000 7f94b4a0e640 1 -- 172.21.15.83:0/1668265215 wait complete. 2024-03-11T19:49:29.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:29 smithi083 ceph-mon[31864]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.0 MiB/s wr, 719 op/s 2024-03-11T19:49:29.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:29 smithi137 ceph-mon[36220]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.0 MiB/s wr, 719 op/s 2024-03-11T19:49:29.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:29 smithi204 ceph-mon[36739]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.0 MiB/s wr, 719 op/s 2024-03-11T19:49:29.809 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:49:29.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:49:29.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:49:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.955+0000 7f02a99c9640 1 -- 172.21.15.83:0/1711172370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02a4101c40 msgr2=0x7f02a41084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.955+0000 7f02a99c9640 1 --2- 172.21.15.83:0/1711172370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02a4101c40 0x7f02a41084d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f029c009ca0 tx=0x7f029c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.955+0000 7f02a99c9640 1 -- 172.21.15.83:0/1711172370 shutdown_connections 2024-03-11T19:49:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.955+0000 7f02a99c9640 1 --2- 172.21.15.83:0/1711172370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02a4101c40 0x7f02a41084d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.955+0000 7f02a99c9640 1 --2- 172.21.15.83:0/1711172370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02a4101280 0x7f02a4101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.955+0000 7f02a99c9640 1 --2- 172.21.15.83:0/1711172370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02a4100090 0x7f02a41004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.955+0000 7f02a99c9640 1 -- 172.21.15.83:0/1711172370 >> 172.21.15.83:0/1711172370 conn(0x7f02a40fb620 msgr2=0x7f02a40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.955+0000 7f02a99c9640 1 -- 172.21.15.83:0/1711172370 shutdown_connections 2024-03-11T19:49:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.955+0000 7f02a99c9640 1 -- 172.21.15.83:0/1711172370 wait complete. 2024-03-11T19:49:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.956+0000 7f02a99c9640 1 Processor -- start 2024-03-11T19:49:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.956+0000 7f02a99c9640 1 -- start start 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.956+0000 7f02a99c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02a4100090 0x7f02a41a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.956+0000 7f02a99c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02a4101280 0x7f02a41a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.956+0000 7f02a99c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02a4101c40 0x7f02a41a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a99c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a41a5360 con 0x7f02a4100090 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a99c9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a41a54d0 con 0x7f02a4101280 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a99c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a41a5690 con 0x7f02a4101c40 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02a4101c40 0x7f02a41a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a37fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02a4101c40 0x7f02a41a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57294/0 (socket says 172.21.15.83:57294) 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a37fe640 1 -- 172.21.15.83:0/974695523 learned_addr learned my addr 172.21.15.83:0/974695523 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f029bfff640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02a4101280 0x7f02a41a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a37fe640 1 -- 172.21.15.83:0/974695523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02a4101280 msgr2=0x7f02a41a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a37fe640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02a4101280 0x7f02a41a2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a37fe640 1 -- 172.21.15.83:0/974695523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02a4100090 msgr2=0x7f02a41a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a37fe640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02a4100090 0x7f02a41a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a37fe640 1 -- 172.21.15.83:0/974695523 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c009900 con 0x7f02a4101c40 2024-03-11T19:49:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.957+0000 7f02a37fe640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02a4101c40 0x7f02a41a4bd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f029c037040 tx=0x7f029c009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.958+0000 7f02a89c7640 1 -- 172.21.15.83:0/974695523 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c0329b0 con 0x7f02a4101c40 2024-03-11T19:49:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.958+0000 7f02a99c9640 1 -- 172.21.15.83:0/974695523 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a41a5910 con 0x7f02a4101c40 2024-03-11T19:49:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.958+0000 7f02a89c7640 1 -- 172.21.15.83:0/974695523 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f029c003e50 con 0x7f02a4101c40 2024-03-11T19:49:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.958+0000 7f02a89c7640 1 -- 172.21.15.83:0/974695523 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c040450 con 0x7f02a4101c40 2024-03-11T19:49:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.958+0000 7f02a99c9640 1 -- 172.21.15.83:0/974695523 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a41a5e10 con 0x7f02a4101c40 2024-03-11T19:49:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.960+0000 7f02a89c7640 1 -- 172.21.15.83:0/974695523 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f029c00a040 con 0x7f02a4101c40 2024-03-11T19:49:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.960+0000 7f02ac595740 1 Processor -- start 2024-03-11T19:49:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.960+0000 7f02ac595740 1 -- start start 2024-03-11T19:49:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.960+0000 7f02ac595740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e64c0 0x5628893e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.960+0000 7f02ac595740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 0x5628893e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.960+0000 7f02ac595740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e8050 0x5628893ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.960+0000 7f02ac595740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562889355f90 con 0x5628893e7690 2024-03-11T19:49:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.960+0000 7f02ac595740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628893534c0 con 0x5628893e8050 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.960+0000 7f02ac595740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288934eca0 con 0x5628893e64c0 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f029bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e64c0 0x5628893e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f029bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e8050 0x5628893ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f029bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e64c0 0x5628893e68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57300/0 (socket says 172.21.15.83:57300) 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f029bfff640 1 -- 172.21.15.83:0/3324116353 learned_addr learned my addr 172.21.15.83:0/3324116353 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 0x5628893e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f02a89c7640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f02900a1a30 0x7f02900a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f029bfff640 1 -- 172.21.15.83:0/3324116353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e8050 msgr2=0x5628893ee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f029bfff640 1 --2- 172.21.15.83:0/3324116353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e8050 0x5628893ee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f029bfff640 1 -- 172.21.15.83:0/3324116353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 msgr2=0x5628893e7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f029bfff640 1 --2- 172.21.15.83:0/3324116353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 0x5628893e7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f029bfff640 1 -- 172.21.15.83:0/3324116353 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628893110b0 con 0x5628893e64c0 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.961+0000 7f02a37fe640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f02900a1a30 0x7f02900a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.962+0000 7f02a89c7640 1 -- 172.21.15.83:0/974695523 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f029c0f0170 con 0x7f02a4101c40 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.962+0000 7f029bfff640 1 --2- 172.21.15.83:0/3324116353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e64c0 0x5628893e68e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f028c013810 tx=0x7f028c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd817fec0be7cf71 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.962+0000 7f0299ffb640 1 -- 172.21.15.83:0/3324116353 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c0130d0 con 0x5628893e64c0 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.962+0000 7f02a37fe640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f02900a1a30 0x7f02900a3ef0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f029c0c5400 tx=0x7f029c0b95a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.962+0000 7f0299ffb640 1 -- 172.21.15.83:0/3324116353 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f028c013230 con 0x5628893e64c0 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.962+0000 7f0299ffb640 1 -- 172.21.15.83:0/3324116353 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c0047a0 con 0x5628893e64c0 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.963+0000 7f02ac595740 1 -- 172.21.15.83:0/3324116353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e64c0 msgr2=0x5628893e68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.963+0000 7f02ac595740 1 --2- 172.21.15.83:0/3324116353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e64c0 0x5628893e68e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f028c013810 tx=0x7f028c005f90 comp rx=0 tx=0).stop 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.963+0000 7f02ac595740 1 -- 172.21.15.83:0/3324116353 shutdown_connections 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.963+0000 7f02ac595740 1 --2- 172.21.15.83:0/3324116353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e8050 0x5628893ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.963+0000 7f02ac595740 1 --2- 172.21.15.83:0/3324116353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 0x5628893e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.963+0000 7f02ac595740 1 --2- 172.21.15.83:0/3324116353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e64c0 0x5628893e68e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.963+0000 7f02ac595740 1 -- 172.21.15.83:0/3324116353 >> 172.21.15.83:0/3324116353 conn(0x5628893e16a0 msgr2=0x5628893e1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.963+0000 7f02ac595740 1 -- 172.21.15.83:0/3324116353 shutdown_connections 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.963+0000 7f02ac595740 1 -- 172.21.15.83:0/3324116353 wait complete. 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.963+0000 7f02ac595740 1 -- start start 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.963+0000 7f02ac595740 1 Processor -- start 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02ac595740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 0x5628893f0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02ac595740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 0x5628893f0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02ac595740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e8050 0x5628893f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02ac595740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628892f3f40 con 0x5628893e7690 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02ac595740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628890224b0 con 0x5628893e64c0 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02ac595740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628892d4670 con 0x5628893e8050 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02ac595740 10 client.0 _ll_register_callbacks cb 0x562889341f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 0x5628893f0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 0x5628893f0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43650/0 (socket says 172.21.15.83:43650) 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f029bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e8050 0x5628893f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02a2ffd640 1 -- 172.21.15.83:0/1311436627 learned_addr learned my addr 172.21.15.83:0/1311436627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f029bfff640 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 0x5628893f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02a2ffd640 1 -- 172.21.15.83:0/1311436627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e8050 msgr2=0x5628893f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02a2ffd640 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e8050 0x5628893f7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02a2ffd640 1 -- 172.21.15.83:0/1311436627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 msgr2=0x5628893f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02a2ffd640 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 0x5628893f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02a2ffd640 1 -- 172.21.15.83:0/1311436627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c01b040 con 0x5628893e7690 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02a2ffd640 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 0x5628893f0c40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f02940116f0 tx=0x7f0294011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35111bb61820de57 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f0282ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f0282ffd640 1 -- 172.21.15.83:0/1311436627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400fcd0 con 0x5628893e7690 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f0282ffd640 1 -- 172.21.15.83:0/1311436627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f029400fe30 con 0x5628893e7690 2024-03-11T19:49:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02ac595740 1 -- 172.21.15.83:0/1311436627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562889328f40 con 0x5628893e7690 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f0282ffd640 1 -- 172.21.15.83:0/1311436627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0294019430 con 0x5628893e7690 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02ac595740 10 client.17301 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.964+0000 7f02ac595740 1 -- 172.21.15.83:0/1311436627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56288934d320 con 0x5628893e7690 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.965+0000 7f0282ffd640 1 -- 172.21.15.83:0/1311436627 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562889328f40 con 0x5628893e7690 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.965+0000 7f0282ffd640 1 -- 172.21.15.83:0/1311436627 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02940247e0 con 0x5628893e7690 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.965+0000 7f02ac595740 10 client.17301 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.965+0000 7f02ac595740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.965+0000 7f02ac595740 20 client.17301 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.965+0000 7f02ac595740 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5628893fc9b0 0x5628893fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.965+0000 7f02ac595740 4 client.17301 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.965+0000 7f02ac595740 1 -- 172.21.15.83:0/1311436627 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562889345310 con 0x5628893fc9b0 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.965+0000 7f029bfff640 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5628893fc9b0 0x5628893fedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.965+0000 7f0282ffd640 10 client.17301 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.965+0000 7f029bfff640 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5628893fc9b0 0x5628893fedd0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c662f36ffb43046 server_cookie=5e64f3fb7209a09f in_seq=0 out_seq=0 2024-03-11T19:49:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f0282ffd640 1 -- 172.21.15.83:0/1311436627 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562889345310 con 0x5628893fc9b0 2024-03-11T19:49:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f0282ffd640 10 client.17301 handle_command_reply: tid=0 2024-03-11T19:49:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f02ac595740 1 client.17301 shutdown 2024-03-11T19:49:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f02ac595740 1 -- 172.21.15.83:0/1311436627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 msgr2=0x5628893f0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f02ac595740 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 0x5628893f0c40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f02940116f0 tx=0x7f0294011bb0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f02ac595740 1 -- 172.21.15.83:0/1311436627 shutdown_connections 2024-03-11T19:49:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f02ac595740 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5628893fc9b0 0x5628893fedd0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f02ac595740 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e8050 0x5628893f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f02ac595740 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e7690 0x5628893f0c40 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f02ac595740 1 --2- 172.21.15.83:0/1311436627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 0x5628893f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f02ac595740 1 -- 172.21.15.83:0/1311436627 >> 172.21.15.83:0/1311436627 conn(0x5628893e16a0 msgr2=0x5628893e08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f0282ffd640 0 client.17301 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f02ac595740 1 -- 172.21.15.83:0/1311436627 shutdown_connections 2024-03-11T19:49:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.967+0000 7f02ac595740 1 -- 172.21.15.83:0/1311436627 wait complete. 2024-03-11T19:49:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.968+0000 7f02ac595740 20 client.17301 trim_cache size 0 max 16384 2024-03-11T19:49:29.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f02ac595740 1 Processor -- start 2024-03-11T19:49:29.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f02ac595740 1 -- start start 2024-03-11T19:49:29.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f02ac595740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e64c0 0x5628893eb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f02ac595740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e7690 0x5628893eb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f02ac595740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e8050 0x5628893f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f02ac595740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562889328f40 con 0x5628893e64c0 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f02ac595740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628893f1880 con 0x5628893e7690 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f02ac595740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628893faf30 con 0x5628893e8050 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f029bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e64c0 0x5628893eb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f029bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e64c0 0x5628893eb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43654/0 (socket says 172.21.15.83:43654) 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f029bfff640 1 -- 172.21.15.83:0/3230305259 learned_addr learned my addr 172.21.15.83:0/3230305259 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f02a2ffd640 1 --2- 172.21.15.83:0/3230305259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e7690 0x5628893eb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.986+0000 7f029bfff640 1 --2- 172.21.15.83:0/3230305259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e8050 0x5628893f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.987+0000 7f029bfff640 1 -- 172.21.15.83:0/3230305259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e8050 msgr2=0x5628893f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.987+0000 7f029bfff640 1 --2- 172.21.15.83:0/3230305259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e8050 0x5628893f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.987+0000 7f029bfff640 1 -- 172.21.15.83:0/3230305259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e7690 msgr2=0x5628893eb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.987+0000 7f029bfff640 1 --2- 172.21.15.83:0/3230305259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e7690 0x5628893eb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.987+0000 7f029bfff640 1 -- 172.21.15.83:0/3230305259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628892e9c00 con 0x5628893e64c0 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.987+0000 7f029bfff640 1 --2- 172.21.15.83:0/3230305259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e64c0 0x5628893eb270 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f028c0137b0 tx=0x7f028c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c2d948b386a852 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:29.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.987+0000 7f02997fa640 1 -- 172.21.15.83:0/3230305259 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c0132f0 con 0x5628893e64c0 2024-03-11T19:49:29.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.987+0000 7f02997fa640 1 -- 172.21.15.83:0/3230305259 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f028c030820 con 0x5628893e64c0 2024-03-11T19:49:29.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.987+0000 7f02997fa640 1 -- 172.21.15.83:0/3230305259 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c005130 con 0x5628893e64c0 2024-03-11T19:49:29.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.988+0000 7f02ac595740 1 -- 172.21.15.83:0/3230305259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e64c0 msgr2=0x5628893eb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:29.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.988+0000 7f02ac595740 1 --2- 172.21.15.83:0/3230305259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e64c0 0x5628893eb270 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f028c0137b0 tx=0x7f028c009be0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.988+0000 7f02ac595740 1 -- 172.21.15.83:0/3230305259 shutdown_connections 2024-03-11T19:49:29.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.988+0000 7f02ac595740 1 --2- 172.21.15.83:0/3230305259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e8050 0x5628893f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.988+0000 7f02ac595740 1 --2- 172.21.15.83:0/3230305259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e7690 0x5628893eb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.988+0000 7f02ac595740 1 --2- 172.21.15.83:0/3230305259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e64c0 0x5628893eb270 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.988+0000 7f02ac595740 1 -- 172.21.15.83:0/3230305259 >> 172.21.15.83:0/3230305259 conn(0x5628893e16a0 msgr2=0x5628893f06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:29.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.988+0000 7f02ac595740 1 -- 172.21.15.83:0/3230305259 shutdown_connections 2024-03-11T19:49:29.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.988+0000 7f02ac595740 1 -- 172.21.15.83:0/3230305259 wait complete. 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.989+0000 7f02ac595740 1 -- start start 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.989+0000 7f02ac595740 1 Processor -- start 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.989+0000 7f02ac595740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 0x5628893f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.989+0000 7f02ac595740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e7690 0x5628893f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.989+0000 7f02ac595740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e8050 0x5628893f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.989+0000 7f02ac595740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628893f6710 con 0x5628893e8050 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.989+0000 7f02ac595740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628893f6880 con 0x5628893e64c0 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.989+0000 7f02ac595740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628893f6a40 con 0x5628893e7690 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.989+0000 7f02ac595740 10 client.0 _ll_register_callbacks cb 0x7f028c009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f029bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 0x5628893f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f02a2ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e7690 0x5628893f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f029bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 0x5628893f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:50122/0 (socket says 172.21.15.83:50122) 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f029bfff640 1 -- 172.21.15.83:0/3657909228 learned_addr learned my addr 172.21.15.83:0/3657909228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f029bfff640 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e8050 0x5628893f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f02a2ffd640 1 -- 172.21.15.83:0/3657909228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 msgr2=0x5628893f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f02a2ffd640 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 0x5628893f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f02a2ffd640 1 -- 172.21.15.83:0/3657909228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e8050 msgr2=0x5628893f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f02a2ffd640 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e8050 0x5628893f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f02a2ffd640 1 -- 172.21.15.83:0/3657909228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c02b040 con 0x5628893e7690 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f029bfff640 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 0x5628893f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f02a2ffd640 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e7690 0x5628893f51c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0294024be0 tx=0x7f029400f800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce1c722a22c4176 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f0282ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f0282ffd640 1 -- 172.21.15.83:0/3657909228 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400fcd0 con 0x5628893e7690 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f02ac595740 1 -- 172.21.15.83:0/3657909228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562889374900 con 0x5628893e7690 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f0282ffd640 1 -- 172.21.15.83:0/3657909228 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0294000e40 con 0x5628893e7690 2024-03-11T19:49:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f0282ffd640 1 -- 172.21.15.83:0/3657909228 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02940096b0 con 0x5628893e7690 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f02ac595740 10 client.26872 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.990+0000 7f02ac595740 1 -- 172.21.15.83:0/3657909228 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56288930f910 con 0x5628893e7690 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.991+0000 7f0282ffd640 1 -- 172.21.15.83:0/3657909228 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f029403c020 con 0x5628893e7690 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.991+0000 7f0282ffd640 1 -- 172.21.15.83:0/3657909228 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0294025720 con 0x5628893e7690 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.991+0000 7f02ac595740 10 client.26872 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.991+0000 7f02ac595740 10 client.26872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.991+0000 7f02ac595740 20 client.26872 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.991+0000 7f02ac595740 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562889455210 0x562889457630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.991+0000 7f02ac595740 4 client.26872 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.991+0000 7f02ac595740 1 -- 172.21.15.83:0/3657909228 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56288904e860 con 0x562889455210 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.991+0000 7f029bfff640 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562889455210 0x562889457630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.991+0000 7f0282ffd640 10 client.26872 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:30.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.992+0000 7f029bfff640 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562889455210 0x562889457630 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=251fc0d7fadcefd1 server_cookie=590acea9c4e21d13 in_seq=0 out_seq=0 2024-03-11T19:49:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.994+0000 7f0282ffd640 1 -- 172.21.15.83:0/3657909228 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56288904e860 con 0x562889455210 2024-03-11T19:49:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.994+0000 7f0282ffd640 10 client.26872 handle_command_reply: tid=0 2024-03-11T19:49:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.994+0000 7f02ac595740 1 client.26872 shutdown 2024-03-11T19:49:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02ac595740 1 -- 172.21.15.83:0/3657909228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e7690 msgr2=0x5628893f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02ac595740 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e7690 0x5628893f51c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0294024be0 tx=0x7f029400f800 comp rx=0 tx=0).stop 2024-03-11T19:49:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02ac595740 1 -- 172.21.15.83:0/3657909228 shutdown_connections 2024-03-11T19:49:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02ac595740 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562889455210 0x562889457630 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02ac595740 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5628893e8050 0x5628893f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f0282ffd640 0 client.26872 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:30.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02ac595740 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628893e7690 0x5628893f51c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02ac595740 1 --2- 172.21.15.83:0/3657909228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628893e64c0 0x5628893f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02ac595740 1 -- 172.21.15.83:0/3657909228 >> 172.21.15.83:0/3657909228 conn(0x5628893e16a0 msgr2=0x5628893f0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:30.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02ac595740 1 -- 172.21.15.83:0/3657909228 shutdown_connections 2024-03-11T19:49:30.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02ac595740 1 -- 172.21.15.83:0/3657909228 wait complete. 2024-03-11T19:49:30.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02ac595740 20 client.26872 trim_cache size 0 max 16384 2024-03-11T19:49:30.003 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:49:30.003 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:49:30.003 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "3cc6a953-ff84-408f-8fac-bea1fda86119", 2024-03-11T19:49:30.004 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:49:30.004 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:49:30.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.995+0000 7f02a99c9640 1 -- 172.21.15.83:0/974695523 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f02900a1a30 msgr2=0x7f02900a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.996+0000 7f02a99c9640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f02900a1a30 0x7f02900a3ef0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f029c0c5400 tx=0x7f029c0b95a0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.996+0000 7f02a99c9640 1 -- 172.21.15.83:0/974695523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02a4101c40 msgr2=0x7f02a41a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.996+0000 7f02a99c9640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02a4101c40 0x7f02a41a4bd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f029c037040 tx=0x7f029c009360 comp rx=0 tx=0).stop 2024-03-11T19:49:30.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.996+0000 7f02a99c9640 1 -- 172.21.15.83:0/974695523 shutdown_connections 2024-03-11T19:49:30.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.996+0000 7f02a99c9640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f02900a1a30 0x7f02900a3ef0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.996+0000 7f02a99c9640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02a4101c40 0x7f02a41a4bd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.996+0000 7f02a99c9640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02a4101280 0x7f02a41a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.996+0000 7f02a99c9640 1 --2- 172.21.15.83:0/974695523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02a4100090 0x7f02a41a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.996+0000 7f02a99c9640 1 -- 172.21.15.83:0/974695523 >> 172.21.15.83:0/974695523 conn(0x7f02a40fb620 msgr2=0x7f02a40696f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:30.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.996+0000 7f02a99c9640 1 -- 172.21.15.83:0/974695523 shutdown_connections 2024-03-11T19:49:30.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:29.996+0000 7f02a99c9640 1 -- 172.21.15.83:0/974695523 wait complete. 2024-03-11T19:49:30.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:49:30.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:49:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.178+0000 7f45e8e64640 1 -- 172.21.15.83:0/374607652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e41012a0 msgr2=0x7f45e4101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.178+0000 7f45e8e64640 1 --2- 172.21.15.83:0/374607652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e41012a0 0x7f45e4101720 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009900 tx=0x7f45cc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:49:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.179+0000 7f45e8e64640 1 -- 172.21.15.83:0/374607652 shutdown_connections 2024-03-11T19:49:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.179+0000 7f45e8e64640 1 --2- 172.21.15.83:0/374607652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45e4101c60 0x7f45e41084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.179+0000 7f45e8e64640 1 --2- 172.21.15.83:0/374607652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e41012a0 0x7f45e4101720 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.179+0000 7f45e8e64640 1 --2- 172.21.15.83:0/374607652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45e4100590 0x7f45e41009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.179+0000 7f45e8e64640 1 -- 172.21.15.83:0/374607652 >> 172.21.15.83:0/374607652 conn(0x7f45e40fad60 msgr2=0x7f45e40fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:30.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.179+0000 7f45e8e64640 1 -- 172.21.15.83:0/374607652 shutdown_connections 2024-03-11T19:49:30.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.179+0000 7f45e8e64640 1 -- 172.21.15.83:0/374607652 wait complete. 2024-03-11T19:49:30.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.180+0000 7f45e8e64640 1 Processor -- start 2024-03-11T19:49:30.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.180+0000 7f45e8e64640 1 -- start start 2024-03-11T19:49:30.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.180+0000 7f45e8e64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e4100590 0x7f45e4197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.180+0000 7f45e8e64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45e41012a0 0x7f45e4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.180+0000 7f45e259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e4100590 0x7f45e4197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.180+0000 7f45e8e64640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45e4101c60 0x7f45e419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.180+0000 7f45e8e64640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e419ed10 con 0x7f45e4100590 2024-03-11T19:49:30.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.180+0000 7f45e259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e4100590 0x7f45e4197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43694/0 (socket says 172.21.15.83:43694) 2024-03-11T19:49:30.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.180+0000 7f45e259c640 1 -- 172.21.15.83:0/3834840985 learned_addr learned my addr 172.21.15.83:0/3834840985 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.180+0000 7f45e8e64640 1 -- 172.21.15.83:0/3834840985 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e419ee80 con 0x7f45e4101c60 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45e8e64640 1 -- 172.21.15.83:0/3834840985 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e419f040 con 0x7f45e41012a0 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45d9d9b640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45e41012a0 0x7f45e4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45e2d9d640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45e4101c60 0x7f45e419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45e259c640 1 -- 172.21.15.83:0/3834840985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45e41012a0 msgr2=0x7f45e4197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45e259c640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45e41012a0 0x7f45e4197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45e259c640 1 -- 172.21.15.83:0/3834840985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45e4101c60 msgr2=0x7f45e419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45e259c640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45e4101c60 0x7f45e419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45e259c640 1 -- 172.21.15.83:0/3834840985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc009560 con 0x7f45e4100590 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45e2d9d640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45e4101c60 0x7f45e419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45d9d9b640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45e41012a0 0x7f45e4197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45e259c640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e4100590 0x7f45e4197220 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f45d400e9c0 tx=0x7f45d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45dbfff640 1 -- 172.21.15.83:0/3834840985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d40096e0 con 0x7f45e4100590 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45dbfff640 1 -- 172.21.15.83:0/3834840985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45d400cd80 con 0x7f45e4100590 2024-03-11T19:49:30.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45e8e64640 1 -- 172.21.15.83:0/3834840985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e419f320 con 0x7f45e4100590 2024-03-11T19:49:30.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45dbfff640 1 -- 172.21.15.83:0/3834840985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4010430 con 0x7f45e4100590 2024-03-11T19:49:30.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.181+0000 7f45e8e64640 1 -- 172.21.15.83:0/3834840985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e419f7b0 con 0x7f45e4100590 2024-03-11T19:49:30.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.185+0000 7f45eba38740 1 Processor -- start 2024-03-11T19:49:30.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.185+0000 7f45dbfff640 1 -- 172.21.15.83:0/3834840985 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f45d4003680 con 0x7f45e4100590 2024-03-11T19:49:30.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45eba38740 1 -- start start 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45eba38740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c804a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45eba38740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 0x562b80c85050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45eba38740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80c87b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45eba38740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80be7280 con 0x562b80c80a70 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45eba38740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80be47b0 con 0x562b80c80080 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45eba38740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80bdff90 con 0x562b80c856c0 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45d9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80c87b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45d9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80c87b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57356/0 (socket says 172.21.15.83:57356) 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45d9d9b640 1 -- 172.21.15.83:0/991752308 learned_addr learned my addr 172.21.15.83:0/991752308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45e2d9d640 1 --2- 172.21.15.83:0/991752308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c804a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45e259c640 1 --2- 172.21.15.83:0/991752308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 0x562b80c85050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45d9d9b640 1 -- 172.21.15.83:0/991752308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 msgr2=0x562b80c804a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:30.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45d9d9b640 1 --2- 172.21.15.83:0/991752308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c804a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.186+0000 7f45d9d9b640 1 -- 172.21.15.83:0/991752308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 msgr2=0x562b80c85050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:30.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.187+0000 7f45d9d9b640 1 --2- 172.21.15.83:0/991752308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 0x562b80c85050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.187+0000 7f45d9d9b640 1 -- 172.21.15.83:0/991752308 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b80b9e570 con 0x562b80c856c0 2024-03-11T19:49:30.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.187+0000 7f45e259c640 1 --2- 172.21.15.83:0/991752308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 0x562b80c85050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:49:30.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.187+0000 7f45dbfff640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45d00a1a30 0x7f45d00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.187+0000 7f45e2d9d640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45d00a1a30 0x7f45d00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.187+0000 7f45dbfff640 1 -- 172.21.15.83:0/3834840985 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f45d4014070 con 0x7f45e4100590 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.187+0000 7f45d9d9b640 1 --2- 172.21.15.83:0/991752308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80c87b30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005ab0 tx=0x7f45cc005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d4e8280c5c071 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.187+0000 7f45d8d99640 1 -- 172.21.15.83:0/991752308 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004740 con 0x562b80c856c0 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.187+0000 7f45d8d99640 1 -- 172.21.15.83:0/991752308 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45cc0048a0 con 0x562b80c856c0 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.187+0000 7f45d8d99640 1 -- 172.21.15.83:0/991752308 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc005650 con 0x562b80c856c0 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.187+0000 7f45e2d9d640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45d00a1a30 0x7f45d00a3ef0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x562b80c76480 tx=0x7f45dc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.188+0000 7f45eba38740 1 -- 172.21.15.83:0/991752308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 msgr2=0x562b80c87b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.188+0000 7f45eba38740 1 --2- 172.21.15.83:0/991752308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80c87b30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005ab0 tx=0x7f45cc005d50 comp rx=0 tx=0).stop 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.188+0000 7f45eba38740 1 -- 172.21.15.83:0/991752308 shutdown_connections 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.188+0000 7f45eba38740 1 --2- 172.21.15.83:0/991752308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80c87b30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.188+0000 7f45eba38740 1 --2- 172.21.15.83:0/991752308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 0x562b80c85050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.188+0000 7f45eba38740 1 --2- 172.21.15.83:0/991752308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c804a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.188+0000 7f45eba38740 1 -- 172.21.15.83:0/991752308 >> 172.21.15.83:0/991752308 conn(0x562b80c72770 msgr2=0x562b80c72b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.188+0000 7f45eba38740 1 -- 172.21.15.83:0/991752308 shutdown_connections 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.188+0000 7f45eba38740 1 -- 172.21.15.83:0/991752308 wait complete. 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45eba38740 1 -- start start 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45eba38740 1 Processor -- start 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45eba38740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c89a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45eba38740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 0x562b80c89f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45eba38740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80c90fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45eba38740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b808b3430 con 0x562b80c80a70 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45eba38740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8086fe00 con 0x562b80c80080 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45eba38740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80b64fd0 con 0x562b80c856c0 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45d9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 0x562b80c89f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45d9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 0x562b80c89f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43712/0 (socket says 172.21.15.83:43712) 2024-03-11T19:49:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45d9d9b640 1 -- 172.21.15.83:0/3008922333 learned_addr learned my addr 172.21.15.83:0/3008922333 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:30.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45eba38740 10 client.0 _ll_register_callbacks cb 0x562b80bbafd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:30.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.189+0000 7f45e2d9d640 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80c90fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.190+0000 7f45e259c640 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c89a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.190+0000 7f45d9d9b640 1 -- 172.21.15.83:0/3008922333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 msgr2=0x562b80c90fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:30.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.190+0000 7f45d9d9b640 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80c90fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.190+0000 7f45d9d9b640 1 -- 172.21.15.83:0/3008922333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 msgr2=0x562b80c89a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:30.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.190+0000 7f45d9d9b640 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c89a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.190+0000 7f45d9d9b640 1 -- 172.21.15.83:0/3008922333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e4108db0 con 0x562b80c80a70 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.191+0000 7f45d9d9b640 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 0x562b80c89f80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f45cc006010 tx=0x7f45cc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b14a7b125ca144e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.191+0000 7f45c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.192+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3008922333 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc04a650 con 0x562b80c80a70 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.192+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3008922333 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f45cc04a7b0 con 0x562b80c80a70 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.192+0000 7f45eba38740 1 -- 172.21.15.83:0/3008922333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b808df7e0 con 0x562b80c80a70 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.192+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3008922333 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45cc04a970 con 0x562b80c80a70 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.192+0000 7f45eba38740 10 client.17319 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.192+0000 7f45eba38740 1 -- 172.21.15.83:0/3008922333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562b80bde610 con 0x562b80c80a70 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.192+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3008922333 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562b808df7e0 con 0x562b80c80a70 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.193+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3008922333 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f45cc03c050 con 0x562b80c80a70 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.193+0000 7f45eba38740 10 client.17319 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.193+0000 7f45eba38740 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.193+0000 7f45eba38740 20 client.17319 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.193+0000 7f45eba38740 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562b80c95860 0x562b80c97c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.193+0000 7f45eba38740 4 client.17319 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.193+0000 7f45eba38740 1 -- 172.21.15.83:0/3008922333 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b8091c720 con 0x562b80c95860 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.193+0000 7f45e259c640 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562b80c95860 0x562b80c97c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.193+0000 7f45e259c640 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562b80c95860 0x562b80c97c80 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ed013dbe289242 server_cookie=7d40a07ac6cf1d47 in_seq=0 out_seq=0 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.193+0000 7f45c9ffb640 10 client.17319 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.194+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3008922333 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b8091c720 con 0x562b80c95860 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.194+0000 7f45c9ffb640 10 client.17319 handle_command_reply: tid=0 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.194+0000 7f45eba38740 1 client.17319 shutdown 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45eba38740 1 -- 172.21.15.83:0/3008922333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 msgr2=0x562b80c89f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45eba38740 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 0x562b80c89f80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f45cc006010 tx=0x7f45cc004740 comp rx=0 tx=0).stop 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45eba38740 1 -- 172.21.15.83:0/3008922333 shutdown_connections 2024-03-11T19:49:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45eba38740 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562b80c95860 0x562b80c97c80 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45eba38740 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80c90fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45c9ffb640 0 client.17319 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:30.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45eba38740 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80a70 0x562b80c89f80 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45eba38740 1 --2- 172.21.15.83:0/3008922333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c89a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45eba38740 1 -- 172.21.15.83:0/3008922333 >> 172.21.15.83:0/3008922333 conn(0x562b80c72770 msgr2=0x562b80c7f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:30.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45eba38740 1 -- 172.21.15.83:0/3008922333 shutdown_connections 2024-03-11T19:49:30.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45eba38740 1 -- 172.21.15.83:0/3008922333 wait complete. 2024-03-11T19:49:30.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.195+0000 7f45eba38740 20 client.17319 trim_cache size 0 max 16384 2024-03-11T19:49:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.216+0000 7f45eba38740 1 Processor -- start 2024-03-11T19:49:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45eba38740 1 -- start start 2024-03-11T19:49:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45eba38740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c81810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45eba38740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c80a70 0x562b80c81d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45eba38740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c856c0 0x562b80c90f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45eba38740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80c6deb0 con 0x562b80c856c0 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45eba38740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8091c720 con 0x562b80c80080 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45eba38740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b808df7e0 con 0x562b80c80a70 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45d9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c80a70 0x562b80c81d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45e2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c856c0 0x562b80c90f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45e259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c81810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45d9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c80a70 0x562b80c81d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57374/0 (socket says 172.21.15.83:57374) 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45d9d9b640 1 -- 172.21.15.83:0/3333358966 learned_addr learned my addr 172.21.15.83:0/3333358966 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45e259c640 1 -- 172.21.15.83:0/3333358966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c80a70 msgr2=0x562b80c81d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45e259c640 1 --2- 172.21.15.83:0/3333358966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c80a70 0x562b80c81d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45e259c640 1 -- 172.21.15.83:0/3333358966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c856c0 msgr2=0x562b80c90f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45e259c640 1 --2- 172.21.15.83:0/3333358966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c856c0 0x562b80c90f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45e259c640 1 -- 172.21.15.83:0/3333358966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b80b7aac0 con 0x562b80c80080 2024-03-11T19:49:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.217+0000 7f45e2d9d640 1 --2- 172.21.15.83:0/3333358966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c856c0 0x562b80c90f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:49:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.218+0000 7f45e259c640 1 --2- 172.21.15.83:0/3333358966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c81810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x562b80c80e70 tx=0x7f45d40ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c22632b3aaffda06 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.218+0000 7f45cbfff640 1 -- 172.21.15.83:0/3333358966 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d409a110 con 0x562b80c80080 2024-03-11T19:49:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.218+0000 7f45cbfff640 1 -- 172.21.15.83:0/3333358966 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45d409aeb0 con 0x562b80c80080 2024-03-11T19:49:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.218+0000 7f45cbfff640 1 -- 172.21.15.83:0/3333358966 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4098bd0 con 0x562b80c80080 2024-03-11T19:49:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.218+0000 7f45eba38740 1 -- 172.21.15.83:0/3333358966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 msgr2=0x562b80c81810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.218+0000 7f45eba38740 1 --2- 172.21.15.83:0/3333358966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c81810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x562b80c80e70 tx=0x7f45d40ba040 comp rx=0 tx=0).stop 2024-03-11T19:49:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 -- 172.21.15.83:0/3333358966 shutdown_connections 2024-03-11T19:49:30.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 --2- 172.21.15.83:0/3333358966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c856c0 0x562b80c90f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 --2- 172.21.15.83:0/3333358966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c80a70 0x562b80c81d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 --2- 172.21.15.83:0/3333358966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80080 0x562b80c81810 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 -- 172.21.15.83:0/3333358966 >> 172.21.15.83:0/3333358966 conn(0x562b80c72770 msgr2=0x562b80c89a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 -- 172.21.15.83:0/3333358966 shutdown_connections 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 -- 172.21.15.83:0/3333358966 wait complete. 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 -- start start 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 Processor -- start 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80080 0x562b80c8fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80a70 0x562b80c90430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80cebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45e259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80080 0x562b80c8fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.219+0000 7f45eba38740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80c90b30 con 0x562b80c80080 2024-03-11T19:49:30.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.220+0000 7f45eba38740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80cec290 con 0x562b80c80a70 2024-03-11T19:49:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.220+0000 7f45e259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80080 0x562b80c8fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43724/0 (socket says 172.21.15.83:43724) 2024-03-11T19:49:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.220+0000 7f45eba38740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b80cec3d0 con 0x562b80c856c0 2024-03-11T19:49:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.220+0000 7f45e259c640 1 -- 172.21.15.83:0/3164771396 learned_addr learned my addr 172.21.15.83:0/3164771396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:49:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.220+0000 7f45eba38740 10 client.0 _ll_register_callbacks cb 0x7f45d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:49:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.220+0000 7f45d9d9b640 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80a70 0x562b80c90430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.220+0000 7f45e2d9d640 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80cebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.221+0000 7f45e2d9d640 1 -- 172.21.15.83:0/3164771396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80a70 msgr2=0x562b80c90430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.221+0000 7f45e2d9d640 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80a70 0x562b80c90430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.221+0000 7f45e2d9d640 1 -- 172.21.15.83:0/3164771396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80080 msgr2=0x562b80c8fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:49:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.221+0000 7f45e2d9d640 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80080 0x562b80c8fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.221+0000 7f45e2d9d640 1 -- 172.21.15.83:0/3164771396 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc010550 con 0x562b80c856c0 2024-03-11T19:49:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.221+0000 7f45d9d9b640 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80a70 0x562b80c90430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:49:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.221+0000 7f45e2d9d640 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80cebd50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f45dc010d30 tx=0x7f45dc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0bb8947ed78fdd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:49:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.221+0000 7f45c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:49:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.229+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3164771396 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc016070 con 0x562b80c856c0 2024-03-11T19:49:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.229+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3164771396 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f45dc010e20 con 0x562b80c856c0 2024-03-11T19:49:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.231+0000 7f45eba38740 1 -- 172.21.15.83:0/3164771396 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45cc0048a0 con 0x562b80c856c0 2024-03-11T19:49:30.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.235+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3164771396 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45dc004b20 con 0x562b80c856c0 2024-03-11T19:49:30.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.244+0000 7f45eba38740 10 client.26890 fetch_fsmap learned FSMap version 32 2024-03-11T19:49:30.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.244+0000 7f45eba38740 1 -- 172.21.15.83:0/3164771396 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f45e419f7b0 con 0x562b80c856c0 2024-03-11T19:49:30.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.246+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3164771396 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45cc0048a0 con 0x562b80c856c0 2024-03-11T19:49:30.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.246+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3164771396 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f45dc029700 con 0x562b80c856c0 2024-03-11T19:49:30.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.249+0000 7f45eba38740 10 client.26890 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:49:30.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.249+0000 7f45eba38740 10 client.26890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:49:30.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.249+0000 7f45eba38740 20 client.26890 populate_metadata read hostname 'smithi083' 2024-03-11T19:49:30.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.249+0000 7f45eba38740 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562b80cedf30 0x562b80cf0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:49:30.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.249+0000 7f45eba38740 4 client.26890 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:49:30.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.249+0000 7f45eba38740 1 -- 172.21.15.83:0/3164771396 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562b8096b890 con 0x562b80cedf30 2024-03-11T19:49:30.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.249+0000 7f45e259c640 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562b80cedf30 0x562b80cf0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:49:30.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.250+0000 7f45e259c640 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562b80cedf30 0x562b80cf0350 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9786611e108cd3a4 server_cookie=bff710436ac0101c in_seq=0 out_seq=0 2024-03-11T19:49:30.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.250+0000 7f45c9ffb640 10 client.26890 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:30.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45c9ffb640 1 -- 172.21.15.83:0/3164771396 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562b8096b890 con 0x562b80cedf30 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45c9ffb640 10 client.26890 handle_command_reply: tid=0 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 1 client.26890 shutdown 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 1 -- 172.21.15.83:0/3164771396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 msgr2=0x562b80cebd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80cebd50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f45dc010d30 tx=0x7f45dc004ab0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 1 -- 172.21.15.83:0/3164771396 shutdown_connections 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562b80cedf30 0x562b80cf0350 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b80c856c0 0x562b80cebd50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b80c80a70 0x562b80c90430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45c9ffb640 0 client.26890 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 1 --2- 172.21.15.83:0/3164771396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b80c80080 0x562b80c8fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 1 -- 172.21.15.83:0/3164771396 >> 172.21.15.83:0/3164771396 conn(0x562b80c72770 msgr2=0x562b80c89910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 1 -- 172.21.15.83:0/3164771396 shutdown_connections 2024-03-11T19:49:30.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 1 -- 172.21.15.83:0/3164771396 wait complete. 2024-03-11T19:49:30.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.253+0000 7f45eba38740 20 client.26890 trim_cache size 0 max 16384 2024-03-11T19:49:30.259 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:49:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:49:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:49:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "3cc6a953-ff84-408f-8fac-bea1fda86119": { 2024-03-11T19:49:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:49:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "3cc6a953-ff84-408f-8fac-bea1fda86119", 2024-03-11T19:49:30.259 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:49:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:49:30.260 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:49:30.260 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:49:30.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.254+0000 7f45e8e64640 1 -- 172.21.15.83:0/3834840985 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45d00a1a30 msgr2=0x7f45d00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.254+0000 7f45e8e64640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45d00a1a30 0x7f45d00a3ef0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x562b80c76480 tx=0x7f45dc009500 comp rx=0 tx=0).stop 2024-03-11T19:49:30.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.254+0000 7f45e8e64640 1 -- 172.21.15.83:0/3834840985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e4100590 msgr2=0x7f45e4197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:49:30.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.254+0000 7f45e8e64640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e4100590 0x7f45e4197220 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f45d400e9c0 tx=0x7f45d400ee80 comp rx=0 tx=0).stop 2024-03-11T19:49:30.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.255+0000 7f45e8e64640 1 -- 172.21.15.83:0/3834840985 shutdown_connections 2024-03-11T19:49:30.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.255+0000 7f45e8e64640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f45d00a1a30 0x7f45d00a3ef0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.255+0000 7f45e8e64640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45e4101c60 0x7f45e419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.255+0000 7f45e8e64640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45e41012a0 0x7f45e4197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.255+0000 7f45e8e64640 1 --2- 172.21.15.83:0/3834840985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e4100590 0x7f45e4197220 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:49:30.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.255+0000 7f45e8e64640 1 -- 172.21.15.83:0/3834840985 >> 172.21.15.83:0/3834840985 conn(0x7f45e40fad60 msgr2=0x7f45e40fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:49:30.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.256+0000 7f45e8e64640 1 -- 172.21.15.83:0/3834840985 shutdown_connections 2024-03-11T19:49:30.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:49:30.256+0000 7f45e8e64640 1 -- 172.21.15.83:0/3834840985 wait complete. 2024-03-11T19:49:30.273 INFO:tasks.cephfs.filesystem:scrub status for tag:3cc6a953-ff84-408f-8fac-bea1fda86119 - {'path': '/', 'tag': '3cc6a953-ff84-408f-8fac-bea1fda86119', 'options': 'recursive,force'} 2024-03-11T19:49:30.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:30 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:49:30.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:30 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:30.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:30 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:49:30.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:30 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:30.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:30 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:49:30.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:30 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:31 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:49:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:31 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:31 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:49:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:31 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:49:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:31 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:31 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:31.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:31 smithi083 ceph-mon[31864]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.3 MiB/s wr, 598 op/s 2024-03-11T19:49:31.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:31 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:49:31.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:31 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:31.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:31 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:49:31.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:31 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:49:31.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:31 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:31.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:31 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:31.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:31 smithi137 ceph-mon[36220]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.3 MiB/s wr, 598 op/s 2024-03-11T19:49:31.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:31 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:49:31.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:31 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:31.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:31 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:49:31.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:31 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:49:31.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:31 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:31.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:31 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:49:31.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:31 smithi204 ceph-mon[36739]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.3 MiB/s wr, 598 op/s 2024-03-11T19:49:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 615.0 s, 1947.4 tps, lat 51.018 ms stddev 60.212 2024-03-11T19:49:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:32 smithi083 ceph-mon[31864]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.7 MiB/s wr, 672 op/s 2024-03-11T19:49:33.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:32 smithi137 ceph-mon[36220]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.7 MiB/s wr, 672 op/s 2024-03-11T19:49:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:32 smithi204 ceph-mon[36739]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 8.7 MiB/s wr, 672 op/s 2024-03-11T19:49:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:34 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:35.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:34 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:34 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:49:35.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:35 smithi204 ceph-mon[36739]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.9 MiB/s wr, 722 op/s 2024-03-11T19:49:35.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:35 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:49:36.026 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:35 smithi137 ceph-mon[36220]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.9 MiB/s wr, 722 op/s 2024-03-11T19:49:36.026 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:35 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:49:36.033 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:35 smithi083 ceph-mon[31864]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.9 MiB/s wr, 722 op/s 2024-03-11T19:49:36.033 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:35 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:49:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:36 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:49:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:36 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:49:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:36 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:37.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:36 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:49:37.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:36 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:49:37.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:37.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:36 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:36 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:49:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:36 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:49:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:36 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 620.0 s, 2295.6 tps, lat 43.104 ms stddev 52.157 2024-03-11T19:49:38.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:38.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:37 smithi083 ceph-mon[31864]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.9 MiB/s wr, 637 op/s 2024-03-11T19:49:38.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:38.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:38.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:37 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:38.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:38.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:37 smithi137 ceph-mon[36220]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.9 MiB/s wr, 637 op/s 2024-03-11T19:49:38.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:38.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:38.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:37 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:38.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:38.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:37 smithi204 ceph-mon[36739]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.9 MiB/s wr, 637 op/s 2024-03-11T19:49:38.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:38.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:38.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:37 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:39.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:39.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:39.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:49:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:49:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:49:40.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:39 smithi083 ceph-mon[31864]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 859 op/s 2024-03-11T19:49:40.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:39 smithi137 ceph-mon[36220]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 859 op/s 2024-03-11T19:49:40.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:39 smithi204 ceph-mon[36739]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 859 op/s 2024-03-11T19:49:41.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:40 smithi083 ceph-mon[31864]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 9.0 MiB/s wr, 739 op/s 2024-03-11T19:49:41.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:40 smithi137 ceph-mon[36220]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 9.0 MiB/s wr, 739 op/s 2024-03-11T19:49:41.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:40 smithi204 ceph-mon[36739]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 9.0 MiB/s wr, 739 op/s 2024-03-11T19:49:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 625.0 s, 2658.8 tps, lat 38.304 ms stddev 47.040 2024-03-11T19:49:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:42 smithi083 ceph-mon[31864]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 MiB/s wr, 715 op/s 2024-03-11T19:49:43.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:42 smithi137 ceph-mon[36220]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 MiB/s wr, 715 op/s 2024-03-11T19:49:43.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:42 smithi204 ceph-mon[36739]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.1 MiB/s wr, 715 op/s 2024-03-11T19:49:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:44 smithi083 ceph-mon[31864]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.1 MiB/s wr, 787 op/s 2024-03-11T19:49:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:44 smithi137 ceph-mon[36220]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.1 MiB/s wr, 787 op/s 2024-03-11T19:49:45.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:44 smithi204 ceph-mon[36739]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.1 MiB/s wr, 787 op/s 2024-03-11T19:49:47.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:46 smithi083 ceph-mon[31864]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.3 MiB/s wr, 693 op/s 2024-03-11T19:49:47.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:46 smithi137 ceph-mon[36220]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.3 MiB/s wr, 693 op/s 2024-03-11T19:49:47.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:46 smithi204 ceph-mon[36739]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.3 MiB/s wr, 693 op/s 2024-03-11T19:49:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 630.0 s, 2545.4 tps, lat 39.232 ms stddev 46.892 2024-03-11T19:49:49.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:48 smithi083 ceph-mon[31864]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 834 op/s 2024-03-11T19:49:49.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:48 smithi137 ceph-mon[36220]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 834 op/s 2024-03-11T19:49:49.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:48 smithi204 ceph-mon[36739]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 MiB/s wr, 834 op/s 2024-03-11T19:49:51.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:50 smithi083 ceph-mon[31864]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.1 MiB/s wr, 637 op/s 2024-03-11T19:49:51.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:50 smithi137 ceph-mon[36220]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.1 MiB/s wr, 637 op/s 2024-03-11T19:49:51.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:50 smithi204 ceph-mon[36739]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.1 MiB/s wr, 637 op/s 2024-03-11T19:49:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 635.0 s, 1734.8 tps, lat 57.569 ms stddev 61.605 2024-03-11T19:49:53.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:52 smithi083 ceph-mon[31864]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 MiB/s wr, 640 op/s 2024-03-11T19:49:53.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:52 smithi137 ceph-mon[36220]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 MiB/s wr, 640 op/s 2024-03-11T19:49:53.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:52 smithi204 ceph-mon[36739]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.9 MiB/s wr, 640 op/s 2024-03-11T19:49:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:54 smithi083 ceph-mon[31864]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 10 MiB/s wr, 727 op/s 2024-03-11T19:49:55.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:54 smithi137 ceph-mon[36220]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 10 MiB/s wr, 727 op/s 2024-03-11T19:49:55.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:54 smithi204 ceph-mon[36739]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 10 MiB/s wr, 727 op/s 2024-03-11T19:49:57.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:56 smithi083 ceph-mon[31864]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.7 MiB/s wr, 575 op/s 2024-03-11T19:49:57.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:56 smithi137 ceph-mon[36220]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.7 MiB/s wr, 575 op/s 2024-03-11T19:49:57.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:56 smithi204 ceph-mon[36739]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.7 MiB/s wr, 575 op/s 2024-03-11T19:49:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 640.0 s, 2439.8 tps, lat 40.704 ms stddev 49.543 2024-03-11T19:49:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:49:58 smithi083 ceph-mon[31864]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 706 op/s 2024-03-11T19:49:59.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:49:58 smithi137 ceph-mon[36220]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 706 op/s 2024-03-11T19:49:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:49:58 smithi204 ceph-mon[36739]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 706 op/s 2024-03-11T19:50:00.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:50:00.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:50:00.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:00 smithi083 ceph-mon[31864]: overall HEALTH_OK 2024-03-11T19:50:00.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:00 smithi137 ceph-mon[36220]: overall HEALTH_OK 2024-03-11T19:50:00.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:00 smithi204 ceph-mon[36739]: overall HEALTH_OK 2024-03-11T19:50:00.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.447+0000 7fd3a1302640 1 -- 172.21.15.83:0/2126517267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd39c0788a0 msgr2=0x7fd39c07ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.447+0000 7fd3a1302640 1 --2- 172.21.15.83:0/2126517267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd39c0788a0 0x7fd39c07ad30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd39000af00 tx=0x7fd39002fe80 comp rx=0 tx=0).stop 2024-03-11T19:50:00.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.447+0000 7fd3a1302640 1 -- 172.21.15.83:0/2126517267 shutdown_connections 2024-03-11T19:50:00.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.447+0000 7fd3a1302640 1 --2- 172.21.15.83:0/2126517267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd39c0788a0 0x7fd39c07ad30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.447+0000 7fd3a1302640 1 --2- 172.21.15.83:0/2126517267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd39c075ef0 0x7fd39c078360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.447+0000 7fd3a1302640 1 --2- 172.21.15.83:0/2126517267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c073010 0x7fd39c073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.447+0000 7fd3a1302640 1 -- 172.21.15.83:0/2126517267 >> 172.21.15.83:0/2126517267 conn(0x7fd39c06e200 msgr2=0x7fd39c070640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:00.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.451+0000 7fd3a1302640 1 -- 172.21.15.83:0/2126517267 shutdown_connections 2024-03-11T19:50:00.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.452+0000 7fd3a1302640 1 -- 172.21.15.83:0/2126517267 wait complete. 2024-03-11T19:50:00.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.452+0000 7fd3a1302640 1 Processor -- start 2024-03-11T19:50:00.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.453+0000 7fd3a1302640 1 -- start start 2024-03-11T19:50:00.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.453+0000 7fd3a1302640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c073010 0x7fd39c1b96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.453+0000 7fd3a1302640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd39c075ef0 0x7fd39c1bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.453+0000 7fd3a1302640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd39c1bc1a0 0x7fd39c1be630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.453+0000 7fd3a1302640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c1bedc0 con 0x7fd39c073010 2024-03-11T19:50:00.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.453+0000 7fd3a1302640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c1aeed0 con 0x7fd39c075ef0 2024-03-11T19:50:00.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.453+0000 7fd3a1302640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c1af090 con 0x7fd39c1bc1a0 2024-03-11T19:50:00.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd39c1bc1a0 0x7fd39c1be630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd39c1bc1a0 0x7fd39c1be630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37354/0 (socket says 172.21.15.83:37354) 2024-03-11T19:50:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39b7fe640 1 -- 172.21.15.83:0/895984362 learned_addr learned my addr 172.21.15.83:0/895984362 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd39c075ef0 0x7fd39c1bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39affd640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c073010 0x7fd39c1b96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39b7fe640 1 -- 172.21.15.83:0/895984362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd39c075ef0 msgr2=0x7fd39c1bbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39b7fe640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd39c075ef0 0x7fd39c1bbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39b7fe640 1 -- 172.21.15.83:0/895984362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c073010 msgr2=0x7fd39c1b96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39b7fe640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c073010 0x7fd39c1b96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39b7fe640 1 -- 172.21.15.83:0/895984362 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3900081d0 con 0x7fd39c1bc1a0 2024-03-11T19:50:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39affd640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c073010 0x7fd39c1b96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:50:00.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.454+0000 7fd39b7fe640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd39c1bc1a0 0x7fd39c1be630 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd390030490 tx=0x7fd39000a670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:00.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.455+0000 7fd383fff640 1 -- 172.21.15.83:0/895984362 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39000aaa0 con 0x7fd39c1bc1a0 2024-03-11T19:50:00.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.455+0000 7fd383fff640 1 -- 172.21.15.83:0/895984362 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd390012250 con 0x7fd39c1bc1a0 2024-03-11T19:50:00.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.455+0000 7fd383fff640 1 -- 172.21.15.83:0/895984362 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3900116a0 con 0x7fd39c1bc1a0 2024-03-11T19:50:00.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.455+0000 7fd3a1302640 1 -- 172.21.15.83:0/895984362 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39c1af310 con 0x7fd39c1bc1a0 2024-03-11T19:50:00.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.456+0000 7fd3a1302640 1 -- 172.21.15.83:0/895984362 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39c1af850 con 0x7fd39c1bc1a0 2024-03-11T19:50:00.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.459+0000 7fd3a3fb5740 1 Processor -- start 2024-03-11T19:50:00.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.460+0000 7fd3a3fb5740 1 -- start start 2024-03-11T19:50:00.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.460+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dc2d0 0x5651253dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.460+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 0x5651253e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.460+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x5651253e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.460+0000 7fd3a3fb5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56512534c280 con 0x5651253e1e30 2024-03-11T19:50:00.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.460+0000 7fd3a3fb5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651253497b0 con 0x5651253dc2d0 2024-03-11T19:50:00.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.460+0000 7fd3a3fb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125344f90 con 0x5651253dccc0 2024-03-11T19:50:00.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.461+0000 7fd383fff640 1 -- 172.21.15.83:0/895984362 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd390011800 con 0x7fd39c1bc1a0 2024-03-11T19:50:00.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.461+0000 7fd39a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dc2d0 0x5651253dc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.461+0000 7fd39a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dc2d0 0x5651253dc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51460/0 (socket says 172.21.15.83:51460) 2024-03-11T19:50:00.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.461+0000 7fd39a7fc640 1 -- 172.21.15.83:0/3460972050 learned_addr learned my addr 172.21.15.83:0/3460972050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:00.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.461+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/3460972050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x5651253e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.461+0000 7fd383fff640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3840a1b70 0x7fd3840a4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.462+0000 7fd383fff640 1 -- 172.21.15.83:0/895984362 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd3900f0660 con 0x7fd39c1bc1a0 2024-03-11T19:50:00.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.462+0000 7fd39affd640 1 --2- 172.21.15.83:0/3460972050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 0x5651253e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.462+0000 7fd39b7fe640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3840a1b70 0x7fd3840a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.462+0000 7fd39a7fc640 1 -- 172.21.15.83:0/3460972050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 msgr2=0x5651253e17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:00.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.462+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/3460972050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 0x5651253e17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.462+0000 7fd39a7fc640 1 -- 172.21.15.83:0/3460972050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 msgr2=0x5651253e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:00.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.462+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/3460972050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x5651253e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.462+0000 7fd39a7fc640 1 -- 172.21.15.83:0/3460972050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565125303570 con 0x5651253dc2d0 2024-03-11T19:50:00.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.464+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/3460972050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dc2d0 0x5651253dc6f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd39400db70 tx=0x7fd394009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71b16266a2d775a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:00.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.467+0000 7fd3817fa640 1 -- 172.21.15.83:0/3460972050 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39400ac70 con 0x5651253dc2d0 2024-03-11T19:50:00.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.467+0000 7fd3817fa640 1 -- 172.21.15.83:0/3460972050 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd39400add0 con 0x5651253dc2d0 2024-03-11T19:50:00.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.467+0000 7fd3817fa640 1 -- 172.21.15.83:0/3460972050 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39400bb90 con 0x5651253dc2d0 2024-03-11T19:50:00.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.469+0000 7fd39b7fe640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3840a1b70 0x7fd3840a4030 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd3900084c0 tx=0x7fd39000a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:00.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.470+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/3460972050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dc2d0 msgr2=0x5651253dc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.470+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/3460972050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dc2d0 0x5651253dc6f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd39400db70 tx=0x7fd394009800 comp rx=0 tx=0).stop 2024-03-11T19:50:00.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.470+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/3460972050 shutdown_connections 2024-03-11T19:50:00.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.470+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/3460972050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x5651253e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.470+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/3460972050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 0x5651253e17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.470+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/3460972050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dc2d0 0x5651253dc6f0 secure :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd39400db70 tx=0x7fd394009800 comp rx=0 tx=0).stop 2024-03-11T19:50:00.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.470+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/3460972050 >> 172.21.15.83:0/3460972050 conn(0x5651253d77f0 msgr2=0x5651253d7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:00.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.472+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/3460972050 shutdown_connections 2024-03-11T19:50:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.472+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/3460972050 wait complete. 2024-03-11T19:50:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.473+0000 7fd3a3fb5740 1 -- start start 2024-03-11T19:50:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.473+0000 7fd3a3fb5740 1 Processor -- start 2024-03-11T19:50:00.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.473+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 0x5651253e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.473+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x5651253e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.473+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253eba20 0x5651253ede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.473+0000 7fd3a3fb5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125018430 con 0x5651253e1e30 2024-03-11T19:50:00.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.473+0000 7fd3a3fb5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x565124fd4e00 con 0x5651253dccc0 2024-03-11T19:50:00.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.473+0000 7fd3a3fb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651252c9fd0 con 0x5651253eba20 2024-03-11T19:50:00.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.473+0000 7fd3a3fb5740 10 client.0 _ll_register_callbacks cb 0x56512531ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:00.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.477+0000 7fd39a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 0x5651253e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.477+0000 7fd39a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253eba20 0x5651253ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.477+0000 7fd39a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 0x5651253e64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51476/0 (socket says 172.21.15.83:51476) 2024-03-11T19:50:00.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.477+0000 7fd39a7fc640 1 -- 172.21.15.83:0/1463378426 learned_addr learned my addr 172.21.15.83:0/1463378426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:00.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.478+0000 7fd39affd640 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x5651253e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.478+0000 7fd39a7fc640 1 -- 172.21.15.83:0/1463378426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253eba20 msgr2=0x5651253ede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:00.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.478+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253eba20 0x5651253ede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.478+0000 7fd39a7fc640 1 -- 172.21.15.83:0/1463378426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 msgr2=0x5651253e69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:00.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.478+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x5651253e69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.478+0000 7fd39a7fc640 1 -- 172.21.15.83:0/1463378426 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd394015040 con 0x5651253dccc0 2024-03-11T19:50:00.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.478+0000 7fd39affd640 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x5651253e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:50:00.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.478+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 0x5651253e64b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d9b0 tx=0x7fd394009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=409fe3ac7fba5e40 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:00.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.479+0000 7fd37eff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:50:00.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.479+0000 7fd37eff5640 1 -- 172.21.15.83:0/1463378426 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39401f070 con 0x5651253dccc0 2024-03-11T19:50:00.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.479+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/1463378426 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651250447e0 con 0x5651253dccc0 2024-03-11T19:50:00.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.479+0000 7fd37eff5640 1 -- 172.21.15.83:0/1463378426 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd394003a30 con 0x5651253dccc0 2024-03-11T19:50:00.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.479+0000 7fd37eff5640 1 -- 172.21.15.83:0/1463378426 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd394003bf0 con 0x5651253dccc0 2024-03-11T19:50:00.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.479+0000 7fd3a3fb5740 10 client.26792 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:00.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.479+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/1463378426 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x565125343610 con 0x5651253dccc0 2024-03-11T19:50:00.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.480+0000 7fd37eff5640 1 -- 172.21.15.83:0/1463378426 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5651250447e0 con 0x5651253dccc0 2024-03-11T19:50:00.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.480+0000 7fd37eff5640 1 -- 172.21.15.83:0/1463378426 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd3940041e0 con 0x5651253dccc0 2024-03-11T19:50:00.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.481+0000 7fd3a3fb5740 10 client.26792 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:00.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.481+0000 7fd3a3fb5740 10 client.26792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:00.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.481+0000 7fd3a3fb5740 20 client.26792 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:00.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.481+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5651253f28c0 0x5651253f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.481+0000 7fd3a3fb5740 4 client.26792 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:00.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.481+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/1463378426 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565125081720 con 0x5651253f28c0 2024-03-11T19:50:00.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.481+0000 7fd39affd640 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5651253f28c0 0x5651253f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.481+0000 7fd39affd640 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5651253f28c0 0x5651253f4cb0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17744a3bc99a830e server_cookie=bb86193f53b19866 in_seq=0 out_seq=0 2024-03-11T19:50:00.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.481+0000 7fd37eff5640 10 client.26792 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd37eff5640 1 -- 172.21.15.83:0/1463378426 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565125081720 con 0x5651253f28c0 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd37eff5640 10 client.26792 handle_command_reply: tid=0 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd3a3fb5740 1 client.26792 shutdown 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/1463378426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 msgr2=0x5651253e64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 0x5651253e64b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d9b0 tx=0x7fd394009540 comp rx=0 tx=0).stop 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/1463378426 shutdown_connections 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5651253f28c0 0x5651253f4cb0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253eba20 0x5651253ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x5651253e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/1463378426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 0x5651253e64b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/1463378426 >> 172.21.15.83:0/1463378426 conn(0x5651253d77f0 msgr2=0x5651253d6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:00.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd37eff5640 0 client.26792 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:00.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/1463378426 shutdown_connections 2024-03-11T19:50:00.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.483+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/1463378426 wait complete. 2024-03-11T19:50:00.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.484+0000 7fd3a3fb5740 20 client.26792 trim_cache size 0 max 16384 2024-03-11T19:50:00.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.507+0000 7fd3a3fb5740 1 Processor -- start 2024-03-11T19:50:00.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.507+0000 7fd3a3fb5740 1 -- start start 2024-03-11T19:50:00.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.507+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 0x5651253f9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.507+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253e1e30 0x565125402b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253eba20 0x565125403090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd39affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253e1e30 0x565125402b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd39affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253e1e30 0x565125402b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37378/0 (socket says 172.21.15.83:37378) 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd39a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 0x5651253f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd39a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 0x5651253f9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51488/0 (socket says 172.21.15.83:51488) 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd39affd640 1 -- 172.21.15.83:0/677284712 learned_addr learned my addr 172.21.15.83:0/677284712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/677284712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125081720 con 0x5651253eba20 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/677284712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651250447e0 con 0x5651253dccc0 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/677284712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651253d7660 con 0x5651253e1e30 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/677284712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253eba20 0x565125403090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd39affd640 1 -- 172.21.15.83:0/677284712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 msgr2=0x5651253f9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.508+0000 7fd39affd640 1 --2- 172.21.15.83:0/677284712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 0x5651253f9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.509+0000 7fd39affd640 1 -- 172.21.15.83:0/677284712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253eba20 msgr2=0x565125403090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.509+0000 7fd39affd640 1 --2- 172.21.15.83:0/677284712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253eba20 0x565125403090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.509+0000 7fd39affd640 1 -- 172.21.15.83:0/677284712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651252dfac0 con 0x5651253e1e30 2024-03-11T19:50:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.509+0000 7fd39affd640 1 --2- 172.21.15.83:0/677284712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253e1e30 0x565125402b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5651253dd0c0 tx=0x7fd38c009170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9f3605eaad7c5a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:00.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.509+0000 7fd380ff9640 1 -- 172.21.15.83:0/677284712 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c013cd0 con 0x5651253e1e30 2024-03-11T19:50:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.509+0000 7fd380ff9640 1 -- 172.21.15.83:0/677284712 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd38c01b070 con 0x5651253e1e30 2024-03-11T19:50:00.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.510+0000 7fd380ff9640 1 -- 172.21.15.83:0/677284712 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c01e670 con 0x5651253e1e30 2024-03-11T19:50:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.511+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/677284712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253e1e30 msgr2=0x565125402b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.511+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/677284712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253e1e30 0x565125402b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5651253dd0c0 tx=0x7fd38c009170 comp rx=0 tx=0).stop 2024-03-11T19:50:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.511+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/677284712 shutdown_connections 2024-03-11T19:50:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.511+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/677284712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253eba20 0x565125403090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.511+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/677284712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253e1e30 0x565125402b50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.511+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/677284712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253dccc0 0x5651253f9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.511+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/677284712 >> 172.21.15.83:0/677284712 conn(0x5651253d77f0 msgr2=0x5651253d7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.511+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/677284712 shutdown_connections 2024-03-11T19:50:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.511+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/677284712 wait complete. 2024-03-11T19:50:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd3a3fb5740 1 -- start start 2024-03-11T19:50:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd3a3fb5740 1 Processor -- start 2024-03-11T19:50:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 0x5651254027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x565125402d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd3a3fb5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253eba20 0x565125447f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd3a3fb5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125448460 con 0x5651253e1e30 2024-03-11T19:50:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd3a3fb5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651254485a0 con 0x5651253eba20 2024-03-11T19:50:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd3a3fb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565125448730 con 0x5651253dccc0 2024-03-11T19:50:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd39a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 0x5651254027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd3a3fb5740 10 client.0 _ll_register_callbacks cb 0x7fd394009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd39a7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 0x5651254027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37386/0 (socket says 172.21.15.83:37386) 2024-03-11T19:50:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.512+0000 7fd39a7fc640 1 -- 172.21.15.83:0/2637286858 learned_addr learned my addr 172.21.15.83:0/2637286858 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.513+0000 7fd39affd640 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x565125402d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.513+0000 7fd39a7fc640 1 -- 172.21.15.83:0/2637286858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253eba20 msgr2=0x565125447f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.513+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253eba20 0x565125447f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.513+0000 7fd39a7fc640 1 -- 172.21.15.83:0/2637286858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 msgr2=0x565125402d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.513+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x565125402d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.513+0000 7fd39a7fc640 1 -- 172.21.15.83:0/2637286858 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c009750 con 0x5651253dccc0 2024-03-11T19:50:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.513+0000 7fd39a7fc640 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 0x5651254027c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d9b0 tx=0x7fd394004a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e666945b17bae42 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.513+0000 7fd37eff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:50:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.514+0000 7fd37eff5640 1 -- 172.21.15.83:0/2637286858 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39401f040 con 0x5651253dccc0 2024-03-11T19:50:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.514+0000 7fd37eff5640 1 -- 172.21.15.83:0/2637286858 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd394009e80 con 0x5651253dccc0 2024-03-11T19:50:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.514+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/2637286858 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd39400add0 con 0x5651253dccc0 2024-03-11T19:50:00.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.514+0000 7fd37eff5640 1 -- 172.21.15.83:0/2637286858 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd39400ac40 con 0x5651253dccc0 2024-03-11T19:50:00.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.514+0000 7fd3a3fb5740 10 client.26902 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:00.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.514+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/2637286858 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5651253056a0 con 0x5651253dccc0 2024-03-11T19:50:00.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.514+0000 7fd37eff5640 1 -- 172.21.15.83:0/2637286858 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd39400dc60 con 0x5651253dccc0 2024-03-11T19:50:00.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.515+0000 7fd37eff5640 1 -- 172.21.15.83:0/2637286858 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd39401b6d0 con 0x5651253dccc0 2024-03-11T19:50:00.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.515+0000 7fd3a3fb5740 10 client.26902 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:00.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.515+0000 7fd3a3fb5740 10 client.26902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:00.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.515+0000 7fd3a3fb5740 20 client.26902 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.515+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565125449f60 0x56512544c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.515+0000 7fd3a3fb5740 4 client.26902 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.515+0000 7fd39affd640 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565125449f60 0x56512544c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.515+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/2637286858 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fd39400add0 con 0x565125449f60 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.516+0000 7fd39affd640 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565125449f60 0x56512544c380 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76cba9d68eff77d server_cookie=ba96ee17ab79e786 in_seq=0 out_seq=0 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.516+0000 7fd37eff5640 10 client.26902 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.516+0000 7fd37eff5640 1 -- 172.21.15.83:0/2637286858 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fd39400add0 con 0x565125449f60 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.516+0000 7fd37eff5640 10 client.26902 handle_command_reply: tid=0 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.516+0000 7fd3a3fb5740 1 client.26902 shutdown 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.517+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/2637286858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 msgr2=0x5651254027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.517+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 0x5651254027c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d9b0 tx=0x7fd394004a00 comp rx=0 tx=0).stop 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.517+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/2637286858 shutdown_connections 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.517+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x565125449f60 0x56512544c380 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.517+0000 7fd37eff5640 0 client.26902 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:00.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.517+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651253eba20 0x565125447f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.517+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651253e1e30 0x565125402d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.517+0000 7fd3a3fb5740 1 --2- 172.21.15.83:0/2637286858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651253dccc0 0x5651254027c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.517+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/2637286858 >> 172.21.15.83:0/2637286858 conn(0x5651253d77f0 msgr2=0x5651253f9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:00.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.517+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/2637286858 shutdown_connections 2024-03-11T19:50:00.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.517+0000 7fd3a3fb5740 1 -- 172.21.15.83:0/2637286858 wait complete. 2024-03-11T19:50:00.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.518+0000 7fd3a3fb5740 20 client.26902 trim_cache size 0 max 16384 2024-03-11T19:50:00.524 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:50:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:50:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:50:00.525 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.518+0000 7fd3a1302640 1 -- 172.21.15.83:0/895984362 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3840a1b70 msgr2=0x7fd3840a4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.518+0000 7fd3a1302640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3840a1b70 0x7fd3840a4030 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd3900084c0 tx=0x7fd39000a040 comp rx=0 tx=0).stop 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.518+0000 7fd3a1302640 1 -- 172.21.15.83:0/895984362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd39c1bc1a0 msgr2=0x7fd39c1be630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.518+0000 7fd3a1302640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd39c1bc1a0 0x7fd39c1be630 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd390030490 tx=0x7fd39000a670 comp rx=0 tx=0).stop 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.518+0000 7fd3a1302640 1 -- 172.21.15.83:0/895984362 shutdown_connections 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.518+0000 7fd3a1302640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd3840a1b70 0x7fd3840a4030 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.518+0000 7fd3a1302640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd39c1bc1a0 0x7fd39c1be630 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.518+0000 7fd3a1302640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd39c075ef0 0x7fd39c1bbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.518+0000 7fd3a1302640 1 --2- 172.21.15.83:0/895984362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd39c073010 0x7fd39c1b96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.518+0000 7fd3a1302640 1 -- 172.21.15.83:0/895984362 >> 172.21.15.83:0/895984362 conn(0x7fd39c06e200 msgr2=0x7fd39c070340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.519+0000 7fd3a1302640 1 -- 172.21.15.83:0/895984362 shutdown_connections 2024-03-11T19:50:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.519+0000 7fd3a1302640 1 -- 172.21.15.83:0/895984362 wait complete. 2024-03-11T19:50:00.539 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:50:00.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:50:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.734+0000 7fcd07fff640 1 -- 172.21.15.83:0/4146159562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c005220 con 0x7fcd101011c0 2024-03-11T19:50:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.734+0000 7fcd1789d640 1 -- 172.21.15.83:0/4146159562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd101011c0 msgr2=0x7fcd101015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.734+0000 7fcd1789d640 1 --2- 172.21.15.83:0/4146159562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd101011c0 0x7fcd101015e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009900 tx=0x7fcd0c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.735+0000 7fcd1789d640 1 -- 172.21.15.83:0/4146159562 shutdown_connections 2024-03-11T19:50:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.735+0000 7fcd1789d640 1 --2- 172.21.15.83:0/4146159562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd10102d70 0x7fcd10109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.735+0000 7fcd1789d640 1 --2- 172.21.15.83:0/4146159562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd101023b0 0x7fcd10102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.735+0000 7fcd1789d640 1 --2- 172.21.15.83:0/4146159562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd101011c0 0x7fcd101015e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.735+0000 7fcd1789d640 1 -- 172.21.15.83:0/4146159562 >> 172.21.15.83:0/4146159562 conn(0x7fcd100fc730 msgr2=0x7fcd100feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.735+0000 7fcd1789d640 1 -- 172.21.15.83:0/4146159562 shutdown_connections 2024-03-11T19:50:00.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.735+0000 7fcd1789d640 1 -- 172.21.15.83:0/4146159562 wait complete. 2024-03-11T19:50:00.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.735+0000 7fcd1789d640 1 Processor -- start 2024-03-11T19:50:00.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.735+0000 7fcd1789d640 1 -- start start 2024-03-11T19:50:00.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd1789d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd101011c0 0x7fcd10197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd1789d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd101023b0 0x7fcd10197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd1789d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10102d70 0x7fcd1019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd1789d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1019ed40 con 0x7fcd10102d70 2024-03-11T19:50:00.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd1789d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1019eeb0 con 0x7fcd101023b0 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd1789d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1019f070 con 0x7fcd101011c0 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd14e38640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd101023b0 0x7fcd10197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd14e38640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd101023b0 0x7fcd10197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51516/0 (socket says 172.21.15.83:51516) 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd14e38640 1 -- 172.21.15.83:0/3494691827 learned_addr learned my addr 172.21.15.83:0/3494691827 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd15e3a640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10102d70 0x7fcd1019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd15639640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd101011c0 0x7fcd10197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd14e38640 1 -- 172.21.15.83:0/3494691827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd101011c0 msgr2=0x7fcd10197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.736+0000 7fcd14e38640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd101011c0 0x7fcd10197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.737+0000 7fcd14e38640 1 -- 172.21.15.83:0/3494691827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10102d70 msgr2=0x7fcd1019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.737+0000 7fcd14e38640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10102d70 0x7fcd1019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.737+0000 7fcd14e38640 1 -- 172.21.15.83:0/3494691827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c009560 con 0x7fcd101023b0 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.737+0000 7fcd14e38640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd101023b0 0x7fcd10197790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009f20 tx=0x7fcd0000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.737+0000 7fcd067fc640 1 -- 172.21.15.83:0/3494691827 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000cd80 con 0x7fcd101023b0 2024-03-11T19:50:00.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.737+0000 7fcd067fc640 1 -- 172.21.15.83:0/3494691827 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd00013cb0 con 0x7fcd101023b0 2024-03-11T19:50:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.737+0000 7fcd067fc640 1 -- 172.21.15.83:0/3494691827 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0001bca0 con 0x7fcd101023b0 2024-03-11T19:50:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.737+0000 7fcd1789d640 1 -- 172.21.15.83:0/3494691827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd10073410 con 0x7fcd101023b0 2024-03-11T19:50:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.737+0000 7fcd1789d640 1 -- 172.21.15.83:0/3494691827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd10073870 con 0x7fcd101023b0 2024-03-11T19:50:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.740+0000 7fcd1789d640 1 -- 172.21.15.83:0/3494691827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd10108530 con 0x7fcd101023b0 2024-03-11T19:50:00.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.740+0000 7fcd067fc640 1 -- 172.21.15.83:0/3494691827 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fcd000137d0 con 0x7fcd101023b0 2024-03-11T19:50:00.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.741+0000 7fcd067fc640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fccfc0a1b30 0x7fccfc0a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.741+0000 7fcd067fc640 1 -- 172.21.15.83:0/3494691827 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fcd000d18f0 con 0x7fcd101023b0 2024-03-11T19:50:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.741+0000 7fcd15639640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fccfc0a1b30 0x7fccfc0a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:00.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.742+0000 7fcd15639640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fccfc0a1b30 0x7fccfc0a3ff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c02c720 tx=0x7fcd0c02c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:00.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:00.745+0000 7fcd067fc640 1 -- 172.21.15.83:0/3494691827 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd0009ab10 con 0x7fcd101023b0 2024-03-11T19:50:01.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.125+0000 7fcd1789d640 1 -- 172.21.15.83:0/3494691827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd1002ce40 con 0x7fcd101023b0 2024-03-11T19:50:01.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.126+0000 7fcd067fc640 1 -- 172.21.15.83:0/3494691827 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7fcd0009e1d0 con 0x7fcd101023b0 2024-03-11T19:50:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:50:01.131 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:50:01.131 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:50:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 -- 172.21.15.83:0/3494691827 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fccfc0a1b30 msgr2=0x7fccfc0a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fccfc0a1b30 0x7fccfc0a3ff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c02c720 tx=0x7fcd0c02c7f0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 -- 172.21.15.83:0/3494691827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd101023b0 msgr2=0x7fcd10197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd101023b0 0x7fcd10197790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009f20 tx=0x7fcd0000ee80 comp rx=0 tx=0).stop 2024-03-11T19:50:01.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 -- 172.21.15.83:0/3494691827 shutdown_connections 2024-03-11T19:50:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fccfc0a1b30 0x7fccfc0a3ff0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd10102d70 0x7fcd1019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd101023b0 0x7fcd10197790 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 --2- 172.21.15.83:0/3494691827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd101011c0 0x7fcd10197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 -- 172.21.15.83:0/3494691827 >> 172.21.15.83:0/3494691827 conn(0x7fcd100fc730 msgr2=0x7fcd10105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 -- 172.21.15.83:0/3494691827 shutdown_connections 2024-03-11T19:50:01.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.129+0000 7fcd1789d640 1 -- 172.21.15.83:0/3494691827 wait complete. 2024-03-11T19:50:01.147 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:50:01.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:50:01.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.295+0000 7f4802102640 1 -- 172.21.15.83:0/3809487168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47fc100080 msgr2=0x7f47fc1004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.295+0000 7f4802102640 1 --2- 172.21.15.83:0/3809487168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47fc100080 0x7f47fc1004a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009900 tx=0x7f47ec02bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:01.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.295+0000 7f4802102640 1 -- 172.21.15.83:0/3809487168 shutdown_connections 2024-03-11T19:50:01.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.295+0000 7f4802102640 1 --2- 172.21.15.83:0/3809487168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47fc101c30 0x7f47fc1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.295+0000 7f4802102640 1 --2- 172.21.15.83:0/3809487168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47fc101270 0x7f47fc1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.295+0000 7f4802102640 1 --2- 172.21.15.83:0/3809487168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47fc100080 0x7f47fc1004a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.295+0000 7f4802102640 1 -- 172.21.15.83:0/3809487168 >> 172.21.15.83:0/3809487168 conn(0x7f47fc0fb610 msgr2=0x7f47fc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.296+0000 7f4802102640 1 -- 172.21.15.83:0/3809487168 shutdown_connections 2024-03-11T19:50:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.296+0000 7f4802102640 1 -- 172.21.15.83:0/3809487168 wait complete. 2024-03-11T19:50:01.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.296+0000 7f4802102640 1 Processor -- start 2024-03-11T19:50:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f4802102640 1 -- start start 2024-03-11T19:50:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f4802102640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47fc100080 0x7f47fc104010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f4802102640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47fc101270 0x7f47fc102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f4802102640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47fc101c30 0x7f47fc102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f4802102640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc1032f0 con 0x7f47fc101c30 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f4802102640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc103460 con 0x7f47fc100080 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f4802102640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc1a1310 con 0x7f47fc101270 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f47fb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47fc100080 0x7f47fc104010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f47fb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47fc100080 0x7f47fc104010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51540/0 (socket says 172.21.15.83:51540) 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f47fb7fe640 1 -- 172.21.15.83:0/4192024210 learned_addr learned my addr 172.21.15.83:0/4192024210 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f47faffd640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47fc101270 0x7f47fc102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.297+0000 7f47fbfff640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47fc101c30 0x7f47fc102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.298+0000 7f47faffd640 1 -- 172.21.15.83:0/4192024210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47fc100080 msgr2=0x7f47fc104010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.298+0000 7f47faffd640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47fc100080 0x7f47fc104010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.298+0000 7f47faffd640 1 -- 172.21.15.83:0/4192024210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47fc101c30 msgr2=0x7f47fc102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.298+0000 7f47faffd640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47fc101c30 0x7f47fc102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.298+0000 7f47faffd640 1 -- 172.21.15.83:0/4192024210 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec009560 con 0x7f47fc101270 2024-03-11T19:50:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.299+0000 7f47faffd640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47fc101270 0x7f47fc102650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f47f000e9f0 tx=0x7f47f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.300+0000 7f47f8ff9640 1 -- 172.21.15.83:0/4192024210 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f000ccc0 con 0x7f47fc101270 2024-03-11T19:50:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.300+0000 7f47f8ff9640 1 -- 172.21.15.83:0/4192024210 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47f000ce20 con 0x7f47fc101270 2024-03-11T19:50:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.300+0000 7f4802102640 1 -- 172.21.15.83:0/4192024210 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47fc1a15c0 con 0x7f47fc101270 2024-03-11T19:50:01.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.300+0000 7f47f8ff9640 1 -- 172.21.15.83:0/4192024210 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f0004a00 con 0x7f47fc101270 2024-03-11T19:50:01.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.300+0000 7f4802102640 1 -- 172.21.15.83:0/4192024210 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc1a1ad0 con 0x7f47fc101270 2024-03-11T19:50:01.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.303+0000 7f47f8ff9640 1 -- 172.21.15.83:0/4192024210 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f47f00050b0 con 0x7f47fc101270 2024-03-11T19:50:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.303+0000 7f4804db2740 1 Processor -- start 2024-03-11T19:50:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.303+0000 7f4804db2740 1 -- start start 2024-03-11T19:50:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47f8ff9640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f47e40a1b80 0x7f47e40a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f4804db2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0e270 0x557897b0e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f4804db2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0ec60 0x557897b13760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f4804db2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b13dd0 0x557897b16260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f4804db2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897aca960 con 0x557897b0e270 2024-03-11T19:50:01.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f4804db2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897a7e310 con 0x557897b0ec60 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f4804db2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897a7b840 con 0x557897b13dd0 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47f8ff9640 1 -- 172.21.15.83:0/4192024210 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f47f0014070 con 0x7f47fc101270 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47fbfff640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f47e40a1b80 0x7f47e40a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47fb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0e270 0x557897b0e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b13dd0 0x557897b16260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47fb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0e270 0x557897b0e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60084/0 (socket says 172.21.15.83:60084) 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47fbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b13dd0 0x557897b16260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37428/0 (socket says 172.21.15.83:37428) 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47fb7fe640 1 -- 172.21.15.83:0/4037228749 learned_addr learned my addr 172.21.15.83:0/4037228749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47faffd640 1 --2- 172.21.15.83:0/4037228749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0ec60 0x557897b13760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47faffd640 1 -- 172.21.15.83:0/4037228749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b13dd0 msgr2=0x557897b16260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47faffd640 1 --2- 172.21.15.83:0/4037228749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b13dd0 0x557897b16260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47faffd640 1 -- 172.21.15.83:0/4037228749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0e270 msgr2=0x557897b0e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47faffd640 1 --2- 172.21.15.83:0/4037228749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0e270 0x557897b0e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47faffd640 1 -- 172.21.15.83:0/4037228749 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557897a39060 con 0x557897b0ec60 2024-03-11T19:50:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47fbfff640 1 --2- 172.21.15.83:0/4037228749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b13dd0 0x557897b16260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.304+0000 7f47fbfff640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f47e40a1b80 0x7f47e40a4040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f47f4005fd0 tx=0x7f47f4009df0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.305+0000 7f47faffd640 1 --2- 172.21.15.83:0/4037228749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0ec60 0x557897b13760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f47f00cadf0 tx=0x7f47f00054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c0ddf869566cb95 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.305+0000 7f47e1ffb640 1 -- 172.21.15.83:0/4037228749 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f009a210 con 0x557897b0ec60 2024-03-11T19:50:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.305+0000 7f47e1ffb640 1 -- 172.21.15.83:0/4037228749 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47f009e360 con 0x557897b0ec60 2024-03-11T19:50:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.305+0000 7f47e1ffb640 1 -- 172.21.15.83:0/4037228749 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f00a2780 con 0x557897b0ec60 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.306+0000 7f4804db2740 1 -- 172.21.15.83:0/4037228749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0ec60 msgr2=0x557897b13760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.306+0000 7f4804db2740 1 --2- 172.21.15.83:0/4037228749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0ec60 0x557897b13760 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f47f00cadf0 tx=0x7f47f00054e0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.306+0000 7f4804db2740 1 -- 172.21.15.83:0/4037228749 shutdown_connections 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.306+0000 7f4804db2740 1 --2- 172.21.15.83:0/4037228749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b13dd0 0x557897b16260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.306+0000 7f4804db2740 1 --2- 172.21.15.83:0/4037228749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0ec60 0x557897b13760 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.306+0000 7f4804db2740 1 --2- 172.21.15.83:0/4037228749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0e270 0x557897b0e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.306+0000 7f4804db2740 1 -- 172.21.15.83:0/4037228749 >> 172.21.15.83:0/4037228749 conn(0x557897b097e0 msgr2=0x557897b09be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.306+0000 7f4804db2740 1 -- 172.21.15.83:0/4037228749 shutdown_connections 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.306+0000 7f4804db2740 1 -- 172.21.15.83:0/4037228749 wait complete. 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.307+0000 7f4804db2740 1 -- start start 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.307+0000 7f4804db2740 1 Processor -- start 2024-03-11T19:50:01.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.308+0000 7f4804db2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 0x557897b182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.308+0000 7f4804db2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0ec60 0x557897b18820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.308+0000 7f4804db2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b13dd0 0x557897b1f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.308+0000 7f47fb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 0x557897b182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.308+0000 7f47fb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 0x557897b182e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37434/0 (socket says 172.21.15.83:37434) 2024-03-11T19:50:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.308+0000 7f47fb7fe640 1 -- 172.21.15.83:0/1327903718 learned_addr learned my addr 172.21.15.83:0/1327903718 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.308+0000 7f47faffd640 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0ec60 0x557897b18820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.308+0000 7f4804db2740 1 -- 172.21.15.83:0/1327903718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897a77020 con 0x557897b13dd0 2024-03-11T19:50:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.308+0000 7f4804db2740 1 -- 172.21.15.83:0/1327903718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55789774a430 con 0x557897b0ec60 2024-03-11T19:50:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.308+0000 7f4804db2740 1 -- 172.21.15.83:0/1327903718 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897706e00 con 0x557897b0e270 2024-03-11T19:50:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.308+0000 7f47fbfff640 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b13dd0 0x557897b1f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.309+0000 7f4804db2740 10 client.0 _ll_register_callbacks cb 0x557897a1ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:01.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.310+0000 7f47fbfff640 1 -- 172.21.15.83:0/1327903718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 msgr2=0x557897b182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.310+0000 7f47fbfff640 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 0x557897b182e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.310+0000 7f47fbfff640 1 -- 172.21.15.83:0/1327903718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0ec60 msgr2=0x557897b18820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.310+0000 7f47fbfff640 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0ec60 0x557897b18820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.310+0000 7f47fbfff640 1 -- 172.21.15.83:0/1327903718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc1a1ad0 con 0x557897b13dd0 2024-03-11T19:50:01.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.310+0000 7f47fbfff640 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b13dd0 0x557897b1f860 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f47f4010e50 tx=0x7f47f40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b912c7501d4738 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.310+0000 7f47daffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.310+0000 7f47daffd640 1 -- 172.21.15.83:0/1327903718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4029480 con 0x557897b13dd0 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f47daffd640 1 -- 172.21.15.83:0/1327903718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f47f4024070 con 0x557897b13dd0 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f4804db2740 1 -- 172.21.15.83:0/1327903718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578977767e0 con 0x557897b13dd0 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f47daffd640 1 -- 172.21.15.83:0/1327903718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47f402f030 con 0x557897b13dd0 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f4804db2740 10 client.17352 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f4804db2740 1 -- 172.21.15.83:0/1327903718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557897a756a0 con 0x557897b13dd0 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f47daffd640 1 -- 172.21.15.83:0/1327903718 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5578977767e0 con 0x557897b13dd0 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f47daffd640 1 -- 172.21.15.83:0/1327903718 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f47f4011500 con 0x557897b13dd0 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f4804db2740 10 client.17352 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f4804db2740 10 client.17352 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f4804db2740 20 client.17352 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f4804db2740 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x557897b24530 0x557897b26950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f4804db2740 4 client.17352 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:01.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.311+0000 7f4804db2740 1 -- 172.21.15.83:0/1327903718 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578977b3720 con 0x557897b24530 2024-03-11T19:50:01.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.312+0000 7f47fb7fe640 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x557897b24530 0x557897b26950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.312+0000 7f47fb7fe640 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x557897b24530 0x557897b26950 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=196181ed3bac72d7 server_cookie=c58921d945f92ed6 in_seq=0 out_seq=0 2024-03-11T19:50:01.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.312+0000 7f47daffd640 10 client.17352 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:50:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.313+0000 7f47daffd640 1 -- 172.21.15.83:0/1327903718 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578977b3720 con 0x557897b24530 2024-03-11T19:50:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.313+0000 7f47daffd640 10 client.17352 handle_command_reply: tid=0 2024-03-11T19:50:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.313+0000 7f4804db2740 1 client.17352 shutdown 2024-03-11T19:50:01.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.314+0000 7f4804db2740 1 -- 172.21.15.83:0/1327903718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b13dd0 msgr2=0x557897b1f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.314+0000 7f4804db2740 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b13dd0 0x557897b1f860 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f47f4010e50 tx=0x7f47f40096a0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.315+0000 7f4804db2740 1 -- 172.21.15.83:0/1327903718 shutdown_connections 2024-03-11T19:50:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.315+0000 7f4804db2740 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x557897b24530 0x557897b26950 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.315+0000 7f4804db2740 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b13dd0 0x557897b1f860 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.315+0000 7f47daffd640 0 client.17352 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:50:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.315+0000 7f4804db2740 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0ec60 0x557897b18820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.315+0000 7f4804db2740 1 --2- 172.21.15.83:0/1327903718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 0x557897b182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.315+0000 7f4804db2740 1 -- 172.21.15.83:0/1327903718 >> 172.21.15.83:0/1327903718 conn(0x557897b097e0 msgr2=0x557897b08b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.315+0000 7f4804db2740 1 -- 172.21.15.83:0/1327903718 shutdown_connections 2024-03-11T19:50:01.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.315+0000 7f4804db2740 1 -- 172.21.15.83:0/1327903718 wait complete. 2024-03-11T19:50:01.320 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:01 smithi083 ceph-mon[31864]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 MiB/s wr, 633 op/s 2024-03-11T19:50:01.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.316+0000 7f4804db2740 20 client.17352 trim_cache size 0 max 16384 2024-03-11T19:50:01.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.335+0000 7f4804db2740 1 Processor -- start 2024-03-11T19:50:01.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.335+0000 7f4804db2740 1 -- start start 2024-03-11T19:50:01.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.335+0000 7f4804db2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 0x557897b16fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.335+0000 7f4804db2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0ec60 0x557897b1f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.336+0000 7f47fb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 0x557897b16fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.336+0000 7f47fb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 0x557897b16fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37436/0 (socket says 172.21.15.83:37436) 2024-03-11T19:50:01.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.336+0000 7f47fb7fe640 1 -- 172.21.15.83:0/2469049162 learned_addr learned my addr 172.21.15.83:0/2469049162 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.336+0000 7f4804db2740 1 --2- 172.21.15.83:0/2469049162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b13dd0 0x557897b09bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.336+0000 7f4804db2740 1 -- 172.21.15.83:0/2469049162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578979fc130 con 0x557897b0ec60 2024-03-11T19:50:01.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.336+0000 7f4804db2740 1 -- 172.21.15.83:0/2469049162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897afba40 con 0x557897b13dd0 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.336+0000 7f4804db2740 1 -- 172.21.15.83:0/2469049162 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897b194d0 con 0x557897b0e270 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.336+0000 7f47fbfff640 1 --2- 172.21.15.83:0/2469049162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b13dd0 0x557897b09bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.337+0000 7f47faffd640 1 --2- 172.21.15.83:0/2469049162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0ec60 0x557897b1f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.337+0000 7f47fbfff640 1 -- 172.21.15.83:0/2469049162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 msgr2=0x557897b16fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.337+0000 7f47fbfff640 1 --2- 172.21.15.83:0/2469049162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 0x557897b16fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.337+0000 7f47fbfff640 1 -- 172.21.15.83:0/2469049162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0ec60 msgr2=0x557897b1f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.337+0000 7f47fbfff640 1 --2- 172.21.15.83:0/2469049162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0ec60 0x557897b1f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.337+0000 7f47fbfff640 1 -- 172.21.15.83:0/2469049162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557897a11c20 con 0x557897b13dd0 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.337+0000 7f47faffd640 1 --2- 172.21.15.83:0/2469049162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0ec60 0x557897b1f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.337+0000 7f47fbfff640 1 --2- 172.21.15.83:0/2469049162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b13dd0 0x557897b09bc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009990 tx=0x7f47f4005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4589973902292f67 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.337+0000 7f47e17fa640 1 -- 172.21.15.83:0/2469049162 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f402f070 con 0x557897b13dd0 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.337+0000 7f47e17fa640 1 -- 172.21.15.83:0/2469049162 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47f4010800 con 0x557897b13dd0 2024-03-11T19:50:01.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.337+0000 7f47e17fa640 1 -- 172.21.15.83:0/2469049162 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f40296c0 con 0x557897b13dd0 2024-03-11T19:50:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.338+0000 7f4804db2740 1 -- 172.21.15.83:0/2469049162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b13dd0 msgr2=0x557897b09bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.338+0000 7f4804db2740 1 --2- 172.21.15.83:0/2469049162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b13dd0 0x557897b09bc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009990 tx=0x7f47f4005b70 comp rx=0 tx=0).stop 2024-03-11T19:50:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.338+0000 7f4804db2740 1 -- 172.21.15.83:0/2469049162 shutdown_connections 2024-03-11T19:50:01.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.338+0000 7f4804db2740 1 --2- 172.21.15.83:0/2469049162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b13dd0 0x557897b09bc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.338+0000 7f4804db2740 1 --2- 172.21.15.83:0/2469049162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b0ec60 0x557897b1f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.338+0000 7f4804db2740 1 --2- 172.21.15.83:0/2469049162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0e270 0x557897b16fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.338+0000 7f4804db2740 1 -- 172.21.15.83:0/2469049162 >> 172.21.15.83:0/2469049162 conn(0x557897b097e0 msgr2=0x557897b04a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.338+0000 7f4804db2740 1 -- 172.21.15.83:0/2469049162 shutdown_connections 2024-03-11T19:50:01.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.338+0000 7f4804db2740 1 -- 172.21.15.83:0/2469049162 wait complete. 2024-03-11T19:50:01.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.339+0000 7f4804db2740 1 -- start start 2024-03-11T19:50:01.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.339+0000 7f4804db2740 1 Processor -- start 2024-03-11T19:50:01.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.339+0000 7f4804db2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0e270 0x557897b1f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.339+0000 7f4804db2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0ec60 0x557897b1dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.339+0000 7f4804db2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b13dd0 0x557897b1e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.339+0000 7f4804db2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897b1fc50 con 0x557897b13dd0 2024-03-11T19:50:01.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.339+0000 7f4804db2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897b1e590 con 0x557897b0e270 2024-03-11T19:50:01.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.339+0000 7f4804db2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557897b1e750 con 0x557897b0ec60 2024-03-11T19:50:01.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.339+0000 7f4804db2740 10 client.0 _ll_register_callbacks cb 0x7f47f0005360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47faffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0ec60 0x557897b1dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47faffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0ec60 0x557897b1dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37452/0 (socket says 172.21.15.83:37452) 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47faffd640 1 -- 172.21.15.83:0/3168117675 learned_addr learned my addr 172.21.15.83:0/3168117675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47fb7fe640 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0e270 0x557897b1f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47fbfff640 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b13dd0 0x557897b1e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47faffd640 1 -- 172.21.15.83:0/3168117675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0e270 msgr2=0x557897b1f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47faffd640 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0e270 0x557897b1f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47faffd640 1 -- 172.21.15.83:0/3168117675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b13dd0 msgr2=0x557897b1e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47faffd640 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b13dd0 0x557897b1e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47faffd640 1 -- 172.21.15.83:0/3168117675 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557897a756a0 con 0x557897b0ec60 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47fb7fe640 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0e270 0x557897b1f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47faffd640 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0ec60 0x557897b1dae0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f47f0096520 tx=0x7f47f0099db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ecd3f576ec0843a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.340+0000 7f47daffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:50:01.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f47daffd640 1 -- 172.21.15.83:0/3168117675 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f00b7070 con 0x557897b0ec60 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f4804db2740 1 -- 172.21.15.83:0/3168117675 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557897802890 con 0x557897b0ec60 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f47daffd640 1 -- 172.21.15.83:0/3168117675 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f47f00a29b0 con 0x557897b0ec60 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f47daffd640 1 -- 172.21.15.83:0/3168117675 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47f00a2730 con 0x557897b0ec60 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f4804db2740 10 client.26923 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f4804db2740 1 -- 172.21.15.83:0/3168117675 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557897a378c0 con 0x557897b0ec60 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f47daffd640 1 -- 172.21.15.83:0/3168117675 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47f007f080 con 0x557897b0ec60 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f47daffd640 1 -- 172.21.15.83:0/3168117675 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f47f00990d0 con 0x557897b0ec60 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f4804db2740 10 client.26923 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f4804db2740 10 client.26923 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f4804db2740 20 client.26923 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f4804db2740 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x557897b24530 0x557897b8edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f4804db2740 4 client.26923 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f4804db2740 1 -- 172.21.15.83:0/3168117675 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557897722700 con 0x557897b24530 2024-03-11T19:50:01.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.341+0000 7f47fb7fe640 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x557897b24530 0x557897b8edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f47daffd640 10 client.26923 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f47fb7fe640 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x557897b24530 0x557897b8edc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b7135d8dd4a06b7 server_cookie=5ef32b60617e5363 in_seq=0 out_seq=0 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f47daffd640 1 -- 172.21.15.83:0/3168117675 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557897722700 con 0x557897b24530 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f47daffd640 10 client.26923 handle_command_reply: tid=0 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f4804db2740 1 client.26923 shutdown 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f4804db2740 1 -- 172.21.15.83:0/3168117675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0ec60 msgr2=0x557897b1dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f4804db2740 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0ec60 0x557897b1dae0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f47f0096520 tx=0x7f47f0099db0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f4804db2740 1 -- 172.21.15.83:0/3168117675 shutdown_connections 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f4804db2740 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x557897b24530 0x557897b8edc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f4804db2740 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557897b13dd0 0x557897b1e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f47daffd640 0 client.26923 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f4804db2740 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557897b0ec60 0x557897b1dae0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f4804db2740 1 --2- 172.21.15.83:0/3168117675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557897b0e270 0x557897b1f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f4804db2740 1 -- 172.21.15.83:0/3168117675 >> 172.21.15.83:0/3168117675 conn(0x557897b097e0 msgr2=0x557897b09bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.342+0000 7f4804db2740 1 -- 172.21.15.83:0/3168117675 shutdown_connections 2024-03-11T19:50:01.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.343+0000 7f4804db2740 1 -- 172.21.15.83:0/3168117675 wait complete. 2024-03-11T19:50:01.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.343+0000 7f4804db2740 20 client.26923 trim_cache size 0 max 16384 2024-03-11T19:50:01.351 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 -- 172.21.15.83:0/4192024210 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f47e40a1b80 msgr2=0x7f47e40a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f47e40a1b80 0x7f47e40a4040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f47f4005fd0 tx=0x7f47f4009df0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 -- 172.21.15.83:0/4192024210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47fc101270 msgr2=0x7f47fc102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47fc101270 0x7f47fc102650 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f47f000e9f0 tx=0x7f47f000eeb0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 -- 172.21.15.83:0/4192024210 shutdown_connections 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f47e40a1b80 0x7f47e40a4040 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47fc101c30 0x7f47fc102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47fc101270 0x7f47fc102650 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 --2- 172.21.15.83:0/4192024210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47fc100080 0x7f47fc104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 -- 172.21.15.83:0/4192024210 >> 172.21.15.83:0/4192024210 conn(0x7f47fc0fb610 msgr2=0x7f47fc0fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 -- 172.21.15.83:0/4192024210 shutdown_connections 2024-03-11T19:50:01.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.345+0000 7f4802102640 1 -- 172.21.15.83:0/4192024210 wait complete. 2024-03-11T19:50:01.364 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:50:01.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:50:01.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:01 smithi137 ceph-mon[36220]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 MiB/s wr, 633 op/s 2024-03-11T19:50:01.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:01 smithi204 ceph-mon[36739]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 MiB/s wr, 633 op/s 2024-03-11T19:50:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.529+0000 7f203ffff640 1 -- 172.21.15.83:0/629645686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20380a6fa0 msgr2=0x7f20380ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.529+0000 7f203ffff640 1 --2- 172.21.15.83:0/629645686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20380a6fa0 0x7f20380ad830 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2040066e50 tx=0x7f204008ecb0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.529+0000 7f203ffff640 1 -- 172.21.15.83:0/629645686 shutdown_connections 2024-03-11T19:50:01.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.530+0000 7f203ffff640 1 --2- 172.21.15.83:0/629645686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20380a6fa0 0x7f20380ad830 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.530+0000 7f203ffff640 1 --2- 172.21.15.83:0/629645686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380a65e0 0x7f20380a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.530+0000 7f203ffff640 1 --2- 172.21.15.83:0/629645686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20380a53f0 0x7f20380a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.530+0000 7f203ffff640 1 -- 172.21.15.83:0/629645686 >> 172.21.15.83:0/629645686 conn(0x7f20380a0960 msgr2=0x7f20380a2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.532+0000 7f203ffff640 1 -- 172.21.15.83:0/629645686 shutdown_connections 2024-03-11T19:50:01.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.532+0000 7f203ffff640 1 -- 172.21.15.83:0/629645686 wait complete. 2024-03-11T19:50:01.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.533+0000 7f203ffff640 1 Processor -- start 2024-03-11T19:50:01.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.534+0000 7f203ffff640 1 -- start start 2024-03-11T19:50:01.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.534+0000 7f203ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20380a53f0 0x7f203813ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.534+0000 7f203ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20380a65e0 0x7f203813f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.535+0000 7f203ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380a6fa0 0x7f20381464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.535+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20380a65e0 0x7f203813f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.535+0000 7f203ffff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2038146a20 con 0x7f20380a53f0 2024-03-11T19:50:01.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.535+0000 7f203f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380a6fa0 0x7f20381464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.535+0000 7f203f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380a6fa0 0x7f20381464e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51584/0 (socket says 172.21.15.83:51584) 2024-03-11T19:50:01.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.535+0000 7f203f7fe640 1 -- 172.21.15.83:0/4091194273 learned_addr learned my addr 172.21.15.83:0/4091194273 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.535+0000 7f203ffff640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2038146b90 con 0x7f20380a6fa0 2024-03-11T19:50:01.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.535+0000 7f203ffff640 1 -- 172.21.15.83:0/4091194273 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2038146d50 con 0x7f20380a65e0 2024-03-11T19:50:01.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.535+0000 7f203effd640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20380a53f0 0x7f203813ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.536+0000 7f203effd640 1 -- 172.21.15.83:0/4091194273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20380a65e0 msgr2=0x7f203813f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.536+0000 7f203effd640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20380a65e0 0x7f203813f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.536+0000 7f203effd640 1 -- 172.21.15.83:0/4091194273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380a6fa0 msgr2=0x7f20381464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.536+0000 7f203effd640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380a6fa0 0x7f20381464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.536+0000 7f203effd640 1 -- 172.21.15.83:0/4091194273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204004f730 con 0x7f20380a53f0 2024-03-11T19:50:01.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.536+0000 7f203f7fe640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380a6fa0 0x7f20381464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:01.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.536+0000 7f203effd640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20380a53f0 0x7f203813ef30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2030005fd0 tx=0x7f203000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.537+0000 7f2027fff640 1 -- 172.21.15.83:0/4091194273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203000cc40 con 0x7f20380a53f0 2024-03-11T19:50:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.537+0000 7f2027fff640 1 -- 172.21.15.83:0/4091194273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f203000ee70 con 0x7f20380a53f0 2024-03-11T19:50:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.537+0000 7f2027fff640 1 -- 172.21.15.83:0/4091194273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030010430 con 0x7f20380a53f0 2024-03-11T19:50:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.537+0000 7f203ffff640 1 -- 172.21.15.83:0/4091194273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2038146fd0 con 0x7f20380a53f0 2024-03-11T19:50:01.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.538+0000 7f203ffff640 1 -- 172.21.15.83:0/4091194273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2038147400 con 0x7f20380a53f0 2024-03-11T19:50:01.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.541+0000 7f2027fff640 1 -- 172.21.15.83:0/4091194273 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f20300035f0 con 0x7f20380a53f0 2024-03-11T19:50:01.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.541+0000 7f2027fff640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20280a1c30 0x7f20280a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.542+0000 7f2048050740 1 Processor -- start 2024-03-11T19:50:01.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.542+0000 7f2027fff640 1 -- 172.21.15.83:0/4091194273 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2030014070 con 0x7f20380a53f0 2024-03-11T19:50:01.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.542+0000 7f203e7fc640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20280a1c30 0x7f20280a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.542+0000 7f2048050740 1 -- start start 2024-03-11T19:50:01.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f2048050740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x5563265838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f2048050740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x556326588970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f2048050740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x55632658b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f2048050740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55632653f960 con 0x556326583480 2024-03-11T19:50:01.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f2048050740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563264f3310 con 0x556326583e70 2024-03-11T19:50:01.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f2048050740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563264f0840 con 0x556326588fe0 2024-03-11T19:50:01.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x5563265838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x556326588970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x55632658b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x556326588970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51586/0 (socket says 172.21.15.83:51586) 2024-03-11T19:50:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203e7fc640 1 -- 172.21.15.83:0/2714426161 learned_addr learned my addr 172.21.15.83:0/2714426161 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203effd640 1 -- 172.21.15.83:0/2714426161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 msgr2=0x55632658b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203effd640 1 --2- 172.21.15.83:0/2714426161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x55632658b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203effd640 1 -- 172.21.15.83:0/2714426161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 msgr2=0x556326588970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203effd640 1 --2- 172.21.15.83:0/2714426161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x556326588970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203effd640 1 -- 172.21.15.83:0/2714426161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563264ae060 con 0x556326583480 2024-03-11T19:50:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203e7fc640 1 --2- 172.21.15.83:0/2714426161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x556326588970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.543+0000 7f203effd640 1 --2- 172.21.15.83:0/2714426161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x5563265838a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f20300968e0 tx=0x7f2030099de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1be4ca50257a33 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.544+0000 7f203e7fc640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20280a1c30 0x7f20280a40f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f203400a890 tx=0x7f203400a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.544+0000 7f20257fa640 1 -- 172.21.15.83:0/2714426161 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20300ba070 con 0x556326583480 2024-03-11T19:50:01.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.544+0000 7f20257fa640 1 -- 172.21.15.83:0/2714426161 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f203009e5d0 con 0x556326583480 2024-03-11T19:50:01.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.544+0000 7f20257fa640 1 -- 172.21.15.83:0/2714426161 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203009e1b0 con 0x556326583480 2024-03-11T19:50:01.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.545+0000 7f2048050740 1 -- 172.21.15.83:0/2714426161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 msgr2=0x5563265838a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.545+0000 7f2048050740 1 --2- 172.21.15.83:0/2714426161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x5563265838a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f20300968e0 tx=0x7f2030099de0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.545+0000 7f2048050740 1 -- 172.21.15.83:0/2714426161 shutdown_connections 2024-03-11T19:50:01.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.545+0000 7f2048050740 1 --2- 172.21.15.83:0/2714426161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x55632658b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.545+0000 7f2048050740 1 --2- 172.21.15.83:0/2714426161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x556326588970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.545+0000 7f2048050740 1 --2- 172.21.15.83:0/2714426161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x5563265838a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.545+0000 7f2048050740 1 -- 172.21.15.83:0/2714426161 >> 172.21.15.83:0/2714426161 conn(0x55632657e9f0 msgr2=0x55632657edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.546+0000 7f2048050740 1 -- 172.21.15.83:0/2714426161 shutdown_connections 2024-03-11T19:50:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.546+0000 7f2048050740 1 -- 172.21.15.83:0/2714426161 wait complete. 2024-03-11T19:50:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f2048050740 1 -- start start 2024-03-11T19:50:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f2048050740 1 Processor -- start 2024-03-11T19:50:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f2048050740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x556326595d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f2048050740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x556326596270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f2048050740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x55632659d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f2048050740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563264ec020 con 0x556326583480 2024-03-11T19:50:01.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f2048050740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563261bf430 con 0x556326583e70 2024-03-11T19:50:01.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f2048050740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55632617be00 con 0x556326588fe0 2024-03-11T19:50:01.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f2048050740 10 client.0 _ll_register_callbacks cb 0x556326493cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:01.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x556326596270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f203f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x55632659d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f203effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x556326595d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f203f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x55632659d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37492/0 (socket says 172.21.15.83:37492) 2024-03-11T19:50:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f203e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x556326596270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51598/0 (socket says 172.21.15.83:51598) 2024-03-11T19:50:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f203f7fe640 1 -- 172.21.15.83:0/2351173462 learned_addr learned my addr 172.21.15.83:0/2351173462 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f203effd640 1 -- 172.21.15.83:0/2351173462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 msgr2=0x55632659d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f203effd640 1 --2- 172.21.15.83:0/2351173462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x55632659d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f203effd640 1 -- 172.21.15.83:0/2351173462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 msgr2=0x556326596270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.547+0000 7f203effd640 1 --2- 172.21.15.83:0/2351173462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x556326596270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.548+0000 7f203effd640 1 -- 172.21.15.83:0/2351173462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2038147400 con 0x556326583480 2024-03-11T19:50:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.548+0000 7f203effd640 1 --2- 172.21.15.83:0/2351173462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x556326595d30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f203009a180 tx=0x7f203009a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5343fea9de8d17 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.548+0000 7f2022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:50:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.548+0000 7f2022ff5640 1 -- 172.21.15.83:0/2351173462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030098780 con 0x556326583480 2024-03-11T19:50:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.548+0000 7f2022ff5640 1 -- 172.21.15.83:0/2351173462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f20300a2c20 con 0x556326583480 2024-03-11T19:50:01.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.548+0000 7f2048050740 1 -- 172.21.15.83:0/2351173462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563261eb7e0 con 0x556326583480 2024-03-11T19:50:01.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.548+0000 7f2022ff5640 1 -- 172.21.15.83:0/2351173462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20300cb220 con 0x556326583480 2024-03-11T19:50:01.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.548+0000 7f2048050740 10 client.17364 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:01.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.548+0000 7f2048050740 1 -- 172.21.15.83:0/2351173462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5563264ea6a0 con 0x556326583480 2024-03-11T19:50:01.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.549+0000 7f2022ff5640 1 -- 172.21.15.83:0/2351173462 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5563261eb7e0 con 0x556326583480 2024-03-11T19:50:01.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.549+0000 7f2022ff5640 1 -- 172.21.15.83:0/2351173462 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f20300b6070 con 0x556326583480 2024-03-11T19:50:01.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.549+0000 7f2048050740 10 client.17364 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:01.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.549+0000 7f2048050740 10 client.17364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:50:01.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.549+0000 7f2048050740 20 client.17364 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:01.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.549+0000 7f2048050740 1 --2- 172.21.15.83:0/2351173462 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5563265a1b70 0x5563265a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.549+0000 7f2048050740 4 client.17364 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:01.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.549+0000 7f2048050740 1 -- 172.21.15.83:0/2351173462 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556326228720 con 0x5563265a1b70 2024-03-11T19:50:01.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.550+0000 7f203e7fc640 1 --2- 172.21.15.83:0/2351173462 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5563265a1b70 0x5563265a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.550+0000 7f203e7fc640 1 --2- 172.21.15.83:0/2351173462 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5563265a1b70 0x5563265a3f60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9089bd6643be92ef server_cookie=e971726a085b123b in_seq=0 out_seq=0 2024-03-11T19:50:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.550+0000 7f2022ff5640 10 client.17364 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:50:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2022ff5640 1 -- 172.21.15.83:0/2351173462 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556326228720 con 0x5563265a1b70 2024-03-11T19:50:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2022ff5640 10 client.17364 handle_command_reply: tid=0 2024-03-11T19:50:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2048050740 1 client.17364 shutdown 2024-03-11T19:50:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2048050740 1 -- 172.21.15.83:0/2351173462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 msgr2=0x556326595d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2048050740 1 --2- 172.21.15.83:0/2351173462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x556326595d30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f203009a180 tx=0x7f203009a600 comp rx=0 tx=0).stop 2024-03-11T19:50:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2048050740 1 -- 172.21.15.83:0/2351173462 shutdown_connections 2024-03-11T19:50:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2048050740 1 --2- 172.21.15.83:0/2351173462 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5563265a1b70 0x5563265a3f60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2048050740 1 --2- 172.21.15.83:0/2351173462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x55632659d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2048050740 1 --2- 172.21.15.83:0/2351173462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x556326596270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2048050740 1 --2- 172.21.15.83:0/2351173462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x556326595d30 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2048050740 1 -- 172.21.15.83:0/2351173462 >> 172.21.15.83:0/2351173462 conn(0x55632657e9f0 msgr2=0x55632657dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2022ff5640 0 client.17364 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:50:01.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.551+0000 7f2048050740 1 -- 172.21.15.83:0/2351173462 shutdown_connections 2024-03-11T19:50:01.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.552+0000 7f2048050740 1 -- 172.21.15.83:0/2351173462 wait complete. 2024-03-11T19:50:01.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.552+0000 7f2048050740 20 client.17364 trim_cache size 0 max 16384 2024-03-11T19:50:01.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.569+0000 7f2048050740 1 Processor -- start 2024-03-11T19:50:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.569+0000 7f2048050740 1 -- start start 2024-03-11T19:50:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f2048050740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x556326595940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f2048050740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x55632659d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f2048050740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x556326595e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f2048050740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x556326471130 con 0x556326583480 2024-03-11T19:50:01.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f2048050740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55632658bb30 con 0x556326583e70 2024-03-11T19:50:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f2048050740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55632657a1d0 con 0x556326588fe0 2024-03-11T19:50:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f203effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x556326595940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f203effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x556326595940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60174/0 (socket says 172.21.15.83:60174) 2024-03-11T19:50:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f203effd640 1 -- 172.21.15.83:0/3821900425 learned_addr learned my addr 172.21.15.83:0/3821900425 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f203e7fc640 1 --2- 172.21.15.83:0/3821900425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x55632659d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f203f7fe640 1 --2- 172.21.15.83:0/3821900425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x556326595e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f203effd640 1 -- 172.21.15.83:0/3821900425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 msgr2=0x556326595e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f203effd640 1 --2- 172.21.15.83:0/3821900425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x556326595e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f203effd640 1 -- 172.21.15.83:0/3821900425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 msgr2=0x55632659d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f203effd640 1 --2- 172.21.15.83:0/3821900425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x55632659d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.570+0000 7f203effd640 1 -- 172.21.15.83:0/3821900425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556326486c20 con 0x556326583480 2024-03-11T19:50:01.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.571+0000 7f203effd640 1 --2- 172.21.15.83:0/3821900425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x556326595940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2030098730 tx=0x7f2030099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9db4c775188c169 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.572+0000 7f2024ff9640 1 -- 172.21.15.83:0/3821900425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030009400 con 0x556326583480 2024-03-11T19:50:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.572+0000 7f2024ff9640 1 -- 172.21.15.83:0/3821900425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f203009e3c0 con 0x556326583480 2024-03-11T19:50:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.572+0000 7f2024ff9640 1 -- 172.21.15.83:0/3821900425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030098830 con 0x556326583480 2024-03-11T19:50:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.573+0000 7f2048050740 1 -- 172.21.15.83:0/3821900425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 msgr2=0x556326595940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.573+0000 7f2048050740 1 --2- 172.21.15.83:0/3821900425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x556326595940 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2030098730 tx=0x7f2030099f10 comp rx=0 tx=0).stop 2024-03-11T19:50:01.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.573+0000 7f2048050740 1 -- 172.21.15.83:0/3821900425 shutdown_connections 2024-03-11T19:50:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.573+0000 7f2048050740 1 --2- 172.21.15.83:0/3821900425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326588fe0 0x556326595e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.573+0000 7f2048050740 1 --2- 172.21.15.83:0/3821900425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583e70 0x55632659d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.573+0000 7f2048050740 1 --2- 172.21.15.83:0/3821900425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326583480 0x556326595940 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.573+0000 7f2048050740 1 -- 172.21.15.83:0/3821900425 >> 172.21.15.83:0/3821900425 conn(0x55632657e9f0 msgr2=0x5563265851f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.573+0000 7f2048050740 1 -- 172.21.15.83:0/3821900425 shutdown_connections 2024-03-11T19:50:01.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.574+0000 7f2048050740 1 -- 172.21.15.83:0/3821900425 wait complete. 2024-03-11T19:50:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.574+0000 7f2048050740 1 -- start start 2024-03-11T19:50:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.574+0000 7f2048050740 1 Processor -- start 2024-03-11T19:50:01.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f2048050740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583480 0x55632659bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f2048050740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326583e70 0x55632659c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f2048050740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326588fe0 0x55632659c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f203effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583480 0x55632659bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f203effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583480 0x55632659bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51602/0 (socket says 172.21.15.83:51602) 2024-03-11T19:50:01.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f203effd640 1 -- 172.21.15.83:0/3688949305 learned_addr learned my addr 172.21.15.83:0/3688949305 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f2048050740 1 -- 172.21.15.83:0/3688949305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563265f80d0 con 0x556326588fe0 2024-03-11T19:50:01.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f203e7fc640 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326583e70 0x55632659c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f2048050740 1 -- 172.21.15.83:0/3688949305 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563265f8240 con 0x556326583480 2024-03-11T19:50:01.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f2048050740 1 -- 172.21.15.83:0/3688949305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563265f8400 con 0x556326583e70 2024-03-11T19:50:01.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f2048050740 10 client.0 _ll_register_callbacks cb 0x7f203400be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:01.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.575+0000 7f203f7fe640 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326588fe0 0x55632659c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.576+0000 7f203e7fc640 1 -- 172.21.15.83:0/3688949305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583480 msgr2=0x55632659bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.576+0000 7f203e7fc640 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583480 0x55632659bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.576+0000 7f203e7fc640 1 -- 172.21.15.83:0/3688949305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326588fe0 msgr2=0x55632659c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.576+0000 7f203e7fc640 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326588fe0 0x55632659c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.576+0000 7f203e7fc640 1 -- 172.21.15.83:0/3688949305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563264ea6a0 con 0x556326583e70 2024-03-11T19:50:01.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.576+0000 7f203effd640 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583480 0x55632659bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:01.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.576+0000 7f203e7fc640 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326583e70 0x55632659c190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5563265a4890 tx=0x7f2034004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0a1dbda6263b02 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.576+0000 7f2022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:50:01.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.576+0000 7f2022ff5640 1 -- 172.21.15.83:0/3688949305 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034018d10 con 0x556326583e70 2024-03-11T19:50:01.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.576+0000 7f2022ff5640 1 -- 172.21.15.83:0/3688949305 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2034031070 con 0x556326583e70 2024-03-11T19:50:01.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.576+0000 7f2048050740 1 -- 172.21.15.83:0/3688949305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f203009e5d0 con 0x556326583e70 2024-03-11T19:50:01.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f2022ff5640 1 -- 172.21.15.83:0/3688949305 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f203403b030 con 0x556326583e70 2024-03-11T19:50:01.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f2048050740 10 client.26935 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:01.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f2048050740 1 -- 172.21.15.83:0/3688949305 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5563264ac8c0 con 0x556326583e70 2024-03-11T19:50:01.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f2022ff5640 1 -- 172.21.15.83:0/3688949305 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f203009e5d0 con 0x556326583e70 2024-03-11T19:50:01.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f2022ff5640 1 -- 172.21.15.83:0/3688949305 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f203401bb60 con 0x556326583e70 2024-03-11T19:50:01.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f2048050740 10 client.26935 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:01.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f2048050740 10 client.26935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:50:01.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f2048050740 20 client.26935 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:01.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f2048050740 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5563265f9ec0 0x5563265fc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f2048050740 4 client.26935 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:50:01.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f2048050740 1 -- 172.21.15.83:0/3688949305 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556326277890 con 0x5563265f9ec0 2024-03-11T19:50:01.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.577+0000 7f203effd640 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5563265f9ec0 0x5563265fc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.578+0000 7f203effd640 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5563265f9ec0 0x5563265fc2e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da668ffd1d57c84b server_cookie=495b5b60f42c5ee2 in_seq=0 out_seq=0 2024-03-11T19:50:01.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.578+0000 7f2022ff5640 10 client.26935 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:50:01.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.578+0000 7f2022ff5640 1 -- 172.21.15.83:0/3688949305 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556326277890 con 0x5563265f9ec0 2024-03-11T19:50:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.578+0000 7f2022ff5640 10 client.26935 handle_command_reply: tid=0 2024-03-11T19:50:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.578+0000 7f2048050740 1 client.26935 shutdown 2024-03-11T19:50:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.579+0000 7f2048050740 1 -- 172.21.15.83:0/3688949305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326583e70 msgr2=0x55632659c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.579+0000 7f2048050740 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326583e70 0x55632659c190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5563265a4890 tx=0x7f2034004730 comp rx=0 tx=0).stop 2024-03-11T19:50:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.580+0000 7f2048050740 1 -- 172.21.15.83:0/3688949305 shutdown_connections 2024-03-11T19:50:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.580+0000 7f2048050740 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5563265f9ec0 0x5563265fc2e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.580+0000 7f2022ff5640 0 client.26935 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:50:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.580+0000 7f2048050740 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556326588fe0 0x55632659c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.580+0000 7f2048050740 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556326583e70 0x55632659c190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.580+0000 7f2048050740 1 --2- 172.21.15.83:0/3688949305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556326583480 0x55632659bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.580+0000 7f2048050740 1 -- 172.21.15.83:0/3688949305 >> 172.21.15.83:0/3688949305 conn(0x55632657e9f0 msgr2=0x556326595920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.580+0000 7f2048050740 1 -- 172.21.15.83:0/3688949305 shutdown_connections 2024-03-11T19:50:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.580+0000 7f2048050740 1 -- 172.21.15.83:0/3688949305 wait complete. 2024-03-11T19:50:01.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.580+0000 7f2048050740 20 client.26935 trim_cache size 0 max 16384 2024-03-11T19:50:01.597 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:50:01.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 -- 172.21.15.83:0/4091194273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20280a1c30 msgr2=0x7f20280a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20280a1c30 0x7f20280a40f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f203400a890 tx=0x7f203400a250 comp rx=0 tx=0).stop 2024-03-11T19:50:01.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 -- 172.21.15.83:0/4091194273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20380a53f0 msgr2=0x7f203813ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20380a53f0 0x7f203813ef30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2030005fd0 tx=0x7f203000ec20 comp rx=0 tx=0).stop 2024-03-11T19:50:01.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 -- 172.21.15.83:0/4091194273 shutdown_connections 2024-03-11T19:50:01.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f20280a1c30 0x7f20280a40f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380a6fa0 0x7f20381464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20380a65e0 0x7f203813f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 --2- 172.21.15.83:0/4091194273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20380a53f0 0x7f203813ef30 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 -- 172.21.15.83:0/4091194273 >> 172.21.15.83:0/4091194273 conn(0x7f20380a0960 msgr2=0x7f2038135cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 -- 172.21.15.83:0/4091194273 shutdown_connections 2024-03-11T19:50:01.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.581+0000 7f203ffff640 1 -- 172.21.15.83:0/4091194273 wait complete. 2024-03-11T19:50:01.601 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:50:01.601 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:50:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.761+0000 7fe787e4e640 1 -- 172.21.15.83:0/4191879569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78010fa00 msgr2=0x7fe780111df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.761+0000 7fe787e4e640 1 --2- 172.21.15.83:0/4191879569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78010fa00 0x7fe780111df0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe778009860 tx=0x7fe77802bb70 comp rx=0 tx=0).stop 2024-03-11T19:50:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.761+0000 7fe787e4e640 1 -- 172.21.15.83:0/4191879569 shutdown_connections 2024-03-11T19:50:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.761+0000 7fe787e4e640 1 --2- 172.21.15.83:0/4191879569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78010fa00 0x7fe780111df0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.761+0000 7fe787e4e640 1 --2- 172.21.15.83:0/4191879569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe780108ae0 0x7fe78010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.761+0000 7fe787e4e640 1 --2- 172.21.15.83:0/4191879569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe780108180 0x7fe7801085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.761+0000 7fe787e4e640 1 -- 172.21.15.83:0/4191879569 >> 172.21.15.83:0/4191879569 conn(0x7fe78006e380 msgr2=0x7fe7800707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.761+0000 7fe787e4e640 1 -- 172.21.15.83:0/4191879569 shutdown_connections 2024-03-11T19:50:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.761+0000 7fe787e4e640 1 -- 172.21.15.83:0/4191879569 wait complete. 2024-03-11T19:50:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.761+0000 7fe787e4e640 1 Processor -- start 2024-03-11T19:50:01.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.761+0000 7fe787e4e640 1 -- start start 2024-03-11T19:50:01.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.762+0000 7fe787e4e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe780108180 0x7fe7801a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.762+0000 7fe787e4e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe780108ae0 0x7fe7801a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.762+0000 7fe787e4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78010fa00 0x7fe7801a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.762+0000 7fe787e4e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7801a9730 con 0x7fe78010fa00 2024-03-11T19:50:01.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.762+0000 7fe787e4e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7801a98a0 con 0x7fe780108180 2024-03-11T19:50:01.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.762+0000 7fe787e4e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7801a9a60 con 0x7fe780108ae0 2024-03-11T19:50:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.762+0000 7fe785bea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe780108180 0x7fe7801a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.762+0000 7fe785bea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe780108180 0x7fe7801a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51612/0 (socket says 172.21.15.83:51612) 2024-03-11T19:50:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.762+0000 7fe785bea640 1 -- 172.21.15.83:0/531151879 learned_addr learned my addr 172.21.15.83:0/531151879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.762+0000 7fe7853e9640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe780108ae0 0x7fe7801a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.762+0000 7fe785bea640 1 -- 172.21.15.83:0/531151879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe780108ae0 msgr2=0x7fe7801a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.763+0000 7fe785bea640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe780108ae0 0x7fe7801a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.763+0000 7fe785bea640 1 -- 172.21.15.83:0/531151879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78010fa00 msgr2=0x7fe7801a91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.763+0000 7fe785bea640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78010fa00 0x7fe7801a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.763+0000 7fe785bea640 1 -- 172.21.15.83:0/531151879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778009490 con 0x7fe780108180 2024-03-11T19:50:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.763+0000 7fe785bea640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe780108180 0x7fe7801a1ca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe77400ebc0 tx=0x7fe77400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.763+0000 7fe77effd640 1 -- 172.21.15.83:0/531151879 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7740099b0 con 0x7fe780108180 2024-03-11T19:50:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.763+0000 7fe787e4e640 1 -- 172.21.15.83:0/531151879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7801a9d40 con 0x7fe780108180 2024-03-11T19:50:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.763+0000 7fe77effd640 1 -- 172.21.15.83:0/531151879 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe77400eec0 con 0x7fe780108180 2024-03-11T19:50:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.763+0000 7fe77effd640 1 -- 172.21.15.83:0/531151879 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe774018aa0 con 0x7fe780108180 2024-03-11T19:50:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.763+0000 7fe787e4e640 1 -- 172.21.15.83:0/531151879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7801aa1e0 con 0x7fe780108180 2024-03-11T19:50:01.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.765+0000 7fe77effd640 1 -- 172.21.15.83:0/531151879 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe774018c00 con 0x7fe780108180 2024-03-11T19:50:01.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe77effd640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe7700a1b80 0x7fe7700a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe78aa27740 1 Processor -- start 2024-03-11T19:50:01.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe78aa27740 1 -- start start 2024-03-11T19:50:01.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe77effd640 1 -- 172.21.15.83:0/531151879 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe774014070 con 0x7fe780108180 2024-03-11T19:50:01.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe7853e9640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe7700a1b80 0x7fe7700a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11a270 0x56137d11a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 0x56137d11f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d122260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe78aa27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137d0d6960 con 0x56137d11ac60 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe78aa27740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137d08a310 con 0x56137d11fdd0 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe78aa27740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137d087840 con 0x56137d11a270 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe7853e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 0x56137d11f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe785bea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11a270 0x56137d11a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe7863eb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d122260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe7863eb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d122260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51620/0 (socket says 172.21.15.83:51620) 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe785bea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11a270 0x56137d11a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37558/0 (socket says 172.21.15.83:37558) 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe7863eb640 1 -- 172.21.15.83:0/2943043442 learned_addr learned my addr 172.21.15.83:0/2943043442 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.767+0000 7fe7853e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 0x56137d11f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60200/0 (socket says 172.21.15.83:60200) 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.768+0000 7fe7863eb640 1 -- 172.21.15.83:0/2943043442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11a270 msgr2=0x56137d11a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.768+0000 7fe7863eb640 1 --2- 172.21.15.83:0/2943043442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11a270 0x56137d11a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.768+0000 7fe7863eb640 1 -- 172.21.15.83:0/2943043442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 msgr2=0x56137d11f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.768+0000 7fe7863eb640 1 --2- 172.21.15.83:0/2943043442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 0x56137d11f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.768+0000 7fe7853e9640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe7700a1b80 0x7fe7700a4040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe76c005fd0 tx=0x7fe76c015960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.768+0000 7fe7863eb640 1 -- 172.21.15.83:0/2943043442 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56137d045060 con 0x56137d11fdd0 2024-03-11T19:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.768+0000 7fe7863eb640 1 --2- 172.21.15.83:0/2943043442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d122260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe77802c100 tx=0x7fe7780323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=135d162f3b430dbf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.768+0000 7fe76bfff640 1 -- 172.21.15.83:0/2943043442 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778004240 con 0x56137d11fdd0 2024-03-11T19:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.768+0000 7fe76bfff640 1 -- 172.21.15.83:0/2943043442 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe778034920 con 0x56137d11fdd0 2024-03-11T19:50:01.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.768+0000 7fe76bfff640 1 -- 172.21.15.83:0/2943043442 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77802c510 con 0x56137d11fdd0 2024-03-11T19:50:01.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.769+0000 7fe78aa27740 1 -- 172.21.15.83:0/2943043442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 msgr2=0x56137d122260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.769+0000 7fe78aa27740 1 --2- 172.21.15.83:0/2943043442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d122260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe77802c100 tx=0x7fe7780323d0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.769+0000 7fe78aa27740 1 -- 172.21.15.83:0/2943043442 shutdown_connections 2024-03-11T19:50:01.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.769+0000 7fe78aa27740 1 --2- 172.21.15.83:0/2943043442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d122260 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.769+0000 7fe78aa27740 1 --2- 172.21.15.83:0/2943043442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 0x56137d11f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.769+0000 7fe78aa27740 1 --2- 172.21.15.83:0/2943043442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11a270 0x56137d11a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.769+0000 7fe78aa27740 1 -- 172.21.15.83:0/2943043442 >> 172.21.15.83:0/2943043442 conn(0x56137d1157e0 msgr2=0x56137d115be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.769+0000 7fe78aa27740 1 -- 172.21.15.83:0/2943043442 shutdown_connections 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.769+0000 7fe78aa27740 1 -- 172.21.15.83:0/2943043442 wait complete. 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.769+0000 7fe78aa27740 1 -- start start 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.770+0000 7fe78aa27740 1 Processor -- start 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.770+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11a270 0x56137d12cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.770+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11ac60 0x56137d12d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.770+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d134180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.770+0000 7fe78aa27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137d083020 con 0x56137d11a270 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.770+0000 7fe78aa27740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137cd56430 con 0x56137d11fdd0 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.770+0000 7fe78aa27740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137cd12e00 con 0x56137d11ac60 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.771+0000 7fe78aa27740 10 client.0 _ll_register_callbacks cb 0x56137d02acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:01.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.771+0000 7fe7863eb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.771+0000 7fe7863eb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d134180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51632/0 (socket says 172.21.15.83:51632) 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.771+0000 7fe7863eb640 1 -- 172.21.15.83:0/2441054187 learned_addr learned my addr 172.21.15.83:0/2441054187 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.771+0000 7fe7853e9640 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11ac60 0x56137d12d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.771+0000 7fe785bea640 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11a270 0x56137d12cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.771+0000 7fe7863eb640 1 -- 172.21.15.83:0/2441054187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11ac60 msgr2=0x56137d12d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.771+0000 7fe7863eb640 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11ac60 0x56137d12d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.771+0000 7fe7863eb640 1 -- 172.21.15.83:0/2441054187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11a270 msgr2=0x56137d12cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.771+0000 7fe7863eb640 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11a270 0x56137d12cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.771+0000 7fe7863eb640 1 -- 172.21.15.83:0/2441054187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe780112680 con 0x56137d11fdd0 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.772+0000 7fe7863eb640 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d134180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe778034c40 tx=0x7fe778034890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefb9bc8d576aa60 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.772+0000 7fe7697fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.772+0000 7fe7697fa640 1 -- 172.21.15.83:0/2441054187 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778004240 con 0x56137d11fdd0 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.772+0000 7fe7697fa640 1 -- 172.21.15.83:0/2441054187 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe7780043a0 con 0x56137d11fdd0 2024-03-11T19:50:01.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.772+0000 7fe78aa27740 1 -- 172.21.15.83:0/2441054187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56137cdbf720 con 0x56137d11fdd0 2024-03-11T19:50:01.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.772+0000 7fe7697fa640 1 -- 172.21.15.83:0/2441054187 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe77802c510 con 0x56137d11fdd0 2024-03-11T19:50:01.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.772+0000 7fe78aa27740 10 client.26849 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:01.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.772+0000 7fe78aa27740 1 -- 172.21.15.83:0/2441054187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56137d0816a0 con 0x56137d11fdd0 2024-03-11T19:50:01.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.772+0000 7fe7697fa640 1 -- 172.21.15.83:0/2441054187 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56137cdbf720 con 0x56137d11fdd0 2024-03-11T19:50:01.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.772+0000 7fe7697fa640 1 -- 172.21.15.83:0/2441054187 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe778058070 con 0x56137d11fdd0 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.773+0000 7fe78aa27740 10 client.26849 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.773+0000 7fe78aa27740 10 client.26849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.773+0000 7fe78aa27740 20 client.26849 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.773+0000 7fe78aa27740 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56137d128460 0x56137d12a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.773+0000 7fe78aa27740 4 client.26849 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.773+0000 7fe78aa27740 1 -- 172.21.15.83:0/2441054187 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56137ce0e890 con 0x56137d128460 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.773+0000 7fe785bea640 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56137d128460 0x56137d12a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.773+0000 7fe785bea640 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56137d128460 0x56137d12a880 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d851685f2ff4787c server_cookie=bebf9ce330ae951d in_seq=0 out_seq=0 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.773+0000 7fe7697fa640 10 client.26849 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe7697fa640 1 -- 172.21.15.83:0/2441054187 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56137ce0e890 con 0x56137d128460 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe7697fa640 10 client.26849 handle_command_reply: tid=0 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe78aa27740 1 client.26849 shutdown 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe78aa27740 1 -- 172.21.15.83:0/2441054187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 msgr2=0x56137d134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe78aa27740 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d134180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe778034c40 tx=0x7fe778034890 comp rx=0 tx=0).stop 2024-03-11T19:50:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe78aa27740 1 -- 172.21.15.83:0/2441054187 shutdown_connections 2024-03-11T19:50:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe78aa27740 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56137d128460 0x56137d12a880 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe78aa27740 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11fdd0 0x56137d134180 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe78aa27740 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11ac60 0x56137d12d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe78aa27740 1 --2- 172.21.15.83:0/2441054187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11a270 0x56137d12cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe78aa27740 1 -- 172.21.15.83:0/2441054187 >> 172.21.15.83:0/2441054187 conn(0x56137d1157e0 msgr2=0x56137d114b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.774+0000 7fe7697fa640 0 client.26849 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.775+0000 7fe78aa27740 1 -- 172.21.15.83:0/2441054187 shutdown_connections 2024-03-11T19:50:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.775+0000 7fe78aa27740 1 -- 172.21.15.83:0/2441054187 wait complete. 2024-03-11T19:50:01.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.775+0000 7fe78aa27740 20 client.26849 trim_cache size 0 max 16384 2024-03-11T19:50:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.792+0000 7fe78aa27740 1 Processor -- start 2024-03-11T19:50:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.792+0000 7fe78aa27740 1 -- start start 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.792+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11a270 0x56137d11c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.792+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 0x56137d134140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.792+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11fdd0 0x56137d11c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.792+0000 7fe78aa27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137d008130 con 0x56137d11ac60 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.792+0000 7fe78aa27740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137ce0e890 con 0x56137d11a270 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.792+0000 7fe78aa27740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137d12ddf0 con 0x56137d11fdd0 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.792+0000 7fe7853e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 0x56137d134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.792+0000 7fe785bea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11a270 0x56137d11c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.793+0000 7fe785bea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11a270 0x56137d11c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51644/0 (socket says 172.21.15.83:51644) 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.793+0000 7fe7853e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 0x56137d134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60206/0 (socket says 172.21.15.83:60206) 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.793+0000 7fe785bea640 1 -- 172.21.15.83:0/4048426264 learned_addr learned my addr 172.21.15.83:0/4048426264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.793+0000 7fe7863eb640 1 --2- 172.21.15.83:0/4048426264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11fdd0 0x56137d11c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.793+0000 7fe785bea640 1 -- 172.21.15.83:0/4048426264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11fdd0 msgr2=0x56137d11c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.793+0000 7fe785bea640 1 --2- 172.21.15.83:0/4048426264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11fdd0 0x56137d11c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.793+0000 7fe785bea640 1 -- 172.21.15.83:0/4048426264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 msgr2=0x56137d134140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.793+0000 7fe785bea640 1 --2- 172.21.15.83:0/4048426264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 0x56137d134140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.793+0000 7fe785bea640 1 -- 172.21.15.83:0/4048426264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56137d01dc20 con 0x56137d11a270 2024-03-11T19:50:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.793+0000 7fe7863eb640 1 --2- 172.21.15.83:0/4048426264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11fdd0 0x56137d11c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:01.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.794+0000 7fe785bea640 1 --2- 172.21.15.83:0/4048426264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11a270 0x56137d11c120 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe774000f60 tx=0x7fe7740cb4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b53160de24da6138 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.794+0000 7fe76b7fe640 1 -- 172.21.15.83:0/4048426264 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77409ebd0 con 0x56137d11a270 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.794+0000 7fe76b7fe640 1 -- 172.21.15.83:0/4048426264 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7740a3220 con 0x56137d11a270 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.794+0000 7fe76b7fe640 1 -- 172.21.15.83:0/4048426264 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77409a190 con 0x56137d11a270 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.795+0000 7fe78aa27740 1 -- 172.21.15.83:0/4048426264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11a270 msgr2=0x56137d11c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.795+0000 7fe78aa27740 1 --2- 172.21.15.83:0/4048426264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11a270 0x56137d11c120 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe774000f60 tx=0x7fe7740cb4e0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.795+0000 7fe78aa27740 1 -- 172.21.15.83:0/4048426264 shutdown_connections 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.795+0000 7fe78aa27740 1 --2- 172.21.15.83:0/4048426264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11fdd0 0x56137d11c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.795+0000 7fe78aa27740 1 --2- 172.21.15.83:0/4048426264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11ac60 0x56137d134140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.795+0000 7fe78aa27740 1 --2- 172.21.15.83:0/4048426264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11a270 0x56137d11c120 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.795+0000 7fe78aa27740 1 -- 172.21.15.83:0/4048426264 >> 172.21.15.83:0/4048426264 conn(0x56137d1157e0 msgr2=0x56137d115bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.795+0000 7fe78aa27740 1 -- 172.21.15.83:0/4048426264 shutdown_connections 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.795+0000 7fe78aa27740 1 -- 172.21.15.83:0/4048426264 wait complete. 2024-03-11T19:50:01.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.796+0000 7fe78aa27740 1 -- start start 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.796+0000 7fe78aa27740 1 Processor -- start 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.796+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11a270 0x56137d130d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.796+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11ac60 0x56137d1312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.796+0000 7fe78aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11fdd0 0x56137d196e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.797+0000 7fe7853e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11ac60 0x56137d1312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.797+0000 7fe7853e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11ac60 0x56137d1312a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51652/0 (socket says 172.21.15.83:51652) 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.797+0000 7fe7853e9640 1 -- 172.21.15.83:0/1079904013 learned_addr learned my addr 172.21.15.83:0/1079904013 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.797+0000 7fe78aa27740 1 -- 172.21.15.83:0/1079904013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137d1342c0 con 0x56137d11fdd0 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.797+0000 7fe78aa27740 1 -- 172.21.15.83:0/1079904013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137d134430 con 0x56137d11ac60 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.797+0000 7fe78aa27740 1 -- 172.21.15.83:0/1079904013 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56137d197390 con 0x56137d11a270 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.797+0000 7fe78aa27740 10 client.0 _ll_register_callbacks cb 0x7fe778037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.797+0000 7fe785bea640 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11a270 0x56137d130d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe7863eb640 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11fdd0 0x56137d196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe7853e9640 1 -- 172.21.15.83:0/1079904013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11a270 msgr2=0x56137d130d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe7853e9640 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11a270 0x56137d130d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe7853e9640 1 -- 172.21.15.83:0/1079904013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11fdd0 msgr2=0x56137d196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe7853e9640 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11fdd0 0x56137d196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe7853e9640 1 -- 172.21.15.83:0/1079904013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56137d0816a0 con 0x56137d11ac60 2024-03-11T19:50:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe7853e9640 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11ac60 0x56137d1312a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe76c015fd0 tx=0x7fe76c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca803f39a9701353 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe7697fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:50:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe7697fa640 1 -- 172.21.15.83:0/1079904013 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c026040 con 0x56137d11ac60 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe7697fa640 1 -- 172.21.15.83:0/1079904013 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe76c0037f0 con 0x56137d11ac60 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe78aa27740 1 -- 172.21.15.83:0/1079904013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56137cd827e0 con 0x56137d11ac60 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe7697fa640 1 -- 172.21.15.83:0/1079904013 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe76c0039b0 con 0x56137d11ac60 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe78aa27740 10 client.26855 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.798+0000 7fe78aa27740 1 -- 172.21.15.83:0/1079904013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fe7801aa1e0 con 0x56137d11ac60 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.799+0000 7fe7697fa640 1 -- 172.21.15.83:0/1079904013 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56137cd827e0 con 0x56137d11ac60 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.799+0000 7fe7697fa640 1 -- 172.21.15.83:0/1079904013 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe76c02bc50 con 0x56137d11ac60 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.799+0000 7fe78aa27740 10 client.26855 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.799+0000 7fe78aa27740 10 client.26855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.799+0000 7fe78aa27740 20 client.26855 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.799+0000 7fe78aa27740 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56137d198de0 0x56137d19b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.799+0000 7fe78aa27740 4 client.26855 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.799+0000 7fe78aa27740 1 -- 172.21.15.83:0/1079904013 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56137cd2e700 con 0x56137d198de0 2024-03-11T19:50:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.799+0000 7fe785bea640 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56137d198de0 0x56137d19b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.799+0000 7fe785bea640 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56137d198de0 0x56137d19b200 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d81c1634d7a2df5 server_cookie=2668c3c2b678175 in_seq=0 out_seq=0 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.799+0000 7fe7697fa640 10 client.26855 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe7697fa640 1 -- 172.21.15.83:0/1079904013 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56137cd2e700 con 0x56137d198de0 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe7697fa640 10 client.26855 handle_command_reply: tid=0 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe78aa27740 1 client.26855 shutdown 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe78aa27740 1 -- 172.21.15.83:0/1079904013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11ac60 msgr2=0x56137d1312a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe78aa27740 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11ac60 0x56137d1312a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe76c015fd0 tx=0x7fe76c01b000 comp rx=0 tx=0).stop 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe78aa27740 1 -- 172.21.15.83:0/1079904013 shutdown_connections 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe78aa27740 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56137d198de0 0x56137d19b200 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe78aa27740 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56137d11fdd0 0x56137d196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe7697fa640 0 client.26855 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe78aa27740 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56137d11ac60 0x56137d1312a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe78aa27740 1 --2- 172.21.15.83:0/1079904013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56137d11a270 0x56137d130d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.800+0000 7fe78aa27740 1 -- 172.21.15.83:0/1079904013 >> 172.21.15.83:0/1079904013 conn(0x56137d1157e0 msgr2=0x56137d11c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.801+0000 7fe78aa27740 1 -- 172.21.15.83:0/1079904013 shutdown_connections 2024-03-11T19:50:01.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.801+0000 7fe78aa27740 1 -- 172.21.15.83:0/1079904013 wait complete. 2024-03-11T19:50:01.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.801+0000 7fe78aa27740 20 client.26855 trim_cache size 0 max 16384 2024-03-11T19:50:01.809 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 -- 172.21.15.83:0/531151879 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe7700a1b80 msgr2=0x7fe7700a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe7700a1b80 0x7fe7700a4040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe76c005fd0 tx=0x7fe76c015960 comp rx=0 tx=0).stop 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 -- 172.21.15.83:0/531151879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe780108180 msgr2=0x7fe7801a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe780108180 0x7fe7801a1ca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe77400ebc0 tx=0x7fe77400c6a0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 -- 172.21.15.83:0/531151879 shutdown_connections 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe7700a1b80 0x7fe7700a4040 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe78010fa00 0x7fe7801a91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe780108ae0 0x7fe7801a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 --2- 172.21.15.83:0/531151879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe780108180 0x7fe7801a1ca0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 -- 172.21.15.83:0/531151879 >> 172.21.15.83:0/531151879 conn(0x7fe78006e380 msgr2=0x7fe78010b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 -- 172.21.15.83:0/531151879 shutdown_connections 2024-03-11T19:50:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:01.802+0000 7fe787e4e640 1 -- 172.21.15.83:0/531151879 wait complete. 2024-03-11T19:50:02.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:02 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3494691827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:02.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:02 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3494691827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 645.0 s, 2784.2 tps, lat 36.078 ms stddev 45.339 2024-03-11T19:50:02.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:02 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3494691827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:02.820 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:50:02.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:02.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:50:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.986+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3993416105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f94108ae0 msgr2=0x7f0f9410f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.986+0000 7f0f9a18c640 1 --2- 172.21.15.83:0/3993416105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f94108ae0 0x7f0f9410f4c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009900 tx=0x7f0f8802bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.986+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3993416105 shutdown_connections 2024-03-11T19:50:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.986+0000 7f0f9a18c640 1 --2- 172.21.15.83:0/3993416105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f9410fa00 0x7f0f94111df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.986+0000 7f0f9a18c640 1 --2- 172.21.15.83:0/3993416105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f94108ae0 0x7f0f9410f4c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.986+0000 7f0f9a18c640 1 --2- 172.21.15.83:0/3993416105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f94108180 0x7f0f941085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.986+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3993416105 >> 172.21.15.83:0/3993416105 conn(0x7f0f9406dfe0 msgr2=0x7f0f94070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.986+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3993416105 shutdown_connections 2024-03-11T19:50:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.986+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3993416105 wait complete. 2024-03-11T19:50:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.987+0000 7f0f9a18c640 1 Processor -- start 2024-03-11T19:50:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.987+0000 7f0f9a18c640 1 -- start start 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.987+0000 7f0f9a18c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f94108180 0x7f0f9419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f9a18c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f94108ae0 0x7f0f941a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f9a18c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f9410fa00 0x7f0f9419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f9a18c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f941b6450 con 0x7f0f9410fa00 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f9a18c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f941b65c0 con 0x7f0f94108180 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f94108180 0x7f0f9419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f94108180 0x7f0f9419d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:51670/0 (socket says 172.21.15.83:51670) 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f937fe640 1 -- 172.21.15.83:0/3553953055 learned_addr learned my addr 172.21.15.83:0/3553953055 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f93fff640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f9410fa00 0x7f0f9419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3553953055 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f941b6780 con 0x7f0f94108ae0 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f94108ae0 0x7f0f941a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f937fe640 1 -- 172.21.15.83:0/3553953055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f94108ae0 msgr2=0x7f0f941a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f937fe640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f94108ae0 0x7f0f941a2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f937fe640 1 -- 172.21.15.83:0/3553953055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f9410fa00 msgr2=0x7f0f9419df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f937fe640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f9410fa00 0x7f0f9419df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.988+0000 7f0f937fe640 1 -- 172.21.15.83:0/3553953055 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88009560 con 0x7f0f94108180 2024-03-11T19:50:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.989+0000 7f0f937fe640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f94108180 0x7f0f9419d920 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400e9f0 tx=0x7f0f8400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.989+0000 7f0f90ff9640 1 -- 172.21.15.83:0/3553953055 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84009720 con 0x7f0f94108180 2024-03-11T19:50:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.989+0000 7f0f90ff9640 1 -- 172.21.15.83:0/3553953055 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f8400cd80 con 0x7f0f94108180 2024-03-11T19:50:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.989+0000 7f0f90ff9640 1 -- 172.21.15.83:0/3553953055 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84010690 con 0x7f0f94108180 2024-03-11T19:50:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.989+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3553953055 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f941b6a60 con 0x7f0f94108180 2024-03-11T19:50:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.989+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3553953055 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f941b6f20 con 0x7f0f94108180 2024-03-11T19:50:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.992+0000 7f0f90ff9640 1 -- 172.21.15.83:0/3553953055 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0f840107f0 con 0x7f0f94108180 2024-03-11T19:50:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.993+0000 7f0f90ff9640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0f7c0a1b30 0x7f0f7c0a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.993+0000 7f0f90ff9640 1 -- 172.21.15.83:0/3553953055 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0f84014070 con 0x7f0f94108180 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.993+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0f7c0a1b30 0x7f0f7c0a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.993+0000 7f0f9c9fd740 1 Processor -- start 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f9c9fd740 1 -- start start 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b5690 0x56033f5b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b6050 0x56033f5bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f9c9fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f523f90 con 0x56033f5b6050 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f9c9fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f5214c0 con 0x56033f5b5690 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f9c9fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f51cca0 con 0x56033f5b44c0 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f93fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b6050 0x56033f5bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f93fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b6050 0x56033f5bc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60220/0 (socket says 172.21.15.83:60220) 2024-03-11T19:50:02.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f93fff640 1 -- 172.21.15.83:0/895375858 learned_addr learned my addr 172.21.15.83:0/895375858 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37628/0 (socket says 172.21.15.83:37628) 2024-03-11T19:50:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/895375858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b5690 0x56033f5b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0f7c0a1b30 0x7f0f7c0a3ff0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0f8802c870 tx=0x7f0f88005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f92ffd640 1 -- 172.21.15.83:0/895375858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 msgr2=0x56033f5b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/895375858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f92ffd640 1 -- 172.21.15.83:0/895375858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b6050 msgr2=0x56033f5bc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.994+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/895375858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b6050 0x56033f5bc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.995+0000 7f0f92ffd640 1 -- 172.21.15.83:0/895375858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56033f4df0b0 con 0x56033f5b5690 2024-03-11T19:50:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.995+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/895375858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b5690 0x56033f5b5b10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0f9419ee30 tx=0x7f0f8802e050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f08f4822db6edb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.995+0000 7f0f79ffb640 1 -- 172.21.15.83:0/895375858 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88004240 con 0x56033f5b5690 2024-03-11T19:50:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.995+0000 7f0f79ffb640 1 -- 172.21.15.83:0/895375858 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f880043a0 con 0x56033f5b5690 2024-03-11T19:50:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.995+0000 7f0f79ffb640 1 -- 172.21.15.83:0/895375858 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88003a10 con 0x56033f5b5690 2024-03-11T19:50:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.996+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/895375858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b5690 msgr2=0x56033f5b5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.996+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/895375858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b5690 0x56033f5b5b10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0f9419ee30 tx=0x7f0f8802e050 comp rx=0 tx=0).stop 2024-03-11T19:50:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.996+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/895375858 shutdown_connections 2024-03-11T19:50:03.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.996+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/895375858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b6050 0x56033f5bc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.996+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/895375858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b5690 0x56033f5b5b10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.996+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/895375858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.996+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/895375858 >> 172.21.15.83:0/895375858 conn(0x56033f5af6a0 msgr2=0x56033f5afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.996+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/895375858 shutdown_connections 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.996+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/895375858 wait complete. 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.997+0000 7f0f9c9fd740 1 -- start start 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.997+0000 7f0f9c9fd740 1 Processor -- start 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5be6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b5690 0x56033f5bec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b6050 0x56033f5c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5be6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5be6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37632/0 (socket says 172.21.15.83:37632) 2024-03-11T19:50:03.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/4110582198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f4c1f40 con 0x56033f5b5690 2024-03-11T19:50:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f937fe640 1 -- 172.21.15.83:0/4110582198 learned_addr learned my addr 172.21.15.83:0/4110582198 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/4110582198 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f1f04b0 con 0x56033f5b6050 2024-03-11T19:50:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/4110582198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f4a2670 con 0x56033f5b44c0 2024-03-11T19:50:03.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f9c9fd740 10 client.0 _ll_register_callbacks cb 0x56033f50ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f93fff640 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b6050 0x56033f5c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f937fe640 1 -- 172.21.15.83:0/4110582198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b6050 msgr2=0x56033f5c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f937fe640 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b6050 0x56033f5c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f937fe640 1 -- 172.21.15.83:0/4110582198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b5690 msgr2=0x56033f5bec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.998+0000 7f0f937fe640 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b5690 0x56033f5bec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.999+0000 7f0f937fe640 1 -- 172.21.15.83:0/4110582198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f941126b0 con 0x56033f5b44c0 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.999+0000 7f0f93fff640 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b6050 0x56033f5c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.999+0000 7f0f937fe640 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5be6d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0f8409b7a0 tx=0x7f0f8409c050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fe5e8e03ea24fe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.999+0000 7f0f72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.999+0000 7f0f72ffd640 1 -- 172.21.15.83:0/4110582198 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840b4200 con 0x56033f5b44c0 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.999+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/4110582198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56033f4f6f40 con 0x56033f5b44c0 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.999+0000 7f0f72ffd640 1 -- 172.21.15.83:0/4110582198 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0f840a3b20 con 0x56033f5b44c0 2024-03-11T19:50:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.999+0000 7f0f72ffd640 1 -- 172.21.15.83:0/4110582198 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f840b4360 con 0x56033f5b44c0 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.999+0000 7f0f9c9fd740 10 client.26962 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:02.999+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/4110582198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56033f51b320 con 0x56033f5b44c0 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.000+0000 7f0f72ffd640 1 -- 172.21.15.83:0/4110582198 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56033f4f6f40 con 0x56033f5b44c0 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.000+0000 7f0f72ffd640 1 -- 172.21.15.83:0/4110582198 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0f840a7070 con 0x56033f5b44c0 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.000+0000 7f0f9c9fd740 10 client.26962 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.000+0000 7f0f9c9fd740 10 client.26962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.000+0000 7f0f9c9fd740 20 client.26962 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.000+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56033f5ca950 0x56033f5ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.000+0000 7f0f9c9fd740 4 client.26962 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.000+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/4110582198 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56033f513310 con 0x56033f5ca950 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.001+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56033f5ca950 0x56033f5ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.001+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56033f5ca950 0x56033f5ccd90 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=317dd671331c85ba server_cookie=2035604f1fb6e82c in_seq=0 out_seq=0 2024-03-11T19:50:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.001+0000 7f0f72ffd640 10 client.26962 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.002+0000 7f0f72ffd640 1 -- 172.21.15.83:0/4110582198 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56033f513310 con 0x56033f5ca950 2024-03-11T19:50:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.002+0000 7f0f72ffd640 10 client.26962 handle_command_reply: tid=0 2024-03-11T19:50:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.002+0000 7f0f9c9fd740 1 client.26962 shutdown 2024-03-11T19:50:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.002+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/4110582198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 msgr2=0x56033f5be6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.002+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5be6d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0f8409b7a0 tx=0x7f0f8409c050 comp rx=0 tx=0).stop 2024-03-11T19:50:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.004+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/4110582198 shutdown_connections 2024-03-11T19:50:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.004+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56033f5ca950 0x56033f5ccd90 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.004+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b6050 0x56033f5c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.004+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b5690 0x56033f5bec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.004+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/4110582198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5be6d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.004+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/4110582198 >> 172.21.15.83:0/4110582198 conn(0x56033f5af6a0 msgr2=0x56033f5ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.004+0000 7f0f72ffd640 0 client.26962 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:03.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.006+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/4110582198 shutdown_connections 2024-03-11T19:50:03.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.006+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/4110582198 wait complete. 2024-03-11T19:50:03.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.006+0000 7f0f9c9fd740 20 client.26962 trim_cache size 0 max 16384 2024-03-11T19:50:03.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.030+0000 7f0f9c9fd740 1 Processor -- start 2024-03-11T19:50:03.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.030+0000 7f0f9c9fd740 1 -- start start 2024-03-11T19:50:03.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.030+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b44c0 0x56033f5be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.030+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b5690 0x56033f5c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.030+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b6050 0x56033f5be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.030+0000 7f0f9c9fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f5c8ec0 con 0x56033f5b5690 2024-03-11T19:50:03.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.030+0000 7f0f9c9fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f5c7100 con 0x56033f5b44c0 2024-03-11T19:50:03.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.030+0000 7f0f9c9fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f5ca440 con 0x56033f5b6050 2024-03-11T19:50:03.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.031+0000 7f0f93fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b6050 0x56033f5be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.031+0000 7f0f92ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b5690 0x56033f5c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.031+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b44c0 0x56033f5be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.031+0000 7f0f93fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b6050 0x56033f5be820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37640/0 (socket says 172.21.15.83:37640) 2024-03-11T19:50:03.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.031+0000 7f0f92ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b5690 0x56033f5c5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60236/0 (socket says 172.21.15.83:60236) 2024-03-11T19:50:03.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.031+0000 7f0f93fff640 1 -- 172.21.15.83:0/228785574 learned_addr learned my addr 172.21.15.83:0/228785574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:03.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.031+0000 7f0f937fe640 1 -- 172.21.15.83:0/228785574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b6050 msgr2=0x56033f5be820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.031+0000 7f0f937fe640 1 --2- 172.21.15.83:0/228785574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b6050 0x56033f5be820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.031+0000 7f0f937fe640 1 -- 172.21.15.83:0/228785574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b5690 msgr2=0x56033f5c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.031+0000 7f0f937fe640 1 --2- 172.21.15.83:0/228785574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b5690 0x56033f5c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.031+0000 7f0f937fe640 1 -- 172.21.15.83:0/228785574 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56033f4b7c00 con 0x56033f5b44c0 2024-03-11T19:50:03.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.032+0000 7f0f937fe640 1 --2- 172.21.15.83:0/228785574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b44c0 0x56033f5be2e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0f840981e0 tx=0x7f0f840974b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d7e863d85460f3f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.032+0000 7f0f797fa640 1 -- 172.21.15.83:0/228785574 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840ad070 con 0x56033f5b44c0 2024-03-11T19:50:03.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.032+0000 7f0f797fa640 1 -- 172.21.15.83:0/228785574 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0f8409b170 con 0x56033f5b44c0 2024-03-11T19:50:03.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.032+0000 7f0f797fa640 1 -- 172.21.15.83:0/228785574 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840cc540 con 0x56033f5b44c0 2024-03-11T19:50:03.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.033+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/228785574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b44c0 msgr2=0x56033f5be2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.033+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/228785574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b44c0 0x56033f5be2e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0f840981e0 tx=0x7f0f840974b0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.033+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/228785574 shutdown_connections 2024-03-11T19:50:03.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.033+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/228785574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b6050 0x56033f5be820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.033+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/228785574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b5690 0x56033f5c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.033+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/228785574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b44c0 0x56033f5be2e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.033+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/228785574 >> 172.21.15.83:0/228785574 conn(0x56033f5af6a0 msgr2=0x56033f5b9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.033+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/228785574 shutdown_connections 2024-03-11T19:50:03.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.033+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/228785574 wait complete. 2024-03-11T19:50:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.034+0000 7f0f9c9fd740 1 -- start start 2024-03-11T19:50:03.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.034+0000 7f0f9c9fd740 1 Processor -- start 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b5690 0x56033f5c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f9c9fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b6050 0x56033f5c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f9c9fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f5c2d70 con 0x56033f5b6050 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f9c9fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f5c2ee0 con 0x56033f5b5690 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f9c9fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033f620e60 con 0x56033f5b44c0 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f9c9fd740 10 client.0 _ll_register_callbacks cb 0x7f0f84005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37656/0 (socket says 172.21.15.83:37656) 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f937fe640 1 -- 172.21.15.83:0/1144558174 learned_addr learned my addr 172.21.15.83:0/1144558174 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f93fff640 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b6050 0x56033f5c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.035+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b5690 0x56033f5c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f937fe640 1 -- 172.21.15.83:0/1144558174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b5690 msgr2=0x56033f5c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f937fe640 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b5690 0x56033f5c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f937fe640 1 -- 172.21.15.83:0/1144558174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b6050 msgr2=0x56033f5c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f937fe640 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b6050 0x56033f5c2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f937fe640 1 -- 172.21.15.83:0/1144558174 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56033f51b320 con 0x56033f5b44c0 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f937fe640 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5c57e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400e9c0 tx=0x7f0f8409b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a25c7c7c4292f6a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f72ffd640 1 -- 172.21.15.83:0/1144558174 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8408a070 con 0x56033f5b44c0 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f72ffd640 1 -- 172.21.15.83:0/1144558174 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0f8409cb40 con 0x56033f5b44c0 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/1144558174 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56033f542900 con 0x56033f5b44c0 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f72ffd640 1 -- 172.21.15.83:0/1144558174 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f840ad070 con 0x56033f5b44c0 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f9c9fd740 10 client.26968 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/1144558174 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56033f4dd910 con 0x56033f5b44c0 2024-03-11T19:50:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.036+0000 7f0f72ffd640 1 -- 172.21.15.83:0/1144558174 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56033f542900 con 0x56033f5b44c0 2024-03-11T19:50:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.037+0000 7f0f72ffd640 1 -- 172.21.15.83:0/1144558174 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0f8408d050 con 0x56033f5b44c0 2024-03-11T19:50:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.037+0000 7f0f9c9fd740 10 client.26968 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.037+0000 7f0f9c9fd740 10 client.26968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.037+0000 7f0f9c9fd740 20 client.26968 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.037+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56033f6222b0 0x56033f6246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.037+0000 7f0f9c9fd740 4 client.26968 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:50:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.037+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/1144558174 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56033f21c860 con 0x56033f6222b0 2024-03-11T19:50:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.037+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56033f6222b0 0x56033f6246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.037+0000 7f0f72ffd640 10 client.26968 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.037+0000 7f0f92ffd640 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56033f6222b0 0x56033f6246a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6eca75e9a146ed server_cookie=db7aa3d3334c0742 in_seq=0 out_seq=0 2024-03-11T19:50:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.038+0000 7f0f72ffd640 1 -- 172.21.15.83:0/1144558174 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56033f21c860 con 0x56033f6222b0 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.038+0000 7f0f72ffd640 10 client.26968 handle_command_reply: tid=0 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 1 client.26968 shutdown 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/1144558174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 msgr2=0x56033f5c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5c57e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400e9c0 tx=0x7f0f8409b6f0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/1144558174 shutdown_connections 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56033f6222b0 0x56033f6246a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56033f5b6050 0x56033f5c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56033f5b5690 0x56033f5c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 1 --2- 172.21.15.83:0/1144558174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56033f5b44c0 0x56033f5c57e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f72ffd640 0 client.26968 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/1144558174 >> 172.21.15.83:0/1144558174 conn(0x56033f5af6a0 msgr2=0x56033f5be250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/1144558174 shutdown_connections 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 1 -- 172.21.15.83:0/1144558174 wait complete. 2024-03-11T19:50:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.039+0000 7f0f9c9fd740 20 client.26968 trim_cache size 0 max 16384 2024-03-11T19:50:03.047 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:50:03.047 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:50:03.047 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "aad704c2-f57b-4c19-8231-f2399b3577b1", 2024-03-11T19:50:03.047 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:50:03.047 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3553953055 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0f7c0a1b30 msgr2=0x7f0f7c0a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0f7c0a1b30 0x7f0f7c0a3ff0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0f8802c870 tx=0x7f0f88005fd0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3553953055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f94108180 msgr2=0x7f0f9419d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f94108180 0x7f0f9419d920 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400e9f0 tx=0x7f0f8400edf0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3553953055 shutdown_connections 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0f7c0a1b30 0x7f0f7c0a3ff0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f9410fa00 0x7f0f9419df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f94108ae0 0x7f0f941a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 --2- 172.21.15.83:0/3553953055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f94108180 0x7f0f9419d920 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3553953055 >> 172.21.15.83:0/3553953055 conn(0x7f0f9406dfe0 msgr2=0x7f0f940706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3553953055 shutdown_connections 2024-03-11T19:50:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.040+0000 7f0f9a18c640 1 -- 172.21.15.83:0/3553953055 wait complete. 2024-03-11T19:50:03.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:50:03.061 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:50:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.227+0000 7f90776b5640 1 -- 172.21.15.83:0/1923539724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9070109370 msgr2=0x7f907010fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.227+0000 7f90776b5640 1 --2- 172.21.15.83:0/1923539724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9070109370 0x7f907010fc00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9068009ca0 tx=0x7f906802c100 comp rx=0 tx=0).stop 2024-03-11T19:50:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.227+0000 7f90776b5640 1 -- 172.21.15.83:0/1923539724 shutdown_connections 2024-03-11T19:50:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.227+0000 7f90776b5640 1 --2- 172.21.15.83:0/1923539724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9070109370 0x7f907010fc00 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.227+0000 7f90776b5640 1 --2- 172.21.15.83:0/1923539724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90701089b0 0x7f9070108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.227+0000 7f90776b5640 1 --2- 172.21.15.83:0/1923539724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90701077c0 0x7f9070107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.227+0000 7f90776b5640 1 -- 172.21.15.83:0/1923539724 >> 172.21.15.83:0/1923539724 conn(0x7f9070076770 msgr2=0x7f9070078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.228+0000 7f90776b5640 1 -- 172.21.15.83:0/1923539724 shutdown_connections 2024-03-11T19:50:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.228+0000 7f90776b5640 1 -- 172.21.15.83:0/1923539724 wait complete. 2024-03-11T19:50:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.228+0000 7f90776b5640 1 Processor -- start 2024-03-11T19:50:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f90776b5640 1 -- start start 2024-03-11T19:50:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f90776b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90701077c0 0x7f907007e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f90776b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90701089b0 0x7f907007ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f90776b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9070109370 0x7f90701adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f90776b5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9070081d20 con 0x7f90701077c0 2024-03-11T19:50:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f90776b5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9070081e90 con 0x7f90701089b0 2024-03-11T19:50:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f90776b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9070082050 con 0x7f9070109370 2024-03-11T19:50:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f9075451640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90701077c0 0x7f907007e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f9075451640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90701077c0 0x7f907007e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60268/0 (socket says 172.21.15.83:60268) 2024-03-11T19:50:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f9075451640 1 -- 172.21.15.83:0/803874768 learned_addr learned my addr 172.21.15.83:0/803874768 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f9074c50640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90701089b0 0x7f907007ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f9075c52640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9070109370 0x7f90701adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f9075c52640 1 -- 172.21.15.83:0/803874768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90701089b0 msgr2=0x7f907007ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f9075c52640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90701089b0 0x7f907007ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f9075c52640 1 -- 172.21.15.83:0/803874768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90701077c0 msgr2=0x7f907007e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f9075c52640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90701077c0 0x7f907007e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.229+0000 7f9075c52640 1 -- 172.21.15.83:0/803874768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9068009900 con 0x7f9070109370 2024-03-11T19:50:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.230+0000 7f9075c52640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9070109370 0x7f90701adac0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9068037040 tx=0x7f9068034e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.230+0000 7f90667fc640 1 -- 172.21.15.83:0/803874768 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068034850 con 0x7f9070109370 2024-03-11T19:50:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.230+0000 7f90667fc640 1 -- 172.21.15.83:0/803874768 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9068003e50 con 0x7f9070109370 2024-03-11T19:50:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.231+0000 7f90667fc640 1 -- 172.21.15.83:0/803874768 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068040450 con 0x7f9070109370 2024-03-11T19:50:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.231+0000 7f90776b5640 1 -- 172.21.15.83:0/803874768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90701ae0f0 con 0x7f9070109370 2024-03-11T19:50:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.231+0000 7f90776b5640 1 -- 172.21.15.83:0/803874768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90701ae520 con 0x7f9070109370 2024-03-11T19:50:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.234+0000 7f90667fc640 1 -- 172.21.15.83:0/803874768 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f906800a040 con 0x7f9070109370 2024-03-11T19:50:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.234+0000 7f907a365740 1 Processor -- start 2024-03-11T19:50:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.234+0000 7f907a365740 1 -- start start 2024-03-11T19:50:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f907a365740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f32d0 0x5595562f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f907a365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595562f3cc0 0x5595562f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f907a365740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f8e30 0x5595562fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f907a365740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559556263280 con 0x5595562f32d0 2024-03-11T19:50:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f907a365740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595562607b0 con 0x5595562f8e30 2024-03-11T19:50:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f907a365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955625bf90 con 0x5595562f3cc0 2024-03-11T19:50:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f90667fc640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f905c0a1ab0 0x7f905c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f9074c50640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f32d0 0x5595562f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f9074c50640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f32d0 0x5595562f36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60270/0 (socket says 172.21.15.83:60270) 2024-03-11T19:50:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f9074c50640 1 -- 172.21.15.83:0/409646752 learned_addr learned my addr 172.21.15.83:0/409646752 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f9074c50640 1 --2- 172.21.15.83:0/409646752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f8e30 0x5595562fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f90667fc640 1 -- 172.21.15.83:0/803874768 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f90680f0190 con 0x7f9070109370 2024-03-11T19:50:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.235+0000 7f9075c52640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f905c0a1ab0 0x7f905c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.236+0000 7f9074c50640 1 -- 172.21.15.83:0/409646752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595562f3cc0 msgr2=0x5595562f87c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:50:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.236+0000 7f9074c50640 1 --2- 172.21.15.83:0/409646752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595562f3cc0 0x5595562f87c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.236+0000 7f9074c50640 1 -- 172.21.15.83:0/409646752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f8e30 msgr2=0x5595562fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.236+0000 7f9074c50640 1 --2- 172.21.15.83:0/409646752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f8e30 0x5595562fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.236+0000 7f9074c50640 1 -- 172.21.15.83:0/409646752 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55955621a570 con 0x5595562f32d0 2024-03-11T19:50:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.236+0000 7f9074c50640 1 --2- 172.21.15.83:0/409646752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f32d0 0x5595562f36f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f90600149a0 tx=0x7f906000c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be5f91c3ef3f602 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.236+0000 7f9075c52640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f905c0a1ab0 0x7f905c0a3f70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f90680049d0 tx=0x7f9068034f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.237+0000 7f905b7fe640 1 -- 172.21.15.83:0/409646752 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90600142d0 con 0x5595562f32d0 2024-03-11T19:50:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.237+0000 7f905b7fe640 1 -- 172.21.15.83:0/409646752 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f906001a020 con 0x5595562f32d0 2024-03-11T19:50:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.237+0000 7f905b7fe640 1 -- 172.21.15.83:0/409646752 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906000fed0 con 0x5595562f32d0 2024-03-11T19:50:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.238+0000 7f907a365740 1 -- 172.21.15.83:0/409646752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f32d0 msgr2=0x5595562f36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.238+0000 7f907a365740 1 --2- 172.21.15.83:0/409646752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f32d0 0x5595562f36f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f90600149a0 tx=0x7f906000c4d0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.238+0000 7f907a365740 1 -- 172.21.15.83:0/409646752 shutdown_connections 2024-03-11T19:50:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.238+0000 7f907a365740 1 --2- 172.21.15.83:0/409646752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f8e30 0x5595562fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.238+0000 7f907a365740 1 --2- 172.21.15.83:0/409646752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595562f3cc0 0x5595562f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.238+0000 7f907a365740 1 --2- 172.21.15.83:0/409646752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f32d0 0x5595562f36f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.238+0000 7f907a365740 1 -- 172.21.15.83:0/409646752 >> 172.21.15.83:0/409646752 conn(0x5595562ee7f0 msgr2=0x5595562eebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.238+0000 7f907a365740 1 -- 172.21.15.83:0/409646752 shutdown_connections 2024-03-11T19:50:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.238+0000 7f907a365740 1 -- 172.21.15.83:0/409646752 wait complete. 2024-03-11T19:50:03.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.239+0000 7f907a365740 1 -- start start 2024-03-11T19:50:03.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.239+0000 7f907a365740 1 Processor -- start 2024-03-11T19:50:03.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.239+0000 7f907a365740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x5595562fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.239+0000 7f907a365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595562f3cc0 0x5595562fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.239+0000 7f907a365740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f8e30 0x559556304820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.239+0000 7f907a365740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559555f2f430 con 0x5595562f8e30 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.239+0000 7f907a365740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559555eebe00 con 0x5595562f32d0 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.239+0000 7f907a365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595561e0fd0 con 0x5595562f3cc0 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.239+0000 7f907a365740 10 client.0 _ll_register_callbacks cb 0x559556236fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9074c50640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f8e30 0x559556304820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9074c50640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f8e30 0x559556304820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60286/0 (socket says 172.21.15.83:60286) 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9074c50640 1 -- 172.21.15.83:0/3215807245 learned_addr learned my addr 172.21.15.83:0/3215807245 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9074c50640 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x5595562fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9075451640 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595562f3cc0 0x5595562fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9074c50640 1 -- 172.21.15.83:0/3215807245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595562f3cc0 msgr2=0x5595562fd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9074c50640 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595562f3cc0 0x5595562fd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9074c50640 1 -- 172.21.15.83:0/3215807245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 msgr2=0x5595562fd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9074c50640 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x5595562fd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9074c50640 1 -- 172.21.15.83:0/3215807245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9060014680 con 0x5595562f8e30 2024-03-11T19:50:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9074c50640 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f8e30 0x559556304820 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9060005fd0 tx=0x7f9060016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6709d419d3b676 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:50:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9058ff9640 1 -- 172.21.15.83:0/3215807245 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060004020 con 0x5595562f8e30 2024-03-11T19:50:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f907a365740 1 -- 172.21.15.83:0/3215807245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559555f5b7e0 con 0x5595562f8e30 2024-03-11T19:50:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.240+0000 7f9058ff9640 1 -- 172.21.15.83:0/3215807245 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f906001b030 con 0x5595562f8e30 2024-03-11T19:50:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.241+0000 7f9058ff9640 1 -- 172.21.15.83:0/3215807245 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9060004180 con 0x5595562f8e30 2024-03-11T19:50:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.241+0000 7f907a365740 10 client.17412 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.241+0000 7f907a365740 1 -- 172.21.15.83:0/3215807245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55955625a610 con 0x5595562f8e30 2024-03-11T19:50:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.242+0000 7f9058ff9640 1 -- 172.21.15.83:0/3215807245 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9060034020 con 0x5595562f8e30 2024-03-11T19:50:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.242+0000 7f9058ff9640 1 -- 172.21.15.83:0/3215807245 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9060004680 con 0x5595562f8e30 2024-03-11T19:50:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.242+0000 7f907a365740 10 client.17412 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.242+0000 7f907a365740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.242+0000 7f907a365740 20 client.17412 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.242+0000 7f907a365740 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5595563093c0 0x55955630b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.242+0000 7f907a365740 4 client.17412 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.242+0000 7f907a365740 1 -- 172.21.15.83:0/3215807245 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559555f98720 con 0x5595563093c0 2024-03-11T19:50:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.243+0000 7f9075451640 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5595563093c0 0x55955630b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.243+0000 7f9058ff9640 10 client.17412 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.243+0000 7f9075451640 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5595563093c0 0x55955630b7e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f3b987d2643c579 server_cookie=26a4c51dfc7fff9c in_seq=0 out_seq=0 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.244+0000 7f9058ff9640 1 -- 172.21.15.83:0/3215807245 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559555f98720 con 0x5595563093c0 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.244+0000 7f9058ff9640 10 client.17412 handle_command_reply: tid=0 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.244+0000 7f907a365740 1 client.17412 shutdown 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.244+0000 7f907a365740 1 -- 172.21.15.83:0/3215807245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f8e30 msgr2=0x559556304820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.244+0000 7f907a365740 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f8e30 0x559556304820 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9060005fd0 tx=0x7f9060016040 comp rx=0 tx=0).stop 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.245+0000 7f907a365740 1 -- 172.21.15.83:0/3215807245 shutdown_connections 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.245+0000 7f907a365740 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5595563093c0 0x55955630b7e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.245+0000 7f907a365740 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f8e30 0x559556304820 secure :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9060005fd0 tx=0x7f9060016040 comp rx=0 tx=0).stop 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.245+0000 7f907a365740 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595562f3cc0 0x5595562fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.245+0000 7f907a365740 1 --2- 172.21.15.83:0/3215807245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x5595562fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.245+0000 7f907a365740 1 -- 172.21.15.83:0/3215807245 >> 172.21.15.83:0/3215807245 conn(0x5595562ee7f0 msgr2=0x5595562eda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.245+0000 7f9058ff9640 0 client.17412 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.245+0000 7f907a365740 1 -- 172.21.15.83:0/3215807245 shutdown_connections 2024-03-11T19:50:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.245+0000 7f907a365740 1 -- 172.21.15.83:0/3215807245 wait complete. 2024-03-11T19:50:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.245+0000 7f907a365740 20 client.17412 trim_cache size 0 max 16384 2024-03-11T19:50:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f907a365740 1 Processor -- start 2024-03-11T19:50:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f907a365740 1 -- start start 2024-03-11T19:50:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f907a365740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x5595562fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f907a365740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x5595562fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f907a365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 0x5595562f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f907a365740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595562fb980 con 0x5595562f3cc0 2024-03-11T19:50:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f907a365740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595562fe310 con 0x5595562f32d0 2024-03-11T19:50:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f907a365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559555f98750 con 0x559556304740 2024-03-11T19:50:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f9075451640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x5595562fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f9075451640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x5595562fd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60288/0 (socket says 172.21.15.83:60288) 2024-03-11T19:50:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f9075451640 1 -- 172.21.15.83:0/2547151627 learned_addr learned my addr 172.21.15.83:0/2547151627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f9074c50640 1 --2- 172.21.15.83:0/2547151627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 0x5595562f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f9074c50640 1 --2- 172.21.15.83:0/2547151627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x5595562fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f9075451640 1 -- 172.21.15.83:0/2547151627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 msgr2=0x5595562f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f9075451640 1 --2- 172.21.15.83:0/2547151627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 0x5595562f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f9075451640 1 -- 172.21.15.83:0/2547151627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 msgr2=0x5595562fd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f9075451640 1 --2- 172.21.15.83:0/2547151627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x5595562fd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f9075451640 1 -- 172.21.15.83:0/2547151627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595561f6ac0 con 0x5595562f3cc0 2024-03-11T19:50:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.270+0000 7f9075451640 1 --2- 172.21.15.83:0/2547151627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x5595562fd7e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x5595562f4730 tx=0x7f906c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800e5dc0de22ea0a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.271+0000 7f905affd640 1 -- 172.21.15.83:0/2547151627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c016ac0 con 0x5595562f3cc0 2024-03-11T19:50:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.271+0000 7f905affd640 1 -- 172.21.15.83:0/2547151627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f906c016c20 con 0x5595562f3cc0 2024-03-11T19:50:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.271+0000 7f905affd640 1 -- 172.21.15.83:0/2547151627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c0143b0 con 0x5595562f3cc0 2024-03-11T19:50:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.272+0000 7f907a365740 1 -- 172.21.15.83:0/2547151627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 msgr2=0x5595562fd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.272+0000 7f907a365740 1 --2- 172.21.15.83:0/2547151627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x5595562fd7e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x5595562f4730 tx=0x7f906c009190 comp rx=0 tx=0).stop 2024-03-11T19:50:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.272+0000 7f907a365740 1 -- 172.21.15.83:0/2547151627 shutdown_connections 2024-03-11T19:50:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.272+0000 7f907a365740 1 --2- 172.21.15.83:0/2547151627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 0x5595562f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.272+0000 7f907a365740 1 --2- 172.21.15.83:0/2547151627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x5595562fd7e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.272+0000 7f907a365740 1 --2- 172.21.15.83:0/2547151627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x5595562fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.272+0000 7f907a365740 1 -- 172.21.15.83:0/2547151627 >> 172.21.15.83:0/2547151627 conn(0x5595562ee7f0 msgr2=0x559556310140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.273+0000 7f907a365740 1 -- 172.21.15.83:0/2547151627 shutdown_connections 2024-03-11T19:50:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.273+0000 7f907a365740 1 -- 172.21.15.83:0/2547151627 wait complete. 2024-03-11T19:50:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.273+0000 7f907a365740 1 -- start start 2024-03-11T19:50:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.273+0000 7f907a365740 1 Processor -- start 2024-03-11T19:50:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.273+0000 7f907a365740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x559556302c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.273+0000 7f907a365740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x559556303190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.273+0000 7f907a365740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 0x5595563036d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.273+0000 7f907a365740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955635f9a0 con 0x5595562f3cc0 2024-03-11T19:50:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.273+0000 7f907a365740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955635fb10 con 0x5595562f32d0 2024-03-11T19:50:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.273+0000 7f907a365740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955635fcd0 con 0x559556304740 2024-03-11T19:50:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.273+0000 7f907a365740 10 client.0 _ll_register_callbacks cb 0x7f906000cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9075451640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x559556303190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9074c50640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 0x5595563036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9074c50640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 0x5595563036d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37696/0 (socket says 172.21.15.83:37696) 2024-03-11T19:50:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9075451640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x559556303190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60298/0 (socket says 172.21.15.83:60298) 2024-03-11T19:50:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9074c50640 1 -- 172.21.15.83:0/2422277331 learned_addr learned my addr 172.21.15.83:0/2422277331 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9074c50640 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x559556302c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9074c50640 1 -- 172.21.15.83:0/2422277331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 msgr2=0x559556302c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9074c50640 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x559556302c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9074c50640 1 -- 172.21.15.83:0/2422277331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 msgr2=0x559556303190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9074c50640 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x559556303190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9074c50640 1 -- 172.21.15.83:0/2422277331 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c009760 con 0x559556304740 2024-03-11T19:50:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9075451640 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x559556303190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9074c50640 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 0x5595563036d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5595562fded0 tx=0x7f906000ce10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b519ec069d62be2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.274+0000 7f9058ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:50:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.275+0000 7f9058ff9640 1 -- 172.21.15.83:0/2422277331 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906000c9d0 con 0x559556304740 2024-03-11T19:50:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.275+0000 7f9058ff9640 1 -- 172.21.15.83:0/2422277331 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f906001b030 con 0x559556304740 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.275+0000 7f907a365740 1 -- 172.21.15.83:0/2422277331 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f906001a020 con 0x559556304740 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.275+0000 7f9058ff9640 1 -- 172.21.15.83:0/2422277331 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f906000fbc0 con 0x559556304740 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.275+0000 7f907a365740 10 client.26983 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.275+0000 7f907a365740 1 -- 172.21.15.83:0/2422277331 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55955621c6a0 con 0x559556304740 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.275+0000 7f9058ff9640 1 -- 172.21.15.83:0/2422277331 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f906001a020 con 0x559556304740 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.276+0000 7f9058ff9640 1 -- 172.21.15.83:0/2422277331 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9060037070 con 0x559556304740 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.276+0000 7f907a365740 10 client.26983 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.276+0000 7f907a365740 10 client.26983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.276+0000 7f907a365740 20 client.26983 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.276+0000 7f907a365740 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559556361430 0x559556363850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.276+0000 7f907a365740 4 client.26983 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.276+0000 7f907a365740 1 -- 172.21.15.83:0/2422277331 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559555fe7890 con 0x559556361430 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.276+0000 7f9075451640 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559556361430 0x559556363850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.276+0000 7f9075451640 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559556361430 0x559556363850 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b4292dd7277dbf server_cookie=dc77f2d81bf46eb8 in_seq=0 out_seq=0 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.276+0000 7f9058ff9640 10 client.26983 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f9058ff9640 1 -- 172.21.15.83:0/2422277331 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559555fe7890 con 0x559556361430 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f9058ff9640 10 client.26983 handle_command_reply: tid=0 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f907a365740 1 client.26983 shutdown 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f907a365740 1 -- 172.21.15.83:0/2422277331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 msgr2=0x5595563036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f907a365740 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 0x5595563036d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5595562fded0 tx=0x7f906000ce10 comp rx=0 tx=0).stop 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f907a365740 1 -- 172.21.15.83:0/2422277331 shutdown_connections 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f907a365740 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559556361430 0x559556363850 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f9058ff9640 0 client.26983 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f907a365740 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559556304740 0x5595563036d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f907a365740 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595562f3cc0 0x559556303190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f907a365740 1 --2- 172.21.15.83:0/2422277331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595562f32d0 0x559556302c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.277+0000 7f907a365740 1 -- 172.21.15.83:0/2422277331 >> 172.21.15.83:0/2422277331 conn(0x5595562ee7f0 msgr2=0x5595562f4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.278+0000 7f907a365740 1 -- 172.21.15.83:0/2422277331 shutdown_connections 2024-03-11T19:50:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.278+0000 7f907a365740 1 -- 172.21.15.83:0/2422277331 wait complete. 2024-03-11T19:50:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.278+0000 7f907a365740 20 client.26983 trim_cache size 0 max 16384 2024-03-11T19:50:03.286 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:50:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:50:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:50:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "aad704c2-f57b-4c19-8231-f2399b3577b1": { 2024-03-11T19:50:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:50:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "aad704c2-f57b-4c19-8231-f2399b3577b1", 2024-03-11T19:50:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:50:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:50:03.286 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:50:03.287 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:50:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 -- 172.21.15.83:0/803874768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f905c0a1ab0 msgr2=0x7f905c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f905c0a1ab0 0x7f905c0a3f70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f90680049d0 tx=0x7f9068034f10 comp rx=0 tx=0).stop 2024-03-11T19:50:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 -- 172.21.15.83:0/803874768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9070109370 msgr2=0x7f90701adac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9070109370 0x7f90701adac0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9068037040 tx=0x7f9068034e20 comp rx=0 tx=0).stop 2024-03-11T19:50:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 -- 172.21.15.83:0/803874768 shutdown_connections 2024-03-11T19:50:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f905c0a1ab0 0x7f905c0a3f70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9070109370 0x7f90701adac0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90701089b0 0x7f907007ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 --2- 172.21.15.83:0/803874768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90701077c0 0x7f907007e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 -- 172.21.15.83:0/803874768 >> 172.21.15.83:0/803874768 conn(0x7f9070076770 msgr2=0x7f907010bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 -- 172.21.15.83:0/803874768 shutdown_connections 2024-03-11T19:50:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:03.279+0000 7f90776b5640 1 -- 172.21.15.83:0/803874768 wait complete. 2024-03-11T19:50:03.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:03 smithi083 ceph-mon[31864]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.5 MiB/s wr, 632 op/s 2024-03-11T19:50:03.300 INFO:tasks.cephfs.filesystem:scrub status for tag:aad704c2-f57b-4c19-8231-f2399b3577b1 - {'path': '/', 'tag': 'aad704c2-f57b-4c19-8231-f2399b3577b1', 'options': 'recursive,force'} 2024-03-11T19:50:03.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:03 smithi137 ceph-mon[36220]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.5 MiB/s wr, 632 op/s 2024-03-11T19:50:03.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:03 smithi204 ceph-mon[36739]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.5 MiB/s wr, 632 op/s 2024-03-11T19:50:04.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:04 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:50:04.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:04 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:04.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:04 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:50:04.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:04 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:50:04.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:04 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:04.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:04 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:50:04.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:04 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:04.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:04 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:04.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:04 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:50:04.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:04 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:04.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:04 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:50:04.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:04 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:50:04.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:04 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:04.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:04 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:50:04.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:04 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:04.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:04 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:04.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:04 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:50:04.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:04 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:04.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:04 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:50:04.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:04 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:50:04.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:04 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:04.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:04 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:50:04.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:04 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:04.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:04 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:05.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:05 smithi137 ceph-mon[36220]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-03-11T19:50:05.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:05 smithi204 ceph-mon[36739]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-03-11T19:50:05.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:05 smithi083 ceph-mon[31864]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-03-11T19:50:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 650.0 s, 2318.0 tps, lat 43.320 ms stddev 54.839 2024-03-11T19:50:08.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:07 smithi137 ceph-mon[36220]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.2 MiB/s wr, 650 op/s 2024-03-11T19:50:08.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:07 smithi204 ceph-mon[36739]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.2 MiB/s wr, 650 op/s 2024-03-11T19:50:08.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:07 smithi083 ceph-mon[31864]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.2 MiB/s wr, 650 op/s 2024-03-11T19:50:10.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:09 smithi137 ceph-mon[36220]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.8 MiB/s wr, 808 op/s 2024-03-11T19:50:10.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:09 smithi204 ceph-mon[36739]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.8 MiB/s wr, 808 op/s 2024-03-11T19:50:10.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:09 smithi083 ceph-mon[31864]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.8 MiB/s wr, 808 op/s 2024-03-11T19:50:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:10 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:50:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:10 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:50:11.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:10 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:50:12.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:11 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:50:12.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:11 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:50:12.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:11 smithi137 ceph-mon[36220]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.9 MiB/s wr, 713 op/s 2024-03-11T19:50:12.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:11 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:50:12.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:11 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:50:12.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:11 smithi204 ceph-mon[36739]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.9 MiB/s wr, 713 op/s 2024-03-11T19:50:12.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:11 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:50:12.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:11 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:50:12.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:11 smithi083 ceph-mon[31864]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.9 MiB/s wr, 713 op/s 2024-03-11T19:50:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 655.0 s, 2256.6 tps, lat 44.171 ms stddev 53.496 2024-03-11T19:50:14.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:13 smithi137 ceph-mon[36220]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-03-11T19:50:14.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:13 smithi204 ceph-mon[36739]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-03-11T19:50:14.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:13 smithi083 ceph-mon[31864]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.8 MiB/s wr, 634 op/s 2024-03-11T19:50:16.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:15 smithi137 ceph-mon[36220]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 MiB/s wr, 765 op/s 2024-03-11T19:50:16.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:15 smithi204 ceph-mon[36739]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 MiB/s wr, 765 op/s 2024-03-11T19:50:16.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:15 smithi083 ceph-mon[31864]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 MiB/s wr, 765 op/s 2024-03-11T19:50:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 660.0 s, 2171.4 tps, lat 46.140 ms stddev 51.849 2024-03-11T19:50:18.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:17 smithi137 ceph-mon[36220]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.9 MiB/s wr, 661 op/s 2024-03-11T19:50:18.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:17 smithi204 ceph-mon[36739]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.9 MiB/s wr, 661 op/s 2024-03-11T19:50:18.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:17 smithi083 ceph-mon[31864]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.9 MiB/s wr, 661 op/s 2024-03-11T19:50:20.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:19 smithi137 ceph-mon[36220]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 761 op/s 2024-03-11T19:50:20.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:19 smithi204 ceph-mon[36739]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 761 op/s 2024-03-11T19:50:20.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:19 smithi083 ceph-mon[31864]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 761 op/s 2024-03-11T19:50:22.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:21 smithi137 ceph-mon[36220]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.3 MiB/s wr, 615 op/s 2024-03-11T19:50:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:21 smithi204 ceph-mon[36739]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.3 MiB/s wr, 615 op/s 2024-03-11T19:50:22.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:21 smithi083 ceph-mon[31864]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.3 MiB/s wr, 615 op/s 2024-03-11T19:50:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 665.0 s, 2272.0 tps, lat 43.796 ms stddev 52.092 2024-03-11T19:50:24.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:23 smithi137 ceph-mon[36220]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.6 MiB/s wr, 543 op/s 2024-03-11T19:50:24.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:23 smithi204 ceph-mon[36739]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.6 MiB/s wr, 543 op/s 2024-03-11T19:50:24.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:23 smithi083 ceph-mon[31864]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.6 MiB/s wr, 543 op/s 2024-03-11T19:50:26.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:25 smithi137 ceph-mon[36220]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 714 op/s 2024-03-11T19:50:26.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:25 smithi204 ceph-mon[36739]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 714 op/s 2024-03-11T19:50:26.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:25 smithi083 ceph-mon[31864]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 714 op/s 2024-03-11T19:50:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 670.0 s, 2021.8 tps, lat 49.117 ms stddev 59.483 2024-03-11T19:50:28.120 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:27 smithi083 ceph-mon[31864]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-03-11T19:50:28.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:27 smithi137 ceph-mon[36220]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-03-11T19:50:28.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:27 smithi204 ceph-mon[36739]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-03-11T19:50:30.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:29 smithi137 ceph-mon[36220]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 721 op/s 2024-03-11T19:50:30.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:29 smithi204 ceph-mon[36739]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 721 op/s 2024-03-11T19:50:30.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:29 smithi083 ceph-mon[31864]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 721 op/s 2024-03-11T19:50:32.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:31 smithi137 ceph-mon[36220]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.7 MiB/s wr, 612 op/s 2024-03-11T19:50:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:31 smithi204 ceph-mon[36739]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.7 MiB/s wr, 612 op/s 2024-03-11T19:50:32.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:31 smithi083 ceph-mon[31864]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.7 MiB/s wr, 612 op/s 2024-03-11T19:50:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 675.0 s, 3055.2 tps, lat 33.076 ms stddev 41.976 2024-03-11T19:50:33.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:50:33.302 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:50:33.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.454+0000 7f5148779640 1 -- 172.21.15.83:0/2859849685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f514010d4a0 msgr2=0x7f514010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.454+0000 7f5148779640 1 --2- 172.21.15.83:0/2859849685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f514010d4a0 0x7f514010d920 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f513001a910 tx=0x7f513003cca0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.457+0000 7f5148779640 1 -- 172.21.15.83:0/2859849685 shutdown_connections 2024-03-11T19:50:33.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.457+0000 7f5148779640 1 --2- 172.21.15.83:0/2859849685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f514010d4a0 0x7f514010d920 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.457+0000 7f5148779640 1 --2- 172.21.15.83:0/2859849685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514010aaf0 0x7f514010cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.457+0000 7f5148779640 1 --2- 172.21.15.83:0/2859849685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5140108180 0x7f514010a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.457+0000 7f5148779640 1 -- 172.21.15.83:0/2859849685 >> 172.21.15.83:0/2859849685 conn(0x7f514006e200 msgr2=0x7f5140070640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:33.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.461+0000 7f5148779640 1 -- 172.21.15.83:0/2859849685 shutdown_connections 2024-03-11T19:50:33.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.462+0000 7f5148779640 1 -- 172.21.15.83:0/2859849685 wait complete. 2024-03-11T19:50:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.463+0000 7f5148779640 1 Processor -- start 2024-03-11T19:50:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.463+0000 7f5148779640 1 -- start start 2024-03-11T19:50:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.464+0000 7f5148779640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140108180 0x7f514019d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.464+0000 7f5148779640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f514010aaf0 0x7f514019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.464+0000 7f5148779640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514010d4a0 0x7f51401a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.464+0000 7f5148779640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51401a5310 con 0x7f514010d4a0 2024-03-11T19:50:33.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.464+0000 7f5148779640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51401a5480 con 0x7f5140108180 2024-03-11T19:50:33.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.464+0000 7f5148779640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51401a5640 con 0x7f514010aaf0 2024-03-11T19:50:33.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.464+0000 7f5146d16640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514010d4a0 0x7f51401a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.464+0000 7f5146d16640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514010d4a0 0x7f51401a4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52278/0 (socket says 172.21.15.83:52278) 2024-03-11T19:50:33.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.464+0000 7f5146d16640 1 -- 172.21.15.83:0/2951828009 learned_addr learned my addr 172.21.15.83:0/2951828009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:33.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.466+0000 7f5145d14640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f514010aaf0 0x7f514019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.469+0000 7f5146d16640 1 -- 172.21.15.83:0/2951828009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f514010aaf0 msgr2=0x7f514019dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.469+0000 7f5146d16640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f514010aaf0 0x7f514019dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.469+0000 7f5146d16640 1 -- 172.21.15.83:0/2951828009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140108180 msgr2=0x7f514019d820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-11T19:50:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.469+0000 7f5146d16640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140108180 0x7f514019d820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.469+0000 7f5146d16640 1 -- 172.21.15.83:0/2951828009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513001a570 con 0x7f514010d4a0 2024-03-11T19:50:33.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.473+0000 7f5146d16640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514010d4a0 0x7f51401a4dd0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f513001a860 tx=0x7f5130004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:33.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.477+0000 7f51377fe640 1 -- 172.21.15.83:0/2951828009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513003da60 con 0x7f514010d4a0 2024-03-11T19:50:33.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.478+0000 7f5148779640 1 -- 172.21.15.83:0/2951828009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51401a58c0 con 0x7f514010d4a0 2024-03-11T19:50:33.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.478+0000 7f5148779640 1 -- 172.21.15.83:0/2951828009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51401a5d70 con 0x7f514010d4a0 2024-03-11T19:50:33.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.478+0000 7f51377fe640 1 -- 172.21.15.83:0/2951828009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f513003dbc0 con 0x7f514010d4a0 2024-03-11T19:50:33.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.478+0000 7f51377fe640 1 -- 172.21.15.83:0/2951828009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130003be0 con 0x7f514010d4a0 2024-03-11T19:50:33.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.482+0000 7f514b41f740 1 Processor -- start 2024-03-11T19:50:33.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.482+0000 7f514b41f740 1 -- start start 2024-03-11T19:50:33.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.483+0000 7f514b41f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749b2d0 0x55c7b749b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.483+0000 7f514b41f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b749bcc0 0x55c7b74a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.483+0000 7f514b41f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b74a0e30 0x55c7b74a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.483+0000 7f514b41f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b740b280 con 0x55c7b74a0e30 2024-03-11T19:50:33.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.483+0000 7f514b41f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b74087b0 con 0x55c7b749b2d0 2024-03-11T19:50:33.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.483+0000 7f514b41f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b7403f90 con 0x55c7b749bcc0 2024-03-11T19:50:33.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.484+0000 7f5146515640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b749bcc0 0x55c7b74a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.484+0000 7f5146515640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b749bcc0 0x55c7b74a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47694/0 (socket says 172.21.15.83:47694) 2024-03-11T19:50:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.484+0000 7f5146515640 1 -- 172.21.15.83:0/3796749977 learned_addr learned my addr 172.21.15.83:0/3796749977 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.485+0000 7f5146515640 1 -- 172.21.15.83:0/3796749977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749b2d0 msgr2=0x55c7b749b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.485+0000 7f5146515640 1 --2- 172.21.15.83:0/3796749977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749b2d0 0x55c7b749b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.485+0000 7f5146515640 1 -- 172.21.15.83:0/3796749977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b74a0e30 msgr2=0x55c7b74a32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.485+0000 7f5146515640 1 --2- 172.21.15.83:0/3796749977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b74a0e30 0x55c7b74a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.485+0000 7f5146515640 1 -- 172.21.15.83:0/3796749977 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7b73c2570 con 0x55c7b749bcc0 2024-03-11T19:50:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.486+0000 7f5146515640 1 --2- 172.21.15.83:0/3796749977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b749bcc0 0x55c7b74a07c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5138009da0 tx=0x7f513800b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c87e4eff96f2f280 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.488+0000 7f5134ff9640 1 -- 172.21.15.83:0/3796749977 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513800e070 con 0x55c7b749bcc0 2024-03-11T19:50:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.489+0000 7f5134ff9640 1 -- 172.21.15.83:0/3796749977 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51380098d0 con 0x55c7b749bcc0 2024-03-11T19:50:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.490+0000 7f514b41f740 1 -- 172.21.15.83:0/3796749977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b749bcc0 msgr2=0x55c7b74a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.490+0000 7f514b41f740 1 --2- 172.21.15.83:0/3796749977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b749bcc0 0x55c7b74a07c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5138009da0 tx=0x7f513800b420 comp rx=0 tx=0).stop 2024-03-11T19:50:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.490+0000 7f51377fe640 1 -- 172.21.15.83:0/2951828009 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5130003d40 con 0x7f514010d4a0 2024-03-11T19:50:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.491+0000 7f51377fe640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f512c0a1c30 0x7f512c0a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.491+0000 7f51377fe640 1 -- 172.21.15.83:0/2951828009 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f51300ff8c0 con 0x7f514010d4a0 2024-03-11T19:50:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.492+0000 7f514b41f740 1 -- 172.21.15.83:0/3796749977 shutdown_connections 2024-03-11T19:50:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.492+0000 7f514b41f740 1 --2- 172.21.15.83:0/3796749977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b74a0e30 0x55c7b74a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.492+0000 7f514b41f740 1 --2- 172.21.15.83:0/3796749977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b749bcc0 0x55c7b74a07c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.492+0000 7f514b41f740 1 --2- 172.21.15.83:0/3796749977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749b2d0 0x55c7b749b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.492+0000 7f514b41f740 1 -- 172.21.15.83:0/3796749977 >> 172.21.15.83:0/3796749977 conn(0x55c7b74967f0 msgr2=0x55c7b7496bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.492+0000 7f5145d14640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f512c0a1c30 0x7f512c0a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.495+0000 7f5145d14640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f512c0a1c30 0x7f512c0a40f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f513c0058c0 tx=0x7f513c0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:33.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.498+0000 7f514b41f740 1 -- 172.21.15.83:0/3796749977 shutdown_connections 2024-03-11T19:50:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.499+0000 7f514b41f740 1 -- 172.21.15.83:0/3796749977 wait complete. 2024-03-11T19:50:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.499+0000 7f514b41f740 1 -- start start 2024-03-11T19:50:33.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.499+0000 7f514b41f740 1 Processor -- start 2024-03-11T19:50:33.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.500+0000 7f514b41f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749b2d0 0x55c7b74a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.500+0000 7f514b41f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749bcc0 0x55c7b74a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.500+0000 7f514b41f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b74ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.500+0000 7f514b41f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b70d7430 con 0x55c7b749bcc0 2024-03-11T19:50:33.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.500+0000 7f514b41f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b7093e00 con 0x55c7b749b2d0 2024-03-11T19:50:33.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.500+0000 7f514b41f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b7388fd0 con 0x55c7b74a0e30 2024-03-11T19:50:33.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.500+0000 7f514b41f740 10 client.0 _ll_register_callbacks cb 0x55c7b73defd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:33.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.501+0000 7f5145d14640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749bcc0 0x55c7b74a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.501+0000 7f5145d14640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749bcc0 0x55c7b74a57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52298/0 (socket says 172.21.15.83:52298) 2024-03-11T19:50:33.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.501+0000 7f5145d14640 1 -- 172.21.15.83:0/3286868508 learned_addr learned my addr 172.21.15.83:0/3286868508 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:33.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.502+0000 7f5146d16640 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b74ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.502+0000 7f5146515640 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749b2d0 0x55c7b74a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.503+0000 7f5146515640 1 -- 172.21.15.83:0/3286868508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 msgr2=0x55c7b74ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:33.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.503+0000 7f5146515640 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b74ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.503+0000 7f5146515640 1 -- 172.21.15.83:0/3286868508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749bcc0 msgr2=0x55c7b74a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:33.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.503+0000 7f5146515640 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749bcc0 0x55c7b74a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.503+0000 7f5146515640 1 -- 172.21.15.83:0/3286868508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5138009a50 con 0x55c7b749b2d0 2024-03-11T19:50:33.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.503+0000 7f5146d16640 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b74ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:50:33.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.504+0000 7f5146515640 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749b2d0 0x55c7b74a52b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55c7b74977b0 tx=0x7f513800bf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75f8ee65fbf4dcbd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:33.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.504+0000 7f5129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:50:33.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.504+0000 7f5129ffb640 1 -- 172.21.15.83:0/3286868508 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513800e040 con 0x55c7b749b2d0 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.504+0000 7f514b41f740 1 -- 172.21.15.83:0/3286868508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7b7140720 con 0x55c7b749b2d0 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.504+0000 7f5129ffb640 1 -- 172.21.15.83:0/3286868508 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5138011da0 con 0x55c7b749b2d0 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.505+0000 7f5129ffb640 1 -- 172.21.15.83:0/3286868508 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5138012430 con 0x55c7b749b2d0 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.505+0000 7f514b41f740 10 client.26891 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.505+0000 7f514b41f740 1 -- 172.21.15.83:0/3286868508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c7b7402610 con 0x55c7b749b2d0 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.505+0000 7f5129ffb640 1 -- 172.21.15.83:0/3286868508 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c7b7140720 con 0x55c7b749b2d0 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.505+0000 7f5129ffb640 1 -- 172.21.15.83:0/3286868508 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f513801d440 con 0x55c7b749b2d0 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.506+0000 7f514b41f740 10 client.26891 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.506+0000 7f514b41f740 10 client.26891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.506+0000 7f514b41f740 20 client.26891 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.506+0000 7f514b41f740 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c7b74b13f0 0x55c7b74b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.506+0000 7f514b41f740 4 client.26891 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:33.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.506+0000 7f514b41f740 1 -- 172.21.15.83:0/3286868508 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7b718f890 con 0x55c7b74b13f0 2024-03-11T19:50:33.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.506+0000 7f5145d14640 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c7b74b13f0 0x55c7b74b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.506+0000 7f5129ffb640 10 client.26891 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:33.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.507+0000 7f5145d14640 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c7b74b13f0 0x55c7b74b3810 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=192f4c10fb990b66 server_cookie=7b6085fc8e78b749 in_seq=0 out_seq=0 2024-03-11T19:50:33.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f5129ffb640 1 -- 172.21.15.83:0/3286868508 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7b718f890 con 0x55c7b74b13f0 2024-03-11T19:50:33.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f5129ffb640 10 client.26891 handle_command_reply: tid=0 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 1 client.26891 shutdown 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 1 -- 172.21.15.83:0/3286868508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749b2d0 msgr2=0x55c7b74a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749b2d0 0x55c7b74a52b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55c7b74977b0 tx=0x7f513800bf70 comp rx=0 tx=0).stop 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 1 -- 172.21.15.83:0/3286868508 shutdown_connections 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c7b74b13f0 0x55c7b74b3810 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b74ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f5129ffb640 0 client.26891 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749bcc0 0x55c7b74a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 1 --2- 172.21.15.83:0/3286868508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749b2d0 0x55c7b74a52b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 1 -- 172.21.15.83:0/3286868508 >> 172.21.15.83:0/3286868508 conn(0x55c7b74967f0 msgr2=0x55c7b7495a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 1 -- 172.21.15.83:0/3286868508 shutdown_connections 2024-03-11T19:50:33.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 1 -- 172.21.15.83:0/3286868508 wait complete. 2024-03-11T19:50:33.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.508+0000 7f514b41f740 20 client.26891 trim_cache size 0 max 16384 2024-03-11T19:50:33.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.532+0000 7f514b41f740 1 Processor -- start 2024-03-11T19:50:33.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.532+0000 7f514b41f740 1 -- start start 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.532+0000 7f514b41f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f514b41f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749bcc0 0x55c7b74ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f5146515640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f5146515640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52306/0 (socket says 172.21.15.83:52306) 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f5146515640 1 -- 172.21.15.83:0/1897900556 learned_addr learned my addr 172.21.15.83:0/1897900556 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f514b41f740 1 --2- 172.21.15.83:0/1897900556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b7495700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f514b41f740 1 -- 172.21.15.83:0/1897900556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b749ae90 con 0x55c7b749b2d0 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f514b41f740 1 -- 172.21.15.83:0/1897900556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b74a3ef0 con 0x55c7b749bcc0 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f514b41f740 1 -- 172.21.15.83:0/1897900556 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b74b8a40 con 0x55c7b74a0e30 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f5146d16640 1 --2- 172.21.15.83:0/1897900556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b7495700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f5145d14640 1 --2- 172.21.15.83:0/1897900556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749bcc0 0x55c7b74ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f5146515640 1 -- 172.21.15.83:0/1897900556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 msgr2=0x55c7b7495700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:33.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f5146515640 1 --2- 172.21.15.83:0/1897900556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b7495700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f5146515640 1 -- 172.21.15.83:0/1897900556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749bcc0 msgr2=0x55c7b74ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:33.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f5146515640 1 --2- 172.21.15.83:0/1897900556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749bcc0 0x55c7b74ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.533+0000 7f5146515640 1 -- 172.21.15.83:0/1897900556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7b739eac0 con 0x55c7b749b2d0 2024-03-11T19:50:33.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.534+0000 7f5146515640 1 --2- 172.21.15.83:0/1897900556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74951c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f513800d010 tx=0x7f513800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4feffb96e9b18a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:33.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.534+0000 7f512bfff640 1 -- 172.21.15.83:0/1897900556 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513802a070 con 0x55c7b749b2d0 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.534+0000 7f512bfff640 1 -- 172.21.15.83:0/1897900556 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5138007000 con 0x55c7b749b2d0 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.534+0000 7f512bfff640 1 -- 172.21.15.83:0/1897900556 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513800e040 con 0x55c7b749b2d0 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.535+0000 7f514b41f740 1 -- 172.21.15.83:0/1897900556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 msgr2=0x55c7b74951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.535+0000 7f514b41f740 1 --2- 172.21.15.83:0/1897900556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74951c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f513800d010 tx=0x7f513800b420 comp rx=0 tx=0).stop 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.535+0000 7f514b41f740 1 -- 172.21.15.83:0/1897900556 shutdown_connections 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.535+0000 7f514b41f740 1 --2- 172.21.15.83:0/1897900556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b7495700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.535+0000 7f514b41f740 1 --2- 172.21.15.83:0/1897900556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749bcc0 0x55c7b74ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.535+0000 7f514b41f740 1 --2- 172.21.15.83:0/1897900556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74951c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.535+0000 7f514b41f740 1 -- 172.21.15.83:0/1897900556 >> 172.21.15.83:0/1897900556 conn(0x55c7b74967f0 msgr2=0x55c7b749ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.535+0000 7f514b41f740 1 -- 172.21.15.83:0/1897900556 shutdown_connections 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.535+0000 7f514b41f740 1 -- 172.21.15.83:0/1897900556 wait complete. 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.535+0000 7f514b41f740 1 -- start start 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f514b41f740 1 Processor -- start 2024-03-11T19:50:33.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f514b41f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74aa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f514b41f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749bcc0 0x55c7b74aab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f514b41f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b74abb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f514b41f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b74ac0c0 con 0x55c7b749b2d0 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f514b41f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b74ac230 con 0x55c7b749bcc0 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f514b41f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7b74ac3f0 con 0x55c7b74a0e30 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f514b41f740 10 client.0 _ll_register_callbacks cb 0x7f51380095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5146515640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5146d16640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b74abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5146515640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74aa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52322/0 (socket says 172.21.15.83:52322) 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5146515640 1 -- 172.21.15.83:0/2228240825 learned_addr learned my addr 172.21.15.83:0/2228240825 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5145d14640 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749bcc0 0x55c7b74aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5146515640 1 -- 172.21.15.83:0/2228240825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 msgr2=0x55c7b74abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:33.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5146515640 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b74abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5146515640 1 -- 172.21.15.83:0/2228240825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749bcc0 msgr2=0x55c7b74aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5146515640 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749bcc0 0x55c7b74aab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5146515640 1 -- 172.21.15.83:0/2228240825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7b7402610 con 0x55c7b749b2d0 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5146d16640 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b74abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5146515640 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74aa630 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f513800b9a0 tx=0x7f51380071e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ee82e359ee0073 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.536+0000 7f5129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f5129ffb640 1 -- 172.21.15.83:0/2228240825 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513802a050 con 0x55c7b749b2d0 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f5129ffb640 1 -- 172.21.15.83:0/2228240825 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f513801d580 con 0x55c7b749b2d0 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f514b41f740 1 -- 172.21.15.83:0/2228240825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7b71037e0 con 0x55c7b749b2d0 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f5129ffb640 1 -- 172.21.15.83:0/2228240825 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f513800e070 con 0x55c7b749b2d0 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f514b41f740 10 client.17433 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f514b41f740 1 -- 172.21.15.83:0/2228240825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c7b73c46a0 con 0x55c7b749b2d0 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f5129ffb640 1 -- 172.21.15.83:0/2228240825 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c7b71037e0 con 0x55c7b749b2d0 2024-03-11T19:50:33.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f5129ffb640 1 -- 172.21.15.83:0/2228240825 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5138026aa0 con 0x55c7b749b2d0 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f514b41f740 10 client.17433 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f514b41f740 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f514b41f740 20 client.17433 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f514b41f740 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c7b7508440 0x55c7b750a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f514b41f740 4 client.17433 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f514b41f740 1 -- 172.21.15.83:0/2228240825 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c7b70f24e0 con 0x55c7b7508440 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.537+0000 7f5145d14640 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c7b7508440 0x55c7b750a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f5129ffb640 10 client.17433 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f5145d14640 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c7b7508440 0x55c7b750a860 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aba7068e16a54c4f server_cookie=355b58402a90fc87 in_seq=0 out_seq=0 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f5129ffb640 1 -- 172.21.15.83:0/2228240825 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c7b70f24e0 con 0x55c7b7508440 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f5129ffb640 10 client.17433 handle_command_reply: tid=0 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 1 client.17433 shutdown 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 1 -- 172.21.15.83:0/2228240825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 msgr2=0x55c7b74aa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74aa630 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f513800b9a0 tx=0x7f51380071e0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 1 -- 172.21.15.83:0/2228240825 shutdown_connections 2024-03-11T19:50:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c7b7508440 0x55c7b750a860 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c7b74a0e30 0x55c7b74abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7b749bcc0 0x55c7b74aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 1 --2- 172.21.15.83:0/2228240825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7b749b2d0 0x55c7b74aa630 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 1 -- 172.21.15.83:0/2228240825 >> 172.21.15.83:0/2228240825 conn(0x55c7b74967f0 msgr2=0x55c7b7499aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f5129ffb640 0 client.17433 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 1 -- 172.21.15.83:0/2228240825 shutdown_connections 2024-03-11T19:50:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 1 -- 172.21.15.83:0/2228240825 wait complete. 2024-03-11T19:50:33.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.538+0000 7f514b41f740 20 client.17433 trim_cache size 0 max 16384 2024-03-11T19:50:33.549 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:50:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:50:33.550 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:50:33.550 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:50:33.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.539+0000 7f5148779640 1 -- 172.21.15.83:0/2951828009 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f512c0a1c30 msgr2=0x7f512c0a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.539+0000 7f5148779640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f512c0a1c30 0x7f512c0a40f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f513c0058c0 tx=0x7f513c0046c0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.539+0000 7f5148779640 1 -- 172.21.15.83:0/2951828009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514010d4a0 msgr2=0x7f51401a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.539+0000 7f5148779640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514010d4a0 0x7f51401a4dd0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f513001a860 tx=0x7f5130004510 comp rx=0 tx=0).stop 2024-03-11T19:50:33.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.540+0000 7f5148779640 1 -- 172.21.15.83:0/2951828009 shutdown_connections 2024-03-11T19:50:33.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.540+0000 7f5148779640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f512c0a1c30 0x7f512c0a40f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.540+0000 7f5148779640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f514010d4a0 0x7f51401a4dd0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.540+0000 7f5148779640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f514010aaf0 0x7f514019dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.540+0000 7f5148779640 1 --2- 172.21.15.83:0/2951828009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5140108180 0x7f514019d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.540+0000 7f5148779640 1 -- 172.21.15.83:0/2951828009 >> 172.21.15.83:0/2951828009 conn(0x7f514006e200 msgr2=0x7f514010b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:33.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.541+0000 7f5148779640 1 -- 172.21.15.83:0/2951828009 shutdown_connections 2024-03-11T19:50:33.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.541+0000 7f5148779640 1 -- 172.21.15.83:0/2951828009 wait complete. 2024-03-11T19:50:33.559 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:50:33.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:50:33.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.697+0000 7fa2db50f640 1 -- 172.21.15.83:0/4180552750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d41011b0 msgr2=0x7fa2d41015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.697+0000 7fa2db50f640 1 --2- 172.21.15.83:0/4180552750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d41011b0 0x7fa2d41015d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009900 tx=0x7fa2d002bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.697+0000 7fa2db50f640 1 -- 172.21.15.83:0/4180552750 shutdown_connections 2024-03-11T19:50:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.697+0000 7fa2db50f640 1 --2- 172.21.15.83:0/4180552750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2d4102d60 0x7fa2d41095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.697+0000 7fa2db50f640 1 --2- 172.21.15.83:0/4180552750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d41023a0 0x7fa2d4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.697+0000 7fa2db50f640 1 --2- 172.21.15.83:0/4180552750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d41011b0 0x7fa2d41015d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.697+0000 7fa2db50f640 1 -- 172.21.15.83:0/4180552750 >> 172.21.15.83:0/4180552750 conn(0x7fa2d40fc720 msgr2=0x7fa2d40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.697+0000 7fa2db50f640 1 -- 172.21.15.83:0/4180552750 shutdown_connections 2024-03-11T19:50:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.697+0000 7fa2db50f640 1 -- 172.21.15.83:0/4180552750 wait complete. 2024-03-11T19:50:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2db50f640 1 Processor -- start 2024-03-11T19:50:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2db50f640 1 -- start start 2024-03-11T19:50:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2db50f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d41011b0 0x7fa2d4197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2db50f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d41023a0 0x7fa2d41977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2db50f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2d4102d60 0x7fa2d419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2db50f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d419ed60 con 0x7fa2d41023a0 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2db50f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d419eed0 con 0x7fa2d41011b0 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2db50f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d419f090 con 0x7fa2d4102d60 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2d8aaa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d41023a0 0x7fa2d41977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2d92ab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d41011b0 0x7fa2d4197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2d92ab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d41011b0 0x7fa2d4197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:48778/0 (socket says 172.21.15.83:48778) 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2d8aaa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d41023a0 0x7fa2d41977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52342/0 (socket says 172.21.15.83:52342) 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2d92ab640 1 -- 172.21.15.83:0/1132576520 learned_addr learned my addr 172.21.15.83:0/1132576520 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.698+0000 7fa2d9aac640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2d4102d60 0x7fa2d419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2d92ab640 1 -- 172.21.15.83:0/1132576520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2d4102d60 msgr2=0x7fa2d419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2d92ab640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2d4102d60 0x7fa2d419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2d92ab640 1 -- 172.21.15.83:0/1132576520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d41023a0 msgr2=0x7fa2d41977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2d92ab640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d41023a0 0x7fa2d41977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2d92ab640 1 -- 172.21.15.83:0/1132576520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0009560 con 0x7fa2d41011b0 2024-03-11T19:50:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2d8aaa640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d41023a0 0x7fa2d41977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2d92ab640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d41011b0 0x7fa2d4197270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0006010 tx=0x7fa2d0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2ca7fc640 1 -- 172.21.15.83:0/1132576520 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00043d0 con 0x7fa2d41011b0 2024-03-11T19:50:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2db50f640 1 -- 172.21.15.83:0/1132576520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d419f310 con 0x7fa2d41011b0 2024-03-11T19:50:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2ca7fc640 1 -- 172.21.15.83:0/1132576520 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2d00052d0 con 0x7fa2d41011b0 2024-03-11T19:50:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2db50f640 1 -- 172.21.15.83:0/1132576520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d419f840 con 0x7fa2d41011b0 2024-03-11T19:50:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.699+0000 7fa2ca7fc640 1 -- 172.21.15.83:0/1132576520 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0003d30 con 0x7fa2d41011b0 2024-03-11T19:50:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.700+0000 7fa2db50f640 1 -- 172.21.15.83:0/1132576520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d41084d0 con 0x7fa2d41011b0 2024-03-11T19:50:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.701+0000 7fa2ca7fc640 1 -- 172.21.15.83:0/1132576520 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa2d0005440 con 0x7fa2d41011b0 2024-03-11T19:50:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.702+0000 7fa2ca7fc640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2c00a1a30 0x7fa2c00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.702+0000 7fa2ca7fc640 1 -- 172.21.15.83:0/1132576520 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa2d00efcf0 con 0x7fa2d41011b0 2024-03-11T19:50:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.702+0000 7fa2d8aaa640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2c00a1a30 0x7fa2c00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:33.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.703+0000 7fa2d8aaa640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2c00a1a30 0x7fa2c00a3ef0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4198780 tx=0x7fa2c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:33.706+0000 7fa2ca7fc640 1 -- 172.21.15.83:0/1132576520 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa2d00b8fb0 con 0x7fa2d41011b0 2024-03-11T19:50:34.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.025+0000 7fa2db50f640 1 -- 172.21.15.83:0/1132576520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2d410cc10 con 0x7fa2d41011b0 2024-03-11T19:50:34.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.027+0000 7fa2ca7fc640 1 -- 172.21.15.83:0/1132576520 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7fa2d0043090 con 0x7fa2d41011b0 2024-03-11T19:50:34.032 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:50:34.032 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:50:34.033 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:50:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 -- 172.21.15.83:0/1132576520 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2c00a1a30 msgr2=0x7fa2c00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2c00a1a30 0x7fa2c00a3ef0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4198780 tx=0x7fa2c4005f90 comp rx=0 tx=0).stop 2024-03-11T19:50:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 -- 172.21.15.83:0/1132576520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d41011b0 msgr2=0x7fa2d4197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d41011b0 0x7fa2d4197270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0006010 tx=0x7fa2d0005e80 comp rx=0 tx=0).stop 2024-03-11T19:50:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 -- 172.21.15.83:0/1132576520 shutdown_connections 2024-03-11T19:50:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa2c00a1a30 0x7fa2c00a3ef0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa2d4102d60 0x7fa2d419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d41023a0 0x7fa2d41977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 --2- 172.21.15.83:0/1132576520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d41011b0 0x7fa2d4197270 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 -- 172.21.15.83:0/1132576520 >> 172.21.15.83:0/1132576520 conn(0x7fa2d40fc720 msgr2=0x7fa2d41056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 -- 172.21.15.83:0/1132576520 shutdown_connections 2024-03-11T19:50:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.029+0000 7fa2db50f640 1 -- 172.21.15.83:0/1132576520 wait complete. 2024-03-11T19:50:34.047 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:50:34.048 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:50:34.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:33 smithi137 ceph-mon[36220]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 581 op/s 2024-03-11T19:50:34.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:33 smithi204 ceph-mon[36739]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 581 op/s 2024-03-11T19:50:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.197+0000 7f5a3bfff640 1 -- 172.21.15.83:0/3669536273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c02cea0 con 0x7f5a44101c30 2024-03-11T19:50:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.197+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3669536273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a44101c30 msgr2=0x7f5a441084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.197+0000 7f5a4b7e2640 1 --2- 172.21.15.83:0/3669536273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a44101c30 0x7f5a441084c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009ca0 tx=0x7f5a3c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.197+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3669536273 shutdown_connections 2024-03-11T19:50:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.197+0000 7f5a4b7e2640 1 --2- 172.21.15.83:0/3669536273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a44101c30 0x7f5a441084c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.197+0000 7f5a4b7e2640 1 --2- 172.21.15.83:0/3669536273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a44101270 0x7f5a441016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.197+0000 7f5a4b7e2640 1 --2- 172.21.15.83:0/3669536273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44100080 0x7f5a441004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.197+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3669536273 >> 172.21.15.83:0/3669536273 conn(0x7f5a440fb610 msgr2=0x7f5a440fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.197+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3669536273 shutdown_connections 2024-03-11T19:50:34.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.197+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3669536273 wait complete. 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a4b7e2640 1 Processor -- start 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a4b7e2640 1 -- start start 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a4b7e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44100080 0x7f5a44070940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a4b7e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a44101270 0x7f5a44070e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a4b7e2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a44101c30 0x7f5a44071e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a4b7e2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a440723b0 con 0x7f5a44101270 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a4b7e2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a44072520 con 0x7f5a44100080 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a4b7e2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a440726e0 con 0x7f5a44101c30 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a48d7d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a44101270 0x7f5a44070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a48d7d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a44101270 0x7f5a44070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52358/0 (socket says 172.21.15.83:52358) 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a48d7d640 1 -- 172.21.15.83:0/3863886895 learned_addr learned my addr 172.21.15.83:0/3863886895 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a4957e640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44100080 0x7f5a44070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.198+0000 7f5a49d7f640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a44101c30 0x7f5a44071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a4957e640 1 -- 172.21.15.83:0/3863886895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a44101c30 msgr2=0x7f5a44071e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a4957e640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a44101c30 0x7f5a44071e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a4957e640 1 -- 172.21.15.83:0/3863886895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a44101270 msgr2=0x7f5a44070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a4957e640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a44101270 0x7f5a44070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a4957e640 1 -- 172.21.15.83:0/3863886895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c009900 con 0x7f5a44100080 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a48d7d640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a44101270 0x7f5a44070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a49d7f640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a44101c30 0x7f5a44071e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a4957e640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44100080 0x7f5a44070940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000e9c0 tx=0x7f5a4000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a3a7fc640 1 -- 172.21.15.83:0/3863886895 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000ccc0 con 0x7f5a44100080 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3863886895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a440729c0 con 0x7f5a44100080 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a3a7fc640 1 -- 172.21.15.83:0/3863886895 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a4000ce20 con 0x7f5a44100080 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a3a7fc640 1 -- 172.21.15.83:0/3863886895 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40010430 con 0x7f5a44100080 2024-03-11T19:50:34.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.199+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3863886895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a44072ed0 con 0x7f5a44100080 2024-03-11T19:50:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a3a7fc640 1 -- 172.21.15.83:0/3863886895 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5a40003f60 con 0x7f5a44100080 2024-03-11T19:50:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a4e3b2740 1 Processor -- start 2024-03-11T19:50:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a4e3b2740 1 -- start start 2024-03-11T19:50:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 0x55a9f7203690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f7208760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f720b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a4e3b2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f71bf960 con 0x55a9f7203270 2024-03-11T19:50:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a4e3b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f7173310 con 0x55a9f7203c60 2024-03-11T19:50:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a4e3b2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f7170840 con 0x55a9f7208dd0 2024-03-11T19:50:34.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a49d7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 0x55a9f7203690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a48d7d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f720b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a48d7d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f720b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47776/0 (socket says 172.21.15.83:47776) 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a49d7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 0x55a9f7203690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52360/0 (socket says 172.21.15.83:52360) 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a48d7d640 1 -- 172.21.15.83:0/3722586131 learned_addr learned my addr 172.21.15.83:0/3722586131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a4957e640 1 --2- 172.21.15.83:0/3722586131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f7208760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.202+0000 7f5a3a7fc640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a300a1a30 0x7f5a300a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a49d7f640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a300a1a30 0x7f5a300a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a48d7d640 1 -- 172.21.15.83:0/3722586131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 msgr2=0x55a9f7208760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a48d7d640 1 --2- 172.21.15.83:0/3722586131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f7208760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a48d7d640 1 -- 172.21.15.83:0/3722586131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 msgr2=0x55a9f7203690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a48d7d640 1 --2- 172.21.15.83:0/3722586131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 0x55a9f7203690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a48d7d640 1 -- 172.21.15.83:0/3722586131 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f712e060 con 0x55a9f7208dd0 2024-03-11T19:50:34.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a3a7fc640 1 -- 172.21.15.83:0/3863886895 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5a40014070 con 0x7f5a44100080 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a4957e640 1 --2- 172.21.15.83:0/3722586131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f7208760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a48d7d640 1 --2- 172.21.15.83:0/3722586131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f720b260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a340115f0 tx=0x7f5a34011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8e5c90c4891d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a49d7f640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a300a1a30 0x7f5a300a3ef0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c004af0 tx=0x7f5a3c02c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a2f7fe640 1 -- 172.21.15.83:0/3722586131 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3400f6c0 con 0x55a9f7208dd0 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a2f7fe640 1 -- 172.21.15.83:0/3722586131 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a3400fd00 con 0x55a9f7208dd0 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.203+0000 7f5a2f7fe640 1 -- 172.21.15.83:0/3722586131 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a34019e20 con 0x55a9f7208dd0 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.204+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/3722586131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 msgr2=0x55a9f720b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.204+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/3722586131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f720b260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a340115f0 tx=0x7f5a34011ab0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.204+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/3722586131 shutdown_connections 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.204+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/3722586131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f720b260 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.204+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/3722586131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f7208760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.204+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/3722586131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 0x55a9f7203690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.204+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/3722586131 >> 172.21.15.83:0/3722586131 conn(0x55a9f71fe7e0 msgr2=0x55a9f71febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.204+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/3722586131 shutdown_connections 2024-03-11T19:50:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.204+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/3722586131 wait complete. 2024-03-11T19:50:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.204+0000 7f5a4e3b2740 1 -- start start 2024-03-11T19:50:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.204+0000 7f5a4e3b2740 1 Processor -- start 2024-03-11T19:50:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.205+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203270 0x55a9f720d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.205+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203c60 0x55a9f720d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.205+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f7214a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.205+0000 7f5a4e3b2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f716c020 con 0x55a9f7203c60 2024-03-11T19:50:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.205+0000 7f5a4e3b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f6e3f430 con 0x55a9f7203270 2024-03-11T19:50:34.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.205+0000 7f5a4e3b2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f6dfbe00 con 0x55a9f7208dd0 2024-03-11T19:50:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.205+0000 7f5a4e3b2740 10 client.0 _ll_register_callbacks cb 0x55a9f7113cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.206+0000 7f5a4957e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203270 0x55a9f720d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.206+0000 7f5a4957e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203270 0x55a9f720d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:48834/0 (socket says 172.21.15.83:48834) 2024-03-11T19:50:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.206+0000 7f5a4957e640 1 -- 172.21.15.83:0/2255323647 learned_addr learned my addr 172.21.15.83:0/2255323647 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.206+0000 7f5a49d7f640 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f7214a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.206+0000 7f5a49d7f640 1 -- 172.21.15.83:0/2255323647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203270 msgr2=0x55a9f720d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.206+0000 7f5a49d7f640 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203270 0x55a9f720d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.206+0000 7f5a49d7f640 1 -- 172.21.15.83:0/2255323647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203c60 msgr2=0x55a9f720d9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:50:34.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.206+0000 7f5a49d7f640 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203c60 0x55a9f720d9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.206+0000 7f5a49d7f640 1 -- 172.21.15.83:0/2255323647 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a34009dc0 con 0x55a9f7208dd0 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.207+0000 7f5a49d7f640 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f7214a00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c006010 tx=0x7f5a3c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edcce4fb966aa942 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.207+0000 7f5a2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.207+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2255323647 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c003d80 con 0x55a9f7208dd0 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.207+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2255323647 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9f6e6b7e0 con 0x55a9f7208dd0 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.207+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2255323647 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5a3c004040 con 0x55a9f7208dd0 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.207+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2255323647 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a3c004200 con 0x55a9f7208dd0 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.207+0000 7f5a4e3b2740 10 client.27010 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.207+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2255323647 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a9f716a6a0 con 0x55a9f7208dd0 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.208+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2255323647 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a9f6e6b7e0 con 0x55a9f7208dd0 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.208+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2255323647 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5a3c05f020 con 0x55a9f7208dd0 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.208+0000 7f5a4e3b2740 10 client.27010 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:34.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.208+0000 7f5a4e3b2740 10 client.27010 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.208+0000 7f5a4e3b2740 20 client.27010 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.208+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55a9f72196d0 0x55a9f721bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.208+0000 7f5a4e3b2740 4 client.27010 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.208+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2255323647 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9f6ea8720 con 0x55a9f72196d0 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.208+0000 7f5a4957e640 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55a9f72196d0 0x55a9f721bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.209+0000 7f5a2cff9640 10 client.27010 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.209+0000 7f5a4957e640 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55a9f72196d0 0x55a9f721bb30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8ea070995429fb3 server_cookie=b5f001c2e84a132e in_seq=0 out_seq=0 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.209+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2255323647 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9f6ea8720 con 0x55a9f72196d0 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.209+0000 7f5a2cff9640 10 client.27010 handle_command_reply: tid=0 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 1 client.27010 shutdown 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2255323647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 msgr2=0x55a9f7214a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f7214a00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c006010 tx=0x7f5a3c0095c0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2255323647 shutdown_connections 2024-03-11T19:50:34.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55a9f72196d0 0x55a9f721bb30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f7214a00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203c60 0x55a9f720d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2255323647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203270 0x55a9f720d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2255323647 >> 172.21.15.83:0/2255323647 conn(0x55a9f71fe7e0 msgr2=0x55a9f71fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a2cff9640 0 client.27010 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:50:34.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2255323647 shutdown_connections 2024-03-11T19:50:34.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2255323647 wait complete. 2024-03-11T19:50:34.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.210+0000 7f5a4e3b2740 20 client.27010 trim_cache size 0 max 16384 2024-03-11T19:50:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.226+0000 7f5a4e3b2740 1 Processor -- start 2024-03-11T19:50:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.226+0000 7f5a4e3b2740 1 -- start start 2024-03-11T19:50:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.226+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7203270 0x55a9f7204350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.226+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f7204890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.226+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7208dd0 0x55a9f720d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a48d7d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f7204890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a49d7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7208dd0 0x55a9f720d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a4957e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7203270 0x55a9f7204350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a48d7d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f7204890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:48844/0 (socket says 172.21.15.83:48844) 2024-03-11T19:50:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a49d7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7208dd0 0x55a9f720d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52362/0 (socket says 172.21.15.83:52362) 2024-03-11T19:50:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a48d7d640 1 -- 172.21.15.83:0/2476387246 learned_addr learned my addr 172.21.15.83:0/2476387246 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a4957e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7203270 0x55a9f7204350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47790/0 (socket says 172.21.15.83:47790) 2024-03-11T19:50:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2476387246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f70f1130 con 0x55a9f7208dd0 2024-03-11T19:50:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2476387246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f720e670 con 0x55a9f7203c60 2024-03-11T19:50:34.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2476387246 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f7204090 con 0x55a9f7203270 2024-03-11T19:50:34.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a4957e640 1 -- 172.21.15.83:0/2476387246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 msgr2=0x55a9f7204890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a4957e640 1 --2- 172.21.15.83:0/2476387246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f7204890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a4957e640 1 -- 172.21.15.83:0/2476387246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7208dd0 msgr2=0x55a9f720d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a4957e640 1 --2- 172.21.15.83:0/2476387246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7208dd0 0x55a9f720d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a4957e640 1 -- 172.21.15.83:0/2476387246 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f7106c20 con 0x55a9f7203270 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a48d7d640 1 --2- 172.21.15.83:0/2476387246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f7204890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.227+0000 7f5a4957e640 1 --2- 172.21.15.83:0/2476387246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7203270 0x55a9f7204350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5a4009a4d0 tx=0x7f5a400cb230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a243ec17a9981ad server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.228+0000 7f5a2effd640 1 -- 172.21.15.83:0/2476387246 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4009a7d0 con 0x55a9f7203270 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.228+0000 7f5a2effd640 1 -- 172.21.15.83:0/2476387246 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a4009e920 con 0x55a9f7203270 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.228+0000 7f5a2effd640 1 -- 172.21.15.83:0/2476387246 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400a2ae0 con 0x55a9f7203270 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.228+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2476387246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7203270 msgr2=0x55a9f7204350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.228+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2476387246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7203270 0x55a9f7204350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5a4009a4d0 tx=0x7f5a400cb230 comp rx=0 tx=0).stop 2024-03-11T19:50:34.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.228+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2476387246 shutdown_connections 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.228+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2476387246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7208dd0 0x55a9f720d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.228+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2476387246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f7204890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.228+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2476387246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7203270 0x55a9f7204350 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.228+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2476387246 >> 172.21.15.83:0/2476387246 conn(0x55a9f71fe7e0 msgr2=0x55a9f72051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.228+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2476387246 shutdown_connections 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2476387246 wait complete. 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4e3b2740 1 -- start start 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4e3b2740 1 Processor -- start 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 0x55a9f726c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f726c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4e3b2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f7273600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4e3b2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f7273b40 con 0x55a9f7203270 2024-03-11T19:50:34.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4e3b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f7273c80 con 0x55a9f7203c60 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4e3b2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f7273df0 con 0x55a9f7208dd0 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4e3b2740 10 client.0 _ll_register_callbacks cb 0x7f5a34009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4957e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 0x55a9f726c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4957e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 0x55a9f726c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52368/0 (socket says 172.21.15.83:52368) 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.229+0000 7f5a4957e640 1 -- 172.21.15.83:0/2823997909 learned_addr learned my addr 172.21.15.83:0/2823997909 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a48d7d640 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f726c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a49d7f640 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f7273600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a4957e640 1 -- 172.21.15.83:0/2823997909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 msgr2=0x55a9f7273600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a4957e640 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f7273600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a4957e640 1 -- 172.21.15.83:0/2823997909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 msgr2=0x55a9f726c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a4957e640 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f726c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a4957e640 1 -- 172.21.15.83:0/2823997909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9f712e060 con 0x55a9f7203270 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a48d7d640 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f726c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a4957e640 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 0x55a9f726c080 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000e990 tx=0x7f5a40003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca21f6c5cf55fba server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2823997909 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4009a7d0 con 0x55a9f7203270 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2823997909 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5a400b3210 con 0x55a9f7203270 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.230+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2823997909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a3400fd00 con 0x55a9f7203270 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2823997909 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a40096e40 con 0x55a9f7203270 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a4e3b2740 10 client.17454 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2823997909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f5a44072ed0 con 0x55a9f7203270 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2823997909 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a4007d020 con 0x55a9f7203270 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2823997909 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5a4001f060 con 0x55a9f7203270 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a4e3b2740 10 client.17454 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a4e3b2740 10 client.17454 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:50:34.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a4e3b2740 20 client.17454 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:34.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55a9f7275540 0x55a9f7277960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a4e3b2740 4 client.17454 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:50:34.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2823997909 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a9f6ef7890 con 0x55a9f7275540 2024-03-11T19:50:34.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.231+0000 7f5a48d7d640 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55a9f7275540 0x55a9f7277960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a2cff9640 10 client.17454 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:50:34.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a48d7d640 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55a9f7275540 0x55a9f7277960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f83b2ba10871d04c server_cookie=e85d49cf67c044f7 in_seq=0 out_seq=0 2024-03-11T19:50:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a2cff9640 1 -- 172.21.15.83:0/2823997909 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a9f6ef7890 con 0x55a9f7275540 2024-03-11T19:50:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a2cff9640 10 client.17454 handle_command_reply: tid=0 2024-03-11T19:50:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 1 client.17454 shutdown 2024-03-11T19:50:34.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2823997909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 msgr2=0x55a9f726c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 0x55a9f726c080 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000e990 tx=0x7f5a40003980 comp rx=0 tx=0).stop 2024-03-11T19:50:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2823997909 shutdown_connections 2024-03-11T19:50:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55a9f7275540 0x55a9f7277960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a9f7208dd0 0x55a9f7273600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a2cff9640 0 client.17454 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:50:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f7203c60 0x55a9f726c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 1 --2- 172.21.15.83:0/2823997909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9f7203270 0x55a9f726c080 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2823997909 >> 172.21.15.83:0/2823997909 conn(0x55a9f71fe7e0 msgr2=0x55a9f72051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2823997909 shutdown_connections 2024-03-11T19:50:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 1 -- 172.21.15.83:0/2823997909 wait complete. 2024-03-11T19:50:34.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.232+0000 7f5a4e3b2740 20 client.17454 trim_cache size 0 max 16384 2024-03-11T19:50:34.244 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3863886895 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a300a1a30 msgr2=0x7f5a300a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a300a1a30 0x7f5a300a3ef0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c004af0 tx=0x7f5a3c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3863886895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44100080 msgr2=0x7f5a44070940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44100080 0x7f5a44070940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000e9c0 tx=0x7f5a4000ee80 comp rx=0 tx=0).stop 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3863886895 shutdown_connections 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a300a1a30 0x7f5a300a3ef0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a44101c30 0x7f5a44071e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a44101270 0x7f5a44070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 --2- 172.21.15.83:0/3863886895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44100080 0x7f5a44070940 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3863886895 >> 172.21.15.83:0/3863886895 conn(0x7f5a440fb610 msgr2=0x7f5a44104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3863886895 shutdown_connections 2024-03-11T19:50:34.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.233+0000 7f5a4b7e2640 1 -- 172.21.15.83:0/3863886895 wait complete. 2024-03-11T19:50:34.251 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:33 smithi083 ceph-mon[31864]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 581 op/s 2024-03-11T19:50:34.252 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:50:34.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:50:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.434+0000 7f2437f63640 1 -- 172.21.15.83:0/1723089944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24301008e0 msgr2=0x7f2430102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.434+0000 7f2437f63640 1 --2- 172.21.15.83:0/1723089944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24301008e0 0x7f2430102cd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f242c009900 tx=0x7f242c02bcd0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.434+0000 7f2437f63640 1 -- 172.21.15.83:0/1723089944 shutdown_connections 2024-03-11T19:50:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.434+0000 7f2437f63640 1 --2- 172.21.15.83:0/1723089944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2430105d10 0x7f24301081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.434+0000 7f2437f63640 1 --2- 172.21.15.83:0/1723089944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2430103210 0x7f2430105680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.434+0000 7f2437f63640 1 --2- 172.21.15.83:0/1723089944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24301008e0 0x7f2430102cd0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.434+0000 7f2437f63640 1 -- 172.21.15.83:0/1723089944 >> 172.21.15.83:0/1723089944 conn(0x7f24300fa940 msgr2=0x7f24300fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.434+0000 7f2437f63640 1 -- 172.21.15.83:0/1723089944 shutdown_connections 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.434+0000 7f2437f63640 1 -- 172.21.15.83:0/1723089944 wait complete. 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.434+0000 7f2437f63640 1 Processor -- start 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.434+0000 7f2437f63640 1 -- start start 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f2437f63640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24301008e0 0x7f2430195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f2437f63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2430103210 0x7f2430196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f2437f63640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2430105d10 0x7f243019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f2437f63640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243019da90 con 0x7f2430103210 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f2437f63640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243019dc00 con 0x7f24301008e0 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f2437f63640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243019ddc0 con 0x7f2430105d10 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f2435cff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24301008e0 0x7f2430195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f2435cff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24301008e0 0x7f2430195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:48868/0 (socket says 172.21.15.83:48868) 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f2435cff640 1 -- 172.21.15.83:0/2234082641 learned_addr learned my addr 172.21.15.83:0/2234082641 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f24354fe640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2430103210 0x7f2430196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f2436500640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2430105d10 0x7f243019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f24354fe640 1 -- 172.21.15.83:0/2234082641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2430105d10 msgr2=0x7f243019d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f24354fe640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2430105d10 0x7f243019d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f24354fe640 1 -- 172.21.15.83:0/2234082641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24301008e0 msgr2=0x7f2430195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f24354fe640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24301008e0 0x7f2430195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f24354fe640 1 -- 172.21.15.83:0/2234082641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c009560 con 0x7f2430103210 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f2435cff640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24301008e0 0x7f2430195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.435+0000 7f24354fe640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2430103210 0x7f2430196530 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f242000e9c0 tx=0x7f242000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.436+0000 7f2426ffd640 1 -- 172.21.15.83:0/2234082641 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242000ccc0 con 0x7f2430103210 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.436+0000 7f2426ffd640 1 -- 172.21.15.83:0/2234082641 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f242000ce20 con 0x7f2430103210 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.436+0000 7f2437f63640 1 -- 172.21.15.83:0/2234082641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243019e0a0 con 0x7f2430103210 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.436+0000 7f2426ffd640 1 -- 172.21.15.83:0/2234082641 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420018880 con 0x7f2430103210 2024-03-11T19:50:34.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.436+0000 7f2437f63640 1 -- 172.21.15.83:0/2234082641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243019e600 con 0x7f2430103210 2024-03-11T19:50:34.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.439+0000 7f243ac0b740 1 Processor -- start 2024-03-11T19:50:34.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.440+0000 7f243ac0b740 1 -- start start 2024-03-11T19:50:34.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.440+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2c610 0x564460b2ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.440+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2cf70 0x564460b3a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.440+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b3a610 0x564460b3ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.440+0000 7f243ac0b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564460ae8960 con 0x564460b2c610 2024-03-11T19:50:34.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.440+0000 7f243ac0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564460a9c310 con 0x564460b3a610 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.440+0000 7f243ac0b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564460a99840 con 0x564460b2cf70 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2436500640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2c610 0x564460b2ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2435cff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2cf70 0x564460b3a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2436500640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2c610 0x564460b2ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52388/0 (socket says 172.21.15.83:52388) 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2436500640 1 -- 172.21.15.83:0/2405946715 learned_addr learned my addr 172.21.15.83:0/2405946715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2435cff640 1 -- 172.21.15.83:0/2405946715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b3a610 msgr2=0x564460b3ca60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2435cff640 1 --2- 172.21.15.83:0/2405946715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b3a610 0x564460b3ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2435cff640 1 -- 172.21.15.83:0/2405946715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2c610 msgr2=0x564460b2ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2435cff640 1 --2- 172.21.15.83:0/2405946715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2c610 0x564460b2ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2435cff640 1 -- 172.21.15.83:0/2405946715 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564460a57060 con 0x564460b2cf70 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2436500640 1 --2- 172.21.15.83:0/2405946715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2c610 0x564460b2ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2426ffd640 1 -- 172.21.15.83:0/2234082641 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f24200189e0 con 0x7f2430103210 2024-03-11T19:50:34.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f2435cff640 1 --2- 172.21.15.83:0/2405946715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2cf70 0x564460b3a040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f242c009850 tx=0x7f242c004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa234599bad26694 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f241bfff640 1 -- 172.21.15.83:0/2405946715 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0042d0 con 0x564460b2cf70 2024-03-11T19:50:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f241bfff640 1 -- 172.21.15.83:0/2405946715 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f242c02cdd0 con 0x564460b2cf70 2024-03-11T19:50:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.441+0000 7f241bfff640 1 -- 172.21.15.83:0/2405946715 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0035f0 con 0x564460b2cf70 2024-03-11T19:50:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.442+0000 7f2426ffd640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f241c0a1ab0 0x7f241c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.442+0000 7f2426ffd640 1 -- 172.21.15.83:0/2234082641 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f2420014070 con 0x7f2430103210 2024-03-11T19:50:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.442+0000 7f2436500640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f241c0a1ab0 0x7f241c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.442+0000 7f243ac0b740 1 -- 172.21.15.83:0/2405946715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2cf70 msgr2=0x564460b3a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.442+0000 7f243ac0b740 1 --2- 172.21.15.83:0/2405946715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2cf70 0x564460b3a040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f242c009850 tx=0x7f242c004af0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.442+0000 7f243ac0b740 1 -- 172.21.15.83:0/2405946715 shutdown_connections 2024-03-11T19:50:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.442+0000 7f243ac0b740 1 --2- 172.21.15.83:0/2405946715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b3a610 0x564460b3ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.442+0000 7f243ac0b740 1 --2- 172.21.15.83:0/2405946715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2cf70 0x564460b3a040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.442+0000 7f243ac0b740 1 --2- 172.21.15.83:0/2405946715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2c610 0x564460b2ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.442+0000 7f243ac0b740 1 -- 172.21.15.83:0/2405946715 >> 172.21.15.83:0/2405946715 conn(0x564460b28e20 msgr2=0x564460b27370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.442+0000 7f2436500640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f241c0a1ab0 0x7f241c0a3f70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x564460b2d4f0 tx=0x7f2428009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.443+0000 7f243ac0b740 1 -- 172.21.15.83:0/2405946715 shutdown_connections 2024-03-11T19:50:34.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.443+0000 7f243ac0b740 1 -- 172.21.15.83:0/2405946715 wait complete. 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.443+0000 7f243ac0b740 1 -- start start 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.443+0000 7f243ac0b740 1 Processor -- start 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 0x564460b3ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2cf70 0x564460b3f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b3a610 0x564460b461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f243ac0b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564460a95020 con 0x564460b3a610 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f243ac0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564460768430 con 0x564460b2c610 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f243ac0b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564460724e00 con 0x564460b2cf70 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f243ac0b740 10 client.0 _ll_register_callbacks cb 0x564460a3ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f2435cff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 0x564460b3ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f2435cff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 0x564460b3ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:48884/0 (socket says 172.21.15.83:48884) 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f2435cff640 1 -- 172.21.15.83:0/414855774 learned_addr learned my addr 172.21.15.83:0/414855774 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f24354fe640 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2cf70 0x564460b3f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f2436500640 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b3a610 0x564460b461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f2435cff640 1 -- 172.21.15.83:0/414855774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2cf70 msgr2=0x564460b3f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f2435cff640 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2cf70 0x564460b3f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f2435cff640 1 -- 172.21.15.83:0/414855774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b3a610 msgr2=0x564460b461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f2435cff640 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b3a610 0x564460b461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f2435cff640 1 -- 172.21.15.83:0/414855774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2430108db0 con 0x564460b2c610 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f2435cff640 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 0x564460b3ec40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f242c02c2b0 tx=0x7f242c004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5c855b68fc7f1e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.444+0000 7f24197fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f24197fa640 1 -- 172.21.15.83:0/414855774 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c004df0 con 0x564460b2c610 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f24197fa640 1 -- 172.21.15.83:0/414855774 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f242c0035f0 con 0x564460b2c610 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f243ac0b740 1 -- 172.21.15.83:0/414855774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644607947e0 con 0x564460b2c610 2024-03-11T19:50:34.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f24197fa640 1 -- 172.21.15.83:0/414855774 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f242c0037b0 con 0x564460b2c610 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f243ac0b740 10 client.26930 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f243ac0b740 1 -- 172.21.15.83:0/414855774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564460a936a0 con 0x564460b2c610 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f24197fa640 1 -- 172.21.15.83:0/414855774 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5644607947e0 con 0x564460b2c610 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f24197fa640 1 -- 172.21.15.83:0/414855774 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f242c05a070 con 0x564460b2c610 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f243ac0b740 10 client.26930 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f243ac0b740 10 client.26930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f243ac0b740 20 client.26930 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f243ac0b740 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564460b4aae0 0x564460b4cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f243ac0b740 4 client.26930 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f243ac0b740 1 -- 172.21.15.83:0/414855774 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644607d1720 con 0x564460b4aae0 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.445+0000 7f24354fe640 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564460b4aae0 0x564460b4cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.446+0000 7f24354fe640 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564460b4aae0 0x564460b4cf00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b270111a6d284562 server_cookie=bb09c31c0fac8723 in_seq=0 out_seq=0 2024-03-11T19:50:34.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.446+0000 7f24197fa640 10 client.26930 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.446+0000 7f24197fa640 1 -- 172.21.15.83:0/414855774 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644607d1720 con 0x564460b4aae0 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.446+0000 7f24197fa640 10 client.26930 handle_command_reply: tid=0 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.446+0000 7f243ac0b740 1 client.26930 shutdown 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.446+0000 7f243ac0b740 1 -- 172.21.15.83:0/414855774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 msgr2=0x564460b3ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.446+0000 7f243ac0b740 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 0x564460b3ec40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f242c02c2b0 tx=0x7f242c004580 comp rx=0 tx=0).stop 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.447+0000 7f243ac0b740 1 -- 172.21.15.83:0/414855774 shutdown_connections 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.447+0000 7f243ac0b740 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564460b4aae0 0x564460b4cf00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.447+0000 7f243ac0b740 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b3a610 0x564460b461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.447+0000 7f24197fa640 0 client.26930 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.447+0000 7f243ac0b740 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2cf70 0x564460b3f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.447+0000 7f243ac0b740 1 --2- 172.21.15.83:0/414855774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 0x564460b3ec40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.447+0000 7f243ac0b740 1 -- 172.21.15.83:0/414855774 >> 172.21.15.83:0/414855774 conn(0x564460b28e20 msgr2=0x564460b26b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.447+0000 7f243ac0b740 1 -- 172.21.15.83:0/414855774 shutdown_connections 2024-03-11T19:50:34.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.447+0000 7f243ac0b740 1 -- 172.21.15.83:0/414855774 wait complete. 2024-03-11T19:50:34.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.447+0000 7f243ac0b740 20 client.26930 trim_cache size 0 max 16384 2024-03-11T19:50:34.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.463+0000 7f243ac0b740 1 Processor -- start 2024-03-11T19:50:34.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.464+0000 7f243ac0b740 1 -- start start 2024-03-11T19:50:34.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.464+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2c610 0x564460b2b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.464+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2cf70 0x564460b2be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.464+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b3a610 0x564460b46180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.464+0000 7f243ac0b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564460a1a130 con 0x564460b3a610 2024-03-11T19:50:34.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.464+0000 7f243ac0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644607947e0 con 0x564460b2cf70 2024-03-11T19:50:34.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.464+0000 7f243ac0b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564460b28c90 con 0x564460b2c610 2024-03-11T19:50:34.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.468+0000 7f2436500640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b3a610 0x564460b46180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.468+0000 7f2436500640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b3a610 0x564460b46180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52404/0 (socket says 172.21.15.83:52404) 2024-03-11T19:50:34.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.468+0000 7f2436500640 1 -- 172.21.15.83:0/4294351578 learned_addr learned my addr 172.21.15.83:0/4294351578 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.468+0000 7f24354fe640 1 --2- 172.21.15.83:0/4294351578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2cf70 0x564460b2be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.469+0000 7f2435cff640 1 --2- 172.21.15.83:0/4294351578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2c610 0x564460b2b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.469+0000 7f24354fe640 1 -- 172.21.15.83:0/4294351578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2c610 msgr2=0x564460b2b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.469+0000 7f24354fe640 1 --2- 172.21.15.83:0/4294351578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2c610 0x564460b2b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.469+0000 7f24354fe640 1 -- 172.21.15.83:0/4294351578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b3a610 msgr2=0x564460b46180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.469+0000 7f24354fe640 1 --2- 172.21.15.83:0/4294351578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b3a610 0x564460b46180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.469+0000 7f24354fe640 1 -- 172.21.15.83:0/4294351578 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564460a2fc20 con 0x564460b2cf70 2024-03-11T19:50:34.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.471+0000 7f24354fe640 1 --2- 172.21.15.83:0/4294351578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2cf70 0x564460b2be60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f24200cb200 tx=0x7f24200cd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=443624aeec6d3281 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.475+0000 7f241b7fe640 1 -- 172.21.15.83:0/4294351578 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420083ea0 con 0x564460b2cf70 2024-03-11T19:50:34.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.475+0000 7f241b7fe640 1 -- 172.21.15.83:0/4294351578 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24200a2dc0 con 0x564460b2cf70 2024-03-11T19:50:34.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.475+0000 7f241b7fe640 1 -- 172.21.15.83:0/4294351578 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24200ba0b0 con 0x564460b2cf70 2024-03-11T19:50:34.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.476+0000 7f243ac0b740 1 -- 172.21.15.83:0/4294351578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2cf70 msgr2=0x564460b2be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.477+0000 7f243ac0b740 1 --2- 172.21.15.83:0/4294351578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2cf70 0x564460b2be60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f24200cb200 tx=0x7f24200cd040 comp rx=0 tx=0).stop 2024-03-11T19:50:34.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.480+0000 7f243ac0b740 1 -- 172.21.15.83:0/4294351578 shutdown_connections 2024-03-11T19:50:34.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.480+0000 7f243ac0b740 1 --2- 172.21.15.83:0/4294351578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b3a610 0x564460b46180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.480+0000 7f243ac0b740 1 --2- 172.21.15.83:0/4294351578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2cf70 0x564460b2be60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.480+0000 7f243ac0b740 1 --2- 172.21.15.83:0/4294351578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b2c610 0x564460b2b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.480+0000 7f243ac0b740 1 -- 172.21.15.83:0/4294351578 >> 172.21.15.83:0/4294351578 conn(0x564460b28e20 msgr2=0x564460b2b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.485+0000 7f243ac0b740 1 -- 172.21.15.83:0/4294351578 shutdown_connections 2024-03-11T19:50:34.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.485+0000 7f243ac0b740 1 -- 172.21.15.83:0/4294351578 wait complete. 2024-03-11T19:50:34.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.485+0000 7f243ac0b740 1 -- start start 2024-03-11T19:50:34.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.486+0000 7f243ac0b740 1 Processor -- start 2024-03-11T19:50:34.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.486+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 0x564460b46050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.486+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2cf70 0x564460b42af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.487+0000 7f243ac0b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b3a610 0x564460b43030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.487+0000 7f243ac0b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564460b43640 con 0x564460b2cf70 2024-03-11T19:50:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.487+0000 7f243ac0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564460b317e0 con 0x564460b2c610 2024-03-11T19:50:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.487+0000 7f243ac0b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564460b319a0 con 0x564460b3a610 2024-03-11T19:50:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.487+0000 7f243ac0b740 10 client.0 _ll_register_callbacks cb 0x7f242c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.487+0000 7f24354fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2cf70 0x564460b42af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.487+0000 7f24354fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2cf70 0x564460b42af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52410/0 (socket says 172.21.15.83:52410) 2024-03-11T19:50:34.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.487+0000 7f24354fe640 1 -- 172.21.15.83:0/1339501967 learned_addr learned my addr 172.21.15.83:0/1339501967 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.488+0000 7f24354fe640 1 -- 172.21.15.83:0/1339501967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b3a610 msgr2=0x564460b43030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:50:34.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.488+0000 7f2435cff640 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 0x564460b46050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.488+0000 7f24354fe640 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b3a610 0x564460b43030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.488+0000 7f24354fe640 1 -- 172.21.15.83:0/1339501967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 msgr2=0x564460b46050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.488+0000 7f24354fe640 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 0x564460b46050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.488+0000 7f24354fe640 1 -- 172.21.15.83:0/1339501967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243019e600 con 0x564460b2cf70 2024-03-11T19:50:34.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.489+0000 7f24354fe640 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2cf70 0x564460b42af0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2420010e20 tx=0x7f242000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692f88d9bac68f82 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.489+0000 7f24197fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:50:34.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.489+0000 7f24197fa640 1 -- 172.21.15.83:0/1339501967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420083ea0 con 0x564460b2cf70 2024-03-11T19:50:34.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.489+0000 7f24197fa640 1 -- 172.21.15.83:0/1339501967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f24200ba040 con 0x564460b2cf70 2024-03-11T19:50:34.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.489+0000 7f243ac0b740 1 -- 172.21.15.83:0/1339501967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f242c02cdd0 con 0x564460b2cf70 2024-03-11T19:50:34.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.490+0000 7f24197fa640 1 -- 172.21.15.83:0/1339501967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f242007b080 con 0x564460b2cf70 2024-03-11T19:50:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.493+0000 7f243ac0b740 10 client.17472 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.493+0000 7f243ac0b740 1 -- 172.21.15.83:0/1339501967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564460a558c0 con 0x564460b2cf70 2024-03-11T19:50:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.495+0000 7f24197fa640 1 -- 172.21.15.83:0/1339501967 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f242c02cdd0 con 0x564460b2cf70 2024-03-11T19:50:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.496+0000 7f24197fa640 1 -- 172.21.15.83:0/1339501967 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2420082020 con 0x564460b2cf70 2024-03-11T19:50:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.496+0000 7f243ac0b740 10 client.17472 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.496+0000 7f243ac0b740 10 client.17472 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:50:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.496+0000 7f243ac0b740 20 client.17472 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.496+0000 7f243ac0b740 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564460b33060 0x564460b35480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.496+0000 7f243ac0b740 4 client.17472 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:50:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.496+0000 7f243ac0b740 1 -- 172.21.15.83:0/1339501967 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564460820890 con 0x564460b33060 2024-03-11T19:50:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.496+0000 7f2435cff640 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564460b33060 0x564460b35480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.497+0000 7f24197fa640 10 client.17472 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.497+0000 7f2435cff640 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564460b33060 0x564460b35480 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f216934007e41af5 server_cookie=f236cbc50e208088 in_seq=0 out_seq=0 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.497+0000 7f24197fa640 1 -- 172.21.15.83:0/1339501967 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564460820890 con 0x564460b33060 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.497+0000 7f24197fa640 10 client.17472 handle_command_reply: tid=0 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.497+0000 7f243ac0b740 1 client.17472 shutdown 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.497+0000 7f243ac0b740 1 -- 172.21.15.83:0/1339501967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2cf70 msgr2=0x564460b42af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.497+0000 7f243ac0b740 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2cf70 0x564460b42af0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2420010e20 tx=0x7f242000cf90 comp rx=0 tx=0).stop 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.497+0000 7f243ac0b740 1 -- 172.21.15.83:0/1339501967 shutdown_connections 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.497+0000 7f243ac0b740 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564460b33060 0x564460b35480 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.498+0000 7f243ac0b740 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564460b3a610 0x564460b43030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.498+0000 7f24197fa640 0 client.17472 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:50:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.498+0000 7f243ac0b740 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564460b2cf70 0x564460b42af0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.498+0000 7f243ac0b740 1 --2- 172.21.15.83:0/1339501967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564460b2c610 0x564460b46050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.498+0000 7f243ac0b740 1 -- 172.21.15.83:0/1339501967 >> 172.21.15.83:0/1339501967 conn(0x564460b28e20 msgr2=0x564460b262f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.498+0000 7f243ac0b740 1 -- 172.21.15.83:0/1339501967 shutdown_connections 2024-03-11T19:50:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.498+0000 7f243ac0b740 1 -- 172.21.15.83:0/1339501967 wait complete. 2024-03-11T19:50:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.498+0000 7f243ac0b740 20 client.17472 trim_cache size 0 max 16384 2024-03-11T19:50:34.505 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:50:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 -- 172.21.15.83:0/2234082641 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f241c0a1ab0 msgr2=0x7f241c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f241c0a1ab0 0x7f241c0a3f70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x564460b2d4f0 tx=0x7f2428009600 comp rx=0 tx=0).stop 2024-03-11T19:50:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 -- 172.21.15.83:0/2234082641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2430103210 msgr2=0x7f2430196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2430103210 0x7f2430196530 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f242000e9c0 tx=0x7f242000ee80 comp rx=0 tx=0).stop 2024-03-11T19:50:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 -- 172.21.15.83:0/2234082641 shutdown_connections 2024-03-11T19:50:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f241c0a1ab0 0x7f241c0a3f70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2430105d10 0x7f243019d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2430103210 0x7f2430196530 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 --2- 172.21.15.83:0/2234082641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24301008e0 0x7f2430195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 -- 172.21.15.83:0/2234082641 >> 172.21.15.83:0/2234082641 conn(0x7f24300fa940 msgr2=0x7f2430103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 -- 172.21.15.83:0/2234082641 shutdown_connections 2024-03-11T19:50:34.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.499+0000 7f2437f63640 1 -- 172.21.15.83:0/2234082641 wait complete. 2024-03-11T19:50:34.517 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:50:34.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.645+0000 7f9a957c9640 1 -- 172.21.15.83:0/690372332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a90108180 msgr2=0x7f9a9010a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.645+0000 7f9a957c9640 1 --2- 172.21.15.83:0/690372332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a90108180 0x7f9a9010a5b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009be0 tx=0x7f9a8802bef0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.645+0000 7f9a957c9640 1 -- 172.21.15.83:0/690372332 shutdown_connections 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.645+0000 7f9a957c9640 1 --2- 172.21.15.83:0/690372332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a9010d4a0 0x7f9a9010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.645+0000 7f9a957c9640 1 --2- 172.21.15.83:0/690372332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a9010aaf0 0x7f9a9010cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.645+0000 7f9a957c9640 1 --2- 172.21.15.83:0/690372332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a90108180 0x7f9a9010a5b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.645+0000 7f9a957c9640 1 -- 172.21.15.83:0/690372332 >> 172.21.15.83:0/690372332 conn(0x7f9a9006dfe0 msgr2=0x7f9a90070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.645+0000 7f9a957c9640 1 -- 172.21.15.83:0/690372332 shutdown_connections 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.645+0000 7f9a957c9640 1 -- 172.21.15.83:0/690372332 wait complete. 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a957c9640 1 Processor -- start 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a957c9640 1 -- start start 2024-03-11T19:50:34.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a957c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90108180 0x7f9a901a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a957c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a9010aaf0 0x7f9a901a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a957c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a9010d4a0 0x7f9a901ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a957c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a901adb60 con 0x7f9a90108180 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a957c9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a901adcd0 con 0x7f9a9010d4a0 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a957c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a901ade90 con 0x7f9a9010aaf0 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90108180 0x7f9a901a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a8e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a9010aaf0 0x7f9a901a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90108180 0x7f9a901a6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52426/0 (socket says 172.21.15.83:52426) 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a8e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a9010aaf0 0x7f9a901a65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47884/0 (socket says 172.21.15.83:47884) 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.646+0000 7f9a8effd640 1 -- 172.21.15.83:0/1239892296 learned_addr learned my addr 172.21.15.83:0/1239892296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a8f7fe640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a9010d4a0 0x7f9a901ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a8e7fc640 1 -- 172.21.15.83:0/1239892296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a9010d4a0 msgr2=0x7f9a901ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a8e7fc640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a9010d4a0 0x7f9a901ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a8e7fc640 1 -- 172.21.15.83:0/1239892296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90108180 msgr2=0x7f9a901a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a8e7fc640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90108180 0x7f9a901a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a8e7fc640 1 -- 172.21.15.83:0/1239892296 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a88009840 con 0x7f9a9010aaf0 2024-03-11T19:50:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a8e7fc640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a9010aaf0 0x7f9a901a65b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9a80005fd0 tx=0x7f9a8000dd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a77fff640 1 -- 172.21.15.83:0/1239892296 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80009c30 con 0x7f9a9010aaf0 2024-03-11T19:50:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a957c9640 1 -- 172.21.15.83:0/1239892296 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a901ae170 con 0x7f9a9010aaf0 2024-03-11T19:50:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a77fff640 1 -- 172.21.15.83:0/1239892296 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a80004510 con 0x7f9a9010aaf0 2024-03-11T19:50:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a77fff640 1 -- 172.21.15.83:0/1239892296 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80003e00 con 0x7f9a9010aaf0 2024-03-11T19:50:34.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.647+0000 7f9a957c9640 1 -- 172.21.15.83:0/1239892296 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a901ae680 con 0x7f9a9010aaf0 2024-03-11T19:50:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a983a0740 1 Processor -- start 2024-03-11T19:50:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a983a0740 1 -- start start 2024-03-11T19:50:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a77fff640 1 -- 172.21.15.83:0/1239892296 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9a80003f60 con 0x7f9a9010aaf0 2024-03-11T19:50:34.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395dfd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 0x561395e02760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395e02dd0 0x561395e05260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a983a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561395db9960 con 0x561395dfdc60 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a983a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561395d6d310 con 0x561395dfd270 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a983a0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561395d6a840 con 0x561395e02dd0 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 0x561395e02760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395dfd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395dfd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:48948/0 (socket says 172.21.15.83:48948) 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395e02dd0 0x561395e05260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 0x561395e02760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52434/0 (socket says 172.21.15.83:52434) 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8f7fe640 1 -- 172.21.15.83:0/3614981682 learned_addr learned my addr 172.21.15.83:0/3614981682 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8f7fe640 1 -- 172.21.15.83:0/3614981682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395e02dd0 msgr2=0x561395e05260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8f7fe640 1 --2- 172.21.15.83:0/3614981682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395e02dd0 0x561395e05260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8f7fe640 1 -- 172.21.15.83:0/3614981682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 msgr2=0x561395e02760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8f7fe640 1 --2- 172.21.15.83:0/3614981682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 0x561395e02760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8f7fe640 1 -- 172.21.15.83:0/3614981682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561395d28060 con 0x561395dfd270 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.650+0000 7f9a8effd640 1 --2- 172.21.15.83:0/3614981682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 0x561395e02760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.651+0000 7f9a77fff640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a780a1a30 0x7f9a780a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.651+0000 7f9a8f7fe640 1 --2- 172.21.15.83:0/3614981682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395dfd690 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009bc0 tx=0x7f9a84011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb7d41baaadb3319 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.651+0000 7f9a8effd640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a780a1a30 0x7f9a780a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.651+0000 7f9a77fff640 1 -- 172.21.15.83:0/1239892296 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9a800ccac0 con 0x7f9a9010aaf0 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.651+0000 7f9a757fa640 1 -- 172.21.15.83:0/3614981682 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84004230 con 0x561395dfd270 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.651+0000 7f9a757fa640 1 -- 172.21.15.83:0/3614981682 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a84009e30 con 0x561395dfd270 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.651+0000 7f9a757fa640 1 -- 172.21.15.83:0/3614981682 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84004e10 con 0x561395dfd270 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.652+0000 7f9a983a0740 1 -- 172.21.15.83:0/3614981682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 msgr2=0x561395dfd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.652+0000 7f9a8effd640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a780a1a30 0x7f9a780a3ef0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x561395dfe060 tx=0x7f9a880333e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.652+0000 7f9a983a0740 1 --2- 172.21.15.83:0/3614981682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395dfd690 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009bc0 tx=0x7f9a84011810 comp rx=0 tx=0).stop 2024-03-11T19:50:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.653+0000 7f9a983a0740 1 -- 172.21.15.83:0/3614981682 shutdown_connections 2024-03-11T19:50:34.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.653+0000 7f9a983a0740 1 --2- 172.21.15.83:0/3614981682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395e02dd0 0x561395e05260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.653+0000 7f9a983a0740 1 --2- 172.21.15.83:0/3614981682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 0x561395e02760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.653+0000 7f9a983a0740 1 --2- 172.21.15.83:0/3614981682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395dfd690 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.653+0000 7f9a983a0740 1 -- 172.21.15.83:0/3614981682 >> 172.21.15.83:0/3614981682 conn(0x561395df87e0 msgr2=0x561395df8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.654+0000 7f9a983a0740 1 -- 172.21.15.83:0/3614981682 shutdown_connections 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.654+0000 7f9a983a0740 1 -- 172.21.15.83:0/3614981682 wait complete. 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.654+0000 7f9a983a0740 1 -- start start 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a983a0740 1 Processor -- start 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395e074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 0x561395e07a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395e02dd0 0x561395e0ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a983a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561395d66020 con 0x561395e02dd0 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a983a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561395a39430 con 0x561395dfd270 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a983a0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613959f5e00 con 0x561395dfdc60 2024-03-11T19:50:34.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a983a0740 10 client.0 _ll_register_callbacks cb 0x561395d0dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395e074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395e074e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:48960/0 (socket says 172.21.15.83:48960) 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a8effd640 1 -- 172.21.15.83:0/1193991239 learned_addr learned my addr 172.21.15.83:0/1193991239 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a8f7fe640 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395e02dd0 0x561395e0ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.655+0000 7f9a8e7fc640 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 0x561395e07a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a8effd640 1 -- 172.21.15.83:0/1193991239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 msgr2=0x561395e07a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a8effd640 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 0x561395e07a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a8effd640 1 -- 172.21.15.83:0/1193991239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395e02dd0 msgr2=0x561395e0ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a8effd640 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395e02dd0 0x561395e0ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a8effd640 1 -- 172.21.15.83:0/1193991239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a84009870 con 0x561395dfd270 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a8effd640 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395e074e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9a8802c500 tx=0x7f9a88009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3bcec10016b21c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:50:34.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a72ff5640 1 -- 172.21.15.83:0/1193991239 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88004710 con 0x561395dfd270 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a72ff5640 1 -- 172.21.15.83:0/1193991239 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9a88034430 con 0x561395dfd270 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a983a0740 1 -- 172.21.15.83:0/1193991239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561395a657e0 con 0x561395dfd270 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a72ff5640 1 -- 172.21.15.83:0/1193991239 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a88004870 con 0x561395dfd270 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a983a0740 10 client.26945 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a983a0740 1 -- 172.21.15.83:0/1193991239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561395d646a0 con 0x561395dfd270 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.656+0000 7f9a72ff5640 1 -- 172.21.15.83:0/1193991239 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561395a657e0 con 0x561395dfd270 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.657+0000 7f9a72ff5640 1 -- 172.21.15.83:0/1193991239 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a88049800 con 0x561395dfd270 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.657+0000 7f9a983a0740 10 client.26945 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.657+0000 7f9a983a0740 10 client.26945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.657+0000 7f9a983a0740 20 client.26945 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.657+0000 7f9a983a0740 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x561395e356e0 0x561395e37b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.657+0000 7f9a983a0740 4 client.26945 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:34.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.657+0000 7f9a983a0740 1 -- 172.21.15.83:0/1193991239 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561395aa2720 con 0x561395e356e0 2024-03-11T19:50:34.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.657+0000 7f9a8e7fc640 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x561395e356e0 0x561395e37b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.657+0000 7f9a72ff5640 10 client.26945 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:34.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.657+0000 7f9a8e7fc640 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x561395e356e0 0x561395e37b00 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=decdc7a034a64aed server_cookie=38818deda1148e0f in_seq=0 out_seq=0 2024-03-11T19:50:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a72ff5640 1 -- 172.21.15.83:0/1193991239 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561395aa2720 con 0x561395e356e0 2024-03-11T19:50:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a72ff5640 10 client.26945 handle_command_reply: tid=0 2024-03-11T19:50:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a983a0740 1 client.26945 shutdown 2024-03-11T19:50:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a983a0740 1 -- 172.21.15.83:0/1193991239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 msgr2=0x561395e074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a983a0740 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395e074e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9a8802c500 tx=0x7f9a88009500 comp rx=0 tx=0).stop 2024-03-11T19:50:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a983a0740 1 -- 172.21.15.83:0/1193991239 shutdown_connections 2024-03-11T19:50:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a983a0740 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x561395e356e0 0x561395e37b00 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a983a0740 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395e02dd0 0x561395e0ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a983a0740 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 0x561395e07a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a983a0740 1 --2- 172.21.15.83:0/1193991239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395e074e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a983a0740 1 -- 172.21.15.83:0/1193991239 >> 172.21.15.83:0/1193991239 conn(0x561395df87e0 msgr2=0x561395df7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a72ff5640 0 client.26945 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a983a0740 1 -- 172.21.15.83:0/1193991239 shutdown_connections 2024-03-11T19:50:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.658+0000 7f9a983a0740 1 -- 172.21.15.83:0/1193991239 wait complete. 2024-03-11T19:50:34.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.659+0000 7f9a983a0740 20 client.26945 trim_cache size 0 max 16384 2024-03-11T19:50:34.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.689+0000 7f9a983a0740 1 Processor -- start 2024-03-11T19:50:34.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.689+0000 7f9a983a0740 1 -- start start 2024-03-11T19:50:34.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.689+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfd270 0x561395e3c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.689+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 0x561395e0ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.689+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395e02dd0 0x561395dff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.689+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfd270 0x561395e3c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.689+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfd270 0x561395e3c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47892/0 (socket says 172.21.15.83:47892) 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.689+0000 7f9a8effd640 1 -- 172.21.15.83:0/2224628726 learned_addr learned my addr 172.21.15.83:0/2224628726 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.689+0000 7f9a983a0740 1 -- 172.21.15.83:0/2224628726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561395ceb130 con 0x561395dfdc60 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.689+0000 7f9a983a0740 1 -- 172.21.15.83:0/2224628726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561395e02b30 con 0x561395e02dd0 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.689+0000 7f9a983a0740 1 -- 172.21.15.83:0/2224628726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561395df92d0 con 0x561395dfd270 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.690+0000 7f9a8f7fe640 1 --2- 172.21.15.83:0/2224628726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395e02dd0 0x561395dff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.690+0000 7f9a8e7fc640 1 --2- 172.21.15.83:0/2224628726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 0x561395e0ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.690+0000 7f9a8effd640 1 -- 172.21.15.83:0/2224628726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395e02dd0 msgr2=0x561395dff1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.690+0000 7f9a8effd640 1 --2- 172.21.15.83:0/2224628726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395e02dd0 0x561395dff1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.690+0000 7f9a8effd640 1 -- 172.21.15.83:0/2224628726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 msgr2=0x561395e0ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.690+0000 7f9a8effd640 1 --2- 172.21.15.83:0/2224628726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 0x561395e0ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.690+0000 7f9a8effd640 1 -- 172.21.15.83:0/2224628726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561395d00c20 con 0x561395dfd270 2024-03-11T19:50:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.698+0000 7f9a8effd640 1 --2- 172.21.15.83:0/2224628726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfd270 0x561395e3c4a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a88049b90 tx=0x7f9a88003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28f38ffbf9164026 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.704+0000 7f9a74ff9640 1 -- 172.21.15.83:0/2224628726 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88004610 con 0x561395dfd270 2024-03-11T19:50:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.704+0000 7f9a74ff9640 1 -- 172.21.15.83:0/2224628726 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a8802c890 con 0x561395dfd270 2024-03-11T19:50:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.704+0000 7f9a74ff9640 1 -- 172.21.15.83:0/2224628726 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88034b00 con 0x561395dfd270 2024-03-11T19:50:34.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.705+0000 7f9a983a0740 1 -- 172.21.15.83:0/2224628726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfd270 msgr2=0x561395e3c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.705+0000 7f9a983a0740 1 --2- 172.21.15.83:0/2224628726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfd270 0x561395e3c4a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a88049b90 tx=0x7f9a88003f80 comp rx=0 tx=0).stop 2024-03-11T19:50:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.707+0000 7f9a983a0740 1 -- 172.21.15.83:0/2224628726 shutdown_connections 2024-03-11T19:50:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.707+0000 7f9a983a0740 1 --2- 172.21.15.83:0/2224628726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395e02dd0 0x561395dff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.707+0000 7f9a983a0740 1 --2- 172.21.15.83:0/2224628726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395dfdc60 0x561395e0ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.707+0000 7f9a983a0740 1 --2- 172.21.15.83:0/2224628726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfd270 0x561395e3c4a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.707+0000 7f9a983a0740 1 -- 172.21.15.83:0/2224628726 >> 172.21.15.83:0/2224628726 conn(0x561395df87e0 msgr2=0x561395df8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.710+0000 7f9a983a0740 1 -- 172.21.15.83:0/2224628726 shutdown_connections 2024-03-11T19:50:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.710+0000 7f9a983a0740 1 -- 172.21.15.83:0/2224628726 wait complete. 2024-03-11T19:50:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.710+0000 7f9a983a0740 1 -- start start 2024-03-11T19:50:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.710+0000 7f9a983a0740 1 Processor -- start 2024-03-11T19:50:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395e0e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 0x561395e0e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a983a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395e02dd0 0x561395e0c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395e0e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395e0e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:48974/0 (socket says 172.21.15.83:48974) 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a8e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 0x561395e0e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a8e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 0x561395e0e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47898/0 (socket says 172.21.15.83:47898) 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a8e7fc640 1 -- 172.21.15.83:0/4164993982 learned_addr learned my addr 172.21.15.83:0/4164993982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a983a0740 1 -- 172.21.15.83:0/4164993982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561395e0cf70 con 0x561395e02dd0 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a983a0740 1 -- 172.21.15.83:0/4164993982 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561395e0d0e0 con 0x561395dfd270 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a983a0740 1 -- 172.21.15.83:0/4164993982 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561395e0d2a0 con 0x561395dfdc60 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a983a0740 10 client.0 _ll_register_callbacks cb 0x7f9a840097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a8f7fe640 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395e02dd0 0x561395e0c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a8e7fc640 1 -- 172.21.15.83:0/4164993982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 msgr2=0x561395e0e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a8e7fc640 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395e0e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a8e7fc640 1 -- 172.21.15.83:0/4164993982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395e02dd0 msgr2=0x561395e0c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a8e7fc640 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395e02dd0 0x561395e0c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.711+0000 7f9a8e7fc640 1 -- 172.21.15.83:0/4164993982 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561395d646a0 con 0x561395dfdc60 2024-03-11T19:50:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.714+0000 7f9a8e7fc640 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 0x561395e0e890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a80095f40 tx=0x7f9a800cce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fca7188489e0d38 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.714+0000 7f9a72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:50:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.718+0000 7f9a72ff5640 1 -- 172.21.15.83:0/4164993982 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80099620 con 0x561395dfdc60 2024-03-11T19:50:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.718+0000 7f9a72ff5640 1 -- 172.21.15.83:0/4164993982 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9a8000be30 con 0x561395dfdc60 2024-03-11T19:50:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.719+0000 7f9a983a0740 1 -- 172.21.15.83:0/4164993982 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a84009e30 con 0x561395dfdc60 2024-03-11T19:50:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.720+0000 7f9a72ff5640 1 -- 172.21.15.83:0/4164993982 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a80099440 con 0x561395dfdc60 2024-03-11T19:50:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.724+0000 7f9a983a0740 10 client.27049 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.724+0000 7f9a983a0740 1 -- 172.21.15.83:0/4164993982 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561395d268c0 con 0x561395dfdc60 2024-03-11T19:50:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.727+0000 7f9a72ff5640 1 -- 172.21.15.83:0/4164993982 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a84009e30 con 0x561395dfdc60 2024-03-11T19:50:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.727+0000 7f9a72ff5640 1 -- 172.21.15.83:0/4164993982 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a80084040 con 0x561395dfdc60 2024-03-11T19:50:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.727+0000 7f9a983a0740 10 client.27049 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.727+0000 7f9a983a0740 10 client.27049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.727+0000 7f9a983a0740 20 client.27049 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.727+0000 7f9a983a0740 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x561395e8d070 0x561395e8f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.727+0000 7f9a983a0740 4 client.27049 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:50:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.727+0000 7f9a983a0740 1 -- 172.21.15.83:0/4164993982 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561395af1890 con 0x561395e8d070 2024-03-11T19:50:34.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.728+0000 7f9a8effd640 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x561395e8d070 0x561395e8f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:34.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.731+0000 7f9a72ff5640 10 client.27049 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:34.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.732+0000 7f9a8effd640 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x561395e8d070 0x561395e8f490 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bcb688f5f4e689d server_cookie=f1b9238a1cb7adf9 in_seq=0 out_seq=0 2024-03-11T19:50:34.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.733+0000 7f9a72ff5640 1 -- 172.21.15.83:0/4164993982 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561395af1890 con 0x561395e8d070 2024-03-11T19:50:34.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.733+0000 7f9a72ff5640 10 client.27049 handle_command_reply: tid=0 2024-03-11T19:50:34.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.734+0000 7f9a983a0740 1 client.27049 shutdown 2024-03-11T19:50:34.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.734+0000 7f9a983a0740 1 -- 172.21.15.83:0/4164993982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 msgr2=0x561395e0e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.734+0000 7f9a983a0740 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 0x561395e0e890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a80095f40 tx=0x7f9a800cce70 comp rx=0 tx=0).stop 2024-03-11T19:50:34.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.735+0000 7f9a983a0740 1 -- 172.21.15.83:0/4164993982 shutdown_connections 2024-03-11T19:50:34.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.735+0000 7f9a983a0740 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x561395e8d070 0x561395e8f490 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.735+0000 7f9a983a0740 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561395e02dd0 0x561395e0c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.735+0000 7f9a983a0740 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561395dfdc60 0x561395e0e890 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.735+0000 7f9a983a0740 1 --2- 172.21.15.83:0/4164993982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561395dfd270 0x561395e0e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.735+0000 7f9a983a0740 1 -- 172.21.15.83:0/4164993982 >> 172.21.15.83:0/4164993982 conn(0x561395df87e0 msgr2=0x561395e3c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.735+0000 7f9a72ff5640 0 client.27049 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:34.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.735+0000 7f9a983a0740 1 -- 172.21.15.83:0/4164993982 shutdown_connections 2024-03-11T19:50:34.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.735+0000 7f9a983a0740 1 -- 172.21.15.83:0/4164993982 wait complete. 2024-03-11T19:50:34.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.735+0000 7f9a983a0740 20 client.27049 trim_cache size 0 max 16384 2024-03-11T19:50:34.741 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:50:34.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.737+0000 7f9a957c9640 1 -- 172.21.15.83:0/1239892296 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a780a1a30 msgr2=0x7f9a780a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.737+0000 7f9a957c9640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a780a1a30 0x7f9a780a3ef0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x561395dfe060 tx=0x7f9a880333e0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.737+0000 7f9a957c9640 1 -- 172.21.15.83:0/1239892296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a9010aaf0 msgr2=0x7f9a901a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:34.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.737+0000 7f9a957c9640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a9010aaf0 0x7f9a901a65b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9a80005fd0 tx=0x7f9a8000dd20 comp rx=0 tx=0).stop 2024-03-11T19:50:34.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.738+0000 7f9a957c9640 1 -- 172.21.15.83:0/1239892296 shutdown_connections 2024-03-11T19:50:34.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.738+0000 7f9a957c9640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a780a1a30 0x7f9a780a3ef0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.738+0000 7f9a957c9640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a9010d4a0 0x7f9a901ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.738+0000 7f9a957c9640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a9010aaf0 0x7f9a901a65b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.738+0000 7f9a957c9640 1 --2- 172.21.15.83:0/1239892296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a90108180 0x7f9a901a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:34.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.738+0000 7f9a957c9640 1 -- 172.21.15.83:0/1239892296 >> 172.21.15.83:0/1239892296 conn(0x7f9a9006dfe0 msgr2=0x7f9a9010c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:34.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.738+0000 7f9a957c9640 1 -- 172.21.15.83:0/1239892296 shutdown_connections 2024-03-11T19:50:34.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:34.738+0000 7f9a957c9640 1 -- 172.21.15.83:0/1239892296 wait complete. 2024-03-11T19:50:35.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:34 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1132576520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:35.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:34 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1132576520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:35.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:34 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1132576520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:50:35.764 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:50:35.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:50:35.765 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:50:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.912+0000 7f78e1d0c640 1 -- 172.21.15.83:0/1048241765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78dc1023a0 msgr2=0x7f78dc102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.912+0000 7f78e1d0c640 1 --2- 172.21.15.83:0/1048241765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78dc1023a0 0x7f78dc102820 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009900 tx=0x7f78d002bc30 comp rx=0 tx=0).stop 2024-03-11T19:50:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.912+0000 7f78e1d0c640 1 -- 172.21.15.83:0/1048241765 shutdown_connections 2024-03-11T19:50:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.912+0000 7f78e1d0c640 1 --2- 172.21.15.83:0/1048241765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78dc102d60 0x7f78dc1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.912+0000 7f78e1d0c640 1 --2- 172.21.15.83:0/1048241765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78dc1023a0 0x7f78dc102820 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.912+0000 7f78e1d0c640 1 --2- 172.21.15.83:0/1048241765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78dc1011b0 0x7f78dc1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.912+0000 7f78e1d0c640 1 -- 172.21.15.83:0/1048241765 >> 172.21.15.83:0/1048241765 conn(0x7f78dc0fc720 msgr2=0x7f78dc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:35.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.913+0000 7f78e1d0c640 1 -- 172.21.15.83:0/1048241765 shutdown_connections 2024-03-11T19:50:35.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.913+0000 7f78e1d0c640 1 -- 172.21.15.83:0/1048241765 wait complete. 2024-03-11T19:50:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.914+0000 7f78e1d0c640 1 Processor -- start 2024-03-11T19:50:35.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78e1d0c640 1 -- start start 2024-03-11T19:50:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78e1d0c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78dc1011b0 0x7f78dc197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78e1d0c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78dc1023a0 0x7f78dc1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78e1d0c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78dc102d60 0x7f78dc19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78e1d0c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc19ed50 con 0x7f78dc1011b0 2024-03-11T19:50:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78e1d0c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc19eec0 con 0x7f78dc1023a0 2024-03-11T19:50:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78dc1011b0 0x7f78dc197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78e1d0c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc19f080 con 0x7f78dc102d60 2024-03-11T19:50:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78dc1011b0 0x7f78dc197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52488/0 (socket says 172.21.15.83:52488) 2024-03-11T19:50:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78db7fe640 1 -- 172.21.15.83:0/3823954143 learned_addr learned my addr 172.21.15.83:0/3823954143 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78daffd640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78dc1023a0 0x7f78dc1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.915+0000 7f78dbfff640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78dc102d60 0x7f78dc19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78daffd640 1 -- 172.21.15.83:0/3823954143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78dc102d60 msgr2=0x7f78dc19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78daffd640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78dc102d60 0x7f78dc19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78daffd640 1 -- 172.21.15.83:0/3823954143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78dc1011b0 msgr2=0x7f78dc197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78daffd640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78dc1011b0 0x7f78dc197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78daffd640 1 -- 172.21.15.83:0/3823954143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0009560 con 0x7f78dc1023a0 2024-03-11T19:50:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78db7fe640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78dc1011b0 0x7f78dc197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78daffd640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78dc1023a0 0x7f78dc1977a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000c00 tx=0x7f78d0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78d8ff9640 1 -- 172.21.15.83:0/3823954143 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0004aa0 con 0x7f78dc1023a0 2024-03-11T19:50:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78e1d0c640 1 -- 172.21.15.83:0/3823954143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78dc19f300 con 0x7f78dc1023a0 2024-03-11T19:50:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78d8ff9640 1 -- 172.21.15.83:0/3823954143 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78d00052d0 con 0x7f78dc1023a0 2024-03-11T19:50:35.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78d8ff9640 1 -- 172.21.15.83:0/3823954143 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0003e80 con 0x7f78dc1023a0 2024-03-11T19:50:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.916+0000 7f78e1d0c640 1 -- 172.21.15.83:0/3823954143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78dc19f7e0 con 0x7f78dc1023a0 2024-03-11T19:50:35.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.918+0000 7f78d8ff9640 1 -- 172.21.15.83:0/3823954143 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f78d0005440 con 0x7f78dc1023a0 2024-03-11T19:50:35.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78e49c1740 1 Processor -- start 2024-03-11T19:50:35.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78e49c1740 1 -- start start 2024-03-11T19:50:35.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78d8ff9640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f78c40a1a30 0x7f78c40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bc8f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 0x55d92bc90a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bc91850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78e49c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92bbfef90 con 0x55d92bc8f3f0 2024-03-11T19:50:35.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78e49c1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92bbfc4c0 con 0x55d92bc90610 2024-03-11T19:50:35.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78e49c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92bbf7ca0 con 0x55d92bc90fb0 2024-03-11T19:50:35.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78d8ff9640 1 -- 172.21.15.83:0/3823954143 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f78d00ef7d0 con 0x7f78dc1023a0 2024-03-11T19:50:35.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78dbfff640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f78c40a1a30 0x7f78c40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bc8f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bc91850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78daffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 0x55d92bc90a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bc8f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52498/0 (socket says 172.21.15.83:52498) 2024-03-11T19:50:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bc91850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47918/0 (socket says 172.21.15.83:47918) 2024-03-11T19:50:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78db7fe640 1 -- 172.21.15.83:0/1899378228 learned_addr learned my addr 172.21.15.83:0/1899378228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.919+0000 7f78daffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 0x55d92bc90a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:48998/0 (socket says 172.21.15.83:48998) 2024-03-11T19:50:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78daffd640 1 -- 172.21.15.83:0/1899378228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 msgr2=0x55d92bc91850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78daffd640 1 --2- 172.21.15.83:0/1899378228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bc91850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78daffd640 1 -- 172.21.15.83:0/1899378228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 msgr2=0x55d92bc8f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78daffd640 1 --2- 172.21.15.83:0/1899378228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bc8f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78daffd640 1 -- 172.21.15.83:0/1899378228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d92bbba0b0 con 0x55d92bc90610 2024-03-11T19:50:35.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78db7fe640 1 --2- 172.21.15.83:0/1899378228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bc8f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78dbfff640 1 --2- 172.21.15.83:0/1899378228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bc91850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78daffd640 1 --2- 172.21.15.83:0/1899378228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 0x55d92bc90a70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f78d00ed2b0 tx=0x7f78d0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc81c02782e21de server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78dbfff640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f78c40a1a30 0x7f78c40a3ef0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f78d4005fd0 tx=0x7f78d4009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78c1ffb640 1 -- 172.21.15.83:0/1899378228 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00bc750 con 0x55d92bc90610 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78c1ffb640 1 -- 172.21.15.83:0/1899378228 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78d00c08a0 con 0x55d92bc90610 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.920+0000 7f78c1ffb640 1 -- 172.21.15.83:0/1899378228 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00b8dc0 con 0x55d92bc90610 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.921+0000 7f78e49c1740 1 -- 172.21.15.83:0/1899378228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 msgr2=0x55d92bc90a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.921+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1899378228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 0x55d92bc90a70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f78d00ed2b0 tx=0x7f78d0005fd0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.921+0000 7f78e49c1740 1 -- 172.21.15.83:0/1899378228 shutdown_connections 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.921+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1899378228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bc91850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.921+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1899378228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 0x55d92bc90a70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.921+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1899378228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bc8f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.921+0000 7f78e49c1740 1 -- 172.21.15.83:0/1899378228 >> 172.21.15.83:0/1899378228 conn(0x55d92bc8a6a0 msgr2=0x55d92bc8aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.921+0000 7f78e49c1740 1 -- 172.21.15.83:0/1899378228 shutdown_connections 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.921+0000 7f78e49c1740 1 -- 172.21.15.83:0/1899378228 wait complete. 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.921+0000 7f78e49c1740 1 -- start start 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78e49c1740 1 Processor -- start 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc8f3f0 0x55d92bca1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78e49c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92bb9cf40 con 0x55d92bc90610 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78e49c1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92b8cb4b0 con 0x55d92bc8f3f0 2024-03-11T19:50:35.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52500/0 (socket says 172.21.15.83:52500) 2024-03-11T19:50:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78daffd640 1 -- 172.21.15.83:0/3444760166 learned_addr learned my addr 172.21.15.83:0/3444760166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc8f3f0 0x55d92bca1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78e49c1740 1 -- 172.21.15.83:0/3444760166 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92bb7d670 con 0x55d92bc90fb0 2024-03-11T19:50:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78dbfff640 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78e49c1740 10 client.0 _ll_register_callbacks cb 0x55d92bbeaf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78daffd640 1 -- 172.21.15.83:0/3444760166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 msgr2=0x55d92bca9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78daffd640 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78daffd640 1 -- 172.21.15.83:0/3444760166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc8f3f0 msgr2=0x55d92bca1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78daffd640 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc8f3f0 0x55d92bca1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78daffd640 1 -- 172.21.15.83:0/3444760166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78dc19f7e0 con 0x55d92bc90610 2024-03-11T19:50:35.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.922+0000 7f78db7fe640 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc8f3f0 0x55d92bca1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:50:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.923+0000 7f78daffd640 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca2230 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f78d00b8bb0 tx=0x7f78d00b8ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fceca084b28e4b05 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.923+0000 7f78baffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:50:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.923+0000 7f78baffd640 1 -- 172.21.15.83:0/3444760166 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00dc310 con 0x55d92bc90610 2024-03-11T19:50:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.923+0000 7f78baffd640 1 -- 172.21.15.83:0/3444760166 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f78d00c4ef0 con 0x55d92bc90610 2024-03-11T19:50:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.923+0000 7f78e49c1740 1 -- 172.21.15.83:0/3444760166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d92bbd1f40 con 0x55d92bc90610 2024-03-11T19:50:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.923+0000 7f78baffd640 1 -- 172.21.15.83:0/3444760166 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78d00c08a0 con 0x55d92bc90610 2024-03-11T19:50:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.923+0000 7f78e49c1740 10 client.17502 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.923+0000 7f78e49c1740 1 -- 172.21.15.83:0/3444760166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d92bbf6320 con 0x55d92bc90610 2024-03-11T19:50:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.924+0000 7f78baffd640 1 -- 172.21.15.83:0/3444760166 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d92bbd1f40 con 0x55d92bc90610 2024-03-11T19:50:35.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.924+0000 7f78baffd640 1 -- 172.21.15.83:0/3444760166 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f78d00b3030 con 0x55d92bc90610 2024-03-11T19:50:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.925+0000 7f78e49c1740 10 client.17502 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.925+0000 7f78e49c1740 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.925+0000 7f78e49c1740 20 client.17502 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.925+0000 7f78e49c1740 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d92bcad9d0 0x55d92bcafdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.926+0000 7f78e49c1740 4 client.17502 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.926+0000 7f78e49c1740 1 -- 172.21.15.83:0/3444760166 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d92bbee310 con 0x55d92bcad9d0 2024-03-11T19:50:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.926+0000 7f78db7fe640 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d92bcad9d0 0x55d92bcafdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.926+0000 7f78db7fe640 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d92bcad9d0 0x55d92bcafdc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eefeb5396c8608 server_cookie=7c0dc66868849b33 in_seq=0 out_seq=0 2024-03-11T19:50:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.927+0000 7f78baffd640 10 client.17502 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.927+0000 7f78baffd640 1 -- 172.21.15.83:0/3444760166 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d92bbee310 con 0x55d92bcad9d0 2024-03-11T19:50:35.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.927+0000 7f78baffd640 10 client.17502 handle_command_reply: tid=0 2024-03-11T19:50:35.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.930+0000 7f78e49c1740 1 client.17502 shutdown 2024-03-11T19:50:35.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.934+0000 7f78e49c1740 1 -- 172.21.15.83:0/3444760166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 msgr2=0x55d92bca2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:35.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.934+0000 7f78e49c1740 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca2230 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f78d00b8bb0 tx=0x7f78d00b8ff0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.940+0000 7f78e49c1740 1 -- 172.21.15.83:0/3444760166 shutdown_connections 2024-03-11T19:50:35.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.940+0000 7f78e49c1740 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d92bcad9d0 0x55d92bcafdc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.940+0000 7f78e49c1740 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.940+0000 7f78e49c1740 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca2230 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.940+0000 7f78e49c1740 1 --2- 172.21.15.83:0/3444760166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc8f3f0 0x55d92bca1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.940+0000 7f78e49c1740 1 -- 172.21.15.83:0/3444760166 >> 172.21.15.83:0/3444760166 conn(0x55d92bc8a6a0 msgr2=0x55d92bc898e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:35.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.941+0000 7f78baffd640 0 client.17502 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:35.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.942+0000 7f78e49c1740 1 -- 172.21.15.83:0/3444760166 shutdown_connections 2024-03-11T19:50:35.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.942+0000 7f78e49c1740 1 -- 172.21.15.83:0/3444760166 wait complete. 2024-03-11T19:50:35.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.942+0000 7f78e49c1740 20 client.17502 trim_cache size 0 max 16384 2024-03-11T19:50:35.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.974+0000 7f78e49c1740 1 Processor -- start 2024-03-11T19:50:35.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.974+0000 7f78e49c1740 1 -- start start 2024-03-11T19:50:35.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.974+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bca0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.974+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 0x55d92bca9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.974+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.974+0000 7f78e49c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92bc7f350 con 0x55d92bc8f3f0 2024-03-11T19:50:35.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.974+0000 7f78e49c1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92bc90070 con 0x55d92bc90610 2024-03-11T19:50:35.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.974+0000 7f78e49c1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92bbee310 con 0x55d92bc90fb0 2024-03-11T19:50:35.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bca0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bca0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52516/0 (socket says 172.21.15.83:52516) 2024-03-11T19:50:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78db7fe640 1 -- 172.21.15.83:0/1705313777 learned_addr learned my addr 172.21.15.83:0/1705313777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78daffd640 1 --2- 172.21.15.83:0/1705313777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 0x55d92bca9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78dbfff640 1 --2- 172.21.15.83:0/1705313777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78db7fe640 1 -- 172.21.15.83:0/1705313777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 msgr2=0x55d92bca0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78db7fe640 1 --2- 172.21.15.83:0/1705313777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78db7fe640 1 -- 172.21.15.83:0/1705313777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 msgr2=0x55d92bca9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78db7fe640 1 --2- 172.21.15.83:0/1705313777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 0x55d92bca9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78db7fe640 1 -- 172.21.15.83:0/1705313777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d92bb92c00 con 0x55d92bc8f3f0 2024-03-11T19:50:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78dbfff640 1 --2- 172.21.15.83:0/1705313777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:50:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78db7fe640 1 --2- 172.21.15.83:0/1705313777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bca0350 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x55d92bcb0680 tx=0x7f78cc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f318231fa0a145 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:35.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78c17fa640 1 -- 172.21.15.83:0/1705313777 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc016040 con 0x55d92bc8f3f0 2024-03-11T19:50:35.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78c17fa640 1 -- 172.21.15.83:0/1705313777 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78cc017040 con 0x55d92bc8f3f0 2024-03-11T19:50:35.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.975+0000 7f78c17fa640 1 -- 172.21.15.83:0/1705313777 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc004a90 con 0x55d92bc8f3f0 2024-03-11T19:50:35.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.976+0000 7f78e49c1740 1 -- 172.21.15.83:0/1705313777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 msgr2=0x55d92bca0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:35.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.976+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1705313777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bca0350 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x55d92bcb0680 tx=0x7f78cc00b410 comp rx=0 tx=0).stop 2024-03-11T19:50:35.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.976+0000 7f78e49c1740 1 -- 172.21.15.83:0/1705313777 shutdown_connections 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.976+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1705313777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.976+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1705313777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc90610 0x55d92bca9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.976+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1705313777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc8f3f0 0x55d92bca0350 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.976+0000 7f78e49c1740 1 -- 172.21.15.83:0/1705313777 >> 172.21.15.83:0/1705313777 conn(0x55d92bc8a6a0 msgr2=0x55d92bc9cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.976+0000 7f78e49c1740 1 -- 172.21.15.83:0/1705313777 shutdown_connections 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.976+0000 7f78e49c1740 1 -- 172.21.15.83:0/1705313777 wait complete. 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78e49c1740 1 -- start start 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78e49c1740 1 Processor -- start 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc8f3f0 0x55d92bca9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78e49c1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78e49c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92bca6850 con 0x55d92bc90610 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78daffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52526/0 (socket says 172.21.15.83:52526) 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78e49c1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92bd03ed0 con 0x55d92bc8f3f0 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78daffd640 1 -- 172.21.15.83:0/1341053700 learned_addr learned my addr 172.21.15.83:0/1341053700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78e49c1740 1 -- 172.21.15.83:0/1341053700 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d92bd04090 con 0x55d92bc90fb0 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78e49c1740 10 client.0 _ll_register_callbacks cb 0x7f78d0005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.977+0000 7f78db7fe640 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc8f3f0 0x55d92bca9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78dbfff640 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78daffd640 1 -- 172.21.15.83:0/1341053700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 msgr2=0x55d92bca6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78daffd640 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78daffd640 1 -- 172.21.15.83:0/1341053700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc8f3f0 msgr2=0x55d92bca9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78daffd640 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc8f3f0 0x55d92bca9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78daffd640 1 -- 172.21.15.83:0/1341053700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc00bae0 con 0x55d92bc90610 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78daffd640 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca5bf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f78d00bc450 tx=0x7f78d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db37c9dbea6f5ef2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78baffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78baffd640 1 -- 172.21.15.83:0/1341053700 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00dc2b0 con 0x55d92bc90610 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78baffd640 1 -- 172.21.15.83:0/1341053700 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f78d00baa50 con 0x55d92bc90610 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78e49c1740 1 -- 172.21.15.83:0/1341053700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d92bc1d900 con 0x55d92bc90610 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78baffd640 1 -- 172.21.15.83:0/1341053700 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78d00c08a0 con 0x55d92bc90610 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78e49c1740 10 client.17508 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78e49c1740 1 -- 172.21.15.83:0/1341053700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d92bbb8910 con 0x55d92bc90610 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.978+0000 7f78baffd640 1 -- 172.21.15.83:0/1341053700 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78d00a1080 con 0x55d92bc90610 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.979+0000 7f78baffd640 1 -- 172.21.15.83:0/1341053700 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f78d003c070 con 0x55d92bc90610 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.979+0000 7f78e49c1740 10 client.17508 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.979+0000 7f78e49c1740 10 client.17508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.979+0000 7f78e49c1740 20 client.17508 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.979+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d92bd055d0 0x55d92bd079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.979+0000 7f78e49c1740 4 client.17508 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.979+0000 7f78e49c1740 1 -- 172.21.15.83:0/1341053700 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d92b8f7860 con 0x55d92bd055d0 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.979+0000 7f78db7fe640 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d92bd055d0 0x55d92bd079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.979+0000 7f78db7fe640 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d92bd055d0 0x55d92bd079f0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3176250d411d5470 server_cookie=3d366cb88b5eda91 in_seq=0 out_seq=0 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.979+0000 7f78baffd640 10 client.17508 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.981+0000 7f78baffd640 1 -- 172.21.15.83:0/1341053700 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d92b8f7860 con 0x55d92bd055d0 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.981+0000 7f78baffd640 10 client.17508 handle_command_reply: tid=0 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.981+0000 7f78e49c1740 1 client.17508 shutdown 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.981+0000 7f78e49c1740 1 -- 172.21.15.83:0/1341053700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 msgr2=0x55d92bca5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.981+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca5bf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f78d00bc450 tx=0x7f78d0005fd0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.983+0000 7f78e49c1740 1 -- 172.21.15.83:0/1341053700 shutdown_connections 2024-03-11T19:50:35.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.983+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d92bd055d0 0x55d92bd079f0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.983+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d92bc90fb0 0x55d92bca6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.983+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d92bc90610 0x55d92bca5bf0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.983+0000 7f78e49c1740 1 --2- 172.21.15.83:0/1341053700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d92bc8f3f0 0x55d92bca9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.983+0000 7f78e49c1740 1 -- 172.21.15.83:0/1341053700 >> 172.21.15.83:0/1341053700 conn(0x55d92bc8a6a0 msgr2=0x55d92bca02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.984+0000 7f78baffd640 0 client.17508 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.984+0000 7f78e49c1740 1 -- 172.21.15.83:0/1341053700 shutdown_connections 2024-03-11T19:50:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.984+0000 7f78e49c1740 1 -- 172.21.15.83:0/1341053700 wait complete. 2024-03-11T19:50:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.984+0000 7f78e49c1740 20 client.17508 trim_cache size 0 max 16384 2024-03-11T19:50:35.992 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:50:35.993 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:50:35.993 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "bbec0d3c-4b9b-46d4-9f7d-b41b8414762d", 2024-03-11T19:50:35.993 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:50:35.993 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:50:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.984+0000 7f78e1d0c640 1 -- 172.21.15.83:0/3823954143 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f78c40a1a30 msgr2=0x7f78c40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.984+0000 7f78e1d0c640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f78c40a1a30 0x7f78c40a3ef0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f78d4005fd0 tx=0x7f78d4009f10 comp rx=0 tx=0).stop 2024-03-11T19:50:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.985+0000 7f78e1d0c640 1 -- 172.21.15.83:0/3823954143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78dc1023a0 msgr2=0x7f78dc1977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.985+0000 7f78e1d0c640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78dc1023a0 0x7f78dc1977a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000c00 tx=0x7f78d0004270 comp rx=0 tx=0).stop 2024-03-11T19:50:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.985+0000 7f78e1d0c640 1 -- 172.21.15.83:0/3823954143 shutdown_connections 2024-03-11T19:50:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.985+0000 7f78e1d0c640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f78c40a1a30 0x7f78c40a3ef0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.985+0000 7f78e1d0c640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78dc102d60 0x7f78dc19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.985+0000 7f78e1d0c640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78dc1023a0 0x7f78dc1977a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.985+0000 7f78e1d0c640 1 --2- 172.21.15.83:0/3823954143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78dc1011b0 0x7f78dc197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.985+0000 7f78e1d0c640 1 -- 172.21.15.83:0/3823954143 >> 172.21.15.83:0/3823954143 conn(0x7f78dc0fc720 msgr2=0x7f78dc105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:35.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.985+0000 7f78e1d0c640 1 -- 172.21.15.83:0/3823954143 shutdown_connections 2024-03-11T19:50:35.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:35.985+0000 7f78e1d0c640 1 -- 172.21.15.83:0/3823954143 wait complete. 2024-03-11T19:50:36.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:50:36.003 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:50:36.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:35 smithi083 ceph-mon[31864]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 778 op/s 2024-03-11T19:50:36.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:35 smithi137 ceph-mon[36220]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 778 op/s 2024-03-11T19:50:36.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:35 smithi204 ceph-mon[36739]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.8 MiB/s wr, 778 op/s 2024-03-11T19:50:36.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.161+0000 7f48a7881640 1 -- 172.21.15.83:0/1488997854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48a01077c0 msgr2=0x7f48a0107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.161+0000 7f48a7881640 1 --2- 172.21.15.83:0/1488997854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48a01077c0 0x7f48a0107be0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4898009ca0 tx=0x7f489802c140 comp rx=0 tx=0).stop 2024-03-11T19:50:36.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.162+0000 7f48a7881640 1 -- 172.21.15.83:0/1488997854 shutdown_connections 2024-03-11T19:50:36.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.162+0000 7f48a7881640 1 --2- 172.21.15.83:0/1488997854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48a0109370 0x7f48a010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.162+0000 7f48a7881640 1 --2- 172.21.15.83:0/1488997854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48a01089b0 0x7f48a0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.162+0000 7f48a7881640 1 --2- 172.21.15.83:0/1488997854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48a01077c0 0x7f48a0107be0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.162+0000 7f48a7881640 1 -- 172.21.15.83:0/1488997854 >> 172.21.15.83:0/1488997854 conn(0x7f48a0076770 msgr2=0x7f48a0078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:36.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.170+0000 7f48a7881640 1 -- 172.21.15.83:0/1488997854 shutdown_connections 2024-03-11T19:50:36.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.173+0000 7f48a7881640 1 -- 172.21.15.83:0/1488997854 wait complete. 2024-03-11T19:50:36.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.173+0000 7f48a7881640 1 Processor -- start 2024-03-11T19:50:36.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.174+0000 7f48a7881640 1 -- start start 2024-03-11T19:50:36.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.174+0000 7f48a7881640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48a01077c0 0x7f48a007fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.174+0000 7f48a7881640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48a01089b0 0x7f48a00801d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.175+0000 7f48a7881640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48a0109370 0x7f48a00811c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.175+0000 7f48a4e1c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48a01089b0 0x7f48a00801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.175+0000 7f48a4e1c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48a01089b0 0x7f48a00801d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47968/0 (socket says 172.21.15.83:47968) 2024-03-11T19:50:36.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.175+0000 7f48a4e1c640 1 -- 172.21.15.83:0/2998990675 learned_addr learned my addr 172.21.15.83:0/2998990675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.175+0000 7f48a7881640 1 -- 172.21.15.83:0/2998990675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0081700 con 0x7f48a0109370 2024-03-11T19:50:36.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.175+0000 7f48a561d640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48a01077c0 0x7f48a007fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.175+0000 7f48a7881640 1 -- 172.21.15.83:0/2998990675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0081870 con 0x7f48a01077c0 2024-03-11T19:50:36.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.175+0000 7f48a7881640 1 -- 172.21.15.83:0/2998990675 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0081a30 con 0x7f48a01089b0 2024-03-11T19:50:36.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.179+0000 7f48a5e1e640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48a0109370 0x7f48a00811c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.179+0000 7f48a4e1c640 1 -- 172.21.15.83:0/2998990675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48a01077c0 msgr2=0x7f48a007fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.179+0000 7f48a4e1c640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48a01077c0 0x7f48a007fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.179+0000 7f48a4e1c640 1 -- 172.21.15.83:0/2998990675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48a0109370 msgr2=0x7f48a00811c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.179+0000 7f48a4e1c640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48a0109370 0x7f48a00811c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.179+0000 7f48a4e1c640 1 -- 172.21.15.83:0/2998990675 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4898009900 con 0x7f48a01089b0 2024-03-11T19:50:36.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.179+0000 7f48a561d640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48a01077c0 0x7f48a007fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:50:36.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.179+0000 7f48a4e1c640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48a01089b0 0x7f48a00801d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f489000d950 tx=0x7f489000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:36.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.180+0000 7f48967fc640 1 -- 172.21.15.83:0/2998990675 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4890009c30 con 0x7f48a01089b0 2024-03-11T19:50:36.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.180+0000 7f48a7881640 1 -- 172.21.15.83:0/2998990675 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a0081d10 con 0x7f48a01089b0 2024-03-11T19:50:36.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.180+0000 7f48a7881640 1 -- 172.21.15.83:0/2998990675 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a0082040 con 0x7f48a01089b0 2024-03-11T19:50:36.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.180+0000 7f48967fc640 1 -- 172.21.15.83:0/2998990675 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4890009d90 con 0x7f48a01089b0 2024-03-11T19:50:36.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.182+0000 7f48967fc640 1 -- 172.21.15.83:0/2998990675 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000f7e0 con 0x7f48a01089b0 2024-03-11T19:50:36.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.188+0000 7f48aa452740 1 Processor -- start 2024-03-11T19:50:36.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.188+0000 7f48aa452740 1 -- start start 2024-03-11T19:50:36.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.195+0000 7f48aa452740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 0x5567e0da06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.195+0000 7f48aa452740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da0cc0 0x5567e0da57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.195+0000 7f48aa452740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da5e30 0x5567e0da82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.195+0000 7f48aa452740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e0d10280 con 0x5567e0da0cc0 2024-03-11T19:50:36.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.195+0000 7f48aa452740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e0d0d7b0 con 0x5567e0da5e30 2024-03-11T19:50:36.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.195+0000 7f48aa452740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e0d08f90 con 0x5567e0da02d0 2024-03-11T19:50:36.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.195+0000 7f48967fc640 1 -- 172.21.15.83:0/2998990675 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f489000f940 con 0x7f48a01089b0 2024-03-11T19:50:36.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.196+0000 7f48967fc640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f488c0a1c30 0x7f488c0a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.197+0000 7f48967fc640 1 -- 172.21.15.83:0/2998990675 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f48900cdb70 con 0x7f48a01089b0 2024-03-11T19:50:36.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.197+0000 7f48a561d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da0cc0 0x5567e0da57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.197+0000 7f48a4e1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da5e30 0x5567e0da82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.197+0000 7f48a4e1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da5e30 0x5567e0da82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:49064/0 (socket says 172.21.15.83:49064) 2024-03-11T19:50:36.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.197+0000 7f48a4e1c640 1 -- 172.21.15.83:0/93699878 learned_addr learned my addr 172.21.15.83:0/93699878 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:36.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.197+0000 7f48a5e1e640 1 --2- 172.21.15.83:0/93699878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 0x5567e0da06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.198+0000 7f48a5e1e640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f488c0a1c30 0x7f488c0a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.198+0000 7f48a561d640 1 -- 172.21.15.83:0/93699878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 msgr2=0x5567e0da06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:36.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.198+0000 7f48a561d640 1 --2- 172.21.15.83:0/93699878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 0x5567e0da06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.198+0000 7f48a561d640 1 -- 172.21.15.83:0/93699878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da5e30 msgr2=0x5567e0da82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:36.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.198+0000 7f48a561d640 1 --2- 172.21.15.83:0/93699878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da5e30 0x5567e0da82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.198+0000 7f48a561d640 1 -- 172.21.15.83:0/93699878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e0cc7570 con 0x5567e0da0cc0 2024-03-11T19:50:36.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.200+0000 7f48a561d640 1 --2- 172.21.15.83:0/93699878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da0cc0 0x5567e0da57c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f489802c750 tx=0x7f4898005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f083ab48e6b9a6f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:36.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.200+0000 7f488b7fe640 1 -- 172.21.15.83:0/93699878 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48980327b0 con 0x5567e0da0cc0 2024-03-11T19:50:36.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.200+0000 7f488b7fe640 1 -- 172.21.15.83:0/93699878 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4898032910 con 0x5567e0da0cc0 2024-03-11T19:50:36.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.201+0000 7f488b7fe640 1 -- 172.21.15.83:0/93699878 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898004040 con 0x5567e0da0cc0 2024-03-11T19:50:36.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.201+0000 7f48aa452740 1 -- 172.21.15.83:0/93699878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da0cc0 msgr2=0x5567e0da57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.201+0000 7f48aa452740 1 --2- 172.21.15.83:0/93699878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da0cc0 0x5567e0da57c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f489802c750 tx=0x7f4898005fa0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.202+0000 7f48a5e1e640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f488c0a1c30 0x7f488c0a40f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x5567e0da1520 tx=0x7f489c004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:36.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.202+0000 7f48aa452740 1 -- 172.21.15.83:0/93699878 shutdown_connections 2024-03-11T19:50:36.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.202+0000 7f48aa452740 1 --2- 172.21.15.83:0/93699878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da5e30 0x5567e0da82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.202+0000 7f48aa452740 1 --2- 172.21.15.83:0/93699878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da0cc0 0x5567e0da57c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.202+0000 7f48aa452740 1 --2- 172.21.15.83:0/93699878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 0x5567e0da06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.202+0000 7f48aa452740 1 -- 172.21.15.83:0/93699878 >> 172.21.15.83:0/93699878 conn(0x5567e0d9b7f0 msgr2=0x5567e0d9bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:36.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.203+0000 7f48aa452740 1 -- 172.21.15.83:0/93699878 shutdown_connections 2024-03-11T19:50:36.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.203+0000 7f48aa452740 1 -- 172.21.15.83:0/93699878 wait complete. 2024-03-11T19:50:36.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.203+0000 7f48aa452740 1 -- start start 2024-03-11T19:50:36.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.204+0000 7f48aa452740 1 Processor -- start 2024-03-11T19:50:36.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.204+0000 7f48aa452740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da02d0 0x5567e0daa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.204+0000 7f48aa452740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da0cc0 0x5567e0daa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.205+0000 7f48aa452740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da5e30 0x5567e0db1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.205+0000 7f48a4e1c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da0cc0 0x5567e0daa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.205+0000 7f48a4e1c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da0cc0 0x5567e0daa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47980/0 (socket says 172.21.15.83:47980) 2024-03-11T19:50:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.205+0000 7f48a5e1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da5e30 0x5567e0db1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.205+0000 7f48a5e1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da5e30 0x5567e0db1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52572/0 (socket says 172.21.15.83:52572) 2024-03-11T19:50:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.205+0000 7f48a4e1c640 1 -- 172.21.15.83:0/2042930988 learned_addr learned my addr 172.21.15.83:0/2042930988 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.205+0000 7f48aa452740 1 -- 172.21.15.83:0/2042930988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e09dc430 con 0x5567e0da5e30 2024-03-11T19:50:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.205+0000 7f48aa452740 1 -- 172.21.15.83:0/2042930988 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e0998e00 con 0x5567e0da02d0 2024-03-11T19:50:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.205+0000 7f48aa452740 1 -- 172.21.15.83:0/2042930988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e0c8dfd0 con 0x5567e0da0cc0 2024-03-11T19:50:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.205+0000 7f48aa452740 10 client.0 _ll_register_callbacks cb 0x5567e0ce3fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.206+0000 7f48a4e1c640 1 -- 172.21.15.83:0/2042930988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da02d0 msgr2=0x5567e0daa280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:50:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.206+0000 7f48a4e1c640 1 --2- 172.21.15.83:0/2042930988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da02d0 0x5567e0daa280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.206+0000 7f48a4e1c640 1 -- 172.21.15.83:0/2042930988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da5e30 msgr2=0x5567e0db1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:36.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.206+0000 7f48a4e1c640 1 --2- 172.21.15.83:0/2042930988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da5e30 0x5567e0db1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.206+0000 7f48a4e1c640 1 -- 172.21.15.83:0/2042930988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a0110500 con 0x5567e0da0cc0 2024-03-11T19:50:36.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.206+0000 7f48a4e1c640 1 --2- 172.21.15.83:0/2042930988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da0cc0 0x5567e0daa7c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f489009a7f0 tx=0x7f489009a540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34463b968ccbf5ec server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.206+0000 7f48aa452740 1 -- 172.21.15.83:0/2042930988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567e0a087e0 con 0x5567e0da0cc0 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.207+0000 7f4888ff9640 10 client.27079 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.207+0000 7f4888ff9640 1 -- 172.21.15.83:0/2042930988 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48900a3290 con 0x5567e0da0cc0 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.207+0000 7f4888ff9640 1 -- 172.21.15.83:0/2042930988 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f489000fbf0 con 0x5567e0da0cc0 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.207+0000 7f4888ff9640 1 -- 172.21.15.83:0/2042930988 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f489000fdb0 con 0x5567e0da0cc0 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.207+0000 7f48aa452740 10 client.27079 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.207+0000 7f48aa452740 1 -- 172.21.15.83:0/2042930988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x5567e0d07610 con 0x5567e0da0cc0 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.208+0000 7f4888ff9640 1 -- 172.21.15.83:0/2042930988 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4890099f40 con 0x5567e0da0cc0 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.208+0000 7f48aa452740 10 client.27079 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.208+0000 7f48aa452740 10 client.27079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.208+0000 7f48aa452740 20 client.27079 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.208+0000 7f48aa452740 1 --2- 172.21.15.83:0/2042930988 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5567e0db6180 0x5567e0db8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.208+0000 7f48aa452740 4 client.27079 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:50:36.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.208+0000 7f48aa452740 1 -- 172.21.15.83:0/2042930988 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567e0a087e0 con 0x5567e0db6180 2024-03-11T19:50:36.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.208+0000 7f48a561d640 1 --2- 172.21.15.83:0/2042930988 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5567e0db6180 0x5567e0db8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.213+0000 7f48a561d640 1 --2- 172.21.15.83:0/2042930988 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5567e0db6180 0x5567e0db8600 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72f732ccba75546e server_cookie=1b65c9ad81a07268 in_seq=0 out_seq=0 2024-03-11T19:50:36.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.214+0000 7f4888ff9640 10 client.27079 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f4888ff9640 1 -- 172.21.15.83:0/2042930988 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567e0a087e0 con 0x5567e0db6180 2024-03-11T19:50:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f4888ff9640 10 client.27079 handle_command_reply: tid=0 2024-03-11T19:50:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f48aa452740 1 client.27079 shutdown 2024-03-11T19:50:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f48aa452740 1 -- 172.21.15.83:0/2042930988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da0cc0 msgr2=0x5567e0daa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f48aa452740 1 --2- 172.21.15.83:0/2042930988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da0cc0 0x5567e0daa7c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f489009a7f0 tx=0x7f489009a540 comp rx=0 tx=0).stop 2024-03-11T19:50:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f48aa452740 1 -- 172.21.15.83:0/2042930988 shutdown_connections 2024-03-11T19:50:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f48aa452740 1 --2- 172.21.15.83:0/2042930988 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5567e0db6180 0x5567e0db8600 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f48aa452740 1 --2- 172.21.15.83:0/2042930988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da5e30 0x5567e0db1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f48aa452740 1 --2- 172.21.15.83:0/2042930988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da0cc0 0x5567e0daa7c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f48aa452740 1 --2- 172.21.15.83:0/2042930988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da02d0 0x5567e0daa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f48aa452740 1 -- 172.21.15.83:0/2042930988 >> 172.21.15.83:0/2042930988 conn(0x5567e0d9b7f0 msgr2=0x5567e0d9aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:36.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.215+0000 7f4888ff9640 0 client.27079 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:36.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.216+0000 7f48aa452740 1 -- 172.21.15.83:0/2042930988 shutdown_connections 2024-03-11T19:50:36.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.216+0000 7f48aa452740 1 -- 172.21.15.83:0/2042930988 wait complete. 2024-03-11T19:50:36.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.216+0000 7f48aa452740 20 client.27079 trim_cache size 0 max 16384 2024-03-11T19:50:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.240+0000 7f48aa452740 1 Processor -- start 2024-03-11T19:50:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.240+0000 7f48aa452740 1 -- start start 2024-03-11T19:50:36.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.240+0000 7f48aa452740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da02d0 0x5567e0da1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48aa452740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da0cc0 0x5567e0da2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48aa452740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da5e30 0x5567e0db17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48aa452740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e0d9ff60 con 0x5567e0da02d0 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48aa452740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e0da10f0 con 0x5567e0da0cc0 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48aa452740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e0db2cd0 con 0x5567e0da5e30 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48a561d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da02d0 0x5567e0da1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48a561d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da02d0 0x5567e0da1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52580/0 (socket says 172.21.15.83:52580) 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48a561d640 1 -- 172.21.15.83:0/162946182 learned_addr learned my addr 172.21.15.83:0/162946182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48a4e1c640 1 --2- 172.21.15.83:0/162946182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da0cc0 0x5567e0da2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48a5e1e640 1 --2- 172.21.15.83:0/162946182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da5e30 0x5567e0db17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48a561d640 1 -- 172.21.15.83:0/162946182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da5e30 msgr2=0x5567e0db17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:36.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48a561d640 1 --2- 172.21.15.83:0/162946182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da5e30 0x5567e0db17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48a561d640 1 -- 172.21.15.83:0/162946182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da0cc0 msgr2=0x5567e0da2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:36.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48a561d640 1 --2- 172.21.15.83:0/162946182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da0cc0 0x5567e0da2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48a561d640 1 -- 172.21.15.83:0/162946182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567e0ca3ac0 con 0x5567e0da02d0 2024-03-11T19:50:36.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.241+0000 7f48a561d640 1 --2- 172.21.15.83:0/162946182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da02d0 0x5567e0da1ec0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f489802c6d0 tx=0x7f4898005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33358cc0b4822a9e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:36.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.242+0000 7f488affd640 1 -- 172.21.15.83:0/162946182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489804f050 con 0x5567e0da02d0 2024-03-11T19:50:36.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.242+0000 7f488affd640 1 -- 172.21.15.83:0/162946182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4898009780 con 0x5567e0da02d0 2024-03-11T19:50:36.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.242+0000 7f488affd640 1 -- 172.21.15.83:0/162946182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48980525d0 con 0x5567e0da02d0 2024-03-11T19:50:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.242+0000 7f48aa452740 1 -- 172.21.15.83:0/162946182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da02d0 msgr2=0x5567e0da1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.242+0000 7f48aa452740 1 --2- 172.21.15.83:0/162946182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da02d0 0x5567e0da1ec0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f489802c6d0 tx=0x7f4898005fa0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.243+0000 7f48aa452740 1 -- 172.21.15.83:0/162946182 shutdown_connections 2024-03-11T19:50:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.243+0000 7f48aa452740 1 --2- 172.21.15.83:0/162946182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da5e30 0x5567e0db17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.243+0000 7f48aa452740 1 --2- 172.21.15.83:0/162946182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da0cc0 0x5567e0da2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.243+0000 7f48aa452740 1 --2- 172.21.15.83:0/162946182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da02d0 0x5567e0da1ec0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.243+0000 7f48aa452740 1 -- 172.21.15.83:0/162946182 >> 172.21.15.83:0/162946182 conn(0x5567e0d9b7f0 msgr2=0x5567e0daa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.243+0000 7f48aa452740 1 -- 172.21.15.83:0/162946182 shutdown_connections 2024-03-11T19:50:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.243+0000 7f48aa452740 1 -- 172.21.15.83:0/162946182 wait complete. 2024-03-11T19:50:36.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.244+0000 7f48aa452740 1 -- start start 2024-03-11T19:50:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.244+0000 7f48aa452740 1 Processor -- start 2024-03-11T19:50:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48aa452740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 0x5567e0daf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48aa452740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da0cc0 0x5567e0dafb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48a561d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 0x5567e0daf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48a561d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 0x5567e0daf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:47994/0 (socket says 172.21.15.83:47994) 2024-03-11T19:50:36.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48a561d640 1 -- 172.21.15.83:0/1229507226 learned_addr learned my addr 172.21.15.83:0/1229507226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:50:36.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48aa452740 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da5e30 0x5567e0db0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48aa452740 1 -- 172.21.15.83:0/1229507226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e0db1050 con 0x5567e0da5e30 2024-03-11T19:50:36.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48aa452740 1 -- 172.21.15.83:0/1229507226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e0db11c0 con 0x5567e0da0cc0 2024-03-11T19:50:36.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48aa452740 1 -- 172.21.15.83:0/1229507226 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567e0db1380 con 0x5567e0da02d0 2024-03-11T19:50:36.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48aa452740 10 client.0 _ll_register_callbacks cb 0x7f4898037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:50:36.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48a4e1c640 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da0cc0 0x5567e0dafb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.245+0000 7f48a5e1e640 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da5e30 0x5567e0db0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f48a561d640 1 -- 172.21.15.83:0/1229507226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da0cc0 msgr2=0x5567e0dafb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f48a561d640 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da0cc0 0x5567e0dafb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f48a561d640 1 -- 172.21.15.83:0/1229507226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da5e30 msgr2=0x5567e0db0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f48a561d640 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da5e30 0x5567e0db0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f48a561d640 1 -- 172.21.15.83:0/1229507226 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567e0cc7570 con 0x5567e0da02d0 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f48a561d640 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 0x5567e0daf5c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f48980325e0 tx=0x7f4898000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab75a05f4113b586 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f4888ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f4888ff9640 1 -- 172.21.15.83:0/1229507226 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898052350 con 0x5567e0da02d0 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f4888ff9640 1 -- 172.21.15.83:0/1229507226 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4898052570 con 0x5567e0da02d0 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f48aa452740 1 -- 172.21.15.83:0/1229507226 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567e0a45720 con 0x5567e0da02d0 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f4888ff9640 1 -- 172.21.15.83:0/1229507226 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4898052730 con 0x5567e0da02d0 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f48aa452740 10 client.27085 fetch_fsmap learned FSMap version 32 2024-03-11T19:50:36.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f48aa452740 1 -- 172.21.15.83:0/1229507226 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5567e0cc96a0 con 0x5567e0da02d0 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.246+0000 7f4888ff9640 1 -- 172.21.15.83:0/1229507226 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5567e0a45720 con 0x5567e0da02d0 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f4888ff9640 1 -- 172.21.15.83:0/1229507226 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4898052070 con 0x5567e0da02d0 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f48aa452740 10 client.27085 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f48aa452740 10 client.27085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f48aa452740 20 client.27085 populate_metadata read hostname 'smithi083' 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f48aa452740 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5567e0e0d290 0x5567e0e0f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f48aa452740 4 client.27085 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f48aa452740 1 -- 172.21.15.83:0/1229507226 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5567e0a94890 con 0x5567e0e0d290 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f48a4e1c640 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5567e0e0d290 0x5567e0e0f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f48a4e1c640 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5567e0e0d290 0x5567e0e0f6b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca369cb86637c31 server_cookie=5cde844336022bdf in_seq=0 out_seq=0 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f4888ff9640 10 client.27085 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f4888ff9640 1 -- 172.21.15.83:0/1229507226 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5567e0a94890 con 0x5567e0e0d290 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f4888ff9640 10 client.27085 handle_command_reply: tid=0 2024-03-11T19:50:36.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.247+0000 7f48aa452740 1 client.27085 shutdown 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f48aa452740 1 -- 172.21.15.83:0/1229507226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 msgr2=0x5567e0daf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f48aa452740 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 0x5567e0daf5c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f48980325e0 tx=0x7f4898000f40 comp rx=0 tx=0).stop 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f48aa452740 1 -- 172.21.15.83:0/1229507226 shutdown_connections 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f48aa452740 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5567e0e0d290 0x5567e0e0f6b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f48aa452740 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567e0da5e30 0x5567e0db0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f48aa452740 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567e0da0cc0 0x5567e0dafb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f4888ff9640 0 client.27085 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f48aa452740 1 --2- 172.21.15.83:0/1229507226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5567e0da02d0 0x5567e0daf5c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f48aa452740 1 -- 172.21.15.83:0/1229507226 >> 172.21.15.83:0/1229507226 conn(0x5567e0d9b7f0 msgr2=0x5567e0d9eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f48aa452740 1 -- 172.21.15.83:0/1229507226 shutdown_connections 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f48aa452740 1 -- 172.21.15.83:0/1229507226 wait complete. 2024-03-11T19:50:36.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.248+0000 7f48aa452740 20 client.27085 trim_cache size 0 max 16384 2024-03-11T19:50:36.258 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:50:36.258 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:50:36.258 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:50:36.258 INFO:teuthology.orchestra.run.smithi083.stdout: "bbec0d3c-4b9b-46d4-9f7d-b41b8414762d": { 2024-03-11T19:50:36.258 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:50:36.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "bbec0d3c-4b9b-46d4-9f7d-b41b8414762d", 2024-03-11T19:50:36.258 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:50:36.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:50:36.258 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:50:36.258 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:50:36.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 -- 172.21.15.83:0/2998990675 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f488c0a1c30 msgr2=0x7f488c0a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f488c0a1c30 0x7f488c0a40f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x5567e0da1520 tx=0x7f489c004730 comp rx=0 tx=0).stop 2024-03-11T19:50:36.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 -- 172.21.15.83:0/2998990675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48a01089b0 msgr2=0x7f48a00801d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:50:36.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48a01089b0 0x7f48a00801d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f489000d950 tx=0x7f489000de10 comp rx=0 tx=0).stop 2024-03-11T19:50:36.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 -- 172.21.15.83:0/2998990675 shutdown_connections 2024-03-11T19:50:36.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f488c0a1c30 0x7f488c0a40f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48a0109370 0x7f48a00811c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48a01089b0 0x7f48a00801d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 --2- 172.21.15.83:0/2998990675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48a01077c0 0x7f48a007fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:50:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 -- 172.21.15.83:0/2998990675 >> 172.21.15.83:0/2998990675 conn(0x7f48a0076770 msgr2=0x7f48a010bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:50:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 -- 172.21.15.83:0/2998990675 shutdown_connections 2024-03-11T19:50:36.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:50:36.249+0000 7f48a7881640 1 -- 172.21.15.83:0/2998990675 wait complete. 2024-03-11T19:50:36.268 INFO:tasks.cephfs.filesystem:scrub status for tag:bbec0d3c-4b9b-46d4-9f7d-b41b8414762d - {'path': '/', 'tag': 'bbec0d3c-4b9b-46d4-9f7d-b41b8414762d', 'options': 'recursive,force'} 2024-03-11T19:50:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:36 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:50:37.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:36 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:36 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:50:37.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:36 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:50:37.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:36 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:36 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:50:37.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:36 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:36 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:36 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:50:37.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:36 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:36 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:50:37.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:36 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:50:37.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:36 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:36 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:50:37.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:36 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:36 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:36 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:50:37.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:36 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:36 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:50:37.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:36 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:50:37.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:36 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:36 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:50:37.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:36 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:36 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:50:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 680.0 s, 1728.2 tps, lat 57.551 ms stddev 63.154 2024-03-11T19:50:38.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:37 smithi204 ceph-mon[36739]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 656 op/s 2024-03-11T19:50:38.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:37 smithi083 ceph-mon[31864]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 656 op/s 2024-03-11T19:50:38.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:37 smithi137 ceph-mon[36220]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 656 op/s 2024-03-11T19:50:39.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:38 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:50:39.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:38 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:50:39.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:38 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:50:40.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:40 smithi083 ceph-mon[31864]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 744 op/s 2024-03-11T19:50:40.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:40 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:50:40.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:40 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:50:40.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:40 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:50:40.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:40 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:50:40.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:40 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:50:40.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:40 smithi137 ceph-mon[36220]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 744 op/s 2024-03-11T19:50:40.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:40 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:50:40.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:40 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:50:40.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:40 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:50:40.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:40 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:50:40.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:40 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:50:40.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:40 smithi204 ceph-mon[36739]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 744 op/s 2024-03-11T19:50:40.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:40 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:50:40.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:40 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:50:40.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:40 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:50:40.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:40 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:50:40.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:40 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:50:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 685.0 s, 1338.8 tps, lat 74.131 ms stddev 85.085 2024-03-11T19:50:42.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:42 smithi083 ceph-mon[31864]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 MiB/s wr, 667 op/s 2024-03-11T19:50:42.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:42 smithi137 ceph-mon[36220]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 MiB/s wr, 667 op/s 2024-03-11T19:50:42.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:42 smithi204 ceph-mon[36739]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 MiB/s wr, 667 op/s 2024-03-11T19:50:44.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:44 smithi083 ceph-mon[31864]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 MiB/s wr, 598 op/s 2024-03-11T19:50:44.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:44 smithi137 ceph-mon[36220]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 MiB/s wr, 598 op/s 2024-03-11T19:50:44.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:44 smithi204 ceph-mon[36739]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 MiB/s wr, 598 op/s 2024-03-11T19:50:46.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:46 smithi083 ceph-mon[31864]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.8 MiB/s wr, 766 op/s 2024-03-11T19:50:46.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:46 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:50:46.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:46 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:50:46.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:46 smithi137 ceph-mon[36220]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.8 MiB/s wr, 766 op/s 2024-03-11T19:50:46.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:46 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:50:46.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:46 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:50:46.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:46 smithi204 ceph-mon[36739]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.8 MiB/s wr, 766 op/s 2024-03-11T19:50:46.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:46 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:50:46.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:46 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:50:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 690.0 s, 1787.6 tps, lat 56.162 ms stddev 58.656 2024-03-11T19:50:47.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:47 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:50:47.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:47 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:50:47.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:47 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:50:48.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:48 smithi083 ceph-mon[31864]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.6 MiB/s wr, 585 op/s 2024-03-11T19:50:48.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:48 smithi137 ceph-mon[36220]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.6 MiB/s wr, 585 op/s 2024-03-11T19:50:48.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:48 smithi204 ceph-mon[36739]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.6 MiB/s wr, 585 op/s 2024-03-11T19:50:50.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:50 smithi083 ceph-mon[31864]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.8 MiB/s wr, 657 op/s 2024-03-11T19:50:50.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:50 smithi137 ceph-mon[36220]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.8 MiB/s wr, 657 op/s 2024-03-11T19:50:50.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:50 smithi204 ceph-mon[36739]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.8 MiB/s wr, 657 op/s 2024-03-11T19:50:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 695.0 s, 1758.0 tps, lat 57.417 ms stddev 59.950 2024-03-11T19:50:52.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:52 smithi083 ceph-mon[31864]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-03-11T19:50:52.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:52 smithi137 ceph-mon[36220]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-03-11T19:50:52.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:52 smithi204 ceph-mon[36739]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.7 MiB/s wr, 605 op/s 2024-03-11T19:50:54.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:54 smithi083 ceph-mon[31864]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-03-11T19:50:54.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:54 smithi137 ceph-mon[36220]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-03-11T19:50:54.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:54 smithi204 ceph-mon[36739]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-03-11T19:50:56.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:56 smithi083 ceph-mon[31864]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 MiB/s wr, 683 op/s 2024-03-11T19:50:56.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:56 smithi137 ceph-mon[36220]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 MiB/s wr, 683 op/s 2024-03-11T19:50:56.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:56 smithi204 ceph-mon[36739]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 MiB/s wr, 683 op/s 2024-03-11T19:50:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 700.0 s, 1880.0 tps, lat 53.080 ms stddev 54.799 2024-03-11T19:50:58.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:50:58 smithi083 ceph-mon[31864]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.6 MiB/s wr, 534 op/s 2024-03-11T19:50:58.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:50:58 smithi137 ceph-mon[36220]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.6 MiB/s wr, 534 op/s 2024-03-11T19:50:58.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:50:58 smithi204 ceph-mon[36739]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.6 MiB/s wr, 534 op/s 2024-03-11T19:51:00.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:00 smithi083 ceph-mon[31864]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-03-11T19:51:00.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:00 smithi137 ceph-mon[36220]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-03-11T19:51:00.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:00 smithi204 ceph-mon[36739]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.4 MiB/s wr, 705 op/s 2024-03-11T19:51:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 705.0 s, 2114.0 tps, lat 47.305 ms stddev 54.182 2024-03-11T19:51:02.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:02 smithi083 ceph-mon[31864]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.7 MiB/s wr, 609 op/s 2024-03-11T19:51:02.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:02 smithi137 ceph-mon[36220]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.7 MiB/s wr, 609 op/s 2024-03-11T19:51:02.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:02 smithi204 ceph-mon[36739]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 6.7 MiB/s wr, 609 op/s 2024-03-11T19:51:04.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:04 smithi083 ceph-mon[31864]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2024-03-11T19:51:04.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:04 smithi137 ceph-mon[36220]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2024-03-11T19:51:04.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:04 smithi204 ceph-mon[36739]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2024-03-11T19:51:06.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:51:06.269 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:51:06.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.396+0000 7fab82ef4640 1 -- 172.21.15.83:0/3429219446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab7c0ff7f0 msgr2=0x7fab7c0ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.396+0000 7fab82ef4640 1 --2- 172.21.15.83:0/3429219446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab7c0ff7f0 0x7fab7c0ffc10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fab7000aa10 tx=0x7fab7002cf70 comp rx=0 tx=0).stop 2024-03-11T19:51:06.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.396+0000 7fab82ef4640 1 -- 172.21.15.83:0/3429219446 shutdown_connections 2024-03-11T19:51:06.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.396+0000 7fab82ef4640 1 --2- 172.21.15.83:0/3429219446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab7c1013a0 0x7fab7c107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.396+0000 7fab82ef4640 1 --2- 172.21.15.83:0/3429219446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab7c1009e0 0x7fab7c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.396+0000 7fab82ef4640 1 --2- 172.21.15.83:0/3429219446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab7c0ff7f0 0x7fab7c0ffc10 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.396+0000 7fab82ef4640 1 -- 172.21.15.83:0/3429219446 >> 172.21.15.83:0/3429219446 conn(0x7fab7c0fad60 msgr2=0x7fab7c0fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:06.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.396+0000 7fab82ef4640 1 -- 172.21.15.83:0/3429219446 shutdown_connections 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.396+0000 7fab82ef4640 1 -- 172.21.15.83:0/3429219446 wait complete. 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab82ef4640 1 Processor -- start 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab82ef4640 1 -- start start 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab82ef4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab7c0ff7f0 0x7fab7c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab82ef4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab7c1009e0 0x7fab7c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab82ef4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab7c1013a0 0x7fab7c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab82ef4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c19ee60 con 0x7fab7c1009e0 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab82ef4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c19efd0 con 0x7fab7c1013a0 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab82ef4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c19f190 con 0x7fab7c0ff7f0 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab7c1009e0 0x7fab7c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab7c1009e0 0x7fab7c1978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40230/0 (socket says 172.21.15.83:40230) 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab7bfff640 1 -- 172.21.15.83:0/2850700307 learned_addr learned my addr 172.21.15.83:0/2850700307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab80c90640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab7c0ff7f0 0x7fab7c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab81491640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab7c1013a0 0x7fab7c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab7bfff640 1 -- 172.21.15.83:0/2850700307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab7c0ff7f0 msgr2=0x7fab7c197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab7bfff640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab7c0ff7f0 0x7fab7c197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab7bfff640 1 -- 172.21.15.83:0/2850700307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab7c1013a0 msgr2=0x7fab7c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab7bfff640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab7c1013a0 0x7fab7c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab7bfff640 1 -- 172.21.15.83:0/2850700307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7000a670 con 0x7fab7c1009e0 2024-03-11T19:51:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.397+0000 7fab7bfff640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab7c1009e0 0x7fab7c1978b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fab6800d950 tx=0x7fab6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.398+0000 7fab79ffb640 1 -- 172.21.15.83:0/2850700307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68009c30 con 0x7fab7c1009e0 2024-03-11T19:51:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.398+0000 7fab82ef4640 1 -- 172.21.15.83:0/2850700307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7c19f470 con 0x7fab7c1009e0 2024-03-11T19:51:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.398+0000 7fab79ffb640 1 -- 172.21.15.83:0/2850700307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab68009d90 con 0x7fab7c1009e0 2024-03-11T19:51:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.398+0000 7fab79ffb640 1 -- 172.21.15.83:0/2850700307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6800f7e0 con 0x7fab7c1009e0 2024-03-11T19:51:06.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.398+0000 7fab82ef4640 1 -- 172.21.15.83:0/2850700307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7c19f9d0 con 0x7fab7c1009e0 2024-03-11T19:51:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.400+0000 7fab79ffb640 1 -- 172.21.15.83:0/2850700307 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fab68010ad0 con 0x7fab7c1009e0 2024-03-11T19:51:06.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.400+0000 7fab85ba1740 1 Processor -- start 2024-03-11T19:51:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.401+0000 7fab79ffb640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fab6c0a1a80 0x7fab6c0a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.401+0000 7fab79ffb640 1 -- 172.21.15.83:0/2850700307 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fab6809a010 con 0x7fab7c1009e0 2024-03-11T19:51:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.401+0000 7fab85ba1740 1 -- start start 2024-03-11T19:51:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.402+0000 7fab81491640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fab6c0a1a80 0x7fab6c0a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.402+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d3b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.402+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d3ba50 0x55c007d40030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.402+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 0x55c007d42b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.402+0000 7fab85ba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c007ca2280 con 0x55c007d406a0 2024-03-11T19:51:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.402+0000 7fab85ba1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c007c9f7b0 con 0x55c007d3ba50 2024-03-11T19:51:06.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.402+0000 7fab85ba1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c007c9af90 con 0x55c007d3b060 2024-03-11T19:51:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.402+0000 7fab80c90640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d3b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.402+0000 7fab80c90640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d3b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57570/0 (socket says 172.21.15.83:57570) 2024-03-11T19:51:06.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab80c90640 1 -- 172.21.15.83:0/1347201303 learned_addr learned my addr 172.21.15.83:0/1347201303 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab81491640 1 --2- 172.21.15.83:0/1347201303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 0x55c007d42b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab81491640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fab6c0a1a80 0x7fab6c0a3f40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fab74005fd0 tx=0x7fab74011810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab80c90640 1 -- 172.21.15.83:0/1347201303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d3ba50 msgr2=0x55c007d40030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab7bfff640 1 --2- 172.21.15.83:0/1347201303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d3ba50 0x55c007d40030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab80c90640 1 --2- 172.21.15.83:0/1347201303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d3ba50 0x55c007d40030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab80c90640 1 -- 172.21.15.83:0/1347201303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 msgr2=0x55c007d42b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab80c90640 1 --2- 172.21.15.83:0/1347201303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 0x55c007d42b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab80c90640 1 -- 172.21.15.83:0/1347201303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c007c59570 con 0x55c007d3b060 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab80c90640 1 --2- 172.21.15.83:0/1347201303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d3b480 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fab7000aa10 tx=0x7fab70005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15550218aada4fc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab66ffd640 1 -- 172.21.15.83:0/1347201303 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7002db20 con 0x55c007d3b060 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab66ffd640 1 -- 172.21.15.83:0/1347201303 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab7002dc80 con 0x55c007d3b060 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.403+0000 7fab66ffd640 1 -- 172.21.15.83:0/1347201303 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70035c50 con 0x55c007d3b060 2024-03-11T19:51:06.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.404+0000 7fab85ba1740 1 -- 172.21.15.83:0/1347201303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 msgr2=0x55c007d3b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.404+0000 7fab85ba1740 1 --2- 172.21.15.83:0/1347201303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d3b480 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fab7000aa10 tx=0x7fab70005c00 comp rx=0 tx=0).stop 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.404+0000 7fab85ba1740 1 -- 172.21.15.83:0/1347201303 shutdown_connections 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.404+0000 7fab85ba1740 1 --2- 172.21.15.83:0/1347201303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 0x55c007d42b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.404+0000 7fab85ba1740 1 --2- 172.21.15.83:0/1347201303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d3ba50 0x55c007d40030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.404+0000 7fab85ba1740 1 --2- 172.21.15.83:0/1347201303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d3b480 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.404+0000 7fab85ba1740 1 -- 172.21.15.83:0/1347201303 >> 172.21.15.83:0/1347201303 conn(0x55c007d2d770 msgr2=0x55c007d2db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.404+0000 7fab85ba1740 1 -- 172.21.15.83:0/1347201303 shutdown_connections 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.404+0000 7fab85ba1740 1 -- 172.21.15.83:0/1347201303 wait complete. 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab85ba1740 1 -- start start 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab85ba1740 1 Processor -- start 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3b060 0x55c007d449f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3ba50 0x55c007d44f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 0x55c007d4bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab85ba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c00796e430 con 0x55c007d3b060 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab85ba1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c00792ae00 con 0x55c007d406a0 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab85ba1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c007c1ffd0 con 0x55c007d3ba50 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab85ba1740 10 client.0 _ll_register_callbacks cb 0x55c007c75fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab80c90640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3b060 0x55c007d449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab80c90640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3b060 0x55c007d449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40242/0 (socket says 172.21.15.83:40242) 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab80c90640 1 -- 172.21.15.83:0/3896470968 learned_addr learned my addr 172.21.15.83:0/3896470968 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab7bfff640 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3ba50 0x55c007d44f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.405+0000 7fab81491640 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 0x55c007d4bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.406+0000 7fab80c90640 1 -- 172.21.15.83:0/3896470968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3ba50 msgr2=0x55c007d44f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.406+0000 7fab80c90640 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3ba50 0x55c007d44f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.406+0000 7fab80c90640 1 -- 172.21.15.83:0/3896470968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 msgr2=0x55c007d4bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.406+0000 7fab80c90640 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 0x55c007d4bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.406+0000 7fab80c90640 1 -- 172.21.15.83:0/3896470968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7c108530 con 0x55c007d3b060 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.406+0000 7fab5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.406+0000 7fab80c90640 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3b060 0x55c007d449f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fab70037040 tx=0x7fab7002db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2236b0a9e061473 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.406+0000 7fab5bfff640 1 -- 172.21.15.83:0/3896470968 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70035bd0 con 0x55c007d3b060 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.406+0000 7fab5bfff640 1 -- 172.21.15.83:0/3896470968 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fab70035d30 con 0x55c007d3b060 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.406+0000 7fab85ba1740 1 -- 172.21.15.83:0/3896470968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c00799a7e0 con 0x55c007d3b060 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.407+0000 7fab5bfff640 1 -- 172.21.15.83:0/3896470968 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab70004e60 con 0x55c007d3b060 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.407+0000 7fab85ba1740 10 client.17538 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.407+0000 7fab85ba1740 1 -- 172.21.15.83:0/3896470968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c007c99610 con 0x55c007d3b060 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.407+0000 7fab5bfff640 1 -- 172.21.15.83:0/3896470968 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c00799a7e0 con 0x55c007d3b060 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.407+0000 7fab5bfff640 1 -- 172.21.15.83:0/3896470968 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fab7003d040 con 0x55c007d3b060 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.407+0000 7fab85ba1740 10 client.17538 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.407+0000 7fab85ba1740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.407+0000 7fab85ba1740 20 client.17538 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.407+0000 7fab85ba1740 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c007d50920 0x55c007d52d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.407+0000 7fab85ba1740 4 client.17538 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.407+0000 7fab85ba1740 1 -- 172.21.15.83:0/3896470968 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0079d7720 con 0x55c007d50920 2024-03-11T19:51:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.408+0000 7fab7bfff640 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c007d50920 0x55c007d52d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.408+0000 7fab5bfff640 10 client.17538 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:06.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.408+0000 7fab7bfff640 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c007d50920 0x55c007d52d40 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e869ec8962ad002 server_cookie=5d5777a4bd836b0a in_seq=0 out_seq=0 2024-03-11T19:51:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab5bfff640 1 -- 172.21.15.83:0/3896470968 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0079d7720 con 0x55c007d50920 2024-03-11T19:51:06.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab5bfff640 10 client.17538 handle_command_reply: tid=0 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 1 client.17538 shutdown 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 1 -- 172.21.15.83:0/3896470968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3b060 msgr2=0x55c007d449f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3b060 0x55c007d449f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fab70037040 tx=0x7fab7002db20 comp rx=0 tx=0).stop 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 1 -- 172.21.15.83:0/3896470968 shutdown_connections 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c007d50920 0x55c007d52d40 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab5bfff640 0 client.17538 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 0x55c007d4bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3ba50 0x55c007d44f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 1 --2- 172.21.15.83:0/3896470968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3b060 0x55c007d449f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 1 -- 172.21.15.83:0/3896470968 >> 172.21.15.83:0/3896470968 conn(0x55c007d2d770 msgr2=0x55c007d3a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 1 -- 172.21.15.83:0/3896470968 shutdown_connections 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 1 -- 172.21.15.83:0/3896470968 wait complete. 2024-03-11T19:51:06.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.409+0000 7fab85ba1740 20 client.17538 trim_cache size 0 max 16384 2024-03-11T19:51:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.435+0000 7fab85ba1740 1 Processor -- start 2024-03-11T19:51:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.435+0000 7fab85ba1740 1 -- start start 2024-03-11T19:51:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.435+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d2c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.435+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d3ba50 0x55c007d4bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.436+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 0x55c007d2c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.436+0000 7fab85ba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0079d7720 con 0x55c007d406a0 2024-03-11T19:51:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.436+0000 7fab85ba1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c00799a7e0 con 0x55c007d3ba50 2024-03-11T19:51:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.436+0000 7fab85ba1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c007d29320 con 0x55c007d3b060 2024-03-11T19:51:06.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.438+0000 7fab81491640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 0x55c007d2c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.438+0000 7fab81491640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 0x55c007d2c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40248/0 (socket says 172.21.15.83:40248) 2024-03-11T19:51:06.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.438+0000 7fab81491640 1 -- 172.21.15.83:0/331550928 learned_addr learned my addr 172.21.15.83:0/331550928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:06.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.441+0000 7fab7bfff640 1 --2- 172.21.15.83:0/331550928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d3ba50 0x55c007d4bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.445+0000 7fab80c90640 1 --2- 172.21.15.83:0/331550928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d2c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.447+0000 7fab81491640 1 -- 172.21.15.83:0/331550928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 msgr2=0x55c007d2c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:06.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.447+0000 7fab81491640 1 --2- 172.21.15.83:0/331550928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d2c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.447+0000 7fab81491640 1 -- 172.21.15.83:0/331550928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d3ba50 msgr2=0x55c007d4bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:06.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.447+0000 7fab81491640 1 --2- 172.21.15.83:0/331550928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d3ba50 0x55c007d4bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.447+0000 7fab81491640 1 -- 172.21.15.83:0/331550928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c007c35ac0 con 0x55c007d406a0 2024-03-11T19:51:06.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.462+0000 7fab81491640 1 --2- 172.21.15.83:0/331550928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 0x55c007d2c700 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fab7400f990 tx=0x7fab74018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff67a26bdc25b3f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:06.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.467+0000 7fab667fc640 1 -- 172.21.15.83:0/331550928 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab74023070 con 0x55c007d406a0 2024-03-11T19:51:06.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.467+0000 7fab667fc640 1 -- 172.21.15.83:0/331550928 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab7401ee90 con 0x55c007d406a0 2024-03-11T19:51:06.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.467+0000 7fab667fc640 1 -- 172.21.15.83:0/331550928 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab74027d30 con 0x55c007d406a0 2024-03-11T19:51:06.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.470+0000 7fab85ba1740 1 -- 172.21.15.83:0/331550928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 msgr2=0x55c007d2c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.470+0000 7fab85ba1740 1 --2- 172.21.15.83:0/331550928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 0x55c007d2c700 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fab7400f990 tx=0x7fab74018000 comp rx=0 tx=0).stop 2024-03-11T19:51:06.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.471+0000 7fab85ba1740 1 -- 172.21.15.83:0/331550928 shutdown_connections 2024-03-11T19:51:06.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.471+0000 7fab85ba1740 1 --2- 172.21.15.83:0/331550928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d406a0 0x55c007d2c700 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.471+0000 7fab85ba1740 1 --2- 172.21.15.83:0/331550928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d3ba50 0x55c007d4bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.471+0000 7fab85ba1740 1 --2- 172.21.15.83:0/331550928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d2c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.471+0000 7fab85ba1740 1 -- 172.21.15.83:0/331550928 >> 172.21.15.83:0/331550928 conn(0x55c007d2d770 msgr2=0x55c007d3c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:06.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.472+0000 7fab85ba1740 1 -- 172.21.15.83:0/331550928 shutdown_connections 2024-03-11T19:51:06.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.475+0000 7fab85ba1740 1 -- 172.21.15.83:0/331550928 wait complete. 2024-03-11T19:51:06.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.475+0000 7fab85ba1740 1 -- start start 2024-03-11T19:51:06.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.475+0000 7fab85ba1740 1 Processor -- start 2024-03-11T19:51:06.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.476+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d4a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.476+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3ba50 0x55c007d4aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.476+0000 7fab85ba1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 0x55c007d4b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.477+0000 7fab85ba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c007da6f00 con 0x55c007d3ba50 2024-03-11T19:51:06.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.477+0000 7fab85ba1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c007da7070 con 0x55c007d406a0 2024-03-11T19:51:06.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.477+0000 7fab81491640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 0x55c007d4b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.477+0000 7fab81491640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 0x55c007d4b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32894/0 (socket says 172.21.15.83:32894) 2024-03-11T19:51:06.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.477+0000 7fab81491640 1 -- 172.21.15.83:0/4193247607 learned_addr learned my addr 172.21.15.83:0/4193247607 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:06.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.477+0000 7fab85ba1740 1 -- 172.21.15.83:0/4193247607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c007da7230 con 0x55c007d3b060 2024-03-11T19:51:06.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.477+0000 7fab85ba1740 10 client.0 _ll_register_callbacks cb 0x7fab7000a890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:06.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.478+0000 7fab7bfff640 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3ba50 0x55c007d4aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.479+0000 7fab81491640 1 -- 172.21.15.83:0/4193247607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 msgr2=0x55c007d4a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:06.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.479+0000 7fab81491640 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d4a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.479+0000 7fab81491640 1 -- 172.21.15.83:0/4193247607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3ba50 msgr2=0x55c007d4aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:06.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.479+0000 7fab81491640 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3ba50 0x55c007d4aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.479+0000 7fab81491640 1 -- 172.21.15.83:0/4193247607 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7400f720 con 0x55c007d406a0 2024-03-11T19:51:06.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.480+0000 7fab81491640 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 0x55c007d4b3f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fab7402a040 tx=0x7fab74027b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b90b1f91ae61a134 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:06.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.482+0000 7fab5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:51:06.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.482+0000 7fab5bfff640 1 -- 172.21.15.83:0/4193247607 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab74023030 con 0x55c007d406a0 2024-03-11T19:51:06.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.482+0000 7fab85ba1740 1 -- 172.21.15.83:0/4193247607 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab7002dc80 con 0x55c007d406a0 2024-03-11T19:51:06.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.482+0000 7fab5bfff640 1 -- 172.21.15.83:0/4193247607 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fab74024070 con 0x55c007d406a0 2024-03-11T19:51:06.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.484+0000 7fab5bfff640 1 -- 172.21.15.83:0/4193247607 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab74015050 con 0x55c007d406a0 2024-03-11T19:51:06.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.484+0000 7fab85ba1740 10 client.26996 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:06.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.484+0000 7fab85ba1740 1 -- 172.21.15.83:0/4193247607 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c007c5b6a0 con 0x55c007d406a0 2024-03-11T19:51:06.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.487+0000 7fab5bfff640 1 -- 172.21.15.83:0/4193247607 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab74003e50 con 0x55c007d406a0 2024-03-11T19:51:06.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.487+0000 7fab5bfff640 1 -- 172.21.15.83:0/4193247607 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fab7402e070 con 0x55c007d406a0 2024-03-11T19:51:06.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.487+0000 7fab85ba1740 10 client.26996 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:06.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.487+0000 7fab85ba1740 10 client.26996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:06.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.487+0000 7fab85ba1740 20 client.26996 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:06.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.487+0000 7fab85ba1740 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c007da8bd0 0x55c007daaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.487+0000 7fab85ba1740 4 client.26996 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:51:06.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.487+0000 7fab85ba1740 1 -- 172.21.15.83:0/4193247607 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fab7002dc80 con 0x55c007da8bd0 2024-03-11T19:51:06.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.491+0000 7fab80c90640 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c007da8bd0 0x55c007daaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.493+0000 7fab80c90640 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c007da8bd0 0x55c007daaff0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe323ca82f939916 server_cookie=d2712b5ebdeb8ed0 in_seq=0 out_seq=0 2024-03-11T19:51:06.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.493+0000 7fab5bfff640 10 client.26996 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:06.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.493+0000 7fab5bfff640 1 -- 172.21.15.83:0/4193247607 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fab7002dc80 con 0x55c007da8bd0 2024-03-11T19:51:06.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.493+0000 7fab5bfff640 10 client.26996 handle_command_reply: tid=0 2024-03-11T19:51:06.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.493+0000 7fab85ba1740 1 client.26996 shutdown 2024-03-11T19:51:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.494+0000 7fab85ba1740 1 -- 172.21.15.83:0/4193247607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 msgr2=0x55c007d4b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.494+0000 7fab85ba1740 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 0x55c007d4b3f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fab7402a040 tx=0x7fab74027b10 comp rx=0 tx=0).stop 2024-03-11T19:51:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.494+0000 7fab85ba1740 1 -- 172.21.15.83:0/4193247607 shutdown_connections 2024-03-11T19:51:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.494+0000 7fab85ba1740 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c007da8bd0 0x55c007daaff0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.494+0000 7fab85ba1740 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c007d406a0 0x55c007d4b3f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.494+0000 7fab85ba1740 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c007d3ba50 0x55c007d4aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.494+0000 7fab85ba1740 1 --2- 172.21.15.83:0/4193247607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c007d3b060 0x55c007d4a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.494+0000 7fab85ba1740 1 -- 172.21.15.83:0/4193247607 >> 172.21.15.83:0/4193247607 conn(0x55c007d2d770 msgr2=0x55c007d39e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.494+0000 7fab5bfff640 0 client.26996 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.494+0000 7fab85ba1740 1 -- 172.21.15.83:0/4193247607 shutdown_connections 2024-03-11T19:51:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.495+0000 7fab85ba1740 1 -- 172.21.15.83:0/4193247607 wait complete. 2024-03-11T19:51:06.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.495+0000 7fab85ba1740 20 client.26996 trim_cache size 0 max 16384 2024-03-11T19:51:06.500 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:51:06.500 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:51:06.500 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:51:06.500 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:51:06.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.496+0000 7fab82ef4640 1 -- 172.21.15.83:0/2850700307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fab6c0a1a80 msgr2=0x7fab6c0a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.496+0000 7fab82ef4640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fab6c0a1a80 0x7fab6c0a3f40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fab74005fd0 tx=0x7fab74011810 comp rx=0 tx=0).stop 2024-03-11T19:51:06.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.496+0000 7fab82ef4640 1 -- 172.21.15.83:0/2850700307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab7c1009e0 msgr2=0x7fab7c1978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.496+0000 7fab82ef4640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab7c1009e0 0x7fab7c1978b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fab6800d950 tx=0x7fab6800de10 comp rx=0 tx=0).stop 2024-03-11T19:51:06.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.500+0000 7fab82ef4640 1 -- 172.21.15.83:0/2850700307 shutdown_connections 2024-03-11T19:51:06.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.500+0000 7fab82ef4640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fab6c0a1a80 0x7fab6c0a3f40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.500+0000 7fab82ef4640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fab7c1013a0 0x7fab7c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.500+0000 7fab82ef4640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab7c1009e0 0x7fab7c1978b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.500+0000 7fab82ef4640 1 --2- 172.21.15.83:0/2850700307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab7c0ff7f0 0x7fab7c197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.500+0000 7fab82ef4640 1 -- 172.21.15.83:0/2850700307 >> 172.21.15.83:0/2850700307 conn(0x7fab7c0fad60 msgr2=0x7fab7c103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:06.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.500+0000 7fab82ef4640 1 -- 172.21.15.83:0/2850700307 shutdown_connections 2024-03-11T19:51:06.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.500+0000 7fab82ef4640 1 -- 172.21.15.83:0/2850700307 wait complete. 2024-03-11T19:51:06.530 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:51:06.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:51:06.563 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:06 smithi083 ceph-mon[31864]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.4 MiB/s wr, 707 op/s 2024-03-11T19:51:06.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.672+0000 7fc54e7fc640 1 -- 172.21.15.83:0/2557963874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54002cc60 con 0x7fc55010aaf0 2024-03-11T19:51:06.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.672+0000 7fc55604e640 1 -- 172.21.15.83:0/2557963874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc55010aaf0 msgr2=0x7fc55010cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.672+0000 7fc55604e640 1 --2- 172.21.15.83:0/2557963874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc55010aaf0 0x7fc55010cf80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc540009900 tx=0x7fc54002bc80 comp rx=0 tx=0).stop 2024-03-11T19:51:06.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.672+0000 7fc55604e640 1 -- 172.21.15.83:0/2557963874 shutdown_connections 2024-03-11T19:51:06.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.672+0000 7fc55604e640 1 --2- 172.21.15.83:0/2557963874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55010d4c0 0x7fc55010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.672+0000 7fc55604e640 1 --2- 172.21.15.83:0/2557963874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc55010aaf0 0x7fc55010cf80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.672+0000 7fc55604e640 1 --2- 172.21.15.83:0/2557963874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc550108180 0x7fc55010a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.672+0000 7fc55604e640 1 -- 172.21.15.83:0/2557963874 >> 172.21.15.83:0/2557963874 conn(0x7fc55006dff0 msgr2=0x7fc550070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:06.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.674+0000 7fc55604e640 1 -- 172.21.15.83:0/2557963874 shutdown_connections 2024-03-11T19:51:06.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.674+0000 7fc55604e640 1 -- 172.21.15.83:0/2557963874 wait complete. 2024-03-11T19:51:06.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.675+0000 7fc55604e640 1 Processor -- start 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.675+0000 7fc55604e640 1 -- start start 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.675+0000 7fc55604e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc55010aaf0 0x7fc5501a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.675+0000 7fc55604e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55010d4c0 0x7fc5501a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.675+0000 7fc55604e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5501ab800 0x7fc5501adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.675+0000 7fc55604e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5501ae130 con 0x7fc55010aaf0 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.675+0000 7fc55604e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5501ae2a0 con 0x7fc55010d4c0 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.675+0000 7fc55604e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5501ae460 con 0x7fc5501ab800 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.675+0000 7fc54effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55010d4c0 0x7fc5501a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.675+0000 7fc54effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55010d4c0 0x7fc5501a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32912/0 (socket says 172.21.15.83:32912) 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.675+0000 7fc54effd640 1 -- 172.21.15.83:0/4053994132 learned_addr learned my addr 172.21.15.83:0/4053994132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.676+0000 7fc54effd640 1 -- 172.21.15.83:0/4053994132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5501ab800 msgr2=0x7fc5501adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.676+0000 7fc54effd640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5501ab800 0x7fc5501adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.676+0000 7fc54effd640 1 -- 172.21.15.83:0/4053994132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc55010aaf0 msgr2=0x7fc5501a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.676+0000 7fc54effd640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc55010aaf0 0x7fc5501a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.676+0000 7fc54effd640 1 -- 172.21.15.83:0/4053994132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc540009560 con 0x7fc55010d4c0 2024-03-11T19:51:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.676+0000 7fc54effd640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55010d4c0 0x7fc5501a67f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc540005ab0 tx=0x7fc5400042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.677+0000 7fc54cff9640 1 -- 172.21.15.83:0/4053994132 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540004df0 con 0x7fc55010d4c0 2024-03-11T19:51:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.677+0000 7fc54cff9640 1 -- 172.21.15.83:0/4053994132 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc540005430 con 0x7fc55010d4c0 2024-03-11T19:51:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.677+0000 7fc55604e640 1 -- 172.21.15.83:0/4053994132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5501ae6e0 con 0x7fc55010d4c0 2024-03-11T19:51:06.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.677+0000 7fc55604e640 1 -- 172.21.15.83:0/4053994132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5501aec90 con 0x7fc55010d4c0 2024-03-11T19:51:06.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.679+0000 7fc54cff9640 1 -- 172.21.15.83:0/4053994132 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54003e450 con 0x7fc55010d4c0 2024-03-11T19:51:06.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.682+0000 7fc55604e640 1 -- 172.21.15.83:0/4053994132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc550070af0 con 0x7fc55010d4c0 2024-03-11T19:51:06.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.682+0000 7fc54cff9640 1 -- 172.21.15.83:0/4053994132 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc54003e5b0 con 0x7fc55010d4c0 2024-03-11T19:51:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.683+0000 7fc54cff9640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5380a1b80 0x7fc5380a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:06.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.683+0000 7fc54f7fe640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5380a1b80 0x7fc5380a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.683+0000 7fc54cff9640 1 -- 172.21.15.83:0/4053994132 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc5400f0f20 con 0x7fc55010d4c0 2024-03-11T19:51:06.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.684+0000 7fc54f7fe640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5380a1b80 0x7fc5380a4040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc5500731e0 tx=0x7fc544006c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:06.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:06.690+0000 7fc54cff9640 1 -- 172.21.15.83:0/4053994132 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc5400ba140 con 0x7fc55010d4c0 2024-03-11T19:51:06.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:06 smithi137 ceph-mon[36220]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.4 MiB/s wr, 707 op/s 2024-03-11T19:51:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:06 smithi204 ceph-mon[36739]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.4 MiB/s wr, 707 op/s 2024-03-11T19:51:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.011+0000 7fc55604e640 1 -- 172.21.15.83:0/4053994132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc55002cda0 con 0x7fc55010d4c0 2024-03-11T19:51:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.012+0000 7fc54cff9640 1 -- 172.21.15.83:0/4053994132 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7fc540043020 con 0x7fc55010d4c0 2024-03-11T19:51:07.016 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:51:07.016 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:51:07.017 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:51:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.015+0000 7fc55604e640 1 -- 172.21.15.83:0/4053994132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5380a1b80 msgr2=0x7fc5380a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.015+0000 7fc55604e640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5380a1b80 0x7fc5380a4040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc5500731e0 tx=0x7fc544006c10 comp rx=0 tx=0).stop 2024-03-11T19:51:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.015+0000 7fc55604e640 1 -- 172.21.15.83:0/4053994132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55010d4c0 msgr2=0x7fc5501a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.015+0000 7fc55604e640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55010d4c0 0x7fc5501a67f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc540005ab0 tx=0x7fc5400042e0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.016+0000 7fc55604e640 1 -- 172.21.15.83:0/4053994132 shutdown_connections 2024-03-11T19:51:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.016+0000 7fc55604e640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc5380a1b80 0x7fc5380a4040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.016+0000 7fc55604e640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5501ab800 0x7fc5501adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.016+0000 7fc55604e640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55010d4c0 0x7fc5501a67f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.016+0000 7fc55604e640 1 --2- 172.21.15.83:0/4053994132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc55010aaf0 0x7fc5501a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.016+0000 7fc55604e640 1 -- 172.21.15.83:0/4053994132 >> 172.21.15.83:0/4053994132 conn(0x7fc55006dff0 msgr2=0x7fc5501085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.016+0000 7fc55604e640 1 -- 172.21.15.83:0/4053994132 shutdown_connections 2024-03-11T19:51:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.016+0000 7fc55604e640 1 -- 172.21.15.83:0/4053994132 wait complete. 2024-03-11T19:51:07.033 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:51:07.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:51:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.228+0000 7f0cf5955640 1 -- 172.21.15.83:0/2464898839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf0100b00 msgr2=0x7f0cf0100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.228+0000 7f0cf5955640 1 --2- 172.21.15.83:0/2464898839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf0100b00 0x7f0cf0100f80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009900 tx=0x7f0ce002bc30 comp rx=0 tx=0).stop 2024-03-11T19:51:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.228+0000 7f0cf5955640 1 -- 172.21.15.83:0/2464898839 shutdown_connections 2024-03-11T19:51:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.228+0000 7f0cf5955640 1 --2- 172.21.15.83:0/2464898839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cf0105800 0x7f0cf0107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.228+0000 7f0cf5955640 1 --2- 172.21.15.83:0/2464898839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf0100b00 0x7f0cf0100f80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.228+0000 7f0cf5955640 1 --2- 172.21.15.83:0/2464898839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cf00676e0 0x7f0cf01005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.228+0000 7f0cf5955640 1 -- 172.21.15.83:0/2464898839 >> 172.21.15.83:0/2464898839 conn(0x7f0cf00fa940 msgr2=0x7f0cf00fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.228+0000 7f0cf5955640 1 -- 172.21.15.83:0/2464898839 shutdown_connections 2024-03-11T19:51:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.228+0000 7f0cf5955640 1 -- 172.21.15.83:0/2464898839 wait complete. 2024-03-11T19:51:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.228+0000 7f0cf5955640 1 Processor -- start 2024-03-11T19:51:07.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0cf5955640 1 -- start start 2024-03-11T19:51:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0cf5955640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf00676e0 0x7f0cf010af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0cf5955640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cf0100b00 0x7f0cf0109540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0cf5955640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cf0105800 0x7f0cf0109a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0ceeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf00676e0 0x7f0cf010af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0ceeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf00676e0 0x7f0cf010af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40304/0 (socket says 172.21.15.83:40304) 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0ceeffd640 1 -- 172.21.15.83:0/4100722446 learned_addr learned my addr 172.21.15.83:0/4100722446 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0cf5955640 1 -- 172.21.15.83:0/4100722446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf010a1e0 con 0x7f0cf00676e0 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0cf5955640 1 -- 172.21.15.83:0/4100722446 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf010a350 con 0x7f0cf0100b00 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0cf5955640 1 -- 172.21.15.83:0/4100722446 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf01a1320 con 0x7f0cf0105800 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cf0100b00 0x7f0cf0109540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.229+0000 7f0cef7fe640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cf0105800 0x7f0cf0109a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.230+0000 7f0ceeffd640 1 -- 172.21.15.83:0/4100722446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cf0105800 msgr2=0x7f0cf0109a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.230+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cf0105800 0x7f0cf0109a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.230+0000 7f0ceeffd640 1 -- 172.21.15.83:0/4100722446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cf0100b00 msgr2=0x7f0cf0109540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.230+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cf0100b00 0x7f0cf0109540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.230+0000 7f0ceeffd640 1 -- 172.21.15.83:0/4100722446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce0009560 con 0x7f0cf00676e0 2024-03-11T19:51:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.230+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf00676e0 0x7f0cf010af00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00e9f0 tx=0x7f0cdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.230+0000 7f0cf4953640 1 -- 172.21.15.83:0/4100722446 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc009700 con 0x7f0cf00676e0 2024-03-11T19:51:07.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.230+0000 7f0cf5955640 1 -- 172.21.15.83:0/4100722446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf01a15d0 con 0x7f0cf00676e0 2024-03-11T19:51:07.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.230+0000 7f0cf5955640 1 -- 172.21.15.83:0/4100722446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf01a1b00 con 0x7f0cf00676e0 2024-03-11T19:51:07.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.230+0000 7f0cf4953640 1 -- 172.21.15.83:0/4100722446 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cdc004500 con 0x7f0cf00676e0 2024-03-11T19:51:07.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.230+0000 7f0cf4953640 1 -- 172.21.15.83:0/4100722446 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc010430 con 0x7f0cf00676e0 2024-03-11T19:51:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cf4953640 1 -- 172.21.15.83:0/4100722446 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0cdc00cd80 con 0x7f0cf00676e0 2024-03-11T19:51:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cf85fd740 1 Processor -- start 2024-03-11T19:51:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cf85fd740 1 -- start start 2024-03-11T19:51:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae29270 0x55fd3ae29690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae2e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae31260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3ade5960 con 0x55fd3ae2edd0 2024-03-11T19:51:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3ad99310 con 0x55fd3ae29c60 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3ad96840 con 0x55fd3ae29270 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cf4953640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cd40a1ab0 0x7f0cd40a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cee7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae31260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cee7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae31260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40320/0 (socket says 172.21.15.83:40320) 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cee7fc640 1 -- 172.21.15.83:0/753747201 learned_addr learned my addr 172.21.15.83:0/753747201 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/753747201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae2e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cef7fe640 1 --2- 172.21.15.83:0/753747201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae29270 0x55fd3ae29690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cef7fe640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cd40a1ab0 0x7f0cd40a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.233+0000 7f0cf4953640 1 -- 172.21.15.83:0/4100722446 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0cdc014070 con 0x7f0cf00676e0 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.234+0000 7f0cee7fc640 1 -- 172.21.15.83:0/753747201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae29270 msgr2=0x55fd3ae29690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.234+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/753747201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae29270 0x55fd3ae29690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.234+0000 7f0cee7fc640 1 -- 172.21.15.83:0/753747201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 msgr2=0x55fd3ae2e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.234+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/753747201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae2e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.234+0000 7f0cee7fc640 1 -- 172.21.15.83:0/753747201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd3ad54060 con 0x55fd3ae2edd0 2024-03-11T19:51:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.234+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/753747201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae2e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:51:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.234+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/753747201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae31260 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009900 tx=0x7f0ce0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41da6efa4fef4c69 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.234+0000 7f0cef7fe640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cd40a1ab0 0x7f0cd40a3f70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55fd3ae2a4c0 tx=0x7f0ce8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.234+0000 7f0ce5ffb640 1 -- 172.21.15.83:0/753747201 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce002c960 con 0x55fd3ae2edd0 2024-03-11T19:51:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.234+0000 7f0ce5ffb640 1 -- 172.21.15.83:0/753747201 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ce002cac0 con 0x55fd3ae2edd0 2024-03-11T19:51:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.234+0000 7f0ce5ffb640 1 -- 172.21.15.83:0/753747201 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce002c960 con 0x55fd3ae2edd0 2024-03-11T19:51:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.236+0000 7f0cf85fd740 1 -- 172.21.15.83:0/753747201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae2edd0 msgr2=0x55fd3ae31260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.236+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/753747201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae31260 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009900 tx=0x7f0ce0005d50 comp rx=0 tx=0).stop 2024-03-11T19:51:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.236+0000 7f0cf85fd740 1 -- 172.21.15.83:0/753747201 shutdown_connections 2024-03-11T19:51:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.236+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/753747201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae31260 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.236+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/753747201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae2e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.236+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/753747201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae29270 0x55fd3ae29690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.236+0000 7f0cf85fd740 1 -- 172.21.15.83:0/753747201 >> 172.21.15.83:0/753747201 conn(0x55fd3ae247e0 msgr2=0x55fd3ae24be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.236+0000 7f0cf85fd740 1 -- 172.21.15.83:0/753747201 shutdown_connections 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.237+0000 7f0cf85fd740 1 -- 172.21.15.83:0/753747201 wait complete. 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.237+0000 7f0cf85fd740 1 -- start start 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.237+0000 7f0cf85fd740 1 Processor -- start 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae33930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae3a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3ad92020 con 0x55fd3ae29270 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3aa65430 con 0x55fd3ae29c60 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3aa21e00 con 0x55fd3ae2edd0 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0cf85fd740 10 client.0 _ll_register_callbacks cb 0x55fd3ad39cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0ceeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0ceeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae333f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40326/0 (socket says 172.21.15.83:40326) 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0cef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae3a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0ceeffd640 1 -- 172.21.15.83:0/1905210695 learned_addr learned my addr 172.21.15.83:0/1905210695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae33930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0ceeffd640 1 -- 172.21.15.83:0/1905210695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae2edd0 msgr2=0x55fd3ae3a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae3a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0ceeffd640 1 -- 172.21.15.83:0/1905210695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 msgr2=0x55fd3ae33930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae33930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0ceeffd640 1 -- 172.21.15.83:0/1905210695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf00ff750 con 0x55fd3ae29270 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0cef7fe640 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae3a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae333f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc09a600 tx=0x7f0cdc015f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11552c63986470 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.238+0000 7f0cd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:07.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/1905210695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc052070 con 0x55fd3ae29270 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/1905210695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0cdc083e40 con 0x55fd3ae29270 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cf85fd740 1 -- 172.21.15.83:0/1905210695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd3aa917e0 con 0x55fd3ae29270 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/1905210695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cdc09e7c0 con 0x55fd3ae29270 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cf85fd740 10 client.17559 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cf85fd740 1 -- 172.21.15.83:0/1905210695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fd3ad906a0 con 0x55fd3ae29270 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/1905210695 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fd3aa917e0 con 0x55fd3ae29270 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/1905210695 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0cdc01f060 con 0x55fd3ae29270 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cf85fd740 10 client.17559 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cf85fd740 10 client.17559 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cf85fd740 20 client.17559 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd3ae3f580 0x55fd3ae41990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.239+0000 7f0cf85fd740 4 client.17559 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.240+0000 7f0cf85fd740 1 -- 172.21.15.83:0/1905210695 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd3aace720 con 0x55fd3ae3f580 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.240+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd3ae3f580 0x55fd3ae41990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.240+0000 7f0cd2ffd640 10 client.17559 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.240+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd3ae3f580 0x55fd3ae41990 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed7c9bebe9d29bf5 server_cookie=3b8518134fcbb07d in_seq=0 out_seq=0 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/1905210695 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd3aace720 con 0x55fd3ae3f580 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cd2ffd640 10 client.17559 handle_command_reply: tid=0 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 1 client.17559 shutdown 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 1 -- 172.21.15.83:0/1905210695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 msgr2=0x55fd3ae333f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae333f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc09a600 tx=0x7f0cdc015f40 comp rx=0 tx=0).stop 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 1 -- 172.21.15.83:0/1905210695 shutdown_connections 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd3ae3f580 0x55fd3ae41990 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae3a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cd2ffd640 0 client.17559 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae33930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/1905210695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae333f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 1 -- 172.21.15.83:0/1905210695 >> 172.21.15.83:0/1905210695 conn(0x55fd3ae247e0 msgr2=0x55fd3ae23b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 1 -- 172.21.15.83:0/1905210695 shutdown_connections 2024-03-11T19:51:07.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 1 -- 172.21.15.83:0/1905210695 wait complete. 2024-03-11T19:51:07.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.241+0000 7f0cf85fd740 20 client.17559 trim_cache size 0 max 16384 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.255+0000 7f0cf85fd740 1 Processor -- start 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0cf85fd740 1 -- start start 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae41970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae29c60 0x55fd3ae3a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae24bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3ad17130 con 0x55fd3ae29270 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3ae3bdf0 con 0x55fd3ae2edd0 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3ae2ac90 con 0x55fd3ae29c60 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0ceeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae41970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0ceeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae41970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40328/0 (socket says 172.21.15.83:40328) 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0cef7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae24bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0ceeffd640 1 -- 172.21.15.83:0/3722103778 learned_addr learned my addr 172.21.15.83:0/3722103778 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/3722103778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae29c60 0x55fd3ae3a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0ceeffd640 1 -- 172.21.15.83:0/3722103778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae29c60 msgr2=0x55fd3ae3a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/3722103778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae29c60 0x55fd3ae3a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0ceeffd640 1 -- 172.21.15.83:0/3722103778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae2edd0 msgr2=0x55fd3ae24bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/3722103778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae24bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0ceeffd640 1 -- 172.21.15.83:0/3722103778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd3ad2cc20 con 0x55fd3ae29270 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/3722103778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae29c60 0x55fd3ae3a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.256+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/3722103778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae41970 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc096ff0 tx=0x7f0cdc085050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910b56e03abe1d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0ce57fa640 1 -- 172.21.15.83:0/3722103778 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc093070 con 0x55fd3ae29270 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0ce57fa640 1 -- 172.21.15.83:0/3722103778 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cdc083e40 con 0x55fd3ae29270 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0ce57fa640 1 -- 172.21.15.83:0/3722103778 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0b3130 con 0x55fd3ae29270 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0cf85fd740 1 -- 172.21.15.83:0/3722103778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 msgr2=0x55fd3ae41970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/3722103778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae41970 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc096ff0 tx=0x7f0cdc085050 comp rx=0 tx=0).stop 2024-03-11T19:51:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0cf85fd740 1 -- 172.21.15.83:0/3722103778 shutdown_connections 2024-03-11T19:51:07.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/3722103778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae24bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/3722103778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae29c60 0x55fd3ae3a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/3722103778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae41970 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0cf85fd740 1 -- 172.21.15.83:0/3722103778 >> 172.21.15.83:0/3722103778 conn(0x55fd3ae247e0 msgr2=0x55fd3ae1fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0cf85fd740 1 -- 172.21.15.83:0/3722103778 shutdown_connections 2024-03-11T19:51:07.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.257+0000 7f0cf85fd740 1 -- 172.21.15.83:0/3722103778 wait complete. 2024-03-11T19:51:07.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0cf85fd740 1 -- start start 2024-03-11T19:51:07.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0cf85fd740 1 Processor -- start 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae3a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae38da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0cf85fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3ae398c0 con 0x55fd3ae29270 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3ae39a30 con 0x55fd3ae29c60 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0cf85fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3ae39bf0 con 0x55fd3ae2edd0 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0cf85fd740 10 client.0 _ll_register_callbacks cb 0x7f0ce80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0ceeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae3a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0ceeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae3a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40340/0 (socket says 172.21.15.83:40340) 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0ceeffd640 1 -- 172.21.15.83:0/2474383199 learned_addr learned my addr 172.21.15.83:0/2474383199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0cef7fe640 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae38da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0ceeffd640 1 -- 172.21.15.83:0/2474383199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae2edd0 msgr2=0x55fd3ae392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0ceeffd640 1 -- 172.21.15.83:0/2474383199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 msgr2=0x55fd3ae38da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae38da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.258+0000 7f0ceeffd640 1 -- 172.21.15.83:0/2474383199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd3ad54060 con 0x55fd3ae29270 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0ceeffd640 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae3a720 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0a28e0 tx=0x7f0cdc003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f0dc2be44c39f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/2474383199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc093070 con 0x55fd3ae29270 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/2474383199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0cdc0850c0 con 0x55fd3ae29270 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cf85fd740 1 -- 172.21.15.83:0/2474383199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ce002cac0 con 0x55fd3ae29270 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/2474383199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cdc085300 con 0x55fd3ae29270 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cf85fd740 10 client.17565 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cf85fd740 1 -- 172.21.15.83:0/2474383199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fd3ad906a0 con 0x55fd3ae29270 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/2474383199 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ce002cac0 con 0x55fd3ae29270 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/2474383199 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0cdc099ac0 con 0x55fd3ae29270 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cf85fd740 10 client.17565 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:07.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cf85fd740 10 client.17565 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cf85fd740 20 client.17565 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd3ae3f580 0x55fd3ae98d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.259+0000 7f0cf85fd740 4 client.17565 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.260+0000 7f0cf85fd740 1 -- 172.21.15.83:0/2474383199 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd3ab1d890 con 0x55fd3ae3f580 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.260+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd3ae3f580 0x55fd3ae98d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.261+0000 7f0cee7fc640 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd3ae3f580 0x55fd3ae98d80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10cd91bb8cff6596 server_cookie=b39fea12c10427ef in_seq=0 out_seq=0 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.261+0000 7f0cd2ffd640 10 client.17565 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.261+0000 7f0cd2ffd640 1 -- 172.21.15.83:0/2474383199 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd3ab1d890 con 0x55fd3ae3f580 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.262+0000 7f0cd2ffd640 10 client.17565 handle_command_reply: tid=0 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.262+0000 7f0cf85fd740 1 client.17565 shutdown 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.262+0000 7f0cf85fd740 1 -- 172.21.15.83:0/2474383199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 msgr2=0x55fd3ae3a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.262+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae3a720 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0a28e0 tx=0x7f0cdc003a00 comp rx=0 tx=0).stop 2024-03-11T19:51:07.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.262+0000 7f0cf85fd740 1 -- 172.21.15.83:0/2474383199 shutdown_connections 2024-03-11T19:51:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.262+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd3ae3f580 0x55fd3ae98d80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.262+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd3ae2edd0 0x55fd3ae392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.262+0000 7f0cd2ffd640 0 client.17565 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:51:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.262+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd3ae29c60 0x55fd3ae38da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.262+0000 7f0cf85fd740 1 --2- 172.21.15.83:0/2474383199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd3ae29270 0x55fd3ae3a720 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.263+0000 7f0cf85fd740 1 -- 172.21.15.83:0/2474383199 >> 172.21.15.83:0/2474383199 conn(0x55fd3ae247e0 msgr2=0x55fd3ae24bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.263+0000 7f0cf85fd740 1 -- 172.21.15.83:0/2474383199 shutdown_connections 2024-03-11T19:51:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.263+0000 7f0cf85fd740 1 -- 172.21.15.83:0/2474383199 wait complete. 2024-03-11T19:51:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.263+0000 7f0cf85fd740 20 client.17565 trim_cache size 0 max 16384 2024-03-11T19:51:07.272 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:51:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.263+0000 7f0cf5955640 1 -- 172.21.15.83:0/4100722446 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cd40a1ab0 msgr2=0x7f0cd40a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.263+0000 7f0cf5955640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cd40a1ab0 0x7f0cd40a3f70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55fd3ae2a4c0 tx=0x7f0ce8009500 comp rx=0 tx=0).stop 2024-03-11T19:51:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.263+0000 7f0cf5955640 1 -- 172.21.15.83:0/4100722446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf00676e0 msgr2=0x7f0cf010af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.263+0000 7f0cf5955640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf00676e0 0x7f0cf010af00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00e9f0 tx=0x7f0cdc00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.264+0000 7f0cf5955640 1 -- 172.21.15.83:0/4100722446 shutdown_connections 2024-03-11T19:51:07.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.264+0000 7f0cf5955640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0cd40a1ab0 0x7f0cd40a3f70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.264+0000 7f0cf5955640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cf0105800 0x7f0cf0109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.264+0000 7f0cf5955640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cf0100b00 0x7f0cf0109540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.264+0000 7f0cf5955640 1 --2- 172.21.15.83:0/4100722446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf00676e0 0x7f0cf010af00 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.264+0000 7f0cf5955640 1 -- 172.21.15.83:0/4100722446 >> 172.21.15.83:0/4100722446 conn(0x7f0cf00fa940 msgr2=0x7f0cf01017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.264+0000 7f0cf5955640 1 -- 172.21.15.83:0/4100722446 shutdown_connections 2024-03-11T19:51:07.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.264+0000 7f0cf5955640 1 -- 172.21.15.83:0/4100722446 wait complete. 2024-03-11T19:51:07.283 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:51:07.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:51:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.423+0000 7f7785c0f640 1 -- 172.21.15.83:0/1646994427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780108990 msgr2=0x7f7780108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.423+0000 7f7785c0f640 1 --2- 172.21.15.83:0/1646994427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780108990 0x7f7780108e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7774009900 tx=0x7f777402bc30 comp rx=0 tx=0).stop 2024-03-11T19:51:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.423+0000 7f7785c0f640 1 -- 172.21.15.83:0/1646994427 shutdown_connections 2024-03-11T19:51:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.423+0000 7f7785c0f640 1 --2- 172.21.15.83:0/1646994427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7780109350 0x7f778010fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.423+0000 7f7785c0f640 1 --2- 172.21.15.83:0/1646994427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780108990 0x7f7780108e10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.423+0000 7f7785c0f640 1 --2- 172.21.15.83:0/1646994427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77801077a0 0x7f7780107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.424+0000 7f7785c0f640 1 -- 172.21.15.83:0/1646994427 >> 172.21.15.83:0/1646994427 conn(0x7f7780076740 msgr2=0x7f7780078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.424+0000 7f7785c0f640 1 -- 172.21.15.83:0/1646994427 shutdown_connections 2024-03-11T19:51:07.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.424+0000 7f7785c0f640 1 -- 172.21.15.83:0/1646994427 wait complete. 2024-03-11T19:51:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.424+0000 7f7785c0f640 1 Processor -- start 2024-03-11T19:51:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f7785c0f640 1 -- start start 2024-03-11T19:51:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f7785c0f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77801077a0 0x7f778019d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f7785c0f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7780108990 0x7f778019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f7785c0f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780109350 0x7f77801a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f7785c0f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77801a5330 con 0x7f7780108990 2024-03-11T19:51:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f7785c0f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77801a54a0 con 0x7f7780109350 2024-03-11T19:51:07.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f7785c0f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77801a5660 con 0x7f77801077a0 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f777effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7780108990 0x7f778019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f777f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77801077a0 0x7f778019d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f777effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7780108990 0x7f778019dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40360/0 (socket says 172.21.15.83:40360) 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f777effd640 1 -- 172.21.15.83:0/2385222785 learned_addr learned my addr 172.21.15.83:0/2385222785 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f777ffff640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780109350 0x7f77801a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f777effd640 1 -- 172.21.15.83:0/2385222785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77801077a0 msgr2=0x7f778019d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f777effd640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77801077a0 0x7f778019d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f777effd640 1 -- 172.21.15.83:0/2385222785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780109350 msgr2=0x7f77801a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f777effd640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780109350 0x7f77801a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f777effd640 1 -- 172.21.15.83:0/2385222785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7774009560 con 0x7f7780108990 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.425+0000 7f777f7fe640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77801077a0 0x7f778019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.426+0000 7f777effd640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7780108990 0x7f778019dd80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7774006010 tx=0x7f7774005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.426+0000 7f777cff9640 1 -- 172.21.15.83:0/2385222785 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740043d0 con 0x7f7780108990 2024-03-11T19:51:07.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.426+0000 7f777cff9640 1 -- 172.21.15.83:0/2385222785 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77740052d0 con 0x7f7780108990 2024-03-11T19:51:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.426+0000 7f7785c0f640 1 -- 172.21.15.83:0/2385222785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77801a58e0 con 0x7f7780108990 2024-03-11T19:51:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.426+0000 7f777cff9640 1 -- 172.21.15.83:0/2385222785 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774003cc0 con 0x7f7780108990 2024-03-11T19:51:07.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.426+0000 7f7785c0f640 1 -- 172.21.15.83:0/2385222785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77801a5df0 con 0x7f7780108990 2024-03-11T19:51:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.428+0000 7f777cff9640 1 -- 172.21.15.83:0/2385222785 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7774005440 con 0x7f7780108990 2024-03-11T19:51:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.428+0000 7f77887da740 1 Processor -- start 2024-03-11T19:51:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.428+0000 7f77887da740 1 -- start start 2024-03-11T19:51:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f77887da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894e270 0x5555e894e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f777cff9640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f77680a1a30 0x7f77680a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f77887da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 0x5555e8953760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f77887da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e8956260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f77887da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e890a960 con 0x5555e894e270 2024-03-11T19:51:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f77887da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e88be310 con 0x5555e894ec60 2024-03-11T19:51:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f77887da740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e88bb840 con 0x5555e8953dd0 2024-03-11T19:51:07.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f777f7fe640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f77680a1a30 0x7f77680a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f777ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e8956260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f777cff9640 1 -- 172.21.15.83:0/2385222785 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f77740ef570 con 0x7f7780108990 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f777effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 0x5555e8953760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f777ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894e270 0x5555e894e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f777effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 0x5555e8953760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32980/0 (socket says 172.21.15.83:32980) 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f777effd640 1 -- 172.21.15.83:0/3408095371 learned_addr learned my addr 172.21.15.83:0/3408095371 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.429+0000 7f777ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894e270 0x5555e894e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40376/0 (socket says 172.21.15.83:40376) 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.430+0000 7f777effd640 1 -- 172.21.15.83:0/3408095371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 msgr2=0x5555e8956260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.430+0000 7f777effd640 1 --2- 172.21.15.83:0/3408095371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e8956260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.430+0000 7f777effd640 1 -- 172.21.15.83:0/3408095371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894e270 msgr2=0x5555e894e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.430+0000 7f777effd640 1 --2- 172.21.15.83:0/3408095371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894e270 0x5555e894e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.430+0000 7f777effd640 1 -- 172.21.15.83:0/3408095371 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555e8879060 con 0x5555e894ec60 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.430+0000 7f777f7fe640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f77680a1a30 0x7f77680a3ef0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f77801087f0 tx=0x7f7770005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.430+0000 7f777effd640 1 --2- 172.21.15.83:0/3408095371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 0x5555e8953760 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f77740b8f60 tx=0x7f777402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17a596a691f51989 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.430+0000 7f7765ffb640 1 -- 172.21.15.83:0/3408095371 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740bc0d0 con 0x5555e894ec60 2024-03-11T19:51:07.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.430+0000 7f7765ffb640 1 -- 172.21.15.83:0/3408095371 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77740bbef0 con 0x5555e894ec60 2024-03-11T19:51:07.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.430+0000 7f7765ffb640 1 -- 172.21.15.83:0/3408095371 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740c4dc0 con 0x5555e894ec60 2024-03-11T19:51:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.431+0000 7f77887da740 1 -- 172.21.15.83:0/3408095371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 msgr2=0x5555e8953760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.431+0000 7f77887da740 1 --2- 172.21.15.83:0/3408095371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 0x5555e8953760 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f77740b8f60 tx=0x7f777402bc30 comp rx=0 tx=0).stop 2024-03-11T19:51:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.431+0000 7f77887da740 1 -- 172.21.15.83:0/3408095371 shutdown_connections 2024-03-11T19:51:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.431+0000 7f77887da740 1 --2- 172.21.15.83:0/3408095371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e8956260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.431+0000 7f77887da740 1 --2- 172.21.15.83:0/3408095371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 0x5555e8953760 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.431+0000 7f77887da740 1 --2- 172.21.15.83:0/3408095371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894e270 0x5555e894e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.432+0000 7f77887da740 1 -- 172.21.15.83:0/3408095371 >> 172.21.15.83:0/3408095371 conn(0x5555e89497e0 msgr2=0x5555e8949be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.432+0000 7f77887da740 1 -- 172.21.15.83:0/3408095371 shutdown_connections 2024-03-11T19:51:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.432+0000 7f77887da740 1 -- 172.21.15.83:0/3408095371 wait complete. 2024-03-11T19:51:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.433+0000 7f77887da740 1 -- start start 2024-03-11T19:51:07.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.433+0000 7f77887da740 1 Processor -- start 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.433+0000 7f77887da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894e270 0x5555e8958070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.433+0000 7f77887da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 0x5555e89585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.433+0000 7f77887da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e895f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.433+0000 7f77887da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e88b7020 con 0x5555e894ec60 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.433+0000 7f77887da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e858a430 con 0x5555e894e270 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.433+0000 7f77887da740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e8546e00 con 0x5555e8953dd0 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.433+0000 7f77887da740 10 client.0 _ll_register_callbacks cb 0x5555e885ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.433+0000 7f777f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894e270 0x5555e8958070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f777effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 0x5555e89585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f777f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894e270 0x5555e8958070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:32994/0 (socket says 172.21.15.83:32994) 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f777f7fe640 1 -- 172.21.15.83:0/686771887 learned_addr learned my addr 172.21.15.83:0/686771887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f777ffff640 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e895f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f777f7fe640 1 -- 172.21.15.83:0/686771887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 msgr2=0x5555e895f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f777f7fe640 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e895f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f777f7fe640 1 -- 172.21.15.83:0/686771887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 msgr2=0x5555e89585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f777f7fe640 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 0x5555e89585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f777f7fe640 1 -- 172.21.15.83:0/686771887 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77801a5df0 con 0x5555e894e270 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f777effd640 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 0x5555e89585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f777f7fe640 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894e270 0x5555e8958070 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f777000fb90 tx=0x7f7770004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a93abf8f7b0488c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f775effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f775effd640 1 -- 172.21.15.83:0/686771887 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777001cbd0 con 0x5555e894e270 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f775effd640 1 -- 172.21.15.83:0/686771887 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f777001cd30 con 0x5555e894e270 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f77887da740 1 -- 172.21.15.83:0/686771887 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555e85b67e0 con 0x5555e894e270 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f775effd640 1 -- 172.21.15.83:0/686771887 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7770026420 con 0x5555e894e270 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f77887da740 10 client.27029 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.434+0000 7f77887da740 1 -- 172.21.15.83:0/686771887 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5555e88b56a0 con 0x5555e894e270 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.435+0000 7f775effd640 1 -- 172.21.15.83:0/686771887 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5555e85b67e0 con 0x5555e894e270 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.435+0000 7f775effd640 1 -- 172.21.15.83:0/686771887 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f777000e4d0 con 0x5555e894e270 2024-03-11T19:51:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.435+0000 7f77887da740 10 client.27029 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.435+0000 7f77887da740 10 client.27029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:51:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.435+0000 7f77887da740 20 client.27029 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.435+0000 7f77887da740 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5555e89642a0 0x5555e89666c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.435+0000 7f77887da740 4 client.27029 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.435+0000 7f77887da740 1 -- 172.21.15.83:0/686771887 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555e85f3720 con 0x5555e89642a0 2024-03-11T19:51:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.435+0000 7f777effd640 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5555e89642a0 0x5555e89666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f775effd640 10 client.27029 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:51:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.435+0000 7f777effd640 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5555e89642a0 0x5555e89666c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cbf659620517b46 server_cookie=7624f6e81109d87e in_seq=0 out_seq=0 2024-03-11T19:51:07.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f775effd640 1 -- 172.21.15.83:0/686771887 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5555e85f3720 con 0x5555e89642a0 2024-03-11T19:51:07.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f775effd640 10 client.27029 handle_command_reply: tid=0 2024-03-11T19:51:07.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f77887da740 1 client.27029 shutdown 2024-03-11T19:51:07.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f77887da740 1 -- 172.21.15.83:0/686771887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894e270 msgr2=0x5555e8958070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f77887da740 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894e270 0x5555e8958070 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f777000fb90 tx=0x7f7770004730 comp rx=0 tx=0).stop 2024-03-11T19:51:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f77887da740 1 -- 172.21.15.83:0/686771887 shutdown_connections 2024-03-11T19:51:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f77887da740 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5555e89642a0 0x5555e89666c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f775effd640 0 client.27029 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:51:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f77887da740 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e895f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f77887da740 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 0x5555e89585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f77887da740 1 --2- 172.21.15.83:0/686771887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894e270 0x5555e8958070 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f77887da740 1 -- 172.21.15.83:0/686771887 >> 172.21.15.83:0/686771887 conn(0x5555e89497e0 msgr2=0x5555e8948b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.436+0000 7f77887da740 1 -- 172.21.15.83:0/686771887 shutdown_connections 2024-03-11T19:51:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.437+0000 7f77887da740 1 -- 172.21.15.83:0/686771887 wait complete. 2024-03-11T19:51:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.437+0000 7f77887da740 20 client.27029 trim_cache size 0 max 16384 2024-03-11T19:51:07.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.456+0000 7f77887da740 1 Processor -- start 2024-03-11T19:51:07.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.456+0000 7f77887da740 1 -- start start 2024-03-11T19:51:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.456+0000 7f77887da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894e270 0x5555e8953690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.456+0000 7f77887da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 0x5555e895f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.456+0000 7f77887da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e8957c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.456+0000 7f77887da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e883c130 con 0x5555e894ec60 2024-03-11T19:51:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.456+0000 7f77887da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e85b6810 con 0x5555e894e270 2024-03-11T19:51:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.456+0000 7f77887da740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e893ba40 con 0x5555e8953dd0 2024-03-11T19:51:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.456+0000 7f777effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 0x5555e895f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.456+0000 7f777effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 0x5555e895f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40388/0 (socket says 172.21.15.83:40388) 2024-03-11T19:51:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.456+0000 7f777effd640 1 -- 172.21.15.83:0/371728008 learned_addr learned my addr 172.21.15.83:0/371728008 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.457+0000 7f777ffff640 1 --2- 172.21.15.83:0/371728008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e8957c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.457+0000 7f777ffff640 1 -- 172.21.15.83:0/371728008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894e270 msgr2=0x5555e8953690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:51:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.457+0000 7f777ffff640 1 --2- 172.21.15.83:0/371728008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894e270 0x5555e8953690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.457+0000 7f777ffff640 1 -- 172.21.15.83:0/371728008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 msgr2=0x5555e895f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.457+0000 7f777ffff640 1 --2- 172.21.15.83:0/371728008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 0x5555e895f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.457+0000 7f777ffff640 1 -- 172.21.15.83:0/371728008 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555e8851c20 con 0x5555e8953dd0 2024-03-11T19:51:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.457+0000 7f777ffff640 1 --2- 172.21.15.83:0/371728008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e8957c80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5555e894f4c0 tx=0x7f7778004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d510a95f1d7ffd80 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.457+0000 7f77657fa640 1 -- 172.21.15.83:0/371728008 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7778004d20 con 0x5555e8953dd0 2024-03-11T19:51:07.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.458+0000 7f77657fa640 1 -- 172.21.15.83:0/371728008 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7778004e80 con 0x5555e8953dd0 2024-03-11T19:51:07.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.458+0000 7f77657fa640 1 -- 172.21.15.83:0/371728008 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777801e410 con 0x5555e8953dd0 2024-03-11T19:51:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.458+0000 7f77887da740 1 -- 172.21.15.83:0/371728008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 msgr2=0x5555e8957c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.458+0000 7f77887da740 1 --2- 172.21.15.83:0/371728008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e8957c80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5555e894f4c0 tx=0x7f7778004ab0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.458+0000 7f77887da740 1 -- 172.21.15.83:0/371728008 shutdown_connections 2024-03-11T19:51:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.458+0000 7f77887da740 1 --2- 172.21.15.83:0/371728008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e8953dd0 0x5555e8957c80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.458+0000 7f77887da740 1 --2- 172.21.15.83:0/371728008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e894ec60 0x5555e895f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.458+0000 7f77887da740 1 --2- 172.21.15.83:0/371728008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894e270 0x5555e8953690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.458+0000 7f77887da740 1 -- 172.21.15.83:0/371728008 >> 172.21.15.83:0/371728008 conn(0x5555e89497e0 msgr2=0x5555e8949bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.458+0000 7f77887da740 1 -- 172.21.15.83:0/371728008 shutdown_connections 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.458+0000 7f77887da740 1 -- 172.21.15.83:0/371728008 wait complete. 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f77887da740 1 -- start start 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f77887da740 1 Processor -- start 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f77887da740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e894e270 0x5555e895deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f77887da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 0x5555e895e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f77887da740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e8953dd0 0x5555e895e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f77887da740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e89cb860 con 0x5555e8953dd0 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f77887da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e89cb9d0 con 0x5555e894ec60 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f77887da740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555e89cbb90 con 0x5555e894e270 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f77887da740 10 client.0 _ll_register_callbacks cb 0x7f7774005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f777ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e8953dd0 0x5555e895e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f777ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e8953dd0 0x5555e895e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40404/0 (socket says 172.21.15.83:40404) 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f777ffff640 1 -- 172.21.15.83:0/1021625155 learned_addr learned my addr 172.21.15.83:0/1021625155 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f777f7fe640 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e894e270 0x5555e895deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.459+0000 7f777effd640 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 0x5555e895e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f777ffff640 1 -- 172.21.15.83:0/1021625155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e894e270 msgr2=0x5555e895deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f777ffff640 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e894e270 0x5555e895deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f777ffff640 1 -- 172.21.15.83:0/1021625155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 msgr2=0x5555e895e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f777ffff640 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 0x5555e895e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f777ffff640 1 -- 172.21.15.83:0/1021625155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77780107f0 con 0x5555e8953dd0 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f777ffff640 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e8953dd0 0x5555e895e930 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7778004a80 tx=0x7f777800fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c428c21f8f2a8f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f775effd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f775effd640 1 -- 172.21.15.83:0/1021625155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777800fee0 con 0x5555e8953dd0 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f775effd640 1 -- 172.21.15.83:0/1021625155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7778004d20 con 0x5555e8953dd0 2024-03-11T19:51:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f77887da740 1 -- 172.21.15.83:0/1021625155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77740bbef0 con 0x5555e8953dd0 2024-03-11T19:51:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f775effd640 1 -- 172.21.15.83:0/1021625155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7778004f20 con 0x5555e8953dd0 2024-03-11T19:51:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f77887da740 10 client.17583 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f77887da740 1 -- 172.21.15.83:0/1021625155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5555e88778c0 con 0x5555e8953dd0 2024-03-11T19:51:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.460+0000 7f775effd640 1 -- 172.21.15.83:0/1021625155 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77740bbef0 con 0x5555e8953dd0 2024-03-11T19:51:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.461+0000 7f775effd640 1 -- 172.21.15.83:0/1021625155 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7778026040 con 0x5555e8953dd0 2024-03-11T19:51:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.461+0000 7f77887da740 10 client.17583 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.461+0000 7f77887da740 10 client.17583 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:51:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.461+0000 7f77887da740 20 client.17583 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.461+0000 7f77887da740 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5555e89cd640 0x5555e89cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.461+0000 7f77887da740 4 client.17583 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:51:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.461+0000 7f77887da740 1 -- 172.21.15.83:0/1021625155 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5555e8642890 con 0x5555e89cd640 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.461+0000 7f777f7fe640 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5555e89cd640 0x5555e89cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.461+0000 7f777f7fe640 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5555e89cd640 0x5555e89cfa60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=818c3021af295e9c server_cookie=371b0b47c497d6a3 in_seq=0 out_seq=0 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.461+0000 7f775effd640 10 client.17583 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f775effd640 1 -- 172.21.15.83:0/1021625155 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5555e8642890 con 0x5555e89cd640 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f775effd640 10 client.17583 handle_command_reply: tid=0 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 1 client.17583 shutdown 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 1 -- 172.21.15.83:0/1021625155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e8953dd0 msgr2=0x5555e895e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e8953dd0 0x5555e895e930 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7778004a80 tx=0x7f777800fb60 comp rx=0 tx=0).stop 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 1 -- 172.21.15.83:0/1021625155 shutdown_connections 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5555e89cd640 0x5555e89cfa60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555e8953dd0 0x5555e895e930 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f775effd640 0 client.17583 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555e894ec60 0x5555e895e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 1 --2- 172.21.15.83:0/1021625155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555e894e270 0x5555e895deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 1 -- 172.21.15.83:0/1021625155 >> 172.21.15.83:0/1021625155 conn(0x5555e89497e0 msgr2=0x5555e8957c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 1 -- 172.21.15.83:0/1021625155 shutdown_connections 2024-03-11T19:51:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 1 -- 172.21.15.83:0/1021625155 wait complete. 2024-03-11T19:51:07.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.462+0000 7f77887da740 20 client.17583 trim_cache size 0 max 16384 2024-03-11T19:51:07.472 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 -- 172.21.15.83:0/2385222785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f77680a1a30 msgr2=0x7f77680a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f77680a1a30 0x7f77680a3ef0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f77801087f0 tx=0x7f7770005f90 comp rx=0 tx=0).stop 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 -- 172.21.15.83:0/2385222785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7780108990 msgr2=0x7f778019dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7780108990 0x7f778019dd80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7774006010 tx=0x7f7774005e80 comp rx=0 tx=0).stop 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 -- 172.21.15.83:0/2385222785 shutdown_connections 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f77680a1a30 0x7f77680a3ef0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780109350 0x7f77801a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7780108990 0x7f778019dd80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 --2- 172.21.15.83:0/2385222785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77801077a0 0x7f778019d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 -- 172.21.15.83:0/2385222785 >> 172.21.15.83:0/2385222785 conn(0x7f7780076740 msgr2=0x7f778010bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 -- 172.21.15.83:0/2385222785 shutdown_connections 2024-03-11T19:51:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.463+0000 7f7785c0f640 1 -- 172.21.15.83:0/2385222785 wait complete. 2024-03-11T19:51:07.480 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:51:07.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:51:07.499 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 710.0 s, 1607.0 tps, lat 61.878 ms stddev 68.605 2024-03-11T19:51:07.549 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:07 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/4053994132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:51:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.658+0000 7f4118b25640 1 -- 172.21.15.83:0/558947963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4114100090 msgr2=0x7f41141004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.658+0000 7f4118b25640 1 --2- 172.21.15.83:0/558947963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4114100090 0x7f41141004b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4104009900 tx=0x7f410402bc30 comp rx=0 tx=0).stop 2024-03-11T19:51:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.659+0000 7f4118b25640 1 -- 172.21.15.83:0/558947963 shutdown_connections 2024-03-11T19:51:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.659+0000 7f4118b25640 1 --2- 172.21.15.83:0/558947963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4114101c40 0x7f41141084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.659+0000 7f4118b25640 1 --2- 172.21.15.83:0/558947963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4114101280 0x7f4114101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.659+0000 7f4118b25640 1 --2- 172.21.15.83:0/558947963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4114100090 0x7f41141004b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.659+0000 7f4118b25640 1 -- 172.21.15.83:0/558947963 >> 172.21.15.83:0/558947963 conn(0x7f41140fb620 msgr2=0x7f41140fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.659+0000 7f4118b25640 1 -- 172.21.15.83:0/558947963 shutdown_connections 2024-03-11T19:51:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.659+0000 7f4118b25640 1 -- 172.21.15.83:0/558947963 wait complete. 2024-03-11T19:51:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.659+0000 7f4118b25640 1 Processor -- start 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.659+0000 7f4118b25640 1 -- start start 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4118b25640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4114100090 0x7f41141972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4118b25640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4114101280 0x7f4114197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4118b25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4114101c40 0x7f411419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4118b25640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411419edb0 con 0x7f4114101c40 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4118b25640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411419ef20 con 0x7f4114100090 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4118b25640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411419f0e0 con 0x7f4114101280 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4112d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4114101c40 0x7f411419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4112d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4114101c40 0x7f411419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40424/0 (socket says 172.21.15.83:40424) 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4112d9d640 1 -- 172.21.15.83:0/250867522 learned_addr learned my addr 172.21.15.83:0/250867522 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f411259c640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4114100090 0x7f41141972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f410bfff640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4114101280 0x7f4114197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4112d9d640 1 -- 172.21.15.83:0/250867522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4114101280 msgr2=0x7f4114197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4112d9d640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4114101280 0x7f4114197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4112d9d640 1 -- 172.21.15.83:0/250867522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4114100090 msgr2=0x7f41141972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4112d9d640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4114100090 0x7f41141972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4112d9d640 1 -- 172.21.15.83:0/250867522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4104009560 con 0x7f4114101c40 2024-03-11T19:51:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f411259c640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4114100090 0x7f41141972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4112d9d640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4114101c40 0x7f411419e870 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f410c009fd0 tx=0x7f410c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f410b7fe640 1 -- 172.21.15.83:0/250867522 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c00bea0 con 0x7f4114101c40 2024-03-11T19:51:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f410b7fe640 1 -- 172.21.15.83:0/250867522 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f410c009940 con 0x7f4114101c40 2024-03-11T19:51:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f4118b25640 1 -- 172.21.15.83:0/250867522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411419f3c0 con 0x7f4114101c40 2024-03-11T19:51:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.660+0000 7f410b7fe640 1 -- 172.21.15.83:0/250867522 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0106c0 con 0x7f4114101c40 2024-03-11T19:51:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.661+0000 7f4118b25640 1 -- 172.21.15.83:0/250867522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411419f880 con 0x7f4114101c40 2024-03-11T19:51:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.662+0000 7f410b7fe640 1 -- 172.21.15.83:0/250867522 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f410c004240 con 0x7f4114101c40 2024-03-11T19:51:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f411b7d2740 1 Processor -- start 2024-03-11T19:51:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f411b7d2740 1 -- start start 2024-03-11T19:51:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5c680 0x55d3c9c5caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 0x55d3c9c61b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 0x55d3c9c64670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f411b7d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9c14960 con 0x55d3c9c621e0 2024-03-11T19:51:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f411b7d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9bc8310 con 0x55d3c9c5c680 2024-03-11T19:51:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f411b7d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9bc5840 con 0x55d3c9c5d070 2024-03-11T19:51:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f410b7fe640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f41000a1a30 0x7f41000a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f410bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 0x55d3c9c64670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f411259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 0x55d3c9c61b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f4112d9d640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f41000a1a30 0x7f41000a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f411259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 0x55d3c9c61b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57758/0 (socket says 172.21.15.83:57758) 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f410bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 0x55d3c9c64670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40430/0 (socket says 172.21.15.83:40430) 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f411259c640 1 -- 172.21.15.83:0/728897121 learned_addr learned my addr 172.21.15.83:0/728897121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.663+0000 7f410bfff640 1 --2- 172.21.15.83:0/728897121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5c680 0x55d3c9c5caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.664+0000 7f410b7fe640 1 -- 172.21.15.83:0/250867522 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f410c0ccfc0 con 0x7f4114101c40 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.664+0000 7f411259c640 1 -- 172.21.15.83:0/728897121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5c680 msgr2=0x55d3c9c5caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.664+0000 7f411259c640 1 --2- 172.21.15.83:0/728897121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5c680 0x55d3c9c5caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.664+0000 7f411259c640 1 -- 172.21.15.83:0/728897121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 msgr2=0x55d3c9c64670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.664+0000 7f411259c640 1 --2- 172.21.15.83:0/728897121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 0x55d3c9c64670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.664+0000 7f411259c640 1 -- 172.21.15.83:0/728897121 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3c9b83060 con 0x55d3c9c5d070 2024-03-11T19:51:07.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.665+0000 7f411259c640 1 --2- 172.21.15.83:0/728897121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 0x55d3c9c61b70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4104009900 tx=0x7f4104005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c141d2d68c01cb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.666+0000 7f4112d9d640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f41000a1a30 0x7f41000a3ef0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f410c06d360 tx=0x7f410c06d390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.666+0000 7f4108ff9640 1 -- 172.21.15.83:0/728897121 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104004800 con 0x55d3c9c5d070 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.666+0000 7f4108ff9640 1 -- 172.21.15.83:0/728897121 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4104004960 con 0x55d3c9c5d070 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.666+0000 7f4108ff9640 1 -- 172.21.15.83:0/728897121 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41040035f0 con 0x55d3c9c5d070 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.667+0000 7f411b7d2740 1 -- 172.21.15.83:0/728897121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 msgr2=0x55d3c9c61b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.667+0000 7f411b7d2740 1 --2- 172.21.15.83:0/728897121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 0x55d3c9c61b70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4104009900 tx=0x7f4104005d50 comp rx=0 tx=0).stop 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.667+0000 7f411b7d2740 1 -- 172.21.15.83:0/728897121 shutdown_connections 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.667+0000 7f411b7d2740 1 --2- 172.21.15.83:0/728897121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 0x55d3c9c64670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.667+0000 7f411b7d2740 1 --2- 172.21.15.83:0/728897121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 0x55d3c9c61b70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.667+0000 7f411b7d2740 1 --2- 172.21.15.83:0/728897121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5c680 0x55d3c9c5caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.667+0000 7f411b7d2740 1 -- 172.21.15.83:0/728897121 >> 172.21.15.83:0/728897121 conn(0x55d3c9c537e0 msgr2=0x55d3c9c53be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.667+0000 7f411b7d2740 1 -- 172.21.15.83:0/728897121 shutdown_connections 2024-03-11T19:51:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.667+0000 7f411b7d2740 1 -- 172.21.15.83:0/728897121 wait complete. 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.667+0000 7f411b7d2740 1 -- start start 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.667+0000 7f411b7d2740 1 Processor -- start 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5c680 0x55d3c9c66840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5d070 0x55d3c9c66d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c621e0 0x55d3c9c6ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411b7d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9bc1020 con 0x55d3c9c5d070 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411b7d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9894430 con 0x55d3c9c621e0 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411b7d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9850e00 con 0x55d3c9c5c680 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411b7d2740 10 client.0 _ll_register_callbacks cb 0x55d3c9b68cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5d070 0x55d3c9c66d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5d070 0x55d3c9c66d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40440/0 (socket says 172.21.15.83:40440) 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411259c640 1 -- 172.21.15.83:0/964277026 learned_addr learned my addr 172.21.15.83:0/964277026 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f410bfff640 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5c680 0x55d3c9c66840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f410bfff640 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c621e0 0x55d3c9c6ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411259c640 1 -- 172.21.15.83:0/964277026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5c680 msgr2=0x55d3c9c66840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411259c640 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5c680 0x55d3c9c66840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411259c640 1 -- 172.21.15.83:0/964277026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c621e0 msgr2=0x55d3c9c6ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411259c640 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c621e0 0x55d3c9c6ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411259c640 1 -- 172.21.15.83:0/964277026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4114108dd0 con 0x55d3c9c5d070 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f411259c640 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5d070 0x55d3c9c66d80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f410402c170 tx=0x7f4104005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d794d1817880b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f40f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f40f9ffb640 1 -- 172.21.15.83:0/964277026 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104004450 con 0x55d3c9c5d070 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.668+0000 7f40f9ffb640 1 -- 172.21.15.83:0/964277026 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f410404e070 con 0x55d3c9c5d070 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f411b7d2740 1 -- 172.21.15.83:0/964277026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3c98c07e0 con 0x55d3c9c5d070 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f40f9ffb640 1 -- 172.21.15.83:0/964277026 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41040050f0 con 0x55d3c9c5d070 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f411b7d2740 10 client.17595 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f411b7d2740 1 -- 172.21.15.83:0/964277026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d3c9bbf6a0 con 0x55d3c9c5d070 2024-03-11T19:51:07.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f40f9ffb640 1 -- 172.21.15.83:0/964277026 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d3c98c07e0 con 0x55d3c9c5d070 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f40f9ffb640 1 -- 172.21.15.83:0/964277026 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f410405b070 con 0x55d3c9c5d070 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f411b7d2740 10 client.17595 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f411b7d2740 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f411b7d2740 20 client.17595 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f411b7d2740 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d3c9c72640 0x55d3c9c74a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f411b7d2740 4 client.17595 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f411b7d2740 1 -- 172.21.15.83:0/964277026 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3c98fd720 con 0x55d3c9c72640 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.669+0000 7f410bfff640 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d3c9c72640 0x55d3c9c74a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f410bfff640 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d3c9c72640 0x55d3c9c74a60 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f00f59ab3f613bb server_cookie=8e93d2445b563318 in_seq=0 out_seq=0 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f40f9ffb640 10 client.17595 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f40f9ffb640 1 -- 172.21.15.83:0/964277026 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d3c98fd720 con 0x55d3c9c72640 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f40f9ffb640 10 client.17595 handle_command_reply: tid=0 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f411b7d2740 1 client.17595 shutdown 2024-03-11T19:51:07.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f411b7d2740 1 -- 172.21.15.83:0/964277026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5d070 msgr2=0x55d3c9c66d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f411b7d2740 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5d070 0x55d3c9c66d80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f410402c170 tx=0x7f4104005000 comp rx=0 tx=0).stop 2024-03-11T19:51:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f411b7d2740 1 -- 172.21.15.83:0/964277026 shutdown_connections 2024-03-11T19:51:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f411b7d2740 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d3c9c72640 0x55d3c9c74a60 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f411b7d2740 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c621e0 0x55d3c9c6ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f411b7d2740 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5d070 0x55d3c9c66d80 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f411b7d2740 1 --2- 172.21.15.83:0/964277026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5c680 0x55d3c9c66840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f411b7d2740 1 -- 172.21.15.83:0/964277026 >> 172.21.15.83:0/964277026 conn(0x55d3c9c537e0 msgr2=0x55d3c9c52b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f40f9ffb640 0 client.17595 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.670+0000 7f411b7d2740 1 -- 172.21.15.83:0/964277026 shutdown_connections 2024-03-11T19:51:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.671+0000 7f411b7d2740 1 -- 172.21.15.83:0/964277026 wait complete. 2024-03-11T19:51:07.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.671+0000 7f411b7d2740 20 client.17595 trim_cache size 0 max 16384 2024-03-11T19:51:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f411b7d2740 1 Processor -- start 2024-03-11T19:51:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f411b7d2740 1 -- start start 2024-03-11T19:51:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5c680 0x55d3c9c793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5d070 0x55d3c9c6dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c621e0 0x55d3c9c5e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f411b7d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9b46130 con 0x55d3c9c5c680 2024-03-11T19:51:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f411b7d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9c45a40 con 0x55d3c9c5d070 2024-03-11T19:51:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f411b7d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9c54750 con 0x55d3c9c621e0 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f410bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5c680 0x55d3c9c793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f410bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5c680 0x55d3c9c793c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40446/0 (socket says 172.21.15.83:40446) 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f410bfff640 1 -- 172.21.15.83:0/199993533 learned_addr learned my addr 172.21.15.83:0/199993533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.685+0000 7f411259c640 1 --2- 172.21.15.83:0/199993533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5d070 0x55d3c9c6dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.686+0000 7f410bfff640 1 --2- 172.21.15.83:0/199993533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c621e0 0x55d3c9c5e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.686+0000 7f410bfff640 1 -- 172.21.15.83:0/199993533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c621e0 msgr2=0x55d3c9c5e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.686+0000 7f410bfff640 1 --2- 172.21.15.83:0/199993533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c621e0 0x55d3c9c5e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.686+0000 7f410bfff640 1 -- 172.21.15.83:0/199993533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5d070 msgr2=0x55d3c9c6dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.686+0000 7f410bfff640 1 --2- 172.21.15.83:0/199993533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5d070 0x55d3c9c6dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.686+0000 7f410bfff640 1 -- 172.21.15.83:0/199993533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3c9b5bc20 con 0x55d3c9c5c680 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.686+0000 7f410bfff640 1 --2- 172.21.15.83:0/199993533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5c680 0x55d3c9c793c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009730 tx=0x7f40fc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295a4f54aab3d9a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.686+0000 7f40fbfff640 1 -- 172.21.15.83:0/199993533 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc016040 con 0x55d3c9c5c680 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.686+0000 7f40fbfff640 1 -- 172.21.15.83:0/199993533 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40fc026e90 con 0x55d3c9c5c680 2024-03-11T19:51:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.686+0000 7f40fbfff640 1 -- 172.21.15.83:0/199993533 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc004990 con 0x55d3c9c5c680 2024-03-11T19:51:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.687+0000 7f411b7d2740 1 -- 172.21.15.83:0/199993533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5c680 msgr2=0x55d3c9c793c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.687+0000 7f411b7d2740 1 --2- 172.21.15.83:0/199993533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5c680 0x55d3c9c793c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009730 tx=0x7f40fc005f90 comp rx=0 tx=0).stop 2024-03-11T19:51:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.688+0000 7f411b7d2740 1 -- 172.21.15.83:0/199993533 shutdown_connections 2024-03-11T19:51:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.688+0000 7f411b7d2740 1 --2- 172.21.15.83:0/199993533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c621e0 0x55d3c9c5e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.688+0000 7f411b7d2740 1 --2- 172.21.15.83:0/199993533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5d070 0x55d3c9c6dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.688+0000 7f411b7d2740 1 --2- 172.21.15.83:0/199993533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c5c680 0x55d3c9c793c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.688+0000 7f411b7d2740 1 -- 172.21.15.83:0/199993533 >> 172.21.15.83:0/199993533 conn(0x55d3c9c537e0 msgr2=0x55d3c9c65e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.688+0000 7f411b7d2740 1 -- 172.21.15.83:0/199993533 shutdown_connections 2024-03-11T19:51:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.688+0000 7f411b7d2740 1 -- 172.21.15.83:0/199993533 wait complete. 2024-03-11T19:51:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.689+0000 7f411b7d2740 1 -- start start 2024-03-11T19:51:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.689+0000 7f411b7d2740 1 Processor -- start 2024-03-11T19:51:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.689+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5c680 0x55d3c9c6bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.689+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 0x55d3c9c6c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.689+0000 7f411b7d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 0x55d3c9c6d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.689+0000 7f411b7d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9c6d7f0 con 0x55d3c9c621e0 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.689+0000 7f411b7d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9c6d960 con 0x55d3c9c5c680 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.689+0000 7f411b7d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3c9c6db20 con 0x55d3c9c5d070 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.689+0000 7f411b7d2740 10 client.0 _ll_register_callbacks cb 0x7f4104009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f410bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 0x55d3c9c6d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f411259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 0x55d3c9c6c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f410bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 0x55d3c9c6d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40452/0 (socket says 172.21.15.83:40452) 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f410bfff640 1 -- 172.21.15.83:0/346520972 learned_addr learned my addr 172.21.15.83:0/346520972 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f411259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 0x55d3c9c6c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57792/0 (socket says 172.21.15.83:57792) 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f410bfff640 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5c680 0x55d3c9c6bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f410bfff640 1 -- 172.21.15.83:0/346520972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 msgr2=0x55d3c9c6c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f410bfff640 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 0x55d3c9c6c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f410bfff640 1 -- 172.21.15.83:0/346520972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5c680 msgr2=0x55d3c9c6bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f410bfff640 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5c680 0x55d3c9c6bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f410bfff640 1 -- 172.21.15.83:0/346520972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc019040 con 0x55d3c9c621e0 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f410bfff640 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 0x55d3c9c6d2b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f40fc014010 tx=0x7f40fc009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7333e4f8db7b91 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f40f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f40f9ffb640 1 -- 172.21.15.83:0/346520972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc016040 con 0x55d3c9c621e0 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f40f9ffb640 1 -- 172.21.15.83:0/346520972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f40fc0046c0 con 0x55d3c9c621e0 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f411b7d2740 1 -- 172.21.15.83:0/346520972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4104004960 con 0x55d3c9c621e0 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f40f9ffb640 1 -- 172.21.15.83:0/346520972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40fc0048c0 con 0x55d3c9c621e0 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f411b7d2740 10 client.17601 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.690+0000 7f411b7d2740 1 -- 172.21.15.83:0/346520972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d3c9b818c0 con 0x55d3c9c621e0 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.691+0000 7f40f9ffb640 1 -- 172.21.15.83:0/346520972 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4104004960 con 0x55d3c9c621e0 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.691+0000 7f40f9ffb640 1 -- 172.21.15.83:0/346520972 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40fc0265b0 con 0x55d3c9c621e0 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.691+0000 7f411b7d2740 10 client.17601 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:07.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.691+0000 7f411b7d2740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.691+0000 7f411b7d2740 20 client.17601 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.691+0000 7f411b7d2740 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d3c9cc9cf0 0x55d3c9ccc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.691+0000 7f411b7d2740 4 client.17601 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.691+0000 7f411b7d2740 1 -- 172.21.15.83:0/346520972 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d3c994c890 con 0x55d3c9cc9cf0 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.691+0000 7f411259c640 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d3c9cc9cf0 0x55d3c9ccc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.691+0000 7f40f9ffb640 10 client.17601 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.691+0000 7f411259c640 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d3c9cc9cf0 0x55d3c9ccc110 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f0c725c78201e64 server_cookie=69c9149a79acb8ea in_seq=0 out_seq=0 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f40f9ffb640 1 -- 172.21.15.83:0/346520972 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d3c994c890 con 0x55d3c9cc9cf0 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f40f9ffb640 10 client.17601 handle_command_reply: tid=0 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 1 client.17601 shutdown 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 1 -- 172.21.15.83:0/346520972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 msgr2=0x55d3c9c6d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 0x55d3c9c6d2b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f40fc014010 tx=0x7f40fc009ac0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 1 -- 172.21.15.83:0/346520972 shutdown_connections 2024-03-11T19:51:07.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d3c9cc9cf0 0x55d3c9ccc110 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3c9c621e0 0x55d3c9c6d2b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f40f9ffb640 0 client.17601 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d3c9c5d070 0x55d3c9c6c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 1 --2- 172.21.15.83:0/346520972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3c9c5c680 0x55d3c9c6bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 1 -- 172.21.15.83:0/346520972 >> 172.21.15.83:0/346520972 conn(0x55d3c9c537e0 msgr2=0x55d3c9c5e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 1 -- 172.21.15.83:0/346520972 shutdown_connections 2024-03-11T19:51:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 1 -- 172.21.15.83:0/346520972 wait complete. 2024-03-11T19:51:07.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f411b7d2740 20 client.17601 trim_cache size 0 max 16384 2024-03-11T19:51:07.701 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:51:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f4118b25640 1 -- 172.21.15.83:0/250867522 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f41000a1a30 msgr2=0x7f41000a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f4118b25640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f41000a1a30 0x7f41000a3ef0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f410c06d360 tx=0x7f410c06d390 comp rx=0 tx=0).stop 2024-03-11T19:51:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f4118b25640 1 -- 172.21.15.83:0/250867522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4114101c40 msgr2=0x7f411419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:07.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.692+0000 7f4118b25640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4114101c40 0x7f411419e870 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f410c009fd0 tx=0x7f410c00b260 comp rx=0 tx=0).stop 2024-03-11T19:51:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.693+0000 7f4118b25640 1 -- 172.21.15.83:0/250867522 shutdown_connections 2024-03-11T19:51:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.693+0000 7f4118b25640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f41000a1a30 0x7f41000a3ef0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.693+0000 7f4118b25640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4114101c40 0x7f411419e870 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.693+0000 7f4118b25640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4114101280 0x7f4114197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.693+0000 7f4118b25640 1 --2- 172.21.15.83:0/250867522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4114100090 0x7f41141972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.693+0000 7f4118b25640 1 -- 172.21.15.83:0/250867522 >> 172.21.15.83:0/250867522 conn(0x7f41140fb620 msgr2=0x7f4114069750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.693+0000 7f4118b25640 1 -- 172.21.15.83:0/250867522 shutdown_connections 2024-03-11T19:51:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:07.693+0000 7f4118b25640 1 -- 172.21.15.83:0/250867522 wait complete. 2024-03-11T19:51:07.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:07 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/4053994132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:51:07.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:07 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/4053994132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:51:08.715 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:51:08.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:51:08.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:51:08.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:08 smithi083 ceph-mon[31864]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 MiB/s wr, 597 op/s 2024-03-11T19:51:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.854+0000 7f35db7d7640 1 -- 172.21.15.83:0/2103849370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d40ff830 msgr2=0x7f35d40ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.854+0000 7f35db7d7640 1 --2- 172.21.15.83:0/2103849370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d40ff830 0x7f35d40ffc50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d960 tx=0x7f35d002db00 comp rx=0 tx=0).stop 2024-03-11T19:51:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.854+0000 7f35db7d7640 1 -- 172.21.15.83:0/2103849370 shutdown_connections 2024-03-11T19:51:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.854+0000 7f35db7d7640 1 --2- 172.21.15.83:0/2103849370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d41013e0 0x7f35d4107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.854+0000 7f35db7d7640 1 --2- 172.21.15.83:0/2103849370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4100a20 0x7f35d4100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.854+0000 7f35db7d7640 1 --2- 172.21.15.83:0/2103849370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d40ff830 0x7f35d40ffc50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.854+0000 7f35db7d7640 1 -- 172.21.15.83:0/2103849370 >> 172.21.15.83:0/2103849370 conn(0x7f35d40fad80 msgr2=0x7f35d40fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.854+0000 7f35db7d7640 1 -- 172.21.15.83:0/2103849370 shutdown_connections 2024-03-11T19:51:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.854+0000 7f35db7d7640 1 -- 172.21.15.83:0/2103849370 wait complete. 2024-03-11T19:51:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35db7d7640 1 Processor -- start 2024-03-11T19:51:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35db7d7640 1 -- start start 2024-03-11T19:51:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35db7d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40ff830 0x7f35d4195050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35db7d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4100a20 0x7f35d4195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35db7d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d41013e0 0x7f35d419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35db7d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d419cb40 con 0x7f35d40ff830 2024-03-11T19:51:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35db7d7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d419ccb0 con 0x7f35d4100a20 2024-03-11T19:51:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35db7d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d419ce70 con 0x7f35d41013e0 2024-03-11T19:51:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35d9573640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40ff830 0x7f35d4195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35d8d72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4100a20 0x7f35d4195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35d8d72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4100a20 0x7f35d4195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:33082/0 (socket says 172.21.15.83:33082) 2024-03-11T19:51:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35d9573640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40ff830 0x7f35d4195050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40472/0 (socket says 172.21.15.83:40472) 2024-03-11T19:51:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35d8d72640 1 -- 172.21.15.83:0/1673814088 learned_addr learned my addr 172.21.15.83:0/1673814088 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.855+0000 7f35d9d74640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d41013e0 0x7f35d419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.856+0000 7f35d8d72640 1 -- 172.21.15.83:0/1673814088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d41013e0 msgr2=0x7f35d419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.856+0000 7f35d8d72640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d41013e0 0x7f35d419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.856+0000 7f35d8d72640 1 -- 172.21.15.83:0/1673814088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40ff830 msgr2=0x7f35d4195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.856+0000 7f35d8d72640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40ff830 0x7f35d4195050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.856+0000 7f35d8d72640 1 -- 172.21.15.83:0/1673814088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d000d590 con 0x7f35d4100a20 2024-03-11T19:51:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.856+0000 7f35d8d72640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4100a20 0x7f35d4195590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e9f0 tx=0x7f35c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.856+0000 7f35ca7fc640 1 -- 172.21.15.83:0/1673814088 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4009700 con 0x7f35d4100a20 2024-03-11T19:51:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.856+0000 7f35ca7fc640 1 -- 172.21.15.83:0/1673814088 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35c400cd80 con 0x7f35d4100a20 2024-03-11T19:51:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.856+0000 7f35db7d7640 1 -- 172.21.15.83:0/1673814088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d419d150 con 0x7f35d4100a20 2024-03-11T19:51:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.856+0000 7f35ca7fc640 1 -- 172.21.15.83:0/1673814088 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4018920 con 0x7f35d4100a20 2024-03-11T19:51:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.856+0000 7f35db7d7640 1 -- 172.21.15.83:0/1673814088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d419d660 con 0x7f35d4100a20 2024-03-11T19:51:08.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.858+0000 7f35ca7fc640 1 -- 172.21.15.83:0/1673814088 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f35c4018a80 con 0x7f35d4100a20 2024-03-11T19:51:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35de3ad740 1 Processor -- start 2024-03-11T19:51:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35de3ad740 1 -- start start 2024-03-11T19:51:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bb28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bb3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 0x559e54bba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35de3ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54b21f90 con 0x559e54bb4050 2024-03-11T19:51:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35de3ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54b1f4c0 con 0x559e54bb3690 2024-03-11T19:51:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35de3ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54b1aca0 con 0x559e54bb24c0 2024-03-11T19:51:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35ca7fc640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f35c00a1a30 0x7f35c00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35d9d74640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bb28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35d9d74640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bb28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57812/0 (socket says 172.21.15.83:57812) 2024-03-11T19:51:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35d9d74640 1 -- 172.21.15.83:0/173761729 learned_addr learned my addr 172.21.15.83:0/173761729 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35ca7fc640 1 -- 172.21.15.83:0/1673814088 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f35c4014070 con 0x7f35d4100a20 2024-03-11T19:51:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.859+0000 7f35d8d72640 1 --2- 172.21.15.83:0/173761729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 0x559e54bba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.860+0000 7f35d9573640 1 --2- 172.21.15.83:0/173761729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bb3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.860+0000 7f35d9d74640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f35c00a1a30 0x7f35c00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.860+0000 7f35d9d74640 1 -- 172.21.15.83:0/173761729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 msgr2=0x559e54bb3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.860+0000 7f35d9d74640 1 --2- 172.21.15.83:0/173761729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bb3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.860+0000 7f35d9d74640 1 -- 172.21.15.83:0/173761729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 msgr2=0x559e54bba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.860+0000 7f35d9d74640 1 --2- 172.21.15.83:0/173761729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 0x559e54bba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.860+0000 7f35d9d74640 1 -- 172.21.15.83:0/173761729 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e54add0b0 con 0x559e54bb24c0 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.861+0000 7f35d9d74640 1 --2- 172.21.15.83:0/173761729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bb28e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0040c0 tx=0x7f35cc004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a86d36c01298cd74 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.861+0000 7f35bf7fe640 1 -- 172.21.15.83:0/173761729 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc010040 con 0x559e54bb24c0 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.861+0000 7f35bf7fe640 1 -- 172.21.15.83:0/173761729 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35cc00bdf0 con 0x559e54bb24c0 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.861+0000 7f35bf7fe640 1 -- 172.21.15.83:0/173761729 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc01a940 con 0x559e54bb24c0 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.862+0000 7f35de3ad740 1 -- 172.21.15.83:0/173761729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 msgr2=0x559e54bb28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.862+0000 7f35d9d74640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f35c00a1a30 0x7f35c00a3ef0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f35cc023000 tx=0x7f35cc0193b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.862+0000 7f35de3ad740 1 --2- 172.21.15.83:0/173761729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bb28e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0040c0 tx=0x7f35cc004580 comp rx=0 tx=0).stop 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.862+0000 7f35de3ad740 1 -- 172.21.15.83:0/173761729 shutdown_connections 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.862+0000 7f35de3ad740 1 --2- 172.21.15.83:0/173761729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 0x559e54bba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.862+0000 7f35de3ad740 1 --2- 172.21.15.83:0/173761729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bb3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.862+0000 7f35de3ad740 1 --2- 172.21.15.83:0/173761729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bb28e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.862+0000 7f35de3ad740 1 -- 172.21.15.83:0/173761729 >> 172.21.15.83:0/173761729 conn(0x559e54bad6a0 msgr2=0x559e54badaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35de3ad740 1 -- 172.21.15.83:0/173761729 shutdown_connections 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35de3ad740 1 -- 172.21.15.83:0/173761729 wait complete. 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35de3ad740 1 -- start start 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35de3ad740 1 Processor -- start 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bbc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bbca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 0x559e54bc3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35de3ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54abff40 con 0x559e54bb4050 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35de3ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e547ee4b0 con 0x559e54bb3690 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35de3ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54aa0670 con 0x559e54bb24c0 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35de3ad740 10 client.0 _ll_register_callbacks cb 0x559e54b0df70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35d9d74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 0x559e54bc3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35d9573640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bbc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35d8d72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bbca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.863+0000 7f35d9d74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 0x559e54bc3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40496/0 (socket says 172.21.15.83:40496) 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35d9d74640 1 -- 172.21.15.83:0/3061803055 learned_addr learned my addr 172.21.15.83:0/3061803055 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35d9573640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bbc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57828/0 (socket says 172.21.15.83:57828) 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35d9d74640 1 -- 172.21.15.83:0/3061803055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 msgr2=0x559e54bbc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35d9d74640 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bbc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35d9d74640 1 -- 172.21.15.83:0/3061803055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 msgr2=0x559e54bbca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35d9d74640 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bbca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35d9d74640 1 -- 172.21.15.83:0/3061803055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35cc003c30 con 0x559e54bb4050 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35d8d72640 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bbca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35d9d74640 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 0x559e54bc3aa0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f35cc000f80 tx=0x7f35cc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf3fbc8e461700f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35bcff9640 1 -- 172.21.15.83:0/3061803055 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc010070 con 0x559e54bb4050 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35bcff9640 1 -- 172.21.15.83:0/3061803055 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f35cc003a70 con 0x559e54bb4050 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.864+0000 7f35de3ad740 1 -- 172.21.15.83:0/3061803055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e54af4f40 con 0x559e54bb4050 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35bcff9640 1 -- 172.21.15.83:0/3061803055 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35cc0199a0 con 0x559e54bb4050 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35de3ad740 10 client.17613 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35de3ad740 1 -- 172.21.15.83:0/3061803055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559e54b19320 con 0x559e54bb4050 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35bcff9640 1 -- 172.21.15.83:0/3061803055 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559e54af4f40 con 0x559e54bb4050 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35bcff9640 1 -- 172.21.15.83:0/3061803055 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f35cc016040 con 0x559e54bb4050 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35de3ad740 10 client.17613 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35de3ad740 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35de3ad740 20 client.17613 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:08.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559e54bc8550 0x559e54bca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35de3ad740 4 client.17613 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35de3ad740 1 -- 172.21.15.83:0/3061803055 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e54b11310 con 0x559e54bc8550 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.865+0000 7f35d9573640 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559e54bc8550 0x559e54bca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.866+0000 7f35d9573640 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559e54bc8550 0x559e54bca940 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd900ac7012ac8c0 server_cookie=94419c4bedf47596 in_seq=0 out_seq=0 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.866+0000 7f35bcff9640 10 client.17613 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35bcff9640 1 -- 172.21.15.83:0/3061803055 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559e54b11310 con 0x559e54bc8550 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35bcff9640 10 client.17613 handle_command_reply: tid=0 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 1 client.17613 shutdown 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 1 -- 172.21.15.83:0/3061803055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 msgr2=0x559e54bc3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 0x559e54bc3aa0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f35cc000f80 tx=0x7f35cc00a620 comp rx=0 tx=0).stop 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 1 -- 172.21.15.83:0/3061803055 shutdown_connections 2024-03-11T19:51:08.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559e54bc8550 0x559e54bca940 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bb4050 0x559e54bc3aa0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bbca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3061803055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bbc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35bcff9640 0 client.17613 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 1 -- 172.21.15.83:0/3061803055 >> 172.21.15.83:0/3061803055 conn(0x559e54bad6a0 msgr2=0x559e54bac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 1 -- 172.21.15.83:0/3061803055 shutdown_connections 2024-03-11T19:51:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 1 -- 172.21.15.83:0/3061803055 wait complete. 2024-03-11T19:51:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.867+0000 7f35de3ad740 20 client.17613 trim_cache size 0 max 16384 2024-03-11T19:51:08.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:08 smithi137 ceph-mon[36220]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 MiB/s wr, 597 op/s 2024-03-11T19:51:08.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.886+0000 7f35de3ad740 1 Processor -- start 2024-03-11T19:51:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.886+0000 7f35de3ad740 1 -- start start 2024-03-11T19:51:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.886+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bb7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.886+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bb7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.886+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bc3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.886+0000 7f35de3ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54bba8c0 con 0x559e54bad6a0 2024-03-11T19:51:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.886+0000 7f35de3ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54bae0f0 con 0x559e54bb3690 2024-03-11T19:51:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.886+0000 7f35de3ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54b11310 con 0x559e54bb24c0 2024-03-11T19:51:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.886+0000 7f35d9573640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bb7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35d9573640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bb7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40500/0 (socket says 172.21.15.83:40500) 2024-03-11T19:51:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35d9573640 1 -- 172.21.15.83:0/1826117270 learned_addr learned my addr 172.21.15.83:0/1826117270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35d8d72640 1 --2- 172.21.15.83:0/1826117270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bb7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35d9d74640 1 --2- 172.21.15.83:0/1826117270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bc3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35d9573640 1 -- 172.21.15.83:0/1826117270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 msgr2=0x559e54bb7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35d9573640 1 --2- 172.21.15.83:0/1826117270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bb7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35d9573640 1 -- 172.21.15.83:0/1826117270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 msgr2=0x559e54bc3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35d9573640 1 --2- 172.21.15.83:0/1826117270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bc3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35d9573640 1 -- 172.21.15.83:0/1826117270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e54ab5c00 con 0x559e54bad6a0 2024-03-11T19:51:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35d9573640 1 --2- 172.21.15.83:0/1826117270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bb7050 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d960 tx=0x7f35d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980aab4268baf039 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35beffd640 1 -- 172.21.15.83:0/1826117270 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0044030 con 0x559e54bad6a0 2024-03-11T19:51:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35beffd640 1 -- 172.21.15.83:0/1826117270 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35d0045040 con 0x559e54bad6a0 2024-03-11T19:51:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.887+0000 7f35beffd640 1 -- 172.21.15.83:0/1826117270 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d004b830 con 0x559e54bad6a0 2024-03-11T19:51:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.888+0000 7f35de3ad740 1 -- 172.21.15.83:0/1826117270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 msgr2=0x559e54bb7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.888+0000 7f35de3ad740 1 --2- 172.21.15.83:0/1826117270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bb7050 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d960 tx=0x7f35d0000bc0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.888+0000 7f35de3ad740 1 -- 172.21.15.83:0/1826117270 shutdown_connections 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.888+0000 7f35de3ad740 1 --2- 172.21.15.83:0/1826117270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb3690 0x559e54bc3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.888+0000 7f35de3ad740 1 --2- 172.21.15.83:0/1826117270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb24c0 0x559e54bb7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.888+0000 7f35de3ad740 1 --2- 172.21.15.83:0/1826117270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bb7050 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.888+0000 7f35de3ad740 1 -- 172.21.15.83:0/1826117270 >> 172.21.15.83:0/1826117270 conn(0x559e54bbaa00 msgr2=0x559e54bcf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.888+0000 7f35de3ad740 1 -- 172.21.15.83:0/1826117270 shutdown_connections 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.888+0000 7f35de3ad740 1 -- 172.21.15.83:0/1826117270 wait complete. 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35de3ad740 1 -- start start 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35de3ad740 1 Processor -- start 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bc3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb24c0 0x559e54bc2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35de3ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb3690 0x559e54bc2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35de3ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54bc2c70 con 0x559e54bad6a0 2024-03-11T19:51:08.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35de3ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54bc2de0 con 0x559e54bb24c0 2024-03-11T19:51:08.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35de3ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e54c1ea10 con 0x559e54bb3690 2024-03-11T19:51:08.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35de3ad740 10 client.0 _ll_register_callbacks cb 0x7f35cc00abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35d9573640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bc3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35d9573640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bc3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40504/0 (socket says 172.21.15.83:40504) 2024-03-11T19:51:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35d9573640 1 -- 172.21.15.83:0/3453607397 learned_addr learned my addr 172.21.15.83:0/3453607397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35d8d72640 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb24c0 0x559e54bc2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.889+0000 7f35d9d74640 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb3690 0x559e54bc2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35d9573640 1 -- 172.21.15.83:0/3453607397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb3690 msgr2=0x559e54bc2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35d9573640 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb3690 0x559e54bc2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35d9573640 1 -- 172.21.15.83:0/3453607397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb24c0 msgr2=0x559e54bc2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35d9573640 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb24c0 0x559e54bc2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35d9573640 1 -- 172.21.15.83:0/3453607397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d4108570 con 0x559e54bad6a0 2024-03-11T19:51:08.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35d9573640 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bc3a20 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f35d0037a30 tx=0x7f35d00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1de48642955042 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35bcff9640 1 -- 172.21.15.83:0/3453607397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0044070 con 0x559e54bad6a0 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35bcff9640 1 -- 172.21.15.83:0/3453607397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f35d0033070 con 0x559e54bad6a0 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35de3ad740 1 -- 172.21.15.83:0/3453607397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e54b40900 con 0x559e54bad6a0 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35bcff9640 1 -- 172.21.15.83:0/3453607397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35d0037e70 con 0x559e54bad6a0 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35de3ad740 10 client.17619 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35de3ad740 1 -- 172.21.15.83:0/3453607397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559e54adb910 con 0x559e54bad6a0 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35bcff9640 1 -- 172.21.15.83:0/3453607397 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559e54b40900 con 0x559e54bad6a0 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35bcff9640 1 -- 172.21.15.83:0/3453607397 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f35d002f4b0 con 0x559e54bad6a0 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35de3ad740 10 client.17619 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.890+0000 7f35de3ad740 10 client.17619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.891+0000 7f35de3ad740 20 client.17619 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.891+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559e54bb4050 0x559e54c22120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:08.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.891+0000 7f35de3ad740 4 client.17619 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.891+0000 7f35de3ad740 1 -- 172.21.15.83:0/3453607397 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559e5481a860 con 0x559e54bb4050 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.891+0000 7f35d8d72640 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559e54bb4050 0x559e54c22120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.891+0000 7f35bcff9640 10 client.17619 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.891+0000 7f35d8d72640 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559e54bb4050 0x559e54c22120 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b7f62062202b22 server_cookie=bd7d5c9b6ca32702 in_seq=0 out_seq=0 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.892+0000 7f35bcff9640 1 -- 172.21.15.83:0/3453607397 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559e5481a860 con 0x559e54bb4050 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.892+0000 7f35bcff9640 10 client.17619 handle_command_reply: tid=0 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.892+0000 7f35de3ad740 1 client.17619 shutdown 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.892+0000 7f35de3ad740 1 -- 172.21.15.83:0/3453607397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 msgr2=0x559e54bc3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.892+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bc3a20 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f35d0037a30 tx=0x7f35d00043c0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.893+0000 7f35de3ad740 1 -- 172.21.15.83:0/3453607397 shutdown_connections 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.893+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x559e54bb4050 0x559e54c22120 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.893+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e54bb3690 0x559e54bc2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.893+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e54bb24c0 0x559e54bc2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.893+0000 7f35de3ad740 1 --2- 172.21.15.83:0/3453607397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e54bad6a0 0x559e54bc3a20 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.893+0000 7f35de3ad740 1 -- 172.21.15.83:0/3453607397 >> 172.21.15.83:0/3453607397 conn(0x559e54bbaa00 msgr2=0x559e54bb7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:08.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.894+0000 7f35bcff9640 0 client.17619 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:08.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35de3ad740 1 -- 172.21.15.83:0/3453607397 shutdown_connections 2024-03-11T19:51:08.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35de3ad740 1 -- 172.21.15.83:0/3453607397 wait complete. 2024-03-11T19:51:08.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35de3ad740 20 client.17619 trim_cache size 0 max 16384 2024-03-11T19:51:08.905 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:51:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:51:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "87e59027-ac48-4e79-b44f-387f7b4f4186", 2024-03-11T19:51:08.905 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:51:08.905 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:51:08.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 -- 172.21.15.83:0/1673814088 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f35c00a1a30 msgr2=0x7f35c00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f35c00a1a30 0x7f35c00a3ef0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f35cc023000 tx=0x7f35cc0193b0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 -- 172.21.15.83:0/1673814088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4100a20 msgr2=0x7f35d4195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4100a20 0x7f35d4195590 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e9f0 tx=0x7f35c400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 -- 172.21.15.83:0/1673814088 shutdown_connections 2024-03-11T19:51:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f35c00a1a30 0x7f35c00a3ef0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35d41013e0 0x7f35d419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35d4100a20 0x7f35d4195590 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 --2- 172.21.15.83:0/1673814088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35d40ff830 0x7f35d4195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 -- 172.21.15.83:0/1673814088 >> 172.21.15.83:0/1673814088 conn(0x7f35d40fad80 msgr2=0x7f35d4103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 -- 172.21.15.83:0/1673814088 shutdown_connections 2024-03-11T19:51:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:08.895+0000 7f35db7d7640 1 -- 172.21.15.83:0/1673814088 wait complete. 2024-03-11T19:51:08.908 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:08 smithi204 ceph-mon[36739]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 MiB/s wr, 597 op/s 2024-03-11T19:51:08.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:51:08.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:51:09.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.069+0000 7faf0e89d640 1 -- 172.21.15.83:0/3507263393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08102d30 msgr2=0x7faf081095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.069+0000 7faf0e89d640 1 --2- 172.21.15.83:0/3507263393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08102d30 0x7faf081095c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faefc009860 tx=0x7faefc02bb70 comp rx=0 tx=0).stop 2024-03-11T19:51:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.069+0000 7faf0e89d640 1 -- 172.21.15.83:0/3507263393 shutdown_connections 2024-03-11T19:51:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.069+0000 7faf0e89d640 1 --2- 172.21.15.83:0/3507263393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08102d30 0x7faf081095c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.069+0000 7faf0e89d640 1 --2- 172.21.15.83:0/3507263393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf08102370 0x7faf081027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.069+0000 7faf0e89d640 1 --2- 172.21.15.83:0/3507263393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf08101180 0x7faf081015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.069+0000 7faf0e89d640 1 -- 172.21.15.83:0/3507263393 >> 172.21.15.83:0/3507263393 conn(0x7faf080fc6f0 msgr2=0x7faf080feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.069+0000 7faf0e89d640 1 -- 172.21.15.83:0/3507263393 shutdown_connections 2024-03-11T19:51:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.069+0000 7faf0e89d640 1 -- 172.21.15.83:0/3507263393 wait complete. 2024-03-11T19:51:09.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf0e89d640 1 Processor -- start 2024-03-11T19:51:09.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf0e89d640 1 -- start start 2024-03-11T19:51:09.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf0e89d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08101180 0x7faf081a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf0e89d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf08102370 0x7faf081a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf0e89d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf08102d30 0x7faf081a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf0e89d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf081a3f90 con 0x7faf08101180 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf0e89d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf081a4100 con 0x7faf08102370 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf0e89d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf081a42c0 con 0x7faf08102d30 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf07fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08101180 0x7faf081a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf07fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08101180 0x7faf081a6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40522/0 (socket says 172.21.15.83:40522) 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf07fff640 1 -- 172.21.15.83:0/4090959676 learned_addr learned my addr 172.21.15.83:0/4090959676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf077fe640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf08102370 0x7faf081a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.070+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf08102d30 0x7faf081a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf07fff640 1 -- 172.21.15.83:0/4090959676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf08102d30 msgr2=0x7faf081a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf07fff640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf08102d30 0x7faf081a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf07fff640 1 -- 172.21.15.83:0/4090959676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf08102370 msgr2=0x7faf081a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf07fff640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf08102370 0x7faf081a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf07fff640 1 -- 172.21.15.83:0/4090959676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc009490 con 0x7faf08101180 2024-03-11T19:51:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf08102d30 0x7faf081a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:51:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf077fe640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf08102370 0x7faf081a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf07fff640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08101180 0x7faf081a6ce0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faef800d770 tx=0x7faef800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf057fa640 1 -- 172.21.15.83:0/4090959676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8004440 con 0x7faf08101180 2024-03-11T19:51:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf057fa640 1 -- 172.21.15.83:0/4090959676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faef800de80 con 0x7faf08101180 2024-03-11T19:51:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf057fa640 1 -- 172.21.15.83:0/4090959676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8010460 con 0x7faf08101180 2024-03-11T19:51:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf0e89d640 1 -- 172.21.15.83:0/4090959676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf08073660 con 0x7faf08101180 2024-03-11T19:51:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.071+0000 7faf0e89d640 1 -- 172.21.15.83:0/4090959676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf08073b20 con 0x7faf08101180 2024-03-11T19:51:09.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.075+0000 7faf057fa640 1 -- 172.21.15.83:0/4090959676 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7faef80097c0 con 0x7faf08101180 2024-03-11T19:51:09.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf11477740 1 Processor -- start 2024-03-11T19:51:09.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf11477740 1 -- start start 2024-03-11T19:51:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf11477740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae34e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf057fa640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faef00a1b30 0x7faef00a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf11477740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 0x55ebae3537c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf11477740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae3562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf11477740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebae2be280 con 0x55ebae34ecc0 2024-03-11T19:51:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf11477740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebae2bb7b0 con 0x55ebae353e30 2024-03-11T19:51:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf11477740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebae2b6f90 con 0x55ebae34e2d0 2024-03-11T19:51:09.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf057fa640 1 -- 172.21.15.83:0/4090959676 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7faef80cd2b0 con 0x7faf08101180 2024-03-11T19:51:09.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf0ce3a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae34e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf0ce3a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae34e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57880/0 (socket says 172.21.15.83:57880) 2024-03-11T19:51:09.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.076+0000 7faf0ce3a640 1 -- 172.21.15.83:0/1635948473 learned_addr learned my addr 172.21.15.83:0/1635948473 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:09.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/1635948473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae3562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faf077fe640 1 --2- 172.21.15.83:0/1635948473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 0x55ebae3537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faf0ce3a640 1 -- 172.21.15.83:0/1635948473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 msgr2=0x55ebae3562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:09.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/1635948473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae3562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faf0ce3a640 1 -- 172.21.15.83:0/1635948473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 msgr2=0x55ebae3537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:09.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/1635948473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 0x55ebae3537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faf0ce3a640 1 -- 172.21.15.83:0/1635948473 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebae275570 con 0x55ebae34e2d0 2024-03-11T19:51:09.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faf07fff640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faef00a1b30 0x7faef00a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faf077fe640 1 --2- 172.21.15.83:0/1635948473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 0x55ebae3537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/1635948473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae34e6f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faefc0057c0 tx=0x7faefc02e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9914edd8f5c5344b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faeee7fc640 1 -- 172.21.15.83:0/1635948473 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0043c0 con 0x55ebae34e2d0 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faf07fff640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faef00a1b30 0x7faef00a3ff0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faef80a24a0 tx=0x7faef8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faeee7fc640 1 -- 172.21.15.83:0/1635948473 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faefc034920 con 0x55ebae34e2d0 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.077+0000 7faeee7fc640 1 -- 172.21.15.83:0/1635948473 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc032600 con 0x55ebae34e2d0 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.078+0000 7faf11477740 1 -- 172.21.15.83:0/1635948473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 msgr2=0x55ebae34e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.078+0000 7faf11477740 1 --2- 172.21.15.83:0/1635948473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae34e6f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faefc0057c0 tx=0x7faefc02e040 comp rx=0 tx=0).stop 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.078+0000 7faf11477740 1 -- 172.21.15.83:0/1635948473 shutdown_connections 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.078+0000 7faf11477740 1 --2- 172.21.15.83:0/1635948473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae3562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.078+0000 7faf11477740 1 --2- 172.21.15.83:0/1635948473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 0x55ebae3537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.078+0000 7faf11477740 1 --2- 172.21.15.83:0/1635948473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae34e6f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.078+0000 7faf11477740 1 -- 172.21.15.83:0/1635948473 >> 172.21.15.83:0/1635948473 conn(0x55ebae3497f0 msgr2=0x55ebae349bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:09.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.078+0000 7faf11477740 1 -- 172.21.15.83:0/1635948473 shutdown_connections 2024-03-11T19:51:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.078+0000 7faf11477740 1 -- 172.21.15.83:0/1635948473 wait complete. 2024-03-11T19:51:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.079+0000 7faf11477740 1 -- start start 2024-03-11T19:51:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.079+0000 7faf11477740 1 Processor -- start 2024-03-11T19:51:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.080+0000 7faf11477740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae34e2d0 0x55ebae358280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.080+0000 7faf11477740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 0x55ebae3587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.080+0000 7faf11477740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae353e30 0x55ebae35f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.080+0000 7faf0ce3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae34e2d0 0x55ebae358280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.080+0000 7faf0ce3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae34e2d0 0x55ebae358280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:33130/0 (socket says 172.21.15.83:33130) 2024-03-11T19:51:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.080+0000 7faf0ce3a640 1 -- 172.21.15.83:0/3273682008 learned_addr learned my addr 172.21.15.83:0/3273682008 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:09.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.080+0000 7faf11477740 1 -- 172.21.15.83:0/3273682008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebadf8a430 con 0x55ebae353e30 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.080+0000 7faf11477740 1 -- 172.21.15.83:0/3273682008 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebadf46e00 con 0x55ebae34e2d0 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.080+0000 7faf11477740 1 -- 172.21.15.83:0/3273682008 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebae23bfd0 con 0x55ebae34ecc0 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.080+0000 7faf077fe640 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 0x55ebae3587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.080+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae353e30 0x55ebae35f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faf11477740 10 client.0 _ll_register_callbacks cb 0x55ebae291fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faf077fe640 1 -- 172.21.15.83:0/3273682008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae34e2d0 msgr2=0x55ebae358280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faf077fe640 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae34e2d0 0x55ebae358280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faf077fe640 1 -- 172.21.15.83:0/3273682008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae353e30 msgr2=0x55ebae35f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faf077fe640 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae353e30 0x55ebae35f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faf077fe640 1 -- 172.21.15.83:0/3273682008 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf08109ec0 con 0x55ebae34ecc0 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae34e2d0 0x55ebae358280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faf077fe640 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 0x55ebae3587c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55ebae34a7b0 tx=0x7faf000117d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93d11346b74f4ab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:09.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faee77fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faee77fe640 1 -- 172.21.15.83:0/3273682008 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00003970 con 0x55ebae34ecc0 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faee77fe640 1 -- 172.21.15.83:0/3273682008 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7faf00003ad0 con 0x55ebae34ecc0 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faf11477740 1 -- 172.21.15.83:0/3273682008 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebadff3720 con 0x55ebae34ecc0 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faee77fe640 1 -- 172.21.15.83:0/3273682008 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf00003c90 con 0x55ebae34ecc0 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faf11477740 10 client.27184 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.081+0000 7faf11477740 1 -- 172.21.15.83:0/3273682008 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ebae2b5610 con 0x55ebae34ecc0 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.082+0000 7faee77fe640 1 -- 172.21.15.83:0/3273682008 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ebadff3720 con 0x55ebae34ecc0 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.082+0000 7faee77fe640 1 -- 172.21.15.83:0/3273682008 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faf00012550 con 0x55ebae34ecc0 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.082+0000 7faf11477740 10 client.27184 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.082+0000 7faf11477740 10 client.27184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.082+0000 7faf11477740 20 client.27184 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.082+0000 7faf11477740 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ebae364450 0x55ebae366870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.082+0000 7faf11477740 4 client.27184 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.082+0000 7faf11477740 1 -- 172.21.15.83:0/3273682008 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebae042890 con 0x55ebae364450 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.082+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ebae364450 0x55ebae366870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.082+0000 7faee77fe640 10 client.27184 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.082+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ebae364450 0x55ebae366870 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd975fe8c3d8c404 server_cookie=2eb11bc720698fb7 in_seq=0 out_seq=0 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faee77fe640 1 -- 172.21.15.83:0/3273682008 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ebae042890 con 0x55ebae364450 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faee77fe640 10 client.27184 handle_command_reply: tid=0 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faf11477740 1 client.27184 shutdown 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faf11477740 1 -- 172.21.15.83:0/3273682008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 msgr2=0x55ebae3587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faf11477740 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 0x55ebae3587c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55ebae34a7b0 tx=0x7faf000117d0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faf11477740 1 -- 172.21.15.83:0/3273682008 shutdown_connections 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faf11477740 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ebae364450 0x55ebae366870 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faf11477740 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae353e30 0x55ebae35f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faf11477740 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 0x55ebae3587c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faee77fe640 0 client.27184 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faf11477740 1 --2- 172.21.15.83:0/3273682008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae34e2d0 0x55ebae358280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.083+0000 7faf11477740 1 -- 172.21.15.83:0/3273682008 >> 172.21.15.83:0/3273682008 conn(0x55ebae3497f0 msgr2=0x55ebae348a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.084+0000 7faf11477740 1 -- 172.21.15.83:0/3273682008 shutdown_connections 2024-03-11T19:51:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.084+0000 7faf11477740 1 -- 172.21.15.83:0/3273682008 wait complete. 2024-03-11T19:51:09.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.084+0000 7faf11477740 20 client.27184 trim_cache size 0 max 16384 2024-03-11T19:51:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.107+0000 7faf11477740 1 Processor -- start 2024-03-11T19:51:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.107+0000 7faf11477740 1 -- start start 2024-03-11T19:51:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.107+0000 7faf11477740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34e2d0 0x55ebae358280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.107+0000 7faf11477740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 0x55ebae3587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.107+0000 7faf11477740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae35f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.107+0000 7faf11477740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebadff3720 con 0x55ebae34e2d0 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.107+0000 7faf11477740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebae359400 con 0x55ebae353e30 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.107+0000 7faf11477740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebae34a7e0 con 0x55ebae34ecc0 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.108+0000 7faf077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 0x55ebae3587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.108+0000 7faf0ce3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae35f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.108+0000 7faf077fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 0x55ebae3587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57894/0 (socket says 172.21.15.83:57894) 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.108+0000 7faf077fe640 1 -- 172.21.15.83:0/182200060 learned_addr learned my addr 172.21.15.83:0/182200060 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.108+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/182200060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34e2d0 0x55ebae358280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.108+0000 7faf0ce3a640 1 -- 172.21.15.83:0/182200060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 msgr2=0x55ebae3587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.108+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/182200060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 0x55ebae3587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.108+0000 7faf0ce3a640 1 -- 172.21.15.83:0/182200060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34e2d0 msgr2=0x55ebae358280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:09.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.108+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/182200060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34e2d0 0x55ebae358280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.108+0000 7faf0ce3a640 1 -- 172.21.15.83:0/182200060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebae251ac0 con 0x55ebae353e30 2024-03-11T19:51:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.108+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/182200060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae35f7c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faefc032530 tx=0x7faefc035f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95fb85d790345c2e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.109+0000 7faeedffb640 1 -- 172.21.15.83:0/182200060 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc059b80 con 0x55ebae353e30 2024-03-11T19:51:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.109+0000 7faeedffb640 1 -- 172.21.15.83:0/182200060 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faefc057050 con 0x55ebae353e30 2024-03-11T19:51:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.109+0000 7faeedffb640 1 -- 172.21.15.83:0/182200060 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc059890 con 0x55ebae353e30 2024-03-11T19:51:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.109+0000 7faf11477740 1 -- 172.21.15.83:0/182200060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 msgr2=0x55ebae35f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.109+0000 7faf11477740 1 --2- 172.21.15.83:0/182200060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae35f7c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faefc032530 tx=0x7faefc035f90 comp rx=0 tx=0).stop 2024-03-11T19:51:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 -- 172.21.15.83:0/182200060 shutdown_connections 2024-03-11T19:51:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 --2- 172.21.15.83:0/182200060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae35f7c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 --2- 172.21.15.83:0/182200060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34ecc0 0x55ebae3587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 --2- 172.21.15.83:0/182200060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34e2d0 0x55ebae358280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 -- 172.21.15.83:0/182200060 >> 172.21.15.83:0/182200060 conn(0x55ebae3497f0 msgr2=0x55ebae36b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 -- 172.21.15.83:0/182200060 shutdown_connections 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 -- 172.21.15.83:0/182200060 wait complete. 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 -- start start 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 Processor -- start 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae35f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 0x55ebae35ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae35e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebae35e9e0 con 0x55ebae34ecc0 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebae35eb50 con 0x55ebae353e30 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.110+0000 7faf11477740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebae3ba980 con 0x55ebae34e2d0 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf11477740 10 client.0 _ll_register_callbacks cb 0x7faefc037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:09.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf0ce3a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae35f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf0ce3a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae35f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:57910/0 (socket says 172.21.15.83:57910) 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf0ce3a640 1 -- 172.21.15.83:0/3187297744 learned_addr learned my addr 172.21.15.83:0/3187297744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf077fe640 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 0x55ebae35ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae35e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf077fe640 1 -- 172.21.15.83:0/3187297744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 msgr2=0x55ebae35f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf077fe640 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae35f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf077fe640 1 -- 172.21.15.83:0/3187297744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 msgr2=0x55ebae35e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf077fe640 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae35e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf077fe640 1 -- 172.21.15.83:0/3187297744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebae275570 con 0x55ebae34ecc0 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae35f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf077fe640 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 0x55ebae35ddf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7faf00013b00 tx=0x7faf000096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd63886e4d995e9c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faee77fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faee77fe640 1 -- 172.21.15.83:0/3187297744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00012b70 con 0x55ebae34ecc0 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faee77fe640 1 -- 172.21.15.83:0/3187297744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7faf00012cd0 con 0x55ebae34ecc0 2024-03-11T19:51:09.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.111+0000 7faf11477740 1 -- 172.21.15.83:0/3187297744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebadfb67e0 con 0x55ebae34ecc0 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.113+0000 7faee77fe640 1 -- 172.21.15.83:0/3187297744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf00012e90 con 0x55ebae34ecc0 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.114+0000 7faf11477740 10 client.17634 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.114+0000 7faf11477740 1 -- 172.21.15.83:0/3187297744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ebae2776a0 con 0x55ebae34ecc0 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.114+0000 7faee77fe640 1 -- 172.21.15.83:0/3187297744 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ebadfb67e0 con 0x55ebae34ecc0 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.114+0000 7faee77fe640 1 -- 172.21.15.83:0/3187297744 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faf00005400 con 0x55ebae34ecc0 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.114+0000 7faf11477740 10 client.17634 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.114+0000 7faf11477740 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.114+0000 7faf11477740 20 client.17634 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.114+0000 7faf11477740 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ebae364450 0x55ebae3be3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.114+0000 7faf11477740 4 client.17634 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.114+0000 7faf11477740 1 -- 172.21.15.83:0/3187297744 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ebadfa54e0 con 0x55ebae364450 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.115+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ebae364450 0x55ebae3be3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:09.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.115+0000 7faf0ce3a640 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ebae364450 0x55ebae3be3b0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52a2e1c73ccb0c2 server_cookie=520ac19e38316e12 in_seq=0 out_seq=0 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.115+0000 7faee77fe640 10 client.17634 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faee77fe640 1 -- 172.21.15.83:0/3187297744 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55ebadfa54e0 con 0x55ebae364450 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faee77fe640 10 client.17634 handle_command_reply: tid=0 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faf11477740 1 client.17634 shutdown 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faf11477740 1 -- 172.21.15.83:0/3187297744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 msgr2=0x55ebae35ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faf11477740 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 0x55ebae35ddf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7faf00013b00 tx=0x7faf000096d0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faf11477740 1 -- 172.21.15.83:0/3187297744 shutdown_connections 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faf11477740 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ebae364450 0x55ebae3be3b0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faf11477740 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebae353e30 0x55ebae35e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faf11477740 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ebae34ecc0 0x55ebae35ddf0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faee77fe640 0 client.17634 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faf11477740 1 --2- 172.21.15.83:0/3187297744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ebae34e2d0 0x55ebae35f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faf11477740 1 -- 172.21.15.83:0/3187297744 >> 172.21.15.83:0/3187297744 conn(0x55ebae3497f0 msgr2=0x55ebae36b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:09.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faf11477740 1 -- 172.21.15.83:0/3187297744 shutdown_connections 2024-03-11T19:51:09.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.116+0000 7faf11477740 1 -- 172.21.15.83:0/3187297744 wait complete. 2024-03-11T19:51:09.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf11477740 20 client.17634 trim_cache size 0 max 16384 2024-03-11T19:51:09.125 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:51:09.125 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (159 inodes in the stack)", 2024-03-11T19:51:09.125 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:51:09.125 INFO:teuthology.orchestra.run.smithi083.stdout: "87e59027-ac48-4e79-b44f-387f7b4f4186": { 2024-03-11T19:51:09.125 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:51:09.125 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "87e59027-ac48-4e79-b44f-387f7b4f4186", 2024-03-11T19:51:09.125 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:51:09.125 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:51:09.126 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:51:09.126 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:51:09.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 -- 172.21.15.83:0/4090959676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faef00a1b30 msgr2=0x7faef00a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faef00a1b30 0x7faef00a3ff0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faef80a24a0 tx=0x7faef8000bc0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 -- 172.21.15.83:0/4090959676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08101180 msgr2=0x7faf081a6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:09.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08101180 0x7faf081a6ce0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faef800d770 tx=0x7faef800dc30 comp rx=0 tx=0).stop 2024-03-11T19:51:09.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 -- 172.21.15.83:0/4090959676 shutdown_connections 2024-03-11T19:51:09.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faef00a1b30 0x7faef00a3ff0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf08102d30 0x7faf081a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf08102370 0x7faf081a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 --2- 172.21.15.83:0/4090959676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf08101180 0x7faf081a6ce0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:09.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 -- 172.21.15.83:0/4090959676 >> 172.21.15.83:0/4090959676 conn(0x7faf080fc6f0 msgr2=0x7faf08105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:09.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 -- 172.21.15.83:0/4090959676 shutdown_connections 2024-03-11T19:51:09.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:09.117+0000 7faf0e89d640 1 -- 172.21.15.83:0/4090959676 wait complete. 2024-03-11T19:51:09.135 INFO:tasks.cephfs.filesystem:scrub status for tag:87e59027-ac48-4e79-b44f-387f7b4f4186 - {'path': '/', 'tag': '87e59027-ac48-4e79-b44f-387f7b4f4186', 'options': 'recursive,force'} 2024-03-11T19:51:09.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:09 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:51:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:09 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:09 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:51:09.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:09 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:09 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:51:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:09 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:09 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:51:09.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:09 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:09 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:51:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:09 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:09 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:51:09.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:09 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:10.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:10 smithi083 ceph-mon[31864]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 MiB/s wr, 756 op/s 2024-03-11T19:51:10.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:10 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:51:10.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:10 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:10.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:10 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:51:10.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:10 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:10 smithi137 ceph-mon[36220]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 MiB/s wr, 756 op/s 2024-03-11T19:51:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:10 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:51:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:10 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:10 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:51:10.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:10 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:10 smithi204 ceph-mon[36739]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 MiB/s wr, 756 op/s 2024-03-11T19:51:10.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:10 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:51:10.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:10 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:10.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:10 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:51:10.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:10 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 715.0 s, 1993.8 tps, lat 50.307 ms stddev 54.404 2024-03-11T19:51:12.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:12 smithi083 ceph-mon[31864]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.8 MiB/s wr, 590 op/s 2024-03-11T19:51:12.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:12 smithi137 ceph-mon[36220]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.8 MiB/s wr, 590 op/s 2024-03-11T19:51:12.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:12 smithi204 ceph-mon[36739]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.8 MiB/s wr, 590 op/s 2024-03-11T19:51:14.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:14 smithi083 ceph-mon[31864]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-03-11T19:51:14.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:14 smithi137 ceph-mon[36220]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-03-11T19:51:14.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:14 smithi204 ceph-mon[36739]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-03-11T19:51:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:16 smithi083 ceph-mon[31864]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.4 MiB/s wr, 743 op/s 2024-03-11T19:51:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:16 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:51:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:16 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:51:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:16 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:51:16.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:16 smithi137 ceph-mon[36220]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.4 MiB/s wr, 743 op/s 2024-03-11T19:51:16.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:16 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:51:16.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:16 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:51:16.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:16 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:51:16.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:16 smithi204 ceph-mon[36739]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.4 MiB/s wr, 743 op/s 2024-03-11T19:51:16.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:16 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:51:16.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:16 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:51:16.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:16 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:51:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 720.0 s, 1765.2 tps, lat 56.849 ms stddev 63.056 2024-03-11T19:51:18.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:18 smithi083 ceph-mon[31864]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 570 op/s 2024-03-11T19:51:18.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:18 smithi137 ceph-mon[36220]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 570 op/s 2024-03-11T19:51:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:18 smithi204 ceph-mon[36739]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 570 op/s 2024-03-11T19:51:20.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:20 smithi083 ceph-mon[31864]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 708 op/s 2024-03-11T19:51:20.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:20 smithi137 ceph-mon[36220]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 708 op/s 2024-03-11T19:51:20.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:20 smithi204 ceph-mon[36739]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 708 op/s 2024-03-11T19:51:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 725.0 s, 1667.4 tps, lat 59.322 ms stddev 68.874 2024-03-11T19:51:22.820 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:22 smithi083 ceph-mon[31864]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-03-11T19:51:22.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:22 smithi137 ceph-mon[36220]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-03-11T19:51:22.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:22 smithi204 ceph-mon[36739]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-03-11T19:51:24.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:24 smithi083 ceph-mon[31864]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.0 MiB/s wr, 613 op/s 2024-03-11T19:51:24.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:24 smithi137 ceph-mon[36220]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.0 MiB/s wr, 613 op/s 2024-03-11T19:51:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:24 smithi204 ceph-mon[36739]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.0 MiB/s wr, 613 op/s 2024-03-11T19:51:26.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:26 smithi083 ceph-mon[31864]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 664 op/s 2024-03-11T19:51:26.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:26 smithi137 ceph-mon[36220]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 664 op/s 2024-03-11T19:51:26.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:26 smithi204 ceph-mon[36739]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.3 MiB/s wr, 664 op/s 2024-03-11T19:51:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 730.0 s, 1496.8 tps, lat 67.297 ms stddev 72.379 2024-03-11T19:51:28.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:28 smithi083 ceph-mon[31864]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 552 op/s 2024-03-11T19:51:28.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:28 smithi137 ceph-mon[36220]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 552 op/s 2024-03-11T19:51:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:28 smithi204 ceph-mon[36739]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 MiB/s wr, 552 op/s 2024-03-11T19:51:30.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:30 smithi083 ceph-mon[31864]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 713 op/s 2024-03-11T19:51:30.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:30 smithi137 ceph-mon[36220]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 713 op/s 2024-03-11T19:51:30.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:30 smithi204 ceph-mon[36739]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 713 op/s 2024-03-11T19:51:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 735.0 s, 1664.4 tps, lat 59.602 ms stddev 65.024 2024-03-11T19:51:32.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:32 smithi083 ceph-mon[31864]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.6 MiB/s wr, 583 op/s 2024-03-11T19:51:32.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:32 smithi137 ceph-mon[36220]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.6 MiB/s wr, 583 op/s 2024-03-11T19:51:32.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:32 smithi204 ceph-mon[36739]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.6 MiB/s wr, 583 op/s 2024-03-11T19:51:34.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:34 smithi137 ceph-mon[36220]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.8 MiB/s wr, 524 op/s 2024-03-11T19:51:34.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:34 smithi204 ceph-mon[36739]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.8 MiB/s wr, 524 op/s 2024-03-11T19:51:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:34 smithi083 ceph-mon[31864]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.8 MiB/s wr, 524 op/s 2024-03-11T19:51:36.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:36 smithi137 ceph-mon[36220]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 MiB/s wr, 653 op/s 2024-03-11T19:51:36.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:36 smithi204 ceph-mon[36739]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 MiB/s wr, 653 op/s 2024-03-11T19:51:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:36 smithi083 ceph-mon[31864]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 MiB/s wr, 653 op/s 2024-03-11T19:51:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 740.0 s, 1614.0 tps, lat 62.305 ms stddev 70.094 2024-03-11T19:51:38.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:38 smithi137 ceph-mon[36220]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-03-11T19:51:38.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:38 smithi204 ceph-mon[36739]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-03-11T19:51:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:38 smithi083 ceph-mon[31864]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-03-11T19:51:39.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:51:39.136 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:51:39.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.294+0000 7f4de2c64640 1 -- 172.21.15.83:0/3497918704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ddc108180 msgr2=0x7f4ddc10a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.294+0000 7f4de2c64640 1 --2- 172.21.15.83:0/3497918704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ddc108180 0x7f4ddc10a5b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4dd401acb0 tx=0x7f4dd403d150 comp rx=0 tx=0).stop 2024-03-11T19:51:39.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.294+0000 7f4de2c64640 1 -- 172.21.15.83:0/3497918704 shutdown_connections 2024-03-11T19:51:39.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.294+0000 7f4de2c64640 1 --2- 172.21.15.83:0/3497918704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc10d4a0 0x7f4ddc10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.294+0000 7f4de2c64640 1 --2- 172.21.15.83:0/3497918704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ddc10aaf0 0x7f4ddc10cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.294+0000 7f4de2c64640 1 --2- 172.21.15.83:0/3497918704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ddc108180 0x7f4ddc10a5b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.294+0000 7f4de2c64640 1 -- 172.21.15.83:0/3497918704 >> 172.21.15.83:0/3497918704 conn(0x7f4ddc06e200 msgr2=0x7f4ddc070640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:39.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.294+0000 7f4de2c64640 1 -- 172.21.15.83:0/3497918704 shutdown_connections 2024-03-11T19:51:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.295+0000 7f4de2c64640 1 -- 172.21.15.83:0/3497918704 wait complete. 2024-03-11T19:51:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.295+0000 7f4de2c64640 1 Processor -- start 2024-03-11T19:51:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.295+0000 7f4de2c64640 1 -- start start 2024-03-11T19:51:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.296+0000 7f4de2c64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108180 0x7f4ddc19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.296+0000 7f4de2c64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ddc10aaf0 0x7f4ddc19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.296+0000 7f4de2c64640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ddc10d4a0 0x7f4ddc1a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.296+0000 7f4de2c64640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc1a5170 con 0x7f4ddc108180 2024-03-11T19:51:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.296+0000 7f4de2c64640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc1a52e0 con 0x7f4ddc10d4a0 2024-03-11T19:51:39.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.296+0000 7f4de2c64640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc1a54a0 con 0x7f4ddc10aaf0 2024-03-11T19:51:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4de1201640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ddc10d4a0 0x7f4ddc1a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ddc10aaf0 0x7f4ddc19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4de1201640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ddc10d4a0 0x7f4ddc1a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37028/0 (socket says 172.21.15.83:37028) 2024-03-11T19:51:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4de1201640 1 -- 172.21.15.83:0/2750259105 learned_addr learned my addr 172.21.15.83:0/2750259105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4de0a00640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108180 0x7f4ddc19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4de1201640 1 -- 172.21.15.83:0/2750259105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ddc10aaf0 msgr2=0x7f4ddc19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4de1201640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ddc10aaf0 0x7f4ddc19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4de1201640 1 -- 172.21.15.83:0/2750259105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108180 msgr2=0x7f4ddc19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4de1201640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108180 0x7f4ddc19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4de1201640 1 -- 172.21.15.83:0/2750259105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd401a910 con 0x7f4ddc10d4a0 2024-03-11T19:51:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4de1201640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ddc10d4a0 0x7f4ddc1a4c30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800c970 tx=0x7f4dc800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:39.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4dd9ffb640 1 -- 172.21.15.83:0/2750259105 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8007bb0 con 0x7f4ddc10d4a0 2024-03-11T19:51:39.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4dd9ffb640 1 -- 172.21.15.83:0/2750259105 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dc8007d10 con 0x7f4ddc10d4a0 2024-03-11T19:51:39.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4dd9ffb640 1 -- 172.21.15.83:0/2750259105 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8018770 con 0x7f4ddc10d4a0 2024-03-11T19:51:39.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.297+0000 7f4de2c64640 1 -- 172.21.15.83:0/2750259105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ddc1a5780 con 0x7f4ddc10d4a0 2024-03-11T19:51:39.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.298+0000 7f4de2c64640 1 -- 172.21.15.83:0/2750259105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc1a5c40 con 0x7f4ddc10d4a0 2024-03-11T19:51:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.300+0000 7f4dd9ffb640 1 -- 172.21.15.83:0/2750259105 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4dc800f450 con 0x7f4ddc10d4a0 2024-03-11T19:51:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de5833740 1 Processor -- start 2024-03-11T19:51:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de5833740 1 -- start start 2024-03-11T19:51:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de5833740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f64112d0 0x55d8f64116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de5833740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6411cc0 0x55d8f64167c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4dd9ffb640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4dc00a1b70 0x7f4dc00a4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de5833740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6416e30 0x55d8f64192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de5833740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f6381280 con 0x55d8f6411cc0 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6411cc0 0x55d8f64167c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6411cc0 0x55d8f64167c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49044/0 (socket says 172.21.15.83:49044) 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de0a00640 1 -- 172.21.15.83:0/177820578 learned_addr learned my addr 172.21.15.83:0/177820578 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de5833740 1 -- 172.21.15.83:0/177820578 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f637e7b0 con 0x55d8f6416e30 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de5833740 1 -- 172.21.15.83:0/177820578 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f6379f90 con 0x55d8f64112d0 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4dd9ffb640 1 -- 172.21.15.83:0/2750259105 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4dc80cd320 con 0x7f4ddc10d4a0 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.301+0000 7f4de1201640 1 --2- 172.21.15.83:0/177820578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6416e30 0x55d8f64192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.302+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/177820578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f64112d0 0x55d8f64116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.302+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4dc00a1b70 0x7f4dc00a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.302+0000 7f4de0a00640 1 -- 172.21.15.83:0/177820578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f64112d0 msgr2=0x55d8f64116f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.302+0000 7f4de0a00640 1 --2- 172.21.15.83:0/177820578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f64112d0 0x55d8f64116f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.302+0000 7f4de0a00640 1 -- 172.21.15.83:0/177820578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6416e30 msgr2=0x55d8f64192c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.302+0000 7f4de0a00640 1 --2- 172.21.15.83:0/177820578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6416e30 0x55d8f64192c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.302+0000 7f4de0a00640 1 -- 172.21.15.83:0/177820578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8f6338570 con 0x55d8f6411cc0 2024-03-11T19:51:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.302+0000 7f4de1201640 1 --2- 172.21.15.83:0/177820578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6416e30 0x55d8f64192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.302+0000 7f4de0a00640 1 --2- 172.21.15.83:0/177820578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6411cc0 0x55d8f64167c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4dd401acb0 tx=0x7f4dd4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6f773aa03d108c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.302+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4dc00a1b70 0x7f4dc00a4030 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55d8f6412520 tx=0x7f4dcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.303+0000 7f4dd2ffd640 1 -- 172.21.15.83:0/177820578 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4043710 con 0x55d8f6411cc0 2024-03-11T19:51:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.303+0000 7f4dd2ffd640 1 -- 172.21.15.83:0/177820578 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dd4003e50 con 0x55d8f6411cc0 2024-03-11T19:51:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.303+0000 7f4dd2ffd640 1 -- 172.21.15.83:0/177820578 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4004460 con 0x55d8f6411cc0 2024-03-11T19:51:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.303+0000 7f4de5833740 1 -- 172.21.15.83:0/177820578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6411cc0 msgr2=0x55d8f64167c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.303+0000 7f4de5833740 1 --2- 172.21.15.83:0/177820578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6411cc0 0x55d8f64167c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4dd401acb0 tx=0x7f4dd4005ef0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.303+0000 7f4de5833740 1 -- 172.21.15.83:0/177820578 shutdown_connections 2024-03-11T19:51:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.303+0000 7f4de5833740 1 --2- 172.21.15.83:0/177820578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6416e30 0x55d8f64192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.303+0000 7f4de5833740 1 --2- 172.21.15.83:0/177820578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6411cc0 0x55d8f64167c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.303+0000 7f4de5833740 1 --2- 172.21.15.83:0/177820578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f64112d0 0x55d8f64116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.303+0000 7f4de5833740 1 -- 172.21.15.83:0/177820578 >> 172.21.15.83:0/177820578 conn(0x55d8f640c7f0 msgr2=0x55d8f640cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.304+0000 7f4de5833740 1 -- 172.21.15.83:0/177820578 shutdown_connections 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.304+0000 7f4de5833740 1 -- 172.21.15.83:0/177820578 wait complete. 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.304+0000 7f4de5833740 1 -- start start 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.304+0000 7f4de5833740 1 Processor -- start 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de5833740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 0x55d8f641b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de5833740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 0x55d8f641b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de5833740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 0x55d8f64227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de5833740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f604d430 con 0x55d8f64112d0 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de5833740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f6009e00 con 0x55d8f6411cc0 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de5833740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f62fefd0 con 0x55d8f6416e30 2024-03-11T19:51:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 0x55d8f641b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de5833740 10 client.0 _ll_register_callbacks cb 0x55d8f6354fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 0x55d8f641b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 0x55d8f641b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49052/0 (socket says 172.21.15.83:49052) 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de0a00640 1 -- 172.21.15.83:0/713284855 learned_addr learned my addr 172.21.15.83:0/713284855 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 0x55d8f641b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37052/0 (socket says 172.21.15.83:37052) 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de1201640 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 0x55d8f64227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de0a00640 1 -- 172.21.15.83:0/713284855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 msgr2=0x55d8f64227c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de0a00640 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 0x55d8f64227c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de0a00640 1 -- 172.21.15.83:0/713284855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 msgr2=0x55d8f641b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de0a00640 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 0x55d8f641b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de0a00640 1 -- 172.21.15.83:0/713284855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc10e460 con 0x55d8f64112d0 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de1201640 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 0x55d8f64227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 0x55d8f641b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4de0a00640 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 0x55d8f641b240 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4dd403d760 tx=0x7f4dd4043dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f29811a5f33f86 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4dbffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.305+0000 7f4dbffff640 1 -- 172.21.15.83:0/713284855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4003b80 con 0x55d8f64112d0 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4dbffff640 1 -- 172.21.15.83:0/713284855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4dd405d070 con 0x55d8f64112d0 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4de5833740 1 -- 172.21.15.83:0/713284855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8f60797e0 con 0x55d8f64112d0 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4dbffff640 1 -- 172.21.15.83:0/713284855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dd4003ce0 con 0x55d8f64112d0 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4de5833740 10 client.17643 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4de5833740 1 -- 172.21.15.83:0/713284855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d8f6378610 con 0x55d8f64112d0 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4dbffff640 1 -- 172.21.15.83:0/713284855 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d8f60797e0 con 0x55d8f64112d0 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4dbffff640 1 -- 172.21.15.83:0/713284855 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4dd404fa00 con 0x55d8f64112d0 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4de5833740 10 client.17643 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4de5833740 10 client.17643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4de5833740 20 client.17643 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4de5833740 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d8f6426f80 0x55d8f6429370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4de5833740 4 client.17643 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4de5833740 1 -- 172.21.15.83:0/713284855 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8f60b6720 con 0x55d8f6426f80 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.306+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d8f6426f80 0x55d8f6429370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.307+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d8f6426f80 0x55d8f6429370 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3beb83a2d6a480a server_cookie=9cef426acc286073 in_seq=0 out_seq=0 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.307+0000 7f4dbffff640 10 client.17643 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.308+0000 7f4dbffff640 1 -- 172.21.15.83:0/713284855 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d8f60b6720 con 0x55d8f6426f80 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.308+0000 7f4dbffff640 10 client.17643 handle_command_reply: tid=0 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.308+0000 7f4de5833740 1 client.17643 shutdown 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.308+0000 7f4de5833740 1 -- 172.21.15.83:0/713284855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 msgr2=0x55d8f641b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.308+0000 7f4de5833740 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 0x55d8f641b240 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4dd403d760 tx=0x7f4dd4043dc0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.308+0000 7f4de5833740 1 -- 172.21.15.83:0/713284855 shutdown_connections 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.308+0000 7f4de5833740 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d8f6426f80 0x55d8f6429370 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.308+0000 7f4dbffff640 0 client.17643 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.308+0000 7f4de5833740 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 0x55d8f64227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.308+0000 7f4de5833740 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 0x55d8f641b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.308+0000 7f4de5833740 1 --2- 172.21.15.83:0/713284855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 0x55d8f641b240 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.309+0000 7f4de5833740 1 -- 172.21.15.83:0/713284855 >> 172.21.15.83:0/713284855 conn(0x55d8f640c7f0 msgr2=0x55d8f640ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:39.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.309+0000 7f4de5833740 1 -- 172.21.15.83:0/713284855 shutdown_connections 2024-03-11T19:51:39.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.309+0000 7f4de5833740 1 -- 172.21.15.83:0/713284855 wait complete. 2024-03-11T19:51:39.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.309+0000 7f4de5833740 20 client.17643 trim_cache size 0 max 16384 2024-03-11T19:51:39.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.327+0000 7f4de5833740 1 Processor -- start 2024-03-11T19:51:39.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.327+0000 7f4de5833740 1 -- start start 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.327+0000 7f4de5833740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 0x55d8f640b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.327+0000 7f4de5833740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 0x55d8f6422750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.327+0000 7f4de5833740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 0x55d8f640b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.327+0000 7f4de5833740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f641c2b0 con 0x55d8f64112d0 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.327+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 0x55d8f640b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 0x55d8f640b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49062/0 (socket says 172.21.15.83:49062) 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4de0a00640 1 -- 172.21.15.83:0/2636140067 learned_addr learned my addr 172.21.15.83:0/2636140067 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/2636140067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 0x55d8f6422750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4de5833740 1 -- 172.21.15.83:0/2636140067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f60b6720 con 0x55d8f6411cc0 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4de5833740 1 -- 172.21.15.83:0/2636140067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f60797e0 con 0x55d8f6416e30 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4de1201640 1 --2- 172.21.15.83:0/2636140067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 0x55d8f640b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4ddbfff640 1 -- 172.21.15.83:0/2636140067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 msgr2=0x55d8f640b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:39.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/2636140067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 0x55d8f640b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4ddbfff640 1 -- 172.21.15.83:0/2636140067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 msgr2=0x55d8f640b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:39.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/2636140067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 0x55d8f640b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4ddbfff640 1 -- 172.21.15.83:0/2636140067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8f6314ac0 con 0x55d8f6411cc0 2024-03-11T19:51:39.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4de1201640 1 --2- 172.21.15.83:0/2636140067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 0x55d8f640b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:39.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/2636140067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 0x55d8f6422750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55d8f6429c80 tx=0x7f4dcc005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b63bcb03c62e8346 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:39.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4dd27fc640 1 -- 172.21.15.83:0/2636140067 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc029070 con 0x55d8f6411cc0 2024-03-11T19:51:39.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4dd27fc640 1 -- 172.21.15.83:0/2636140067 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dcc00cb90 con 0x55d8f6411cc0 2024-03-11T19:51:39.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.328+0000 7f4dd27fc640 1 -- 172.21.15.83:0/2636140067 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc02b890 con 0x55d8f6411cc0 2024-03-11T19:51:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.329+0000 7f4de5833740 1 -- 172.21.15.83:0/2636140067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 msgr2=0x55d8f6422750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.329+0000 7f4de5833740 1 --2- 172.21.15.83:0/2636140067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 0x55d8f6422750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55d8f6429c80 tx=0x7f4dcc005850 comp rx=0 tx=0).stop 2024-03-11T19:51:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.329+0000 7f4de5833740 1 -- 172.21.15.83:0/2636140067 shutdown_connections 2024-03-11T19:51:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.329+0000 7f4de5833740 1 --2- 172.21.15.83:0/2636140067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6416e30 0x55d8f640b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.329+0000 7f4de5833740 1 --2- 172.21.15.83:0/2636140067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f6411cc0 0x55d8f6422750 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.329+0000 7f4de5833740 1 --2- 172.21.15.83:0/2636140067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f64112d0 0x55d8f640b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.329+0000 7f4de5833740 1 -- 172.21.15.83:0/2636140067 >> 172.21.15.83:0/2636140067 conn(0x55d8f640c7f0 msgr2=0x55d8f6412f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.329+0000 7f4de5833740 1 -- 172.21.15.83:0/2636140067 shutdown_connections 2024-03-11T19:51:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.329+0000 7f4de5833740 1 -- 172.21.15.83:0/2636140067 wait complete. 2024-03-11T19:51:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.329+0000 7f4de5833740 1 -- start start 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.329+0000 7f4de5833740 1 Processor -- start 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de5833740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f64112d0 0x55d8f6422210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de5833740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6411cc0 0x55d8f6422750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de5833740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6416e30 0x55d8f6420940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de5833740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f6421040 con 0x55d8f6416e30 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de5833740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f64211b0 con 0x55d8f64112d0 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de5833740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8f6421370 con 0x55d8f6411cc0 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de5833740 10 client.0 _ll_register_callbacks cb 0x7f4dcc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de1201640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6416e30 0x55d8f6420940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f64112d0 0x55d8f6422210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de1201640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6416e30 0x55d8f6420940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49072/0 (socket says 172.21.15.83:49072) 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de0a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f64112d0 0x55d8f6422210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:37066/0 (socket says 172.21.15.83:37066) 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4de1201640 1 -- 172.21.15.83:0/1357056991 learned_addr learned my addr 172.21.15.83:0/1357056991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6411cc0 0x55d8f6422750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4ddbfff640 1 -- 172.21.15.83:0/1357056991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f64112d0 msgr2=0x55d8f6422210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f64112d0 0x55d8f6422210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4ddbfff640 1 -- 172.21.15.83:0/1357056991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6416e30 msgr2=0x55d8f6420940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6416e30 0x55d8f6420940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.330+0000 7f4ddbfff640 1 -- 172.21.15.83:0/1357056991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc1a5780 con 0x55d8f6411cc0 2024-03-11T19:51:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.331+0000 7f4de1201640 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6416e30 0x55d8f6420940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.331+0000 7f4ddbfff640 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6411cc0 0x55d8f6422750 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009b20 tx=0x7f4dcc005ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b244ed6acfd832 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.332+0000 7f4dbffff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:51:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.332+0000 7f4dbffff640 1 -- 172.21.15.83:0/1357056991 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc029040 con 0x55d8f6411cc0 2024-03-11T19:51:39.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4dbffff640 1 -- 172.21.15.83:0/1357056991 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4dcc00cd00 con 0x55d8f6411cc0 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4de5833740 1 -- 172.21.15.83:0/1357056991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8f6412af0 con 0x55d8f6411cc0 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4dbffff640 1 -- 172.21.15.83:0/1357056991 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dcc02b810 con 0x55d8f6411cc0 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4de5833740 10 client.27208 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4de5833740 1 -- 172.21.15.83:0/1357056991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f4dcc01b040 con 0x55d8f6411cc0 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4dbffff640 1 -- 172.21.15.83:0/1357056991 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dcc043020 con 0x55d8f6411cc0 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4dbffff640 1 -- 172.21.15.83:0/1357056991 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4dcc003bb0 con 0x55d8f6411cc0 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4de5833740 10 client.27208 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4de5833740 10 client.27208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4de5833740 20 client.27208 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4de5833740 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d8f6426f80 0x55d8f6480a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4de5833740 4 client.27208 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.333+0000 7f4de5833740 1 -- 172.21.15.83:0/1357056991 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d8f6412af0 con 0x55d8f6426f80 2024-03-11T19:51:39.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4de0a00640 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d8f6426f80 0x55d8f6480a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4dbffff640 10 client.27208 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4de0a00640 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d8f6426f80 0x55d8f6480a30 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a10409091bcce9f4 server_cookie=8b43dfcf7d283ae0 in_seq=0 out_seq=0 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4dbffff640 1 -- 172.21.15.83:0/1357056991 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d8f6412af0 con 0x55d8f6426f80 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4dbffff640 10 client.27208 handle_command_reply: tid=0 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4de5833740 1 client.27208 shutdown 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4de5833740 1 -- 172.21.15.83:0/1357056991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6411cc0 msgr2=0x55d8f6422750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4de5833740 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6411cc0 0x55d8f6422750 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009b20 tx=0x7f4dcc005ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4de5833740 1 -- 172.21.15.83:0/1357056991 shutdown_connections 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4de5833740 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d8f6426f80 0x55d8f6480a30 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4dbffff640 0 client.27208 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4de5833740 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d8f6416e30 0x55d8f6420940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4de5833740 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8f6411cc0 0x55d8f6422750 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4de5833740 1 --2- 172.21.15.83:0/1357056991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8f64112d0 0x55d8f6422210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.334+0000 7f4de5833740 1 -- 172.21.15.83:0/1357056991 >> 172.21.15.83:0/1357056991 conn(0x55d8f640c7f0 msgr2=0x55d8f640faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de5833740 1 -- 172.21.15.83:0/1357056991 shutdown_connections 2024-03-11T19:51:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de5833740 1 -- 172.21.15.83:0/1357056991 wait complete. 2024-03-11T19:51:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de5833740 20 client.27208 trim_cache size 0 max 16384 2024-03-11T19:51:39.343 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:51:39.343 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:51:39.343 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:51:39.343 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:51:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 -- 172.21.15.83:0/2750259105 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4dc00a1b70 msgr2=0x7f4dc00a4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4dc00a1b70 0x7f4dc00a4030 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55d8f6412520 tx=0x7f4dcc005f90 comp rx=0 tx=0).stop 2024-03-11T19:51:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 -- 172.21.15.83:0/2750259105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ddc10d4a0 msgr2=0x7f4ddc1a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ddc10d4a0 0x7f4ddc1a4c30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800c970 tx=0x7f4dc800ce30 comp rx=0 tx=0).stop 2024-03-11T19:51:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 -- 172.21.15.83:0/2750259105 shutdown_connections 2024-03-11T19:51:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4dc00a1b70 0x7f4dc00a4030 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ddc10d4a0 0x7f4ddc1a4c30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ddc10aaf0 0x7f4ddc19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 --2- 172.21.15.83:0/2750259105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ddc108180 0x7f4ddc19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 -- 172.21.15.83:0/2750259105 >> 172.21.15.83:0/2750259105 conn(0x7f4ddc06e200 msgr2=0x7f4ddc108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 -- 172.21.15.83:0/2750259105 shutdown_connections 2024-03-11T19:51:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.335+0000 7f4de2c64640 1 -- 172.21.15.83:0/2750259105 wait complete. 2024-03-11T19:51:39.352 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:51:39.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:51:39.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.486+0000 7f756fde2640 1 -- 172.21.15.83:0/4142560554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7568101290 msgr2=0x7f7568101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.486+0000 7f756fde2640 1 --2- 172.21.15.83:0/4142560554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7568101290 0x7f7568101710 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7558009900 tx=0x7f755802bc30 comp rx=0 tx=0).stop 2024-03-11T19:51:39.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.487+0000 7f756fde2640 1 -- 172.21.15.83:0/4142560554 shutdown_connections 2024-03-11T19:51:39.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.487+0000 7f756fde2640 1 --2- 172.21.15.83:0/4142560554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7568101c50 0x7f75681084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.487+0000 7f756fde2640 1 --2- 172.21.15.83:0/4142560554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7568101290 0x7f7568101710 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.487+0000 7f756fde2640 1 --2- 172.21.15.83:0/4142560554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75681000a0 0x7f75681004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.487+0000 7f756fde2640 1 -- 172.21.15.83:0/4142560554 >> 172.21.15.83:0/4142560554 conn(0x7f75680fb610 msgr2=0x7f75680fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:39.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.488+0000 7f756fde2640 1 -- 172.21.15.83:0/4142560554 shutdown_connections 2024-03-11T19:51:39.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.488+0000 7f756fde2640 1 -- 172.21.15.83:0/4142560554 wait complete. 2024-03-11T19:51:39.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.488+0000 7f756fde2640 1 Processor -- start 2024-03-11T19:51:39.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756fde2640 1 -- start start 2024-03-11T19:51:39.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756fde2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75681000a0 0x7f7568199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756fde2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7568101290 0x7f7568199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756fde2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7568101c50 0x7f75681a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756fde2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75681a0f30 con 0x7f7568101c50 2024-03-11T19:51:39.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756fde2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75681a10a0 con 0x7f7568101290 2024-03-11T19:51:39.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756fde2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75681a1260 con 0x7f75681000a0 2024-03-11T19:51:39.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756db7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75681000a0 0x7f7568199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756db7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75681000a0 0x7f7568199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:43174/0 (socket says 172.21.15.83:43174) 2024-03-11T19:51:39.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756db7e640 1 -- 172.21.15.83:0/1662148926 learned_addr learned my addr 172.21.15.83:0/1662148926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756d37d640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7568101290 0x7f7568199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756e37f640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7568101c50 0x7f75681a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756db7e640 1 -- 172.21.15.83:0/1662148926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7568101290 msgr2=0x7f7568199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756db7e640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7568101290 0x7f7568199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756db7e640 1 -- 172.21.15.83:0/1662148926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7568101c50 msgr2=0x7f75681a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756db7e640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7568101c50 0x7f75681a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756db7e640 1 -- 172.21.15.83:0/1662148926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7558009560 con 0x7f75681000a0 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.489+0000 7f756d37d640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7568101290 0x7f7568199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.490+0000 7f756db7e640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75681000a0 0x7f7568199440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f756400e9c0 tx=0x7f756400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.490+0000 7f755effd640 1 -- 172.21.15.83:0/1662148926 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564009700 con 0x7f75681000a0 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.490+0000 7f755effd640 1 -- 172.21.15.83:0/1662148926 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f756400cd80 con 0x7f75681000a0 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.491+0000 7f755effd640 1 -- 172.21.15.83:0/1662148926 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564010430 con 0x7f75681000a0 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.491+0000 7f756fde2640 1 -- 172.21.15.83:0/1662148926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75681a1540 con 0x7f75681000a0 2024-03-11T19:51:39.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.491+0000 7f756fde2640 1 -- 172.21.15.83:0/1662148926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75681a1a20 con 0x7f75681000a0 2024-03-11T19:51:39.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.491+0000 7f756fde2640 1 -- 172.21.15.83:0/1662148926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756810bab0 con 0x7f75681000a0 2024-03-11T19:51:39.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.493+0000 7f755effd640 1 -- 172.21.15.83:0/1662148926 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7564003680 con 0x7f75681000a0 2024-03-11T19:51:39.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.494+0000 7f755effd640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75540a1a30 0x7f75540a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.494+0000 7f756d37d640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75540a1a30 0x7f75540a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.494+0000 7f755effd640 1 -- 172.21.15.83:0/1662148926 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7564014070 con 0x7f75681000a0 2024-03-11T19:51:39.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.494+0000 7f756d37d640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75540a1a30 0x7f75540a3ef0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7558009230 tx=0x7f7558005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:39.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.497+0000 7f755effd640 1 -- 172.21.15.83:0/1662148926 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f75640969c0 con 0x7f75681000a0 2024-03-11T19:51:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.831+0000 7f756fde2640 1 -- 172.21.15.83:0/1662148926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f756810bc60 con 0x7f75681000a0 2024-03-11T19:51:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.832+0000 7f755effd640 1 -- 172.21.15.83:0/1662148926 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f756409a080 con 0x7f75681000a0 2024-03-11T19:51:39.837 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:51:39.838 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:51:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:51:39.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 -- 172.21.15.83:0/1662148926 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75540a1a30 msgr2=0x7f75540a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75540a1a30 0x7f75540a3ef0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7558009230 tx=0x7f7558005fd0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 -- 172.21.15.83:0/1662148926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75681000a0 msgr2=0x7f7568199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75681000a0 0x7f7568199440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f756400e9c0 tx=0x7f756400ee80 comp rx=0 tx=0).stop 2024-03-11T19:51:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 -- 172.21.15.83:0/1662148926 shutdown_connections 2024-03-11T19:51:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f75540a1a30 0x7f75540a3ef0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7568101c50 0x7f75681a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7568101290 0x7f7568199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 --2- 172.21.15.83:0/1662148926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75681000a0 0x7f7568199440 secure :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f756400e9c0 tx=0x7f756400ee80 comp rx=0 tx=0).stop 2024-03-11T19:51:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 -- 172.21.15.83:0/1662148926 >> 172.21.15.83:0/1662148926 conn(0x7f75680fb610 msgr2=0x7f75680fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 -- 172.21.15.83:0/1662148926 shutdown_connections 2024-03-11T19:51:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.837+0000 7f756fde2640 1 -- 172.21.15.83:0/1662148926 wait complete. 2024-03-11T19:51:39.865 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:51:39.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:51:39.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.988+0000 7effc1883640 1 -- 172.21.15.83:0/3532207593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effbc100080 msgr2=0x7effbc1004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.988+0000 7effc1883640 1 --2- 172.21.15.83:0/3532207593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effbc100080 0x7effbc1004a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7effac009900 tx=0x7effac02bc30 comp rx=0 tx=0).stop 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.988+0000 7effc1883640 1 -- 172.21.15.83:0/3532207593 shutdown_connections 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.988+0000 7effc1883640 1 --2- 172.21.15.83:0/3532207593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effbc101c30 0x7effbc1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.988+0000 7effc1883640 1 --2- 172.21.15.83:0/3532207593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effbc101270 0x7effbc1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.988+0000 7effc1883640 1 --2- 172.21.15.83:0/3532207593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effbc100080 0x7effbc1004a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.988+0000 7effc1883640 1 -- 172.21.15.83:0/3532207593 >> 172.21.15.83:0/3532207593 conn(0x7effbc0fb610 msgr2=0x7effbc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.989+0000 7effc1883640 1 -- 172.21.15.83:0/3532207593 shutdown_connections 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.989+0000 7effc1883640 1 -- 172.21.15.83:0/3532207593 wait complete. 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.989+0000 7effc1883640 1 Processor -- start 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.989+0000 7effc1883640 1 -- start start 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.989+0000 7effc1883640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effbc100080 0x7effbc068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.989+0000 7effc1883640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effbc101270 0x7effbc19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.989+0000 7effc1883640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effbc101c30 0x7effbc19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.989+0000 7effc1883640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc19ede0 con 0x7effbc100080 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.989+0000 7effc1883640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc19ef50 con 0x7effbc101270 2024-03-11T19:51:39.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.989+0000 7effc1883640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc19f110 con 0x7effbc101c30 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effbaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effbc100080 0x7effbc068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effbaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effbc100080 0x7effbc068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36756/0 (socket says 172.21.15.83:36756) 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effbaffd640 1 -- 172.21.15.83:0/1696583559 learned_addr learned my addr 172.21.15.83:0/1696583559 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effbb7fe640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effbc101c30 0x7effbc19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effb27fc640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effbc101270 0x7effbc19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effbaffd640 1 -- 172.21.15.83:0/1696583559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effbc101c30 msgr2=0x7effbc19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effbaffd640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effbc101c30 0x7effbc19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effbaffd640 1 -- 172.21.15.83:0/1696583559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effbc101270 msgr2=0x7effbc19bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effbaffd640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effbc101270 0x7effbc19bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effbaffd640 1 -- 172.21.15.83:0/1696583559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac009560 con 0x7effbc100080 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effbb7fe640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effbc101c30 0x7effbc19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effbaffd640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effbc100080 0x7effbc068e50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7effac005ab0 tx=0x7effac005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effc0881640 1 -- 172.21.15.83:0/1696583559 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac0043d0 con 0x7effbc100080 2024-03-11T19:51:39.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effc1883640 1 -- 172.21.15.83:0/1696583559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc19f330 con 0x7effbc100080 2024-03-11T19:51:39.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effc0881640 1 -- 172.21.15.83:0/1696583559 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effac0052d0 con 0x7effbc100080 2024-03-11T19:51:39.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effc0881640 1 -- 172.21.15.83:0/1696583559 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac003d40 con 0x7effbc100080 2024-03-11T19:51:39.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.990+0000 7effc1883640 1 -- 172.21.15.83:0/1696583559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc19f890 con 0x7effbc100080 2024-03-11T19:51:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effc444c740 1 Processor -- start 2024-03-11T19:51:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effc444c740 1 -- start start 2024-03-11T19:51:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effc444c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d26490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effc444c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26a60 0x5637c4d2aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effc444c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d2dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effc444c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4cd9960 con 0x5637c4d2b660 2024-03-11T19:51:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effc444c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4c8d310 con 0x5637c4d26070 2024-03-11T19:51:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effc444c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4c8a840 con 0x5637c4d26a60 2024-03-11T19:51:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d26490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d26490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:42202/0 (socket says 172.21.15.83:42202) 2024-03-11T19:51:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effbb7fe640 1 -- 172.21.15.83:0/3628688321 learned_addr learned my addr 172.21.15.83:0/3628688321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effb27fc640 1 --2- 172.21.15.83:0/3628688321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d2dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.994+0000 7effbb7fe640 1 -- 172.21.15.83:0/3628688321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26a60 msgr2=0x5637c4d2aff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.995+0000 7effbb7fe640 1 --2- 172.21.15.83:0/3628688321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26a60 0x5637c4d2aff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.995+0000 7effbb7fe640 1 -- 172.21.15.83:0/3628688321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 msgr2=0x5637c4d2dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.995+0000 7effbb7fe640 1 --2- 172.21.15.83:0/3628688321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d2dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.995+0000 7effbb7fe640 1 -- 172.21.15.83:0/3628688321 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637c4c48060 con 0x5637c4d26070 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.995+0000 7effb27fc640 1 --2- 172.21.15.83:0/3628688321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d2dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.995+0000 7effbb7fe640 1 --2- 172.21.15.83:0/3628688321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d26490 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7effb40117e0 tx=0x7effb4011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2129a49fe547d9eb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.995+0000 7effc0881640 1 -- 172.21.15.83:0/1696583559 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7effac005440 con 0x7effbc100080 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.995+0000 7effb1ffb640 1 -- 172.21.15.83:0/3628688321 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4011310 con 0x5637c4d26070 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.995+0000 7effb1ffb640 1 -- 172.21.15.83:0/3628688321 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effb4003e50 con 0x5637c4d26070 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.995+0000 7effb1ffb640 1 -- 172.21.15.83:0/3628688321 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4004500 con 0x5637c4d26070 2024-03-11T19:51:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc0881640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7effa80a1a30 0x7effa80a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc444c740 1 -- 172.21.15.83:0/3628688321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 msgr2=0x5637c4d26490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc444c740 1 --2- 172.21.15.83:0/3628688321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d26490 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7effb40117e0 tx=0x7effb4011ca0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effbaffd640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7effa80a1a30 0x7effa80a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc444c740 1 -- 172.21.15.83:0/3628688321 shutdown_connections 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc444c740 1 --2- 172.21.15.83:0/3628688321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d2dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc444c740 1 --2- 172.21.15.83:0/3628688321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26a60 0x5637c4d2aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc444c740 1 --2- 172.21.15.83:0/3628688321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d26490 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc444c740 1 -- 172.21.15.83:0/3628688321 >> 172.21.15.83:0/3628688321 conn(0x5637c4d18760 msgr2=0x5637c4d18b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc0881640 1 -- 172.21.15.83:0/1696583559 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7effac0ef470 con 0x7effbc100080 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc444c740 1 -- 172.21.15.83:0/3628688321 shutdown_connections 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc444c740 1 -- 172.21.15.83:0/3628688321 wait complete. 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effc444c740 1 -- start start 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.996+0000 7effbaffd640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7effa80a1a30 0x7effa80a3ef0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x5637c4d26f00 tx=0x7effac0a7040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effc444c740 1 Processor -- start 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effc444c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26070 0x5637c4d2fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effc444c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26a60 0x5637c4d301f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effc444c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d37230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effc444c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4c86020 con 0x5637c4d2b660 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effc444c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4959430 con 0x5637c4d26a60 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effc444c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4915e00 con 0x5637c4d26070 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effc444c740 10 client.0 _ll_register_callbacks cb 0x5637c4c2dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d37230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effb27fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26a60 0x5637c4d301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effb27fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26a60 0x5637c4d301f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:42218/0 (socket says 172.21.15.83:42218) 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effb27fc640 1 -- 172.21.15.83:0/1958801168 learned_addr learned my addr 172.21.15.83:0/1958801168 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d37230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36772/0 (socket says 172.21.15.83:36772) 2024-03-11T19:51:40.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effbb7fe640 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26070 0x5637c4d2fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effb27fc640 1 -- 172.21.15.83:0/1958801168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26070 msgr2=0x5637c4d2fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effb27fc640 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26070 0x5637c4d2fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effb27fc640 1 -- 172.21.15.83:0/1958801168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 msgr2=0x5637c4d37230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.997+0000 7effb27fc640 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d37230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7effb27fc640 1 -- 172.21.15.83:0/1958801168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb40114c0 con 0x5637c4d26a60 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7effbb7fe640 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d37230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7effb27fc640 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26a60 0x5637c4d301f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7effa4012700 tx=0x7effa4012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb1f7a1cb299f71c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7eff9affd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7eff9affd640 1 -- 172.21.15.83:0/1958801168 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4017070 con 0x5637c4d26a60 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7effc444c740 1 -- 172.21.15.83:0/1958801168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637c49857e0 con 0x5637c4d26a60 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7eff9affd640 1 -- 172.21.15.83:0/1958801168 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7effa4004050 con 0x5637c4d26a60 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7eff9affd640 1 -- 172.21.15.83:0/1958801168 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7effa4004210 con 0x5637c4d26a60 2024-03-11T19:51:40.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7effc444c740 10 client.27119 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7effc444c740 1 -- 172.21.15.83:0/1958801168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5637c4c846a0 con 0x5637c4d26a60 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7eff9affd640 1 -- 172.21.15.83:0/1958801168 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5637c49857e0 con 0x5637c4d26a60 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.998+0000 7eff9affd640 1 -- 172.21.15.83:0/1958801168 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7effa4025070 con 0x5637c4d26a60 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.999+0000 7effc444c740 10 client.27119 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.999+0000 7effc444c740 10 client.27119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.999+0000 7effc444c740 20 client.27119 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.999+0000 7effc444c740 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5637c4d3c010 0x5637c4d3e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.999+0000 7effc444c740 4 client.27119 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.999+0000 7effc444c740 1 -- 172.21.15.83:0/1958801168 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637c49c2720 con 0x5637c4d3c010 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:39.999+0000 7effbb7fe640 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5637c4d3c010 0x5637c4d3e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.000+0000 7effbb7fe640 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5637c4d3c010 0x5637c4d3e430 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99000965bed451f8 server_cookie=31ff923fbbb7bd16 in_seq=0 out_seq=0 2024-03-11T19:51:40.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.000+0000 7eff9affd640 10 client.27119 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:51:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.000+0000 7eff9affd640 1 -- 172.21.15.83:0/1958801168 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5637c49c2720 con 0x5637c4d3c010 2024-03-11T19:51:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.000+0000 7eff9affd640 10 client.27119 handle_command_reply: tid=0 2024-03-11T19:51:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 1 client.27119 shutdown 2024-03-11T19:51:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 1 -- 172.21.15.83:0/1958801168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26a60 msgr2=0x5637c4d301f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26a60 0x5637c4d301f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7effa4012700 tx=0x7effa4012bc0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 1 -- 172.21.15.83:0/1958801168 shutdown_connections 2024-03-11T19:51:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5637c4d3c010 0x5637c4d3e430 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d37230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7eff9affd640 0 client.27119 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:51:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26a60 0x5637c4d301f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 1 --2- 172.21.15.83:0/1958801168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26070 0x5637c4d2fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 1 -- 172.21.15.83:0/1958801168 >> 172.21.15.83:0/1958801168 conn(0x5637c4d18760 msgr2=0x5637c4d24e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 1 -- 172.21.15.83:0/1958801168 shutdown_connections 2024-03-11T19:51:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 1 -- 172.21.15.83:0/1958801168 wait complete. 2024-03-11T19:51:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.001+0000 7effc444c740 20 client.27119 trim_cache size 0 max 16384 2024-03-11T19:51:40.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.017+0000 7effc444c740 1 Processor -- start 2024-03-11T19:51:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.017+0000 7effc444c740 1 -- start start 2024-03-11T19:51:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.018+0000 7effc444c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d27ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.018+0000 7effc444c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26a60 0x5637c4d28020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.018+0000 7effc444c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.018+0000 7effc444c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4c0b130 con 0x5637c4d2b660 2024-03-11T19:51:40.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.018+0000 7effc444c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4d1c2b0 con 0x5637c4d26070 2024-03-11T19:51:40.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.018+0000 7effc444c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4d14320 con 0x5637c4d26a60 2024-03-11T19:51:40.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.018+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d27ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.018+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d27ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:42220/0 (socket says 172.21.15.83:42220) 2024-03-11T19:51:40.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.018+0000 7effbb7fe640 1 -- 172.21.15.83:0/2853077159 learned_addr learned my addr 172.21.15.83:0/2853077159 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.018+0000 7effbb7fe640 1 --2- 172.21.15.83:0/2853077159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.019+0000 7effbb7fe640 1 -- 172.21.15.83:0/2853077159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26a60 msgr2=0x5637c4d28020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:51:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.019+0000 7effbb7fe640 1 --2- 172.21.15.83:0/2853077159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26a60 0x5637c4d28020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.019+0000 7effbb7fe640 1 -- 172.21.15.83:0/2853077159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 msgr2=0x5637c4d371f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.019+0000 7effbb7fe640 1 --2- 172.21.15.83:0/2853077159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d371f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.019+0000 7effbb7fe640 1 -- 172.21.15.83:0/2853077159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637c4c20c20 con 0x5637c4d26070 2024-03-11T19:51:40.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.019+0000 7effbb7fe640 1 --2- 172.21.15.83:0/2853077159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d27ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7effb40117e0 tx=0x7effb400ff50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b40464f948f3a277 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.019+0000 7effb17fa640 1 -- 172.21.15.83:0/2853077159 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb402e450 con 0x5637c4d26070 2024-03-11T19:51:40.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.019+0000 7effb17fa640 1 -- 172.21.15.83:0/2853077159 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effb4005110 con 0x5637c4d26070 2024-03-11T19:51:40.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.019+0000 7effb17fa640 1 -- 172.21.15.83:0/2853077159 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb402d6d0 con 0x5637c4d26070 2024-03-11T19:51:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.020+0000 7effc444c740 1 -- 172.21.15.83:0/2853077159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 msgr2=0x5637c4d27ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.020+0000 7effc444c740 1 --2- 172.21.15.83:0/2853077159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d27ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7effb40117e0 tx=0x7effb400ff50 comp rx=0 tx=0).stop 2024-03-11T19:51:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.020+0000 7effc444c740 1 -- 172.21.15.83:0/2853077159 shutdown_connections 2024-03-11T19:51:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.020+0000 7effc444c740 1 --2- 172.21.15.83:0/2853077159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d2b660 0x5637c4d371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.020+0000 7effc444c740 1 --2- 172.21.15.83:0/2853077159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26a60 0x5637c4d28020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.020+0000 7effc444c740 1 --2- 172.21.15.83:0/2853077159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d26070 0x5637c4d27ae0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.020+0000 7effc444c740 1 -- 172.21.15.83:0/2853077159 >> 172.21.15.83:0/2853077159 conn(0x5637c4d18760 msgr2=0x5637c4d18b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.020+0000 7effc444c740 1 -- 172.21.15.83:0/2853077159 shutdown_connections 2024-03-11T19:51:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.020+0000 7effc444c740 1 -- 172.21.15.83:0/2853077159 wait complete. 2024-03-11T19:51:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effc444c740 1 -- start start 2024-03-11T19:51:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effc444c740 1 Processor -- start 2024-03-11T19:51:40.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effc444c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26070 0x5637c4d35c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effc444c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d26a60 0x5637c4d36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effc444c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d2b660 0x5637c4d366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effc444c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4d92530 con 0x5637c4d26a60 2024-03-11T19:51:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effc444c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4d926a0 con 0x5637c4d2b660 2024-03-11T19:51:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effc444c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637c4d92860 con 0x5637c4d26070 2024-03-11T19:51:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effc444c740 10 client.0 _ll_register_callbacks cb 0x7effb4009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effb27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d26a60 0x5637c4d36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d2b660 0x5637c4d366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effb27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d26a60 0x5637c4d36190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36794/0 (socket says 172.21.15.83:36794) 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effb27fc640 1 -- 172.21.15.83:0/3520595888 learned_addr learned my addr 172.21.15.83:0/3520595888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effbb7fe640 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26070 0x5637c4d35c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effbb7fe640 1 -- 172.21.15.83:0/3520595888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26070 msgr2=0x5637c4d35c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effbb7fe640 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26070 0x5637c4d35c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effbb7fe640 1 -- 172.21.15.83:0/3520595888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d26a60 msgr2=0x5637c4d36190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.021+0000 7effbb7fe640 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d26a60 0x5637c4d36190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7effbb7fe640 1 -- 172.21.15.83:0/3520595888 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637c4c48060 con 0x5637c4d2b660 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7effbb7fe640 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d2b660 0x5637c4d366d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effb40117b0 tx=0x7effb4032040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6e4f08285441c9c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7eff9affd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7eff9affd640 1 -- 172.21.15.83:0/3520595888 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb402e650 con 0x5637c4d2b660 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7eff9affd640 1 -- 172.21.15.83:0/3520595888 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7effb401f070 con 0x5637c4d2b660 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7effc444c740 1 -- 172.21.15.83:0/3520595888 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effb4003e50 con 0x5637c4d2b660 2024-03-11T19:51:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7eff9affd640 1 -- 172.21.15.83:0/3520595888 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7effb402e7b0 con 0x5637c4d2b660 2024-03-11T19:51:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7effc444c740 10 client.27125 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7effc444c740 1 -- 172.21.15.83:0/3520595888 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5637c4c468c0 con 0x5637c4d2b660 2024-03-11T19:51:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7eff9affd640 1 -- 172.21.15.83:0/3520595888 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effb4003e50 con 0x5637c4d2b660 2024-03-11T19:51:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7eff9affd640 1 -- 172.21.15.83:0/3520595888 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7effb4048070 con 0x5637c4d2b660 2024-03-11T19:51:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7effc444c740 10 client.27125 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7effc444c740 10 client.27125 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:51:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7effc444c740 20 client.27125 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7effc444c740 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5637c4d93e20 0x5637c4d96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7effc444c740 4 client.27125 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:51:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.022+0000 7effc444c740 1 -- 172.21.15.83:0/3520595888 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5637c4a11890 con 0x5637c4d93e20 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.023+0000 7effb27fc640 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5637c4d93e20 0x5637c4d96240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.023+0000 7eff9affd640 10 client.27125 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.023+0000 7effb27fc640 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5637c4d93e20 0x5637c4d96240 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5864215cca059d26 server_cookie=6a2511ae26c4db6b in_seq=0 out_seq=0 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.023+0000 7eff9affd640 1 -- 172.21.15.83:0/3520595888 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637c4a11890 con 0x5637c4d93e20 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.023+0000 7eff9affd640 10 client.27125 handle_command_reply: tid=0 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.023+0000 7effc444c740 1 client.27125 shutdown 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.023+0000 7effc444c740 1 -- 172.21.15.83:0/3520595888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d2b660 msgr2=0x5637c4d366d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.023+0000 7effc444c740 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d2b660 0x5637c4d366d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effb40117b0 tx=0x7effb4032040 comp rx=0 tx=0).stop 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.024+0000 7effc444c740 1 -- 172.21.15.83:0/3520595888 shutdown_connections 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.024+0000 7effc444c740 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5637c4d93e20 0x5637c4d96240 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.024+0000 7effc444c740 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637c4d2b660 0x5637c4d366d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.024+0000 7effc444c740 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637c4d26a60 0x5637c4d36190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.024+0000 7effc444c740 1 --2- 172.21.15.83:0/3520595888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637c4d26070 0x5637c4d35c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.024+0000 7effc444c740 1 -- 172.21.15.83:0/3520595888 >> 172.21.15.83:0/3520595888 conn(0x5637c4d18760 msgr2=0x5637c4d27ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.024+0000 7eff9affd640 0 client.27125 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:51:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.024+0000 7effc444c740 1 -- 172.21.15.83:0/3520595888 shutdown_connections 2024-03-11T19:51:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.024+0000 7effc444c740 1 -- 172.21.15.83:0/3520595888 wait complete. 2024-03-11T19:51:40.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.024+0000 7effc444c740 20 client.27125 trim_cache size 0 max 16384 2024-03-11T19:51:40.035 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:51:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 -- 172.21.15.83:0/1696583559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7effa80a1a30 msgr2=0x7effa80a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7effa80a1a30 0x7effa80a3ef0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x5637c4d26f00 tx=0x7effac0a7040 comp rx=0 tx=0).stop 2024-03-11T19:51:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 -- 172.21.15.83:0/1696583559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effbc100080 msgr2=0x7effbc068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effbc100080 0x7effbc068e50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7effac005ab0 tx=0x7effac005e50 comp rx=0 tx=0).stop 2024-03-11T19:51:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 -- 172.21.15.83:0/1696583559 shutdown_connections 2024-03-11T19:51:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7effa80a1a30 0x7effa80a3ef0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effbc101c30 0x7effbc19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effbc101270 0x7effbc19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 --2- 172.21.15.83:0/1696583559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effbc100080 0x7effbc068e50 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 -- 172.21.15.83:0/1696583559 >> 172.21.15.83:0/1696583559 conn(0x7effbc0fb610 msgr2=0x7effbc104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 -- 172.21.15.83:0/1696583559 shutdown_connections 2024-03-11T19:51:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.025+0000 7effc1883640 1 -- 172.21.15.83:0/1696583559 wait complete. 2024-03-11T19:51:40.041 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:51:40.041 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:51:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.212+0000 7f55fbf93640 1 -- 172.21.15.83:0/623722501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f41023a0 msgr2=0x7f55f4102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.212+0000 7f55fbf93640 1 --2- 172.21.15.83:0/623722501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f41023a0 0x7f55f4102820 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009900 tx=0x7f55e402bc30 comp rx=0 tx=0).stop 2024-03-11T19:51:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.212+0000 7f55fbf93640 1 -- 172.21.15.83:0/623722501 shutdown_connections 2024-03-11T19:51:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.212+0000 7f55fbf93640 1 --2- 172.21.15.83:0/623722501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f4102d60 0x7f55f41095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.212+0000 7f55fbf93640 1 --2- 172.21.15.83:0/623722501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f41023a0 0x7f55f4102820 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.212+0000 7f55fbf93640 1 --2- 172.21.15.83:0/623722501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f41011b0 0x7f55f41015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.212+0000 7f55fbf93640 1 -- 172.21.15.83:0/623722501 >> 172.21.15.83:0/623722501 conn(0x7f55f40fc720 msgr2=0x7f55f40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55fbf93640 1 -- 172.21.15.83:0/623722501 shutdown_connections 2024-03-11T19:51:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55fbf93640 1 -- 172.21.15.83:0/623722501 wait complete. 2024-03-11T19:51:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55fbf93640 1 Processor -- start 2024-03-11T19:51:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55fbf93640 1 -- start start 2024-03-11T19:51:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55fbf93640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f41011b0 0x7f55f4197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55fbf93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f41023a0 0x7f55f4197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55fbf93640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f4102d60 0x7f55f419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55fbf93640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f419ed30 con 0x7f55f41023a0 2024-03-11T19:51:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55fbf93640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f419eea0 con 0x7f55f4102d60 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55fbf93640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f419f060 con 0x7f55f41011b0 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55f952e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f41023a0 0x7f55f4197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55f952e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f41023a0 0x7f55f4197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36804/0 (socket says 172.21.15.83:36804) 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.213+0000 7f55f952e640 1 -- 172.21.15.83:0/1085612569 learned_addr learned my addr 172.21.15.83:0/1085612569 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f41011b0 0x7f55f4197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55fa530640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f4102d60 0x7f55f419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55f952e640 1 -- 172.21.15.83:0/1085612569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f41011b0 msgr2=0x7f55f4197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55f952e640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f41011b0 0x7f55f4197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55f952e640 1 -- 172.21.15.83:0/1085612569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f4102d60 msgr2=0x7f55f419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55f952e640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f4102d60 0x7f55f419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55f952e640 1 -- 172.21.15.83:0/1085612569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e4009560 con 0x7f55f41023a0 2024-03-11T19:51:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55f952e640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f41023a0 0x7f55f4197780 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f55e4006010 tx=0x7f55e4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55eaffd640 1 -- 172.21.15.83:0/1085612569 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e40043d0 con 0x7f55f41023a0 2024-03-11T19:51:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55fbf93640 1 -- 172.21.15.83:0/1085612569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f419f2e0 con 0x7f55f41023a0 2024-03-11T19:51:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55eaffd640 1 -- 172.21.15.83:0/1085612569 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55e40052d0 con 0x7f55f41023a0 2024-03-11T19:51:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55eaffd640 1 -- 172.21.15.83:0/1085612569 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4003cc0 con 0x7f55f41023a0 2024-03-11T19:51:40.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.214+0000 7f55fbf93640 1 -- 172.21.15.83:0/1085612569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f419f840 con 0x7f55f41023a0 2024-03-11T19:51:40.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55feb6f740 1 Processor -- start 2024-03-11T19:51:40.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55feb6f740 1 -- start start 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1651690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca1656760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1659260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55feb6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca160d960 con 0x564ca1651270 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55feb6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca15c1310 con 0x564ca1656dd0 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55feb6f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca15be840 con 0x564ca1651c60 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55fa530640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1651690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55fa530640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1651690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36818/0 (socket says 172.21.15.83:36818) 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55fa530640 1 -- 172.21.15.83:0/2628052176 learned_addr learned my addr 172.21.15.83:0/2628052176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.218+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/2628052176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca1656760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55fa530640 1 -- 172.21.15.83:0/2628052176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 msgr2=0x564ca1656760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55fa530640 1 --2- 172.21.15.83:0/2628052176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca1656760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55fa530640 1 -- 172.21.15.83:0/2628052176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 msgr2=0x564ca1659260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55fa530640 1 --2- 172.21.15.83:0/2628052176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1659260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55fa530640 1 -- 172.21.15.83:0/2628052176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ca157c060 con 0x564ca1651270 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/2628052176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca1656760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55eaffd640 1 -- 172.21.15.83:0/1085612569 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f55e4005440 con 0x7f55f41023a0 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55fa530640 1 --2- 172.21.15.83:0/2628052176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1651690 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009fd0 tx=0x7f55ec0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698dffa8ace0d4b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55dffff640 1 -- 172.21.15.83:0/2628052176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec004d20 con 0x564ca1651270 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55dffff640 1 -- 172.21.15.83:0/2628052176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55ec004e80 con 0x564ca1651270 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55dffff640 1 -- 172.21.15.83:0/2628052176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec005280 con 0x564ca1651270 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.219+0000 7f55eaffd640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55e00a1a30 0x7f55e00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55eaffd640 1 -- 172.21.15.83:0/1085612569 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f55e40ef570 con 0x7f55f41023a0 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55e00a1a30 0x7f55e00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55feb6f740 1 -- 172.21.15.83:0/2628052176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 msgr2=0x564ca1651690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55feb6f740 1 --2- 172.21.15.83:0/2628052176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1651690 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009fd0 tx=0x7f55ec0118d0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55e00a1a30 0x7f55e00a3ef0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x564ca1652060 tx=0x7f55f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55feb6f740 1 -- 172.21.15.83:0/2628052176 shutdown_connections 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55feb6f740 1 --2- 172.21.15.83:0/2628052176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1659260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55feb6f740 1 --2- 172.21.15.83:0/2628052176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca1656760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55feb6f740 1 --2- 172.21.15.83:0/2628052176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1651690 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55feb6f740 1 -- 172.21.15.83:0/2628052176 >> 172.21.15.83:0/2628052176 conn(0x564ca164c7e0 msgr2=0x564ca164cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55feb6f740 1 -- 172.21.15.83:0/2628052176 shutdown_connections 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.220+0000 7f55feb6f740 1 -- 172.21.15.83:0/2628052176 wait complete. 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55feb6f740 1 -- start start 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55feb6f740 1 Processor -- start 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1651270 0x564ca165b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651c60 0x564ca165ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1656dd0 0x564ca1662a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55feb6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca15ba020 con 0x564ca1651c60 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55feb6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca128d430 con 0x564ca1651270 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55feb6f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca1249e00 con 0x564ca1656dd0 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55feb6f740 10 client.0 _ll_register_callbacks cb 0x564ca1561cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55f952e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651c60 0x564ca165ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55f952e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651c60 0x564ca165ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36824/0 (socket says 172.21.15.83:36824) 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55f952e640 1 -- 172.21.15.83:0/4037655013 learned_addr learned my addr 172.21.15.83:0/4037655013 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55fa530640 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1656dd0 0x564ca1662a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1651270 0x564ca165b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55f952e640 1 -- 172.21.15.83:0/4037655013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1656dd0 msgr2=0x564ca1662a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55f952e640 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1656dd0 0x564ca1662a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.221+0000 7f55f952e640 1 -- 172.21.15.83:0/4037655013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1651270 msgr2=0x564ca165b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.222+0000 7f55f952e640 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1651270 0x564ca165b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.222+0000 7f55f952e640 1 -- 172.21.15.83:0/4037655013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ec009c80 con 0x564ca1651c60 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.222+0000 7f55fa530640 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1656dd0 0x564ca1662a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.222+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1651270 0x564ca165b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.222+0000 7f55f952e640 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651c60 0x564ca165ba50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f55e40b8f60 tx=0x7f55e40bc0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95906d7f23ca08ab server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.222+0000 7f55dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.222+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4037655013 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e40bcc90 con 0x564ca1651c60 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4037655013 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f55e40bc590 con 0x564ca1651c60 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55feb6f740 1 -- 172.21.15.83:0/4037655013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ca12f6720 con 0x564ca1651c60 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4037655013 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55e40c06e0 con 0x564ca1651c60 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55feb6f740 10 client.17685 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55feb6f740 1 -- 172.21.15.83:0/4037655013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564ca15b86a0 con 0x564ca1651c60 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4037655013 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564ca12f6720 con 0x564ca1651c60 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4037655013 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55e403c070 con 0x564ca1651c60 2024-03-11T19:51:40.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55feb6f740 10 client.17685 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55feb6f740 10 client.17685 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:51:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55feb6f740 20 client.17685 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564ca166b860 0x564ca166dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55feb6f740 4 client.17685 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55feb6f740 1 -- 172.21.15.83:0/4037655013 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ca1345890 con 0x564ca166b860 2024-03-11T19:51:40.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.223+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564ca166b860 0x564ca166dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.224+0000 7f55dd7fa640 10 client.17685 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:51:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.224+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564ca166b860 0x564ca166dc50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60ec56c1f4b7c418 server_cookie=3c53f86a68650f77 in_seq=0 out_seq=0 2024-03-11T19:51:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.224+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4037655013 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564ca1345890 con 0x564ca166b860 2024-03-11T19:51:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.224+0000 7f55dd7fa640 10 client.17685 handle_command_reply: tid=0 2024-03-11T19:51:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.224+0000 7f55feb6f740 1 client.17685 shutdown 2024-03-11T19:51:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55feb6f740 1 -- 172.21.15.83:0/4037655013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651c60 msgr2=0x564ca165ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651c60 0x564ca165ba50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f55e40b8f60 tx=0x7f55e40bc0d0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55feb6f740 1 -- 172.21.15.83:0/4037655013 shutdown_connections 2024-03-11T19:51:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564ca166b860 0x564ca166dc50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1656dd0 0x564ca1662a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55dd7fa640 0 client.17685 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:51:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651c60 0x564ca165ba50 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4037655013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1651270 0x564ca165b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55feb6f740 1 -- 172.21.15.83:0/4037655013 >> 172.21.15.83:0/4037655013 conn(0x564ca164c7e0 msgr2=0x564ca164bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55feb6f740 1 -- 172.21.15.83:0/4037655013 shutdown_connections 2024-03-11T19:51:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55feb6f740 1 -- 172.21.15.83:0/4037655013 wait complete. 2024-03-11T19:51:40.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.225+0000 7f55feb6f740 20 client.17685 trim_cache size 0 max 16384 2024-03-11T19:51:40.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.240+0000 7f55feb6f740 1 Processor -- start 2024-03-11T19:51:40.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.241+0000 7f55feb6f740 1 -- start start 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.241+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca165b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.241+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca165ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.241+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1662a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.241+0000 7f55feb6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca153f130 con 0x564ca1651270 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.241+0000 7f55feb6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca12f6720 con 0x564ca1656dd0 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.241+0000 7f55feb6f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca165c700 con 0x564ca1651c60 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.241+0000 7f55f9d2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca165b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.241+0000 7f55f9d2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca165b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36828/0 (socket says 172.21.15.83:36828) 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.241+0000 7f55f9d2f640 1 -- 172.21.15.83:0/1949695347 learned_addr learned my addr 172.21.15.83:0/1949695347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55fa530640 1 --2- 172.21.15.83:0/1949695347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1662a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55f952e640 1 --2- 172.21.15.83:0/1949695347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca165ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55f9d2f640 1 -- 172.21.15.83:0/1949695347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 msgr2=0x564ca165ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/1949695347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca165ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55f9d2f640 1 -- 172.21.15.83:0/1949695347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 msgr2=0x564ca1662a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/1949695347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1662a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55f9d2f640 1 -- 172.21.15.83:0/1949695347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ca1554c20 con 0x564ca1651270 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55fa530640 1 --2- 172.21.15.83:0/1949695347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1662a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/1949695347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca165b510 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x564ca164d750 tx=0x7f55f001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85794d8ba7c1a4a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55df7fe640 1 -- 172.21.15.83:0/1949695347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0017070 con 0x564ca1651270 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55df7fe640 1 -- 172.21.15.83:0/1949695347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55f0023040 con 0x564ca1651270 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.242+0000 7f55df7fe640 1 -- 172.21.15.83:0/1949695347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0004e40 con 0x564ca1651270 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.243+0000 7f55feb6f740 1 -- 172.21.15.83:0/1949695347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 msgr2=0x564ca165b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.243+0000 7f55feb6f740 1 --2- 172.21.15.83:0/1949695347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca165b510 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x564ca164d750 tx=0x7f55f001b000 comp rx=0 tx=0).stop 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.243+0000 7f55feb6f740 1 -- 172.21.15.83:0/1949695347 shutdown_connections 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.243+0000 7f55feb6f740 1 --2- 172.21.15.83:0/1949695347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1662a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.243+0000 7f55feb6f740 1 --2- 172.21.15.83:0/1949695347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca165ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.243+0000 7f55feb6f740 1 --2- 172.21.15.83:0/1949695347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca165b510 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.243+0000 7f55feb6f740 1 -- 172.21.15.83:0/1949695347 >> 172.21.15.83:0/1949695347 conn(0x564ca164c7e0 msgr2=0x564ca164cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.243+0000 7f55feb6f740 1 -- 172.21.15.83:0/1949695347 shutdown_connections 2024-03-11T19:51:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.243+0000 7f55feb6f740 1 -- 172.21.15.83:0/1949695347 wait complete. 2024-03-11T19:51:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55feb6f740 1 -- start start 2024-03-11T19:51:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55feb6f740 1 Processor -- start 2024-03-11T19:51:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1662750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca1660dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55feb6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1661310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55feb6f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca1661850 con 0x564ca1651270 2024-03-11T19:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55feb6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca16619c0 con 0x564ca1656dd0 2024-03-11T19:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55feb6f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ca1661b30 con 0x564ca1651c60 2024-03-11T19:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55feb6f740 10 client.0 _ll_register_callbacks cb 0x7f55ec009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55f9d2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1662750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55f9d2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1662750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36832/0 (socket says 172.21.15.83:36832) 2024-03-11T19:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55f9d2f640 1 -- 172.21.15.83:0/4059982263 learned_addr learned my addr 172.21.15.83:0/4059982263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55f952e640 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca1660dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55fa530640 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1661310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55f9d2f640 1 -- 172.21.15.83:0/4059982263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 msgr2=0x564ca1660dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca1660dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55f9d2f640 1 -- 172.21.15.83:0/4059982263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 msgr2=0x564ca1661310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1661310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55f9d2f640 1 -- 172.21.15.83:0/4059982263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ca157c060 con 0x564ca1651270 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.244+0000 7f55f952e640 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca1660dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.245+0000 7f55f9d2f640 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1662750 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f55f0019040 tx=0x7f55f001df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace815a1be2aa3bb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.245+0000 7f55dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.245+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4059982263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0017040 con 0x564ca1651270 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.245+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4059982263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f55f00047c0 con 0x564ca1651270 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.245+0000 7f55feb6f740 1 -- 172.21.15.83:0/4059982263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ca12b97e0 con 0x564ca1651270 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.245+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4059982263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55f0042030 con 0x564ca1651270 2024-03-11T19:51:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.245+0000 7f55feb6f740 10 client.17691 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.245+0000 7f55feb6f740 1 -- 172.21.15.83:0/4059982263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f55f00098e0 con 0x564ca1651270 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.246+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4059982263 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f55f002a730 con 0x564ca1651270 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.246+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4059982263 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55f0031020 con 0x564ca1651270 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.246+0000 7f55feb6f740 10 client.17691 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.246+0000 7f55feb6f740 10 client.17691 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.246+0000 7f55feb6f740 20 client.17691 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.246+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564ca16c3500 0x564ca16c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.246+0000 7f55feb6f740 4 client.17691 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.246+0000 7f55feb6f740 1 -- 172.21.15.83:0/4059982263 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564ca12b97e0 con 0x564ca16c3500 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.246+0000 7f55f952e640 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564ca16c3500 0x564ca16c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55f952e640 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564ca16c3500 0x564ca16c5920 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8604f9e54988dc1 server_cookie=e3f089ab79fe4a05 in_seq=0 out_seq=0 2024-03-11T19:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55dd7fa640 10 client.17691 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55dd7fa640 1 -- 172.21.15.83:0/4059982263 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564ca12b97e0 con 0x564ca16c3500 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55dd7fa640 10 client.17691 handle_command_reply: tid=0 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 1 client.17691 shutdown 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 1 -- 172.21.15.83:0/4059982263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 msgr2=0x564ca1662750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1662750 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f55f0019040 tx=0x7f55f001df90 comp rx=0 tx=0).stop 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 1 -- 172.21.15.83:0/4059982263 shutdown_connections 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x564ca16c3500 0x564ca16c5920 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ca1656dd0 0x564ca1661310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55dd7fa640 0 client.17691 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564ca1651c60 0x564ca1660dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 1 --2- 172.21.15.83:0/4059982263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ca1651270 0x564ca1662750 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 1 -- 172.21.15.83:0/4059982263 >> 172.21.15.83:0/4059982263 conn(0x564ca164c7e0 msgr2=0x564ca165b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 1 -- 172.21.15.83:0/4059982263 shutdown_connections 2024-03-11T19:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 1 -- 172.21.15.83:0/4059982263 wait complete. 2024-03-11T19:51:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.247+0000 7f55feb6f740 20 client.17691 trim_cache size 0 max 16384 2024-03-11T19:51:40.258 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 -- 172.21.15.83:0/1085612569 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55e00a1a30 msgr2=0x7f55e00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55e00a1a30 0x7f55e00a3ef0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x564ca1652060 tx=0x7f55f0005f90 comp rx=0 tx=0).stop 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 -- 172.21.15.83:0/1085612569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f41023a0 msgr2=0x7f55f4197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f41023a0 0x7f55f4197780 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f55e4006010 tx=0x7f55e4005e80 comp rx=0 tx=0).stop 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 -- 172.21.15.83:0/1085612569 shutdown_connections 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f55e00a1a30 0x7f55e00a3ef0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f4102d60 0x7f55f419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55f41023a0 0x7f55f4197780 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 --2- 172.21.15.83:0/1085612569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f41011b0 0x7f55f4197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 -- 172.21.15.83:0/1085612569 >> 172.21.15.83:0/1085612569 conn(0x7f55f40fc720 msgr2=0x7f55f41056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 -- 172.21.15.83:0/1085612569 shutdown_connections 2024-03-11T19:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.248+0000 7f55fbf93640 1 -- 172.21.15.83:0/1085612569 wait complete. 2024-03-11T19:51:40.267 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:51:40.267 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:51:40.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.422+0000 7fa35ea02640 1 -- 172.21.15.83:0/3784698634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa358100d90 msgr2=0x7fa358101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.422+0000 7fa35ea02640 1 --2- 172.21.15.83:0/3784698634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa358100d90 0x7fa358101210 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa350009ca0 tx=0x7fa35002c120 comp rx=0 tx=0).stop 2024-03-11T19:51:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.422+0000 7fa35ea02640 1 -- 172.21.15.83:0/3784698634 shutdown_connections 2024-03-11T19:51:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.422+0000 7fa35ea02640 1 --2- 172.21.15.83:0/3784698634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa358100d90 0x7fa358101210 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.422+0000 7fa35ea02640 1 --2- 172.21.15.83:0/3784698634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3581003f0 0x7fa358100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.422+0000 7fa35ea02640 1 --2- 172.21.15.83:0/3784698634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3580ffa90 0x7fa3580ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.422+0000 7fa35ea02640 1 -- 172.21.15.83:0/3784698634 >> 172.21.15.83:0/3784698634 conn(0x7fa3580fb630 msgr2=0x7fa3580fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.423+0000 7fa35ea02640 1 -- 172.21.15.83:0/3784698634 shutdown_connections 2024-03-11T19:51:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.423+0000 7fa35ea02640 1 -- 172.21.15.83:0/3784698634 wait complete. 2024-03-11T19:51:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.423+0000 7fa35ea02640 1 Processor -- start 2024-03-11T19:51:40.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.423+0000 7fa35ea02640 1 -- start start 2024-03-11T19:51:40.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa35ea02640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3580ffa90 0x7fa358199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa35ea02640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3581003f0 0x7fa3581999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa35ea02640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa358100d90 0x7fa3581a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa35ea02640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581a0f80 con 0x7fa3581003f0 2024-03-11T19:51:40.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa35ea02640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581a10f0 con 0x7fa358100d90 2024-03-11T19:51:40.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa35ea02640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581a12b0 con 0x7fa3580ffa90 2024-03-11T19:51:40.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3581003f0 0x7fa3581999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3581003f0 0x7fa3581999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36842/0 (socket says 172.21.15.83:36842) 2024-03-11T19:51:40.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa3577fe640 1 -- 172.21.15.83:0/3050698749 learned_addr learned my addr 172.21.15.83:0/3050698749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa357fff640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3580ffa90 0x7fa358199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa35cf9f640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa358100d90 0x7fa3581a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa3577fe640 1 -- 172.21.15.83:0/3050698749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3580ffa90 msgr2=0x7fa358199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa3577fe640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3580ffa90 0x7fa358199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa3577fe640 1 -- 172.21.15.83:0/3050698749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa358100d90 msgr2=0x7fa3581a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa3577fe640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa358100d90 0x7fa3581a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa3577fe640 1 -- 172.21.15.83:0/3050698749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34c009490 con 0x7fa3581003f0 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa357fff640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3580ffa90 0x7fa358199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa3577fe640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3581003f0 0x7fa3581999d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00ea60 tx=0x7fa34c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.424+0000 7fa3557fa640 1 -- 172.21.15.83:0/3050698749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00cde0 con 0x7fa3581003f0 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.425+0000 7fa35ea02640 1 -- 172.21.15.83:0/3050698749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa350009900 con 0x7fa3581003f0 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.425+0000 7fa3557fa640 1 -- 172.21.15.83:0/3050698749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa34c0035f0 con 0x7fa3581003f0 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.425+0000 7fa3557fa640 1 -- 172.21.15.83:0/3050698749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c010690 con 0x7fa3581003f0 2024-03-11T19:51:40.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.425+0000 7fa35ea02640 1 -- 172.21.15.83:0/3050698749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3581a1960 con 0x7fa3581003f0 2024-03-11T19:51:40.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.428+0000 7fa3557fa640 1 -- 172.21.15.83:0/3050698749 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa34c003af0 con 0x7fa3581003f0 2024-03-11T19:51:40.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa3615d2740 1 Processor -- start 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa3615d2740 1 -- start start 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 0x55953c2fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa3557fa640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa3400a1a80 0x7fa3400a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c2fdc60 0x55953c302760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c302dd0 0x55953c305260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa3615d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953c2b9960 con 0x55953c302dd0 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa3615d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953c26d310 con 0x55953c2fd270 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa3615d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953c26a840 con 0x55953c2fdc60 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa357fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 0x55953c2fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa357fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 0x55953c2fd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:42288/0 (socket says 172.21.15.83:42288) 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa357fff640 1 -- 172.21.15.83:0/3698959915 learned_addr learned my addr 172.21.15.83:0/3698959915 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa3577fe640 1 --2- 172.21.15.83:0/3698959915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c2fdc60 0x55953c302760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa35cf9f640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa3400a1a80 0x7fa3400a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa3557fa640 1 -- 172.21.15.83:0/3050698749 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa34c014070 con 0x7fa3581003f0 2024-03-11T19:51:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa357fff640 1 -- 172.21.15.83:0/3698959915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c2fdc60 msgr2=0x55953c302760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa357fff640 1 --2- 172.21.15.83:0/3698959915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c2fdc60 0x55953c302760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.429+0000 7fa357fff640 1 -- 172.21.15.83:0/3698959915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c302dd0 msgr2=0x55953c305260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.430+0000 7fa35cf9f640 1 --2- 172.21.15.83:0/3698959915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c302dd0 0x55953c305260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.430+0000 7fa357fff640 1 --2- 172.21.15.83:0/3698959915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c302dd0 0x55953c305260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.430+0000 7fa357fff640 1 -- 172.21.15.83:0/3698959915 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55953c228060 con 0x55953c2fd270 2024-03-11T19:51:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.430+0000 7fa35cf9f640 1 --2- 172.21.15.83:0/3698959915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c302dd0 0x55953c305260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:51:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.430+0000 7fa357fff640 1 --2- 172.21.15.83:0/3698959915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 0x55953c2fd690 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa3480115f0 tx=0x7fa348011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb26c12f6e158a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.430+0000 7fa35cf9f640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa3400a1a80 0x7fa3400a3f40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa350006010 tx=0x7fa35002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.430+0000 7fa33e7fc640 1 -- 172.21.15.83:0/3698959915 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34800f640 con 0x55953c2fd270 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.430+0000 7fa33e7fc640 1 -- 172.21.15.83:0/3698959915 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa34800fc80 con 0x55953c2fd270 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.430+0000 7fa33e7fc640 1 -- 172.21.15.83:0/3698959915 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348019d00 con 0x55953c2fd270 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.431+0000 7fa3615d2740 1 -- 172.21.15.83:0/3698959915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 msgr2=0x55953c2fd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.431+0000 7fa3615d2740 1 --2- 172.21.15.83:0/3698959915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 0x55953c2fd690 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa3480115f0 tx=0x7fa348011ab0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.431+0000 7fa3615d2740 1 -- 172.21.15.83:0/3698959915 shutdown_connections 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.431+0000 7fa3615d2740 1 --2- 172.21.15.83:0/3698959915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c302dd0 0x55953c305260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.431+0000 7fa3615d2740 1 --2- 172.21.15.83:0/3698959915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c2fdc60 0x55953c302760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.431+0000 7fa3615d2740 1 --2- 172.21.15.83:0/3698959915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 0x55953c2fd690 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.431+0000 7fa3615d2740 1 -- 172.21.15.83:0/3698959915 >> 172.21.15.83:0/3698959915 conn(0x55953c2f87e0 msgr2=0x55953c2f8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.431+0000 7fa3615d2740 1 -- 172.21.15.83:0/3698959915 shutdown_connections 2024-03-11T19:51:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.431+0000 7fa3615d2740 1 -- 172.21.15.83:0/3698959915 wait complete. 2024-03-11T19:51:40.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.431+0000 7fa3615d2740 1 -- start start 2024-03-11T19:51:40.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.432+0000 7fa3615d2740 1 Processor -- start 2024-03-11T19:51:40.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.434+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fd270 0x55953c307480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.434+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fdc60 0x55953c3079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.434+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 0x55953c30ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.434+0000 7fa3615d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953c266020 con 0x55953c2fd270 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.434+0000 7fa35cf9f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 0x55953c30ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.434+0000 7fa35cf9f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 0x55953c30ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37420/0 (socket says 172.21.15.83:37420) 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.434+0000 7fa35cf9f640 1 -- 172.21.15.83:0/2299633871 learned_addr learned my addr 172.21.15.83:0/2299633871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.434+0000 7fa3615d2740 1 -- 172.21.15.83:0/2299633871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953bf39430 con 0x55953c2fdc60 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.434+0000 7fa3615d2740 1 -- 172.21.15.83:0/2299633871 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953bef5e00 con 0x55953c302dd0 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.434+0000 7fa3577fe640 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fdc60 0x55953c3079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa3615d2740 10 client.0 _ll_register_callbacks cb 0x55953c20dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa357fff640 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fd270 0x55953c307480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa35cf9f640 1 -- 172.21.15.83:0/2299633871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fdc60 msgr2=0x55953c3079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa35cf9f640 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fdc60 0x55953c3079c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa35cf9f640 1 -- 172.21.15.83:0/2299633871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fd270 msgr2=0x55953c307480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa35cf9f640 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fd270 0x55953c307480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa35cf9f640 1 -- 172.21.15.83:0/2299633871 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa348009d40 con 0x55953c302dd0 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa357fff640 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fd270 0x55953c307480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa3577fe640 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fdc60 0x55953c3079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa35cf9f640 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 0x55953c30ea00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa350009ca0 tx=0x7fa35002c120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57ca843e82379637 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa3377fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa3377fe640 1 -- 172.21.15.83:0/2299633871 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35004c630 con 0x55953c302dd0 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa3615d2740 1 -- 172.21.15.83:0/2299633871 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55953bf657e0 con 0x55953c302dd0 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa3377fe640 1 -- 172.21.15.83:0/2299633871 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa35004c790 con 0x55953c302dd0 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.435+0000 7fa3377fe640 1 -- 172.21.15.83:0/2299633871 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa35004c950 con 0x55953c302dd0 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.436+0000 7fa3615d2740 10 client.27256 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.436+0000 7fa3615d2740 1 -- 172.21.15.83:0/2299633871 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55953c2646a0 con 0x55953c302dd0 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.436+0000 7fa3377fe640 1 -- 172.21.15.83:0/2299633871 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55953bf657e0 con 0x55953c302dd0 2024-03-11T19:51:40.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.436+0000 7fa3377fe640 1 -- 172.21.15.83:0/2299633871 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa35004a040 con 0x55953c302dd0 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.436+0000 7fa3615d2740 10 client.27256 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.436+0000 7fa3615d2740 10 client.27256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.436+0000 7fa3615d2740 20 client.27256 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.436+0000 7fa3615d2740 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55953c3136d0 0x55953c315af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.436+0000 7fa3615d2740 4 client.27256 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.436+0000 7fa3615d2740 1 -- 172.21.15.83:0/2299633871 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55953bfa2720 con 0x55953c3136d0 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.436+0000 7fa357fff640 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55953c3136d0 0x55953c315af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.437+0000 7fa357fff640 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55953c3136d0 0x55953c315af0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68be837744599915 server_cookie=6ffa3ccfeb75bf3b in_seq=0 out_seq=0 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.437+0000 7fa3377fe640 10 client.27256 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.437+0000 7fa3377fe640 1 -- 172.21.15.83:0/2299633871 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55953bfa2720 con 0x55953c3136d0 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.437+0000 7fa3377fe640 10 client.27256 handle_command_reply: tid=0 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.437+0000 7fa3615d2740 1 client.27256 shutdown 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.438+0000 7fa3615d2740 1 -- 172.21.15.83:0/2299633871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 msgr2=0x55953c30ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.438+0000 7fa3615d2740 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 0x55953c30ea00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa350009ca0 tx=0x7fa35002c120 comp rx=0 tx=0).stop 2024-03-11T19:51:40.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.438+0000 7fa3615d2740 1 -- 172.21.15.83:0/2299633871 shutdown_connections 2024-03-11T19:51:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.438+0000 7fa3615d2740 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55953c3136d0 0x55953c315af0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.438+0000 7fa3615d2740 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 0x55953c30ea00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.438+0000 7fa3615d2740 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fdc60 0x55953c3079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.438+0000 7fa3615d2740 1 --2- 172.21.15.83:0/2299633871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fd270 0x55953c307480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.438+0000 7fa3615d2740 1 -- 172.21.15.83:0/2299633871 >> 172.21.15.83:0/2299633871 conn(0x55953c2f87e0 msgr2=0x55953c2f7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.438+0000 7fa3377fe640 0 client.27256 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.439+0000 7fa3615d2740 1 -- 172.21.15.83:0/2299633871 shutdown_connections 2024-03-11T19:51:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.439+0000 7fa3615d2740 1 -- 172.21.15.83:0/2299633871 wait complete. 2024-03-11T19:51:40.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.439+0000 7fa3615d2740 20 client.27256 trim_cache size 0 max 16384 2024-03-11T19:51:40.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.455+0000 7fa3615d2740 1 Processor -- start 2024-03-11T19:51:40.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3615d2740 1 -- start start 2024-03-11T19:51:40.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 0x55953c307480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fdc60 0x55953c3079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 0x55953c30e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3615d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953c1eb130 con 0x55953c2fdc60 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3615d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953c2feb30 con 0x55953c2fd270 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3615d2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953c2f45c0 con 0x55953c302dd0 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fdc60 0x55953c3079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fdc60 0x55953c3079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36862/0 (socket says 172.21.15.83:36862) 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3577fe640 1 -- 172.21.15.83:0/1537400589 learned_addr learned my addr 172.21.15.83:0/1537400589 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa357fff640 1 --2- 172.21.15.83:0/1537400589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 0x55953c307480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa35cf9f640 1 --2- 172.21.15.83:0/1537400589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 0x55953c30e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3577fe640 1 -- 172.21.15.83:0/1537400589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 msgr2=0x55953c30e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3577fe640 1 --2- 172.21.15.83:0/1537400589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 0x55953c30e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3577fe640 1 -- 172.21.15.83:0/1537400589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 msgr2=0x55953c307480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3577fe640 1 --2- 172.21.15.83:0/1537400589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 0x55953c307480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.456+0000 7fa3577fe640 1 -- 172.21.15.83:0/1537400589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55953c200c20 con 0x55953c2fdc60 2024-03-11T19:51:40.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.457+0000 7fa3577fe640 1 --2- 172.21.15.83:0/1537400589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fdc60 0x55953c3079c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa34c01f000 tx=0x7fa34c007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4c920f3ae39c85 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.457+0000 7fa33dffb640 1 -- 172.21.15.83:0/1537400589 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c09a7c0 con 0x55953c2fdc60 2024-03-11T19:51:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.457+0000 7fa33dffb640 1 -- 172.21.15.83:0/1537400589 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa34c09e910 con 0x55953c2fdc60 2024-03-11T19:51:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.457+0000 7fa33dffb640 1 -- 172.21.15.83:0/1537400589 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c096e30 con 0x55953c2fdc60 2024-03-11T19:51:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.458+0000 7fa3615d2740 1 -- 172.21.15.83:0/1537400589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fdc60 msgr2=0x55953c3079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.458+0000 7fa3615d2740 1 --2- 172.21.15.83:0/1537400589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fdc60 0x55953c3079c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa34c01f000 tx=0x7fa34c007b90 comp rx=0 tx=0).stop 2024-03-11T19:51:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.459+0000 7fa3615d2740 1 -- 172.21.15.83:0/1537400589 shutdown_connections 2024-03-11T19:51:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.459+0000 7fa3615d2740 1 --2- 172.21.15.83:0/1537400589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c302dd0 0x55953c30e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.459+0000 7fa3615d2740 1 --2- 172.21.15.83:0/1537400589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fdc60 0x55953c3079c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.459+0000 7fa3615d2740 1 --2- 172.21.15.83:0/1537400589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c2fd270 0x55953c307480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.459+0000 7fa3615d2740 1 -- 172.21.15.83:0/1537400589 >> 172.21.15.83:0/1537400589 conn(0x55953c2f87e0 msgr2=0x55953c2f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.459+0000 7fa3615d2740 1 -- 172.21.15.83:0/1537400589 shutdown_connections 2024-03-11T19:51:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.459+0000 7fa3615d2740 1 -- 172.21.15.83:0/1537400589 wait complete. 2024-03-11T19:51:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.459+0000 7fa3615d2740 1 -- start start 2024-03-11T19:51:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.460+0000 7fa3615d2740 1 Processor -- start 2024-03-11T19:51:40.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.460+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fd270 0x55953c30e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.460+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c2fdc60 0x55953c30cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.460+0000 7fa3615d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c302dd0 0x55953c30d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.460+0000 7fa3615d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953c30d800 con 0x55953c2fd270 2024-03-11T19:51:40.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.460+0000 7fa3615d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953c30d970 con 0x55953c302dd0 2024-03-11T19:51:40.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.460+0000 7fa35cf9f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c302dd0 0x55953c30d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.461+0000 7fa35cf9f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c302dd0 0x55953c30d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:42314/0 (socket says 172.21.15.83:42314) 2024-03-11T19:51:40.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.461+0000 7fa35cf9f640 1 -- 172.21.15.83:0/579175099 learned_addr learned my addr 172.21.15.83:0/579175099 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:40.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.461+0000 7fa3615d2740 1 -- 172.21.15.83:0/579175099 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953c30db30 con 0x55953c2fdc60 2024-03-11T19:51:40.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.461+0000 7fa3615d2740 10 client.0 _ll_register_callbacks cb 0x7fa348009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:40.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.464+0000 7fa3577fe640 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c2fdc60 0x55953c30cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.469+0000 7fa3577fe640 1 -- 172.21.15.83:0/579175099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c302dd0 msgr2=0x55953c30d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.469+0000 7fa3577fe640 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c302dd0 0x55953c30d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.469+0000 7fa3577fe640 1 -- 172.21.15.83:0/579175099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fd270 msgr2=0x55953c30e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.469+0000 7fa3577fe640 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fd270 0x55953c30e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.469+0000 7fa3577fe640 1 -- 172.21.15.83:0/579175099 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3581a1960 con 0x55953c2fdc60 2024-03-11T19:51:40.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.471+0000 7fa3577fe640 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c2fdc60 0x55953c30cd80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa34c097190 tx=0x7fa34c098cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3974a3a74b7a9bf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:40.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.472+0000 7fa3377fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:51:40.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.473+0000 7fa3377fe640 1 -- 172.21.15.83:0/579175099 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c003970 con 0x55953c2fdc60 2024-03-11T19:51:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.473+0000 7fa3377fe640 1 -- 172.21.15.83:0/579175099 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa34c07eab0 con 0x55953c2fdc60 2024-03-11T19:51:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.477+0000 7fa3615d2740 1 -- 172.21.15.83:0/579175099 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa34800fc80 con 0x55953c2fdc60 2024-03-11T19:51:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.480+0000 7fa3377fe640 1 -- 172.21.15.83:0/579175099 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa34c09a7c0 con 0x55953c2fdc60 2024-03-11T19:51:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.481+0000 7fa3615d2740 10 client.27262 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:40.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.482+0000 7fa3615d2740 1 -- 172.21.15.83:0/579175099 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55953c2268c0 con 0x55953c2fdc60 2024-03-11T19:51:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.487+0000 7fa3377fe640 1 -- 172.21.15.83:0/579175099 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa34c0b7020 con 0x55953c2fdc60 2024-03-11T19:51:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.487+0000 7fa3377fe640 1 -- 172.21.15.83:0/579175099 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa34c0b3020 con 0x55953c2fdc60 2024-03-11T19:51:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.487+0000 7fa3615d2740 10 client.27262 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.487+0000 7fa3615d2740 10 client.27262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.487+0000 7fa3615d2740 20 client.27262 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.488+0000 7fa3615d2740 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55953c36afa0 0x55953c36d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.488+0000 7fa3615d2740 4 client.27262 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:51:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.488+0000 7fa3615d2740 1 -- 172.21.15.83:0/579175099 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fa34800fc80 con 0x55953c36afa0 2024-03-11T19:51:40.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.489+0000 7fa357fff640 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55953c36afa0 0x55953c36d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:40.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.496+0000 7fa357fff640 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55953c36afa0 0x55953c36d3c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69476a4dfa0c173 server_cookie=cb10d8ffd1ecd923 in_seq=0 out_seq=0 2024-03-11T19:51:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.496+0000 7fa3377fe640 10 client.27262 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.497+0000 7fa3377fe640 1 -- 172.21.15.83:0/579175099 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fa34800fc80 con 0x55953c36afa0 2024-03-11T19:51:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.497+0000 7fa3377fe640 10 client.27262 handle_command_reply: tid=0 2024-03-11T19:51:40.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.497+0000 7fa3615d2740 1 client.27262 shutdown 2024-03-11T19:51:40.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.501+0000 7fa3615d2740 1 -- 172.21.15.83:0/579175099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c2fdc60 msgr2=0x55953c30cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.501+0000 7fa3615d2740 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c2fdc60 0x55953c30cd80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa34c097190 tx=0x7fa34c098cd0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.501+0000 7fa3615d2740 1 -- 172.21.15.83:0/579175099 shutdown_connections 2024-03-11T19:51:40.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.501+0000 7fa3615d2740 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55953c36afa0 0x55953c36d3c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.501+0000 7fa3615d2740 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55953c302dd0 0x55953c30d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.501+0000 7fa3615d2740 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55953c2fdc60 0x55953c30cd80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.501+0000 7fa3615d2740 1 --2- 172.21.15.83:0/579175099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55953c2fd270 0x55953c30e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.501+0000 7fa3615d2740 1 -- 172.21.15.83:0/579175099 >> 172.21.15.83:0/579175099 conn(0x55953c2f87e0 msgr2=0x55953c307480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.501+0000 7fa3377fe640 0 client.27262 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.502+0000 7fa3615d2740 1 -- 172.21.15.83:0/579175099 shutdown_connections 2024-03-11T19:51:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.502+0000 7fa3615d2740 1 -- 172.21.15.83:0/579175099 wait complete. 2024-03-11T19:51:40.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.502+0000 7fa3615d2740 20 client.27262 trim_cache size 0 max 16384 2024-03-11T19:51:40.507 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:51:40.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.504+0000 7fa35ea02640 1 -- 172.21.15.83:0/3050698749 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa3400a1a80 msgr2=0x7fa3400a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.504+0000 7fa35ea02640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa3400a1a80 0x7fa3400a3f40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa350006010 tx=0x7fa35002e040 comp rx=0 tx=0).stop 2024-03-11T19:51:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.504+0000 7fa35ea02640 1 -- 172.21.15.83:0/3050698749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3581003f0 msgr2=0x7fa3581999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.505+0000 7fa35ea02640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3581003f0 0x7fa3581999d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00ea60 tx=0x7fa34c00ef20 comp rx=0 tx=0).stop 2024-03-11T19:51:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.505+0000 7fa35ea02640 1 -- 172.21.15.83:0/3050698749 shutdown_connections 2024-03-11T19:51:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.505+0000 7fa35ea02640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa3400a1a80 0x7fa3400a3f40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.505+0000 7fa35ea02640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa358100d90 0x7fa3581a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.505+0000 7fa35ea02640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3581003f0 0x7fa3581999d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.505+0000 7fa35ea02640 1 --2- 172.21.15.83:0/3050698749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3580ffa90 0x7fa358199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.505+0000 7fa35ea02640 1 -- 172.21.15.83:0/3050698749 >> 172.21.15.83:0/3050698749 conn(0x7fa3580fb630 msgr2=0x7fa3580fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.505+0000 7fa35ea02640 1 -- 172.21.15.83:0/3050698749 shutdown_connections 2024-03-11T19:51:40.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:40.505+0000 7fa35ea02640 1 -- 172.21.15.83:0/3050698749 wait complete. 2024-03-11T19:51:40.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:40 smithi083 ceph-mon[31864]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.8 MiB/s wr, 705 op/s 2024-03-11T19:51:40.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:40 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1662148926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:51:40.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:40 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:51:40.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:40 smithi137 ceph-mon[36220]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.8 MiB/s wr, 705 op/s 2024-03-11T19:51:40.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:40 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1662148926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:51:40.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:40 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:51:40.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:40 smithi204 ceph-mon[36739]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.8 MiB/s wr, 705 op/s 2024-03-11T19:51:40.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:40 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1662148926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:51:40.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:40 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:51:41.532 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:51:41.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:51:41.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:51:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.707+0000 7f73749e2640 1 -- 172.21.15.83:0/2687096810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7370109350 msgr2=0x7f737010fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.707+0000 7f73749e2640 1 --2- 172.21.15.83:0/2687096810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7370109350 0x7f737010fbe0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7368009ca0 tx=0x7f736802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.708+0000 7f73749e2640 1 -- 172.21.15.83:0/2687096810 shutdown_connections 2024-03-11T19:51:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.708+0000 7f73749e2640 1 --2- 172.21.15.83:0/2687096810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7370109350 0x7f737010fbe0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.708+0000 7f73749e2640 1 --2- 172.21.15.83:0/2687096810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7370108990 0x7f7370108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.708+0000 7f73749e2640 1 --2- 172.21.15.83:0/2687096810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73701077a0 0x7f7370107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.708+0000 7f73749e2640 1 -- 172.21.15.83:0/2687096810 >> 172.21.15.83:0/2687096810 conn(0x7f7370076740 msgr2=0x7f7370078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.708+0000 7f73749e2640 1 -- 172.21.15.83:0/2687096810 shutdown_connections 2024-03-11T19:51:41.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.708+0000 7f73749e2640 1 -- 172.21.15.83:0/2687096810 wait complete. 2024-03-11T19:51:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.708+0000 7f73749e2640 1 Processor -- start 2024-03-11T19:51:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f73749e2640 1 -- start start 2024-03-11T19:51:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f73749e2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73701077a0 0x7f737019d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f73749e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7370108990 0x7f737019dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f73749e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7370109350 0x7f73701a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f736dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7370108990 0x7f737019dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f736dd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7370108990 0x7f737019dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36882/0 (socket says 172.21.15.83:36882) 2024-03-11T19:51:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f736dd9b640 1 -- 172.21.15.83:0/715633382 learned_addr learned my addr 172.21.15.83:0/715633382 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f73749e2640 1 -- 172.21.15.83:0/715633382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73701a52e0 con 0x7f7370108990 2024-03-11T19:51:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f73749e2640 1 -- 172.21.15.83:0/715633382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73701a5450 con 0x7f7370109350 2024-03-11T19:51:41.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f73749e2640 1 -- 172.21.15.83:0/715633382 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73701a5610 con 0x7f73701077a0 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f736ed9d640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7370109350 0x7f73701a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f736e59c640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73701077a0 0x7f737019d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f736dd9b640 1 -- 172.21.15.83:0/715633382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73701077a0 msgr2=0x7f737019d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f736dd9b640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73701077a0 0x7f737019d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f736dd9b640 1 -- 172.21.15.83:0/715633382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7370109350 msgr2=0x7f73701a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f736dd9b640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7370109350 0x7f73701a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f736dd9b640 1 -- 172.21.15.83:0/715633382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7368009900 con 0x7f7370108990 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.709+0000 7f736ed9d640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7370109350 0x7f73701a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.710+0000 7f736dd9b640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7370108990 0x7f737019dd30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f736400e9f0 tx=0x7f736400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.710+0000 7f735f7fe640 1 -- 172.21.15.83:0/715633382 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364009720 con 0x7f7370108990 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.710+0000 7f73749e2640 1 -- 172.21.15.83:0/715633382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73701a58f0 con 0x7f7370108990 2024-03-11T19:51:41.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.710+0000 7f73749e2640 1 -- 172.21.15.83:0/715633382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73701a5e00 con 0x7f7370108990 2024-03-11T19:51:41.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.710+0000 7f735f7fe640 1 -- 172.21.15.83:0/715633382 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f736400cd80 con 0x7f7370108990 2024-03-11T19:51:41.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.710+0000 7f735f7fe640 1 -- 172.21.15.83:0/715633382 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364010690 con 0x7f7370108990 2024-03-11T19:51:41.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.713+0000 7f735f7fe640 1 -- 172.21.15.83:0/715633382 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f73640046c0 con 0x7f7370108990 2024-03-11T19:51:41.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.713+0000 7f73775b1740 1 Processor -- start 2024-03-11T19:51:41.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.713+0000 7f735f7fe640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f73580a1b30 0x7f73580a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f736ed9d640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f73580a1b30 0x7f73580a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f73775b1740 1 -- start start 2024-03-11T19:51:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f735f7fe640 1 -- 172.21.15.83:0/715633382 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7364014070 con 0x7f7370108990 2024-03-11T19:51:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f73775b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f73775b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b75690 0x562022b75b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f73775b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b76050 0x562022b7c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f73775b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562022ae3f90 con 0x562022b76050 2024-03-11T19:51:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f73775b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562022ae14c0 con 0x562022b744c0 2024-03-11T19:51:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f73775b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562022adcca0 con 0x562022b75690 2024-03-11T19:51:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f736dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b75690 0x562022b75b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f736dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b75690 0x562022b75b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37470/0 (socket says 172.21.15.83:37470) 2024-03-11T19:51:41.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f736dd9b640 1 -- 172.21.15.83:0/2146233720 learned_addr learned my addr 172.21.15.83:0/2146233720 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f736ed9d640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f73580a1b30 0x7f73580a3ff0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7368005e30 tx=0x7f73680326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f736ed9d640 1 --2- 172.21.15.83:0/2146233720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b76050 0x562022b7c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.714+0000 7f736e59c640 1 --2- 172.21.15.83:0/2146233720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.715+0000 7f736dd9b640 1 -- 172.21.15.83:0/2146233720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 msgr2=0x562022b748e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.715+0000 7f736dd9b640 1 --2- 172.21.15.83:0/2146233720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b748e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.715+0000 7f736dd9b640 1 -- 172.21.15.83:0/2146233720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b76050 msgr2=0x562022b7c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.715+0000 7f736dd9b640 1 --2- 172.21.15.83:0/2146233720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b76050 0x562022b7c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.715+0000 7f736dd9b640 1 -- 172.21.15.83:0/2146233720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562022a9f0b0 con 0x562022b75690 2024-03-11T19:51:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.715+0000 7f736e59c640 1 --2- 172.21.15.83:0/2146233720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b748e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:51:41.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.715+0000 7f736dd9b640 1 --2- 172.21.15.83:0/2146233720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b75690 0x562022b75b10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7364096a40 tx=0x7f7364099f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ea74281137b945 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:41.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.715+0000 7f735cff9640 1 -- 172.21.15.83:0/2146233720 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736409e730 con 0x562022b75690 2024-03-11T19:51:41.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.715+0000 7f735cff9640 1 -- 172.21.15.83:0/2146233720 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73640a2d80 con 0x562022b75690 2024-03-11T19:51:41.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.715+0000 7f735cff9640 1 -- 172.21.15.83:0/2146233720 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73640850c0 con 0x562022b75690 2024-03-11T19:51:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.716+0000 7f73775b1740 1 -- 172.21.15.83:0/2146233720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b75690 msgr2=0x562022b75b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.716+0000 7f73775b1740 1 --2- 172.21.15.83:0/2146233720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b75690 0x562022b75b10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7364096a40 tx=0x7f7364099f40 comp rx=0 tx=0).stop 2024-03-11T19:51:41.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.716+0000 7f73775b1740 1 -- 172.21.15.83:0/2146233720 shutdown_connections 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.716+0000 7f73775b1740 1 --2- 172.21.15.83:0/2146233720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b76050 0x562022b7c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.716+0000 7f73775b1740 1 --2- 172.21.15.83:0/2146233720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b75690 0x562022b75b10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.716+0000 7f73775b1740 1 --2- 172.21.15.83:0/2146233720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b748e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.716+0000 7f73775b1740 1 -- 172.21.15.83:0/2146233720 >> 172.21.15.83:0/2146233720 conn(0x562022b6f6a0 msgr2=0x562022b6faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.716+0000 7f73775b1740 1 -- 172.21.15.83:0/2146233720 shutdown_connections 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f73775b1740 1 -- 172.21.15.83:0/2146233720 wait complete. 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f73775b1740 1 -- start start 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f73775b1740 1 Processor -- start 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f73775b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b7e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f73775b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b75690 0x562022b7ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f73775b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b76050 0x562022b85cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f73775b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562022a81f40 con 0x562022b75690 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f73775b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620227b04b0 con 0x562022b744c0 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f73775b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562022a62670 con 0x562022b76050 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f736e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b7e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f736e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b7e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:42354/0 (socket says 172.21.15.83:42354) 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f736e59c640 1 -- 172.21.15.83:0/2517634448 learned_addr learned my addr 172.21.15.83:0/2517634448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f736dd9b640 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b75690 0x562022b7ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f736ed9d640 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b76050 0x562022b85cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.717+0000 7f73775b1740 10 client.0 _ll_register_callbacks cb 0x562022acff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f736e59c640 1 -- 172.21.15.83:0/2517634448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b76050 msgr2=0x562022b85cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f736e59c640 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b76050 0x562022b85cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f736e59c640 1 -- 172.21.15.83:0/2517634448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b75690 msgr2=0x562022b7ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f736e59c640 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b75690 0x562022b7ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f736e59c640 1 -- 172.21.15.83:0/2517634448 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73701a5e00 con 0x562022b744c0 2024-03-11T19:51:41.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f73775b1740 1 -- 172.21.15.83:0/2517634448 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562022ab6f40 con 0x562022b744c0 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f736e59c640 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b7e770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7360013710 tx=0x7f7360009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1296e4c196ad7927 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f7355ffb640 10 client.27164 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f7355ffb640 1 -- 172.21.15.83:0/2517634448 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360011dd0 con 0x562022b744c0 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f7355ffb640 1 -- 172.21.15.83:0/2517634448 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7360004020 con 0x562022b744c0 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f7355ffb640 1 -- 172.21.15.83:0/2517634448 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7360004220 con 0x562022b744c0 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f73775b1740 10 client.27164 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.718+0000 7f73775b1740 1 -- 172.21.15.83:0/2517634448 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x562022adb320 con 0x562022b744c0 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.719+0000 7f7355ffb640 1 -- 172.21.15.83:0/2517634448 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7360022020 con 0x562022b744c0 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.719+0000 7f73775b1740 10 client.27164 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.719+0000 7f73775b1740 10 client.27164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.719+0000 7f73775b1740 20 client.27164 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.719+0000 7f73775b1740 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562022b8a6f0 0x562022b8cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.719+0000 7f73775b1740 4 client.27164 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.719+0000 7f73775b1740 1 -- 172.21.15.83:0/2517634448 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562022ad3310 con 0x562022b8a6f0 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.719+0000 7f736dd9b640 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562022b8a6f0 0x562022b8cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.719+0000 7f736dd9b640 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562022b8a6f0 0x562022b8cb50 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62be87a98a80a107 server_cookie=1ebe81e15f8850df in_seq=0 out_seq=0 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.719+0000 7f7355ffb640 10 client.27164 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.720+0000 7f7355ffb640 1 -- 172.21.15.83:0/2517634448 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562022ad3310 con 0x562022b8a6f0 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.720+0000 7f7355ffb640 10 client.27164 handle_command_reply: tid=0 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.720+0000 7f73775b1740 1 client.27164 shutdown 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.720+0000 7f73775b1740 1 -- 172.21.15.83:0/2517634448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 msgr2=0x562022b7e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.720+0000 7f73775b1740 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b7e770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7360013710 tx=0x7f7360009f90 comp rx=0 tx=0).stop 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.720+0000 7f73775b1740 1 -- 172.21.15.83:0/2517634448 shutdown_connections 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.720+0000 7f73775b1740 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562022b8a6f0 0x562022b8cb50 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.720+0000 7f7355ffb640 0 client.27164 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.721+0000 7f73775b1740 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b76050 0x562022b85cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.721+0000 7f73775b1740 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b75690 0x562022b7ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.721+0000 7f73775b1740 1 --2- 172.21.15.83:0/2517634448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b7e770 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.721+0000 7f73775b1740 1 -- 172.21.15.83:0/2517634448 >> 172.21.15.83:0/2517634448 conn(0x562022b6f6a0 msgr2=0x562022b6e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:41.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.721+0000 7f73775b1740 1 -- 172.21.15.83:0/2517634448 shutdown_connections 2024-03-11T19:51:41.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.721+0000 7f73775b1740 1 -- 172.21.15.83:0/2517634448 wait complete. 2024-03-11T19:51:41.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.721+0000 7f73775b1740 20 client.27164 trim_cache size 0 max 16384 2024-03-11T19:51:41.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.758+0000 7f73775b1740 1 Processor -- start 2024-03-11T19:51:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.763+0000 7f73775b1740 1 -- start start 2024-03-11T19:51:41.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.763+0000 7f73775b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b744c0 0x562022b73010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.763+0000 7f73775b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b75690 0x562022b85cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.763+0000 7f73775b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b76050 0x562022b7da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.763+0000 7f73775b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562022b871e0 con 0x562022b76050 2024-03-11T19:51:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.763+0000 7f73775b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562022b91da0 con 0x562022b75690 2024-03-11T19:51:41.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.764+0000 7f73775b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562022b7cc50 con 0x562022b744c0 2024-03-11T19:51:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.764+0000 7f736e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b744c0 0x562022b73010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.764+0000 7f736e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b744c0 0x562022b73010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37486/0 (socket says 172.21.15.83:37486) 2024-03-11T19:51:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.764+0000 7f736e59c640 1 -- 172.21.15.83:0/3686633927 learned_addr learned my addr 172.21.15.83:0/3686633927 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:41.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.764+0000 7f736dd9b640 1 --2- 172.21.15.83:0/3686633927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b75690 0x562022b85cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.765+0000 7f736e59c640 1 -- 172.21.15.83:0/3686633927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b75690 msgr2=0x562022b85cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:41.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.765+0000 7f736e59c640 1 --2- 172.21.15.83:0/3686633927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b75690 0x562022b85cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.765+0000 7f736e59c640 1 -- 172.21.15.83:0/3686633927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b76050 msgr2=0x562022b7da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:41.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.765+0000 7f736e59c640 1 --2- 172.21.15.83:0/3686633927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b76050 0x562022b7da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.765+0000 7f736e59c640 1 -- 172.21.15.83:0/3686633927 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562022a77c00 con 0x562022b744c0 2024-03-11T19:51:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.770+0000 7f736e59c640 1 --2- 172.21.15.83:0/3686633927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b744c0 0x562022b73010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7360009820 tx=0x7f7360016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd4dec6c5885505b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.770+0000 7f7357fff640 1 -- 172.21.15.83:0/3686633927 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360011dd0 con 0x562022b744c0 2024-03-11T19:51:41.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.770+0000 7f7357fff640 1 -- 172.21.15.83:0/3686633927 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562022ab6f40 con 0x562022b744c0 2024-03-11T19:51:41.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.770+0000 7f7357fff640 1 -- 172.21.15.83:0/3686633927 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360025690 con 0x562022b744c0 2024-03-11T19:51:41.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.771+0000 7f73775b1740 1 -- 172.21.15.83:0/3686633927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b744c0 msgr2=0x562022b73010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.771+0000 7f73775b1740 1 --2- 172.21.15.83:0/3686633927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b744c0 0x562022b73010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7360009820 tx=0x7f7360016000 comp rx=0 tx=0).stop 2024-03-11T19:51:41.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.772+0000 7f73775b1740 1 -- 172.21.15.83:0/3686633927 shutdown_connections 2024-03-11T19:51:41.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.772+0000 7f73775b1740 1 --2- 172.21.15.83:0/3686633927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b76050 0x562022b7da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.772+0000 7f73775b1740 1 --2- 172.21.15.83:0/3686633927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b75690 0x562022b85cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.772+0000 7f73775b1740 1 --2- 172.21.15.83:0/3686633927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b744c0 0x562022b73010 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.772+0000 7f73775b1740 1 -- 172.21.15.83:0/3686633927 >> 172.21.15.83:0/3686633927 conn(0x562022b6f6a0 msgr2=0x562022b72900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:41.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.776+0000 7f73775b1740 1 -- 172.21.15.83:0/3686633927 shutdown_connections 2024-03-11T19:51:41.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.777+0000 7f73775b1740 1 -- 172.21.15.83:0/3686633927 wait complete. 2024-03-11T19:51:41.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.777+0000 7f73775b1740 1 -- start start 2024-03-11T19:51:41.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.778+0000 7f73775b1740 1 Processor -- start 2024-03-11T19:51:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.778+0000 7f73775b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b84310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.778+0000 7f73775b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b75690 0x562022b84850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f73775b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b76050 0x562022b84d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f736e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b84310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f736e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b84310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:42380/0 (socket says 172.21.15.83:42380) 2024-03-11T19:51:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f736e59c640 1 -- 172.21.15.83:0/4062815239 learned_addr learned my addr 172.21.15.83:0/4062815239 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f73775b1740 1 -- 172.21.15.83:0/4062815239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x562022b85f60 con 0x562022b75690 2024-03-11T19:51:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f736ed9d640 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b76050 0x562022b84d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f73775b1740 1 -- 172.21.15.83:0/4062815239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562022be0c60 con 0x562022b744c0 2024-03-11T19:51:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f736dd9b640 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b75690 0x562022b84850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f73775b1740 1 -- 172.21.15.83:0/4062815239 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562022be0da0 con 0x562022b76050 2024-03-11T19:51:41.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f73775b1740 10 client.0 _ll_register_callbacks cb 0x7f736400cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:41.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f736ed9d640 1 -- 172.21.15.83:0/4062815239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 msgr2=0x562022b84310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:41.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f736ed9d640 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b84310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.779+0000 7f736ed9d640 1 -- 172.21.15.83:0/4062815239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b75690 msgr2=0x562022b84850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:41.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.780+0000 7f736ed9d640 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b75690 0x562022b84850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.780+0000 7f736ed9d640 1 -- 172.21.15.83:0/4062815239 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562022adb320 con 0x562022b76050 2024-03-11T19:51:41.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.780+0000 7f736ed9d640 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b76050 0x562022b84d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f736802c630 tx=0x7f7368034ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d7ce5db2fccfe server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:41.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.780+0000 7f7355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:51:41.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.780+0000 7f7355ffb640 1 -- 172.21.15.83:0/4062815239 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368004040 con 0x562022b76050 2024-03-11T19:51:41.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.780+0000 7f7355ffb640 1 -- 172.21.15.83:0/4062815239 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7368047040 con 0x562022b76050 2024-03-11T19:51:41.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.781+0000 7f73775b1740 1 -- 172.21.15.83:0/4062815239 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562022b02900 con 0x562022b76050 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.781+0000 7f7355ffb640 1 -- 172.21.15.83:0/4062815239 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73680041a0 con 0x562022b76050 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.781+0000 7f73775b1740 10 client.27280 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.781+0000 7f73775b1740 1 -- 172.21.15.83:0/4062815239 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562022a9d910 con 0x562022b76050 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.782+0000 7f7355ffb640 1 -- 172.21.15.83:0/4062815239 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73680047e0 con 0x562022b76050 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.782+0000 7f7355ffb640 1 -- 172.21.15.83:0/4062815239 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7368061420 con 0x562022b76050 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.782+0000 7f73775b1740 10 client.27280 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.782+0000 7f73775b1740 10 client.27280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.782+0000 7f73775b1740 20 client.27280 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.782+0000 7f73775b1740 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562022b8a6f0 0x562022be4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.782+0000 7f73775b1740 4 client.27280 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.782+0000 7f73775b1740 1 -- 172.21.15.83:0/4062815239 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562022b02900 con 0x562022b8a6f0 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.783+0000 7f736e59c640 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562022b8a6f0 0x562022be4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.783+0000 7f7355ffb640 10 client.27280 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:41.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.783+0000 7f736e59c640 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562022b8a6f0 0x562022be4800 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9646fa99c2c4ec8 server_cookie=7e88efd9a653ad0e in_seq=0 out_seq=0 2024-03-11T19:51:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.784+0000 7f7355ffb640 1 -- 172.21.15.83:0/4062815239 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562022b02900 con 0x562022b8a6f0 2024-03-11T19:51:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.784+0000 7f7355ffb640 10 client.27280 handle_command_reply: tid=0 2024-03-11T19:51:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.784+0000 7f73775b1740 1 client.27280 shutdown 2024-03-11T19:51:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.784+0000 7f73775b1740 1 -- 172.21.15.83:0/4062815239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b76050 msgr2=0x562022b84d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.784+0000 7f73775b1740 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b76050 0x562022b84d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f736802c630 tx=0x7f7368034ae0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f73775b1740 1 -- 172.21.15.83:0/4062815239 shutdown_connections 2024-03-11T19:51:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f73775b1740 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x562022b8a6f0 0x562022be4800 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f73775b1740 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562022b76050 0x562022b84d90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f73775b1740 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562022b75690 0x562022b84850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f73775b1740 1 --2- 172.21.15.83:0/4062815239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562022b744c0 0x562022b84310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f73775b1740 1 -- 172.21.15.83:0/4062815239 >> 172.21.15.83:0/4062815239 conn(0x562022b6f6a0 msgr2=0x562022b73190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:41.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f7355ffb640 0 client.27280 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:41.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f73775b1740 1 -- 172.21.15.83:0/4062815239 shutdown_connections 2024-03-11T19:51:41.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f73775b1740 1 -- 172.21.15.83:0/4062815239 wait complete. 2024-03-11T19:51:41.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f73775b1740 20 client.27280 trim_cache size 0 max 16384 2024-03-11T19:51:41.791 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:51:41.791 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:51:41.791 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "ee4d5a51-bb41-41e8-ace2-f068d0a9c50e", 2024-03-11T19:51:41.791 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:51:41.791 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:51:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f73749e2640 1 -- 172.21.15.83:0/715633382 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f73580a1b30 msgr2=0x7f73580a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.785+0000 7f73749e2640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f73580a1b30 0x7f73580a3ff0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7368005e30 tx=0x7f73680326b0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.786+0000 7f73749e2640 1 -- 172.21.15.83:0/715633382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7370108990 msgr2=0x7f737019dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.786+0000 7f73749e2640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7370108990 0x7f737019dd30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f736400e9f0 tx=0x7f736400edf0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.786+0000 7f73749e2640 1 -- 172.21.15.83:0/715633382 shutdown_connections 2024-03-11T19:51:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.786+0000 7f73749e2640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f73580a1b30 0x7f73580a3ff0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.787+0000 7f73749e2640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7370109350 0x7f73701a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.787+0000 7f73749e2640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7370108990 0x7f737019dd30 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.787+0000 7f73749e2640 1 --2- 172.21.15.83:0/715633382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73701077a0 0x7f737019d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.787+0000 7f73749e2640 1 -- 172.21.15.83:0/715633382 >> 172.21.15.83:0/715633382 conn(0x7f7370076740 msgr2=0x7f737010bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.787+0000 7f73749e2640 1 -- 172.21.15.83:0/715633382 shutdown_connections 2024-03-11T19:51:41.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.787+0000 7f73749e2640 1 -- 172.21.15.83:0/715633382 wait complete. 2024-03-11T19:51:41.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:51:41.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:51:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.968+0000 7f6aef623640 1 -- 172.21.15.83:0/2341441369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ae81089b0 msgr2=0x7f6ae8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.968+0000 7f6aef623640 1 --2- 172.21.15.83:0/2341441369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ae81089b0 0x7f6ae8108e30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009900 tx=0x7f6ad802bc60 comp rx=0 tx=0).stop 2024-03-11T19:51:41.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.977+0000 7f6aef623640 1 -- 172.21.15.83:0/2341441369 shutdown_connections 2024-03-11T19:51:41.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.977+0000 7f6aef623640 1 --2- 172.21.15.83:0/2341441369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae8109370 0x7f6ae810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.977+0000 7f6aef623640 1 --2- 172.21.15.83:0/2341441369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ae81089b0 0x7f6ae8108e30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.977+0000 7f6aef623640 1 --2- 172.21.15.83:0/2341441369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ae81077c0 0x7f6ae8107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.977+0000 7f6aef623640 1 -- 172.21.15.83:0/2341441369 >> 172.21.15.83:0/2341441369 conn(0x7f6ae8076770 msgr2=0x7f6ae8078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.983+0000 7f6aef623640 1 -- 172.21.15.83:0/2341441369 shutdown_connections 2024-03-11T19:51:41.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.985+0000 7f6aef623640 1 -- 172.21.15.83:0/2341441369 wait complete. 2024-03-11T19:51:41.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.985+0000 7f6aef623640 1 Processor -- start 2024-03-11T19:51:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.989+0000 7f6aef623640 1 -- start start 2024-03-11T19:51:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.989+0000 7f6aef623640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ae81077c0 0x7f6ae807e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.989+0000 7f6aef623640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ae81089b0 0x7f6ae807ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.989+0000 7f6aef623640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae8109370 0x7f6ae81adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.990+0000 7f6aef623640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae8081d30 con 0x7f6ae81077c0 2024-03-11T19:51:41.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.990+0000 7f6aecbbe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ae81089b0 0x7f6ae807ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.990+0000 7f6aecbbe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ae81089b0 0x7f6ae807ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:37516/0 (socket says 172.21.15.83:37516) 2024-03-11T19:51:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.990+0000 7f6aecbbe640 1 -- 172.21.15.83:0/3248743415 learned_addr learned my addr 172.21.15.83:0/3248743415 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.990+0000 7f6aef623640 1 -- 172.21.15.83:0/3248743415 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae8081ea0 con 0x7f6ae8109370 2024-03-11T19:51:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.990+0000 7f6aef623640 1 -- 172.21.15.83:0/3248743415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae8082060 con 0x7f6ae81089b0 2024-03-11T19:51:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.990+0000 7f6aecbbe640 1 -- 172.21.15.83:0/3248743415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae8109370 msgr2=0x7f6ae81adb10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-11T19:51:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.990+0000 7f6aecbbe640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae8109370 0x7f6ae81adb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.990+0000 7f6aecbbe640 1 -- 172.21.15.83:0/3248743415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ae81077c0 msgr2=0x7f6ae807e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.990+0000 7f6aecbbe640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ae81077c0 0x7f6ae807e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:41.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.990+0000 7f6aecbbe640 1 -- 172.21.15.83:0/3248743415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad8009560 con 0x7f6ae81089b0 2024-03-11T19:51:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.992+0000 7f6aecbbe640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ae81089b0 0x7f6ae807ed30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8000c00 tx=0x7f6ad8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.992+0000 7f6ade7fc640 1 -- 172.21.15.83:0/3248743415 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8004df0 con 0x7f6ae81089b0 2024-03-11T19:51:41.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.992+0000 7f6aef623640 1 -- 172.21.15.83:0/3248743415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae81ae160 con 0x7f6ae81089b0 2024-03-11T19:51:41.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.992+0000 7f6aef623640 1 -- 172.21.15.83:0/3248743415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae81ae680 con 0x7f6ae81089b0 2024-03-11T19:51:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.996+0000 7f6af21f6740 1 Processor -- start 2024-03-11T19:51:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.998+0000 7f6af21f6740 1 -- start start 2024-03-11T19:51:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.998+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786ddd4c0 0x557786ddd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.998+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786dddeb0 0x557786de29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.998+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786de3020 0x557786de54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.998+0000 7f6af21f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557786d4d280 con 0x557786dddeb0 2024-03-11T19:51:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.998+0000 7f6af21f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557786d4a7b0 con 0x557786ddd4c0 2024-03-11T19:51:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.998+0000 7f6af21f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557786d45f90 con 0x557786de3020 2024-03-11T19:51:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.998+0000 7f6aedbc0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786ddd4c0 0x557786ddd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.998+0000 7f6aedbc0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786ddd4c0 0x557786ddd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:42400/0 (socket says 172.21.15.83:42400) 2024-03-11T19:51:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.998+0000 7f6aedbc0640 1 -- 172.21.15.83:0/1317511220 learned_addr learned my addr 172.21.15.83:0/1317511220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.999+0000 7f6ade7fc640 1 -- 172.21.15.83:0/3248743415 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ad8005430 con 0x7f6ae81089b0 2024-03-11T19:51:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.999+0000 7f6aecbbe640 1 --2- 172.21.15.83:0/1317511220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786de3020 0x557786de54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.999+0000 7f6ade7fc640 1 -- 172.21.15.83:0/3248743415 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8003e80 con 0x7f6ae81089b0 2024-03-11T19:51:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.999+0000 7f6aecbbe640 1 -- 172.21.15.83:0/1317511220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786ddd4c0 msgr2=0x557786ddd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.999+0000 7f6aecbbe640 1 --2- 172.21.15.83:0/1317511220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786ddd4c0 0x557786ddd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.999+0000 7f6aecbbe640 1 -- 172.21.15.83:0/1317511220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786dddeb0 msgr2=0x557786de29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.999+0000 7f6aecbbe640 1 --2- 172.21.15.83:0/1317511220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786dddeb0 0x557786de29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:41.999+0000 7f6aecbbe640 1 -- 172.21.15.83:0/1317511220 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557786d04570 con 0x557786de3020 2024-03-11T19:51:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.000+0000 7f6ade7fc640 1 -- 172.21.15.83:0/3248743415 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6ad803e450 con 0x7f6ae81089b0 2024-03-11T19:51:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.000+0000 7f6aecbbe640 1 --2- 172.21.15.83:0/1317511220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786de3020 0x557786de54b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80ee850 tx=0x7f6ad80f4de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f821cb1184b27837 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.007+0000 7f6ade7fc640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ad40a1bb0 0x7f6ad40a4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.007+0000 7f6ade7fc640 1 -- 172.21.15.83:0/3248743415 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6ad80f0d20 con 0x7f6ae81089b0 2024-03-11T19:51:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.008+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/1317511220 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8005160 con 0x557786de3020 2024-03-11T19:51:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.008+0000 7f6ad2ffd640 1 -- 172.21.15.83:0/1317511220 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ad80052c0 con 0x557786de3020 2024-03-11T19:51:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.009+0000 7f6af21f6740 1 -- 172.21.15.83:0/1317511220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786de3020 msgr2=0x557786de54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.009+0000 7f6af21f6740 1 --2- 172.21.15.83:0/1317511220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786de3020 0x557786de54b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80ee850 tx=0x7f6ad80f4de0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.009+0000 7f6af21f6740 1 -- 172.21.15.83:0/1317511220 shutdown_connections 2024-03-11T19:51:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.009+0000 7f6af21f6740 1 --2- 172.21.15.83:0/1317511220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786de3020 0x557786de54b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.009+0000 7f6af21f6740 1 --2- 172.21.15.83:0/1317511220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786dddeb0 0x557786de29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.009+0000 7f6af21f6740 1 --2- 172.21.15.83:0/1317511220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786ddd4c0 0x557786ddd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.009+0000 7f6af21f6740 1 -- 172.21.15.83:0/1317511220 >> 172.21.15.83:0/1317511220 conn(0x557786dd8b30 msgr2=0x557786dd8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.012+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ad40a1bb0 0x7f6ad40a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.013+0000 7f6af21f6740 1 -- 172.21.15.83:0/1317511220 shutdown_connections 2024-03-11T19:51:42.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.013+0000 7f6af21f6740 1 -- 172.21.15.83:0/1317511220 wait complete. 2024-03-11T19:51:42.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.014+0000 7f6af21f6740 1 -- start start 2024-03-11T19:51:42.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.014+0000 7f6af21f6740 1 Processor -- start 2024-03-11T19:51:42.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.014+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786df7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.014+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 0x557786df8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.014+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 0x557786dff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.014+0000 7f6af21f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557786a19430 con 0x557786ddd4c0 2024-03-11T19:51:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.014+0000 7f6af21f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577869d5e00 con 0x557786dddeb0 2024-03-11T19:51:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.014+0000 7f6af21f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557786ccafd0 con 0x557786dfd460 2024-03-11T19:51:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.014+0000 7f6af21f6740 10 client.0 _ll_register_callbacks cb 0x557786d20fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.015+0000 7f6aecbbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 0x557786df8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.015+0000 7f6aecbbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 0x557786df8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:42414/0 (socket says 172.21.15.83:42414) 2024-03-11T19:51:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.015+0000 7f6aecbbe640 1 -- 172.21.15.83:0/130571552 learned_addr learned my addr 172.21.15.83:0/130571552 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.015+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 0x557786dff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.015+0000 7f6aed3bf640 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786df7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.016+0000 7f6aedbc0640 1 -- 172.21.15.83:0/130571552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 msgr2=0x557786df8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.016+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 0x557786df8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.016+0000 7f6aedbc0640 1 -- 172.21.15.83:0/130571552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 msgr2=0x557786df7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.016+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786df7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.016+0000 7f6aedbc0640 1 -- 172.21.15.83:0/130571552 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae81ae680 con 0x557786dfd460 2024-03-11T19:51:42.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.016+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 0x557786dff850 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4011870 tx=0x7f6ae4012b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3348e9af22df2f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:42.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.016+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ad40a1bb0 0x7f6ad40a4070 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x557786dde710 tx=0x7f6ae4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:42.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.018+0000 7f6ad0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:51:42.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.018+0000 7f6ad0ff9640 1 -- 172.21.15.83:0/130571552 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4004250 con 0x557786dfd460 2024-03-11T19:51:42.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.018+0000 7f6ad0ff9640 1 -- 172.21.15.83:0/130571552 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6ae4018030 con 0x557786dfd460 2024-03-11T19:51:42.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.018+0000 7f6af21f6740 1 -- 172.21.15.83:0/130571552 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557786a82720 con 0x557786dfd460 2024-03-11T19:51:42.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.019+0000 7f6ad0ff9640 1 -- 172.21.15.83:0/130571552 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae40043b0 con 0x557786dfd460 2024-03-11T19:51:42.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.019+0000 7f6af21f6740 10 client.27289 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:42.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.019+0000 7f6af21f6740 1 -- 172.21.15.83:0/130571552 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557786d44610 con 0x557786dfd460 2024-03-11T19:51:42.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.019+0000 7f6ad0ff9640 1 -- 172.21.15.83:0/130571552 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ae402d030 con 0x557786dfd460 2024-03-11T19:51:42.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.019+0000 7f6ad0ff9640 1 -- 172.21.15.83:0/130571552 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6ae4003ba0 con 0x557786dfd460 2024-03-11T19:51:42.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.020+0000 7f6af21f6740 10 client.27289 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:42.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.020+0000 7f6af21f6740 10 client.27289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:42.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.020+0000 7f6af21f6740 20 client.27289 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:42.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.020+0000 7f6af21f6740 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557786e04700 0x557786e06b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.020+0000 7f6af21f6740 4 client.27289 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:51:42.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.020+0000 7f6af21f6740 1 -- 172.21.15.83:0/130571552 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557786ad1890 con 0x557786e04700 2024-03-11T19:51:42.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.024+0000 7f6aed3bf640 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557786e04700 0x557786e06b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.025+0000 7f6aed3bf640 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557786e04700 0x557786e06b20 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=906f3361a7721e61 server_cookie=9773e73c16d733f in_seq=0 out_seq=0 2024-03-11T19:51:42.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.025+0000 7f6ad0ff9640 10 client.27289 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:42.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.026+0000 7f6ad0ff9640 1 -- 172.21.15.83:0/130571552 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557786ad1890 con 0x557786e04700 2024-03-11T19:51:42.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.026+0000 7f6ad0ff9640 10 client.27289 handle_command_reply: tid=0 2024-03-11T19:51:42.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.026+0000 7f6af21f6740 1 client.27289 shutdown 2024-03-11T19:51:42.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.027+0000 7f6af21f6740 1 -- 172.21.15.83:0/130571552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 msgr2=0x557786dff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:42.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.027+0000 7f6af21f6740 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 0x557786dff850 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4011870 tx=0x7f6ae4012b10 comp rx=0 tx=0).stop 2024-03-11T19:51:42.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.027+0000 7f6af21f6740 1 -- 172.21.15.83:0/130571552 shutdown_connections 2024-03-11T19:51:42.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.027+0000 7f6af21f6740 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557786e04700 0x557786e06b20 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.027+0000 7f6af21f6740 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 0x557786dff850 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.027+0000 7f6af21f6740 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 0x557786df8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.027+0000 7f6af21f6740 1 --2- 172.21.15.83:0/130571552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786df7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.027+0000 7f6af21f6740 1 -- 172.21.15.83:0/130571552 >> 172.21.15.83:0/130571552 conn(0x557786dd8b30 msgr2=0x557786dd7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:42.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.028+0000 7f6ad0ff9640 0 client.27289 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:42.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.028+0000 7f6af21f6740 1 -- 172.21.15.83:0/130571552 shutdown_connections 2024-03-11T19:51:42.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.028+0000 7f6af21f6740 1 -- 172.21.15.83:0/130571552 wait complete. 2024-03-11T19:51:42.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.028+0000 7f6af21f6740 20 client.27289 trim_cache size 0 max 16384 2024-03-11T19:51:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:42 smithi083 ceph-mon[31864]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.9 MiB/s wr, 603 op/s 2024-03-11T19:51:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:42 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:42 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:42 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:42 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:42 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:51:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:42 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:51:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:42 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.060+0000 7f6af21f6740 1 Processor -- start 2024-03-11T19:51:42.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.060+0000 7f6af21f6740 1 -- start start 2024-03-11T19:51:42.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.060+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.060+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dddeb0 0x557786e14f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.060+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dfd460 0x557786df7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.060+0000 7f6af21f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557786df7740 con 0x557786ddd4c0 2024-03-11T19:51:42.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.060+0000 7f6af21f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557786dd9580 con 0x557786dfd460 2024-03-11T19:51:42.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.060+0000 7f6af21f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557786df7d30 con 0x557786dddeb0 2024-03-11T19:51:42.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aed3bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aed3bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36940/0 (socket says 172.21.15.83:36940) 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aed3bf640 1 -- 172.21.15.83:0/4000461855 learned_addr learned my addr 172.21.15.83:0/4000461855 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/4000461855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dfd460 0x557786df7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aecbbe640 1 --2- 172.21.15.83:0/4000461855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dddeb0 0x557786e14f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aed3bf640 1 -- 172.21.15.83:0/4000461855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dddeb0 msgr2=0x557786e14f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aed3bf640 1 --2- 172.21.15.83:0/4000461855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dddeb0 0x557786e14f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aed3bf640 1 -- 172.21.15.83:0/4000461855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dfd460 msgr2=0x557786df7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aed3bf640 1 --2- 172.21.15.83:0/4000461855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dfd460 0x557786df7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aed3bf640 1 -- 172.21.15.83:0/4000461855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557786ce0ac0 con 0x557786ddd4c0 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/4000461855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dfd460 0x557786df7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6aed3bf640 1 --2- 172.21.15.83:0/4000461855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14a00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0004b10 tx=0x7f6ae000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a23838c1aadd67 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6ad27fc640 1 -- 172.21.15.83:0/4000461855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0011040 con 0x557786ddd4c0 2024-03-11T19:51:42.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6ad27fc640 1 -- 172.21.15.83:0/4000461855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ae001a070 con 0x557786ddd4c0 2024-03-11T19:51:42.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.061+0000 7f6ad27fc640 1 -- 172.21.15.83:0/4000461855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae001de50 con 0x557786ddd4c0 2024-03-11T19:51:42.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.062+0000 7f6af21f6740 1 -- 172.21.15.83:0/4000461855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 msgr2=0x557786e14a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:42.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.062+0000 7f6af21f6740 1 --2- 172.21.15.83:0/4000461855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14a00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0004b10 tx=0x7f6ae000a7d0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.063+0000 7f6af21f6740 1 -- 172.21.15.83:0/4000461855 shutdown_connections 2024-03-11T19:51:42.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.063+0000 7f6af21f6740 1 --2- 172.21.15.83:0/4000461855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dfd460 0x557786df7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.063+0000 7f6af21f6740 1 --2- 172.21.15.83:0/4000461855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dddeb0 0x557786e14f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.063+0000 7f6af21f6740 1 --2- 172.21.15.83:0/4000461855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14a00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.063+0000 7f6af21f6740 1 -- 172.21.15.83:0/4000461855 >> 172.21.15.83:0/4000461855 conn(0x557786dd8b30 msgr2=0x557786e0b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:42.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.064+0000 7f6af21f6740 1 -- 172.21.15.83:0/4000461855 shutdown_connections 2024-03-11T19:51:42.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.064+0000 7f6af21f6740 1 -- 172.21.15.83:0/4000461855 wait complete. 2024-03-11T19:51:42.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.064+0000 7f6af21f6740 1 -- start start 2024-03-11T19:51:42.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.065+0000 7f6af21f6740 1 Processor -- start 2024-03-11T19:51:42.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.065+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 0x557786e576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6af21f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 0x557786e59c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6af21f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557786e5a340 con 0x557786ddd4c0 2024-03-11T19:51:42.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6af21f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557786e5a4b0 con 0x557786dddeb0 2024-03-11T19:51:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6af21f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557786e5a670 con 0x557786dfd460 2024-03-11T19:51:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6af21f6740 10 client.0 _ll_register_callbacks cb 0x7f6ad80f46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:51:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6aed3bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6aed3bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36954/0 (socket says 172.21.15.83:36954) 2024-03-11T19:51:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6aed3bf640 1 -- 172.21.15.83:0/3359860047 learned_addr learned my addr 172.21.15.83:0/3359860047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:51:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 0x557786e59c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6aecbbe640 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 0x557786e576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6aedbc0640 1 -- 172.21.15.83:0/3359860047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 msgr2=0x557786e576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 0x557786e576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6aedbc0640 1 -- 172.21.15.83:0/3359860047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 msgr2=0x557786e14e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:51:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6aedbc0640 1 -- 172.21.15.83:0/3359860047 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae0009eb0 con 0x557786dfd460 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.066+0000 7f6aecbbe640 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 0x557786e576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6aed3bf640 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6aedbc0640 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 0x557786e59c40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40092e0 tx=0x7f6ae401ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48827ba73aff843f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6ac7fff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6ac7fff640 1 -- 172.21.15.83:0/3359860047 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40058c0 con 0x557786dfd460 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6af21f6740 1 -- 172.21.15.83:0/3359860047 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557786a457e0 con 0x557786dfd460 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6ac7fff640 1 -- 172.21.15.83:0/3359860047 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6ae4018070 con 0x557786dfd460 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6ac7fff640 1 -- 172.21.15.83:0/3359860047 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae4005c00 con 0x557786dfd460 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6af21f6740 10 client.27295 fetch_fsmap learned FSMap version 32 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6af21f6740 1 -- 172.21.15.83:0/3359860047 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557786d066a0 con 0x557786dfd460 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6ac7fff640 1 -- 172.21.15.83:0/3359860047 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557786a457e0 con 0x557786dfd460 2024-03-11T19:51:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6ac7fff640 1 -- 172.21.15.83:0/3359860047 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6ae4003960 con 0x557786dfd460 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.067+0000 7f6af21f6740 10 client.27295 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6af21f6740 10 client.27295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6af21f6740 20 client.27295 populate_metadata read hostname 'smithi083' 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6af21f6740 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557786e5c0b0 0x557786e5e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6af21f6740 4 client.27295 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6af21f6740 1 -- 172.21.15.83:0/3359860047 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557786a344e0 con 0x557786e5c0b0 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6aed3bf640 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557786e5c0b0 0x557786e5e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6aed3bf640 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557786e5c0b0 0x557786e5e4d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2abd15b49cc452d0 server_cookie=901ce469f13ea402 in_seq=0 out_seq=0 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6ac7fff640 10 client.27295 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6ac7fff640 1 -- 172.21.15.83:0/3359860047 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557786a344e0 con 0x557786e5c0b0 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6ac7fff640 10 client.27295 handle_command_reply: tid=0 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6af21f6740 1 client.27295 shutdown 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6af21f6740 1 -- 172.21.15.83:0/3359860047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 msgr2=0x557786e59c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:42.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.068+0000 7f6af21f6740 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 0x557786e59c40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40092e0 tx=0x7f6ae401ece0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6af21f6740 1 -- 172.21.15.83:0/3359860047 shutdown_connections 2024-03-11T19:51:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6af21f6740 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557786e5c0b0 0x557786e5e4d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6af21f6740 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557786dfd460 0x557786e59c40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6ac7fff640 0 client.27295 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:51:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6af21f6740 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557786dddeb0 0x557786e576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6af21f6740 1 --2- 172.21.15.83:0/3359860047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557786ddd4c0 0x557786e14e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6af21f6740 1 -- 172.21.15.83:0/3359860047 >> 172.21.15.83:0/3359860047 conn(0x557786dd8b30 msgr2=0x557786ddeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6af21f6740 1 -- 172.21.15.83:0/3359860047 shutdown_connections 2024-03-11T19:51:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6af21f6740 1 -- 172.21.15.83:0/3359860047 wait complete. 2024-03-11T19:51:42.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6af21f6740 20 client.27295 trim_cache size 0 max 16384 2024-03-11T19:51:42.077 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:51:42.078 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:51:42.078 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:51:42.078 INFO:teuthology.orchestra.run.smithi083.stdout: "ee4d5a51-bb41-41e8-ace2-f068d0a9c50e": { 2024-03-11T19:51:42.078 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:51:42.078 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "ee4d5a51-bb41-41e8-ace2-f068d0a9c50e", 2024-03-11T19:51:42.078 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:51:42.078 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:51:42.078 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:51:42.078 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:51:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6ad3fff640 1 -- 172.21.15.83:0/3248743415 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ad40a1bb0 msgr2=0x7f6ad40a4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6ad3fff640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ad40a1bb0 0x7f6ad40a4070 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x557786dde710 tx=0x7f6ae4005f90 comp rx=0 tx=0).stop 2024-03-11T19:51:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6ad3fff640 1 -- 172.21.15.83:0/3248743415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ae81089b0 msgr2=0x7f6ae807ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:51:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.069+0000 7f6ad3fff640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ae81089b0 0x7f6ae807ed30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8000c00 tx=0x7f6ad8004270 comp rx=0 tx=0).stop 2024-03-11T19:51:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.070+0000 7f6ad3fff640 1 -- 172.21.15.83:0/3248743415 shutdown_connections 2024-03-11T19:51:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.070+0000 7f6ad3fff640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6ad40a1bb0 0x7f6ad40a4070 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.070+0000 7f6ad3fff640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae8109370 0x7f6ae81adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.070+0000 7f6ad3fff640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ae81089b0 0x7f6ae807ed30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.070+0000 7f6ad3fff640 1 --2- 172.21.15.83:0/3248743415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ae81077c0 0x7f6ae807e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:51:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.070+0000 7f6ad3fff640 1 -- 172.21.15.83:0/3248743415 >> 172.21.15.83:0/3248743415 conn(0x7f6ae8076770 msgr2=0x7f6ae810bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:51:42.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.071+0000 7f6ad3fff640 1 -- 172.21.15.83:0/3248743415 shutdown_connections 2024-03-11T19:51:42.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:51:42.071+0000 7f6ad3fff640 1 -- 172.21.15.83:0/3248743415 wait complete. 2024-03-11T19:51:42.090 INFO:tasks.cephfs.filesystem:scrub status for tag:ee4d5a51-bb41-41e8-ace2-f068d0a9c50e - {'path': '/', 'tag': 'ee4d5a51-bb41-41e8-ace2-f068d0a9c50e', 'options': 'recursive,force'} 2024-03-11T19:51:42.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:42 smithi137 ceph-mon[36220]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.9 MiB/s wr, 603 op/s 2024-03-11T19:51:42.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:42 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:42 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:42 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:42 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:42 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:51:42.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:42 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:51:42.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:42 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:42 smithi204 ceph-mon[36739]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.9 MiB/s wr, 603 op/s 2024-03-11T19:51:42.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:42 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:42 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:42 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:42 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:42 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:51:42.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:42 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:51:42.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:42 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:51:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 745.0 s, 1785.0 tps, lat 56.372 ms stddev 64.038 2024-03-11T19:51:43.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:43 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:51:43.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:43 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:43.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:43 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:51:43.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:43 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:51:43.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:43 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:43.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:43 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:51:43.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:43 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:43.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:43 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:43.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:43 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:51:43.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:43 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:43.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:43 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:51:43.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:43 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:51:43.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:43 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:43.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:43 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:51:43.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:43 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:43.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:43 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:43 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:51:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:43 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:43 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:51:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:43 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:51:43.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:43 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:43.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:43 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:51:43.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:43 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:43.407 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:43 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:51:44.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:44 smithi083 ceph-mon[31864]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-03-11T19:51:44.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:44 smithi137 ceph-mon[36220]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-03-11T19:51:44.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:44 smithi204 ceph-mon[36739]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 5.9 MiB/s wr, 593 op/s 2024-03-11T19:51:46.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:46 smithi083 ceph-mon[31864]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.6 MiB/s wr, 687 op/s 2024-03-11T19:51:46.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:46 smithi137 ceph-mon[36220]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.6 MiB/s wr, 687 op/s 2024-03-11T19:51:46.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:46 smithi204 ceph-mon[36739]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.6 MiB/s wr, 687 op/s 2024-03-11T19:51:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 750.0 s, 2523.2 tps, lat 39.257 ms stddev 45.463 2024-03-11T19:51:48.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:48 smithi083 ceph-mon[31864]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.2 MiB/s wr, 619 op/s 2024-03-11T19:51:48.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:48 smithi137 ceph-mon[36220]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.2 MiB/s wr, 619 op/s 2024-03-11T19:51:48.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:48 smithi204 ceph-mon[36739]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.2 MiB/s wr, 619 op/s 2024-03-11T19:51:50.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:50 smithi137 ceph-mon[36220]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.0 MiB/s wr, 732 op/s 2024-03-11T19:51:50.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:50 smithi204 ceph-mon[36739]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.0 MiB/s wr, 732 op/s 2024-03-11T19:51:50.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:50 smithi083 ceph-mon[31864]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.0 MiB/s wr, 732 op/s 2024-03-11T19:51:51.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:51 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:51:51.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:51 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:51:51.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:51 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:51:52.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:52 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:51:52.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:52 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:51:52.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:52 smithi137 ceph-mon[36220]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-03-11T19:51:52.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:52 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:51:52.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:52 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:51:52.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:52 smithi204 ceph-mon[36739]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-03-11T19:51:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 755.0 s, 2208.4 tps, lat 45.565 ms stddev 49.777 2024-03-11T19:51:52.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:52 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:51:52.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:52 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:51:52.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:52 smithi083 ceph-mon[31864]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-03-11T19:51:54.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:54 smithi137 ceph-mon[36220]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.5 MiB/s wr, 584 op/s 2024-03-11T19:51:54.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:54 smithi204 ceph-mon[36739]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.5 MiB/s wr, 584 op/s 2024-03-11T19:51:54.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:54 smithi083 ceph-mon[31864]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.5 MiB/s wr, 584 op/s 2024-03-11T19:51:56.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:56 smithi137 ceph-mon[36220]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 715 op/s 2024-03-11T19:51:56.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:56 smithi204 ceph-mon[36739]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 715 op/s 2024-03-11T19:51:56.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:56 smithi083 ceph-mon[31864]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 MiB/s wr, 715 op/s 2024-03-11T19:51:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 760.0 s, 1557.2 tps, lat 64.095 ms stddev 70.801 2024-03-11T19:51:58.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:51:58 smithi137 ceph-mon[36220]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.3 MiB/s wr, 625 op/s 2024-03-11T19:51:58.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:51:58 smithi204 ceph-mon[36739]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.3 MiB/s wr, 625 op/s 2024-03-11T19:51:58.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:51:58 smithi083 ceph-mon[31864]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.3 MiB/s wr, 625 op/s 2024-03-11T19:52:00.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:00 smithi137 ceph-mon[36220]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-03-11T19:52:00.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:00 smithi204 ceph-mon[36739]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-03-11T19:52:00.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:00 smithi083 ceph-mon[31864]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-03-11T19:52:02.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:02 smithi137 ceph-mon[36220]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.0 MiB/s wr, 609 op/s 2024-03-11T19:52:02.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:02 smithi204 ceph-mon[36739]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.0 MiB/s wr, 609 op/s 2024-03-11T19:52:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 765.0 s, 1627.0 tps, lat 61.364 ms stddev 66.442 2024-03-11T19:52:02.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:02 smithi083 ceph-mon[31864]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.0 MiB/s wr, 609 op/s 2024-03-11T19:52:04.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:04 smithi137 ceph-mon[36220]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 MiB/s wr, 601 op/s 2024-03-11T19:52:04.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:04 smithi204 ceph-mon[36739]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 MiB/s wr, 601 op/s 2024-03-11T19:52:04.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:04 smithi083 ceph-mon[31864]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 MiB/s wr, 601 op/s 2024-03-11T19:52:06.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:06 smithi137 ceph-mon[36220]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 MiB/s wr, 689 op/s 2024-03-11T19:52:06.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:06 smithi204 ceph-mon[36739]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 MiB/s wr, 689 op/s 2024-03-11T19:52:06.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:06 smithi083 ceph-mon[31864]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.7 MiB/s wr, 689 op/s 2024-03-11T19:52:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 770.0 s, 1694.2 tps, lat 58.627 ms stddev 65.639 2024-03-11T19:52:08.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:08 smithi137 ceph-mon[36220]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-03-11T19:52:08.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:08 smithi204 ceph-mon[36739]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-03-11T19:52:08.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:08 smithi083 ceph-mon[31864]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-03-11T19:52:10.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:10 smithi204 ceph-mon[36739]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.3 MiB/s wr, 743 op/s 2024-03-11T19:52:10.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:10 smithi083 ceph-mon[31864]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.3 MiB/s wr, 743 op/s 2024-03-11T19:52:10.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:10 smithi137 ceph-mon[36220]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.3 MiB/s wr, 743 op/s 2024-03-11T19:52:12.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:52:12.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:52:12.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.251+0000 7f6c45688640 1 -- 172.21.15.83:0/1914735673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c40108890 msgr2=0x7f6c40108cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.251+0000 7f6c45688640 1 --2- 172.21.15.83:0/1914735673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c40108890 0x7f6c40108cf0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009be0 tx=0x7f6c3002bfc0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.253+0000 7f6c3f7fe640 1 --2- 172.21.15.83:0/1914735673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c40109230 0x7f6c4010daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.258+0000 7f6c45688640 1 -- 172.21.15.83:0/1914735673 shutdown_connections 2024-03-11T19:52:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.258+0000 7f6c45688640 1 --2- 172.21.15.83:0/1914735673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c40109230 0x7f6c4010daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.258+0000 7f6c45688640 1 --2- 172.21.15.83:0/1914735673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c40108890 0x7f6c40108cf0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.258+0000 7f6c45688640 1 --2- 172.21.15.83:0/1914735673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c40072560 0x7f6c40072980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.258+0000 7f6c45688640 1 -- 172.21.15.83:0/1914735673 >> 172.21.15.83:0/1914735673 conn(0x7f6c4006e060 msgr2=0x7f6c400704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.263+0000 7f6c45688640 1 -- 172.21.15.83:0/1914735673 shutdown_connections 2024-03-11T19:52:12.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.267+0000 7f6c45688640 1 -- 172.21.15.83:0/1914735673 wait complete. 2024-03-11T19:52:12.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.267+0000 7f6c45688640 1 Processor -- start 2024-03-11T19:52:12.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.271+0000 7f6c45688640 1 -- start start 2024-03-11T19:52:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.271+0000 7f6c45688640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c40072560 0x7f6c40117b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.271+0000 7f6c45688640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c40108890 0x7f6c40114570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.271+0000 7f6c45688640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c40109230 0x7f6c40114b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.271+0000 7f6c45688640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c401b61f0 con 0x7f6c40072560 2024-03-11T19:52:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.271+0000 7f6c45688640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c401b6330 con 0x7f6c40109230 2024-03-11T19:52:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.271+0000 7f6c45688640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c401b64a0 con 0x7f6c40108890 2024-03-11T19:52:12.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.272+0000 7f6c3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c40109230 0x7f6c40114b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.272+0000 7f6c3e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c40108890 0x7f6c40114570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.272+0000 7f6c3e7fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c40108890 0x7f6c40114570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:43272/0 (socket says 172.21.15.83:43272) 2024-03-11T19:52:12.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.272+0000 7f6c3e7fc640 1 -- 172.21.15.83:0/1774361660 learned_addr learned my addr 172.21.15.83:0/1774361660 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:12.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.272+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c40072560 0x7f6c40117b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.276+0000 7f6c3f7fe640 1 -- 172.21.15.83:0/1774361660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c40108890 msgr2=0x7f6c40114570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.276+0000 7f6c3f7fe640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c40108890 0x7f6c40114570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.276+0000 7f6c3f7fe640 1 -- 172.21.15.83:0/1774361660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c40072560 msgr2=0x7f6c40117b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.276+0000 7f6c3f7fe640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c40072560 0x7f6c40117b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.276+0000 7f6c3f7fe640 1 -- 172.21.15.83:0/1774361660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c28009560 con 0x7f6c40109230 2024-03-11T19:52:12.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.278+0000 7f6c3f7fe640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c40109230 0x7f6c40114b50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800e940 tx=0x7f6c2800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:12.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.280+0000 7f6c37fff640 1 -- 172.21.15.83:0/1774361660 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2800ccc0 con 0x7f6c40109230 2024-03-11T19:52:12.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.280+0000 7f6c45688640 1 -- 172.21.15.83:0/1774361660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c30009840 con 0x7f6c40109230 2024-03-11T19:52:12.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.280+0000 7f6c45688640 1 -- 172.21.15.83:0/1774361660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c401b6b00 con 0x7f6c40109230 2024-03-11T19:52:12.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.280+0000 7f6c37fff640 1 -- 172.21.15.83:0/1774361660 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c2800ce20 con 0x7f6c40109230 2024-03-11T19:52:12.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.280+0000 7f6c37fff640 1 -- 172.21.15.83:0/1774361660 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28010430 con 0x7f6c40109230 2024-03-11T19:52:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.284+0000 7f6c48264740 1 Processor -- start 2024-03-11T19:52:12.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.284+0000 7f6c37fff640 1 -- 172.21.15.83:0/1774361660 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6c28004030 con 0x7f6c40109230 2024-03-11T19:52:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.284+0000 7f6c48264740 1 -- start start 2024-03-11T19:52:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.285+0000 7f6c48264740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.285+0000 7f6c48264740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c0ea0 0x55c4466c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.285+0000 7f6c3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.285+0000 7f6c3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466c08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37748/0 (socket says 172.21.15.83:37748) 2024-03-11T19:52:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.286+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c0ea0 0x55c4466c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.286+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c0ea0 0x55c4466c59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39054/0 (socket says 172.21.15.83:39054) 2024-03-11T19:52:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.286+0000 7f6c37fff640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6c2c0a1bc0 0x7f6c2c0a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.286+0000 7f6c37fff640 1 -- 172.21.15.83:0/1774361660 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6c28014070 con 0x7f6c40109230 2024-03-11T19:52:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.286+0000 7f6c48264740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c6010 0x55c4466c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.286+0000 7f6c3e7fc640 1 -- 172.21.15.83:0/216019189 learned_addr learned my addr 172.21.15.83:0/216019189 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.286+0000 7f6c48264740 1 -- 172.21.15.83:0/216019189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c446630280 con 0x55c4466c04b0 2024-03-11T19:52:12.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.286+0000 7f6c48264740 1 -- 172.21.15.83:0/216019189 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c44662d7b0 con 0x55c4466c0ea0 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.287+0000 7f6c48264740 1 -- 172.21.15.83:0/216019189 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c446628f90 con 0x55c4466c6010 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.287+0000 7f6c3f7fe640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6c2c0a1bc0 0x7f6c2c0a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.287+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/216019189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c6010 0x55c4466c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.287+0000 7f6c3e7fc640 1 -- 172.21.15.83:0/216019189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c6010 msgr2=0x55c4466c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.287+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/216019189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c6010 0x55c4466c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.287+0000 7f6c3e7fc640 1 -- 172.21.15.83:0/216019189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c0ea0 msgr2=0x55c4466c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.287+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/216019189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c0ea0 0x55c4466c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.287+0000 7f6c3e7fc640 1 -- 172.21.15.83:0/216019189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4465e7570 con 0x55c4466c04b0 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.287+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/216019189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466c08d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009b30 tx=0x7f6c30005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40039e0e5aee224e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.287+0000 7f6c357fa640 1 -- 172.21.15.83:0/216019189 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30033720 con 0x55c4466c04b0 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.288+0000 7f6c3f7fe640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6c2c0a1bc0 0x7f6c2c0a4080 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6c28096ff0 tx=0x7f6c2800ef30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.288+0000 7f6c357fa640 1 -- 172.21.15.83:0/216019189 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c30003970 con 0x55c4466c04b0 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.288+0000 7f6c357fa640 1 -- 172.21.15.83:0/216019189 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30004530 con 0x55c4466c04b0 2024-03-11T19:52:12.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.288+0000 7f6c48264740 1 -- 172.21.15.83:0/216019189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 msgr2=0x55c4466c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.288+0000 7f6c48264740 1 --2- 172.21.15.83:0/216019189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466c08d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009b30 tx=0x7f6c30005e90 comp rx=0 tx=0).stop 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 -- 172.21.15.83:0/216019189 shutdown_connections 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 --2- 172.21.15.83:0/216019189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c6010 0x55c4466c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 --2- 172.21.15.83:0/216019189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c0ea0 0x55c4466c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 --2- 172.21.15.83:0/216019189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466c08d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 -- 172.21.15.83:0/216019189 >> 172.21.15.83:0/216019189 conn(0x55c4466bba20 msgr2=0x55c4466bbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 -- 172.21.15.83:0/216019189 shutdown_connections 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 -- 172.21.15.83:0/216019189 wait complete. 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 -- start start 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 Processor -- start 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c0ea0 0x55c4466caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c6010 0x55c4466d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4462fc430 con 0x55c4466c04b0 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4462b8e00 con 0x55c4466c0ea0 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.289+0000 7f6c48264740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4465adfd0 con 0x55c4466c6010 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c48264740 10 client.0 _ll_register_callbacks cb 0x55c446603fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466ca4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37762/0 (socket says 172.21.15.83:37762) 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c3e7fc640 1 -- 172.21.15.83:0/2896099802 learned_addr learned my addr 172.21.15.83:0/2896099802 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c3effd640 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c0ea0 0x55c4466caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c6010 0x55c4466d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c3e7fc640 1 -- 172.21.15.83:0/2896099802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c6010 msgr2=0x55c4466d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c6010 0x55c4466d1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c3e7fc640 1 -- 172.21.15.83:0/2896099802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c0ea0 msgr2=0x55c4466caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c0ea0 0x55c4466caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c3e7fc640 1 -- 172.21.15.83:0/2896099802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4010e3b0 con 0x55c4466c04b0 2024-03-11T19:52:12.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466ca4e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f6c30049040 tx=0x7f6c30000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2448e10ac83a150d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.290+0000 7f6c267fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:52:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.291+0000 7f6c267fc640 1 -- 172.21.15.83:0/2896099802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30033820 con 0x55c4466c04b0 2024-03-11T19:52:12.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.291+0000 7f6c267fc640 1 -- 172.21.15.83:0/2896099802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6c3002de40 con 0x55c4466c04b0 2024-03-11T19:52:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.291+0000 7f6c48264740 1 -- 172.21.15.83:0/2896099802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4463287e0 con 0x55c4466c04b0 2024-03-11T19:52:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.292+0000 7f6c267fc640 1 -- 172.21.15.83:0/2896099802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c30003df0 con 0x55c4466c04b0 2024-03-11T19:52:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.292+0000 7f6c48264740 10 client.17742 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.292+0000 7f6c48264740 1 -- 172.21.15.83:0/2896099802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c446627610 con 0x55c4466c04b0 2024-03-11T19:52:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.293+0000 7f6c267fc640 1 -- 172.21.15.83:0/2896099802 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c3005f070 con 0x55c4466c04b0 2024-03-11T19:52:12.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.293+0000 7f6c267fc640 1 -- 172.21.15.83:0/2896099802 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6c3003c070 con 0x55c4466c04b0 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.293+0000 7f6c48264740 10 client.17742 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.293+0000 7f6c48264740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.293+0000 7f6c48264740 20 client.17742 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.293+0000 7f6c48264740 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4466d63f0 0x55c4466d8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.294+0000 7f6c48264740 4 client.17742 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.294+0000 7f6c48264740 1 -- 172.21.15.83:0/2896099802 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c446365720 con 0x55c4466d63f0 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.294+0000 7f6c3effd640 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4466d63f0 0x55c4466d8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.295+0000 7f6c267fc640 10 client.17742 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.295+0000 7f6c3effd640 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4466d63f0 0x55c4466d8810 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58146a2d683d5a99 server_cookie=59222a0b3f656bda in_seq=0 out_seq=0 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.296+0000 7f6c267fc640 1 -- 172.21.15.83:0/2896099802 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c446365720 con 0x55c4466d63f0 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.296+0000 7f6c267fc640 10 client.17742 handle_command_reply: tid=0 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.296+0000 7f6c48264740 1 client.17742 shutdown 2024-03-11T19:52:12.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.296+0000 7f6c48264740 1 -- 172.21.15.83:0/2896099802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 msgr2=0x55c4466ca4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.296+0000 7f6c48264740 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466ca4e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f6c30049040 tx=0x7f6c30000f40 comp rx=0 tx=0).stop 2024-03-11T19:52:12.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.296+0000 7f6c48264740 1 -- 172.21.15.83:0/2896099802 shutdown_connections 2024-03-11T19:52:12.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.296+0000 7f6c48264740 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4466d63f0 0x55c4466d8810 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.296+0000 7f6c267fc640 0 client.17742 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:12.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.297+0000 7f6c48264740 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c6010 0x55c4466d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.297+0000 7f6c48264740 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c0ea0 0x55c4466caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.297+0000 7f6c48264740 1 --2- 172.21.15.83:0/2896099802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466ca4e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.297+0000 7f6c48264740 1 -- 172.21.15.83:0/2896099802 >> 172.21.15.83:0/2896099802 conn(0x55c4466bba20 msgr2=0x55c4466bac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:12.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.297+0000 7f6c48264740 1 -- 172.21.15.83:0/2896099802 shutdown_connections 2024-03-11T19:52:12.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.297+0000 7f6c48264740 1 -- 172.21.15.83:0/2896099802 wait complete. 2024-03-11T19:52:12.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.297+0000 7f6c48264740 20 client.17742 trim_cache size 0 max 16384 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c48264740 1 Processor -- start 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c48264740 1 -- start start 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c48264740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c04b0 0x55c4466ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c48264740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c0ea0 0x55c4466d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c48264740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c6010 0x55c4466ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c48264740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4466c8b60 con 0x55c4466c0ea0 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c48264740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c446365720 con 0x55c4466c6010 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c48264740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4466cb6d0 con 0x55c4466c04b0 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c0ea0 0x55c4466d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c0ea0 0x55c4466d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37770/0 (socket says 172.21.15.83:37770) 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c3effd640 1 -- 172.21.15.83:0/1428295897 learned_addr learned my addr 172.21.15.83:0/1428295897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:12.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/1428295897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c04b0 0x55c4466ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.321+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/1428295897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c6010 0x55c4466ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.322+0000 7f6c3effd640 1 -- 172.21.15.83:0/1428295897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c04b0 msgr2=0x55c4466ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.322+0000 7f6c3effd640 1 --2- 172.21.15.83:0/1428295897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c04b0 0x55c4466ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.322+0000 7f6c3effd640 1 -- 172.21.15.83:0/1428295897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c6010 msgr2=0x55c4466ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.322+0000 7f6c3effd640 1 --2- 172.21.15.83:0/1428295897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c6010 0x55c4466ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.322+0000 7f6c3effd640 1 -- 172.21.15.83:0/1428295897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4465c3ac0 con 0x55c4466c0ea0 2024-03-11T19:52:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.322+0000 7f6c3effd640 1 --2- 172.21.15.83:0/1428295897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c0ea0 0x55c4466d19f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x55c4466bc990 tx=0x7f6c38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c596d05d4e16fb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.322+0000 7f6c34ff9640 1 -- 172.21.15.83:0/1428295897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38017040 con 0x55c4466c0ea0 2024-03-11T19:52:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.322+0000 7f6c34ff9640 1 -- 172.21.15.83:0/1428295897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c38011e50 con 0x55c4466c0ea0 2024-03-11T19:52:12.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.322+0000 7f6c34ff9640 1 -- 172.21.15.83:0/1428295897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3801ba90 con 0x55c4466c0ea0 2024-03-11T19:52:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.323+0000 7f6c48264740 1 -- 172.21.15.83:0/1428295897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c0ea0 msgr2=0x55c4466d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.323+0000 7f6c48264740 1 --2- 172.21.15.83:0/1428295897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c0ea0 0x55c4466d19f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x55c4466bc990 tx=0x7f6c38011810 comp rx=0 tx=0).stop 2024-03-11T19:52:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.323+0000 7f6c48264740 1 -- 172.21.15.83:0/1428295897 shutdown_connections 2024-03-11T19:52:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.323+0000 7f6c48264740 1 --2- 172.21.15.83:0/1428295897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c6010 0x55c4466ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.323+0000 7f6c48264740 1 --2- 172.21.15.83:0/1428295897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c0ea0 0x55c4466d19f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.323+0000 7f6c48264740 1 --2- 172.21.15.83:0/1428295897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c04b0 0x55c4466ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.323+0000 7f6c48264740 1 -- 172.21.15.83:0/1428295897 >> 172.21.15.83:0/1428295897 conn(0x55c4466bba20 msgr2=0x55c4466c2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.324+0000 7f6c48264740 1 -- 172.21.15.83:0/1428295897 shutdown_connections 2024-03-11T19:52:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.324+0000 7f6c48264740 1 -- 172.21.15.83:0/1428295897 wait complete. 2024-03-11T19:52:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.324+0000 7f6c48264740 1 -- start start 2024-03-11T19:52:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.324+0000 7f6c48264740 1 Processor -- start 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.324+0000 7f6c48264740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.324+0000 7f6c48264740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c0ea0 0x55c4466d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.324+0000 7f6c48264740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c6010 0x55c44672c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.324+0000 7f6c48264740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c44672cde0 con 0x55c4466c04b0 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.324+0000 7f6c48264740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c44672cf20 con 0x55c4466c6010 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.324+0000 7f6c48264740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c44672d0e0 con 0x55c4466c0ea0 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c48264740 10 client.0 _ll_register_callbacks cb 0x7f6c30009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c0ea0 0x55c4466d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c0ea0 0x55c4466d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:43296/0 (socket says 172.21.15.83:43296) 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c3effd640 1 -- 172.21.15.83:0/1903692234 learned_addr learned my addr 172.21.15.83:0/1903692234 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c6010 0x55c44672c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c3effd640 1 -- 172.21.15.83:0/1903692234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c6010 msgr2=0x55c44672c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c3effd640 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c6010 0x55c44672c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c3effd640 1 -- 172.21.15.83:0/1903692234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 msgr2=0x55c4466d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c3effd640 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c3effd640 1 -- 172.21.15.83:0/1903692234 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3800ba80 con 0x55c4466c0ea0 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c3effd640 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c0ea0 0x55c4466d1050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6c3801b590 tx=0x7f6c38005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=236c67d0367619c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.325+0000 7f6c267fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c267fc640 1 -- 172.21.15.83:0/1903692234 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38017040 con 0x55c4466c0ea0 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c267fc640 1 -- 172.21.15.83:0/1903692234 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6c3802e440 con 0x55c4466c0ea0 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c48264740 1 -- 172.21.15.83:0/1903692234 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c30003970 con 0x55c4466c0ea0 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c267fc640 1 -- 172.21.15.83:0/1903692234 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c38034030 con 0x55c4466c0ea0 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c48264740 10 client.27313 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c48264740 1 -- 172.21.15.83:0/1903692234 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c4465e96a0 con 0x55c4466c0ea0 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c267fc640 1 -- 172.21.15.83:0/1903692234 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c3802eae0 con 0x55c4466c0ea0 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c267fc640 1 -- 172.21.15.83:0/1903692234 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6c380249e0 con 0x55c4466c0ea0 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c48264740 10 client.27313 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:12.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c48264740 10 client.27313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c48264740 20 client.27313 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c48264740 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4466d63f0 0x55c446730bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c48264740 4 client.27313 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:52:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.326+0000 7f6c48264740 1 -- 172.21.15.83:0/1903692234 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f6c30003970 con 0x55c4466d63f0 2024-03-11T19:52:12.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.327+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4466d63f0 0x55c446730bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.327+0000 7f6c3e7fc640 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4466d63f0 0x55c446730bd0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebfa4c054b8091 server_cookie=e536be2a45263364 in_seq=0 out_seq=0 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.327+0000 7f6c267fc640 10 client.27313 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.327+0000 7f6c267fc640 1 -- 172.21.15.83:0/1903692234 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f6c30003970 con 0x55c4466d63f0 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.327+0000 7f6c267fc640 10 client.27313 handle_command_reply: tid=0 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.327+0000 7f6c48264740 1 client.27313 shutdown 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c48264740 1 -- 172.21.15.83:0/1903692234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c0ea0 msgr2=0x55c4466d1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c48264740 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c0ea0 0x55c4466d1050 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6c3801b590 tx=0x7f6c38005740 comp rx=0 tx=0).stop 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c48264740 1 -- 172.21.15.83:0/1903692234 shutdown_connections 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c48264740 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4466d63f0 0x55c446730bd0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c48264740 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4466c6010 0x55c44672c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c48264740 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4466c0ea0 0x55c4466d1050 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c48264740 1 --2- 172.21.15.83:0/1903692234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4466c04b0 0x55c4466d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c48264740 1 -- 172.21.15.83:0/1903692234 >> 172.21.15.83:0/1903692234 conn(0x55c4466bba20 msgr2=0x55c4466ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c267fc640 0 client.27313 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c48264740 1 -- 172.21.15.83:0/1903692234 shutdown_connections 2024-03-11T19:52:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c48264740 1 -- 172.21.15.83:0/1903692234 wait complete. 2024-03-11T19:52:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c48264740 20 client.27313 trim_cache size 0 max 16384 2024-03-11T19:52:12.337 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:52:12.337 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:52:12.338 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:52:12.338 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:52:12.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c45688640 1 -- 172.21.15.83:0/1774361660 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6c2c0a1bc0 msgr2=0x7f6c2c0a4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c45688640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6c2c0a1bc0 0x7f6c2c0a4080 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6c28096ff0 tx=0x7f6c2800ef30 comp rx=0 tx=0).stop 2024-03-11T19:52:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c45688640 1 -- 172.21.15.83:0/1774361660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c40109230 msgr2=0x7f6c40114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c45688640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c40109230 0x7f6c40114b50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800e940 tx=0x7f6c2800ee00 comp rx=0 tx=0).stop 2024-03-11T19:52:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c45688640 1 -- 172.21.15.83:0/1774361660 shutdown_connections 2024-03-11T19:52:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c45688640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6c2c0a1bc0 0x7f6c2c0a4080 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c45688640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c40109230 0x7f6c40114b50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c45688640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c40108890 0x7f6c40114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c45688640 1 --2- 172.21.15.83:0/1774361660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c40072560 0x7f6c40117b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.328+0000 7f6c45688640 1 -- 172.21.15.83:0/1774361660 >> 172.21.15.83:0/1774361660 conn(0x7f6c4006e060 msgr2=0x7f6c40109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.329+0000 7f6c45688640 1 -- 172.21.15.83:0/1774361660 shutdown_connections 2024-03-11T19:52:12.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.329+0000 7f6c45688640 1 -- 172.21.15.83:0/1774361660 wait complete. 2024-03-11T19:52:12.347 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:52:12.347 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:52:12.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:12 smithi083 ceph-mon[31864]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-03-11T19:52:12.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:12 smithi204 ceph-mon[36739]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-03-11T19:52:12.501 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 775.0 s, 1564.9 tps, lat 63.967 ms stddev 70.783 2024-03-11T19:52:12.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.501+0000 7f0e12cd4640 1 -- 172.21.15.83:0/2301408551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c1023a0 msgr2=0x7f0e0c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.501+0000 7f0e12cd4640 1 --2- 172.21.15.83:0/2301408551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c1023a0 0x7f0e0c104810 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009ca0 tx=0x7f0e0402c140 comp rx=0 tx=0).stop 2024-03-11T19:52:12.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.502+0000 7f0e12cd4640 1 -- 172.21.15.83:0/2301408551 shutdown_connections 2024-03-11T19:52:12.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.502+0000 7f0e12cd4640 1 --2- 172.21.15.83:0/2301408551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e0c104d50 0x7f0e0c1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.502+0000 7f0e12cd4640 1 --2- 172.21.15.83:0/2301408551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c1023a0 0x7f0e0c104810 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.502+0000 7f0e12cd4640 1 --2- 172.21.15.83:0/2301408551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e0c0ffa30 0x7f0e0c101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.502+0000 7f0e12cd4640 1 -- 172.21.15.83:0/2301408551 >> 172.21.15.83:0/2301408551 conn(0x7f0e0c0fb630 msgr2=0x7f0e0c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:12.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.503+0000 7f0e12cd4640 1 -- 172.21.15.83:0/2301408551 shutdown_connections 2024-03-11T19:52:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.503+0000 7f0e12cd4640 1 -- 172.21.15.83:0/2301408551 wait complete. 2024-03-11T19:52:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.504+0000 7f0e12cd4640 1 Processor -- start 2024-03-11T19:52:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.504+0000 7f0e12cd4640 1 -- start start 2024-03-11T19:52:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.504+0000 7f0e12cd4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e0c0ffa30 0x7f0e0c10aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.504+0000 7f0e12cd4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c1023a0 0x7f0e0c10b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.505+0000 7f0e10a70640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e0c0ffa30 0x7f0e0c10aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.505+0000 7f0e10a70640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e0c0ffa30 0x7f0e0c10aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:43316/0 (socket says 172.21.15.83:43316) 2024-03-11T19:52:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.505+0000 7f0e10a70640 1 -- 172.21.15.83:0/840787757 learned_addr learned my addr 172.21.15.83:0/840787757 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.505+0000 7f0e12cd4640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e0c104d50 0x7f0e0c06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.505+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c10bb00 con 0x7f0e0c1023a0 2024-03-11T19:52:12.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.505+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c10bc70 con 0x7f0e0c104d50 2024-03-11T19:52:12.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.505+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c06f150 con 0x7f0e0c0ffa30 2024-03-11T19:52:12.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.505+0000 7f0e11271640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e0c104d50 0x7f0e0c06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.506+0000 7f0e0bfff640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c1023a0 0x7f0e0c10b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.506+0000 7f0e11271640 1 -- 172.21.15.83:0/840787757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e0c0ffa30 msgr2=0x7f0e0c10aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.506+0000 7f0e11271640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e0c0ffa30 0x7f0e0c10aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.506+0000 7f0e11271640 1 -- 172.21.15.83:0/840787757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c1023a0 msgr2=0x7f0e0c10b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.506+0000 7f0e11271640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c1023a0 0x7f0e0c10b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.506+0000 7f0e11271640 1 -- 172.21.15.83:0/840787757 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e04009900 con 0x7f0e0c104d50 2024-03-11T19:52:12.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.506+0000 7f0e11271640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e0c104d50 0x7f0e0c06ebc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00b410 tx=0x7f0dfc00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:12.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.507+0000 7f0e09ffb640 1 -- 172.21.15.83:0/840787757 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc011830 con 0x7f0e0c104d50 2024-03-11T19:52:12.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.507+0000 7f0e09ffb640 1 -- 172.21.15.83:0/840787757 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0dfc011e70 con 0x7f0e0c104d50 2024-03-11T19:52:12.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.507+0000 7f0e09ffb640 1 -- 172.21.15.83:0/840787757 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc00f2e0 con 0x7f0e0c104d50 2024-03-11T19:52:12.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.507+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0c06f430 con 0x7f0e0c104d50 2024-03-11T19:52:12.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.507+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c06f890 con 0x7f0e0c104d50 2024-03-11T19:52:12.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.510+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0c0fd120 con 0x7f0e0c104d50 2024-03-11T19:52:12.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.510+0000 7f0e09ffb640 1 -- 172.21.15.83:0/840787757 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0dfc0105f0 con 0x7f0e0c104d50 2024-03-11T19:52:12.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.511+0000 7f0e09ffb640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0df40a1c30 0x7f0df40a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:12.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.511+0000 7f0e09ffb640 1 -- 172.21.15.83:0/840787757 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0dfc0ccdc0 con 0x7f0e0c104d50 2024-03-11T19:52:12.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.511+0000 7f0e10a70640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0df40a1c30 0x7f0df40a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:12.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.512+0000 7f0e10a70640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0df40a1c30 0x7f0df40a40f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009a30 tx=0x7f0e00009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.518+0000 7f0e09ffb640 1 -- 172.21.15.83:0/840787757 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0dfc095fe0 con 0x7f0e0c104d50 2024-03-11T19:52:12.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:12 smithi137 ceph-mon[36220]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-03-11T19:52:12.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.825+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e0c02cda0 con 0x7f0e0c104d50 2024-03-11T19:52:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.826+0000 7f0e09ffb640 1 -- 172.21.15.83:0/840787757 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f0dfc015090 con 0x7f0e0c104d50 2024-03-11T19:52:12.831 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:52:12.831 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:52:12.832 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:52:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0df40a1c30 msgr2=0x7f0df40a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0df40a1c30 0x7f0df40a40f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009a30 tx=0x7f0e00009190 comp rx=0 tx=0).stop 2024-03-11T19:52:12.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e0c104d50 msgr2=0x7f0e0c06ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e0c104d50 0x7f0e0c06ebc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00b410 tx=0x7f0dfc00b8d0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 shutdown_connections 2024-03-11T19:52:12.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0df40a1c30 0x7f0df40a40f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e0c104d50 0x7f0e0c06ebc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c1023a0 0x7f0e0c10b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 --2- 172.21.15.83:0/840787757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e0c0ffa30 0x7f0e0c10aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:12.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 >> 172.21.15.83:0/840787757 conn(0x7f0e0c0fb630 msgr2=0x7f0e0c101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:12.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 shutdown_connections 2024-03-11T19:52:12.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:12.829+0000 7f0e12cd4640 1 -- 172.21.15.83:0/840787757 wait complete. 2024-03-11T19:52:12.851 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:52:12.851 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:52:13.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.053+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3440323539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d0069f90 msgr2=0x7fe2d010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.053+0000 7fe2d51dd640 1 --2- 172.21.15.83:0/3440323539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d0069f90 0x7fe2d010d650 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009860 tx=0x7fe2c402bbb0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.053+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3440323539 shutdown_connections 2024-03-11T19:52:13.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.053+0000 7fe2d51dd640 1 --2- 172.21.15.83:0/3440323539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d010dcc0 0x7fe2d0110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.053+0000 7fe2d51dd640 1 --2- 172.21.15.83:0/3440323539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d0069f90 0x7fe2d010d650 secure :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009860 tx=0x7fe2c402bbb0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.053+0000 7fe2d51dd640 1 --2- 172.21.15.83:0/3440323539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d01083f0 0x7fe2d010a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.053+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3440323539 >> 172.21.15.83:0/3440323539 conn(0x7fe2d006e380 msgr2=0x7fe2d00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.061+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3440323539 shutdown_connections 2024-03-11T19:52:13.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.065+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3440323539 wait complete. 2024-03-11T19:52:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.066+0000 7fe2d51dd640 1 Processor -- start 2024-03-11T19:52:13.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.066+0000 7fe2d51dd640 1 -- start start 2024-03-11T19:52:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.066+0000 7fe2d51dd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d01083f0 0x7fe2d019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.066+0000 7fe2d51dd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d010dcc0 0x7fe2d019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.066+0000 7fe2d51dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d01a3080 0x7fe2d01a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.067+0000 7fe2d51dd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d01a59b0 con 0x7fe2d01a3080 2024-03-11T19:52:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.067+0000 7fe2d51dd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d01a5b20 con 0x7fe2d010dcc0 2024-03-11T19:52:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.067+0000 7fe2d51dd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d01a5ce0 con 0x7fe2d01083f0 2024-03-11T19:52:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.067+0000 7fe2cf59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d01a3080 0x7fe2d01a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.067+0000 7fe2cf59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d01a3080 0x7fe2d01a5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37830/0 (socket says 172.21.15.83:37830) 2024-03-11T19:52:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.067+0000 7fe2cf59e640 1 -- 172.21.15.83:0/3880872598 learned_addr learned my addr 172.21.15.83:0/3880872598 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.070+0000 7fe2ced9d640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d01083f0 0x7fe2d019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.070+0000 7fe2cf59e640 1 -- 172.21.15.83:0/3880872598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d01083f0 msgr2=0x7fe2d019dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.070+0000 7fe2cf59e640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d01083f0 0x7fe2d019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.070+0000 7fe2cf59e640 1 -- 172.21.15.83:0/3880872598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d010dcc0 msgr2=0x7fe2d019e020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:52:13.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.070+0000 7fe2cf59e640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d010dcc0 0x7fe2d019e020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.070+0000 7fe2cf59e640 1 -- 172.21.15.83:0/3880872598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c4009490 con 0x7fe2d01a3080 2024-03-11T19:52:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.070+0000 7fe2cf59e640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d01a3080 0x7fe2d01a5470 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800eaa0 tx=0x7fe2c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.071+0000 7fe2b7fff640 1 -- 172.21.15.83:0/3880872598 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800cdb0 con 0x7fe2d01a3080 2024-03-11T19:52:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.071+0000 7fe2b7fff640 1 -- 172.21.15.83:0/3880872598 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c8005020 con 0x7fe2d01a3080 2024-03-11T19:52:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.071+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3880872598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d01a5fc0 con 0x7fe2d01a3080 2024-03-11T19:52:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.071+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3880872598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d01a63e0 con 0x7fe2d01a3080 2024-03-11T19:52:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.072+0000 7fe2b7fff640 1 -- 172.21.15.83:0/3880872598 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80116d0 con 0x7fe2d01a3080 2024-03-11T19:52:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.078+0000 7fe2b7fff640 1 -- 172.21.15.83:0/3880872598 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe2c8004a00 con 0x7fe2d01a3080 2024-03-11T19:52:13.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.080+0000 7fe2b7fff640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2b80a5ea0 0x7fe2b80a8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.080+0000 7fe2b7fff640 1 -- 172.21.15.83:0/3880872598 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe2c80cd5a0 con 0x7fe2d01a3080 2024-03-11T19:52:13.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.081+0000 7fe2ced9d640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2b80a5ea0 0x7fe2b80a8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.081+0000 7fe2ced9d640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2b80a5ea0 0x7fe2b80a8360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0005fd0 tx=0x7fe2c0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.082+0000 7fe2d7da8740 1 Processor -- start 2024-03-11T19:52:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.082+0000 7fe2d7da8740 1 -- start start 2024-03-11T19:52:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.082+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 0x55f035cffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.082+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d00470 0x55f035d04f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.082+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d055e0 0x55f035d07a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.082+0000 7fe2d7da8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f035cb3960 con 0x55f035d055e0 2024-03-11T19:52:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.082+0000 7fe2d7da8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f035c67310 con 0x55f035d00470 2024-03-11T19:52:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.082+0000 7fe2d7da8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f035c64840 con 0x55f035cffa80 2024-03-11T19:52:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.083+0000 7fe2cf59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d055e0 0x55f035d07a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.083+0000 7fe2cf59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d055e0 0x55f035d07a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37844/0 (socket says 172.21.15.83:37844) 2024-03-11T19:52:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.083+0000 7fe2cf59e640 1 -- 172.21.15.83:0/319017638 learned_addr learned my addr 172.21.15.83:0/319017638 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.083+0000 7fe2ced9d640 1 --2- 172.21.15.83:0/319017638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 0x55f035cffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.083+0000 7fe2cf59e640 1 -- 172.21.15.83:0/319017638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 msgr2=0x55f035cffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.083+0000 7fe2cf59e640 1 --2- 172.21.15.83:0/319017638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 0x55f035cffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.083+0000 7fe2cf59e640 1 -- 172.21.15.83:0/319017638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d00470 msgr2=0x55f035d04f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:52:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.083+0000 7fe2cf59e640 1 --2- 172.21.15.83:0/319017638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d00470 0x55f035d04f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.083+0000 7fe2cf59e640 1 -- 172.21.15.83:0/319017638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f035c22060 con 0x55f035d055e0 2024-03-11T19:52:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.084+0000 7fe2cf59e640 1 --2- 172.21.15.83:0/319017638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d055e0 0x55f035d07a70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8096ef0 tx=0x7fe2c809aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b1ec61e567e811 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.084+0000 7fe2b57fa640 1 -- 172.21.15.83:0/319017638 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80a2cc0 con 0x55f035d055e0 2024-03-11T19:52:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.085+0000 7fe2b57fa640 1 -- 172.21.15.83:0/319017638 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c80ba070 con 0x55f035d055e0 2024-03-11T19:52:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.085+0000 7fe2b57fa640 1 -- 172.21.15.83:0/319017638 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80cd0b0 con 0x55f035d055e0 2024-03-11T19:52:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.086+0000 7fe2d7da8740 1 -- 172.21.15.83:0/319017638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d055e0 msgr2=0x55f035d07a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.086+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/319017638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d055e0 0x55f035d07a70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8096ef0 tx=0x7fe2c809aca0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.086+0000 7fe2d7da8740 1 -- 172.21.15.83:0/319017638 shutdown_connections 2024-03-11T19:52:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.086+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/319017638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d055e0 0x55f035d07a70 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.086+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/319017638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d00470 0x55f035d04f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.086+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/319017638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 0x55f035cffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.086+0000 7fe2d7da8740 1 -- 172.21.15.83:0/319017638 >> 172.21.15.83:0/319017638 conn(0x55f035cf27e0 msgr2=0x55f035cf2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.088+0000 7fe2d7da8740 1 -- 172.21.15.83:0/319017638 shutdown_connections 2024-03-11T19:52:13.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.088+0000 7fe2d7da8740 1 -- 172.21.15.83:0/319017638 wait complete. 2024-03-11T19:52:13.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.088+0000 7fe2d7da8740 1 -- start start 2024-03-11T19:52:13.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.088+0000 7fe2d7da8740 1 Processor -- start 2024-03-11T19:52:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.089+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035cffa80 0x55f035cf8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.089+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d00470 0x55f035cf8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.089+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d055e0 0x55f035cfe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.089+0000 7fe2d7da8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f035c60020 con 0x55f035cffa80 2024-03-11T19:52:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.089+0000 7fe2ce59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d00470 0x55f035cf8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.089+0000 7fe2ce59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d00470 0x55f035cf8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39116/0 (socket says 172.21.15.83:39116) 2024-03-11T19:52:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.089+0000 7fe2ce59c640 1 -- 172.21.15.83:0/2047936617 learned_addr learned my addr 172.21.15.83:0/2047936617 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.089+0000 7fe2cf59e640 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d055e0 0x55f035cfe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.089+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2047936617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f035933430 con 0x55f035d00470 2024-03-11T19:52:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.090+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2047936617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0358efe00 con 0x55f035d055e0 2024-03-11T19:52:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.090+0000 7fe2d7da8740 10 client.0 _ll_register_callbacks cb 0x55f035c07cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.090+0000 7fe2cf59e640 1 -- 172.21.15.83:0/2047936617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d00470 msgr2=0x55f035cf8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.090+0000 7fe2cf59e640 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d00470 0x55f035cf8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.090+0000 7fe2cf59e640 1 -- 172.21.15.83:0/2047936617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035cffa80 msgr2=0x55f035cf8aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:52:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.090+0000 7fe2cf59e640 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035cffa80 0x55f035cf8aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.090+0000 7fe2cf59e640 1 -- 172.21.15.83:0/2047936617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d01a63e0 con 0x55f035d055e0 2024-03-11T19:52:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.090+0000 7fe2ce59c640 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d00470 0x55f035cf8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.090+0000 7fe2cf59e640 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d055e0 0x55f035cfe010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8096200 tx=0x7fe2c80ba1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4df3172ff1e85 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.091+0000 7fe2b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:52:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.091+0000 7fe2b2ff5640 1 -- 172.21.15.83:0/2047936617 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80a2cc0 con 0x55f035d055e0 2024-03-11T19:52:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.091+0000 7fe2b2ff5640 1 -- 172.21.15.83:0/2047936617 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe2c80cd0e0 con 0x55f035d055e0 2024-03-11T19:52:13.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.091+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2047936617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f03595f7e0 con 0x55f035d055e0 2024-03-11T19:52:13.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.091+0000 7fe2b2ff5640 1 -- 172.21.15.83:0/2047936617 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2c80cd2a0 con 0x55f035d055e0 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.091+0000 7fe2d7da8740 10 client.27325 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.091+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2047936617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f035c5e6a0 con 0x55f035d055e0 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.092+0000 7fe2b2ff5640 1 -- 172.21.15.83:0/2047936617 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f03595f7e0 con 0x55f035d055e0 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.092+0000 7fe2b2ff5640 1 -- 172.21.15.83:0/2047936617 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe2c8018030 con 0x55f035d055e0 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.092+0000 7fe2d7da8740 10 client.27325 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.092+0000 7fe2d7da8740 10 client.27325 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.092+0000 7fe2d7da8740 20 client.27325 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.092+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f035d1e170 0x55f035d20590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.092+0000 7fe2d7da8740 4 client.27325 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.092+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2047936617 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f03599c720 con 0x55f035d1e170 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.092+0000 7fe2ced9d640 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f035d1e170 0x55f035d20590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.092+0000 7fe2ced9d640 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f035d1e170 0x55f035d20590 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa91309c39dbf5e5 server_cookie=c6ae3230120c6e29 in_seq=0 out_seq=0 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.092+0000 7fe2b2ff5640 10 client.27325 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2b2ff5640 1 -- 172.21.15.83:0/2047936617 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f03599c720 con 0x55f035d1e170 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2b2ff5640 10 client.27325 handle_command_reply: tid=0 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2d7da8740 1 client.27325 shutdown 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2047936617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d055e0 msgr2=0x55f035cfe010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d055e0 0x55f035cfe010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8096200 tx=0x7fe2c80ba1e0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2047936617 shutdown_connections 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f035d1e170 0x55f035d20590 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d055e0 0x55f035cfe010 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2b2ff5640 0 client.27325 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d00470 0x55f035cf8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2047936617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035cffa80 0x55f035cf8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.093+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2047936617 >> 172.21.15.83:0/2047936617 conn(0x55f035cf27e0 msgr2=0x55f035cf1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.094+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2047936617 shutdown_connections 2024-03-11T19:52:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.094+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2047936617 wait complete. 2024-03-11T19:52:13.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.094+0000 7fe2d7da8740 20 client.27325 trim_cache size 0 max 16384 2024-03-11T19:52:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.109+0000 7fe2d7da8740 1 Processor -- start 2024-03-11T19:52:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.110+0000 7fe2d7da8740 1 -- start start 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.110+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 0x55f035d2e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.110+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d00470 0x55f035d2e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.110+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d055e0 0x55f035cf8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2d7da8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f035be5130 con 0x55f035d00470 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2d7da8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f03599c720 con 0x55f035d055e0 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2d7da8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f03595f7e0 con 0x55f035cffa80 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2ced9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 0x55f035d2e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2cf59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d055e0 0x55f035cf8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2ce59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d00470 0x55f035d2e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2cf59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d055e0 0x55f035cf8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39124/0 (socket says 172.21.15.83:39124) 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2cf59e640 1 -- 172.21.15.83:0/3746965662 learned_addr learned my addr 172.21.15.83:0/3746965662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2ced9d640 1 --2- 172.21.15.83:0/3746965662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 0x55f035d2e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:43354/0 (socket says 172.21.15.83:43354) 2024-03-11T19:52:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2ce59c640 1 -- 172.21.15.83:0/3746965662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 msgr2=0x55f035d2e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2ce59c640 1 --2- 172.21.15.83:0/3746965662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 0x55f035d2e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2ce59c640 1 -- 172.21.15.83:0/3746965662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d055e0 msgr2=0x55f035cf8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2ce59c640 1 --2- 172.21.15.83:0/3746965662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d055e0 0x55f035cf8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2ce59c640 1 -- 172.21.15.83:0/3746965662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f035bfac20 con 0x55f035d00470 2024-03-11T19:52:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2ced9d640 1 --2- 172.21.15.83:0/3746965662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 0x55f035d2e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2ce59c640 1 --2- 172.21.15.83:0/3746965662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d00470 0x55f035d2e9b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009780 tx=0x7fe2c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d2b72b0aa6bf3a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.111+0000 7fe2b4ff9640 1 -- 172.21.15.83:0/3746965662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4034d70 con 0x55f035d00470 2024-03-11T19:52:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.112+0000 7fe2b4ff9640 1 -- 172.21.15.83:0/3746965662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2c4004510 con 0x55f035d00470 2024-03-11T19:52:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.112+0000 7fe2b4ff9640 1 -- 172.21.15.83:0/3746965662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40052a0 con 0x55f035d00470 2024-03-11T19:52:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.112+0000 7fe2d7da8740 1 -- 172.21.15.83:0/3746965662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d00470 msgr2=0x55f035d2e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.112+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/3746965662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d00470 0x55f035d2e9b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009780 tx=0x7fe2c4000bc0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.112+0000 7fe2d7da8740 1 -- 172.21.15.83:0/3746965662 shutdown_connections 2024-03-11T19:52:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.112+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/3746965662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d055e0 0x55f035cf8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.112+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/3746965662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035d00470 0x55f035d2e9b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.112+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/3746965662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035cffa80 0x55f035d2e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.112+0000 7fe2d7da8740 1 -- 172.21.15.83:0/3746965662 >> 172.21.15.83:0/3746965662 conn(0x55f035cf27e0 msgr2=0x55f035d24f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.113+0000 7fe2d7da8740 1 -- 172.21.15.83:0/3746965662 shutdown_connections 2024-03-11T19:52:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.113+0000 7fe2d7da8740 1 -- 172.21.15.83:0/3746965662 wait complete. 2024-03-11T19:52:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.113+0000 7fe2d7da8740 1 -- start start 2024-03-11T19:52:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.113+0000 7fe2d7da8740 1 Processor -- start 2024-03-11T19:52:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.113+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035cffa80 0x55f035d6d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.113+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d00470 0x55f035d6d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.113+0000 7fe2d7da8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d055e0 0x55f035d74340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.113+0000 7fe2d7da8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f035d74880 con 0x55f035cffa80 2024-03-11T19:52:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2ce59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d00470 0x55f035d6d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2ce59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d00470 0x55f035d6d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:43362/0 (socket says 172.21.15.83:43362) 2024-03-11T19:52:13.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2ce59c640 1 -- 172.21.15.83:0/2672314467 learned_addr learned my addr 172.21.15.83:0/2672314467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2ced9d640 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035cffa80 0x55f035d6d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2cf59e640 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d055e0 0x55f035d74340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2672314467 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f035d749c0 con 0x55f035d055e0 2024-03-11T19:52:13.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2672314467 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f035d74b50 con 0x55f035d00470 2024-03-11T19:52:13.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2d7da8740 10 client.0 _ll_register_callbacks cb 0x7fe2c80967c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:13.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2ce59c640 1 -- 172.21.15.83:0/2672314467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d055e0 msgr2=0x55f035d74340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2ce59c640 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d055e0 0x55f035d74340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2ce59c640 1 -- 172.21.15.83:0/2672314467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035cffa80 msgr2=0x55f035d6d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2ce59c640 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035cffa80 0x55f035d6d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.114+0000 7fe2ce59c640 1 -- 172.21.15.83:0/2672314467 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c0014070 con 0x55f035d00470 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2672314467 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2c80ba070 con 0x55f035d00470 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2ce59c640 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d00470 0x55f035d6d8b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4037040 tx=0x7fe2c4034e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4672e40f2826e9f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2b2ff5640 10 client.27331 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2b2ff5640 1 -- 172.21.15.83:0/2672314467 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4005450 con 0x55f035d00470 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2b2ff5640 1 -- 172.21.15.83:0/2672314467 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe2c40055b0 con 0x55f035d00470 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2b2ff5640 1 -- 172.21.15.83:0/2672314467 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2c40046c0 con 0x55f035d00470 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2d7da8740 10 client.27331 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2672314467 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x7fe2d0110d30 con 0x55f035d00470 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2b2ff5640 1 -- 172.21.15.83:0/2672314467 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe2c4046070 con 0x55f035d00470 2024-03-11T19:52:13.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2d7da8740 10 client.27331 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2d7da8740 10 client.27331 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:52:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2d7da8740 20 client.27331 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f035d75fc0 0x55f035d783e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2d7da8740 4 client.27331 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:52:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.115+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2672314467 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f0359eb890 con 0x55f035d75fc0 2024-03-11T19:52:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2ced9d640 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f035d75fc0 0x55f035d783e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2b2ff5640 10 client.27331 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:52:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2ced9d640 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f035d75fc0 0x55f035d783e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7be91137153027e4 server_cookie=8d66e9e80973b1a6 in_seq=0 out_seq=0 2024-03-11T19:52:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2b2ff5640 1 -- 172.21.15.83:0/2672314467 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f0359eb890 con 0x55f035d75fc0 2024-03-11T19:52:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2b2ff5640 10 client.27331 handle_command_reply: tid=0 2024-03-11T19:52:13.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2d7da8740 1 client.27331 shutdown 2024-03-11T19:52:13.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2672314467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d00470 msgr2=0x55f035d6d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d00470 0x55f035d6d8b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4037040 tx=0x7fe2c4034e20 comp rx=0 tx=0).stop 2024-03-11T19:52:13.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2672314467 shutdown_connections 2024-03-11T19:52:13.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55f035d75fc0 0x55f035d783e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f035d055e0 0x55f035d74340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2b2ff5640 0 client.27331 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:52:13.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f035d00470 0x55f035d6d8b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2d7da8740 1 --2- 172.21.15.83:0/2672314467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f035cffa80 0x55f035d6d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2672314467 >> 172.21.15.83:0/2672314467 conn(0x55f035cf27e0 msgr2=0x55f035d01960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.116+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2672314467 shutdown_connections 2024-03-11T19:52:13.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.117+0000 7fe2d7da8740 1 -- 172.21.15.83:0/2672314467 wait complete. 2024-03-11T19:52:13.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.117+0000 7fe2d7da8740 20 client.27331 trim_cache size 0 max 16384 2024-03-11T19:52:13.130 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.117+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3880872598 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2b80a5ea0 msgr2=0x7fe2b80a8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.117+0000 7fe2d51dd640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2b80a5ea0 0x7fe2b80a8360 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0005fd0 tx=0x7fe2c0005850 comp rx=0 tx=0).stop 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.117+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3880872598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d01a3080 msgr2=0x7fe2d01a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.117+0000 7fe2d51dd640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d01a3080 0x7fe2d01a5470 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800eaa0 tx=0x7fe2c800ef60 comp rx=0 tx=0).stop 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.118+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3880872598 shutdown_connections 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.118+0000 7fe2d51dd640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe2b80a5ea0 0x7fe2b80a8360 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.118+0000 7fe2d51dd640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2d01a3080 0x7fe2d01a5470 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.118+0000 7fe2d51dd640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2d010dcc0 0x7fe2d019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.118+0000 7fe2d51dd640 1 --2- 172.21.15.83:0/3880872598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2d01083f0 0x7fe2d019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.118+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3880872598 >> 172.21.15.83:0/3880872598 conn(0x7fe2d006e380 msgr2=0x7fe2d010d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.118+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3880872598 shutdown_connections 2024-03-11T19:52:13.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.118+0000 7fe2d51dd640 1 -- 172.21.15.83:0/3880872598 wait complete. 2024-03-11T19:52:13.135 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:52:13.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:52:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.287+0000 7fdad759e640 1 -- 172.21.15.83:0/879593499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad810d710 msgr2=0x7fdad810fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.287+0000 7fdad759e640 1 --2- 172.21.15.83:0/879593499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad810d710 0x7fdad810fbd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdad000bd70 tx=0x7fdad002dc80 comp rx=0 tx=0).stop 2024-03-11T19:52:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.292+0000 7fdad759e640 1 -- 172.21.15.83:0/879593499 shutdown_connections 2024-03-11T19:52:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.292+0000 7fdad759e640 1 --2- 172.21.15.83:0/879593499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad810d710 0x7fdad810fbd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.292+0000 7fdad759e640 1 --2- 172.21.15.83:0/879593499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad810ac90 0x7fdad810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.292+0000 7fdad759e640 1 --2- 172.21.15.83:0/879593499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad8108360 0x7fdad810a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.292+0000 7fdad759e640 1 -- 172.21.15.83:0/879593499 >> 172.21.15.83:0/879593499 conn(0x7fdad806e250 msgr2=0x7fdad8070670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.296+0000 7fdad759e640 1 -- 172.21.15.83:0/879593499 shutdown_connections 2024-03-11T19:52:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.296+0000 7fdad759e640 1 -- 172.21.15.83:0/879593499 wait complete. 2024-03-11T19:52:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.297+0000 7fdad759e640 1 Processor -- start 2024-03-11T19:52:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.297+0000 7fdad759e640 1 -- start start 2024-03-11T19:52:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.297+0000 7fdad759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad8108360 0x7fdad8118980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.298+0000 7fdad759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad810ac90 0x7fdad8118ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.298+0000 7fdad759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad810d710 0x7fdad81b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.298+0000 7fdad759e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad811df20 con 0x7fdad8108360 2024-03-11T19:52:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.298+0000 7fdad759e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad811e090 con 0x7fdad810ac90 2024-03-11T19:52:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.298+0000 7fdad759e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad811e250 con 0x7fdad810d710 2024-03-11T19:52:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.298+0000 7fdad5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad810ac90 0x7fdad8118ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.298+0000 7fdad5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad810ac90 0x7fdad8118ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39152/0 (socket says 172.21.15.83:39152) 2024-03-11T19:52:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.298+0000 7fdad5d9b640 1 -- 172.21.15.83:0/836836698 learned_addr learned my addr 172.21.15.83:0/836836698 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.298+0000 7fdad659c640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad8108360 0x7fdad8118980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.299+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad810d710 0x7fdad81b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.299+0000 7fdad659c640 1 -- 172.21.15.83:0/836836698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad810d710 msgr2=0x7fdad81b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.299+0000 7fdad659c640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad810d710 0x7fdad81b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.299+0000 7fdad659c640 1 -- 172.21.15.83:0/836836698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad810ac90 msgr2=0x7fdad8118ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.299+0000 7fdad659c640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad810ac90 0x7fdad8118ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.299+0000 7fdad659c640 1 -- 172.21.15.83:0/836836698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad000ba50 con 0x7fdad8108360 2024-03-11T19:52:13.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.308+0000 7fdad659c640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad8108360 0x7fdad8118980 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00d6d0 tx=0x7fdacc00dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.309+0000 7fdac77fe640 1 -- 172.21.15.83:0/836836698 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc005220 con 0x7fdad8108360 2024-03-11T19:52:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.309+0000 7fdac77fe640 1 -- 172.21.15.83:0/836836698 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdacc004c80 con 0x7fdad8108360 2024-03-11T19:52:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.309+0000 7fdac77fe640 1 -- 172.21.15.83:0/836836698 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc009d40 con 0x7fdad8108360 2024-03-11T19:52:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.310+0000 7fdad759e640 1 -- 172.21.15.83:0/836836698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad81b67a0 con 0x7fdad8108360 2024-03-11T19:52:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.310+0000 7fdad759e640 1 -- 172.21.15.83:0/836836698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad81b6ba0 con 0x7fdad8108360 2024-03-11T19:52:13.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.314+0000 7fdadf5cb740 1 Processor -- start 2024-03-11T19:52:13.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.314+0000 7fdadf5cb740 1 -- start start 2024-03-11T19:52:13.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.314+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 0x55b32c1b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.318+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9e50 0x55b32c1be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.318+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1befc0 0x55b32c1c1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.318+0000 7fdadf5cb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32c175960 con 0x55b32c1befc0 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.318+0000 7fdadf5cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32c129310 con 0x55b32c1b9e50 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.318+0000 7fdadf5cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32c126840 con 0x55b32c1b9460 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.318+0000 7fdad659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9e50 0x55b32c1be950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.318+0000 7fdad6d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 0x55b32c1b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.318+0000 7fdad659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9e50 0x55b32c1be950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39154/0 (socket says 172.21.15.83:39154) 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.319+0000 7fdad659c640 1 -- 172.21.15.83:0/1873256347 learned_addr learned my addr 172.21.15.83:0/1873256347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.319+0000 7fdac77fe640 1 -- 172.21.15.83:0/836836698 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdacc0054a0 con 0x7fdad8108360 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.319+0000 7fdad6d9d640 1 -- 172.21.15.83:0/1873256347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9e50 msgr2=0x55b32c1be950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.319+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/1873256347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9e50 0x55b32c1be950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.319+0000 7fdad6d9d640 1 -- 172.21.15.83:0/1873256347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1befc0 msgr2=0x55b32c1c1450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:52:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.319+0000 7fdad5d9b640 1 --2- 172.21.15.83:0/1873256347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1befc0 0x55b32c1c1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.319+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/1873256347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1befc0 0x55b32c1c1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.319+0000 7fdad6d9d640 1 -- 172.21.15.83:0/1873256347 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b32c0e4060 con 0x55b32c1b9460 2024-03-11T19:52:13.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.320+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/1873256347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 0x55b32c1b9880 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdad0006010 tx=0x7fdad002e1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b1e74a70ca5d0d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.320+0000 7fdac4ff9640 1 -- 172.21.15.83:0/1873256347 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad002ea30 con 0x55b32c1b9460 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.320+0000 7fdac4ff9640 1 -- 172.21.15.83:0/1873256347 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdad002eb90 con 0x55b32c1b9460 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.320+0000 7fdac4ff9640 1 -- 172.21.15.83:0/1873256347 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0003d70 con 0x55b32c1b9460 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.321+0000 7fdac77fe640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdac00a1c40 0x7fdac00a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.321+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1873256347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 msgr2=0x55b32c1b9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.321+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/1873256347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 0x55b32c1b9880 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdad0006010 tx=0x7fdad002e1a0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.322+0000 7fdac77fe640 1 -- 172.21.15.83:0/836836698 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdacc0ccbb0 con 0x7fdad8108360 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.322+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1873256347 shutdown_connections 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.322+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/1873256347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1befc0 0x55b32c1c1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.322+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/1873256347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9e50 0x55b32c1be950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.322+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/1873256347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 0x55b32c1b9880 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.322+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1873256347 >> 172.21.15.83:0/1873256347 conn(0x55b32c1b49d0 msgr2=0x55b32c1b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.323+0000 7fdad659c640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdac00a1c40 0x7fdac00a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.324+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1873256347 shutdown_connections 2024-03-11T19:52:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.328+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1873256347 wait complete. 2024-03-11T19:52:13.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.328+0000 7fdad659c640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdac00a1c40 0x7fdac00a4100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdacc099ac0 tx=0x7fdacc0097e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.328+0000 7fdadf5cb740 1 -- start start 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.328+0000 7fdadf5cb740 1 Processor -- start 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.328+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9460 0x55b32c1c3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.329+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 0x55b32c1c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.329+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1befc0 0x55b32c1ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.329+0000 7fdadf5cb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32c122020 con 0x55b32c1b9e50 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.329+0000 7fdad6d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9460 0x55b32c1c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.329+0000 7fdad6d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9460 0x55b32c1c3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39166/0 (socket says 172.21.15.83:39166) 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.329+0000 7fdad6d9d640 1 -- 172.21.15.83:0/1912264123 learned_addr learned my addr 172.21.15.83:0/1912264123 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.329+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1912264123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32bdf5430 con 0x55b32c1b9460 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.329+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1912264123 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32bdb1e00 con 0x55b32c1befc0 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.329+0000 7fdadf5cb740 10 client.0 _ll_register_callbacks cb 0x55b32c0c9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:13.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.329+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1befc0 0x55b32c1ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.329+0000 7fdad5d9b640 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 0x55b32c1c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.330+0000 7fdad6d9d640 1 -- 172.21.15.83:0/1912264123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1befc0 msgr2=0x55b32c1ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.330+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1befc0 0x55b32c1ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.330+0000 7fdad6d9d640 1 -- 172.21.15.83:0/1912264123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 msgr2=0x55b32c1c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.330+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 0x55b32c1c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.330+0000 7fdad6d9d640 1 -- 172.21.15.83:0/1912264123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad81107b0 con 0x55b32c1b9460 2024-03-11T19:52:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.330+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9460 0x55b32c1c3460 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdad0036a00 tx=0x7fdad0036c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e01428631692358 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.330+0000 7fdabdffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:52:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.330+0000 7fdabdffb640 1 -- 172.21.15.83:0/1912264123 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0051070 con 0x55b32c1b9460 2024-03-11T19:52:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.330+0000 7fdabdffb640 1 -- 172.21.15.83:0/1912264123 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdad0048660 con 0x55b32c1b9460 2024-03-11T19:52:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.330+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1912264123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b32be5e720 con 0x55b32c1b9460 2024-03-11T19:52:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.332+0000 7fdabdffb640 1 -- 172.21.15.83:0/1912264123 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdad0048820 con 0x55b32c1b9460 2024-03-11T19:52:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.332+0000 7fdadf5cb740 10 client.27230 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:13.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.332+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1912264123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b32c1206a0 con 0x55b32c1b9460 2024-03-11T19:52:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.333+0000 7fdabdffb640 1 -- 172.21.15.83:0/1912264123 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b32be5e720 con 0x55b32c1b9460 2024-03-11T19:52:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.333+0000 7fdabdffb640 1 -- 172.21.15.83:0/1912264123 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdad003fbc0 con 0x55b32c1b9460 2024-03-11T19:52:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.333+0000 7fdadf5cb740 10 client.27230 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.333+0000 7fdadf5cb740 10 client.27230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:52:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.333+0000 7fdadf5cb740 20 client.27230 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.333+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b32c1cf290 0x55b32c1d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.333+0000 7fdadf5cb740 4 client.27230 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.333+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1912264123 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b32bead890 con 0x55b32c1cf290 2024-03-11T19:52:13.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.334+0000 7fdad5d9b640 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b32c1cf290 0x55b32c1d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.334+0000 7fdabdffb640 10 client.27230 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:52:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.334+0000 7fdad5d9b640 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b32c1cf290 0x55b32c1d16b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=748f837b21c5b25c server_cookie=8d2a1ffcdc8e127a in_seq=0 out_seq=0 2024-03-11T19:52:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.335+0000 7fdabdffb640 1 -- 172.21.15.83:0/1912264123 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b32bead890 con 0x55b32c1cf290 2024-03-11T19:52:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.335+0000 7fdabdffb640 10 client.27230 handle_command_reply: tid=0 2024-03-11T19:52:13.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.335+0000 7fdadf5cb740 1 client.27230 shutdown 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1912264123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9460 msgr2=0x55b32c1c3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9460 0x55b32c1c3460 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdad0036a00 tx=0x7fdad0036c70 comp rx=0 tx=0).stop 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1912264123 shutdown_connections 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b32c1cf290 0x55b32c1d16b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdabdffb640 0 client.27230 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1befc0 0x55b32c1ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 0x55b32c1c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/1912264123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9460 0x55b32c1c3460 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1912264123 >> 172.21.15.83:0/1912264123 conn(0x55b32c1b49d0 msgr2=0x55b32c1b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1912264123 shutdown_connections 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdadf5cb740 1 -- 172.21.15.83:0/1912264123 wait complete. 2024-03-11T19:52:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.336+0000 7fdadf5cb740 20 client.27230 trim_cache size 0 max 16384 2024-03-11T19:52:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.352+0000 7fdadf5cb740 1 Processor -- start 2024-03-11T19:52:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.352+0000 7fdadf5cb740 1 -- start start 2024-03-11T19:52:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.352+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 0x55b32c1ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.352+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 0x55b32c1ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.352+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1befc0 0x55b32c1c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.352+0000 7fdadf5cb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32c0a7130 con 0x55b32c1b9e50 2024-03-11T19:52:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.352+0000 7fdadf5cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32c1b4840 con 0x55b32c1befc0 2024-03-11T19:52:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.352+0000 7fdadf5cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32c1cbf00 con 0x55b32c1b9460 2024-03-11T19:52:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 0x55b32c1ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 0x55b32c1ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37900/0 (socket says 172.21.15.83:37900) 2024-03-11T19:52:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad5d9b640 1 -- 172.21.15.83:0/2727748777 learned_addr learned my addr 172.21.15.83:0/2727748777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/2727748777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 0x55b32c1ba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/2727748777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1befc0 0x55b32c1c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad6d9d640 1 -- 172.21.15.83:0/2727748777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1befc0 msgr2=0x55b32c1c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/2727748777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1befc0 0x55b32c1c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad6d9d640 1 -- 172.21.15.83:0/2727748777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 msgr2=0x55b32c1ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/2727748777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 0x55b32c1ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad6d9d640 1 -- 172.21.15.83:0/2727748777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b32c0bcc20 con 0x55b32c1b9460 2024-03-11T19:52:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad5d9b640 1 --2- 172.21.15.83:0/2727748777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 0x55b32c1ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.353+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/2727748777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 0x55b32c1ba8f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdad004a460 tx=0x7fdad0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e83e3078f5112e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.354+0000 7fdabffff640 1 -- 172.21.15.83:0/2727748777 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0051040 con 0x55b32c1b9460 2024-03-11T19:52:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.354+0000 7fdabffff640 1 -- 172.21.15.83:0/2727748777 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdad0052040 con 0x55b32c1b9460 2024-03-11T19:52:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.354+0000 7fdabffff640 1 -- 172.21.15.83:0/2727748777 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0048660 con 0x55b32c1b9460 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.354+0000 7fdadf5cb740 1 -- 172.21.15.83:0/2727748777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 msgr2=0x55b32c1ba8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.354+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/2727748777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 0x55b32c1ba8f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdad004a460 tx=0x7fdad0005fd0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 -- 172.21.15.83:0/2727748777 shutdown_connections 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/2727748777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1befc0 0x55b32c1c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/2727748777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9e50 0x55b32c1ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/2727748777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1b9460 0x55b32c1ba8f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 -- 172.21.15.83:0/2727748777 >> 172.21.15.83:0/2727748777 conn(0x55b32c1b49d0 msgr2=0x55b32c1bb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 -- 172.21.15.83:0/2727748777 shutdown_connections 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 -- 172.21.15.83:0/2727748777 wait complete. 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 -- start start 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 Processor -- start 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9460 0x55b32c1ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9e50 0x55b32c1c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.355+0000 7fdadf5cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1befc0 0x55b32c1c9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdad6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9460 0x55b32c1ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdad6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9460 0x55b32c1ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37908/0 (socket says 172.21.15.83:37908) 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdad6d9d640 1 -- 172.21.15.83:0/4248927201 learned_addr learned my addr 172.21.15.83:0/4248927201 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdadf5cb740 1 -- 172.21.15.83:0/4248927201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32c1cade0 con 0x55b32c1b9460 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdadf5cb740 1 -- 172.21.15.83:0/4248927201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32c1c96b0 con 0x55b32c1b9e50 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdadf5cb740 1 -- 172.21.15.83:0/4248927201 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32c1c9870 con 0x55b32c1befc0 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1befc0 0x55b32c1c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdadf5cb740 10 client.0 _ll_register_callbacks cb 0x7fdac8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdad6d9d640 1 -- 172.21.15.83:0/4248927201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9e50 msgr2=0x55b32c1c8c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9e50 0x55b32c1c8c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdad6d9d640 1 -- 172.21.15.83:0/4248927201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9460 msgr2=0x55b32c1ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9460 0x55b32c1ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.356+0000 7fdad6d9d640 1 -- 172.21.15.83:0/4248927201 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac80042c0 con 0x55b32c1befc0 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.357+0000 7fdad6d9d640 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1befc0 0x55b32c1c9140 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdad0036aa0 tx=0x7fdad000b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e467536c65921da3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.357+0000 7fdabdffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.357+0000 7fdabdffb640 1 -- 172.21.15.83:0/4248927201 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0051030 con 0x55b32c1befc0 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.357+0000 7fdabdffb640 1 -- 172.21.15.83:0/4248927201 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdad0048660 con 0x55b32c1befc0 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.357+0000 7fdadf5cb740 1 -- 172.21.15.83:0/4248927201 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b32be217e0 con 0x55b32c1befc0 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.357+0000 7fdabdffb640 1 -- 172.21.15.83:0/4248927201 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdad004acc0 con 0x55b32c1befc0 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.357+0000 7fdadf5cb740 10 client.27349 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.357+0000 7fdadf5cb740 1 -- 172.21.15.83:0/4248927201 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b32c1206a0 con 0x55b32c1befc0 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.358+0000 7fdabdffb640 1 -- 172.21.15.83:0/4248927201 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdad004ae90 con 0x55b32c1befc0 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.358+0000 7fdabdffb640 1 -- 172.21.15.83:0/4248927201 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdad0072050 con 0x55b32c1befc0 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.358+0000 7fdadf5cb740 10 client.27349 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.358+0000 7fdadf5cb740 10 client.27349 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:52:13.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.358+0000 7fdadf5cb740 20 client.27349 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.358+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b32c1cf290 0x55b32c2289e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.358+0000 7fdadf5cb740 4 client.27349 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.358+0000 7fdadf5cb740 1 -- 172.21.15.83:0/4248927201 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b32be217e0 con 0x55b32c1cf290 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.358+0000 7fdad5d9b640 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b32c1cf290 0x55b32c2289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.358+0000 7fdad5d9b640 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b32c1cf290 0x55b32c2289e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e8e3bc33db2c26b server_cookie=313b1f45fc189208 in_seq=0 out_seq=0 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdabdffb640 10 client.27349 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdabdffb640 1 -- 172.21.15.83:0/4248927201 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b32be217e0 con 0x55b32c1cf290 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdabdffb640 10 client.27349 handle_command_reply: tid=0 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 1 client.27349 shutdown 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 1 -- 172.21.15.83:0/4248927201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1befc0 msgr2=0x55b32c1c9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1befc0 0x55b32c1c9140 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdad0036aa0 tx=0x7fdad000b710 comp rx=0 tx=0).stop 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 1 -- 172.21.15.83:0/4248927201 shutdown_connections 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55b32c1cf290 0x55b32c2289e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b32c1befc0 0x55b32c1c9140 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdabdffb640 0 client.27349 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:52:13.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b32c1b9e50 0x55b32c1c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 1 --2- 172.21.15.83:0/4248927201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b32c1b9460 0x55b32c1ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 1 -- 172.21.15.83:0/4248927201 >> 172.21.15.83:0/4248927201 conn(0x55b32c1b49d0 msgr2=0x55b32c1bb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 1 -- 172.21.15.83:0/4248927201 shutdown_connections 2024-03-11T19:52:13.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 1 -- 172.21.15.83:0/4248927201 wait complete. 2024-03-11T19:52:13.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.359+0000 7fdadf5cb740 20 client.27349 trim_cache size 0 max 16384 2024-03-11T19:52:13.368 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:52:13.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.360+0000 7fdad759e640 1 -- 172.21.15.83:0/836836698 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdac00a1c40 msgr2=0x7fdac00a4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.360+0000 7fdad759e640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdac00a1c40 0x7fdac00a4100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdacc099ac0 tx=0x7fdacc0097e0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.360+0000 7fdad759e640 1 -- 172.21.15.83:0/836836698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad8108360 msgr2=0x7fdad8118980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.360+0000 7fdad759e640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad8108360 0x7fdad8118980 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00d6d0 tx=0x7fdacc00dad0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.360+0000 7fdad759e640 1 -- 172.21.15.83:0/836836698 shutdown_connections 2024-03-11T19:52:13.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.360+0000 7fdad759e640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdac00a1c40 0x7fdac00a4100 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.360+0000 7fdad759e640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdad810d710 0x7fdad81b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.361+0000 7fdad759e640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdad810ac90 0x7fdad8118ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.361+0000 7fdad759e640 1 --2- 172.21.15.83:0/836836698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdad8108360 0x7fdad8118980 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.361+0000 7fdad759e640 1 -- 172.21.15.83:0/836836698 >> 172.21.15.83:0/836836698 conn(0x7fdad806e250 msgr2=0x7fdad810c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.361+0000 7fdad759e640 1 -- 172.21.15.83:0/836836698 shutdown_connections 2024-03-11T19:52:13.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.361+0000 7fdad759e640 1 -- 172.21.15.83:0/836836698 wait complete. 2024-03-11T19:52:13.377 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:52:13.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:52:13.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:13 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/840787757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:13.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.511+0000 7f6793b13640 1 -- 172.21.15.83:0/2116635835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f678c0ff800 msgr2=0x7f678c0ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.511+0000 7f6793b13640 1 --2- 172.21.15.83:0/2116635835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f678c0ff800 0x7f678c0ffc20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6788009900 tx=0x7f678802bc30 comp rx=0 tx=0).stop 2024-03-11T19:52:13.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.511+0000 7f6793b13640 1 -- 172.21.15.83:0/2116635835 shutdown_connections 2024-03-11T19:52:13.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.511+0000 7f6793b13640 1 --2- 172.21.15.83:0/2116635835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c1013b0 0x7f678c107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.511+0000 7f6793b13640 1 --2- 172.21.15.83:0/2116635835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f678c1009f0 0x7f678c100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.511+0000 7f6793b13640 1 --2- 172.21.15.83:0/2116635835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f678c0ff800 0x7f678c0ffc20 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.511+0000 7f6793b13640 1 -- 172.21.15.83:0/2116635835 >> 172.21.15.83:0/2116635835 conn(0x7f678c0fad70 msgr2=0x7f678c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.511+0000 7f6793b13640 1 -- 172.21.15.83:0/2116635835 shutdown_connections 2024-03-11T19:52:13.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.511+0000 7f6793b13640 1 -- 172.21.15.83:0/2116635835 wait complete. 2024-03-11T19:52:13.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.512+0000 7f6793b13640 1 Processor -- start 2024-03-11T19:52:13.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.512+0000 7f6793b13640 1 -- start start 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.512+0000 7f6793b13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f678c0ff800 0x7f678c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.512+0000 7f6793b13640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c1009f0 0x7f678c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.512+0000 7f6793b13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f678c1013b0 0x7f678c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.512+0000 7f6793b13640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c19ed60 con 0x7f678c0ff800 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.512+0000 7f6793b13640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c19eed0 con 0x7f678c1013b0 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.512+0000 7f6793b13640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c101ce0 con 0x7f678c1009f0 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f67918af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f678c0ff800 0x7f678c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f67918af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f678c0ff800 0x7f678c197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37930/0 (socket says 172.21.15.83:37930) 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f67918af640 1 -- 172.21.15.83:0/3213180310 learned_addr learned my addr 172.21.15.83:0/3213180310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f67910ae640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c1009f0 0x7f678c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f67920b0640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f678c1013b0 0x7f678c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f67918af640 1 -- 172.21.15.83:0/3213180310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c1009f0 msgr2=0x7f678c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f67918af640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c1009f0 0x7f678c1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f67918af640 1 -- 172.21.15.83:0/3213180310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f678c1013b0 msgr2=0x7f678c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f67918af640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f678c1013b0 0x7f678c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f67918af640 1 -- 172.21.15.83:0/3213180310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6788009560 con 0x7f678c0ff800 2024-03-11T19:52:13.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f67918af640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f678c0ff800 0x7f678c197270 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6788000c00 tx=0x7f6788005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f6782ffd640 1 -- 172.21.15.83:0/3213180310 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67880043d0 con 0x7f678c0ff800 2024-03-11T19:52:13.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f6793b13640 1 -- 172.21.15.83:0/3213180310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c101f60 con 0x7f678c0ff800 2024-03-11T19:52:13.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f6782ffd640 1 -- 172.21.15.83:0/3213180310 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67880052d0 con 0x7f678c0ff800 2024-03-11T19:52:13.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f6782ffd640 1 -- 172.21.15.83:0/3213180310 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788003d30 con 0x7f678c0ff800 2024-03-11T19:52:13.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.513+0000 7f6793b13640 1 -- 172.21.15.83:0/3213180310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c1024c0 con 0x7f678c0ff800 2024-03-11T19:52:13.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.515+0000 7f6782ffd640 1 -- 172.21.15.83:0/3213180310 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6788005440 con 0x7f678c0ff800 2024-03-11T19:52:13.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.516+0000 7f67967ca740 1 Processor -- start 2024-03-11T19:52:13.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.516+0000 7f67967ca740 1 -- start start 2024-03-11T19:52:13.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.516+0000 7f67967ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.516+0000 7f67967ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b1470 0x557d584b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.516+0000 7f67967ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b65e0 0x557d584b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.516+0000 7f67967ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d58464960 con 0x557d584b1470 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.516+0000 7f67967ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d58418310 con 0x557d584b65e0 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.516+0000 7f67967ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d58415840 con 0x557d584b0a80 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f6782ffd640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f67780a1a30 0x7f67780a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67920b0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67920b0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:43446/0 (socket says 172.21.15.83:43446) 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67920b0640 1 -- 172.21.15.83:0/1037841760 learned_addr learned my addr 172.21.15.83:0/1037841760 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67910ae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b65e0 0x557d584b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67918af640 1 --2- 172.21.15.83:0/1037841760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b1470 0x557d584b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67920b0640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f67780a1a30 0x7f67780a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67910ae640 1 -- 172.21.15.83:0/1037841760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 msgr2=0x557d584b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67910ae640 1 --2- 172.21.15.83:0/1037841760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67910ae640 1 -- 172.21.15.83:0/1037841760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b1470 msgr2=0x557d584b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67910ae640 1 --2- 172.21.15.83:0/1037841760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b1470 0x557d584b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67910ae640 1 -- 172.21.15.83:0/1037841760 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d583d3060 con 0x557d584b65e0 2024-03-11T19:52:13.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67920b0640 1 --2- 172.21.15.83:0/1037841760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:13.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67918af640 1 --2- 172.21.15.83:0/1037841760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b1470 0x557d584b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:52:13.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f6782ffd640 1 -- 172.21.15.83:0/3213180310 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f67880efc20 con 0x7f678c0ff800 2024-03-11T19:52:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.517+0000 7f67910ae640 1 --2- 172.21.15.83:0/1037841760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b65e0 0x557d584b8a70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f677c009bf0 tx=0x7f677c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c857572e9af2155 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.518+0000 7f67920b0640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f67780a1a30 0x7f67780a3ef0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x557d584b1cd0 tx=0x7f6784004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.518+0000 7f6777fff640 1 -- 172.21.15.83:0/1037841760 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c009da0 con 0x557d584b65e0 2024-03-11T19:52:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.518+0000 7f6777fff640 1 -- 172.21.15.83:0/1037841760 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f677c004510 con 0x557d584b65e0 2024-03-11T19:52:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.518+0000 7f6777fff640 1 -- 172.21.15.83:0/1037841760 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c004b20 con 0x557d584b65e0 2024-03-11T19:52:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.518+0000 7f67967ca740 1 -- 172.21.15.83:0/1037841760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b65e0 msgr2=0x557d584b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.518+0000 7f67967ca740 1 --2- 172.21.15.83:0/1037841760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b65e0 0x557d584b8a70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f677c009bf0 tx=0x7f677c011810 comp rx=0 tx=0).stop 2024-03-11T19:52:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.518+0000 7f67967ca740 1 -- 172.21.15.83:0/1037841760 shutdown_connections 2024-03-11T19:52:13.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.518+0000 7f67967ca740 1 --2- 172.21.15.83:0/1037841760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b65e0 0x557d584b8a70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.518+0000 7f67967ca740 1 --2- 172.21.15.83:0/1037841760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b1470 0x557d584b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.518+0000 7f67967ca740 1 --2- 172.21.15.83:0/1037841760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.518+0000 7f67967ca740 1 -- 172.21.15.83:0/1037841760 >> 172.21.15.83:0/1037841760 conn(0x557d584a37e0 msgr2=0x557d584a3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67967ca740 1 -- 172.21.15.83:0/1037841760 shutdown_connections 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67967ca740 1 -- 172.21.15.83:0/1037841760 wait complete. 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67967ca740 1 -- start start 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67967ca740 1 Processor -- start 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67967ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67967ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 0x557d584a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67967ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67967ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d58411020 con 0x557d584b65e0 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67967ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d580e4430 con 0x557d584b1470 2024-03-11T19:52:13.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67967ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d580a0e00 con 0x557d584b0a80 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67967ca740 10 client.0 _ll_register_callbacks cb 0x557d583b8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.519+0000 7f67920b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67918af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67918af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584a9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:43458/0 (socket says 172.21.15.83:43458) 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67918af640 1 -- 172.21.15.83:0/2497573075 learned_addr learned my addr 172.21.15.83:0/2497573075 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67920b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584aefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37948/0 (socket says 172.21.15.83:37948) 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67910ae640 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 0x557d584a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67910ae640 1 -- 172.21.15.83:0/2497573075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 msgr2=0x557d584a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67910ae640 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584a9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67910ae640 1 -- 172.21.15.83:0/2497573075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 msgr2=0x557d584aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67910ae640 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67910ae640 1 -- 172.21.15.83:0/2497573075 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c009850 con 0x557d584b1470 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67918af640 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:13.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67920b0640 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67910ae640 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 0x557d584a9f80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f677c01b040 tx=0x7f677c005030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1273eb8ca738f62 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67757fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67757fa640 1 -- 172.21.15.83:0/2497573075 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c0054b0 con 0x557d584b1470 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67967ca740 1 -- 172.21.15.83:0/2497573075 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d581107e0 con 0x557d584b1470 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67757fa640 1 -- 172.21.15.83:0/2497573075 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f677c009da0 con 0x557d584b1470 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67757fa640 1 -- 172.21.15.83:0/2497573075 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f677c025470 con 0x557d584b1470 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67967ca740 10 client.27245 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.520+0000 7f67967ca740 1 -- 172.21.15.83:0/2497573075 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557d5840f6a0 con 0x557d584b1470 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.521+0000 7f67757fa640 1 -- 172.21.15.83:0/2497573075 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557d581107e0 con 0x557d584b1470 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.521+0000 7f67757fa640 1 -- 172.21.15.83:0/2497573075 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f677c021040 con 0x557d584b1470 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.521+0000 7f67967ca740 10 client.27245 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.521+0000 7f67967ca740 10 client.27245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:13.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.521+0000 7f67967ca740 20 client.27245 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.521+0000 7f67967ca740 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557d584cf520 0x557d584d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.521+0000 7f67967ca740 4 client.27245 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.521+0000 7f67967ca740 1 -- 172.21.15.83:0/2497573075 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d5814d720 con 0x557d584cf520 2024-03-11T19:52:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.521+0000 7f67918af640 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557d584cf520 0x557d584d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.522+0000 7f67918af640 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557d584cf520 0x557d584d1940 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cbc3bee6115200a server_cookie=45598b20de17f8d in_seq=0 out_seq=0 2024-03-11T19:52:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.522+0000 7f67757fa640 10 client.27245 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.522+0000 7f67757fa640 1 -- 172.21.15.83:0/2497573075 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d5814d720 con 0x557d584cf520 2024-03-11T19:52:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.522+0000 7f67757fa640 10 client.27245 handle_command_reply: tid=0 2024-03-11T19:52:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.523+0000 7f67967ca740 1 client.27245 shutdown 2024-03-11T19:52:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.523+0000 7f67967ca740 1 -- 172.21.15.83:0/2497573075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 msgr2=0x557d584a9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.523+0000 7f67967ca740 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 0x557d584a9f80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f677c01b040 tx=0x7f677c005030 comp rx=0 tx=0).stop 2024-03-11T19:52:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.523+0000 7f67967ca740 1 -- 172.21.15.83:0/2497573075 shutdown_connections 2024-03-11T19:52:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.523+0000 7f67967ca740 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557d584cf520 0x557d584d1940 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.523+0000 7f67967ca740 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.523+0000 7f67967ca740 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 0x557d584a9f80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.523+0000 7f67967ca740 1 --2- 172.21.15.83:0/2497573075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b0a80 0x557d584a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.523+0000 7f67757fa640 0 client.27245 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.523+0000 7f67967ca740 1 -- 172.21.15.83:0/2497573075 >> 172.21.15.83:0/2497573075 conn(0x557d584a37e0 msgr2=0x557d584a2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.524+0000 7f67967ca740 1 -- 172.21.15.83:0/2497573075 shutdown_connections 2024-03-11T19:52:13.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.524+0000 7f67967ca740 1 -- 172.21.15.83:0/2497573075 wait complete. 2024-03-11T19:52:13.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.524+0000 7f67967ca740 20 client.27245 trim_cache size 0 max 16384 2024-03-11T19:52:13.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67967ca740 1 Processor -- start 2024-03-11T19:52:13.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67967ca740 1 -- start start 2024-03-11T19:52:13.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67967ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b0a80 0x557d584d62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67967ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b1470 0x557d584df7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67967ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67967ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d58396130 con 0x557d584b65e0 2024-03-11T19:52:13.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67967ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5814d720 con 0x557d584b0a80 2024-03-11T19:52:13.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67967ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d581107e0 con 0x557d584b1470 2024-03-11T19:52:13.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67920b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67910ae640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b1470 0x557d584df7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67920b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584dfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37958/0 (socket says 172.21.15.83:37958) 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67910ae640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b1470 0x557d584df7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:43468/0 (socket says 172.21.15.83:43468) 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67920b0640 1 -- 172.21.15.83:0/1662579531 learned_addr learned my addr 172.21.15.83:0/1662579531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67918af640 1 --2- 172.21.15.83:0/1662579531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b0a80 0x557d584d62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67910ae640 1 -- 172.21.15.83:0/1662579531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b0a80 msgr2=0x557d584d62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67910ae640 1 --2- 172.21.15.83:0/1662579531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b0a80 0x557d584d62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67910ae640 1 -- 172.21.15.83:0/1662579531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 msgr2=0x557d584dfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67910ae640 1 --2- 172.21.15.83:0/1662579531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584dfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67910ae640 1 -- 172.21.15.83:0/1662579531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d583abc20 con 0x557d584b1470 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.540+0000 7f67920b0640 1 --2- 172.21.15.83:0/1662579531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584dfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.541+0000 7f67910ae640 1 --2- 172.21.15.83:0/1662579531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b1470 0x557d584df7e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f677c0059d0 tx=0x7f677c004bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d4e080166a87b3d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.541+0000 7f67777fe640 1 -- 172.21.15.83:0/1662579531 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c004730 con 0x557d584b1470 2024-03-11T19:52:13.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.541+0000 7f67777fe640 1 -- 172.21.15.83:0/1662579531 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f677c0050a0 con 0x557d584b1470 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.541+0000 7f67777fe640 1 -- 172.21.15.83:0/1662579531 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c025bb0 con 0x557d584b1470 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.542+0000 7f67967ca740 1 -- 172.21.15.83:0/1662579531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b1470 msgr2=0x557d584df7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.542+0000 7f67967ca740 1 --2- 172.21.15.83:0/1662579531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b1470 0x557d584df7e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f677c0059d0 tx=0x7f677c004bb0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.542+0000 7f67967ca740 1 -- 172.21.15.83:0/1662579531 shutdown_connections 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.542+0000 7f67967ca740 1 --2- 172.21.15.83:0/1662579531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b65e0 0x557d584dfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.542+0000 7f67967ca740 1 --2- 172.21.15.83:0/1662579531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b1470 0x557d584df7e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.542+0000 7f67967ca740 1 --2- 172.21.15.83:0/1662579531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b0a80 0x557d584d62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.542+0000 7f67967ca740 1 -- 172.21.15.83:0/1662579531 >> 172.21.15.83:0/1662579531 conn(0x557d584a37e0 msgr2=0x557d584a3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.542+0000 7f67967ca740 1 -- 172.21.15.83:0/1662579531 shutdown_connections 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.542+0000 7f67967ca740 1 -- 172.21.15.83:0/1662579531 wait complete. 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.542+0000 7f67967ca740 1 -- start start 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.542+0000 7f67967ca740 1 Processor -- start 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67967ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b0a80 0x557d58520480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67967ca740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 0x557d585229d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67967ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b65e0 0x557d58524f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67967ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d58525460 con 0x557d584b0a80 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67967ca740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d585255a0 con 0x557d584b1470 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67967ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d585256e0 con 0x557d584b65e0 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67967ca740 10 client.0 _ll_register_callbacks cb 0x7f677c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67918af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b0a80 0x557d58520480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67918af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b0a80 0x557d58520480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37968/0 (socket says 172.21.15.83:37968) 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67918af640 1 -- 172.21.15.83:0/1374537741 learned_addr learned my addr 172.21.15.83:0/1374537741 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67910ae640 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 0x557d585229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67920b0640 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b65e0 0x557d58524f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67918af640 1 -- 172.21.15.83:0/1374537741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b65e0 msgr2=0x557d58524f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67918af640 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b65e0 0x557d58524f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67918af640 1 -- 172.21.15.83:0/1374537741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 msgr2=0x557d585229d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67918af640 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 0x557d585229d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67918af640 1 -- 172.21.15.83:0/1374537741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67840136a0 con 0x557d584b0a80 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67910ae640 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 0x557d585229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67918af640 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b0a80 0x557d58520480 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f67880b91c0 tx=0x7f67880b9130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6dd0434f50d1f15 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67757fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67757fa640 1 -- 172.21.15.83:0/1374537741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67880db070 con 0x557d584b0a80 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67757fa640 1 -- 172.21.15.83:0/1374537741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f67880bd460 con 0x557d584b0a80 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67967ca740 1 -- 172.21.15.83:0/1374537741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d5819c890 con 0x557d584b0a80 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67757fa640 1 -- 172.21.15.83:0/1374537741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67880c5500 con 0x557d584b0a80 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67967ca740 10 client.17802 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.543+0000 7f67967ca740 1 -- 172.21.15.83:0/1374537741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557d5840f6a0 con 0x557d584b0a80 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.544+0000 7f67757fa640 1 -- 172.21.15.83:0/1374537741 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557d5819c890 con 0x557d584b0a80 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.544+0000 7f67757fa640 1 -- 172.21.15.83:0/1374537741 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f67880a4070 con 0x557d584b0a80 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.544+0000 7f67967ca740 10 client.17802 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:13.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.544+0000 7f67967ca740 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.544+0000 7f67967ca740 20 client.17802 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.544+0000 7f67967ca740 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557d58526e00 0x557d585291f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.544+0000 7f67967ca740 4 client.17802 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.544+0000 7f67967ca740 1 -- 172.21.15.83:0/1374537741 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d580bc700 con 0x557d58526e00 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.544+0000 7f67910ae640 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557d58526e00 0x557d585291f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67757fa640 10 client.17802 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67910ae640 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557d58526e00 0x557d585291f0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=292cf53d687e09d0 server_cookie=a06480e9cb08ba0 in_seq=0 out_seq=0 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67757fa640 1 -- 172.21.15.83:0/1374537741 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d580bc700 con 0x557d58526e00 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67757fa640 10 client.17802 handle_command_reply: tid=0 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 1 client.17802 shutdown 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 1 -- 172.21.15.83:0/1374537741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b0a80 msgr2=0x557d58520480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b0a80 0x557d58520480 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f67880b91c0 tx=0x7f67880b9130 comp rx=0 tx=0).stop 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 1 -- 172.21.15.83:0/1374537741 shutdown_connections 2024-03-11T19:52:13.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x557d58526e00 0x557d585291f0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d584b65e0 0x557d58524f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d584b1470 0x557d585229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 1 --2- 172.21.15.83:0/1374537741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557d584b0a80 0x557d58520480 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67757fa640 0 client.17802 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:13.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 1 -- 172.21.15.83:0/1374537741 >> 172.21.15.83:0/1374537741 conn(0x557d584a37e0 msgr2=0x557d584d62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 1 -- 172.21.15.83:0/1374537741 shutdown_connections 2024-03-11T19:52:13.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 1 -- 172.21.15.83:0/1374537741 wait complete. 2024-03-11T19:52:13.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.545+0000 7f67967ca740 20 client.17802 trim_cache size 0 max 16384 2024-03-11T19:52:13.556 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:52:13.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.546+0000 7f6793b13640 1 -- 172.21.15.83:0/3213180310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f67780a1a30 msgr2=0x7f67780a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.546+0000 7f6793b13640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f67780a1a30 0x7f67780a3ef0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x557d584b1cd0 tx=0x7f6784004ab0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.546+0000 7f6793b13640 1 -- 172.21.15.83:0/3213180310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f678c0ff800 msgr2=0x7f678c197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:13.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.546+0000 7f6793b13640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f678c0ff800 0x7f678c197270 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6788000c00 tx=0x7f6788005e80 comp rx=0 tx=0).stop 2024-03-11T19:52:13.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.546+0000 7f6793b13640 1 -- 172.21.15.83:0/3213180310 shutdown_connections 2024-03-11T19:52:13.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.546+0000 7f6793b13640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f67780a1a30 0x7f67780a3ef0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.546+0000 7f6793b13640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f678c1013b0 0x7f678c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.546+0000 7f6793b13640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c1009f0 0x7f678c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:13.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.546+0000 7f6793b13640 1 --2- 172.21.15.83:0/3213180310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f678c0ff800 0x7f678c197270 secure :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6788000c00 tx=0x7f6788005e80 comp rx=0 tx=0).stop 2024-03-11T19:52:13.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.546+0000 7f6793b13640 1 -- 172.21.15.83:0/3213180310 >> 172.21.15.83:0/3213180310 conn(0x7f678c0fad70 msgr2=0x7f678c108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:13.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.547+0000 7f6793b13640 1 -- 172.21.15.83:0/3213180310 shutdown_connections 2024-03-11T19:52:13.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:13.547+0000 7f6793b13640 1 -- 172.21.15.83:0/3213180310 wait complete. 2024-03-11T19:52:13.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:13 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/840787757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:13.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:13 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/840787757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:14.565 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:52:14.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:52:14.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:52:14.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:14 smithi083 ceph-mon[31864]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.3 MiB/s wr, 556 op/s 2024-03-11T19:52:14.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:14 smithi137 ceph-mon[36220]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.3 MiB/s wr, 556 op/s 2024-03-11T19:52:14.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:14 smithi204 ceph-mon[36739]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.3 MiB/s wr, 556 op/s 2024-03-11T19:52:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.696+0000 7f9251b93640 1 -- 172.21.15.83:0/1437613551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c102dd0 msgr2=0x7f924c109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.696+0000 7f9251b93640 1 --2- 172.21.15.83:0/1437613551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c102dd0 0x7f924c109660 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9244009ca0 tx=0x7f924402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.696+0000 7f9251b93640 1 -- 172.21.15.83:0/1437613551 shutdown_connections 2024-03-11T19:52:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.696+0000 7f9251b93640 1 --2- 172.21.15.83:0/1437613551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c102dd0 0x7f924c109660 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.696+0000 7f9251b93640 1 --2- 172.21.15.83:0/1437613551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f924c102410 0x7f924c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.696+0000 7f9251b93640 1 --2- 172.21.15.83:0/1437613551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c0ff440 0x7f924c0ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.696+0000 7f9251b93640 1 -- 172.21.15.83:0/1437613551 >> 172.21.15.83:0/1437613551 conn(0x7f924c0fad70 msgr2=0x7f924c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.696+0000 7f9251b93640 1 -- 172.21.15.83:0/1437613551 shutdown_connections 2024-03-11T19:52:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.696+0000 7f9251b93640 1 -- 172.21.15.83:0/1437613551 wait complete. 2024-03-11T19:52:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f9251b93640 1 Processor -- start 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f9251b93640 1 -- start start 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f9251b93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0ff440 0x7f924c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f9251b93640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f924c102410 0x7f924c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f9251b93640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c102dd0 0x7f924c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f9251b93640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c19ecc0 con 0x7f924c0ff440 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f9251b93640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c19ee30 con 0x7f924c102410 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f9251b93640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c19eff0 con 0x7f924c102dd0 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f924b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0ff440 0x7f924c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f924b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0ff440 0x7f924c1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37976/0 (socket says 172.21.15.83:37976) 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f924b7fe640 1 -- 172.21.15.83:0/131471100 learned_addr learned my addr 172.21.15.83:0/131471100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f924bfff640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c102dd0 0x7f924c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f924affd640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f924c102410 0x7f924c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f924b7fe640 1 -- 172.21.15.83:0/131471100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c102dd0 msgr2=0x7f924c19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f924b7fe640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c102dd0 0x7f924c19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f924b7fe640 1 -- 172.21.15.83:0/131471100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f924c102410 msgr2=0x7f924c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f924b7fe640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f924c102410 0x7f924c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.697+0000 7f924b7fe640 1 -- 172.21.15.83:0/131471100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9244009900 con 0x7f924c0ff440 2024-03-11T19:52:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.698+0000 7f924b7fe640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0ff440 0x7f924c1971d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f923c00e9c0 tx=0x7f923c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.698+0000 7f9248ff9640 1 -- 172.21.15.83:0/131471100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c009700 con 0x7f924c0ff440 2024-03-11T19:52:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.698+0000 7f9248ff9640 1 -- 172.21.15.83:0/131471100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f923c00cd80 con 0x7f924c0ff440 2024-03-11T19:52:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.698+0000 7f9251b93640 1 -- 172.21.15.83:0/131471100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924c19f2d0 con 0x7f924c0ff440 2024-03-11T19:52:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.698+0000 7f9248ff9640 1 -- 172.21.15.83:0/131471100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c010430 con 0x7f924c0ff440 2024-03-11T19:52:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.698+0000 7f9251b93640 1 -- 172.21.15.83:0/131471100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c19f7e0 con 0x7f924c0ff440 2024-03-11T19:52:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.701+0000 7f9248ff9640 1 -- 172.21.15.83:0/131471100 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f923c003680 con 0x7f924c0ff440 2024-03-11T19:52:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f925476f740 1 Processor -- start 2024-03-11T19:52:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f925476f740 1 -- start start 2024-03-11T19:52:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f925476f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 0x55f09c1498e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f925476f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c14ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f925476f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 0x55f09c1518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f925476f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09c0b8f90 con 0x55f09c14b050 2024-03-11T19:52:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f925476f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09c0b64c0 con 0x55f09c1494c0 2024-03-11T19:52:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f925476f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09c0b1ca0 con 0x55f09c14a690 2024-03-11T19:52:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f9248ff9640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f92340a1a30 0x7f92340a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f924affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 0x55f09c1518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f924bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 0x55f09c1498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f924bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 0x55f09c1498e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39270/0 (socket says 172.21.15.83:39270) 2024-03-11T19:52:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f924bfff640 1 -- 172.21.15.83:0/435062366 learned_addr learned my addr 172.21.15.83:0/435062366 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f924bfff640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f92340a1a30 0x7f92340a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f9248ff9640 1 -- 172.21.15.83:0/131471100 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f923c014070 con 0x7f924c0ff440 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f924b7fe640 1 --2- 172.21.15.83:0/435062366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c14ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f924bfff640 1 -- 172.21.15.83:0/435062366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 msgr2=0x55f09c14ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f924bfff640 1 --2- 172.21.15.83:0/435062366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c14ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f924bfff640 1 -- 172.21.15.83:0/435062366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 msgr2=0x55f09c1518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f924bfff640 1 --2- 172.21.15.83:0/435062366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 0x55f09c1518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.702+0000 7f924bfff640 1 -- 172.21.15.83:0/435062366 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f09c0740b0 con 0x55f09c1494c0 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.703+0000 7f924bfff640 1 --2- 172.21.15.83:0/435062366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 0x55f09c1498e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9244009ca0 tx=0x7f9244032ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=694b049dd30da246 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.703+0000 7f924bfff640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f92340a1a30 0x7f92340a3ef0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9244036000 tx=0x7f924403c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.703+0000 7f9231ffb640 1 -- 172.21.15.83:0/435062366 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244004b40 con 0x55f09c1494c0 2024-03-11T19:52:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.703+0000 7f9231ffb640 1 -- 172.21.15.83:0/435062366 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9244003e50 con 0x55f09c1494c0 2024-03-11T19:52:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.703+0000 7f9231ffb640 1 -- 172.21.15.83:0/435062366 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244004280 con 0x55f09c1494c0 2024-03-11T19:52:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.704+0000 7f925476f740 1 -- 172.21.15.83:0/435062366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 msgr2=0x55f09c1498e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.704+0000 7f925476f740 1 --2- 172.21.15.83:0/435062366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 0x55f09c1498e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9244009ca0 tx=0x7f9244032ae0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.704+0000 7f925476f740 1 -- 172.21.15.83:0/435062366 shutdown_connections 2024-03-11T19:52:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.704+0000 7f925476f740 1 --2- 172.21.15.83:0/435062366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 0x55f09c1518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.704+0000 7f925476f740 1 --2- 172.21.15.83:0/435062366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c14ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.704+0000 7f925476f740 1 --2- 172.21.15.83:0/435062366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 0x55f09c1498e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.704+0000 7f925476f740 1 -- 172.21.15.83:0/435062366 >> 172.21.15.83:0/435062366 conn(0x55f09c1446a0 msgr2=0x55f09c144aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.704+0000 7f925476f740 1 -- 172.21.15.83:0/435062366 shutdown_connections 2024-03-11T19:52:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.704+0000 7f925476f740 1 -- 172.21.15.83:0/435062366 wait complete. 2024-03-11T19:52:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.704+0000 7f925476f740 1 -- start start 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.704+0000 7f925476f740 1 Processor -- start 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f925476f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c1494c0 0x55f09c1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f925476f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f925476f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c14b050 0x55f09c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f925476f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09c056f40 con 0x55f09c1494c0 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f925476f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09bd854b0 con 0x55f09c14b050 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f925476f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09c037670 con 0x55f09c14a690 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f925476f740 10 client.0 _ll_register_callbacks cb 0x55f09c0a4f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f924b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c1494c0 0x55f09c1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f924b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c1494c0 0x55f09c1536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37986/0 (socket says 172.21.15.83:37986) 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f924b7fe640 1 -- 172.21.15.83:0/1412805787 learned_addr learned my addr 172.21.15.83:0/1412805787 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f924affd640 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f924bfff640 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c14b050 0x55f09c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f924b7fe640 1 -- 172.21.15.83:0/1412805787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 msgr2=0x55f09c153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f924b7fe640 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f924b7fe640 1 -- 172.21.15.83:0/1412805787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c14b050 msgr2=0x55f09c15ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f924b7fe640 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c14b050 0x55f09c15ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f924b7fe640 1 -- 172.21.15.83:0/1412805787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c109ea0 con 0x55f09c1494c0 2024-03-11T19:52:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f924b7fe640 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c1494c0 0x55f09c1536b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f923c0058c0 tx=0x7f923c09ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec3486b52845ff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f922affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:52:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f922affd640 1 -- 172.21.15.83:0/1412805787 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c09e890 con 0x55f09c1494c0 2024-03-11T19:52:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f925476f740 1 -- 172.21.15.83:0/1412805787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f09c08bf40 con 0x55f09c1494c0 2024-03-11T19:52:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f922affd640 1 -- 172.21.15.83:0/1412805787 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f923c0ba0c0 con 0x55f09c1494c0 2024-03-11T19:52:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f922affd640 1 -- 172.21.15.83:0/1412805787 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f923c0ba280 con 0x55f09c1494c0 2024-03-11T19:52:14.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.705+0000 7f925476f740 10 client.17814 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:14.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.706+0000 7f925476f740 1 -- 172.21.15.83:0/1412805787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f09c0b0320 con 0x55f09c1494c0 2024-03-11T19:52:14.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.706+0000 7f922affd640 1 -- 172.21.15.83:0/1412805787 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f923c07d020 con 0x55f09c1494c0 2024-03-11T19:52:14.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.706+0000 7f922affd640 1 -- 172.21.15.83:0/1412805787 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f923c096040 con 0x55f09c1494c0 2024-03-11T19:52:14.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.706+0000 7f925476f740 10 client.17814 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:14.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.706+0000 7f925476f740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:14.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.706+0000 7f925476f740 20 client.17814 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:14.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.706+0000 7f925476f740 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f09c15f820 0x55f09c161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.706+0000 7f925476f740 4 client.17814 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.706+0000 7f925476f740 1 -- 172.21.15.83:0/1412805787 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f09c0a8310 con 0x55f09c15f820 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.706+0000 7f924affd640 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f09c15f820 0x55f09c161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.707+0000 7f924affd640 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f09c15f820 0x55f09c161c10 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c57a211894532e0 server_cookie=22094145931414cd in_seq=0 out_seq=0 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.707+0000 7f922affd640 10 client.17814 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f922affd640 1 -- 172.21.15.83:0/1412805787 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f09c0a8310 con 0x55f09c15f820 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f922affd640 10 client.17814 handle_command_reply: tid=0 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 1 client.17814 shutdown 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 1 -- 172.21.15.83:0/1412805787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c1494c0 msgr2=0x55f09c1536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c1494c0 0x55f09c1536b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f923c0058c0 tx=0x7f923c09ae40 comp rx=0 tx=0).stop 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 1 -- 172.21.15.83:0/1412805787 shutdown_connections 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f09c15f820 0x55f09c161c10 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c14b050 0x55f09c15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 1 --2- 172.21.15.83:0/1412805787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c1494c0 0x55f09c1536b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f922affd640 0 client.17814 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:14.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 1 -- 172.21.15.83:0/1412805787 >> 172.21.15.83:0/1412805787 conn(0x55f09c1446a0 msgr2=0x55f09c1438e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 1 -- 172.21.15.83:0/1412805787 shutdown_connections 2024-03-11T19:52:14.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 1 -- 172.21.15.83:0/1412805787 wait complete. 2024-03-11T19:52:14.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.708+0000 7f925476f740 20 client.17814 trim_cache size 0 max 16384 2024-03-11T19:52:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f925476f740 1 Processor -- start 2024-03-11T19:52:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f925476f740 1 -- start start 2024-03-11T19:52:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f925476f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c1494c0 0x55f09c1532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f925476f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14a690 0x55f09c15abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f925476f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c14b050 0x55f09c153800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f925476f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09c140180 con 0x55f09c14a690 2024-03-11T19:52:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f925476f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09c14a110 con 0x55f09c14b050 2024-03-11T19:52:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f925476f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09c15de50 con 0x55f09c1494c0 2024-03-11T19:52:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f924affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14a690 0x55f09c15abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f924affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14a690 0x55f09c15abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37990/0 (socket says 172.21.15.83:37990) 2024-03-11T19:52:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f924affd640 1 -- 172.21.15.83:0/3427180388 learned_addr learned my addr 172.21.15.83:0/3427180388 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f924b7fe640 1 --2- 172.21.15.83:0/3427180388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c1494c0 0x55f09c1532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f924bfff640 1 --2- 172.21.15.83:0/3427180388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c14b050 0x55f09c153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f924affd640 1 -- 172.21.15.83:0/3427180388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c1494c0 msgr2=0x55f09c1532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f924affd640 1 --2- 172.21.15.83:0/3427180388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c1494c0 0x55f09c1532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f924affd640 1 -- 172.21.15.83:0/3427180388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c14b050 msgr2=0x55f09c153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f924affd640 1 --2- 172.21.15.83:0/3427180388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c14b050 0x55f09c153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.727+0000 7f924affd640 1 -- 172.21.15.83:0/3427180388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f09c04cc00 con 0x55f09c14a690 2024-03-11T19:52:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f924affd640 1 --2- 172.21.15.83:0/3427180388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14a690 0x55f09c15abc0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9240000c00 tx=0x7f9240005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c728579b36810094 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f92317fa640 1 -- 172.21.15.83:0/3427180388 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9240017030 con 0x55f09c14a690 2024-03-11T19:52:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f92317fa640 1 -- 172.21.15.83:0/3427180388 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9240018040 con 0x55f09c14a690 2024-03-11T19:52:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f92317fa640 1 -- 172.21.15.83:0/3427180388 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924000fad0 con 0x55f09c14a690 2024-03-11T19:52:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f925476f740 1 -- 172.21.15.83:0/3427180388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14a690 msgr2=0x55f09c15abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f925476f740 1 --2- 172.21.15.83:0/3427180388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14a690 0x55f09c15abc0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9240000c00 tx=0x7f9240005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f925476f740 1 -- 172.21.15.83:0/3427180388 shutdown_connections 2024-03-11T19:52:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f925476f740 1 --2- 172.21.15.83:0/3427180388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c14b050 0x55f09c153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f925476f740 1 --2- 172.21.15.83:0/3427180388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14a690 0x55f09c15abc0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f925476f740 1 --2- 172.21.15.83:0/3427180388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c1494c0 0x55f09c1532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f925476f740 1 -- 172.21.15.83:0/3427180388 >> 172.21.15.83:0/3427180388 conn(0x55f09c1446a0 msgr2=0x55f09c14e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f925476f740 1 -- 172.21.15.83:0/3427180388 shutdown_connections 2024-03-11T19:52:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.728+0000 7f925476f740 1 -- 172.21.15.83:0/3427180388 wait complete. 2024-03-11T19:52:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f925476f740 1 -- start start 2024-03-11T19:52:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f925476f740 1 Processor -- start 2024-03-11T19:52:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f925476f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 0x55f09c159b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f925476f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f925476f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 0x55f09c1b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f925476f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09c1b6210 con 0x55f09c14b050 2024-03-11T19:52:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f925476f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09c1b6350 con 0x55f09c1494c0 2024-03-11T19:52:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f925476f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09c1b6490 con 0x55f09c14a690 2024-03-11T19:52:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f925476f740 10 client.0 _ll_register_callbacks cb 0x7f9240005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f924bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 0x55f09c1b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f924bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 0x55f09c1b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38002/0 (socket says 172.21.15.83:38002) 2024-03-11T19:52:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f924bfff640 1 -- 172.21.15.83:0/61037978 learned_addr learned my addr 172.21.15.83:0/61037978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f924b7fe640 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 0x55f09c159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f924affd640 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f924bfff640 1 -- 172.21.15.83:0/61037978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 msgr2=0x55f09c15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f924bfff640 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.729+0000 7f924bfff640 1 -- 172.21.15.83:0/61037978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 msgr2=0x55f09c159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f924bfff640 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 0x55f09c159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f924bfff640 1 -- 172.21.15.83:0/61037978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9240010430 con 0x55f09c14b050 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f924bfff640 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 0x55f09c1b5cd0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f924402c630 tx=0x7f92440328d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9852d98c6d482fe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f922affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f922affd640 1 -- 172.21.15.83:0/61037978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244061070 con 0x55f09c14b050 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f922affd640 1 -- 172.21.15.83:0/61037978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9244032e30 con 0x55f09c14b050 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f925476f740 1 -- 172.21.15.83:0/61037978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f09c0d7900 con 0x55f09c14b050 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f922affd640 1 -- 172.21.15.83:0/61037978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9244004540 con 0x55f09c14b050 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f925476f740 10 client.17820 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f925476f740 1 -- 172.21.15.83:0/61037978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f09c072910 con 0x55f09c14b050 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f922affd640 1 -- 172.21.15.83:0/61037978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f924406c080 con 0x55f09c14b050 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f922affd640 1 -- 172.21.15.83:0/61037978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9244064030 con 0x55f09c14b050 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f925476f740 10 client.17820 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f925476f740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f925476f740 20 client.17820 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f925476f740 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f09c1b7d10 0x55f09c1ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f925476f740 4 client.17820 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.730+0000 7f925476f740 1 -- 172.21.15.83:0/61037978 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f09bdb1860 con 0x55f09c1b7d10 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.731+0000 7f924b7fe640 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f09c1b7d10 0x55f09c1ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.731+0000 7f924b7fe640 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f09c1b7d10 0x55f09c1ba130 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9e2c0dca46544b server_cookie=84d0fae4d4cdc754 in_seq=0 out_seq=0 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.731+0000 7f922affd640 10 client.17820 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f922affd640 1 -- 172.21.15.83:0/61037978 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f09bdb1860 con 0x55f09c1b7d10 2024-03-11T19:52:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f922affd640 10 client.17820 handle_command_reply: tid=0 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 1 client.17820 shutdown 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 1 -- 172.21.15.83:0/61037978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 msgr2=0x55f09c1b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 0x55f09c1b5cd0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f924402c630 tx=0x7f92440328d0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 1 -- 172.21.15.83:0/61037978 shutdown_connections 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f09c1b7d10 0x55f09c1ba130 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f09c14b050 0x55f09c1b5cd0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f922affd640 0 client.17820 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f09c14a690 0x55f09c15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 1 --2- 172.21.15.83:0/61037978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f09c1494c0 0x55f09c159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 1 -- 172.21.15.83:0/61037978 >> 172.21.15.83:0/61037978 conn(0x55f09c1446a0 msgr2=0x55f09c1532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 1 -- 172.21.15.83:0/61037978 shutdown_connections 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 1 -- 172.21.15.83:0/61037978 wait complete. 2024-03-11T19:52:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.733+0000 7f925476f740 20 client.17820 trim_cache size 0 max 16384 2024-03-11T19:52:14.744 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:52:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:52:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "d6a31ab8-062a-44d4-8201-4b4fcc8bd568", 2024-03-11T19:52:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:52:14.744 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:52:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 -- 172.21.15.83:0/131471100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f92340a1a30 msgr2=0x7f92340a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f92340a1a30 0x7f92340a3ef0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9244036000 tx=0x7f924403c000 comp rx=0 tx=0).stop 2024-03-11T19:52:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 -- 172.21.15.83:0/131471100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0ff440 msgr2=0x7f924c1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0ff440 0x7f924c1971d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f923c00e9c0 tx=0x7f923c00ee80 comp rx=0 tx=0).stop 2024-03-11T19:52:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 -- 172.21.15.83:0/131471100 shutdown_connections 2024-03-11T19:52:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f92340a1a30 0x7f92340a3ef0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f924c102dd0 0x7f924c19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f924c102410 0x7f924c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 --2- 172.21.15.83:0/131471100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f924c0ff440 0x7f924c1971d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 -- 172.21.15.83:0/131471100 >> 172.21.15.83:0/131471100 conn(0x7f924c0fad70 msgr2=0x7f924c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 -- 172.21.15.83:0/131471100 shutdown_connections 2024-03-11T19:52:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.734+0000 7f9251b93640 1 -- 172.21.15.83:0/131471100 wait complete. 2024-03-11T19:52:14.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:52:14.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:52:14.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.918+0000 7fc869b77640 1 -- 172.21.15.83:0/1213075087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8641077c0 msgr2=0x7fc864107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.918+0000 7fc869b77640 1 --2- 172.21.15.83:0/1213075087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8641077c0 0x7fc864107be0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009ca0 tx=0x7fc85c02c100 comp rx=0 tx=0).stop 2024-03-11T19:52:14.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.918+0000 7fc869b77640 1 -- 172.21.15.83:0/1213075087 shutdown_connections 2024-03-11T19:52:14.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.918+0000 7fc869b77640 1 --2- 172.21.15.83:0/1213075087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc864109370 0x7fc86410fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.918+0000 7fc869b77640 1 --2- 172.21.15.83:0/1213075087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8641089b0 0x7fc864108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.918+0000 7fc869b77640 1 --2- 172.21.15.83:0/1213075087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8641077c0 0x7fc864107be0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.918+0000 7fc869b77640 1 -- 172.21.15.83:0/1213075087 >> 172.21.15.83:0/1213075087 conn(0x7fc864076770 msgr2=0x7fc864078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.918+0000 7fc869b77640 1 -- 172.21.15.83:0/1213075087 shutdown_connections 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.918+0000 7fc869b77640 1 -- 172.21.15.83:0/1213075087 wait complete. 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.919+0000 7fc869b77640 1 Processor -- start 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.919+0000 7fc869b77640 1 -- start start 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.919+0000 7fc869b77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8641077c0 0x7fc86419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.919+0000 7fc869b77640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8641089b0 0x7fc86419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.919+0000 7fc869b77640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc864109370 0x7fc8641a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.919+0000 7fc869b77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8641a5360 con 0x7fc8641077c0 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.919+0000 7fc869b77640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8641a54d0 con 0x7fc8641089b0 2024-03-11T19:52:14.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.919+0000 7fc869b77640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8641a5690 con 0x7fc864109370 2024-03-11T19:52:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.920+0000 7fc8637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8641077c0 0x7fc86419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.920+0000 7fc8637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8641077c0 0x7fc86419d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38014/0 (socket says 172.21.15.83:38014) 2024-03-11T19:52:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.920+0000 7fc8637fe640 1 -- 172.21.15.83:0/3586653549 learned_addr learned my addr 172.21.15.83:0/3586653549 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.920+0000 7fc863fff640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc864109370 0x7fc8641a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.920+0000 7fc862ffd640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8641089b0 0x7fc86419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.920+0000 7fc863fff640 1 -- 172.21.15.83:0/3586653549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8641089b0 msgr2=0x7fc86419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.920+0000 7fc863fff640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8641089b0 0x7fc86419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.920+0000 7fc863fff640 1 -- 172.21.15.83:0/3586653549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8641077c0 msgr2=0x7fc86419d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.920+0000 7fc863fff640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8641077c0 0x7fc86419d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.920+0000 7fc863fff640 1 -- 172.21.15.83:0/3586653549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc854009490 con 0x7fc864109370 2024-03-11T19:52:14.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.920+0000 7fc863fff640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc864109370 0x7fc8641a4e20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc85400e970 tx=0x7fc85400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.921+0000 7fc860ff9640 1 -- 172.21.15.83:0/3586653549 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85400ccc0 con 0x7fc864109370 2024-03-11T19:52:14.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.921+0000 7fc860ff9640 1 -- 172.21.15.83:0/3586653549 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc85400ce20 con 0x7fc864109370 2024-03-11T19:52:14.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.921+0000 7fc869b77640 1 -- 172.21.15.83:0/3586653549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c009900 con 0x7fc864109370 2024-03-11T19:52:14.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.921+0000 7fc860ff9640 1 -- 172.21.15.83:0/3586653549 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc854010430 con 0x7fc864109370 2024-03-11T19:52:14.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.921+0000 7fc869b77640 1 -- 172.21.15.83:0/3586653549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8641a5cf0 con 0x7fc864109370 2024-03-11T19:52:14.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.923+0000 7fc860ff9640 1 -- 172.21.15.83:0/3586653549 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc854004020 con 0x7fc864109370 2024-03-11T19:52:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.923+0000 7fc86c826740 1 Processor -- start 2024-03-11T19:52:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc86c826740 1 -- start start 2024-03-11T19:52:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc86c826740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 0x5594052aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc86c826740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc86c826740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052afe30 0x5594052b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc86c826740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55940521a280 con 0x5594052aacc0 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc86c826740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594052177b0 con 0x5594052afe30 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc86c826740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559405212f90 con 0x5594052aa2d0 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc860ff9640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc84c0a1ab0 0x7fc84c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc8637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc863fff640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc84c0a1ab0 0x7fc84c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc8637fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38018/0 (socket says 172.21.15.83:38018) 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc8637fe640 1 -- 172.21.15.83:0/2202140523 learned_addr learned my addr 172.21.15.83:0/2202140523 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc862ffd640 1 --2- 172.21.15.83:0/2202140523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 0x5594052aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc862ffd640 1 --2- 172.21.15.83:0/2202140523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052afe30 0x5594052b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc862ffd640 1 -- 172.21.15.83:0/2202140523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052afe30 msgr2=0x5594052b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc862ffd640 1 --2- 172.21.15.83:0/2202140523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052afe30 0x5594052b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc862ffd640 1 -- 172.21.15.83:0/2202140523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 msgr2=0x5594052af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc862ffd640 1 --2- 172.21.15.83:0/2202140523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc862ffd640 1 -- 172.21.15.83:0/2202140523 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594051d1570 con 0x5594052aa2d0 2024-03-11T19:52:14.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc860ff9640 1 -- 172.21.15.83:0/3586653549 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc854014070 con 0x7fc864109370 2024-03-11T19:52:14.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.924+0000 7fc862ffd640 1 --2- 172.21.15.83:0/2202140523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 0x5594052aa6f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc86419ed80 tx=0x7fc858004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4608c702bc611ca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.925+0000 7fc863fff640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc84c0a1ab0 0x7fc84c0a3f70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc854096350 tx=0x7fc8540961e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.925+0000 7fc849ffb640 1 -- 172.21.15.83:0/2202140523 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858004050 con 0x5594052aa2d0 2024-03-11T19:52:14.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.925+0000 7fc849ffb640 1 -- 172.21.15.83:0/2202140523 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc85800fca0 con 0x5594052aa2d0 2024-03-11T19:52:14.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.925+0000 7fc849ffb640 1 -- 172.21.15.83:0/2202140523 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858003950 con 0x5594052aa2d0 2024-03-11T19:52:14.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.926+0000 7fc86c826740 1 -- 172.21.15.83:0/2202140523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 msgr2=0x5594052aa6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.926+0000 7fc86c826740 1 --2- 172.21.15.83:0/2202140523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 0x5594052aa6f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc86419ed80 tx=0x7fc858004730 comp rx=0 tx=0).stop 2024-03-11T19:52:14.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.927+0000 7fc86c826740 1 -- 172.21.15.83:0/2202140523 shutdown_connections 2024-03-11T19:52:14.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.927+0000 7fc86c826740 1 --2- 172.21.15.83:0/2202140523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052afe30 0x5594052b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.927+0000 7fc86c826740 1 --2- 172.21.15.83:0/2202140523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.927+0000 7fc86c826740 1 --2- 172.21.15.83:0/2202140523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 0x5594052aa6f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.927+0000 7fc86c826740 1 -- 172.21.15.83:0/2202140523 >> 172.21.15.83:0/2202140523 conn(0x5594052a57f0 msgr2=0x5594052a5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.928+0000 7fc86c826740 1 -- 172.21.15.83:0/2202140523 shutdown_connections 2024-03-11T19:52:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.928+0000 7fc86c826740 1 -- 172.21.15.83:0/2202140523 wait complete. 2024-03-11T19:52:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.928+0000 7fc86c826740 1 -- start start 2024-03-11T19:52:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc86c826740 1 Processor -- start 2024-03-11T19:52:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc86c826740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 0x5594052b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc86c826740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aacc0 0x5594052b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc86c826740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052afe30 0x5594052bb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc86c826740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x559404ee6430 con 0x5594052afe30 2024-03-11T19:52:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc86c826740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559404ea2e00 con 0x5594052aa2d0 2024-03-11T19:52:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc86c826740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559405197fd0 con 0x5594052aacc0 2024-03-11T19:52:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc86c826740 10 client.0 _ll_register_callbacks cb 0x5594051edfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc862ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052afe30 0x5594052bb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc862ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052afe30 0x5594052bb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38026/0 (socket says 172.21.15.83:38026) 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc862ffd640 1 -- 172.21.15.83:0/2337526148 learned_addr learned my addr 172.21.15.83:0/2337526148 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc8637fe640 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aacc0 0x5594052b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.929+0000 7fc862ffd640 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 0x5594052b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc862ffd640 1 -- 172.21.15.83:0/2337526148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aacc0 msgr2=0x5594052b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc862ffd640 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aacc0 0x5594052b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc862ffd640 1 -- 172.21.15.83:0/2337526148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 msgr2=0x5594052b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc862ffd640 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 0x5594052b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc862ffd640 1 -- 172.21.15.83:0/2337526148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc858019040 con 0x5594052afe30 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc8637fe640 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aacc0 0x5594052b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc862ffd640 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052afe30 0x5594052bb840 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc8580097a0 tx=0x7fc8580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae5de9e476324d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc842ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:52:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc842ffd640 1 -- 172.21.15.83:0/2337526148 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85800f8f0 con 0x5594052afe30 2024-03-11T19:52:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc842ffd640 1 -- 172.21.15.83:0/2337526148 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc85800fa50 con 0x5594052afe30 2024-03-11T19:52:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc86c826740 1 -- 172.21.15.83:0/2337526148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559404f127e0 con 0x5594052afe30 2024-03-11T19:52:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc842ffd640 1 -- 172.21.15.83:0/2337526148 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc858005070 con 0x5594052afe30 2024-03-11T19:52:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc86c826740 10 client.17832 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.930+0000 7fc86c826740 1 -- 172.21.15.83:0/2337526148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559405211610 con 0x5594052afe30 2024-03-11T19:52:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.931+0000 7fc842ffd640 1 -- 172.21.15.83:0/2337526148 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559404f127e0 con 0x5594052afe30 2024-03-11T19:52:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.931+0000 7fc842ffd640 1 -- 172.21.15.83:0/2337526148 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc85801d070 con 0x5594052afe30 2024-03-11T19:52:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.931+0000 7fc86c826740 10 client.17832 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.931+0000 7fc86c826740 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.932+0000 7fc86c826740 20 client.17832 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.932+0000 7fc86c826740 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5594052c0250 0x5594052c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.932+0000 7fc86c826740 4 client.17832 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.932+0000 7fc86c826740 1 -- 172.21.15.83:0/2337526148 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559404f4f720 con 0x5594052c0250 2024-03-11T19:52:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.932+0000 7fc8637fe640 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5594052c0250 0x5594052c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.932+0000 7fc8637fe640 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5594052c0250 0x5594052c2640 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc582f00118da608 server_cookie=fdebe0ef9a4dabf9 in_seq=0 out_seq=0 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.932+0000 7fc842ffd640 10 client.17832 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.933+0000 7fc842ffd640 1 -- 172.21.15.83:0/2337526148 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559404f4f720 con 0x5594052c0250 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.933+0000 7fc842ffd640 10 client.17832 handle_command_reply: tid=0 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.933+0000 7fc86c826740 1 client.17832 shutdown 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc86c826740 1 -- 172.21.15.83:0/2337526148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052afe30 msgr2=0x5594052bb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc86c826740 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052afe30 0x5594052bb840 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc8580097a0 tx=0x7fc8580046c0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc86c826740 1 -- 172.21.15.83:0/2337526148 shutdown_connections 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc86c826740 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5594052c0250 0x5594052c2640 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc86c826740 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052afe30 0x5594052bb840 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc842ffd640 0 client.17832 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc86c826740 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aacc0 0x5594052b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc86c826740 1 --2- 172.21.15.83:0/2337526148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 0x5594052b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc86c826740 1 -- 172.21.15.83:0/2337526148 >> 172.21.15.83:0/2337526148 conn(0x5594052a57f0 msgr2=0x5594052a4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc86c826740 1 -- 172.21.15.83:0/2337526148 shutdown_connections 2024-03-11T19:52:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc86c826740 1 -- 172.21.15.83:0/2337526148 wait complete. 2024-03-11T19:52:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.934+0000 7fc86c826740 20 client.17832 trim_cache size 0 max 16384 2024-03-11T19:52:14.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc86c826740 1 Processor -- start 2024-03-11T19:52:14.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc86c826740 1 -- start start 2024-03-11T19:52:14.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc86c826740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052a57f0 0x5594052b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc86c826740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 0x5594052b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc86c826740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc86c826740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594052b5330 con 0x5594052aacc0 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc86c826740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559404f4f720 con 0x5594052a57f0 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc86c826740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559404f127e0 con 0x5594052aa2d0 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc862ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc8637fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 0x5594052b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc862ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052a57f0 0x5594052b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc8637fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 0x5594052b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:43582/0 (socket says 172.21.15.83:43582) 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc8637fe640 1 -- 172.21.15.83:0/3624188500 learned_addr learned my addr 172.21.15.83:0/3624188500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc8637fe640 1 -- 172.21.15.83:0/3624188500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052a57f0 msgr2=0x5594052b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc8637fe640 1 --2- 172.21.15.83:0/3624188500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052a57f0 0x5594052b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc8637fe640 1 -- 172.21.15.83:0/3624188500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 msgr2=0x5594052bb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc8637fe640 1 --2- 172.21.15.83:0/3624188500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052bb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc8637fe640 1 -- 172.21.15.83:0/3624188500 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594052b33c0 con 0x5594052aa2d0 2024-03-11T19:52:14.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.955+0000 7fc862ffd640 1 --2- 172.21.15.83:0/3624188500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052bb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:14.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.956+0000 7fc8637fe640 1 --2- 172.21.15.83:0/3624188500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 0x5594052b4800 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5594052ab730 tx=0x7fc85c0326d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b08930293b72061 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.956+0000 7fc8497fa640 1 -- 172.21.15.83:0/3624188500 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c045040 con 0x5594052aa2d0 2024-03-11T19:52:14.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.956+0000 7fc8497fa640 1 -- 172.21.15.83:0/3624188500 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc85c047040 con 0x5594052aa2d0 2024-03-11T19:52:14.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.956+0000 7fc8497fa640 1 -- 172.21.15.83:0/3624188500 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c04c590 con 0x5594052aa2d0 2024-03-11T19:52:14.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.957+0000 7fc86c826740 1 -- 172.21.15.83:0/3624188500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 msgr2=0x5594052b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.957+0000 7fc86c826740 1 --2- 172.21.15.83:0/3624188500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 0x5594052b4800 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5594052ab730 tx=0x7fc85c0326d0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.961+0000 7fc86c826740 1 -- 172.21.15.83:0/3624188500 shutdown_connections 2024-03-11T19:52:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.961+0000 7fc86c826740 1 --2- 172.21.15.83:0/3624188500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052bb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.961+0000 7fc86c826740 1 --2- 172.21.15.83:0/3624188500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052aa2d0 0x5594052b4800 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.961+0000 7fc86c826740 1 --2- 172.21.15.83:0/3624188500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052a57f0 0x5594052b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.961+0000 7fc86c826740 1 -- 172.21.15.83:0/3624188500 >> 172.21.15.83:0/3624188500 conn(0x5594052af900 msgr2=0x5594052a5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.969+0000 7fc86c826740 1 -- 172.21.15.83:0/3624188500 shutdown_connections 2024-03-11T19:52:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.971+0000 7fc86c826740 1 -- 172.21.15.83:0/3624188500 wait complete. 2024-03-11T19:52:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.971+0000 7fc86c826740 1 -- start start 2024-03-11T19:52:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.971+0000 7fc86c826740 1 Processor -- start 2024-03-11T19:52:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.975+0000 7fc86c826740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052a57f0 0x5594052b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.975+0000 7fc86c826740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 0x5594052ba530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.975+0000 7fc86c826740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052baa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.975+0000 7fc86c826740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594052bbc40 con 0x5594052aacc0 2024-03-11T19:52:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.975+0000 7fc86c826740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559405316710 con 0x5594052aa2d0 2024-03-11T19:52:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.975+0000 7fc86c826740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559405316850 con 0x5594052a57f0 2024-03-11T19:52:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.975+0000 7fc86c826740 10 client.0 _ll_register_callbacks cb 0x7fc858009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.976+0000 7fc8637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 0x5594052ba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.976+0000 7fc8637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 0x5594052ba530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39380/0 (socket says 172.21.15.83:39380) 2024-03-11T19:52:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.976+0000 7fc8637fe640 1 -- 172.21.15.83:0/2037525888 learned_addr learned my addr 172.21.15.83:0/2037525888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:14.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.977+0000 7fc8637fe640 1 -- 172.21.15.83:0/2037525888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052a57f0 msgr2=0x5594052b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.977+0000 7fc8637fe640 1 --2- 172.21.15.83:0/2037525888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052a57f0 0x5594052b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.977+0000 7fc8637fe640 1 -- 172.21.15.83:0/2037525888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 msgr2=0x5594052baa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:14.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.977+0000 7fc8637fe640 1 --2- 172.21.15.83:0/2037525888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052baa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.977+0000 7fc8637fe640 1 -- 172.21.15.83:0/2037525888 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559405211610 con 0x5594052aa2d0 2024-03-11T19:52:14.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.979+0000 7fc8637fe640 1 --2- 172.21.15.83:0/2037525888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 0x5594052ba530 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc85c005e30 tx=0x7fc85c0324f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=247b4ae322f67756 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:14.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.979+0000 7fc842ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:52:14.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.981+0000 7fc842ffd640 1 -- 172.21.15.83:0/2037525888 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c045040 con 0x5594052aa2d0 2024-03-11T19:52:14.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.981+0000 7fc842ffd640 1 -- 172.21.15.83:0/2037525888 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc85c003b70 con 0x5594052aa2d0 2024-03-11T19:52:14.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.982+0000 7fc86c826740 1 -- 172.21.15.83:0/2037525888 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc85800fca0 con 0x5594052aa2d0 2024-03-11T19:52:14.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.982+0000 7fc842ffd640 1 -- 172.21.15.83:0/2037525888 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc85c034440 con 0x5594052aa2d0 2024-03-11T19:52:14.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.985+0000 7fc86c826740 10 client.27287 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:14.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.985+0000 7fc86c826740 1 -- 172.21.15.83:0/2037525888 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fc864110500 con 0x5594052aa2d0 2024-03-11T19:52:14.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.987+0000 7fc842ffd640 1 -- 172.21.15.83:0/2037525888 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc85c066030 con 0x5594052aa2d0 2024-03-11T19:52:14.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.987+0000 7fc842ffd640 1 -- 172.21.15.83:0/2037525888 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc85c0040c0 con 0x5594052aa2d0 2024-03-11T19:52:14.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.987+0000 7fc86c826740 10 client.27287 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:14.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.987+0000 7fc86c826740 10 client.27287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:14.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.987+0000 7fc86c826740 20 client.27287 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:14.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.987+0000 7fc86c826740 1 --2- 172.21.15.83:0/2037525888 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5594052afe30 0x55940531a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:14.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.987+0000 7fc86c826740 4 client.27287 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:52:14.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.987+0000 7fc86c826740 1 -- 172.21.15.83:0/2037525888 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fc85800fca0 con 0x5594052afe30 2024-03-11T19:52:14.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.988+0000 7fc862ffd640 1 --2- 172.21.15.83:0/2037525888 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5594052afe30 0x55940531a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:14.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.989+0000 7fc842ffd640 10 client.27287 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:14.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.989+0000 7fc862ffd640 1 --2- 172.21.15.83:0/2037525888 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5594052afe30 0x55940531a0c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1560ad5aa706a6 server_cookie=177268f152c64403 in_seq=0 out_seq=0 2024-03-11T19:52:14.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.989+0000 7fc842ffd640 1 -- 172.21.15.83:0/2037525888 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x7fc858009c30 con 0x5594052afe30 2024-03-11T19:52:14.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.989+0000 7fc842ffd640 10 client.27287 handle_command_reply: tid=0 2024-03-11T19:52:14.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.989+0000 7fc86c826740 1 client.27287 shutdown 2024-03-11T19:52:14.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.990+0000 7fc86c826740 1 -- 172.21.15.83:0/2037525888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 msgr2=0x5594052ba530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.990+0000 7fc86c826740 1 --2- 172.21.15.83:0/2037525888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 0x5594052ba530 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc85c005e30 tx=0x7fc85c0324f0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.990+0000 7fc86c826740 1 -- 172.21.15.83:0/2037525888 shutdown_connections 2024-03-11T19:52:14.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.990+0000 7fc86c826740 1 --2- 172.21.15.83:0/2037525888 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5594052afe30 0x55940531a0c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.990+0000 7fc86c826740 1 --2- 172.21.15.83:0/2037525888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594052aacc0 0x5594052baa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.990+0000 7fc842ffd640 0 client.27287 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:14.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.990+0000 7fc86c826740 1 --2- 172.21.15.83:0/2037525888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594052aa2d0 0x5594052ba530 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.990+0000 7fc86c826740 1 --2- 172.21.15.83:0/2037525888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594052a57f0 0x5594052b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.990+0000 7fc86c826740 1 -- 172.21.15.83:0/2037525888 >> 172.21.15.83:0/2037525888 conn(0x5594052af900 msgr2=0x5594052abf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:14.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.990+0000 7fc86c826740 1 -- 172.21.15.83:0/2037525888 shutdown_connections 2024-03-11T19:52:14.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.990+0000 7fc86c826740 1 -- 172.21.15.83:0/2037525888 wait complete. 2024-03-11T19:52:14.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.991+0000 7fc86c826740 20 client.27287 trim_cache size 0 max 16384 2024-03-11T19:52:14.996 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:52:14.997 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (86 inodes in the stack)", 2024-03-11T19:52:14.997 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:52:14.997 INFO:teuthology.orchestra.run.smithi083.stdout: "d6a31ab8-062a-44d4-8201-4b4fcc8bd568": { 2024-03-11T19:52:14.997 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:52:14.997 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "d6a31ab8-062a-44d4-8201-4b4fcc8bd568", 2024-03-11T19:52:14.997 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:52:14.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:52:14.997 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:52:14.997 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:52:14.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 -- 172.21.15.83:0/3586653549 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc84c0a1ab0 msgr2=0x7fc84c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc84c0a1ab0 0x7fc84c0a3f70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc854096350 tx=0x7fc8540961e0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 -- 172.21.15.83:0/3586653549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc864109370 msgr2=0x7fc8641a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:14.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc864109370 0x7fc8641a4e20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc85400e970 tx=0x7fc85400ee30 comp rx=0 tx=0).stop 2024-03-11T19:52:14.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 -- 172.21.15.83:0/3586653549 shutdown_connections 2024-03-11T19:52:14.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc84c0a1ab0 0x7fc84c0a3f70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc864109370 0x7fc8641a4e20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8641089b0 0x7fc86419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 --2- 172.21.15.83:0/3586653549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8641077c0 0x7fc86419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:14.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 -- 172.21.15.83:0/3586653549 >> 172.21.15.83:0/3586653549 conn(0x7fc864076770 msgr2=0x7fc86410bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:15.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 -- 172.21.15.83:0/3586653549 shutdown_connections 2024-03-11T19:52:15.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:14.992+0000 7fc869b77640 1 -- 172.21.15.83:0/3586653549 wait complete. 2024-03-11T19:52:15.011 INFO:tasks.cephfs.filesystem:scrub status for tag:d6a31ab8-062a-44d4-8201-4b4fcc8bd568 - {'path': '/', 'tag': 'd6a31ab8-062a-44d4-8201-4b4fcc8bd568', 'options': 'recursive,force'} 2024-03-11T19:52:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:15 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:52:15.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:15 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:15.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:15 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:52:15.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:15 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:15.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:15 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:52:15.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:15 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:16.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:16 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:52:16.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:16 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:16.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:16 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:52:16.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:16 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:52:16.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:16 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:16.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:16 smithi083 ceph-mon[31864]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.8 MiB/s wr, 704 op/s 2024-03-11T19:52:16.570 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:16 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:16.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:16 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:52:16.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:16 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:16.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:16 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:52:16.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:16 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:52:16.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:16 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:16.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:16 smithi137 ceph-mon[36220]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.8 MiB/s wr, 704 op/s 2024-03-11T19:52:16.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:16 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:16 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:52:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:16 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:16 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:52:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:16 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:52:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:16 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:16 smithi204 ceph-mon[36739]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.8 MiB/s wr, 704 op/s 2024-03-11T19:52:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:16 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 780.0 s, 1733.6 tps, lat 57.197 ms stddev 68.392 2024-03-11T19:52:18.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:18 smithi083 ceph-mon[31864]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.3 MiB/s wr, 572 op/s 2024-03-11T19:52:18.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:18 smithi137 ceph-mon[36220]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.3 MiB/s wr, 572 op/s 2024-03-11T19:52:18.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:18 smithi204 ceph-mon[36739]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.3 MiB/s wr, 572 op/s 2024-03-11T19:52:20.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:20 smithi083 ceph-mon[31864]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 766 op/s 2024-03-11T19:52:20.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:20 smithi137 ceph-mon[36220]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 766 op/s 2024-03-11T19:52:20.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:20 smithi204 ceph-mon[36739]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 766 op/s 2024-03-11T19:52:21.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:21 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:52:21.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:21 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:52:21.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:21 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:52:21.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:21 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:52:21.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:21 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:52:21.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:21 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:52:22.497 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 785.0 s, 1356.2 tps, lat 74.526 ms stddev 84.194 2024-03-11T19:52:22.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:22 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:52:22.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:22 smithi083 ceph-mon[31864]: pgmap v743: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 625 op/s 2024-03-11T19:52:22.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:22 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:52:22.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:22 smithi137 ceph-mon[36220]: pgmap v743: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 625 op/s 2024-03-11T19:52:22.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:22 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:52:22.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:22 smithi204 ceph-mon[36739]: pgmap v743: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 625 op/s 2024-03-11T19:52:24.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:24 smithi083 ceph-mon[31864]: pgmap v744: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 MiB/s wr, 627 op/s 2024-03-11T19:52:24.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:24 smithi137 ceph-mon[36220]: pgmap v744: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 MiB/s wr, 627 op/s 2024-03-11T19:52:24.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:24 smithi204 ceph-mon[36739]: pgmap v744: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 MiB/s wr, 627 op/s 2024-03-11T19:52:26.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:26 smithi083 ceph-mon[31864]: pgmap v745: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-03-11T19:52:26.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:26 smithi137 ceph-mon[36220]: pgmap v745: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-03-11T19:52:26.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:26 smithi204 ceph-mon[36739]: pgmap v745: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 783 op/s 2024-03-11T19:52:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 790.0 s, 1629.6 tps, lat 60.996 ms stddev 68.423 2024-03-11T19:52:28.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:28 smithi083 ceph-mon[31864]: pgmap v746: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 672 op/s 2024-03-11T19:52:28.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:28 smithi137 ceph-mon[36220]: pgmap v746: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 672 op/s 2024-03-11T19:52:28.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:28 smithi204 ceph-mon[36739]: pgmap v746: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 672 op/s 2024-03-11T19:52:30.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:30 smithi083 ceph-mon[31864]: pgmap v747: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 816 op/s 2024-03-11T19:52:30.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:30 smithi137 ceph-mon[36220]: pgmap v747: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 816 op/s 2024-03-11T19:52:30.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:30 smithi204 ceph-mon[36739]: pgmap v747: 97 pgs: 97 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 816 op/s 2024-03-11T19:52:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 795.0 s, 2044.4 tps, lat 49.311 ms stddev 58.293 2024-03-11T19:52:32.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:32 smithi083 ceph-mon[31864]: pgmap v748: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 16 MiB/s wr, 642 op/s 2024-03-11T19:52:32.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:32 smithi137 ceph-mon[36220]: pgmap v748: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 16 MiB/s wr, 642 op/s 2024-03-11T19:52:32.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:32 smithi204 ceph-mon[36739]: pgmap v748: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 16 MiB/s wr, 642 op/s 2024-03-11T19:52:34.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:34 smithi083 ceph-mon[31864]: pgmap v749: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 611 op/s 2024-03-11T19:52:34.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:34 smithi137 ceph-mon[36220]: pgmap v749: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 611 op/s 2024-03-11T19:52:34.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:34 smithi204 ceph-mon[36739]: pgmap v749: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 15 MiB/s wr, 611 op/s 2024-03-11T19:52:36.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:36 smithi083 ceph-mon[31864]: pgmap v750: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 17 MiB/s wr, 740 op/s 2024-03-11T19:52:36.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:36 smithi137 ceph-mon[36220]: pgmap v750: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 17 MiB/s wr, 740 op/s 2024-03-11T19:52:36.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:36 smithi204 ceph-mon[36739]: pgmap v750: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 17 MiB/s wr, 740 op/s 2024-03-11T19:52:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 800.0 s, 1525.2 tps, lat 65.674 ms stddev 67.956 2024-03-11T19:52:38.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:38 smithi083 ceph-mon[31864]: pgmap v751: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 16 MiB/s wr, 621 op/s 2024-03-11T19:52:38.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:38 smithi137 ceph-mon[36220]: pgmap v751: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 16 MiB/s wr, 621 op/s 2024-03-11T19:52:38.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:38 smithi204 ceph-mon[36739]: pgmap v751: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 16 MiB/s wr, 621 op/s 2024-03-11T19:52:40.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:40 smithi083 ceph-mon[31864]: pgmap v752: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 16 MiB/s wr, 713 op/s 2024-03-11T19:52:40.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:40 smithi137 ceph-mon[36220]: pgmap v752: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 16 MiB/s wr, 713 op/s 2024-03-11T19:52:40.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:40 smithi204 ceph-mon[36739]: pgmap v752: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 16 MiB/s wr, 713 op/s 2024-03-11T19:52:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 805.0 s, 1533.0 tps, lat 64.466 ms stddev 70.552 2024-03-11T19:52:42.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:42 smithi083 ceph-mon[31864]: pgmap v753: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 14 MiB/s wr, 620 op/s 2024-03-11T19:52:42.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:42 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:52:42.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:42 smithi137 ceph-mon[36220]: pgmap v753: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 14 MiB/s wr, 620 op/s 2024-03-11T19:52:42.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:42 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:52:42.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:42 smithi204 ceph-mon[36739]: pgmap v753: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 14 MiB/s wr, 620 op/s 2024-03-11T19:52:42.658 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:42 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:52:43.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:43 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:52:43.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:43 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:52:43.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:43 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:52:43.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:43 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:52:43.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:43 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:52:43.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:43 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:52:43.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:43 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:52:43.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:43 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:52:43.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:43 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:52:44.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:44 smithi083 ceph-mon[31864]: pgmap v754: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 12 MiB/s wr, 600 op/s 2024-03-11T19:52:44.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:44 smithi137 ceph-mon[36220]: pgmap v754: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 12 MiB/s wr, 600 op/s 2024-03-11T19:52:44.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:44 smithi204 ceph-mon[36739]: pgmap v754: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 12 MiB/s wr, 600 op/s 2024-03-11T19:52:45.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:52:45.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:52:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.146+0000 7f4cf6088640 1 -- 172.21.15.83:0/1649347376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cf00ff4b0 msgr2=0x7f4cf00ff8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.146+0000 7f4cf6088640 1 --2- 172.21.15.83:0/1649347376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cf00ff4b0 0x7f4cf00ff8d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009900 tx=0x7f4ce002bc30 comp rx=0 tx=0).stop 2024-03-11T19:52:45.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.151+0000 7f4cf6088640 1 -- 172.21.15.83:0/1649347376 shutdown_connections 2024-03-11T19:52:45.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.151+0000 7f4cf6088640 1 --2- 172.21.15.83:0/1649347376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cf0102ee0 0x7f4cf0109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.151+0000 7f4cf6088640 1 --2- 172.21.15.83:0/1649347376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf0102520 0x7f4cf01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.151+0000 7f4cf6088640 1 --2- 172.21.15.83:0/1649347376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cf00ff4b0 0x7f4cf00ff8d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.151+0000 7f4cf6088640 1 -- 172.21.15.83:0/1649347376 >> 172.21.15.83:0/1649347376 conn(0x7f4cf00fadc0 msgr2=0x7f4cf00fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.163+0000 7f4cf6088640 1 -- 172.21.15.83:0/1649347376 shutdown_connections 2024-03-11T19:52:45.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.166+0000 7f4cf6088640 1 -- 172.21.15.83:0/1649347376 wait complete. 2024-03-11T19:52:45.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.167+0000 7f4cf6088640 1 Processor -- start 2024-03-11T19:52:45.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.172+0000 7f4cf6088640 1 -- start start 2024-03-11T19:52:45.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.174+0000 7f4cf6088640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cf00ff4b0 0x7f4cf006f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.174+0000 7f4cf6088640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf0102520 0x7f4cf006ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.174+0000 7f4cf6088640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cf0102ee0 0x7f4cf01a7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.174+0000 7f4cf6088640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf0072f50 con 0x7f4cf0102ee0 2024-03-11T19:52:45.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.174+0000 7f4cf6088640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00730c0 con 0x7f4cf00ff4b0 2024-03-11T19:52:45.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.174+0000 7f4cf6088640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf01a79a0 con 0x7f4cf0102520 2024-03-11T19:52:45.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.174+0000 7f4ceffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cf0102ee0 0x7f4cf01a7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.174+0000 7f4ceffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cf0102ee0 0x7f4cf01a7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38472/0 (socket says 172.21.15.83:38472) 2024-03-11T19:52:45.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.174+0000 7f4ceffff640 1 -- 172.21.15.83:0/2024651817 learned_addr learned my addr 172.21.15.83:0/2024651817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.177+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf0102520 0x7f4cf006ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.177+0000 7f4cef7fe640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cf00ff4b0 0x7f4cf006f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.179+0000 7f4ceffff640 1 -- 172.21.15.83:0/2024651817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf0102520 msgr2=0x7f4cf006ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.179+0000 7f4ceffff640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf0102520 0x7f4cf006ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.179+0000 7f4ceffff640 1 -- 172.21.15.83:0/2024651817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cf00ff4b0 msgr2=0x7f4cf006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.179+0000 7f4ceffff640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cf00ff4b0 0x7f4cf006f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.179+0000 7f4ceffff640 1 -- 172.21.15.83:0/2024651817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce0009560 con 0x7f4cf0102ee0 2024-03-11T19:52:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.182+0000 7f4ceffff640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cf0102ee0 0x7f4cf01a7460 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800eb10 tx=0x7f4ce800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.183+0000 7f4cef7fe640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cf00ff4b0 0x7f4cf006f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:52:45.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.183+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf0102520 0x7f4cf006ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:45.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.183+0000 7f4cecff9640 1 -- 172.21.15.83:0/2024651817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8004b20 con 0x7f4cf0102ee0 2024-03-11T19:52:45.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.183+0000 7f4cecff9640 1 -- 172.21.15.83:0/2024651817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ce8013cb0 con 0x7f4cf0102ee0 2024-03-11T19:52:45.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.183+0000 7f4cecff9640 1 -- 172.21.15.83:0/2024651817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8005610 con 0x7f4cf0102ee0 2024-03-11T19:52:45.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.183+0000 7f4cf6088640 1 -- 172.21.15.83:0/2024651817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf0100370 con 0x7f4cf0102ee0 2024-03-11T19:52:45.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.183+0000 7f4cf6088640 1 -- 172.21.15.83:0/2024651817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf0100830 con 0x7f4cf0102ee0 2024-03-11T19:52:45.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.189+0000 7f4cf8c69740 1 Processor -- start 2024-03-11T19:52:45.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.190+0000 7f4cecff9640 1 -- 172.21.15.83:0/2024651817 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4ce8004c80 con 0x7f4cf0102ee0 2024-03-11T19:52:45.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.190+0000 7f4cf8c69740 1 -- start start 2024-03-11T19:52:45.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.190+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f62f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.190+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f634d0 0x5606a2f67fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f634d0 0x5606a2f67fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4ceeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f62f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f634d0 0x5606a2f67fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38474/0 (socket says 172.21.15.83:38474) 2024-03-11T19:52:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4ceeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f62f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:59866/0 (socket says 172.21.15.83:59866) 2024-03-11T19:52:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cecff9640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cd80a1b80 0x7f4cd80a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f68640 0x5606a2f6aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cef7fe640 1 -- 172.21.15.83:0/360523154 learned_addr learned my addr 172.21.15.83:0/360523154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:45.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cf8c69740 1 -- 172.21.15.83:0/360523154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2eca280 con 0x5606a2f634d0 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cf8c69740 1 -- 172.21.15.83:0/360523154 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2ec77b0 con 0x5606a2f62ae0 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cf8c69740 1 -- 172.21.15.83:0/360523154 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2ec2f90 con 0x5606a2f68640 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4ceffff640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cd80a1b80 0x7f4cd80a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cecff9640 1 -- 172.21.15.83:0/2024651817 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4ce80cd500 con 0x7f4cf0102ee0 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cef7fe640 1 -- 172.21.15.83:0/360523154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f68640 msgr2=0x5606a2f6aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cef7fe640 1 --2- 172.21.15.83:0/360523154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f68640 0x5606a2f6aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cef7fe640 1 -- 172.21.15.83:0/360523154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 msgr2=0x5606a2f62f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cef7fe640 1 --2- 172.21.15.83:0/360523154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f62f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.191+0000 7f4cef7fe640 1 -- 172.21.15.83:0/360523154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606a2e81570 con 0x5606a2f634d0 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.192+0000 7f4cef7fe640 1 --2- 172.21.15.83:0/360523154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f634d0 0x5606a2f67fd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0000c00 tx=0x7f4ce00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbe18b1cb979e28 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.192+0000 7f4ceffff640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cd80a1b80 0x7f4cd80a4040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80647f0 tx=0x7f4ce8096000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.192+0000 7f4cd5ffb640 1 -- 172.21.15.83:0/360523154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0005220 con 0x5606a2f634d0 2024-03-11T19:52:45.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.192+0000 7f4cd5ffb640 1 -- 172.21.15.83:0/360523154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ce0004df0 con 0x5606a2f634d0 2024-03-11T19:52:45.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.192+0000 7f4cd5ffb640 1 -- 172.21.15.83:0/360523154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0004040 con 0x5606a2f634d0 2024-03-11T19:52:45.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.193+0000 7f4cf8c69740 1 -- 172.21.15.83:0/360523154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f634d0 msgr2=0x5606a2f67fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.193+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/360523154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f634d0 0x5606a2f67fd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0000c00 tx=0x7f4ce00058d0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.193+0000 7f4cf8c69740 1 -- 172.21.15.83:0/360523154 shutdown_connections 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.193+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/360523154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f68640 0x5606a2f6aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.193+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/360523154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f634d0 0x5606a2f67fd0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.193+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/360523154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f62f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.193+0000 7f4cf8c69740 1 -- 172.21.15.83:0/360523154 >> 172.21.15.83:0/360523154 conn(0x5606a2f5e000 msgr2=0x5606a2f5e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.193+0000 7f4cf8c69740 1 -- 172.21.15.83:0/360523154 shutdown_connections 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.193+0000 7f4cf8c69740 1 -- 172.21.15.83:0/360523154 wait complete. 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.194+0000 7f4cf8c69740 1 -- start start 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.194+0000 7f4cf8c69740 1 Processor -- start 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.194+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.194+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f6cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4cef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f6cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4ceeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f68640 0x5606a2f73dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4ceeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:59870/0 (socket says 172.21.15.83:59870) 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4ceeffd640 1 -- 172.21.15.83:0/3490264466 learned_addr learned my addr 172.21.15.83:0/3490264466 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4cf8c69740 1 -- 172.21.15.83:0/3490264466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2b96430 con 0x5606a2f68640 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4cf8c69740 1 -- 172.21.15.83:0/3490264466 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2b52e00 con 0x5606a2f62ae0 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4cf8c69740 1 -- 172.21.15.83:0/3490264466 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2e47fd0 con 0x5606a2f634d0 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4cef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f6cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52366/0 (socket says 172.21.15.83:52366) 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4cf8c69740 10 client.0 _ll_register_callbacks cb 0x5606a2e9dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f68640 0x5606a2f73dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4ceeffd640 1 -- 172.21.15.83:0/3490264466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 msgr2=0x5606a2f6cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f6cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4ceeffd640 1 -- 172.21.15.83:0/3490264466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f68640 msgr2=0x5606a2f73dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f68640 0x5606a2f73dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.195+0000 7f4ceeffd640 1 -- 172.21.15.83:0/3490264466 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf0109ef0 con 0x5606a2f62ae0 2024-03-11T19:52:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.196+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c850 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400f490 tx=0x7f4ce4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f5be4911687eef9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.196+0000 7f4cceffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:52:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.196+0000 7f4cceffd640 1 -- 172.21.15.83:0/3490264466 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40122d0 con 0x5606a2f62ae0 2024-03-11T19:52:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.196+0000 7f4cceffd640 1 -- 172.21.15.83:0/3490264466 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4ce4005020 con 0x5606a2f62ae0 2024-03-11T19:52:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.196+0000 7f4cf8c69740 1 -- 172.21.15.83:0/3490264466 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606a2bc27e0 con 0x5606a2f62ae0 2024-03-11T19:52:45.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.196+0000 7f4cceffd640 1 -- 172.21.15.83:0/3490264466 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ce40051e0 con 0x5606a2f62ae0 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.197+0000 7f4cf8c69740 10 client.27299 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.197+0000 7f4cf8c69740 1 -- 172.21.15.83:0/3490264466 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5606a2ec1610 con 0x5606a2f62ae0 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.197+0000 7f4cceffd640 1 -- 172.21.15.83:0/3490264466 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ce4004c80 con 0x5606a2f62ae0 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.197+0000 7f4cceffd640 1 -- 172.21.15.83:0/3490264466 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4ce4010700 con 0x5606a2f62ae0 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.197+0000 7f4cf8c69740 10 client.27299 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.197+0000 7f4cf8c69740 10 client.27299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.197+0000 7f4cf8c69740 20 client.27299 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.197+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5606a2f78950 0x5606a2f7adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.197+0000 7f4cf8c69740 4 client.27299 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.197+0000 7f4cf8c69740 1 -- 172.21.15.83:0/3490264466 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606a2bc27e0 con 0x5606a2f78950 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.198+0000 7f4cef7fe640 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5606a2f78950 0x5606a2f7adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.198+0000 7f4cceffd640 10 client.27299 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:45.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.198+0000 7f4cef7fe640 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5606a2f78950 0x5606a2f7adf0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44c6b87091d42f2a server_cookie=9533010e3a4b1e47 in_seq=0 out_seq=0 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cceffd640 1 -- 172.21.15.83:0/3490264466 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5606a2bc27e0 con 0x5606a2f78950 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cceffd640 10 client.27299 handle_command_reply: tid=0 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 1 client.27299 shutdown 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 1 -- 172.21.15.83:0/3490264466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 msgr2=0x5606a2f6c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c850 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400f490 tx=0x7f4ce4005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 1 -- 172.21.15.83:0/3490264466 shutdown_connections 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5606a2f78950 0x5606a2f7adf0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f68640 0x5606a2f73dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cceffd640 0 client.27299 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f6cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/3490264466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c850 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 1 -- 172.21.15.83:0/3490264466 >> 172.21.15.83:0/3490264466 conn(0x5606a2f5e000 msgr2=0x5606a2f54a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 1 -- 172.21.15.83:0/3490264466 shutdown_connections 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 1 -- 172.21.15.83:0/3490264466 wait complete. 2024-03-11T19:52:45.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.199+0000 7f4cf8c69740 20 client.27299 trim_cache size 0 max 16384 2024-03-11T19:52:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.219+0000 7f4cf8c69740 1 Processor -- start 2024-03-11T19:52:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.219+0000 7f4cf8c69740 1 -- start start 2024-03-11T19:52:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.219+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.219+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f73d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f68640 0x5606a2f6c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4cf8c69740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2f6d8c0 con 0x5606a2f62ae0 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4cf8c69740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2f6b190 con 0x5606a2f68640 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4cf8c69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2f77030 con 0x5606a2f634d0 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4ceeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4ceeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38504/0 (socket says 172.21.15.83:38504) 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4ceeffd640 1 -- 172.21.15.83:0/2241384019 learned_addr learned my addr 172.21.15.83:0/2241384019 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4cef7fe640 1 --2- 172.21.15.83:0/2241384019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f73d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/2241384019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f68640 0x5606a2f6c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4ceeffd640 1 -- 172.21.15.83:0/2241384019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 msgr2=0x5606a2f73d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/2241384019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f73d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4ceeffd640 1 -- 172.21.15.83:0/2241384019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f68640 msgr2=0x5606a2f6c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/2241384019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f68640 0x5606a2f6c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4ceeffd640 1 -- 172.21.15.83:0/2241384019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606a2e5dac0 con 0x5606a2f62ae0 2024-03-11T19:52:45.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/2241384019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c460 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4012d00 tx=0x7f4ce400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de3d9403c0b3ba5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4cd57fa640 1 -- 172.21.15.83:0/2241384019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40122d0 con 0x5606a2f62ae0 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4cd57fa640 1 -- 172.21.15.83:0/2241384019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ce4010910 con 0x5606a2f62ae0 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.220+0000 7f4cd57fa640 1 -- 172.21.15.83:0/2241384019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40035f0 con 0x5606a2f62ae0 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.221+0000 7f4cf8c69740 1 -- 172.21.15.83:0/2241384019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f62ae0 msgr2=0x5606a2f6c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.221+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/2241384019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c460 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4012d00 tx=0x7f4ce400f3c0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.221+0000 7f4cf8c69740 1 -- 172.21.15.83:0/2241384019 shutdown_connections 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.221+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/2241384019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f68640 0x5606a2f6c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.221+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/2241384019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f73d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.221+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/2241384019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f62ae0 0x5606a2f6c460 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.221+0000 7f4cf8c69740 1 -- 172.21.15.83:0/2241384019 >> 172.21.15.83:0/2241384019 conn(0x5606a2f5e000 msgr2=0x5606a2f64510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.221+0000 7f4cf8c69740 1 -- 172.21.15.83:0/2241384019 shutdown_connections 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.221+0000 7f4cf8c69740 1 -- 172.21.15.83:0/2241384019 wait complete. 2024-03-11T19:52:45.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cf8c69740 1 -- start start 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cf8c69740 1 Processor -- start 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f72490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cf8c69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f68640 0x5606a2f72f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cf8c69740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2f740e0 con 0x5606a2f68640 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cf8c69740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2fcef00 con 0x5606a2f62ae0 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cf8c69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a2fcf070 con 0x5606a2f634d0 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cf8c69740 10 client.0 _ll_register_callbacks cb 0x7f4ce80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4ceeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f72490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4ceeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f72490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:59880/0 (socket says 172.21.15.83:59880) 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4ceeffd640 1 -- 172.21.15.83:0/560104471 learned_addr learned my addr 172.21.15.83:0/560104471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:45.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f68640 0x5606a2f72f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cef7fe640 1 -- 172.21.15.83:0/560104471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 msgr2=0x5606a2f72490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cef7fe640 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f72490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cef7fe640 1 -- 172.21.15.83:0/560104471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f68640 msgr2=0x5606a2f72f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cef7fe640 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f68640 0x5606a2f72f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4cef7fe640 1 -- 172.21.15.83:0/560104471 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606a2ec1610 con 0x5606a2f634d0 2024-03-11T19:52:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.222+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f68640 0x5606a2f72f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cef7fe640 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f729d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x5606a2f5efc0 tx=0x7f4ce002dd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b646fa068278618a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cceffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:52:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cceffd640 1 -- 172.21.15.83:0/560104471 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0004240 con 0x5606a2f634d0 2024-03-11T19:52:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cf8c69740 1 -- 172.21.15.83:0/560104471 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ce0004df0 con 0x5606a2f634d0 2024-03-11T19:52:45.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cceffd640 1 -- 172.21.15.83:0/560104471 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4ce004d070 con 0x5606a2f634d0 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cceffd640 1 -- 172.21.15.83:0/560104471 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ce00043a0 con 0x5606a2f634d0 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cf8c69740 10 client.27415 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cf8c69740 1 -- 172.21.15.83:0/560104471 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5606a2e836a0 con 0x5606a2f634d0 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cceffd640 1 -- 172.21.15.83:0/560104471 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ce005f020 con 0x5606a2f634d0 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cceffd640 1 -- 172.21.15.83:0/560104471 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4ce0057020 con 0x5606a2f634d0 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cf8c69740 10 client.27415 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.223+0000 7f4cf8c69740 10 client.27415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.224+0000 7f4cf8c69740 20 client.27415 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.224+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5606a2f78950 0x5606a2fd2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.224+0000 7f4cf8c69740 4 client.27415 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.224+0000 7f4cf8c69740 1 -- 172.21.15.83:0/560104471 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5606a2bff720 con 0x5606a2f78950 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.224+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5606a2f78950 0x5606a2fd2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.224+0000 7f4cceffd640 10 client.27415 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:45.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.224+0000 7f4ceeffd640 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5606a2f78950 0x5606a2fd2910 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c917823d2c6280 server_cookie=b18e59702aecedb1 in_seq=0 out_seq=0 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cceffd640 1 -- 172.21.15.83:0/560104471 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5606a2bff720 con 0x5606a2f78950 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cceffd640 10 client.27415 handle_command_reply: tid=0 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 1 client.27415 shutdown 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 1 -- 172.21.15.83:0/560104471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 msgr2=0x5606a2f729d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f729d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x5606a2f5efc0 tx=0x7f4ce002dd90 comp rx=0 tx=0).stop 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 1 -- 172.21.15.83:0/560104471 shutdown_connections 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5606a2f78950 0x5606a2fd2910 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606a2f68640 0x5606a2f72f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606a2f634d0 0x5606a2f729d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 1 --2- 172.21.15.83:0/560104471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606a2f62ae0 0x5606a2f72490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cceffd640 0 client.27415 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:45.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 1 -- 172.21.15.83:0/560104471 >> 172.21.15.83:0/560104471 conn(0x5606a2f5e000 msgr2=0x5606a2f6c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 1 -- 172.21.15.83:0/560104471 shutdown_connections 2024-03-11T19:52:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 1 -- 172.21.15.83:0/560104471 wait complete. 2024-03-11T19:52:45.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.226+0000 7f4cf8c69740 20 client.27415 trim_cache size 0 max 16384 2024-03-11T19:52:45.238 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:52:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:52:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:52:45.238 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:52:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 -- 172.21.15.83:0/2024651817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cd80a1b80 msgr2=0x7f4cd80a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cd80a1b80 0x7f4cd80a4040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80647f0 tx=0x7f4ce8096000 comp rx=0 tx=0).stop 2024-03-11T19:52:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 -- 172.21.15.83:0/2024651817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cf0102ee0 msgr2=0x7f4cf01a7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cf0102ee0 0x7f4cf01a7460 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800eb10 tx=0x7f4ce800c2d0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 -- 172.21.15.83:0/2024651817 shutdown_connections 2024-03-11T19:52:45.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4cd80a1b80 0x7f4cd80a4040 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4cf0102ee0 0x7f4cf01a7460 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf0102520 0x7f4cf006ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 --2- 172.21.15.83:0/2024651817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cf00ff4b0 0x7f4cf006f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 -- 172.21.15.83:0/2024651817 >> 172.21.15.83:0/2024651817 conn(0x7f4cf00fadc0 msgr2=0x7f4cf0105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 -- 172.21.15.83:0/2024651817 shutdown_connections 2024-03-11T19:52:45.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.227+0000 7f4cf6088640 1 -- 172.21.15.83:0/2024651817 wait complete. 2024-03-11T19:52:45.246 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:52:45.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:52:45.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.396+0000 7f6a89327640 1 -- 172.21.15.83:0/2881987523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a8410aaf0 msgr2=0x7f6a8410cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.396+0000 7f6a89327640 1 --2- 172.21.15.83:0/2881987523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a8410aaf0 0x7f6a8410cf80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009900 tx=0x7f6a7402bc80 comp rx=0 tx=0).stop 2024-03-11T19:52:45.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.397+0000 7f6a89327640 1 -- 172.21.15.83:0/2881987523 shutdown_connections 2024-03-11T19:52:45.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.397+0000 7f6a89327640 1 --2- 172.21.15.83:0/2881987523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8410d4c0 0x7f6a8410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.397+0000 7f6a89327640 1 --2- 172.21.15.83:0/2881987523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a8410aaf0 0x7f6a8410cf80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.397+0000 7f6a89327640 1 --2- 172.21.15.83:0/2881987523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a84108180 0x7f6a8410a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.397+0000 7f6a89327640 1 -- 172.21.15.83:0/2881987523 >> 172.21.15.83:0/2881987523 conn(0x7f6a8406dff0 msgr2=0x7f6a84070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.401+0000 7f6a89327640 1 -- 172.21.15.83:0/2881987523 shutdown_connections 2024-03-11T19:52:45.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.401+0000 7f6a89327640 1 -- 172.21.15.83:0/2881987523 wait complete. 2024-03-11T19:52:45.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.402+0000 7f6a89327640 1 Processor -- start 2024-03-11T19:52:45.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.402+0000 7f6a89327640 1 -- start start 2024-03-11T19:52:45.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.403+0000 7f6a89327640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a84108180 0x7f6a841a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.403+0000 7f6a89327640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8410aaf0 0x7f6a841a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.403+0000 7f6a89327640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8410d4c0 0x7f6a841ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.403+0000 7f6a89327640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a841adba0 con 0x7f6a84108180 2024-03-11T19:52:45.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.403+0000 7f6a89327640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8410f8e0 con 0x7f6a8410d4c0 2024-03-11T19:52:45.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.403+0000 7f6a89327640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8410faa0 con 0x7f6a8410aaf0 2024-03-11T19:52:45.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.407+0000 7f6a837fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8410d4c0 0x7f6a841ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.407+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8410aaf0 0x7f6a841a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.407+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8410aaf0 0x7f6a841a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52394/0 (socket says 172.21.15.83:52394) 2024-03-11T19:52:45.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.407+0000 7f6a827fc640 1 -- 172.21.15.83:0/740557168 learned_addr learned my addr 172.21.15.83:0/740557168 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:45.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.407+0000 7f6a827fc640 1 -- 172.21.15.83:0/740557168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8410d4c0 msgr2=0x7f6a841ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.407+0000 7f6a827fc640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8410d4c0 0x7f6a841ad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.407+0000 7f6a827fc640 1 -- 172.21.15.83:0/740557168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a84108180 msgr2=0x7f6a841a6100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:52:45.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.407+0000 7f6a827fc640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a84108180 0x7f6a841a6100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.407+0000 7f6a827fc640 1 -- 172.21.15.83:0/740557168 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a74009560 con 0x7f6a8410aaf0 2024-03-11T19:52:45.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.409+0000 7f6a827fc640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8410aaf0 0x7f6a841a6640 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6a74004b40 tx=0x7f6a7402cf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.409+0000 7f6a6bfff640 1 -- 172.21.15.83:0/740557168 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74005440 con 0x7f6a8410aaf0 2024-03-11T19:52:45.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.410+0000 7f6a89327640 1 -- 172.21.15.83:0/740557168 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8410fd20 con 0x7f6a8410aaf0 2024-03-11T19:52:45.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.410+0000 7f6a89327640 1 -- 172.21.15.83:0/740557168 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a84110220 con 0x7f6a8410aaf0 2024-03-11T19:52:45.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.410+0000 7f6a6bfff640 1 -- 172.21.15.83:0/740557168 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a74004db0 con 0x7f6a8410aaf0 2024-03-11T19:52:45.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.410+0000 7f6a6bfff640 1 -- 172.21.15.83:0/740557168 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7403e410 con 0x7f6a8410aaf0 2024-03-11T19:52:45.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.413+0000 7f6a6bfff640 1 -- 172.21.15.83:0/740557168 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6a7403e570 con 0x7f6a8410aaf0 2024-03-11T19:52:45.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.414+0000 7f6a6bfff640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a6c0a1b70 0x7f6a6c0a4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.414+0000 7f6a6bfff640 1 -- 172.21.15.83:0/740557168 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6a740f0450 con 0x7f6a8410aaf0 2024-03-11T19:52:45.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.415+0000 7f6a89327640 1 -- 172.21.15.83:0/740557168 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a84070af0 con 0x7f6a8410aaf0 2024-03-11T19:52:45.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.417+0000 7f6a82ffd640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a6c0a1b70 0x7f6a6c0a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.425+0000 7f6a82ffd640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a6c0a1b70 0x7f6a6c0a4030 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6a84072270 tx=0x7f6a78006bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.425+0000 7f6a6bfff640 1 -- 172.21.15.83:0/740557168 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6a740b95e0 con 0x7f6a8410aaf0 2024-03-11T19:52:45.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.733+0000 7f6a89327640 1 -- 172.21.15.83:0/740557168 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a8411bab0 con 0x7f6a8410aaf0 2024-03-11T19:52:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.734+0000 7f6a6bfff640 1 -- 172.21.15.83:0/740557168 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f6a740bb510 con 0x7f6a8410aaf0 2024-03-11T19:52:45.739 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:52:45.739 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:52:45.740 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:52:45.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 -- 172.21.15.83:0/740557168 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a6c0a1b70 msgr2=0x7f6a6c0a4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a6c0a1b70 0x7f6a6c0a4030 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6a84072270 tx=0x7f6a78006bd0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 -- 172.21.15.83:0/740557168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8410aaf0 msgr2=0x7f6a841a6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8410aaf0 0x7f6a841a6640 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6a74004b40 tx=0x7f6a7402cf40 comp rx=0 tx=0).stop 2024-03-11T19:52:45.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 -- 172.21.15.83:0/740557168 shutdown_connections 2024-03-11T19:52:45.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a6c0a1b70 0x7f6a6c0a4030 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a8410d4c0 0x7f6a841ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a8410aaf0 0x7f6a841a6640 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 --2- 172.21.15.83:0/740557168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a84108180 0x7f6a841a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 -- 172.21.15.83:0/740557168 >> 172.21.15.83:0/740557168 conn(0x7f6a8406dff0 msgr2=0x7f6a8410a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 -- 172.21.15.83:0/740557168 shutdown_connections 2024-03-11T19:52:45.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.737+0000 7f6a89327640 1 -- 172.21.15.83:0/740557168 wait complete. 2024-03-11T19:52:45.754 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:52:45.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:52:45.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.907+0000 7f9683172640 1 -- 172.21.15.83:0/1396857879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f967c1024b0 msgr2=0x7f967c102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.907+0000 7f9683172640 1 --2- 172.21.15.83:0/1396857879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f967c1024b0 0x7f967c102930 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9668009900 tx=0x7f966802bc30 comp rx=0 tx=0).stop 2024-03-11T19:52:45.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.907+0000 7f9683172640 1 -- 172.21.15.83:0/1396857879 shutdown_connections 2024-03-11T19:52:45.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.907+0000 7f9683172640 1 --2- 172.21.15.83:0/1396857879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f967c102e70 0x7f967c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.907+0000 7f9683172640 1 --2- 172.21.15.83:0/1396857879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f967c1024b0 0x7f967c102930 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.907+0000 7f9683172640 1 --2- 172.21.15.83:0/1396857879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f967c0ff440 0x7f967c0ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.907+0000 7f9683172640 1 -- 172.21.15.83:0/1396857879 >> 172.21.15.83:0/1396857879 conn(0x7f967c0fad70 msgr2=0x7f967c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.907+0000 7f9683172640 1 -- 172.21.15.83:0/1396857879 shutdown_connections 2024-03-11T19:52:45.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.907+0000 7f9683172640 1 -- 172.21.15.83:0/1396857879 wait complete. 2024-03-11T19:52:45.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f9683172640 1 Processor -- start 2024-03-11T19:52:45.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f9683172640 1 -- start start 2024-03-11T19:52:45.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f9683172640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f967c0ff440 0x7f967c101e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f9683172640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f967c1024b0 0x7f967c100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f9683172640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f967c102e70 0x7f967c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f9683172640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c19b080 con 0x7f967c102e70 2024-03-11T19:52:45.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f9683172640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c100ee0 con 0x7f967c1024b0 2024-03-11T19:52:45.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f9683172640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c1010a0 con 0x7f967c0ff440 2024-03-11T19:52:45.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f967bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f967c1024b0 0x7f967c100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f967bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f967c1024b0 0x7f967c100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:59900/0 (socket says 172.21.15.83:59900) 2024-03-11T19:52:45.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f9680f0e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f967c0ff440 0x7f967c101e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f968170f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f967c102e70 0x7f967c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.908+0000 7f967bfff640 1 -- 172.21.15.83:0/453666868 learned_addr learned my addr 172.21.15.83:0/453666868 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:45.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.909+0000 7f967bfff640 1 -- 172.21.15.83:0/453666868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f967c0ff440 msgr2=0x7f967c101e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.909+0000 7f967bfff640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f967c0ff440 0x7f967c101e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.909+0000 7f967bfff640 1 -- 172.21.15.83:0/453666868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f967c102e70 msgr2=0x7f967c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.909+0000 7f967bfff640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f967c102e70 0x7f967c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.909+0000 7f967bfff640 1 -- 172.21.15.83:0/453666868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9668009560 con 0x7f967c1024b0 2024-03-11T19:52:45.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.909+0000 7f967bfff640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f967c1024b0 0x7f967c100460 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f966802c6a0 tx=0x7f96680040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.909+0000 7f9679ffb640 1 -- 172.21.15.83:0/453666868 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680044c0 con 0x7f967c1024b0 2024-03-11T19:52:45.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.909+0000 7f9683172640 1 -- 172.21.15.83:0/453666868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967c1012c0 con 0x7f967c1024b0 2024-03-11T19:52:45.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.909+0000 7f9679ffb640 1 -- 172.21.15.83:0/453666868 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96680052d0 con 0x7f967c1024b0 2024-03-11T19:52:45.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.909+0000 7f9679ffb640 1 -- 172.21.15.83:0/453666868 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966803e450 con 0x7f967c1024b0 2024-03-11T19:52:45.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.909+0000 7f9683172640 1 -- 172.21.15.83:0/453666868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967c1a5e40 con 0x7f967c1024b0 2024-03-11T19:52:45.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.912+0000 7f9679ffb640 1 -- 172.21.15.83:0/453666868 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9668005440 con 0x7f967c1024b0 2024-03-11T19:52:45.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.912+0000 7f9679ffb640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f966c0a1ab0 0x7f966c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.912+0000 7f9679ffb640 1 -- 172.21.15.83:0/453666868 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f96680efb40 con 0x7f967c1024b0 2024-03-11T19:52:45.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.912+0000 7f9680f0e640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f966c0a1ab0 0x7f966c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9685e16740 1 Processor -- start 2024-03-11T19:52:45.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9685e16740 1 -- start start 2024-03-11T19:52:45.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9680f0e640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f966c0a1ab0 0x7f966c0a3f70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f967c068040 tx=0x7f9670005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9685e16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df391690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9685e16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df391c60 0x55e2df396760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9685e16740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df396dd0 0x55e2df399260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9680f0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df391690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9685e16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2df34d960 con 0x55e2df391270 2024-03-11T19:52:45.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9680f0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df391690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38552/0 (socket says 172.21.15.83:38552) 2024-03-11T19:52:45.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9685e16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2df301310 con 0x55e2df391c60 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9680f0e640 1 -- 172.21.15.83:0/420477277 learned_addr learned my addr 172.21.15.83:0/420477277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9685e16740 1 -- 172.21.15.83:0/420477277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2df2fe840 con 0x55e2df396dd0 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f967bfff640 1 --2- 172.21.15.83:0/420477277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df391c60 0x55e2df396760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f968170f640 1 --2- 172.21.15.83:0/420477277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df396dd0 0x55e2df399260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9680f0e640 1 -- 172.21.15.83:0/420477277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df396dd0 msgr2=0x55e2df399260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9680f0e640 1 --2- 172.21.15.83:0/420477277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df396dd0 0x55e2df399260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9680f0e640 1 -- 172.21.15.83:0/420477277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df391c60 msgr2=0x55e2df396760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9680f0e640 1 --2- 172.21.15.83:0/420477277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df391c60 0x55e2df396760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9680f0e640 1 -- 172.21.15.83:0/420477277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2df2bc060 con 0x55e2df391270 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.913+0000 7f9680f0e640 1 --2- 172.21.15.83:0/420477277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df391690 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f967000e630 tx=0x7f9670014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a277799c1aff8b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.914+0000 7f9666ffd640 1 -- 172.21.15.83:0/420477277 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670026430 con 0x55e2df391270 2024-03-11T19:52:45.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.914+0000 7f9666ffd640 1 -- 172.21.15.83:0/420477277 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9670021070 con 0x55e2df391270 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.914+0000 7f9666ffd640 1 -- 172.21.15.83:0/420477277 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967001d7d0 con 0x55e2df391270 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.915+0000 7f9685e16740 1 -- 172.21.15.83:0/420477277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 msgr2=0x55e2df391690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.915+0000 7f9685e16740 1 --2- 172.21.15.83:0/420477277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df391690 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f967000e630 tx=0x7f9670014000 comp rx=0 tx=0).stop 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 -- 172.21.15.83:0/420477277 shutdown_connections 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 --2- 172.21.15.83:0/420477277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df396dd0 0x55e2df399260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 --2- 172.21.15.83:0/420477277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df391c60 0x55e2df396760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 --2- 172.21.15.83:0/420477277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df391690 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 -- 172.21.15.83:0/420477277 >> 172.21.15.83:0/420477277 conn(0x55e2df38c7e0 msgr2=0x55e2df38cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 -- 172.21.15.83:0/420477277 shutdown_connections 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 -- 172.21.15.83:0/420477277 wait complete. 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 -- start start 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 Processor -- start 2024-03-11T19:52:45.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 0x55e2df39b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391c60 0x55e2df39b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df396dd0 0x55e2df3a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2df2fa020 con 0x55e2df391c60 2024-03-11T19:52:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2defcd430 con 0x55e2df396dd0 2024-03-11T19:52:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2def89e00 con 0x55e2df391270 2024-03-11T19:52:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f9685e16740 10 client.0 _ll_register_callbacks cb 0x55e2df2a1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.916+0000 7f967bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391c60 0x55e2df39b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f967bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391c60 0x55e2df39b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38556/0 (socket says 172.21.15.83:38556) 2024-03-11T19:52:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f967bfff640 1 -- 172.21.15.83:0/3832830382 learned_addr learned my addr 172.21.15.83:0/3832830382 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f9680f0e640 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 0x55e2df39b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f968170f640 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df396dd0 0x55e2df3a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f967bfff640 1 -- 172.21.15.83:0/3832830382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 msgr2=0x55e2df39b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f967bfff640 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 0x55e2df39b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f967bfff640 1 -- 172.21.15.83:0/3832830382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df396dd0 msgr2=0x55e2df3a29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f967bfff640 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df396dd0 0x55e2df3a29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f967bfff640 1 -- 172.21.15.83:0/3832830382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9670017070 con 0x55e2df391c60 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f967bfff640 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391c60 0x55e2df39b970 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f966802cdd0 tx=0x7f96680bd320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d536c8c8ab0c3a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f965bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f965bfff640 1 -- 172.21.15.83:0/3832830382 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680037f0 con 0x55e2df391c60 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.917+0000 7f965bfff640 1 -- 172.21.15.83:0/3832830382 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f96680c5300 con 0x55e2df391c60 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f9685e16740 1 -- 172.21.15.83:0/3832830382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2deff97e0 con 0x55e2df391c60 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f965bfff640 1 -- 172.21.15.83:0/3832830382 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96680c4e70 con 0x55e2df391c60 2024-03-11T19:52:45.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f9685e16740 10 client.17868 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f9685e16740 1 -- 172.21.15.83:0/3832830382 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e2df2f86a0 con 0x55e2df391c60 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f965bfff640 1 -- 172.21.15.83:0/3832830382 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f966809f030 con 0x55e2df391c60 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f965bfff640 1 -- 172.21.15.83:0/3832830382 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f966803c070 con 0x55e2df391c60 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f9685e16740 10 client.17868 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f9685e16740 10 client.17868 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f9685e16740 20 client.17868 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f9685e16740 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e2df3a74f0 0x55e2df3a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f9685e16740 4 client.17868 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f9685e16740 1 -- 172.21.15.83:0/3832830382 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2df036720 con 0x55e2df3a74f0 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.918+0000 7f9680f0e640 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e2df3a74f0 0x55e2df3a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.919+0000 7f9680f0e640 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e2df3a74f0 0x55e2df3a98e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7dc17e945df3660 server_cookie=33971a23bc0eb2a0 in_seq=0 out_seq=0 2024-03-11T19:52:45.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.919+0000 7f965bfff640 10 client.17868 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f965bfff640 1 -- 172.21.15.83:0/3832830382 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e2df036720 con 0x55e2df3a74f0 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f965bfff640 10 client.17868 handle_command_reply: tid=0 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 1 client.17868 shutdown 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 1 -- 172.21.15.83:0/3832830382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391c60 msgr2=0x55e2df39b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391c60 0x55e2df39b970 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f966802cdd0 tx=0x7f96680bd320 comp rx=0 tx=0).stop 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 1 -- 172.21.15.83:0/3832830382 shutdown_connections 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e2df3a74f0 0x55e2df3a98e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f965bfff640 0 client.17868 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df396dd0 0x55e2df3a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391c60 0x55e2df39b970 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 1 --2- 172.21.15.83:0/3832830382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 0x55e2df39b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 1 -- 172.21.15.83:0/3832830382 >> 172.21.15.83:0/3832830382 conn(0x55e2df38c7e0 msgr2=0x55e2df38bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 1 -- 172.21.15.83:0/3832830382 shutdown_connections 2024-03-11T19:52:45.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 1 -- 172.21.15.83:0/3832830382 wait complete. 2024-03-11T19:52:45.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.920+0000 7f9685e16740 20 client.17868 trim_cache size 0 max 16384 2024-03-11T19:52:45.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.934+0000 7f9685e16740 1 Processor -- start 2024-03-11T19:52:45.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.934+0000 7f9685e16740 1 -- start start 2024-03-11T19:52:45.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.934+0000 7f9685e16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df3931b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.934+0000 7f9685e16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df391c60 0x55e2df3936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.934+0000 7f9685e16740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df396dd0 0x55e2df3a2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.934+0000 7f9685e16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2df27f130 con 0x55e2df391270 2024-03-11T19:52:45.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.934+0000 7f9685e16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2df37ea40 con 0x55e2df391c60 2024-03-11T19:52:45.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.934+0000 7f9685e16740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2df38d7d0 con 0x55e2df396dd0 2024-03-11T19:52:45.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f9680f0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df3931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f968170f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df396dd0 0x55e2df3a2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f9680f0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df3931b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38566/0 (socket says 172.21.15.83:38566) 2024-03-11T19:52:45.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f9680f0e640 1 -- 172.21.15.83:0/807330273 learned_addr learned my addr 172.21.15.83:0/807330273 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:45.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f967bfff640 1 --2- 172.21.15.83:0/807330273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df391c60 0x55e2df3936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f9680f0e640 1 -- 172.21.15.83:0/807330273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df396dd0 msgr2=0x55e2df3a2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f9680f0e640 1 --2- 172.21.15.83:0/807330273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df396dd0 0x55e2df3a2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f9680f0e640 1 -- 172.21.15.83:0/807330273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df391c60 msgr2=0x55e2df3936f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f9680f0e640 1 --2- 172.21.15.83:0/807330273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df391c60 0x55e2df3936f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f9680f0e640 1 -- 172.21.15.83:0/807330273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2df294c20 con 0x55e2df391270 2024-03-11T19:52:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f967bfff640 1 --2- 172.21.15.83:0/807330273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df391c60 0x55e2df3936f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f9680f0e640 1 --2- 172.21.15.83:0/807330273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df3931b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f96700052c0 tx=0x7f96700093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a33a3ea3fcf2049 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f96667fc640 1 -- 172.21.15.83:0/807330273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967000ed60 con 0x55e2df391270 2024-03-11T19:52:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f96667fc640 1 -- 172.21.15.83:0/807330273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96700053f0 con 0x55e2df391270 2024-03-11T19:52:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.935+0000 7f96667fc640 1 -- 172.21.15.83:0/807330273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670004930 con 0x55e2df391270 2024-03-11T19:52:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.936+0000 7f9685e16740 1 -- 172.21.15.83:0/807330273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 msgr2=0x55e2df3931b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.936+0000 7f9685e16740 1 --2- 172.21.15.83:0/807330273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df3931b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f96700052c0 tx=0x7f96700093b0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.936+0000 7f9685e16740 1 -- 172.21.15.83:0/807330273 shutdown_connections 2024-03-11T19:52:45.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.936+0000 7f9685e16740 1 --2- 172.21.15.83:0/807330273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df396dd0 0x55e2df3a2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.936+0000 7f9685e16740 1 --2- 172.21.15.83:0/807330273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df391c60 0x55e2df3936f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.936+0000 7f9685e16740 1 --2- 172.21.15.83:0/807330273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391270 0x55e2df3931b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.936+0000 7f9685e16740 1 -- 172.21.15.83:0/807330273 >> 172.21.15.83:0/807330273 conn(0x55e2df38c7e0 msgr2=0x55e2df3ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.936+0000 7f9685e16740 1 -- 172.21.15.83:0/807330273 shutdown_connections 2024-03-11T19:52:45.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.936+0000 7f9685e16740 1 -- 172.21.15.83:0/807330273 wait complete. 2024-03-11T19:52:45.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.936+0000 7f9685e16740 1 -- start start 2024-03-11T19:52:45.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.936+0000 7f9685e16740 1 Processor -- start 2024-03-11T19:52:45.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.937+0000 7f9685e16740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 0x55e2df3a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.937+0000 7f9685e16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391c60 0x55e2df3a2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.937+0000 7f9685e16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df396dd0 0x55e2df3fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.937+0000 7f9685e16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2df3fe110 con 0x55e2df391c60 2024-03-11T19:52:45.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.937+0000 7f9685e16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2df3fe250 con 0x55e2df396dd0 2024-03-11T19:52:45.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.937+0000 7f9685e16740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2df3fe3c0 con 0x55e2df391270 2024-03-11T19:52:45.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.937+0000 7f9685e16740 10 client.0 _ll_register_callbacks cb 0x7f96700099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:45.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.937+0000 7f9680f0e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 0x55e2df3a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.937+0000 7f9680f0e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 0x55e2df3a1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52428/0 (socket says 172.21.15.83:52428) 2024-03-11T19:52:45.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.937+0000 7f9680f0e640 1 -- 172.21.15.83:0/4075387142 learned_addr learned my addr 172.21.15.83:0/4075387142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:45.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f968170f640 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df396dd0 0x55e2df3fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f9680f0e640 1 -- 172.21.15.83:0/4075387142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df396dd0 msgr2=0x55e2df3fdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:45.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f9680f0e640 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df396dd0 0x55e2df3fdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f9680f0e640 1 -- 172.21.15.83:0/4075387142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391c60 msgr2=0x55e2df3a2130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:52:45.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f9680f0e640 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391c60 0x55e2df3a2130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f9680f0e640 1 -- 172.21.15.83:0/4075387142 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2df2f86a0 con 0x55e2df391270 2024-03-11T19:52:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f968170f640 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df396dd0 0x55e2df3fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:52:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f9680f0e640 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 0x55e2df3a1bf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f96700058c0 tx=0x7f9670035f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=183204bd96e2b1bf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f965bfff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:52:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f965bfff640 1 -- 172.21.15.83:0/4075387142 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967000ed60 con 0x55e2df391270 2024-03-11T19:52:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f965bfff640 1 -- 172.21.15.83:0/4075387142 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f96700340f0 con 0x55e2df391270 2024-03-11T19:52:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.938+0000 7f9685e16740 1 -- 172.21.15.83:0/4075387142 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9670021070 con 0x55e2df391270 2024-03-11T19:52:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f965bfff640 1 -- 172.21.15.83:0/4075387142 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f967004c030 con 0x55e2df391270 2024-03-11T19:52:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f9685e16740 10 client.27439 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f9685e16740 1 -- 172.21.15.83:0/4075387142 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e2df2bc060 con 0x55e2df391270 2024-03-11T19:52:45.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f965bfff640 1 -- 172.21.15.83:0/4075387142 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96700046d0 con 0x55e2df391270 2024-03-11T19:52:45.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f965bfff640 1 -- 172.21.15.83:0/4075387142 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9670046460 con 0x55e2df391270 2024-03-11T19:52:45.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f9685e16740 10 client.27439 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:45.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f9685e16740 10 client.27439 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:52:45.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f9685e16740 20 client.27439 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:45.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f9685e16740 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e2df3ffac0 0x55e2df401ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:45.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f9685e16740 4 client.27439 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:52:45.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f9685e16740 1 -- 172.21.15.83:0/4075387142 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f9670021070 con 0x55e2df3ffac0 2024-03-11T19:52:45.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.939+0000 7f967bfff640 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e2df3ffac0 0x55e2df401ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:45.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f965bfff640 10 client.27439 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:52:45.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f967bfff640 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e2df3ffac0 0x55e2df401ee0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f635cd5cb3bafe4 server_cookie=26fd097b40ff59d8 in_seq=0 out_seq=0 2024-03-11T19:52:45.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f965bfff640 1 -- 172.21.15.83:0/4075387142 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f9670021070 con 0x55e2df3ffac0 2024-03-11T19:52:45.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f965bfff640 10 client.27439 handle_command_reply: tid=0 2024-03-11T19:52:45.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 1 client.27439 shutdown 2024-03-11T19:52:45.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 1 -- 172.21.15.83:0/4075387142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 msgr2=0x55e2df3a1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 0x55e2df3a1bf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f96700058c0 tx=0x7f9670035f70 comp rx=0 tx=0).stop 2024-03-11T19:52:45.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 1 -- 172.21.15.83:0/4075387142 shutdown_connections 2024-03-11T19:52:45.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55e2df3ffac0 0x55e2df401ee0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e2df396dd0 0x55e2df3fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f965bfff640 0 client.27439 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:52:45.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2df391c60 0x55e2df3a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 1 --2- 172.21.15.83:0/4075387142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2df391270 0x55e2df3a1bf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 1 -- 172.21.15.83:0/4075387142 >> 172.21.15.83:0/4075387142 conn(0x55e2df38c7e0 msgr2=0x55e2df3931b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 1 -- 172.21.15.83:0/4075387142 shutdown_connections 2024-03-11T19:52:45.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 1 -- 172.21.15.83:0/4075387142 wait complete. 2024-03-11T19:52:45.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.940+0000 7f9685e16740 20 client.27439 trim_cache size 0 max 16384 2024-03-11T19:52:45.954 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:52:45.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 -- 172.21.15.83:0/453666868 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f966c0a1ab0 msgr2=0x7f966c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f966c0a1ab0 0x7f966c0a3f70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f967c068040 tx=0x7f9670005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:45.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 -- 172.21.15.83:0/453666868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f967c1024b0 msgr2=0x7f967c100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:45.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f967c1024b0 0x7f967c100460 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f966802c6a0 tx=0x7f96680040c0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 -- 172.21.15.83:0/453666868 shutdown_connections 2024-03-11T19:52:45.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f966c0a1ab0 0x7f966c0a3f70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f967c102e70 0x7f967c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f967c1024b0 0x7f967c100460 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 --2- 172.21.15.83:0/453666868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f967c0ff440 0x7f967c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:45.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 -- 172.21.15.83:0/453666868 >> 172.21.15.83:0/453666868 conn(0x7f967c0fad70 msgr2=0x7f967c1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:45.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 -- 172.21.15.83:0/453666868 shutdown_connections 2024-03-11T19:52:45.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:45.941+0000 7f9683172640 1 -- 172.21.15.83:0/453666868 wait complete. 2024-03-11T19:52:45.958 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:52:45.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:52:46.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.113+0000 7f37e88c3640 1 -- 172.21.15.83:0/601386261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e4101650 msgr2=0x7f37e4101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.113+0000 7f37e88c3640 1 --2- 172.21.15.83:0/601386261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e4101650 0x7f37e4101a70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009900 tx=0x7f37cc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:52:46.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.114+0000 7f37e88c3640 1 -- 172.21.15.83:0/601386261 shutdown_connections 2024-03-11T19:52:46.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.114+0000 7f37e88c3640 1 --2- 172.21.15.83:0/601386261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e4100c40 0x7f37e4108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.114+0000 7f37e88c3640 1 --2- 172.21.15.83:0/601386261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e41002a0 0x7f37e4100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.114+0000 7f37e88c3640 1 --2- 172.21.15.83:0/601386261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e4101650 0x7f37e4101a70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.114+0000 7f37e88c3640 1 -- 172.21.15.83:0/601386261 >> 172.21.15.83:0/601386261 conn(0x7f37e40fad70 msgr2=0x7f37e40fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.114+0000 7f37e88c3640 1 -- 172.21.15.83:0/601386261 shutdown_connections 2024-03-11T19:52:46.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.114+0000 7f37e88c3640 1 -- 172.21.15.83:0/601386261 wait complete. 2024-03-11T19:52:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37e88c3640 1 Processor -- start 2024-03-11T19:52:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37e88c3640 1 -- start start 2024-03-11T19:52:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37e88c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e41002a0 0x7f37e4104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37e88c3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e4100c40 0x7f37e4102690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37e88c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e4101650 0x7f37e4102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e4100c40 0x7f37e4102690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37e2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e4101650 0x7f37e4102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e4100c40 0x7f37e4102690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:59956/0 (socket says 172.21.15.83:59956) 2024-03-11T19:52:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37dbfff640 1 -- 172.21.15.83:0/3760170905 learned_addr learned my addr 172.21.15.83:0/3760170905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37e88c3640 1 -- 172.21.15.83:0/3760170905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e4103330 con 0x7f37e4101650 2024-03-11T19:52:46.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37e88c3640 1 -- 172.21.15.83:0/3760170905 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e41034a0 con 0x7f37e4100c40 2024-03-11T19:52:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37e88c3640 1 -- 172.21.15.83:0/3760170905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e41a12a0 con 0x7f37e41002a0 2024-03-11T19:52:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.115+0000 7f37e259c640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e41002a0 0x7f37e4104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.116+0000 7f37e2d9d640 1 -- 172.21.15.83:0/3760170905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e41002a0 msgr2=0x7f37e4104050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.116+0000 7f37e2d9d640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e41002a0 0x7f37e4104050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.116+0000 7f37e2d9d640 1 -- 172.21.15.83:0/3760170905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e4100c40 msgr2=0x7f37e4102690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.116+0000 7f37e2d9d640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e4100c40 0x7f37e4102690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.116+0000 7f37e2d9d640 1 -- 172.21.15.83:0/3760170905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc009560 con 0x7f37e4101650 2024-03-11T19:52:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.116+0000 7f37e2d9d640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e4101650 0x7f37e4102bd0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00ea80 tx=0x7f37dc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.117+0000 7f37db7fe640 1 -- 172.21.15.83:0/3760170905 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc00cea0 con 0x7f37e4101650 2024-03-11T19:52:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.117+0000 7f37db7fe640 1 -- 172.21.15.83:0/3760170905 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37dc005020 con 0x7f37e4101650 2024-03-11T19:52:46.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.117+0000 7f37db7fe640 1 -- 172.21.15.83:0/3760170905 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc010690 con 0x7f37e4101650 2024-03-11T19:52:46.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.117+0000 7f37e88c3640 1 -- 172.21.15.83:0/3760170905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e41a1550 con 0x7f37e4101650 2024-03-11T19:52:46.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.117+0000 7f37e88c3640 1 -- 172.21.15.83:0/3760170905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e41a1a00 con 0x7f37e4101650 2024-03-11T19:52:46.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.121+0000 7f37eb574740 1 Processor -- start 2024-03-11T19:52:46.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37db7fe640 1 -- 172.21.15.83:0/3760170905 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f37dc0054e0 con 0x7f37e4101650 2024-03-11T19:52:46.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37eb574740 1 -- start start 2024-03-11T19:52:46.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37eb574740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb041690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37eb574740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb046760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37eb574740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb046dd0 0x5630cb049260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37eb574740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630caffd960 con 0x5630cb041270 2024-03-11T19:52:46.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37eb574740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630cafb1310 con 0x5630cb046dd0 2024-03-11T19:52:46.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37eb574740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630cafae840 con 0x5630cb041c60 2024-03-11T19:52:46.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb041690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb041690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38586/0 (socket says 172.21.15.83:38586) 2024-03-11T19:52:46.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37dbfff640 1 -- 172.21.15.83:0/521465838 learned_addr learned my addr 172.21.15.83:0/521465838 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:46.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37e259c640 1 --2- 172.21.15.83:0/521465838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb046760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.122+0000 7f37dbfff640 1 --2- 172.21.15.83:0/521465838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb046dd0 0x5630cb049260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37e259c640 1 -- 172.21.15.83:0/521465838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb046dd0 msgr2=0x5630cb049260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37e259c640 1 --2- 172.21.15.83:0/521465838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb046dd0 0x5630cb049260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37e259c640 1 -- 172.21.15.83:0/521465838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 msgr2=0x5630cb041690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37e259c640 1 --2- 172.21.15.83:0/521465838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb041690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37e259c640 1 -- 172.21.15.83:0/521465838 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630caf6c060 con 0x5630cb041c60 2024-03-11T19:52:46.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37dbfff640 1 --2- 172.21.15.83:0/521465838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb041690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:46.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37e259c640 1 --2- 172.21.15.83:0/521465838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb046760 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f37cc000c00 tx=0x7f37cc005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e383efd4c7e0cc6f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37db7fe640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37d00a1c00 0x7f37d00a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37d8ff9640 1 -- 172.21.15.83:0/521465838 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc02cea0 con 0x5630cb041c60 2024-03-11T19:52:46.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37dbfff640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37d00a1c00 0x7f37d00a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37d8ff9640 1 -- 172.21.15.83:0/521465838 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37cc02cbe0 con 0x5630cb041c60 2024-03-11T19:52:46.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37d8ff9640 1 -- 172.21.15.83:0/521465838 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0051b0 con 0x5630cb041c60 2024-03-11T19:52:46.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.123+0000 7f37db7fe640 1 -- 172.21.15.83:0/3760170905 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f37dc014070 con 0x7f37e4101650 2024-03-11T19:52:46.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.124+0000 7f37dbfff640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37d00a1c00 0x7f37d00a40c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5630cb0424c0 tx=0x7f37d4009880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.124+0000 7f37eb574740 1 -- 172.21.15.83:0/521465838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 msgr2=0x5630cb046760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.124+0000 7f37eb574740 1 --2- 172.21.15.83:0/521465838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb046760 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f37cc000c00 tx=0x7f37cc005dd0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.124+0000 7f37eb574740 1 -- 172.21.15.83:0/521465838 shutdown_connections 2024-03-11T19:52:46.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.124+0000 7f37eb574740 1 --2- 172.21.15.83:0/521465838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb046dd0 0x5630cb049260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.124+0000 7f37eb574740 1 --2- 172.21.15.83:0/521465838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb046760 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.124+0000 7f37eb574740 1 --2- 172.21.15.83:0/521465838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb041690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.124+0000 7f37eb574740 1 -- 172.21.15.83:0/521465838 >> 172.21.15.83:0/521465838 conn(0x5630cb03c7e0 msgr2=0x5630cb03cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.124+0000 7f37eb574740 1 -- 172.21.15.83:0/521465838 shutdown_connections 2024-03-11T19:52:46.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.124+0000 7f37eb574740 1 -- 172.21.15.83:0/521465838 wait complete. 2024-03-11T19:52:46.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37eb574740 1 -- start start 2024-03-11T19:52:46.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37eb574740 1 Processor -- start 2024-03-11T19:52:46.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37eb574740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb04b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37eb574740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb04b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37eb574740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb046dd0 0x5630cb052980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37eb574740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630cafaa020 con 0x5630cb046dd0 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37eb574740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630cac7d430 con 0x5630cb041270 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37eb574740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630cac39e00 con 0x5630cb041c60 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37eb574740 10 client.0 _ll_register_callbacks cb 0x5630caf51cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37e259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb04b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37e259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb04b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:59974/0 (socket says 172.21.15.83:59974) 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37e259c640 1 -- 172.21.15.83:0/1675225540 learned_addr learned my addr 172.21.15.83:0/1675225540 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37e259c640 1 -- 172.21.15.83:0/1675225540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 msgr2=0x5630cb04b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37e259c640 1 --2- 172.21.15.83:0/1675225540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb04b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37e259c640 1 -- 172.21.15.83:0/1675225540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb046dd0 msgr2=0x5630cb052980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.125+0000 7f37e259c640 1 --2- 172.21.15.83:0/1675225540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb046dd0 0x5630cb052980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.126+0000 7f37e259c640 1 -- 172.21.15.83:0/1675225540 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e4108e00 con 0x5630cb041270 2024-03-11T19:52:46.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.126+0000 7f37e259c640 1 --2- 172.21.15.83:0/1675225540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb04b400 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f37cc036040 tx=0x7f37cc005220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=824011d36fa67123 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.126+0000 7f37c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.126+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1675225540 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc005620 con 0x5630cb041270 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.126+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1675225540 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f37cc047450 con 0x5630cb041270 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.126+0000 7f37eb574740 1 -- 172.21.15.83:0/1675225540 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630caca97e0 con 0x5630cb041270 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1675225540 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37cc047610 con 0x5630cb041270 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37eb574740 10 client.27338 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37eb574740 1 -- 172.21.15.83:0/1675225540 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5630cafa86a0 con 0x5630cb041270 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1675225540 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5630caca97e0 con 0x5630cb041270 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1675225540 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f37cc056050 con 0x5630cb041270 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37eb574740 10 client.27338 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37eb574740 10 client.27338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:52:46.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37eb574740 20 client.27338 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37eb574740 1 --2- 172.21.15.83:0/1675225540 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5630cb0571e0 0x5630cb059640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37eb574740 4 client.27338 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37eb574740 1 -- 172.21.15.83:0/1675225540 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630cace6720 con 0x5630cb0571e0 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.127+0000 7f37dbfff640 1 --2- 172.21.15.83:0/1675225540 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5630cb0571e0 0x5630cb059640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.128+0000 7f37c9ffb640 10 client.27338 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.128+0000 7f37dbfff640 1 --2- 172.21.15.83:0/1675225540 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5630cb0571e0 0x5630cb059640 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad32648b5a0731ac server_cookie=551533a3718ef5a2 in_seq=0 out_seq=0 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1675225540 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5630cace6720 con 0x5630cb0571e0 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37c9ffb640 10 client.27338 handle_command_reply: tid=0 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 1 client.27338 shutdown 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 1 -- 172.21.15.83:0/1675225540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 msgr2=0x5630cb04b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 1 --2- 172.21.15.83:0/1675225540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb04b400 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f37cc036040 tx=0x7f37cc005220 comp rx=0 tx=0).stop 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 1 -- 172.21.15.83:0/1675225540 shutdown_connections 2024-03-11T19:52:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 1 --2- 172.21.15.83:0/1675225540 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5630cb0571e0 0x5630cb059640 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37c9ffb640 0 client.27338 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:52:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 1 --2- 172.21.15.83:0/1675225540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb046dd0 0x5630cb052980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 1 --2- 172.21.15.83:0/1675225540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb04b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 1 --2- 172.21.15.83:0/1675225540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb04b400 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 1 -- 172.21.15.83:0/1675225540 >> 172.21.15.83:0/1675225540 conn(0x5630cb03c7e0 msgr2=0x5630cb03bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 1 -- 172.21.15.83:0/1675225540 shutdown_connections 2024-03-11T19:52:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 1 -- 172.21.15.83:0/1675225540 wait complete. 2024-03-11T19:52:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.129+0000 7f37eb574740 20 client.27338 trim_cache size 0 max 16384 2024-03-11T19:52:46.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.142+0000 7f37eb574740 1 Processor -- start 2024-03-11T19:52:46.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.142+0000 7f37eb574740 1 -- start start 2024-03-11T19:52:46.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37eb574740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb03b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37eb574740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb0528a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37eb574740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb046dd0 0x5630cb03b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37eb574740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630caf2f130 con 0x5630cb046dd0 2024-03-11T19:52:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37eb574740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630cb04c5f0 con 0x5630cb041270 2024-03-11T19:52:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37eb574740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630cb049720 con 0x5630cb041c60 2024-03-11T19:52:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37e259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb03b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37e2d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb046dd0 0x5630cb03b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37e259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb03b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:59988/0 (socket says 172.21.15.83:59988) 2024-03-11T19:52:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37e259c640 1 -- 172.21.15.83:0/2936504772 learned_addr learned my addr 172.21.15.83:0/2936504772 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37dbfff640 1 --2- 172.21.15.83:0/2936504772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb0528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37e2d9d640 1 -- 172.21.15.83:0/2936504772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 msgr2=0x5630cb0528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37e2d9d640 1 --2- 172.21.15.83:0/2936504772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb0528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37e2d9d640 1 -- 172.21.15.83:0/2936504772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 msgr2=0x5630cb03b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37e2d9d640 1 --2- 172.21.15.83:0/2936504772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb03b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37e2d9d640 1 -- 172.21.15.83:0/2936504772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630caf44c20 con 0x5630cb046dd0 2024-03-11T19:52:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37e259c640 1 --2- 172.21.15.83:0/2936504772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb03b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37e2d9d640 1 --2- 172.21.15.83:0/2936504772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb046dd0 0x5630cb03b830 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009fd0 tx=0x7f37dc09a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f43f42cafb5294 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.143+0000 7f37cbfff640 1 -- 172.21.15.83:0/2936504772 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc09e9c0 con 0x5630cb046dd0 2024-03-11T19:52:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.144+0000 7f37cbfff640 1 -- 172.21.15.83:0/2936504772 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37dc0a3010 con 0x5630cb046dd0 2024-03-11T19:52:46.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.144+0000 7f37cbfff640 1 -- 172.21.15.83:0/2936504772 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0b33a0 con 0x5630cb046dd0 2024-03-11T19:52:46.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.144+0000 7f37eb574740 1 -- 172.21.15.83:0/2936504772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb046dd0 msgr2=0x5630cb03b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.144+0000 7f37eb574740 1 --2- 172.21.15.83:0/2936504772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb046dd0 0x5630cb03b830 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009fd0 tx=0x7f37dc09a290 comp rx=0 tx=0).stop 2024-03-11T19:52:46.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.144+0000 7f37eb574740 1 -- 172.21.15.83:0/2936504772 shutdown_connections 2024-03-11T19:52:46.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.144+0000 7f37eb574740 1 --2- 172.21.15.83:0/2936504772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb046dd0 0x5630cb03b830 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.144+0000 7f37eb574740 1 --2- 172.21.15.83:0/2936504772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb0528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.144+0000 7f37eb574740 1 --2- 172.21.15.83:0/2936504772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb041270 0x5630cb03b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.144+0000 7f37eb574740 1 -- 172.21.15.83:0/2936504772 >> 172.21.15.83:0/2936504772 conn(0x5630cb03c7e0 msgr2=0x5630cb043120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.145+0000 7f37eb574740 1 -- 172.21.15.83:0/2936504772 shutdown_connections 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.145+0000 7f37eb574740 1 -- 172.21.15.83:0/2936504772 wait complete. 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.145+0000 7f37eb574740 1 -- start start 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.145+0000 7f37eb574740 1 Processor -- start 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37eb574740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb052a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37eb574740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb051060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37e259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb052a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37e259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb052a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38616/0 (socket says 172.21.15.83:38616) 2024-03-11T19:52:46.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37eb574740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb046dd0 0x5630cb051670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb051060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb051060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52496/0 (socket says 172.21.15.83:52496) 2024-03-11T19:52:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37e259c640 1 -- 172.21.15.83:0/1805347957 learned_addr learned my addr 172.21.15.83:0/1805347957 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37e2d9d640 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb046dd0 0x5630cb051670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37eb574740 1 -- 172.21.15.83:0/1805347957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630cb051d70 con 0x5630cb041270 2024-03-11T19:52:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37eb574740 1 -- 172.21.15.83:0/1805347957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630cb051ee0 con 0x5630cb046dd0 2024-03-11T19:52:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37eb574740 1 -- 172.21.15.83:0/1805347957 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630cb0b5f90 con 0x5630cb041c60 2024-03-11T19:52:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.146+0000 7f37eb574740 10 client.0 _ll_register_callbacks cb 0x7f37dc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37dbfff640 1 -- 172.21.15.83:0/1805347957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb046dd0 msgr2=0x5630cb051670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37dbfff640 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb046dd0 0x5630cb051670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37dbfff640 1 -- 172.21.15.83:0/1805347957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 msgr2=0x5630cb052a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37dbfff640 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb052a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37dbfff640 1 -- 172.21.15.83:0/1805347957 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5630cafa86a0 con 0x5630cb041c60 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37e259c640 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb052a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37dbfff640 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb051060 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x5630cb059f30 tx=0x7f37d4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe1c6d140031c9a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1805347957 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4027030 con 0x5630cb041c60 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37eb574740 1 -- 172.21.15.83:0/1805347957 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37cc02cbe0 con 0x5630cb041c60 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1805347957 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f37d4029430 con 0x5630cb041c60 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1805347957 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37d40295f0 con 0x5630cb041c60 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37eb574740 10 client.27451 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.147+0000 7f37eb574740 1 -- 172.21.15.83:0/1805347957 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f37e41a1a00 con 0x5630cb041c60 2024-03-11T19:52:46.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.148+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1805347957 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37cc02cbe0 con 0x5630cb041c60 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.148+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1805347957 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f37d4010430 con 0x5630cb041c60 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.148+0000 7f37eb574740 10 client.27451 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.148+0000 7f37eb574740 10 client.27451 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.148+0000 7f37eb574740 20 client.27451 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.148+0000 7f37eb574740 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5630cb0571e0 0x5630cb0b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.148+0000 7f37eb574740 4 client.27451 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.148+0000 7f37eb574740 1 -- 172.21.15.83:0/1805347957 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5630cad35890 con 0x5630cb0571e0 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.148+0000 7f37e259c640 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5630cb0571e0 0x5630cb0b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.148+0000 7f37c9ffb640 10 client.27451 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.148+0000 7f37e259c640 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5630cb0571e0 0x5630cb0b9b40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70210e5653e623c4 server_cookie=a2b1580ec75593c3 in_seq=0 out_seq=0 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37c9ffb640 1 -- 172.21.15.83:0/1805347957 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5630cad35890 con 0x5630cb0571e0 2024-03-11T19:52:46.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37c9ffb640 10 client.27451 handle_command_reply: tid=0 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 1 client.27451 shutdown 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 1 -- 172.21.15.83:0/1805347957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 msgr2=0x5630cb051060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb051060 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x5630cb059f30 tx=0x7f37d4005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 1 -- 172.21.15.83:0/1805347957 shutdown_connections 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x5630cb0571e0 0x5630cb0b9b40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5630cb046dd0 0x5630cb051670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5630cb041c60 0x5630cb051060 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 1 --2- 172.21.15.83:0/1805347957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630cb041270 0x5630cb052a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37c9ffb640 0 client.27451 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 1 -- 172.21.15.83:0/1805347957 >> 172.21.15.83:0/1805347957 conn(0x5630cb03c7e0 msgr2=0x5630cb03b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 1 -- 172.21.15.83:0/1805347957 shutdown_connections 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 1 -- 172.21.15.83:0/1805347957 wait complete. 2024-03-11T19:52:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.149+0000 7f37eb574740 20 client.27451 trim_cache size 0 max 16384 2024-03-11T19:52:46.162 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:52:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 -- 172.21.15.83:0/3760170905 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37d00a1c00 msgr2=0x7f37d00a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37d00a1c00 0x7f37d00a40c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5630cb0424c0 tx=0x7f37d4009880 comp rx=0 tx=0).stop 2024-03-11T19:52:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 -- 172.21.15.83:0/3760170905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e4101650 msgr2=0x7f37e4102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e4101650 0x7f37e4102bd0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00ea80 tx=0x7f37dc00ef40 comp rx=0 tx=0).stop 2024-03-11T19:52:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 -- 172.21.15.83:0/3760170905 shutdown_connections 2024-03-11T19:52:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f37d00a1c00 0x7f37d00a40c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37e4101650 0x7f37e4102bd0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e4100c40 0x7f37e4102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 --2- 172.21.15.83:0/3760170905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e41002a0 0x7f37e4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 -- 172.21.15.83:0/3760170905 >> 172.21.15.83:0/3760170905 conn(0x7f37e40fad70 msgr2=0x7f37e40fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 -- 172.21.15.83:0/3760170905 shutdown_connections 2024-03-11T19:52:46.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.150+0000 7f37e88c3640 1 -- 172.21.15.83:0/3760170905 wait complete. 2024-03-11T19:52:46.167 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:52:46.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:52:46.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.306+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/3476431742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88100cf0 msgr2=0x7f9a88101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.306+0000 7f9a8dbe4640 1 --2- 172.21.15.83:0/3476431742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88100cf0 0x7f9a88101170 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9a80009ca0 tx=0x7f9a8002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.306+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/3476431742 shutdown_connections 2024-03-11T19:52:46.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.306+0000 7f9a8dbe4640 1 --2- 172.21.15.83:0/3476431742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88100cf0 0x7f9a88101170 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.306+0000 7f9a8dbe4640 1 --2- 172.21.15.83:0/3476431742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88100350 0x7f9a881007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.306+0000 7f9a8dbe4640 1 --2- 172.21.15.83:0/3476431742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a880ff9f0 0x7f9a880ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.306+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/3476431742 >> 172.21.15.83:0/3476431742 conn(0x7f9a880fb610 msgr2=0x7f9a880fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.306+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/3476431742 shutdown_connections 2024-03-11T19:52:46.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.306+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/3476431742 wait complete. 2024-03-11T19:52:46.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.306+0000 7f9a8dbe4640 1 Processor -- start 2024-03-11T19:52:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.307+0000 7f9a8dbe4640 1 -- start start 2024-03-11T19:52:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.307+0000 7f9a8dbe4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a880ff9f0 0x7f9a8806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.307+0000 7f9a8dbe4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88100350 0x7f9a8806ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.307+0000 7f9a8dbe4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88100cf0 0x7f9a881adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.307+0000 7f9a8dbe4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a88105c20 con 0x7f9a88100350 2024-03-11T19:52:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.307+0000 7f9a8dbe4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a88105d90 con 0x7f9a88100cf0 2024-03-11T19:52:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.307+0000 7f9a8dbe4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a88072f50 con 0x7f9a880ff9f0 2024-03-11T19:52:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.307+0000 7f9a877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a880ff9f0 0x7f9a8806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.307+0000 7f9a87fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88100cf0 0x7f9a881adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.307+0000 7f9a87fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88100cf0 0x7f9a881adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:60020/0 (socket says 172.21.15.83:60020) 2024-03-11T19:52:46.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.307+0000 7f9a87fff640 1 -- 172.21.15.83:0/2606429630 learned_addr learned my addr 172.21.15.83:0/2606429630 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:46.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.308+0000 7f9a87fff640 1 -- 172.21.15.83:0/2606429630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a880ff9f0 msgr2=0x7f9a8806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.308+0000 7f9a87fff640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a880ff9f0 0x7f9a8806f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.308+0000 7f9a87fff640 1 -- 172.21.15.83:0/2606429630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88100350 msgr2=0x7f9a8806ff00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-11T19:52:46.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.308+0000 7f9a87fff640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88100350 0x7f9a8806ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.308+0000 7f9a87fff640 1 -- 172.21.15.83:0/2606429630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a80009900 con 0x7f9a88100cf0 2024-03-11T19:52:46.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.308+0000 7f9a87fff640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88100cf0 0x7f9a881adde0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a80037040 tx=0x7f9a80009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.308+0000 7f9a84ff9640 1 -- 172.21.15.83:0/2606429630 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a800329b0 con 0x7f9a88100cf0 2024-03-11T19:52:46.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.308+0000 7f9a84ff9640 1 -- 172.21.15.83:0/2606429630 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a80003e50 con 0x7f9a88100cf0 2024-03-11T19:52:46.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.308+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/2606429630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a880731d0 con 0x7f9a88100cf0 2024-03-11T19:52:46.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.309+0000 7f9a84ff9640 1 -- 172.21.15.83:0/2606429630 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80040450 con 0x7f9a88100cf0 2024-03-11T19:52:46.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.309+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/2606429630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a881a5800 con 0x7f9a88100cf0 2024-03-11T19:52:46.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.311+0000 7f9a84ff9640 1 -- 172.21.15.83:0/2606429630 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9a8000a040 con 0x7f9a88100cf0 2024-03-11T19:52:46.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.311+0000 7f9a907b0740 1 Processor -- start 2024-03-11T19:52:46.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.311+0000 7f9a907b0740 1 -- start start 2024-03-11T19:52:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.312+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x56439424b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.312+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x564394250760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.312+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x564394253260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.312+0000 7f9a907b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564394207960 con 0x564394250dd0 2024-03-11T19:52:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.312+0000 7f9a907b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643941bb310 con 0x56439424bc60 2024-03-11T19:52:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.312+0000 7f9a907b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643941b8840 con 0x56439424b270 2024-03-11T19:52:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.312+0000 7f9a86ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x564394253260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.312+0000 7f9a86ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x564394253260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38644/0 (socket says 172.21.15.83:38644) 2024-03-11T19:52:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.312+0000 7f9a86ffd640 1 -- 172.21.15.83:0/957935574 learned_addr learned my addr 172.21.15.83:0/957935574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.312+0000 7f9a877fe640 1 --2- 172.21.15.83:0/957935574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x564394250760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.312+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/957935574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x56439424b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.313+0000 7f9a86ffd640 1 -- 172.21.15.83:0/957935574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 msgr2=0x564394250760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.313+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/957935574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x564394250760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.313+0000 7f9a86ffd640 1 -- 172.21.15.83:0/957935574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 msgr2=0x564394253260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.313+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/957935574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x564394253260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.313+0000 7f9a86ffd640 1 -- 172.21.15.83:0/957935574 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564394176060 con 0x56439424b270 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.314+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/957935574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x56439424b690 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0137e0 tx=0x7f9a7c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e1aacf57a65743 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.314+0000 7f9a84ff9640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a700a1ab0 0x7f9a700a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.314+0000 7f9a6dffb640 1 -- 172.21.15.83:0/957935574 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c013310 con 0x56439424b270 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.314+0000 7f9a6dffb640 1 -- 172.21.15.83:0/957935574 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a7c013310 con 0x56439424b270 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.314+0000 7f9a6dffb640 1 -- 172.21.15.83:0/957935574 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c010c70 con 0x56439424b270 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.314+0000 7f9a84ff9640 1 -- 172.21.15.83:0/2606429630 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9a800f0210 con 0x7f9a88100cf0 2024-03-11T19:52:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.314+0000 7f9a877fe640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a700a1ab0 0x7f9a700a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a907b0740 1 -- 172.21.15.83:0/957935574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 msgr2=0x56439424b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a907b0740 1 --2- 172.21.15.83:0/957935574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x56439424b690 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0137e0 tx=0x7f9a7c005850 comp rx=0 tx=0).stop 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a877fe640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a700a1ab0 0x7f9a700a3f70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x56439424c060 tx=0x7f9a78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a907b0740 1 -- 172.21.15.83:0/957935574 shutdown_connections 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a907b0740 1 --2- 172.21.15.83:0/957935574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x564394253260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a907b0740 1 --2- 172.21.15.83:0/957935574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x564394250760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a907b0740 1 --2- 172.21.15.83:0/957935574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x56439424b690 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a907b0740 1 -- 172.21.15.83:0/957935574 >> 172.21.15.83:0/957935574 conn(0x5643942467e0 msgr2=0x564394246be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a907b0740 1 -- 172.21.15.83:0/957935574 shutdown_connections 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a907b0740 1 -- 172.21.15.83:0/957935574 wait complete. 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a907b0740 1 -- start start 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.315+0000 7f9a907b0740 1 Processor -- start 2024-03-11T19:52:46.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x564394255420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x564394255960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x56439425c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a907b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643941b4020 con 0x564394250dd0 2024-03-11T19:52:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a907b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564393e87430 con 0x56439424bc60 2024-03-11T19:52:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a907b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564393e43e00 con 0x56439424b270 2024-03-11T19:52:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a907b0740 10 client.0 _ll_register_callbacks cb 0x56439415bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x564394255420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a877fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x564394255420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52544/0 (socket says 172.21.15.83:52544) 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a877fe640 1 -- 172.21.15.83:0/3174626955 learned_addr learned my addr 172.21.15.83:0/3174626955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a87fff640 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x56439425c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x564394255960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a877fe640 1 -- 172.21.15.83:0/3174626955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 msgr2=0x564394255960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a877fe640 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x564394255960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a877fe640 1 -- 172.21.15.83:0/3174626955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 msgr2=0x56439425c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a877fe640 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x56439425c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.316+0000 7f9a877fe640 1 -- 172.21.15.83:0/3174626955 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c0134c0 con 0x56439424b270 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.317+0000 7f9a877fe640 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x564394255420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9a78004600 tx=0x7f9a78004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c7129d08121373 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.317+0000 7f9a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.317+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3174626955 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78017070 con 0x56439424b270 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.317+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3174626955 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9a78005020 con 0x56439424b270 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.317+0000 7f9a87fff640 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x56439425c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.317+0000 7f9a907b0740 1 -- 172.21.15.83:0/3174626955 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564393eb37e0 con 0x56439424b270 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.317+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x564394255960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.317+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3174626955 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a78009e60 con 0x56439424b270 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.317+0000 7f9a907b0740 10 client.27463 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.317+0000 7f9a907b0740 1 -- 172.21.15.83:0/3174626955 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5643941b26a0 con 0x56439424b270 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.318+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3174626955 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564393eb37e0 con 0x56439424b270 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.318+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3174626955 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a7801e680 con 0x56439424b270 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.318+0000 7f9a907b0740 10 client.27463 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:46.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.318+0000 7f9a907b0740 10 client.27463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.318+0000 7f9a907b0740 20 client.27463 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.318+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564394261640 0x564394263a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.318+0000 7f9a907b0740 4 client.27463 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.318+0000 7f9a907b0740 1 -- 172.21.15.83:0/3174626955 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564393ef0720 con 0x564394261640 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.318+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564394261640 0x564394263a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a66ffd640 10 client.27463 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564394261640 0x564394263a60 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db7014f7c9246944 server_cookie=1dd94fb3549a44a8 in_seq=0 out_seq=0 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3174626955 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564393ef0720 con 0x564394261640 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a66ffd640 10 client.27463 handle_command_reply: tid=0 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a907b0740 1 client.27463 shutdown 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a907b0740 1 -- 172.21.15.83:0/3174626955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 msgr2=0x564394255420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x564394255420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9a78004600 tx=0x7f9a78004730 comp rx=0 tx=0).stop 2024-03-11T19:52:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a907b0740 1 -- 172.21.15.83:0/3174626955 shutdown_connections 2024-03-11T19:52:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x564394261640 0x564394263a60 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a66ffd640 0 client.27463 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x56439425c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x564394255960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3174626955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x564394255420 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.319+0000 7f9a907b0740 1 -- 172.21.15.83:0/3174626955 >> 172.21.15.83:0/3174626955 conn(0x5643942467e0 msgr2=0x564394245b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.320+0000 7f9a907b0740 1 -- 172.21.15.83:0/3174626955 shutdown_connections 2024-03-11T19:52:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.320+0000 7f9a907b0740 1 -- 172.21.15.83:0/3174626955 wait complete. 2024-03-11T19:52:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.320+0000 7f9a907b0740 20 client.27463 trim_cache size 0 max 16384 2024-03-11T19:52:46.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.337+0000 7f9a907b0740 1 Processor -- start 2024-03-11T19:52:46.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.337+0000 7f9a907b0740 1 -- start start 2024-03-11T19:52:46.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.337+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56439424b270 0x564394246bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.337+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x56439425c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.337+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564394250dd0 0x56439424d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.337+0000 7f9a907b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x564394139130 con 0x56439424b270 2024-03-11T19:52:46.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a907b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564394246650 con 0x56439424bc60 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a907b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439424c090 con 0x564394250dd0 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56439424b270 0x564394246bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a87fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564394250dd0 0x56439424d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a87fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564394250dd0 0x56439424d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52558/0 (socket says 172.21.15.83:52558) 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56439424b270 0x564394246bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38668/0 (socket says 172.21.15.83:38668) 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a87fff640 1 -- 172.21.15.83:0/929760418 learned_addr learned my addr 172.21.15.83:0/929760418 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/929760418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x56439425c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a86ffd640 1 -- 172.21.15.83:0/929760418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564394250dd0 msgr2=0x56439424d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/929760418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564394250dd0 0x56439424d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a86ffd640 1 -- 172.21.15.83:0/929760418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56439424b270 msgr2=0x564394246bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/929760418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56439424b270 0x564394246bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a86ffd640 1 -- 172.21.15.83:0/929760418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56439414ec20 con 0x56439424bc60 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a87fff640 1 --2- 172.21.15.83:0/929760418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564394250dd0 0x56439424d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:46.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a877fe640 1 --2- 172.21.15.83:0/929760418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56439424b270 0x564394246bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/929760418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x56439425c8f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c004700 tx=0x7f9a7c009a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b39a6975eea0bdf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a6d7fa640 1 -- 172.21.15.83:0/929760418 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c024040 con 0x56439424bc60 2024-03-11T19:52:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a6d7fa640 1 -- 172.21.15.83:0/929760418 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564393ef0720 con 0x56439424bc60 2024-03-11T19:52:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a6d7fa640 1 -- 172.21.15.83:0/929760418 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c010c70 con 0x56439424bc60 2024-03-11T19:52:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a907b0740 1 -- 172.21.15.83:0/929760418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 msgr2=0x56439425c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a907b0740 1 --2- 172.21.15.83:0/929760418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x56439425c8f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c004700 tx=0x7f9a7c009a30 comp rx=0 tx=0).stop 2024-03-11T19:52:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a907b0740 1 -- 172.21.15.83:0/929760418 shutdown_connections 2024-03-11T19:52:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a907b0740 1 --2- 172.21.15.83:0/929760418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564394250dd0 0x56439424d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a907b0740 1 --2- 172.21.15.83:0/929760418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x56439425c8f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a907b0740 1 --2- 172.21.15.83:0/929760418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56439424b270 0x564394246bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a907b0740 1 -- 172.21.15.83:0/929760418 >> 172.21.15.83:0/929760418 conn(0x5643942467e0 msgr2=0x564394241a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a907b0740 1 -- 172.21.15.83:0/929760418 shutdown_connections 2024-03-11T19:52:46.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.338+0000 7f9a907b0740 1 -- 172.21.15.83:0/929760418 wait complete. 2024-03-11T19:52:46.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a907b0740 1 -- start start 2024-03-11T19:52:46.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a907b0740 1 Processor -- start 2024-03-11T19:52:46.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x56439425c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x56439425aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a907b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x56439425b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a907b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439425bbe0 con 0x564394250dd0 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a907b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56439425bd50 con 0x56439424bc60 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a907b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643942b7ba0 con 0x56439424b270 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a907b0740 10 client.0 _ll_register_callbacks cb 0x7f9a7c009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a87fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x56439425b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a87fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x56439425b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38684/0 (socket says 172.21.15.83:38684) 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.340+0000 7f9a87fff640 1 -- 172.21.15.83:0/3205342862 learned_addr learned my addr 172.21.15.83:0/3205342862 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.341+0000 7f9a86ffd640 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x56439425aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.341+0000 7f9a877fe640 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x56439425c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.341+0000 7f9a87fff640 1 -- 172.21.15.83:0/3205342862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 msgr2=0x56439425c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.341+0000 7f9a87fff640 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x56439425c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.341+0000 7f9a87fff640 1 -- 172.21.15.83:0/3205342862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 msgr2=0x56439425aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:46.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.341+0000 7f9a87fff640 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x56439425aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.341+0000 7f9a87fff640 1 -- 172.21.15.83:0/3205342862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643941b26a0 con 0x564394250dd0 2024-03-11T19:52:46.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.341+0000 7f9a87fff640 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x56439425b490 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9a800043b0 tx=0x7f9a800bd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342dad38f4d8f64c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:46.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.341+0000 7f9a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:52:46.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.342+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3205342862 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a800c1370 con 0x564394250dd0 2024-03-11T19:52:46.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.342+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3205342862 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9a800c5930 con 0x564394250dd0 2024-03-11T19:52:46.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.342+0000 7f9a907b0740 1 -- 172.21.15.83:0/3205342862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a7c013310 con 0x564394250dd0 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.342+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3205342862 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a800bd480 con 0x564394250dd0 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.342+0000 7f9a907b0740 10 client.17901 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.342+0000 7f9a907b0740 1 -- 172.21.15.83:0/3205342862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f9a881a5800 con 0x564394250dd0 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.343+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3205342862 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a7c013310 con 0x564394250dd0 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.343+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3205342862 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9a80004b40 con 0x564394250dd0 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.343+0000 7f9a907b0740 10 client.17901 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.343+0000 7f9a907b0740 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.343+0000 7f9a907b0740 20 client.17901 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.343+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5643942b9380 0x5643942bb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.343+0000 7f9a907b0740 4 client.17901 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.343+0000 7f9a907b0740 1 -- 172.21.15.83:0/3205342862 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564393f3f890 con 0x5643942b9380 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.343+0000 7f9a877fe640 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5643942b9380 0x5643942bb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a877fe640 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5643942b9380 0x5643942bb770 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ca2663d614fb47 server_cookie=21af2fba62109f9d in_seq=0 out_seq=0 2024-03-11T19:52:46.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a66ffd640 10 client.17901 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a66ffd640 1 -- 172.21.15.83:0/3205342862 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564393f3f890 con 0x5643942b9380 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a66ffd640 10 client.17901 handle_command_reply: tid=0 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 1 client.17901 shutdown 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 1 -- 172.21.15.83:0/3205342862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 msgr2=0x56439425b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x56439425b490 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9a800043b0 tx=0x7f9a800bd920 comp rx=0 tx=0).stop 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 1 -- 172.21.15.83:0/3205342862 shutdown_connections 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5643942b9380 0x5643942bb770 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564394250dd0 0x56439425b490 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56439424bc60 0x56439425aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 1 --2- 172.21.15.83:0/3205342862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56439424b270 0x56439425c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 1 -- 172.21.15.83:0/3205342862 >> 172.21.15.83:0/3205342862 conn(0x5643942467e0 msgr2=0x56439424d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a66ffd640 0 client.17901 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 1 -- 172.21.15.83:0/3205342862 shutdown_connections 2024-03-11T19:52:46.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 1 -- 172.21.15.83:0/3205342862 wait complete. 2024-03-11T19:52:46.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a907b0740 20 client.17901 trim_cache size 0 max 16384 2024-03-11T19:52:46.356 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/2606429630 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a700a1ab0 msgr2=0x7f9a700a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a700a1ab0 0x7f9a700a3f70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x56439424c060 tx=0x7f9a78005f90 comp rx=0 tx=0).stop 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/2606429630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88100cf0 msgr2=0x7f9a881adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88100cf0 0x7f9a881adde0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9a80037040 tx=0x7f9a80009360 comp rx=0 tx=0).stop 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/2606429630 shutdown_connections 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9a700a1ab0 0x7f9a700a3f70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a88100cf0 0x7f9a881adde0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a88100350 0x7f9a8806ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 --2- 172.21.15.83:0/2606429630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a880ff9f0 0x7f9a8806f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/2606429630 >> 172.21.15.83:0/2606429630 conn(0x7f9a880fb610 msgr2=0x7f9a880fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/2606429630 shutdown_connections 2024-03-11T19:52:46.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:46.344+0000 7f9a8dbe4640 1 -- 172.21.15.83:0/2606429630 wait complete. 2024-03-11T19:52:46.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:46 smithi083 ceph-mon[31864]: pgmap v755: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 745 op/s 2024-03-11T19:52:46.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:46 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/740557168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:46.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:46 smithi137 ceph-mon[36220]: pgmap v755: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 745 op/s 2024-03-11T19:52:46.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:46 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/740557168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:46.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:46 smithi204 ceph-mon[36739]: pgmap v755: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 15 MiB/s wr, 745 op/s 2024-03-11T19:52:46.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:46 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/740557168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:52:47.365 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:52:47.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:52:47.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:52:47.498 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 810.0 s, 1594.3 tps, lat 63.306 ms stddev 67.915 2024-03-11T19:52:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.520+0000 7f70762c3640 1 -- 172.21.15.83:0/2540048692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7070108180 msgr2=0x7f707010a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.520+0000 7f70762c3640 1 --2- 172.21.15.83:0/2540048692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7070108180 0x7f707010a5b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7068009900 tx=0x7f706802bc80 comp rx=0 tx=0).stop 2024-03-11T19:52:47.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.523+0000 7f70762c3640 1 -- 172.21.15.83:0/2540048692 shutdown_connections 2024-03-11T19:52:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.523+0000 7f70762c3640 1 --2- 172.21.15.83:0/2540048692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f707010d4a0 0x7f707010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.523+0000 7f70762c3640 1 --2- 172.21.15.83:0/2540048692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f707010aaf0 0x7f707010cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.523+0000 7f70762c3640 1 --2- 172.21.15.83:0/2540048692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7070108180 0x7f707010a5b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.523+0000 7f70762c3640 1 -- 172.21.15.83:0/2540048692 >> 172.21.15.83:0/2540048692 conn(0x7f707006dfe0 msgr2=0x7f7070070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.525+0000 7f70762c3640 1 -- 172.21.15.83:0/2540048692 shutdown_connections 2024-03-11T19:52:47.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.525+0000 7f70762c3640 1 -- 172.21.15.83:0/2540048692 wait complete. 2024-03-11T19:52:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.526+0000 7f70762c3640 1 Processor -- start 2024-03-11T19:52:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.526+0000 7f70762c3640 1 -- start start 2024-03-11T19:52:47.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.527+0000 7f70762c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7070108180 0x7f70701a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.527+0000 7f70762c3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f707010aaf0 0x7f70701a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.527+0000 7f706ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7070108180 0x7f70701a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.527+0000 7f706ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7070108180 0x7f70701a6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38696/0 (socket says 172.21.15.83:38696) 2024-03-11T19:52:47.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.527+0000 7f706ffff640 1 -- 172.21.15.83:0/3143270580 learned_addr learned my addr 172.21.15.83:0/3143270580 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:47.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.527+0000 7f70762c3640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f707010d4a0 0x7f70701ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.527+0000 7f70762c3640 1 -- 172.21.15.83:0/3143270580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70701adbf0 con 0x7f7070108180 2024-03-11T19:52:47.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.527+0000 7f70762c3640 1 -- 172.21.15.83:0/3143270580 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70701add60 con 0x7f707010aaf0 2024-03-11T19:52:47.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.527+0000 7f70762c3640 1 -- 172.21.15.83:0/3143270580 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70701adf20 con 0x7f707010d4a0 2024-03-11T19:52:47.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.527+0000 7f706f7fe640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f707010aaf0 0x7f70701a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.528+0000 7f7074860640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f707010d4a0 0x7f70701ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.528+0000 7f706ffff640 1 -- 172.21.15.83:0/3143270580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f707010d4a0 msgr2=0x7f70701ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.528+0000 7f706ffff640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f707010d4a0 0x7f70701ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.528+0000 7f706ffff640 1 -- 172.21.15.83:0/3143270580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f707010aaf0 msgr2=0x7f70701a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.528+0000 7f706ffff640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f707010aaf0 0x7f70701a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.528+0000 7f706ffff640 1 -- 172.21.15.83:0/3143270580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7068009560 con 0x7f7070108180 2024-03-11T19:52:47.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.528+0000 7f706ffff640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7070108180 0x7f70701a6100 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7068006010 tx=0x7f70680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.528+0000 7f706d7fa640 1 -- 172.21.15.83:0/3143270580 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70680044e0 con 0x7f7070108180 2024-03-11T19:52:47.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.528+0000 7f706d7fa640 1 -- 172.21.15.83:0/3143270580 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7068004aa0 con 0x7f7070108180 2024-03-11T19:52:47.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.528+0000 7f70762c3640 1 -- 172.21.15.83:0/3143270580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70701ae1a0 con 0x7f7070108180 2024-03-11T19:52:47.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.529+0000 7f706d7fa640 1 -- 172.21.15.83:0/3143270580 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706803e450 con 0x7f7070108180 2024-03-11T19:52:47.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.529+0000 7f70762c3640 1 -- 172.21.15.83:0/3143270580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70701ae780 con 0x7f7070108180 2024-03-11T19:52:47.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.531+0000 7f706d7fa640 1 -- 172.21.15.83:0/3143270580 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f70680035f0 con 0x7f7070108180 2024-03-11T19:52:47.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.532+0000 7f7078f73740 1 Processor -- start 2024-03-11T19:52:47.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.532+0000 7f7078f73740 1 -- start start 2024-03-11T19:52:47.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.532+0000 7f706d7fa640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f705c0a1b30 0x7f705c0a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.532+0000 7f7074860640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f705c0a1b30 0x7f705c0a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f706d7fa640 1 -- 172.21.15.83:0/3143270580 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f70680ef690 con 0x7f7070108180 2024-03-11T19:52:47.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f7074860640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f705c0a1b30 0x7f705c0a3ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f707006ae70 tx=0x7f7058008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f7078f73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb01a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f7078f73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01b690 0x55b2eb01bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f7078f73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01c050 0x55b2eb0228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f7078f73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eaf89f90 con 0x55b2eb01a4c0 2024-03-11T19:52:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f7078f73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eaf874c0 con 0x55b2eb01b690 2024-03-11T19:52:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f7078f73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eaf82ca0 con 0x55b2eb01c050 2024-03-11T19:52:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f706ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb01a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f706ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb01a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38706/0 (socket says 172.21.15.83:38706) 2024-03-11T19:52:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f7074860640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01c050 0x55b2eb0228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f706ffff640 1 -- 172.21.15.83:0/2801045506 learned_addr learned my addr 172.21.15.83:0/2801045506 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:47.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.533+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01b690 0x55b2eb01bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.534+0000 7f706ffff640 1 -- 172.21.15.83:0/2801045506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01c050 msgr2=0x55b2eb0228e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.534+0000 7f706ffff640 1 --2- 172.21.15.83:0/2801045506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01c050 0x55b2eb0228e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.534+0000 7f706ffff640 1 -- 172.21.15.83:0/2801045506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01b690 msgr2=0x55b2eb01bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.534+0000 7f706ffff640 1 --2- 172.21.15.83:0/2801045506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01b690 0x55b2eb01bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.534+0000 7f706ffff640 1 -- 172.21.15.83:0/2801045506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2eaf450b0 con 0x55b2eb01a4c0 2024-03-11T19:52:47.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.534+0000 7f706ffff640 1 --2- 172.21.15.83:0/2801045506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb01a8e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f70680b8a70 tx=0x7f7068003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad68a712ade9417 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.534+0000 7f70667fc640 1 -- 172.21.15.83:0/2801045506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70680c0760 con 0x55b2eb01a4c0 2024-03-11T19:52:47.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.534+0000 7f70667fc640 1 -- 172.21.15.83:0/2801045506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70680c4db0 con 0x55b2eb01a4c0 2024-03-11T19:52:47.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.534+0000 7f70667fc640 1 -- 172.21.15.83:0/2801045506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70680a70f0 con 0x55b2eb01a4c0 2024-03-11T19:52:47.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.535+0000 7f7078f73740 1 -- 172.21.15.83:0/2801045506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 msgr2=0x55b2eb01a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.535+0000 7f7078f73740 1 --2- 172.21.15.83:0/2801045506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb01a8e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f70680b8a70 tx=0x7f7068003990 comp rx=0 tx=0).stop 2024-03-11T19:52:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.535+0000 7f7078f73740 1 -- 172.21.15.83:0/2801045506 shutdown_connections 2024-03-11T19:52:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.536+0000 7f7078f73740 1 --2- 172.21.15.83:0/2801045506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01c050 0x55b2eb0228e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.536+0000 7f7078f73740 1 --2- 172.21.15.83:0/2801045506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01b690 0x55b2eb01bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.536+0000 7f7078f73740 1 --2- 172.21.15.83:0/2801045506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb01a8e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.536+0000 7f7078f73740 1 -- 172.21.15.83:0/2801045506 >> 172.21.15.83:0/2801045506 conn(0x55b2eb0156a0 msgr2=0x55b2eb015aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.536+0000 7f7078f73740 1 -- 172.21.15.83:0/2801045506 shutdown_connections 2024-03-11T19:52:47.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.536+0000 7f7078f73740 1 -- 172.21.15.83:0/2801045506 wait complete. 2024-03-11T19:52:47.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.536+0000 7f7078f73740 1 -- start start 2024-03-11T19:52:47.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.537+0000 7f7078f73740 1 Processor -- start 2024-03-11T19:52:47.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f7078f73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb024200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f7078f73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01b690 0x55b2eb024740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f7078f73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01c050 0x55b2eb02b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f7078f73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eaf27f40 con 0x55b2eb01a4c0 2024-03-11T19:52:47.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f7078f73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eac564b0 con 0x55b2eb01c050 2024-03-11T19:52:47.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f7078f73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eaf08670 con 0x55b2eb01b690 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f7078f73740 10 client.0 _ll_register_callbacks cb 0x55b2eaf75f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f706ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb024200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f706ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb024200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38718/0 (socket says 172.21.15.83:38718) 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f706ffff640 1 -- 172.21.15.83:0/3147363570 learned_addr learned my addr 172.21.15.83:0/3147363570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f706f7fe640 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01b690 0x55b2eb024740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f7074860640 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01c050 0x55b2eb02b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f706ffff640 1 -- 172.21.15.83:0/3147363570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01b690 msgr2=0x55b2eb024740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f706ffff640 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01b690 0x55b2eb024740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f706ffff640 1 -- 172.21.15.83:0/3147363570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01c050 msgr2=0x55b2eb02b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f706ffff640 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01c050 0x55b2eb02b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f706ffff640 1 -- 172.21.15.83:0/3147363570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70701ae780 con 0x55b2eb01a4c0 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.538+0000 7f706ffff640 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb024200 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f70680c4880 tx=0x7f70680b81f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a9e1bdd9a0dd7e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.539+0000 7f704f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:52:47.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.539+0000 7f704f7fe640 1 -- 172.21.15.83:0/3147363570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70680c4db0 con 0x55b2eb01a4c0 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.539+0000 7f7078f73740 1 -- 172.21.15.83:0/3147363570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2eaf5cf40 con 0x55b2eb01a4c0 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.539+0000 7f704f7fe640 1 -- 172.21.15.83:0/3147363570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f70680c0760 con 0x55b2eb01a4c0 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.539+0000 7f704f7fe640 1 -- 172.21.15.83:0/3147363570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70680d5170 con 0x55b2eb01a4c0 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.539+0000 7f7078f73740 10 client.17910 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.539+0000 7f7078f73740 1 -- 172.21.15.83:0/3147363570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b2eaf81320 con 0x55b2eb01a4c0 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.539+0000 7f704f7fe640 1 -- 172.21.15.83:0/3147363570 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70680c4920 con 0x55b2eb01a4c0 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.539+0000 7f704f7fe640 1 -- 172.21.15.83:0/3147363570 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f706803c070 con 0x55b2eb01a4c0 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.540+0000 7f7078f73740 10 client.17910 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.540+0000 7f7078f73740 10 client.17910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.540+0000 7f7078f73740 20 client.17910 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.540+0000 7f7078f73740 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b2eb02fed0 0x55b2eb0322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.540+0000 7f7078f73740 4 client.17910 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:47.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.540+0000 7f7078f73740 1 -- 172.21.15.83:0/3147363570 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2eaf5cf40 con 0x55b2eb02fed0 2024-03-11T19:52:47.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.540+0000 7f706f7fe640 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b2eb02fed0 0x55b2eb0322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.540+0000 7f704f7fe640 10 client.17910 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.540+0000 7f706f7fe640 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b2eb02fed0 0x55b2eb0322c0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c4887970a5a37c8 server_cookie=19b1d3416c81f171 in_seq=0 out_seq=0 2024-03-11T19:52:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.541+0000 7f704f7fe640 1 -- 172.21.15.83:0/3147363570 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b2eaf5cf40 con 0x55b2eb02fed0 2024-03-11T19:52:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.541+0000 7f704f7fe640 10 client.17910 handle_command_reply: tid=0 2024-03-11T19:52:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.541+0000 7f7078f73740 1 client.17910 shutdown 2024-03-11T19:52:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f7078f73740 1 -- 172.21.15.83:0/3147363570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 msgr2=0x55b2eb024200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f7078f73740 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb024200 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f70680c4880 tx=0x7f70680b81f0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f7078f73740 1 -- 172.21.15.83:0/3147363570 shutdown_connections 2024-03-11T19:52:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f7078f73740 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b2eb02fed0 0x55b2eb0322c0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f704f7fe640 0 client.17910 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f7078f73740 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01c050 0x55b2eb02b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f7078f73740 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01b690 0x55b2eb024740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f7078f73740 1 --2- 172.21.15.83:0/3147363570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb024200 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f7078f73740 1 -- 172.21.15.83:0/3147363570 >> 172.21.15.83:0/3147363570 conn(0x55b2eb0156a0 msgr2=0x55b2eb0148e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f7078f73740 1 -- 172.21.15.83:0/3147363570 shutdown_connections 2024-03-11T19:52:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f7078f73740 1 -- 172.21.15.83:0/3147363570 wait complete. 2024-03-11T19:52:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.542+0000 7f7078f73740 20 client.17910 trim_cache size 0 max 16384 2024-03-11T19:52:47.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.562+0000 7f7078f73740 1 Processor -- start 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.562+0000 7f7078f73740 1 -- start start 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.562+0000 7f7078f73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb0156a0 0x55b2eb023e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.562+0000 7f7078f73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.562+0000 7f7078f73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01b690 0x55b2eb024350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.562+0000 7f7078f73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eb0160f0 con 0x55b2eb01a4c0 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.562+0000 7f7078f73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eb00a350 con 0x55b2eb0156a0 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.562+0000 7f7078f73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eaf5cf40 con 0x55b2eb01b690 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38734/0 (socket says 172.21.15.83:38734) 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f706f7fe640 1 -- 172.21.15.83:0/3857493758 learned_addr learned my addr 172.21.15.83:0/3857493758 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f706ffff640 1 --2- 172.21.15.83:0/3857493758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb0156a0 0x55b2eb023e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f7074860640 1 --2- 172.21.15.83:0/3857493758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01b690 0x55b2eb024350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f706f7fe640 1 -- 172.21.15.83:0/3857493758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01b690 msgr2=0x55b2eb024350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f706f7fe640 1 --2- 172.21.15.83:0/3857493758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01b690 0x55b2eb024350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f706f7fe640 1 -- 172.21.15.83:0/3857493758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb0156a0 msgr2=0x55b2eb023e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f706f7fe640 1 --2- 172.21.15.83:0/3857493758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb0156a0 0x55b2eb023e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f706f7fe640 1 -- 172.21.15.83:0/3857493758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2eaf1dc00 con 0x55b2eb01a4c0 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f706f7fe640 1 --2- 172.21.15.83:0/3857493758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02b710 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x55b2eb016610 tx=0x7f70600119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e476d1c6db9e17 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f7065ffb640 1 -- 172.21.15.83:0/3857493758 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060016040 con 0x55b2eb01a4c0 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f7065ffb640 1 -- 172.21.15.83:0/3857493758 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7060018040 con 0x55b2eb01a4c0 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.563+0000 7f7065ffb640 1 -- 172.21.15.83:0/3857493758 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060004020 con 0x55b2eb01a4c0 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.564+0000 7f7078f73740 1 -- 172.21.15.83:0/3857493758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 msgr2=0x55b2eb02b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.564+0000 7f7078f73740 1 --2- 172.21.15.83:0/3857493758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02b710 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x55b2eb016610 tx=0x7f70600119b0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.564+0000 7f7078f73740 1 -- 172.21.15.83:0/3857493758 shutdown_connections 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.564+0000 7f7078f73740 1 --2- 172.21.15.83:0/3857493758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb01b690 0x55b2eb024350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.564+0000 7f7078f73740 1 --2- 172.21.15.83:0/3857493758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02b710 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.564+0000 7f7078f73740 1 --2- 172.21.15.83:0/3857493758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb0156a0 0x55b2eb023e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.564+0000 7f7078f73740 1 -- 172.21.15.83:0/3857493758 >> 172.21.15.83:0/3857493758 conn(0x55b2eb0229d0 msgr2=0x55b2eb01eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.564+0000 7f7078f73740 1 -- 172.21.15.83:0/3857493758 shutdown_connections 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.564+0000 7f7078f73740 1 -- 172.21.15.83:0/3857493758 wait complete. 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.564+0000 7f7078f73740 1 -- start start 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f7078f73740 1 Processor -- start 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f7078f73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb0156a0 0x55b2eb029ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f7078f73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f7078f73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01b690 0x55b2eb02a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f7078f73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eb02bb10 con 0x55b2eb01a4c0 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f7078f73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eb0863e0 con 0x55b2eb01b690 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f7078f73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2eb0865a0 con 0x55b2eb0156a0 2024-03-11T19:52:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f706ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb0156a0 0x55b2eb029ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f7078f73740 10 client.0 _ll_register_callbacks cb 0x7f7068005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38742/0 (socket says 172.21.15.83:38742) 2024-03-11T19:52:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f706ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb0156a0 0x55b2eb029ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:52630/0 (socket says 172.21.15.83:52630) 2024-03-11T19:52:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f706f7fe640 1 -- 172.21.15.83:0/699232530 learned_addr learned my addr 172.21.15.83:0/699232530 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f7074860640 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01b690 0x55b2eb02a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f706f7fe640 1 -- 172.21.15.83:0/699232530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb0156a0 msgr2=0x55b2eb029ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f706f7fe640 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb0156a0 0x55b2eb029ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f706f7fe640 1 -- 172.21.15.83:0/699232530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01b690 msgr2=0x55b2eb02a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.565+0000 7f706f7fe640 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01b690 0x55b2eb02a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.566+0000 7f706f7fe640 1 -- 172.21.15.83:0/699232530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706000ace0 con 0x55b2eb01a4c0 2024-03-11T19:52:47.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.566+0000 7f706ffff640 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb0156a0 0x55b2eb029ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:52:47.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.566+0000 7f706f7fe640 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02a400 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7060003f80 tx=0x7f706000aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cc8e09e7d3417f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.566+0000 7f704f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:52:47.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.566+0000 7f704f7fe640 1 -- 172.21.15.83:0/699232530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060016070 con 0x55b2eb01a4c0 2024-03-11T19:52:47.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.566+0000 7f704f7fe640 1 -- 172.21.15.83:0/699232530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f706001d840 con 0x55b2eb01a4c0 2024-03-11T19:52:47.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.566+0000 7f7078f73740 1 -- 172.21.15.83:0/699232530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2eafa8900 con 0x55b2eb01a4c0 2024-03-11T19:52:47.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.567+0000 7f704f7fe640 1 -- 172.21.15.83:0/699232530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7060004a40 con 0x55b2eb01a4c0 2024-03-11T19:52:47.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.568+0000 7f7078f73740 10 client.17916 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:47.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.568+0000 7f7078f73740 1 -- 172.21.15.83:0/699232530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b2eaf43910 con 0x55b2eb01a4c0 2024-03-11T19:52:47.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.568+0000 7f704f7fe640 1 -- 172.21.15.83:0/699232530 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b2eafa8900 con 0x55b2eb01a4c0 2024-03-11T19:52:47.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.568+0000 7f704f7fe640 1 -- 172.21.15.83:0/699232530 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7060005100 con 0x55b2eb01a4c0 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.568+0000 7f7078f73740 10 client.17916 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.568+0000 7f7078f73740 10 client.17916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.568+0000 7f7078f73740 20 client.17916 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.568+0000 7f7078f73740 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b2eb01c050 0x55b2eb08a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.568+0000 7f7078f73740 4 client.17916 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.568+0000 7f7078f73740 1 -- 172.21.15.83:0/699232530 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b2eaf79310 con 0x55b2eb01c050 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.568+0000 7f706ffff640 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b2eb01c050 0x55b2eb08a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.569+0000 7f706ffff640 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b2eb01c050 0x55b2eb08a170 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae10d285ec9a2c79 server_cookie=efa3e8a1256f697c in_seq=0 out_seq=0 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.569+0000 7f704f7fe640 10 client.17916 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f704f7fe640 1 -- 172.21.15.83:0/699232530 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b2eaf79310 con 0x55b2eb01c050 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f704f7fe640 10 client.17916 handle_command_reply: tid=0 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 1 client.17916 shutdown 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 1 -- 172.21.15.83:0/699232530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 msgr2=0x55b2eb02a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02a400 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7060003f80 tx=0x7f706000aa40 comp rx=0 tx=0).stop 2024-03-11T19:52:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 1 -- 172.21.15.83:0/699232530 shutdown_connections 2024-03-11T19:52:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b2eb01c050 0x55b2eb08a170 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2eb01b690 0x55b2eb02a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f704f7fe640 0 client.17916 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b2eb01a4c0 0x55b2eb02a400 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 1 --2- 172.21.15.83:0/699232530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2eb0156a0 0x55b2eb029ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 1 -- 172.21.15.83:0/699232530 >> 172.21.15.83:0/699232530 conn(0x55b2eb0229d0 msgr2=0x55b2eb023da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 1 -- 172.21.15.83:0/699232530 shutdown_connections 2024-03-11T19:52:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 1 -- 172.21.15.83:0/699232530 wait complete. 2024-03-11T19:52:47.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.570+0000 7f7078f73740 20 client.17916 trim_cache size 0 max 16384 2024-03-11T19:52:47.579 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:52:47.580 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:52:47.580 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "38cc75fb-3c28-40eb-a66b-9cc9dca372d4", 2024-03-11T19:52:47.580 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:52:47.580 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:52:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 -- 172.21.15.83:0/3143270580 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f705c0a1b30 msgr2=0x7f705c0a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f705c0a1b30 0x7f705c0a3ff0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f707006ae70 tx=0x7f7058008040 comp rx=0 tx=0).stop 2024-03-11T19:52:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 -- 172.21.15.83:0/3143270580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7070108180 msgr2=0x7f70701a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7070108180 0x7f70701a6100 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7068006010 tx=0x7f70680042f0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 -- 172.21.15.83:0/3143270580 shutdown_connections 2024-03-11T19:52:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f705c0a1b30 0x7f705c0a3ff0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f707010d4a0 0x7f70701ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f707010aaf0 0x7f70701a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 --2- 172.21.15.83:0/3143270580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7070108180 0x7f70701a6100 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 -- 172.21.15.83:0/3143270580 >> 172.21.15.83:0/3143270580 conn(0x7f707006dfe0 msgr2=0x7f70701097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 -- 172.21.15.83:0/3143270580 shutdown_connections 2024-03-11T19:52:47.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.571+0000 7f70762c3640 1 -- 172.21.15.83:0/3143270580 wait complete. 2024-03-11T19:52:47.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:52:47.590 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:52:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.743+0000 7f570114f640 1 -- 172.21.15.83:0/830744063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56fc101180 msgr2=0x7f56fc1015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.743+0000 7f570114f640 1 --2- 172.21.15.83:0/830744063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56fc101180 0x7f56fc1015a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009be0 tx=0x7f56f402bf10 comp rx=0 tx=0).stop 2024-03-11T19:52:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.743+0000 7f570114f640 1 -- 172.21.15.83:0/830744063 shutdown_connections 2024-03-11T19:52:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.743+0000 7f570114f640 1 --2- 172.21.15.83:0/830744063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56fc102d30 0x7f56fc1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.743+0000 7f570114f640 1 --2- 172.21.15.83:0/830744063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56fc102370 0x7f56fc1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.743+0000 7f570114f640 1 --2- 172.21.15.83:0/830744063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56fc101180 0x7f56fc1015a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.743+0000 7f570114f640 1 -- 172.21.15.83:0/830744063 >> 172.21.15.83:0/830744063 conn(0x7f56fc0fc6f0 msgr2=0x7f56fc0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.752+0000 7f570114f640 1 -- 172.21.15.83:0/830744063 shutdown_connections 2024-03-11T19:52:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.752+0000 7f570114f640 1 -- 172.21.15.83:0/830744063 wait complete. 2024-03-11T19:52:47.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.752+0000 7f570114f640 1 Processor -- start 2024-03-11T19:52:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.753+0000 7f570114f640 1 -- start start 2024-03-11T19:52:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.753+0000 7f570114f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56fc101180 0x7f56fc107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.753+0000 7f570114f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56fc102370 0x7f56fc107980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.753+0000 7f570114f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56fc102d30 0x7f56fc1089c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.753+0000 7f570114f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc108f00 con 0x7f56fc102d30 2024-03-11T19:52:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.753+0000 7f570114f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc109070 con 0x7f56fc102370 2024-03-11T19:52:47.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.753+0000 7f570114f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc109230 con 0x7f56fc101180 2024-03-11T19:52:47.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.756+0000 7f56fb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56fc102d30 0x7f56fc1089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.756+0000 7f56fb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56fc102d30 0x7f56fc1089c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38762/0 (socket says 172.21.15.83:38762) 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.756+0000 7f56fb59e640 1 -- 172.21.15.83:0/1875859909 learned_addr learned my addr 172.21.15.83:0/1875859909 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.756+0000 7f56fa59c640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56fc102370 0x7f56fc107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.756+0000 7f56fad9d640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56fc101180 0x7f56fc107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.756+0000 7f56fad9d640 1 -- 172.21.15.83:0/1875859909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56fc102370 msgr2=0x7f56fc107980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.756+0000 7f56fad9d640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56fc102370 0x7f56fc107980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.756+0000 7f56fad9d640 1 -- 172.21.15.83:0/1875859909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56fc102d30 msgr2=0x7f56fc1089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.756+0000 7f56fad9d640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56fc102d30 0x7f56fc1089c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.757+0000 7f56fad9d640 1 -- 172.21.15.83:0/1875859909 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f4009840 con 0x7f56fc101180 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.757+0000 7f56fad9d640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56fc101180 0x7f56fc107440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56f4034fd0 tx=0x7f56f4034660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.757+0000 7f56e3fff640 1 -- 172.21.15.83:0/1875859909 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f402cd80 con 0x7f56fc101180 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.757+0000 7f570114f640 1 -- 172.21.15.83:0/1875859909 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56fc1094b0 con 0x7f56fc101180 2024-03-11T19:52:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.757+0000 7f570114f640 1 -- 172.21.15.83:0/1875859909 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc110140 con 0x7f56fc101180 2024-03-11T19:52:47.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.758+0000 7f56e3fff640 1 -- 172.21.15.83:0/1875859909 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56f4034a60 con 0x7f56fc101180 2024-03-11T19:52:47.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.758+0000 7f56e3fff640 1 -- 172.21.15.83:0/1875859909 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f403f660 con 0x7f56fc101180 2024-03-11T19:52:47.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.761+0000 7f56e3fff640 1 -- 172.21.15.83:0/1875859909 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f56f403f7c0 con 0x7f56fc101180 2024-03-11T19:52:47.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.761+0000 7f5703e00740 1 Processor -- start 2024-03-11T19:52:47.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.761+0000 7f5703e00740 1 -- start start 2024-03-11T19:52:47.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.761+0000 7f56e3fff640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56e40a1b70 0x7f56e40a4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.762+0000 7f56e3fff640 1 -- 172.21.15.83:0/1875859909 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f56f40f0d40 con 0x7f56fc101180 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.762+0000 7f5703e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638569692d0 0x5638569696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.762+0000 7f5703e00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563856969cc0 0x56385696e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.762+0000 7f5703e00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56385696ee30 0x5638569712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.762+0000 7f56fad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638569692d0 0x5638569696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.762+0000 7f56fad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638569692d0 0x5638569696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:60106/0 (socket says 172.21.15.83:60106) 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.762+0000 7f56fad9d640 1 -- 172.21.15.83:0/3663060035 learned_addr learned my addr 172.21.15.83:0/3663060035 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.763+0000 7f5703e00740 1 -- 172.21.15.83:0/3663060035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638568d9280 con 0x56385696ee30 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.763+0000 7f5703e00740 1 -- 172.21.15.83:0/3663060035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638568d67b0 con 0x5638569692d0 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.763+0000 7f5703e00740 1 -- 172.21.15.83:0/3663060035 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638568d1f90 con 0x563856969cc0 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.763+0000 7f56fad9d640 1 -- 172.21.15.83:0/3663060035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563856969cc0 msgr2=0x56385696e7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.763+0000 7f56fad9d640 1 --2- 172.21.15.83:0/3663060035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563856969cc0 0x56385696e7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.763+0000 7f56fad9d640 1 -- 172.21.15.83:0/3663060035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56385696ee30 msgr2=0x5638569712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.763+0000 7f56fad9d640 1 --2- 172.21.15.83:0/3663060035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56385696ee30 0x5638569712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.764+0000 7f56fad9d640 1 -- 172.21.15.83:0/3663060035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563856890570 con 0x5638569692d0 2024-03-11T19:52:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.769+0000 7f56fb59e640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56e40a1b70 0x7f56e40a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.769+0000 7f56fad9d640 1 --2- 172.21.15.83:0/3663060035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638569692d0 0x5638569696f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f56f403efd0 tx=0x7f56f40bd720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f592de5ea44270d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.769+0000 7f56e17fa640 1 -- 172.21.15.83:0/3663060035 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f40c6460 con 0x5638569692d0 2024-03-11T19:52:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.769+0000 7f56e17fa640 1 -- 172.21.15.83:0/3663060035 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56f40bbfc0 con 0x5638569692d0 2024-03-11T19:52:47.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.770+0000 7f5703e00740 1 -- 172.21.15.83:0/3663060035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638569692d0 msgr2=0x5638569696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.770+0000 7f5703e00740 1 --2- 172.21.15.83:0/3663060035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638569692d0 0x5638569696f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f56f403efd0 tx=0x7f56f40bd720 comp rx=0 tx=0).stop 2024-03-11T19:52:47.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.771+0000 7f5703e00740 1 -- 172.21.15.83:0/3663060035 shutdown_connections 2024-03-11T19:52:47.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.771+0000 7f5703e00740 1 --2- 172.21.15.83:0/3663060035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56385696ee30 0x5638569712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.771+0000 7f5703e00740 1 --2- 172.21.15.83:0/3663060035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563856969cc0 0x56385696e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.771+0000 7f5703e00740 1 --2- 172.21.15.83:0/3663060035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5638569692d0 0x5638569696f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.771+0000 7f5703e00740 1 -- 172.21.15.83:0/3663060035 >> 172.21.15.83:0/3663060035 conn(0x5638569647f0 msgr2=0x563856964bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.771+0000 7f5703e00740 1 -- 172.21.15.83:0/3663060035 shutdown_connections 2024-03-11T19:52:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.771+0000 7f5703e00740 1 -- 172.21.15.83:0/3663060035 wait complete. 2024-03-11T19:52:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.772+0000 7f5703e00740 1 -- start start 2024-03-11T19:52:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.772+0000 7f5703e00740 1 Processor -- start 2024-03-11T19:52:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.772+0000 7f5703e00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638569692d0 0x563856972fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.772+0000 7f5703e00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563856969cc0 0x563856973500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.772+0000 7f5703e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56385696ee30 0x56385697a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.772+0000 7f5703e00740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638565a5430 con 0x5638569692d0 2024-03-11T19:52:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.772+0000 7f5703e00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563856561e00 con 0x56385696ee30 2024-03-11T19:52:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.772+0000 7f5703e00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563856856fd0 con 0x563856969cc0 2024-03-11T19:52:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.772+0000 7f5703e00740 10 client.0 _ll_register_callbacks cb 0x5638568acfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.773+0000 7f56fb59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56385696ee30 0x56385697a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.773+0000 7f56fb59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56385696ee30 0x56385697a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:60108/0 (socket says 172.21.15.83:60108) 2024-03-11T19:52:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.773+0000 7f56fb59e640 1 -- 172.21.15.83:0/4173641778 learned_addr learned my addr 172.21.15.83:0/4173641778 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.774+0000 7f56fa59c640 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563856969cc0 0x563856973500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.774+0000 7f56fb59e640 1 -- 172.21.15.83:0/4173641778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563856969cc0 msgr2=0x563856973500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.774+0000 7f56fb59e640 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563856969cc0 0x563856973500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.774+0000 7f56fb59e640 1 -- 172.21.15.83:0/4173641778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638569692d0 msgr2=0x563856972fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.774+0000 7f56fb59e640 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638569692d0 0x563856972fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.774+0000 7f56fb59e640 1 -- 172.21.15.83:0/4173641778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc110140 con 0x56385696ee30 2024-03-11T19:52:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.774+0000 7f56fa59c640 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563856969cc0 0x563856973500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:52:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.774+0000 7f56fb59e640 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56385696ee30 0x56385697a540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f56ec01a710 tx=0x7f56ec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e887652a7febbe70 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.774+0000 7f56deff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:52:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.775+0000 7f56deff5640 1 -- 172.21.15.83:0/4173641778 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec01fc60 con 0x56385696ee30 2024-03-11T19:52:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.775+0000 7f5703e00740 1 -- 172.21.15.83:0/4173641778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638565d17e0 con 0x56385696ee30 2024-03-11T19:52:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.775+0000 7f56deff5640 1 -- 172.21.15.83:0/4173641778 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f56ec01fdc0 con 0x56385696ee30 2024-03-11T19:52:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.775+0000 7f56deff5640 1 -- 172.21.15.83:0/4173641778 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56ec03d480 con 0x56385696ee30 2024-03-11T19:52:47.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.775+0000 7f5703e00740 10 client.27386 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:47.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.775+0000 7f5703e00740 1 -- 172.21.15.83:0/4173641778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5638568d0610 con 0x56385696ee30 2024-03-11T19:52:47.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.776+0000 7f56deff5640 1 -- 172.21.15.83:0/4173641778 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5638565d17e0 con 0x56385696ee30 2024-03-11T19:52:47.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.776+0000 7f56deff5640 1 -- 172.21.15.83:0/4173641778 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f56ec03a600 con 0x56385696ee30 2024-03-11T19:52:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.777+0000 7f5703e00740 10 client.27386 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.777+0000 7f5703e00740 10 client.27386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.777+0000 7f5703e00740 20 client.27386 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.777+0000 7f5703e00740 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56385697f270 0x563856981690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.777+0000 7f5703e00740 4 client.27386 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:52:47.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.777+0000 7f5703e00740 1 -- 172.21.15.83:0/4173641778 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56385660e720 con 0x56385697f270 2024-03-11T19:52:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.778+0000 7f56fad9d640 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56385697f270 0x563856981690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.779+0000 7f56fb59e640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56e40a1b70 0x7f56e40a4030 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f56ec022560 tx=0x7f56ec022960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.779+0000 7f56fad9d640 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56385697f270 0x563856981690 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1587d337ba590ded server_cookie=9e15d1f05cbf2eb8 in_seq=0 out_seq=0 2024-03-11T19:52:47.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.779+0000 7f56deff5640 10 client.27386 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.780+0000 7f56deff5640 1 -- 172.21.15.83:0/4173641778 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56385660e720 con 0x56385697f270 2024-03-11T19:52:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.780+0000 7f56deff5640 10 client.27386 handle_command_reply: tid=0 2024-03-11T19:52:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.780+0000 7f5703e00740 1 client.27386 shutdown 2024-03-11T19:52:47.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f5703e00740 1 -- 172.21.15.83:0/4173641778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56385696ee30 msgr2=0x56385697a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f5703e00740 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56385696ee30 0x56385697a540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f56ec01a710 tx=0x7f56ec0046c0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f5703e00740 1 -- 172.21.15.83:0/4173641778 shutdown_connections 2024-03-11T19:52:47.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f5703e00740 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x56385697f270 0x563856981690 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f5703e00740 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56385696ee30 0x56385697a540 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f5703e00740 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563856969cc0 0x563856973500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f5703e00740 1 --2- 172.21.15.83:0/4173641778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638569692d0 0x563856972fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f5703e00740 1 -- 172.21.15.83:0/4173641778 >> 172.21.15.83:0/4173641778 conn(0x5638569647f0 msgr2=0x563856963a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f56deff5640 0 client.27386 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:47.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f5703e00740 1 -- 172.21.15.83:0/4173641778 shutdown_connections 2024-03-11T19:52:47.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f5703e00740 1 -- 172.21.15.83:0/4173641778 wait complete. 2024-03-11T19:52:47.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.781+0000 7f5703e00740 20 client.27386 trim_cache size 0 max 16384 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.812+0000 7f5703e00740 1 Processor -- start 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.812+0000 7f5703e00740 1 -- start start 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.812+0000 7f5703e00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638569692d0 0x56385696ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.813+0000 7f5703e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563856969cc0 0x56385697a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.813+0000 7f5703e00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56385696ee30 0x56385696b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.813+0000 7f5703e00740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385697d7f0 con 0x56385696ee30 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.813+0000 7f5703e00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385697ba30 con 0x563856969cc0 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.813+0000 7f5703e00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56385695ee20 con 0x5638569692d0 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.813+0000 7f56fb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56385696ee30 0x56385696b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56385696ee30 0x56385696b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38776/0 (socket says 172.21.15.83:38776) 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fb59e640 1 -- 172.21.15.83:0/3991548068 learned_addr learned my addr 172.21.15.83:0/3991548068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:47.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fad9d640 1 --2- 172.21.15.83:0/3991548068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638569692d0 0x56385696ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fa59c640 1 --2- 172.21.15.83:0/3991548068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563856969cc0 0x56385697a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fb59e640 1 -- 172.21.15.83:0/3991548068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638569692d0 msgr2=0x56385696ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fb59e640 1 --2- 172.21.15.83:0/3991548068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638569692d0 0x56385696ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fb59e640 1 -- 172.21.15.83:0/3991548068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563856969cc0 msgr2=0x56385697a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fb59e640 1 --2- 172.21.15.83:0/3991548068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563856969cc0 0x56385697a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fb59e640 1 -- 172.21.15.83:0/3991548068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56385686cac0 con 0x56385696ee30 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fad9d640 1 --2- 172.21.15.83:0/3991548068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638569692d0 0x56385696ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fa59c640 1 --2- 172.21.15.83:0/3991548068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563856969cc0 0x56385697a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56fb59e640 1 --2- 172.21.15.83:0/3991548068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56385696ee30 0x56385696b3a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f56ec033040 tx=0x7f56ec03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab021f7984f8cf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.814+0000 7f56e0ff9640 1 -- 172.21.15.83:0/3991548068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec005850 con 0x56385696ee30 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.815+0000 7f56e0ff9640 1 -- 172.21.15.83:0/3991548068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56ec039360 con 0x56385696ee30 2024-03-11T19:52:47.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.815+0000 7f56e0ff9640 1 -- 172.21.15.83:0/3991548068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec039a80 con 0x56385696ee30 2024-03-11T19:52:47.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.815+0000 7f5703e00740 1 -- 172.21.15.83:0/3991548068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56385696ee30 msgr2=0x56385696b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.815+0000 7f5703e00740 1 --2- 172.21.15.83:0/3991548068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56385696ee30 0x56385696b3a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f56ec033040 tx=0x7f56ec03e000 comp rx=0 tx=0).stop 2024-03-11T19:52:47.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.815+0000 7f5703e00740 1 -- 172.21.15.83:0/3991548068 shutdown_connections 2024-03-11T19:52:47.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.815+0000 7f5703e00740 1 --2- 172.21.15.83:0/3991548068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56385696ee30 0x56385696b3a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.815+0000 7f5703e00740 1 --2- 172.21.15.83:0/3991548068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563856969cc0 0x56385697a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.815+0000 7f5703e00740 1 --2- 172.21.15.83:0/3991548068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638569692d0 0x56385696ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.815+0000 7f5703e00740 1 -- 172.21.15.83:0/3991548068 >> 172.21.15.83:0/3991548068 conn(0x5638569647f0 msgr2=0x563856985ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.815+0000 7f5703e00740 1 -- 172.21.15.83:0/3991548068 shutdown_connections 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.815+0000 7f5703e00740 1 -- 172.21.15.83:0/3991548068 wait complete. 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f5703e00740 1 -- start start 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f5703e00740 1 Processor -- start 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f5703e00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638569692d0 0x563856979f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f5703e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563856969cc0 0x56385697a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f5703e00740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385696ee30 0x5638569786a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f5703e00740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563856978d70 con 0x5638569692d0 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f5703e00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563856978ee0 con 0x563856969cc0 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f5703e00740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638569790a0 con 0x56385696ee30 2024-03-11T19:52:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f5703e00740 10 client.0 _ll_register_callbacks cb 0x7f56f4033820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56fad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638569692d0 0x563856979f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56fad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638569692d0 0x563856979f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38790/0 (socket says 172.21.15.83:38790) 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56fad9d640 1 -- 172.21.15.83:0/3053827356 learned_addr learned my addr 172.21.15.83:0/3053827356 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56fa59c640 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563856969cc0 0x56385697a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56fb59e640 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385696ee30 0x5638569786a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56fad9d640 1 -- 172.21.15.83:0/3053827356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385696ee30 msgr2=0x5638569786a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56fad9d640 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385696ee30 0x5638569786a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56fad9d640 1 -- 172.21.15.83:0/3053827356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563856969cc0 msgr2=0x56385697a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56fad9d640 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563856969cc0 0x56385697a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56fad9d640 1 -- 172.21.15.83:0/3053827356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638568d0610 con 0x5638569692d0 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56fad9d640 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638569692d0 0x563856979f70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f56f40bdc50 tx=0x7f56f402bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2204fe8263aebdc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.816+0000 7f56deff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f56deff5640 1 -- 172.21.15.83:0/3053827356 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f40b9f60 con 0x5638569692d0 2024-03-11T19:52:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f56deff5640 1 -- 172.21.15.83:0/3053827356 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f56f40ba490 con 0x5638569692d0 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f5703e00740 1 -- 172.21.15.83:0/3053827356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56f40bbfc0 con 0x5638569692d0 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f56deff5640 1 -- 172.21.15.83:0/3053827356 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56f40c6460 con 0x5638569692d0 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f5703e00740 10 client.17928 fetch_fsmap learned FSMap version 32 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f5703e00740 1 -- 172.21.15.83:0/3053827356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5638568926a0 con 0x5638569692d0 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f56deff5640 1 -- 172.21.15.83:0/3053827356 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56f40bbfc0 con 0x5638569692d0 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f56deff5640 1 -- 172.21.15.83:0/3053827356 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f56f403c070 con 0x5638569692d0 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f5703e00740 10 client.17928 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f5703e00740 10 client.17928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f5703e00740 20 client.17928 populate_metadata read hostname 'smithi083' 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f5703e00740 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5638569d68b0 0x5638569d8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f5703e00740 4 client.17928 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f5703e00740 1 -- 172.21.15.83:0/3053827356 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56385665d890 con 0x5638569d68b0 2024-03-11T19:52:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.817+0000 7f56fa59c640 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5638569d68b0 0x5638569d8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.818+0000 7f56fa59c640 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5638569d68b0 0x5638569d8cd0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b22c753640e1898 server_cookie=b66e1df2152a4645 in_seq=0 out_seq=0 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.818+0000 7f56deff5640 10 client.17928 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.818+0000 7f56deff5640 1 -- 172.21.15.83:0/3053827356 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56385665d890 con 0x5638569d68b0 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.818+0000 7f56deff5640 10 client.17928 handle_command_reply: tid=0 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.818+0000 7f5703e00740 1 client.17928 shutdown 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.818+0000 7f5703e00740 1 -- 172.21.15.83:0/3053827356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638569692d0 msgr2=0x563856979f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.818+0000 7f5703e00740 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638569692d0 0x563856979f70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f56f40bdc50 tx=0x7f56f402bf10 comp rx=0 tx=0).stop 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.818+0000 7f5703e00740 1 -- 172.21.15.83:0/3053827356 shutdown_connections 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.818+0000 7f5703e00740 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5638569d68b0 0x5638569d8cd0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.818+0000 7f5703e00740 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56385696ee30 0x5638569786a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.819+0000 7f5703e00740 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563856969cc0 0x56385697a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.819+0000 7f5703e00740 1 --2- 172.21.15.83:0/3053827356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638569692d0 0x563856979f70 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.819+0000 7f5703e00740 1 -- 172.21.15.83:0/3053827356 >> 172.21.15.83:0/3053827356 conn(0x5638569647f0 msgr2=0x563856967aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.819+0000 7f56deff5640 0 client.17928 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:52:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.819+0000 7f5703e00740 1 -- 172.21.15.83:0/3053827356 shutdown_connections 2024-03-11T19:52:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.819+0000 7f5703e00740 1 -- 172.21.15.83:0/3053827356 wait complete. 2024-03-11T19:52:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.819+0000 7f5703e00740 20 client.17928 trim_cache size 0 max 16384 2024-03-11T19:52:47.830 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:52:47.830 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:52:47.830 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:52:47.830 INFO:teuthology.orchestra.run.smithi083.stdout: "38cc75fb-3c28-40eb-a66b-9cc9dca372d4": { 2024-03-11T19:52:47.830 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:52:47.830 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "38cc75fb-3c28-40eb-a66b-9cc9dca372d4", 2024-03-11T19:52:47.830 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:52:47.831 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:52:47.831 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:52:47.831 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:52:47.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 -- 172.21.15.83:0/1875859909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56e40a1b70 msgr2=0x7f56e40a4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56e40a1b70 0x7f56e40a4030 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f56ec022560 tx=0x7f56ec022960 comp rx=0 tx=0).stop 2024-03-11T19:52:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 -- 172.21.15.83:0/1875859909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56fc101180 msgr2=0x7f56fc107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:52:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56fc101180 0x7f56fc107440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56f4034fd0 tx=0x7f56f4034660 comp rx=0 tx=0).stop 2024-03-11T19:52:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 -- 172.21.15.83:0/1875859909 shutdown_connections 2024-03-11T19:52:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56e40a1b70 0x7f56e40a4030 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56fc102d30 0x7f56fc1089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56fc102370 0x7f56fc107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 --2- 172.21.15.83:0/1875859909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56fc101180 0x7f56fc107440 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:52:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 -- 172.21.15.83:0/1875859909 >> 172.21.15.83:0/1875859909 conn(0x7f56fc0fc6f0 msgr2=0x7f56fc070060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:52:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 -- 172.21.15.83:0/1875859909 shutdown_connections 2024-03-11T19:52:47.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:52:47.820+0000 7f570114f640 1 -- 172.21.15.83:0/1875859909 wait complete. 2024-03-11T19:52:47.836 INFO:tasks.cephfs.filesystem:scrub status for tag:38cc75fb-3c28-40eb-a66b-9cc9dca372d4 - {'path': '/', 'tag': '38cc75fb-3c28-40eb-a66b-9cc9dca372d4', 'options': 'recursive,force'} 2024-03-11T19:52:48.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:48 smithi083 ceph-mon[31864]: pgmap v756: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 624 op/s 2024-03-11T19:52:48.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:48 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:52:48.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:48 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:48.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:48 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:52:48.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:48 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:52:48.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:48 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:48.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:48 smithi137 ceph-mon[36220]: pgmap v756: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 624 op/s 2024-03-11T19:52:48.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:48 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:52:48.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:48 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:48.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:48 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:52:48.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:48 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:52:48.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:48 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:48.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:48 smithi204 ceph-mon[36739]: pgmap v756: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 13 MiB/s wr, 624 op/s 2024-03-11T19:52:48.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:48 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:52:48.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:48 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:48.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:48 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:52:48.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:48 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:52:48.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:48 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:49.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:49 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:52:49.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:49 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:49.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:49 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:49 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:52:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:49 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:49.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:49 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:49 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:52:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:49 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:49.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:49 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:52:50.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:50 smithi083 ceph-mon[31864]: pgmap v757: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 699 op/s 2024-03-11T19:52:50.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:50 smithi137 ceph-mon[36220]: pgmap v757: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 699 op/s 2024-03-11T19:52:50.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:50 smithi204 ceph-mon[36739]: pgmap v757: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 699 op/s 2024-03-11T19:52:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 815.0 s, 1626.1 tps, lat 61.826 ms stddev 70.486 2024-03-11T19:52:52.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:52 smithi083 ceph-mon[31864]: pgmap v758: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 624 op/s 2024-03-11T19:52:52.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:52 smithi137 ceph-mon[36220]: pgmap v758: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 624 op/s 2024-03-11T19:52:52.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:52 smithi204 ceph-mon[36739]: pgmap v758: 97 pgs: 97 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 624 op/s 2024-03-11T19:52:54.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:54 smithi083 ceph-mon[31864]: pgmap v759: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 580 op/s 2024-03-11T19:52:54.641 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:54 smithi137 ceph-mon[36220]: pgmap v759: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 580 op/s 2024-03-11T19:52:54.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:54 smithi204 ceph-mon[36739]: pgmap v759: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 580 op/s 2024-03-11T19:52:56.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:56 smithi083 ceph-mon[31864]: pgmap v760: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 13 MiB/s wr, 709 op/s 2024-03-11T19:52:56.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:56 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:52:56.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:56 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:52:56.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:56 smithi137 ceph-mon[36220]: pgmap v760: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 13 MiB/s wr, 709 op/s 2024-03-11T19:52:56.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:56 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:52:56.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:56 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:52:56.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:56 smithi204 ceph-mon[36739]: pgmap v760: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 13 MiB/s wr, 709 op/s 2024-03-11T19:52:56.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:56 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:52:56.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:56 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:52:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 820.0 s, 1456.2 tps, lat 68.426 ms stddev 77.113 2024-03-11T19:52:57.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:57 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:52:57.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:57 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:52:57.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:57 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:52:58.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:52:58 smithi083 ceph-mon[31864]: pgmap v761: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.5 MiB/s wr, 572 op/s 2024-03-11T19:52:58.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:52:58 smithi137 ceph-mon[36220]: pgmap v761: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.5 MiB/s wr, 572 op/s 2024-03-11T19:52:58.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:52:58 smithi204 ceph-mon[36739]: pgmap v761: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 9.5 MiB/s wr, 572 op/s 2024-03-11T19:53:00.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:00 smithi083 ceph-mon[31864]: pgmap v762: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 681 op/s 2024-03-11T19:53:00.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:00 smithi137 ceph-mon[36220]: pgmap v762: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 681 op/s 2024-03-11T19:53:00.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:00 smithi204 ceph-mon[36739]: pgmap v762: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 681 op/s 2024-03-11T19:53:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 825.0 s, 1319.0 tps, lat 75.801 ms stddev 129.650 2024-03-11T19:53:02.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:02 smithi083 ceph-mon[31864]: pgmap v763: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-03-11T19:53:02.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:02 smithi137 ceph-mon[36220]: pgmap v763: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-03-11T19:53:02.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:02 smithi204 ceph-mon[36739]: pgmap v763: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-03-11T19:53:04.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:04 smithi083 ceph-mon[31864]: pgmap v764: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 1.18k op/s 2024-03-11T19:53:04.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:04 smithi137 ceph-mon[36220]: pgmap v764: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 1.18k op/s 2024-03-11T19:53:04.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:04 smithi204 ceph-mon[36739]: pgmap v764: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 1.18k op/s 2024-03-11T19:53:06.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:06 smithi083 ceph-mon[31864]: pgmap v765: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 MiB/s wr, 1.89k op/s 2024-03-11T19:53:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:06 smithi137 ceph-mon[36220]: pgmap v765: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 MiB/s wr, 1.89k op/s 2024-03-11T19:53:06.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:06 smithi204 ceph-mon[36739]: pgmap v765: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 36 MiB/s wr, 1.89k op/s 2024-03-11T19:53:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 830.0 s, 1644.6 tps, lat 61.041 ms stddev 69.801 2024-03-11T19:53:08.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:08 smithi083 ceph-mon[31864]: pgmap v766: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 MiB/s wr, 1.79k op/s 2024-03-11T19:53:08.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:08 smithi137 ceph-mon[36220]: pgmap v766: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 MiB/s wr, 1.79k op/s 2024-03-11T19:53:08.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:08 smithi204 ceph-mon[36739]: pgmap v766: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 MiB/s wr, 1.79k op/s 2024-03-11T19:53:10.821 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:10 smithi083 ceph-mon[31864]: pgmap v767: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 36 MiB/s wr, 1.96k op/s 2024-03-11T19:53:10.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:10 smithi137 ceph-mon[36220]: pgmap v767: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 36 MiB/s wr, 1.96k op/s 2024-03-11T19:53:10.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:10 smithi204 ceph-mon[36739]: pgmap v767: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 36 MiB/s wr, 1.96k op/s 2024-03-11T19:53:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 835.0 s, 1639.0 tps, lat 60.146 ms stddev 62.849 2024-03-11T19:53:12.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:12 smithi083 ceph-mon[31864]: pgmap v768: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 1.90k op/s 2024-03-11T19:53:12.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:12 smithi137 ceph-mon[36220]: pgmap v768: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 1.90k op/s 2024-03-11T19:53:12.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:12 smithi204 ceph-mon[36739]: pgmap v768: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 1.90k op/s 2024-03-11T19:53:14.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:14 smithi083 ceph-mon[31864]: pgmap v769: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 1.42k op/s 2024-03-11T19:53:14.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:14 smithi137 ceph-mon[36220]: pgmap v769: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 1.42k op/s 2024-03-11T19:53:14.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:14 smithi204 ceph-mon[36739]: pgmap v769: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 25 MiB/s wr, 1.42k op/s 2024-03-11T19:53:16.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:16 smithi083 ceph-mon[31864]: pgmap v770: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 23 MiB/s wr, 1.39k op/s 2024-03-11T19:53:16.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:16 smithi137 ceph-mon[36220]: pgmap v770: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 23 MiB/s wr, 1.39k op/s 2024-03-11T19:53:16.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:16 smithi204 ceph-mon[36739]: pgmap v770: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 23 MiB/s wr, 1.39k op/s 2024-03-11T19:53:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 840.0 s, 1863.6 tps, lat 54.320 ms stddev 69.262 2024-03-11T19:53:17.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:53:17.837 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:53:17.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb4d24640 1 -- 172.21.15.83:0/1310840838 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca8032ab0 con 0x7fccb0100d50 2024-03-11T19:53:17.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb7f8a640 1 -- 172.21.15.83:0/1310840838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb0100d50 msgr2=0x7fccb01011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:17.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb7f8a640 1 --2- 172.21.15.83:0/1310840838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb0100d50 0x7fccb01011d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009ca0 tx=0x7fcca802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb7f8a640 1 -- 172.21.15.83:0/1310840838 shutdown_connections 2024-03-11T19:53:17.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb7f8a640 1 --2- 172.21.15.83:0/1310840838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb0100d50 0x7fccb01011d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb7f8a640 1 --2- 172.21.15.83:0/1310840838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb01003b0 0x7fccb0100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb7f8a640 1 --2- 172.21.15.83:0/1310840838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb00ffa50 0x7fccb00ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb7f8a640 1 -- 172.21.15.83:0/1310840838 >> 172.21.15.83:0/1310840838 conn(0x7fccb00fb5f0 msgr2=0x7fccb00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:17.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb7f8a640 1 -- 172.21.15.83:0/1310840838 shutdown_connections 2024-03-11T19:53:17.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb7f8a640 1 -- 172.21.15.83:0/1310840838 wait complete. 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb7f8a640 1 Processor -- start 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.984+0000 7fccb7f8a640 1 -- start start 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb7f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb00ffa50 0x7fccb01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb7f8a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb01003b0 0x7fccb0199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb7f8a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb0100d50 0x7fccb01a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb7f8a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb01a0ed0 con 0x7fccb00ffa50 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb7f8a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb01a1040 con 0x7fccb01003b0 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb7f8a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb01a1200 con 0x7fccb0100d50 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb5d26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb00ffa50 0x7fccb01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb5d26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb00ffa50 0x7fccb01993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40568/0 (socket says 172.21.15.83:40568) 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb5d26640 1 -- 172.21.15.83:0/2054721584 learned_addr learned my addr 172.21.15.83:0/2054721584 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb5525640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb01003b0 0x7fccb0199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb6527640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb0100d50 0x7fccb01a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb5d26640 1 -- 172.21.15.83:0/2054721584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb0100d50 msgr2=0x7fccb01a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb5d26640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb0100d50 0x7fccb01a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb5d26640 1 -- 172.21.15.83:0/2054721584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb01003b0 msgr2=0x7fccb0199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb5d26640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb01003b0 0x7fccb0199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb5d26640 1 -- 172.21.15.83:0/2054721584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8009900 con 0x7fccb00ffa50 2024-03-11T19:53:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb5d26640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb00ffa50 0x7fccb01993e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e9f0 tx=0x7fcca000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fcca6ffd640 1 -- 172.21.15.83:0/2054721584 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0009700 con 0x7fccb00ffa50 2024-03-11T19:53:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb7f8a640 1 -- 172.21.15.83:0/2054721584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb01a14e0 con 0x7fccb00ffa50 2024-03-11T19:53:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fcca6ffd640 1 -- 172.21.15.83:0/2054721584 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca000cd80 con 0x7fccb00ffa50 2024-03-11T19:53:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fcca6ffd640 1 -- 172.21.15.83:0/2054721584 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0018920 con 0x7fccb00ffa50 2024-03-11T19:53:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.985+0000 7fccb7f8a640 1 -- 172.21.15.83:0/2054721584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb01a19f0 con 0x7fccb00ffa50 2024-03-11T19:53:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.988+0000 7fccbac33740 1 Processor -- start 2024-03-11T19:53:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.988+0000 7fcca6ffd640 1 -- 172.21.15.83:0/2054721584 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fcca0018a80 con 0x7fccb00ffa50 2024-03-11T19:53:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccbac33740 1 -- start start 2024-03-11T19:53:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccbac33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 0x55b29c8386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccbac33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 0x55b29c83d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccbac33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 0x55b29c8402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fcca6ffd640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcc9c0a1a30 0x7fcc9c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccbac33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c7a8280 con 0x55b29c838cc0 2024-03-11T19:53:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccbac33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c7a57b0 con 0x55b29c83de30 2024-03-11T19:53:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccb6527640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 0x55b29c8386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccb6527640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 0x55b29c8386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45636/0 (socket says 172.21.15.83:45636) 2024-03-11T19:53:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccb6527640 1 -- 172.21.15.83:0/2232054902 learned_addr learned my addr 172.21.15.83:0/2232054902 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccb5d26640 1 --2- 172.21.15.83:0/2232054902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 0x55b29c83d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccbac33740 1 -- 172.21.15.83:0/2232054902 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c7a0f90 con 0x55b29c8382d0 2024-03-11T19:53:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fcca6ffd640 1 -- 172.21.15.83:0/2054721584 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fcca0014070 con 0x7fccb00ffa50 2024-03-11T19:53:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.989+0000 7fccb5525640 1 --2- 172.21.15.83:0/2232054902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 0x55b29c8402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.990+0000 7fccb6527640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcc9c0a1a30 0x7fcc9c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.990+0000 7fccb6527640 1 -- 172.21.15.83:0/2232054902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 msgr2=0x55b29c8402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.990+0000 7fccb6527640 1 --2- 172.21.15.83:0/2232054902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 0x55b29c8402c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.990+0000 7fccb6527640 1 -- 172.21.15.83:0/2232054902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 msgr2=0x55b29c83d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.990+0000 7fccb6527640 1 --2- 172.21.15.83:0/2232054902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 0x55b29c83d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.990+0000 7fccb6527640 1 -- 172.21.15.83:0/2232054902 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b29c75f570 con 0x55b29c8382d0 2024-03-11T19:53:18.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.990+0000 7fccb6527640 1 --2- 172.21.15.83:0/2232054902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 0x55b29c8386f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009430 tx=0x7fcca80327b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=648ece6ba4ea5cf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.990+0000 7fccb6527640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcc9c0a1a30 0x7fcc9c0a3ef0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcca8032710 tx=0x7fcca8048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.990+0000 7fcc9bfff640 1 -- 172.21.15.83:0/2232054902 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca8004d40 con 0x55b29c8382d0 2024-03-11T19:53:18.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.990+0000 7fcc9bfff640 1 -- 172.21.15.83:0/2232054902 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca8032b20 con 0x55b29c8382d0 2024-03-11T19:53:18.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.990+0000 7fcc9bfff640 1 -- 172.21.15.83:0/2232054902 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca8004540 con 0x55b29c8382d0 2024-03-11T19:53:18.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.991+0000 7fccbac33740 1 -- 172.21.15.83:0/2232054902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 msgr2=0x55b29c8386f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.991+0000 7fccbac33740 1 --2- 172.21.15.83:0/2232054902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 0x55b29c8386f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009430 tx=0x7fcca80327b0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.991+0000 7fccbac33740 1 -- 172.21.15.83:0/2232054902 shutdown_connections 2024-03-11T19:53:18.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.991+0000 7fccbac33740 1 --2- 172.21.15.83:0/2232054902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 0x55b29c8402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.991+0000 7fccbac33740 1 --2- 172.21.15.83:0/2232054902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 0x55b29c83d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.991+0000 7fccbac33740 1 --2- 172.21.15.83:0/2232054902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 0x55b29c8386f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.991+0000 7fccbac33740 1 -- 172.21.15.83:0/2232054902 >> 172.21.15.83:0/2232054902 conn(0x55b29c8337f0 msgr2=0x55b29c833bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.991+0000 7fccbac33740 1 -- 172.21.15.83:0/2232054902 shutdown_connections 2024-03-11T19:53:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.991+0000 7fccbac33740 1 -- 172.21.15.83:0/2232054902 wait complete. 2024-03-11T19:53:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.991+0000 7fccbac33740 1 -- start start 2024-03-11T19:53:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.992+0000 7fccbac33740 1 Processor -- start 2024-03-11T19:53:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.992+0000 7fccbac33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c8382d0 0x55b29c8422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.992+0000 7fccbac33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c838cc0 0x55b29c842810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.992+0000 7fccbac33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c83de30 0x55b29c849850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.992+0000 7fccb5525640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c838cc0 0x55b29c842810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.992+0000 7fccb5525640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c838cc0 0x55b29c842810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45652/0 (socket says 172.21.15.83:45652) 2024-03-11T19:53:18.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.992+0000 7fccb5d26640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c8382d0 0x55b29c8422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.992+0000 7fccb5d26640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c8382d0 0x55b29c8422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39092/0 (socket says 172.21.15.83:39092) 2024-03-11T19:53:18.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccb5525640 1 -- 172.21.15.83:0/1823388384 learned_addr learned my addr 172.21.15.83:0/1823388384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccbac33740 1 -- 172.21.15.83:0/1823388384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c474430 con 0x55b29c83de30 2024-03-11T19:53:18.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccbac33740 1 -- 172.21.15.83:0/1823388384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c430e00 con 0x55b29c8382d0 2024-03-11T19:53:18.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccbac33740 1 -- 172.21.15.83:0/1823388384 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c725fd0 con 0x55b29c838cc0 2024-03-11T19:53:18.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccbac33740 10 client.0 _ll_register_callbacks cb 0x55b29c77bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:18.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccb5525640 1 -- 172.21.15.83:0/1823388384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c8382d0 msgr2=0x55b29c8422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccb5525640 1 --2- 172.21.15.83:0/1823388384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c8382d0 0x55b29c8422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccb5525640 1 -- 172.21.15.83:0/1823388384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c83de30 msgr2=0x55b29c849850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccb5525640 1 --2- 172.21.15.83:0/1823388384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c83de30 0x55b29c849850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccb5525640 1 -- 172.21.15.83:0/1823388384 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb010c0a0 con 0x55b29c838cc0 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccb5525640 1 --2- 172.21.15.83:0/1823388384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c838cc0 0x55b29c842810 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fccac012720 tx=0x7fccac012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94ccfd48e8dff647 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fccbac33740 1 -- 172.21.15.83:0/1823388384 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b29c4a07e0 con 0x55b29c838cc0 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fcc997fa640 10 client.27508 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fcc997fa640 1 -- 172.21.15.83:0/1823388384 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac017070 con 0x55b29c838cc0 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.993+0000 7fcc997fa640 1 -- 172.21.15.83:0/1823388384 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fccac00f8c0 con 0x55b29c838cc0 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.994+0000 7fcc997fa640 1 -- 172.21.15.83:0/1823388384 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccac00fa80 con 0x55b29c838cc0 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.994+0000 7fccbac33740 10 client.27508 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.994+0000 7fccbac33740 1 -- 172.21.15.83:0/1823388384 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x55b29c79f610 con 0x55b29c838cc0 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.994+0000 7fcc997fa640 1 -- 172.21.15.83:0/1823388384 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fccac00fbe0 con 0x55b29c838cc0 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.994+0000 7fccbac33740 10 client.27508 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:18.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.994+0000 7fccbac33740 10 client.27508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:18.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.995+0000 7fccbac33740 20 client.27508 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:18.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.995+0000 7fccbac33740 1 --2- 172.21.15.83:0/1823388384 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b29c84e3a0 0x55b29c8507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.995+0000 7fccbac33740 4 client.27508 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:18.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.995+0000 7fccbac33740 1 -- 172.21.15.83:0/1823388384 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b29c4a07e0 con 0x55b29c84e3a0 2024-03-11T19:53:18.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.995+0000 7fccb5d26640 1 --2- 172.21.15.83:0/1823388384 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b29c84e3a0 0x55b29c8507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.995+0000 7fccb5d26640 1 --2- 172.21.15.83:0/1823388384 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b29c84e3a0 0x55b29c8507d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed76e0ecf97a1139 server_cookie=509d6c86dc0bcd50 in_seq=0 out_seq=0 2024-03-11T19:53:18.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.995+0000 7fcc997fa640 10 client.27508 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:18.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fcc997fa640 1 -- 172.21.15.83:0/1823388384 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b29c4a07e0 con 0x55b29c84e3a0 2024-03-11T19:53:18.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fcc997fa640 10 client.27508 handle_command_reply: tid=0 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 1 client.27508 shutdown 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 1 -- 172.21.15.83:0/1823388384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c838cc0 msgr2=0x55b29c842810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 1 --2- 172.21.15.83:0/1823388384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c838cc0 0x55b29c842810 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fccac012720 tx=0x7fccac012be0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 1 -- 172.21.15.83:0/1823388384 shutdown_connections 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 1 --2- 172.21.15.83:0/1823388384 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b29c84e3a0 0x55b29c8507d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 1 --2- 172.21.15.83:0/1823388384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c83de30 0x55b29c849850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 1 --2- 172.21.15.83:0/1823388384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c838cc0 0x55b29c842810 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 1 --2- 172.21.15.83:0/1823388384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c8382d0 0x55b29c8422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fcc997fa640 0 client.27508 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 1 -- 172.21.15.83:0/1823388384 >> 172.21.15.83:0/1823388384 conn(0x55b29c8337f0 msgr2=0x55b29c832a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 1 -- 172.21.15.83:0/1823388384 shutdown_connections 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 1 -- 172.21.15.83:0/1823388384 wait complete. 2024-03-11T19:53:18.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:17.996+0000 7fccbac33740 20 client.27508 trim_cache size 0 max 16384 2024-03-11T19:53:18.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.014+0000 7fccbac33740 1 Processor -- start 2024-03-11T19:53:18.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.014+0000 7fccbac33740 1 -- start start 2024-03-11T19:53:18.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.014+0000 7fccbac33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c8382d0 0x55b29c842290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.014+0000 7fccbac33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c838cc0 0x55b29c8427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.014+0000 7fccbac33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 0x55b29c849810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.014+0000 7fccbac33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c84ad20 con 0x55b29c8382d0 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.014+0000 7fccbac33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c82f0d0 con 0x55b29c83de30 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.014+0000 7fccbac33740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c82de50 con 0x55b29c838cc0 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.014+0000 7fccb5d26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c8382d0 0x55b29c842290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.015+0000 7fccb5d26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c8382d0 0x55b29c842290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40590/0 (socket says 172.21.15.83:40590) 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.015+0000 7fccb5d26640 1 -- 172.21.15.83:0/16288893 learned_addr learned my addr 172.21.15.83:0/16288893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.015+0000 7fccb5d26640 1 -- 172.21.15.83:0/16288893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c838cc0 msgr2=0x55b29c8427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.015+0000 7fccb5d26640 1 --2- 172.21.15.83:0/16288893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c838cc0 0x55b29c8427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.015+0000 7fccb5d26640 1 -- 172.21.15.83:0/16288893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 msgr2=0x55b29c849810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.015+0000 7fccb5d26640 1 --2- 172.21.15.83:0/16288893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 0x55b29c849810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.015+0000 7fccb5d26640 1 -- 172.21.15.83:0/16288893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b29c73bac0 con 0x55b29c8382d0 2024-03-11T19:53:18.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.016+0000 7fccb5d26640 1 --2- 172.21.15.83:0/16288893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c8382d0 0x55b29c842290 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcca009a4b0 tx=0x7fcca001c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf71d57898725 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.016+0000 7fcc9b7fe640 1 -- 172.21.15.83:0/16288893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca009e900 con 0x55b29c8382d0 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.016+0000 7fcc9b7fe640 1 -- 172.21.15.83:0/16288893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca00a2f50 con 0x55b29c8382d0 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.016+0000 7fcc9b7fe640 1 -- 172.21.15.83:0/16288893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca009aa10 con 0x55b29c8382d0 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.017+0000 7fccbac33740 1 -- 172.21.15.83:0/16288893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c8382d0 msgr2=0x55b29c842290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.017+0000 7fccbac33740 1 --2- 172.21.15.83:0/16288893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c8382d0 0x55b29c842290 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcca009a4b0 tx=0x7fcca001c830 comp rx=0 tx=0).stop 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.017+0000 7fccbac33740 1 -- 172.21.15.83:0/16288893 shutdown_connections 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.017+0000 7fccbac33740 1 --2- 172.21.15.83:0/16288893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 0x55b29c849810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.017+0000 7fccbac33740 1 --2- 172.21.15.83:0/16288893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c838cc0 0x55b29c8427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.017+0000 7fccbac33740 1 --2- 172.21.15.83:0/16288893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c8382d0 0x55b29c842290 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.017+0000 7fccbac33740 1 -- 172.21.15.83:0/16288893 >> 172.21.15.83:0/16288893 conn(0x55b29c8337f0 msgr2=0x55b29c855170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccbac33740 1 -- 172.21.15.83:0/16288893 shutdown_connections 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccbac33740 1 -- 172.21.15.83:0/16288893 wait complete. 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccbac33740 1 -- start start 2024-03-11T19:53:18.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccbac33740 1 Processor -- start 2024-03-11T19:53:18.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccbac33740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 0x55b29c8466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccbac33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 0x55b29c846be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccbac33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 0x55b29c8ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccbac33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c847120 con 0x55b29c838cc0 2024-03-11T19:53:18.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccbac33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c847290 con 0x55b29c83de30 2024-03-11T19:53:18.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccbac33740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29c8ad630 con 0x55b29c8382d0 2024-03-11T19:53:18.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccbac33740 10 client.0 _ll_register_callbacks cb 0x7fcca8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:18.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccb5525640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 0x55b29c846be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.018+0000 7fccb5525640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 0x55b29c846be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40604/0 (socket says 172.21.15.83:40604) 2024-03-11T19:53:18.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fccb5525640 1 -- 172.21.15.83:0/3623690111 learned_addr learned my addr 172.21.15.83:0/3623690111 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fccb5d26640 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 0x55b29c8466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fccb6527640 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 0x55b29c8ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fccb5d26640 1 -- 172.21.15.83:0/3623690111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 msgr2=0x55b29c8ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fccb5d26640 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 0x55b29c8ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fccb5d26640 1 -- 172.21.15.83:0/3623690111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 msgr2=0x55b29c846be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fccb5d26640 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 0x55b29c846be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fccb5d26640 1 -- 172.21.15.83:0/3623690111 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb01a19f0 con 0x55b29c8382d0 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fccb5525640 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 0x55b29c846be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fccb5d26640 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 0x55b29c8466a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55b29c84ae60 tx=0x7fcca009a870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaff4d9292822708 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fcc997fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fcc997fa640 1 -- 172.21.15.83:0/3623690111 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca009e980 con 0x55b29c8382d0 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.019+0000 7fcc997fa640 1 -- 172.21.15.83:0/3623690111 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fcca00b3180 con 0x55b29c8382d0 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fccbac33740 1 -- 172.21.15.83:0/3623690111 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcca8032b20 con 0x55b29c8382d0 2024-03-11T19:53:18.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fcc997fa640 1 -- 172.21.15.83:0/3623690111 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcca0096e20 con 0x55b29c8382d0 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fccbac33740 10 client.27511 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fccbac33740 1 -- 172.21.15.83:0/3623690111 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b29c7616a0 con 0x55b29c8382d0 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fcc997fa640 1 -- 172.21.15.83:0/3623690111 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcca8032b20 con 0x55b29c8382d0 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fcc997fa640 1 -- 172.21.15.83:0/3623690111 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcca00ba0b0 con 0x55b29c8382d0 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fccbac33740 10 client.27511 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fccbac33740 10 client.27511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fccbac33740 20 client.27511 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fccbac33740 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b29c8aed90 0x55b29c8b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fccbac33740 4 client.27511 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fccbac33740 1 -- 172.21.15.83:0/3623690111 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b29c4dd720 con 0x55b29c8aed90 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.020+0000 7fccb5525640 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b29c8aed90 0x55b29c8b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fcc997fa640 10 client.27511 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:18.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fccb5525640 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b29c8aed90 0x55b29c8b11b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bd7ef1a0bc8dd9 server_cookie=8d35cb70d8a9fe0e in_seq=0 out_seq=0 2024-03-11T19:53:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fcc997fa640 1 -- 172.21.15.83:0/3623690111 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b29c4dd720 con 0x55b29c8aed90 2024-03-11T19:53:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fcc997fa640 10 client.27511 handle_command_reply: tid=0 2024-03-11T19:53:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fccbac33740 1 client.27511 shutdown 2024-03-11T19:53:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fccbac33740 1 -- 172.21.15.83:0/3623690111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 msgr2=0x55b29c8466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fccbac33740 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 0x55b29c8466a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55b29c84ae60 tx=0x7fcca009a870 comp rx=0 tx=0).stop 2024-03-11T19:53:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fccbac33740 1 -- 172.21.15.83:0/3623690111 shutdown_connections 2024-03-11T19:53:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fccbac33740 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55b29c8aed90 0x55b29c8b11b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fccbac33740 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b29c83de30 0x55b29c8ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fccbac33740 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b29c838cc0 0x55b29c846be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fcc997fa640 0 client.27511 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fccbac33740 1 --2- 172.21.15.83:0/3623690111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b29c8382d0 0x55b29c8466a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.021+0000 7fccbac33740 1 -- 172.21.15.83:0/3623690111 >> 172.21.15.83:0/3623690111 conn(0x55b29c8337f0 msgr2=0x55b29c842210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccbac33740 1 -- 172.21.15.83:0/3623690111 shutdown_connections 2024-03-11T19:53:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccbac33740 1 -- 172.21.15.83:0/3623690111 wait complete. 2024-03-11T19:53:18.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccbac33740 20 client.27511 trim_cache size 0 max 16384 2024-03-11T19:53:18.035 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:53:18.035 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:53:18.035 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:53:18.035 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:53:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 -- 172.21.15.83:0/2054721584 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcc9c0a1a30 msgr2=0x7fcc9c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcc9c0a1a30 0x7fcc9c0a3ef0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcca8032710 tx=0x7fcca8048000 comp rx=0 tx=0).stop 2024-03-11T19:53:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 -- 172.21.15.83:0/2054721584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb00ffa50 msgr2=0x7fccb01993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb00ffa50 0x7fccb01993e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e9f0 tx=0x7fcca000eeb0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 -- 172.21.15.83:0/2054721584 shutdown_connections 2024-03-11T19:53:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fcc9c0a1a30 0x7fcc9c0a3ef0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fccb0100d50 0x7fccb01a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccb01003b0 0x7fccb0199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 --2- 172.21.15.83:0/2054721584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb00ffa50 0x7fccb01993e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 -- 172.21.15.83:0/2054721584 >> 172.21.15.83:0/2054721584 conn(0x7fccb00fb5f0 msgr2=0x7fccb00fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 -- 172.21.15.83:0/2054721584 shutdown_connections 2024-03-11T19:53:18.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.022+0000 7fccb7f8a640 1 -- 172.21.15.83:0/2054721584 wait complete. 2024-03-11T19:53:18.039 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:53:18.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:53:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.194+0000 7ff39f9b6640 1 -- 172.21.15.83:0/1112641777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff398104550 msgr2=0x7ff398106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.194+0000 7ff39f9b6640 1 --2- 172.21.15.83:0/1112641777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff398104550 0x7ff398106940 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff388009860 tx=0x7ff38802bb70 comp rx=0 tx=0).stop 2024-03-11T19:53:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.194+0000 7ff39f9b6640 1 -- 172.21.15.83:0/1112641777 shutdown_connections 2024-03-11T19:53:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.194+0000 7ff39f9b6640 1 --2- 172.21.15.83:0/1112641777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff398106e80 0x7ff3981092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.194+0000 7ff39f9b6640 1 --2- 172.21.15.83:0/1112641777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff398104550 0x7ff398106940 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.194+0000 7ff39f9b6640 1 --2- 172.21.15.83:0/1112641777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398101c20 0x7ff398104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.194+0000 7ff39f9b6640 1 -- 172.21.15.83:0/1112641777 >> 172.21.15.83:0/1112641777 conn(0x7ff3980fb610 msgr2=0x7ff3980fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.194+0000 7ff39f9b6640 1 -- 172.21.15.83:0/1112641777 shutdown_connections 2024-03-11T19:53:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.194+0000 7ff39f9b6640 1 -- 172.21.15.83:0/1112641777 wait complete. 2024-03-11T19:53:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.195+0000 7ff39f9b6640 1 Processor -- start 2024-03-11T19:53:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.195+0000 7ff39f9b6640 1 -- start start 2024-03-11T19:53:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.196+0000 7ff39f9b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff398101c20 0x7ff398194e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.196+0000 7ff39f9b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398104550 0x7ff3981953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.196+0000 7ff39f9b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff398106e80 0x7ff3981a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.196+0000 7ff39f9b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff398195cb0 con 0x7ff398104550 2024-03-11T19:53:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.196+0000 7ff39f9b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39806aba0 con 0x7ff398101c20 2024-03-11T19:53:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.196+0000 7ff39f9b6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39806ad60 con 0x7ff398106e80 2024-03-11T19:53:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.196+0000 7ff39cf51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398104550 0x7ff3981953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.196+0000 7ff39cf51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398104550 0x7ff3981953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40618/0 (socket says 172.21.15.83:40618) 2024-03-11T19:53:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.196+0000 7ff39cf51640 1 -- 172.21.15.83:0/3382007847 learned_addr learned my addr 172.21.15.83:0/3382007847 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.197+0000 7ff39cf51640 1 -- 172.21.15.83:0/3382007847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff398106e80 msgr2=0x7ff3981a5040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-11T19:53:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.197+0000 7ff39cf51640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff398106e80 0x7ff3981a5040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.197+0000 7ff39cf51640 1 -- 172.21.15.83:0/3382007847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff398101c20 msgr2=0x7ff398194e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-11T19:53:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.197+0000 7ff39cf51640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff398101c20 0x7ff398194e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.197+0000 7ff39cf51640 1 -- 172.21.15.83:0/3382007847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff388009490 con 0x7ff398104550 2024-03-11T19:53:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.197+0000 7ff39cf51640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398104550 0x7ff3981953b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff38802c0b0 tx=0x7ff3880045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.199+0000 7ff38e7fc640 1 -- 172.21.15.83:0/3382007847 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38802c510 con 0x7ff398104550 2024-03-11T19:53:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.200+0000 7ff38e7fc640 1 -- 172.21.15.83:0/3382007847 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff38802cb50 con 0x7ff398104550 2024-03-11T19:53:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.200+0000 7ff39f9b6640 1 -- 172.21.15.83:0/3382007847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39806af80 con 0x7ff398104550 2024-03-11T19:53:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.200+0000 7ff38e7fc640 1 -- 172.21.15.83:0/3382007847 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388032860 con 0x7ff398104550 2024-03-11T19:53:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.200+0000 7ff39f9b6640 1 -- 172.21.15.83:0/3382007847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39806b430 con 0x7ff398104550 2024-03-11T19:53:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.202+0000 7ff39f9b6640 1 -- 172.21.15.83:0/3382007847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff398067f70 con 0x7ff398104550 2024-03-11T19:53:18.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.203+0000 7ff38e7fc640 1 -- 172.21.15.83:0/3382007847 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff38803b040 con 0x7ff398104550 2024-03-11T19:53:18.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.204+0000 7ff38e7fc640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3840a1c00 0x7ff3840a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.204+0000 7ff39d752640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3840a1c00 0x7ff3840a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.204+0000 7ff38e7fc640 1 -- 172.21.15.83:0/3382007847 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff3880ef000 con 0x7ff398104550 2024-03-11T19:53:18.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.205+0000 7ff39d752640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3840a1c00 0x7ff3840a40c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff3981965a0 tx=0x7ff394009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.209+0000 7ff38e7fc640 1 -- 172.21.15.83:0/3382007847 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff3880b8220 con 0x7ff398104550 2024-03-11T19:53:18.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.529+0000 7ff39f9b6640 1 -- 172.21.15.83:0/3382007847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3981089d0 con 0x7ff398104550 2024-03-11T19:53:18.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.531+0000 7ff38e7fc640 1 -- 172.21.15.83:0/3382007847 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7ff3981089d0 con 0x7ff398104550 2024-03-11T19:53:18.535 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:53:18.535 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:53:18.536 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:53:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.534+0000 7ff383fff640 1 -- 172.21.15.83:0/3382007847 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3840a1c00 msgr2=0x7ff3840a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.534+0000 7ff383fff640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3840a1c00 0x7ff3840a40c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff3981965a0 tx=0x7ff394009260 comp rx=0 tx=0).stop 2024-03-11T19:53:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.535+0000 7ff383fff640 1 -- 172.21.15.83:0/3382007847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398104550 msgr2=0x7ff3981953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.535+0000 7ff383fff640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398104550 0x7ff3981953b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff38802c0b0 tx=0x7ff3880045a0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.535+0000 7ff383fff640 1 -- 172.21.15.83:0/3382007847 shutdown_connections 2024-03-11T19:53:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.535+0000 7ff383fff640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff3840a1c00 0x7ff3840a40c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.535+0000 7ff383fff640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff398106e80 0x7ff3981a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.535+0000 7ff383fff640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398104550 0x7ff3981953b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.535+0000 7ff383fff640 1 --2- 172.21.15.83:0/3382007847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff398101c20 0x7ff398194e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.535+0000 7ff383fff640 1 -- 172.21.15.83:0/3382007847 >> 172.21.15.83:0/3382007847 conn(0x7ff3980fb610 msgr2=0x7ff398101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.536+0000 7ff383fff640 1 -- 172.21.15.83:0/3382007847 shutdown_connections 2024-03-11T19:53:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.536+0000 7ff383fff640 1 -- 172.21.15.83:0/3382007847 wait complete. 2024-03-11T19:53:18.559 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:53:18.559 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:53:18.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.702+0000 7fe522b77640 1 -- 172.21.15.83:0/1066965944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c101c70 msgr2=0x7fe51c108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.702+0000 7fe522b77640 1 --2- 172.21.15.83:0/1066965944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c101c70 0x7fe51c108500 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe514009ca0 tx=0x7fe51402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.702+0000 7fe522b77640 1 -- 172.21.15.83:0/1066965944 shutdown_connections 2024-03-11T19:53:18.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.702+0000 7fe522b77640 1 --2- 172.21.15.83:0/1066965944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c101c70 0x7fe51c108500 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.702+0000 7fe522b77640 1 --2- 172.21.15.83:0/1066965944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c1012b0 0x7fe51c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.702+0000 7fe522b77640 1 --2- 172.21.15.83:0/1066965944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe51c1005a0 0x7fe51c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.702+0000 7fe522b77640 1 -- 172.21.15.83:0/1066965944 >> 172.21.15.83:0/1066965944 conn(0x7fe51c0fad70 msgr2=0x7fe51c0fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.702+0000 7fe522b77640 1 -- 172.21.15.83:0/1066965944 shutdown_connections 2024-03-11T19:53:18.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.702+0000 7fe522b77640 1 -- 172.21.15.83:0/1066965944 wait complete. 2024-03-11T19:53:18.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe522b77640 1 Processor -- start 2024-03-11T19:53:18.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe522b77640 1 -- start start 2024-03-11T19:53:18.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe522b77640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c1005a0 0x7fe51c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe522b77640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe51c1012b0 0x7fe51c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe522b77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c101c70 0x7fe51c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe522b77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19ecd0 con 0x7fe51c101c70 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe522b77640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19ee40 con 0x7fe51c1012b0 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe522b77640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c19f000 con 0x7fe51c1005a0 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe521114640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c101c70 0x7fe51c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe521114640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c101c70 0x7fe51c19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40638/0 (socket says 172.21.15.83:40638) 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe521114640 1 -- 172.21.15.83:0/2810569200 learned_addr learned my addr 172.21.15.83:0/2810569200 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe520913640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c1005a0 0x7fe51c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.703+0000 7fe51bfff640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe51c1012b0 0x7fe51c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.704+0000 7fe521114640 1 -- 172.21.15.83:0/2810569200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c1005a0 msgr2=0x7fe51c1971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.704+0000 7fe521114640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c1005a0 0x7fe51c1971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.704+0000 7fe521114640 1 -- 172.21.15.83:0/2810569200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe51c1012b0 msgr2=0x7fe51c197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.704+0000 7fe521114640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe51c1012b0 0x7fe51c197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.704+0000 7fe521114640 1 -- 172.21.15.83:0/2810569200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe514009900 con 0x7fe51c101c70 2024-03-11T19:53:18.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.704+0000 7fe521114640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c101c70 0x7fe51c19e790 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe514037040 tx=0x7fe514034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.704+0000 7fe519ffb640 1 -- 172.21.15.83:0/2810569200 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514032eb0 con 0x7fe51c101c70 2024-03-11T19:53:18.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.704+0000 7fe522b77640 1 -- 172.21.15.83:0/2810569200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c19f280 con 0x7fe51c101c70 2024-03-11T19:53:18.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.704+0000 7fe519ffb640 1 -- 172.21.15.83:0/2810569200 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe514003e50 con 0x7fe51c101c70 2024-03-11T19:53:18.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.704+0000 7fe519ffb640 1 -- 172.21.15.83:0/2810569200 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514040450 con 0x7fe51c101c70 2024-03-11T19:53:18.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.704+0000 7fe522b77640 1 -- 172.21.15.83:0/2810569200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c19f780 con 0x7fe51c101c70 2024-03-11T19:53:18.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.708+0000 7fe519ffb640 1 -- 172.21.15.83:0/2810569200 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe51400a040 con 0x7fe51c101c70 2024-03-11T19:53:18.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.709+0000 7fe525824740 1 Processor -- start 2024-03-11T19:53:18.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.709+0000 7fe525824740 1 -- start start 2024-03-11T19:53:18.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.709+0000 7fe525824740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562105690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.709+0000 7fe525824740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105c60 0x55556210a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.709+0000 7fe525824740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x55556210d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.709+0000 7fe525824740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555620c1960 con 0x55556210add0 2024-03-11T19:53:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.709+0000 7fe525824740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562075310 con 0x555562105270 2024-03-11T19:53:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.709+0000 7fe525824740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562072840 con 0x555562105c60 2024-03-11T19:53:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.711+0000 7fe519ffb640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe50c0a1a30 0x7fe50c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.711+0000 7fe519ffb640 1 -- 172.21.15.83:0/2810569200 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe5140f01e0 con 0x7fe51c101c70 2024-03-11T19:53:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.713+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562105690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.713+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562105690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39164/0 (socket says 172.21.15.83:39164) 2024-03-11T19:53:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.713+0000 7fe51bfff640 1 -- 172.21.15.83:0/808093298 learned_addr learned my addr 172.21.15.83:0/808093298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.713+0000 7fe51bfff640 1 --2- 172.21.15.83:0/808093298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x55556210d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.714+0000 7fe51bfff640 1 -- 172.21.15.83:0/808093298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105c60 msgr2=0x55556210a760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:53:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.714+0000 7fe51bfff640 1 --2- 172.21.15.83:0/808093298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105c60 0x55556210a760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.714+0000 7fe51bfff640 1 -- 172.21.15.83:0/808093298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 msgr2=0x55556210d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.714+0000 7fe51bfff640 1 --2- 172.21.15.83:0/808093298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x55556210d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.714+0000 7fe51bfff640 1 -- 172.21.15.83:0/808093298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555562030060 con 0x555562105270 2024-03-11T19:53:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.718+0000 7fe521114640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe50c0a1a30 0x7fe50c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.720+0000 7fe51bfff640 1 --2- 172.21.15.83:0/808093298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562105690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe50800b6c0 tx=0x7fe50800b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e9b5da1b9dc413 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.723+0000 7fe506ffd640 1 -- 172.21.15.83:0/808093298 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508012d00 con 0x555562105270 2024-03-11T19:53:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.723+0000 7fe506ffd640 1 -- 172.21.15.83:0/808093298 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe508012e60 con 0x555562105270 2024-03-11T19:53:18.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.723+0000 7fe506ffd640 1 -- 172.21.15.83:0/808093298 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508003850 con 0x555562105270 2024-03-11T19:53:18.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.724+0000 7fe525824740 1 -- 172.21.15.83:0/808093298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 msgr2=0x555562105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.724+0000 7fe525824740 1 --2- 172.21.15.83:0/808093298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562105690 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe50800b6c0 tx=0x7fe50800b650 comp rx=0 tx=0).stop 2024-03-11T19:53:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.727+0000 7fe521114640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe50c0a1a30 0x7fe50c0a3ef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe5140b9660 tx=0x7fe51402c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.731+0000 7fe525824740 1 -- 172.21.15.83:0/808093298 shutdown_connections 2024-03-11T19:53:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.731+0000 7fe525824740 1 --2- 172.21.15.83:0/808093298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x55556210d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.731+0000 7fe525824740 1 --2- 172.21.15.83:0/808093298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105c60 0x55556210a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.731+0000 7fe525824740 1 --2- 172.21.15.83:0/808093298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562105690 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.731+0000 7fe525824740 1 -- 172.21.15.83:0/808093298 >> 172.21.15.83:0/808093298 conn(0x5555621007e0 msgr2=0x555562100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.735+0000 7fe525824740 1 -- 172.21.15.83:0/808093298 shutdown_connections 2024-03-11T19:53:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.735+0000 7fe525824740 1 -- 172.21.15.83:0/808093298 wait complete. 2024-03-11T19:53:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.735+0000 7fe525824740 1 -- start start 2024-03-11T19:53:18.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.736+0000 7fe525824740 1 Processor -- start 2024-03-11T19:53:18.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.736+0000 7fe525824740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105270 0x55556210f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.736+0000 7fe525824740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 0x55556210f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.737+0000 7fe525824740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x5555621168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.737+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105270 0x55556210f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.737+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105270 0x55556210f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45702/0 (socket says 172.21.15.83:45702) 2024-03-11T19:53:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.737+0000 7fe520913640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 0x55556210f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.737+0000 7fe520913640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 0x55556210f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39166/0 (socket says 172.21.15.83:39166) 2024-03-11T19:53:18.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.737+0000 7fe51bfff640 1 -- 172.21.15.83:0/466816025 learned_addr learned my addr 172.21.15.83:0/466816025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.737+0000 7fe525824740 1 -- 172.21.15.83:0/466816025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556206e020 con 0x55556210add0 2024-03-11T19:53:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.737+0000 7fe525824740 1 -- 172.21.15.83:0/466816025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555561d41430 con 0x555562105c60 2024-03-11T19:53:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.737+0000 7fe525824740 1 -- 172.21.15.83:0/466816025 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555561cfde00 con 0x555562105270 2024-03-11T19:53:18.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.737+0000 7fe525824740 10 client.0 _ll_register_callbacks cb 0x555562015cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.744+0000 7fe51bfff640 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x5555621168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.744+0000 7fe51bfff640 1 -- 172.21.15.83:0/466816025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 msgr2=0x55556210f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.744+0000 7fe51bfff640 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 0x55556210f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.744+0000 7fe51bfff640 1 -- 172.21.15.83:0/466816025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 msgr2=0x5555621168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.744+0000 7fe51bfff640 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x5555621168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.744+0000 7fe51bfff640 1 -- 172.21.15.83:0/466816025 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50801a040 con 0x555562105270 2024-03-11T19:53:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.745+0000 7fe51bfff640 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105270 0x55556210f330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c010 tx=0x7fe50800b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79c974124aa2ca04 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.745+0000 7fe4fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:53:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.745+0000 7fe4fbfff640 1 -- 172.21.15.83:0/466816025 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5080044e0 con 0x555562105270 2024-03-11T19:53:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.745+0000 7fe4fbfff640 1 -- 172.21.15.83:0/466816025 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe508017040 con 0x555562105270 2024-03-11T19:53:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.746+0000 7fe525824740 1 -- 172.21.15.83:0/466816025 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555561d6d7e0 con 0x555562105270 2024-03-11T19:53:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.746+0000 7fe4fbfff640 1 -- 172.21.15.83:0/466816025 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe508005040 con 0x555562105270 2024-03-11T19:53:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.746+0000 7fe525824740 10 client.27523 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.746+0000 7fe525824740 1 -- 172.21.15.83:0/466816025 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55556206c6a0 con 0x555562105270 2024-03-11T19:53:18.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.746+0000 7fe4fbfff640 1 -- 172.21.15.83:0/466816025 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe508024070 con 0x555562105270 2024-03-11T19:53:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.748+0000 7fe4fbfff640 1 -- 172.21.15.83:0/466816025 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe508004a90 con 0x555562105270 2024-03-11T19:53:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.748+0000 7fe525824740 10 client.27523 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.748+0000 7fe525824740 10 client.27523 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:53:18.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.748+0000 7fe525824740 20 client.27523 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.748+0000 7fe525824740 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55556211b310 0x55556211d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.748+0000 7fe525824740 4 client.27523 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.748+0000 7fe525824740 1 -- 172.21.15.83:0/466816025 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555561d6d7e0 con 0x55556211b310 2024-03-11T19:53:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.748+0000 7fe520913640 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55556211b310 0x55556211d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.749+0000 7fe520913640 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55556211b310 0x55556211d730 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e6ccfde59d448dc server_cookie=83c806c07348c7dd in_seq=0 out_seq=0 2024-03-11T19:53:18.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.749+0000 7fe4fbfff640 10 client.27523 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:53:18.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.750+0000 7fe4fbfff640 1 -- 172.21.15.83:0/466816025 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555561d6d7e0 con 0x55556211b310 2024-03-11T19:53:18.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.750+0000 7fe4fbfff640 10 client.27523 handle_command_reply: tid=0 2024-03-11T19:53:18.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.751+0000 7fe525824740 1 client.27523 shutdown 2024-03-11T19:53:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.752+0000 7fe525824740 1 -- 172.21.15.83:0/466816025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105270 msgr2=0x55556210f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.752+0000 7fe525824740 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105270 0x55556210f330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c010 tx=0x7fe50800b710 comp rx=0 tx=0).stop 2024-03-11T19:53:18.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.752+0000 7fe525824740 1 -- 172.21.15.83:0/466816025 shutdown_connections 2024-03-11T19:53:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.752+0000 7fe525824740 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55556211b310 0x55556211d730 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.752+0000 7fe525824740 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x5555621168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.752+0000 7fe4fbfff640 0 client.27523 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:53:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.752+0000 7fe525824740 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 0x55556210f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.752+0000 7fe525824740 1 --2- 172.21.15.83:0/466816025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105270 0x55556210f330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.752+0000 7fe525824740 1 -- 172.21.15.83:0/466816025 >> 172.21.15.83:0/466816025 conn(0x5555621007e0 msgr2=0x5555620ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.756+0000 7fe525824740 1 -- 172.21.15.83:0/466816025 shutdown_connections 2024-03-11T19:53:18.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.756+0000 7fe525824740 1 -- 172.21.15.83:0/466816025 wait complete. 2024-03-11T19:53:18.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.756+0000 7fe525824740 20 client.27523 trim_cache size 0 max 16384 2024-03-11T19:53:18.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.777+0000 7fe525824740 1 Processor -- start 2024-03-11T19:53:18.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.777+0000 7fe525824740 1 -- start start 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe525824740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562107020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe525824740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105c60 0x555562107560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe525824740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x555562116840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe525824740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555561ff3130 con 0x55556210add0 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe525824740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555561d6d7e0 con 0x555562105270 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe525824740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555620f2a40 con 0x555562105c60 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x555562116840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562107020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39178/0 (socket says 172.21.15.83:39178) 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe51bfff640 1 -- 172.21.15.83:0/424993619 learned_addr learned my addr 172.21.15.83:0/424993619 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe520913640 1 --2- 172.21.15.83:0/424993619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105c60 0x555562107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe51bfff640 1 -- 172.21.15.83:0/424993619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105c60 msgr2=0x555562107560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe51bfff640 1 --2- 172.21.15.83:0/424993619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105c60 0x555562107560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe51bfff640 1 -- 172.21.15.83:0/424993619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 msgr2=0x555562116840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.778+0000 7fe51bfff640 1 --2- 172.21.15.83:0/424993619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x555562116840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.779+0000 7fe51bfff640 1 -- 172.21.15.83:0/424993619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555562008c20 con 0x555562105270 2024-03-11T19:53:18.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.779+0000 7fe51bfff640 1 --2- 172.21.15.83:0/424993619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562107020 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe508005d00 tx=0x7fe508000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72957a1b9cb422e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.779+0000 7fe5067fc640 1 -- 172.21.15.83:0/424993619 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508010950 con 0x555562105270 2024-03-11T19:53:18.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.779+0000 7fe5067fc640 1 -- 172.21.15.83:0/424993619 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe508024030 con 0x555562105270 2024-03-11T19:53:18.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.779+0000 7fe5067fc640 1 -- 172.21.15.83:0/424993619 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50800be80 con 0x555562105270 2024-03-11T19:53:18.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.780+0000 7fe525824740 1 -- 172.21.15.83:0/424993619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 msgr2=0x555562107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.780+0000 7fe525824740 1 --2- 172.21.15.83:0/424993619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562107020 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe508005d00 tx=0x7fe508000bc0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.780+0000 7fe525824740 1 -- 172.21.15.83:0/424993619 shutdown_connections 2024-03-11T19:53:18.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.780+0000 7fe525824740 1 --2- 172.21.15.83:0/424993619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55556210add0 0x555562116840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.780+0000 7fe525824740 1 --2- 172.21.15.83:0/424993619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555562105c60 0x555562107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.780+0000 7fe525824740 1 --2- 172.21.15.83:0/424993619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105270 0x555562107020 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.780+0000 7fe525824740 1 -- 172.21.15.83:0/424993619 >> 172.21.15.83:0/424993619 conn(0x5555621007e0 msgr2=0x555562100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.780+0000 7fe525824740 1 -- 172.21.15.83:0/424993619 shutdown_connections 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.780+0000 7fe525824740 1 -- 172.21.15.83:0/424993619 wait complete. 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.780+0000 7fe525824740 1 -- start start 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe525824740 1 Processor -- start 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe525824740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555562105270 0x5555621164f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe525824740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 0x555562114b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe525824740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55556210add0 0x555562115040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe525824740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562116ce0 con 0x555562105270 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe525824740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555562115580 con 0x555562105c60 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe525824740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555621156f0 con 0x55556210add0 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe525824740 10 client.0 _ll_register_callbacks cb 0x7fe50800bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555562105270 0x5555621164f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555562105270 0x5555621164f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40672/0 (socket says 172.21.15.83:40672) 2024-03-11T19:53:18.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe51bfff640 1 -- 172.21.15.83:0/2677687747 learned_addr learned my addr 172.21.15.83:0/2677687747 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe520913640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 0x555562114b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe51bfff640 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55556210add0 0x555562115040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe520913640 1 -- 172.21.15.83:0/2677687747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55556210add0 msgr2=0x555562115040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe520913640 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55556210add0 0x555562115040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe520913640 1 -- 172.21.15.83:0/2677687747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555562105270 msgr2=0x5555621164f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe520913640 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555562105270 0x5555621164f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe520913640 1 -- 172.21.15.83:0/2677687747 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55556206c6a0 con 0x555562105c60 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.781+0000 7fe51bfff640 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555562105270 0x5555621164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.782+0000 7fe520913640 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 0x555562114b00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55556211e080 tx=0x7fe51000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fe7390d0b0ad5f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.782+0000 7fe4fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.782+0000 7fe4fbfff640 1 -- 172.21.15.83:0/2677687747 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510017050 con 0x555562105c60 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.782+0000 7fe4fbfff640 1 -- 172.21.15.83:0/2677687747 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe510005220 con 0x555562105c60 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.782+0000 7fe525824740 1 -- 172.21.15.83:0/2677687747 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe508012e60 con 0x555562105c60 2024-03-11T19:53:18.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.782+0000 7fe4fbfff640 1 -- 172.21.15.83:0/2677687747 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5100053e0 con 0x555562105c60 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.782+0000 7fe525824740 10 client.27425 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.782+0000 7fe525824740 1 -- 172.21.15.83:0/2677687747 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55556202e8c0 con 0x555562105c60 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.782+0000 7fe4fbfff640 1 -- 172.21.15.83:0/2677687747 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe508012e60 con 0x555562105c60 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.783+0000 7fe4fbfff640 1 -- 172.21.15.83:0/2677687747 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe510020540 con 0x555562105c60 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.783+0000 7fe525824740 10 client.27425 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.783+0000 7fe525824740 10 client.27425 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.783+0000 7fe525824740 20 client.27425 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.783+0000 7fe525824740 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555562172f20 0x555562175340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.783+0000 7fe525824740 4 client.27425 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.783+0000 7fe525824740 1 -- 172.21.15.83:0/2677687747 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555561daa720 con 0x555562172f20 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.783+0000 7fe51bfff640 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555562172f20 0x555562175340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe51bfff640 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555562172f20 0x555562175340 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=454780661d609d25 server_cookie=92c88b210d2c7ca5 in_seq=0 out_seq=0 2024-03-11T19:53:18.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe4fbfff640 10 client.27425 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe4fbfff640 1 -- 172.21.15.83:0/2677687747 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555561daa720 con 0x555562172f20 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe4fbfff640 10 client.27425 handle_command_reply: tid=0 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe525824740 1 client.27425 shutdown 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe525824740 1 -- 172.21.15.83:0/2677687747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 msgr2=0x555562114b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe525824740 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 0x555562114b00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55556211e080 tx=0x7fe51000b690 comp rx=0 tx=0).stop 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe525824740 1 -- 172.21.15.83:0/2677687747 shutdown_connections 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe525824740 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x555562172f20 0x555562175340 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe4fbfff640 0 client.27425 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe525824740 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55556210add0 0x555562115040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe525824740 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555562105c60 0x555562114b00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe525824740 1 --2- 172.21.15.83:0/2677687747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555562105270 0x5555621164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe525824740 1 -- 172.21.15.83:0/2677687747 >> 172.21.15.83:0/2677687747 conn(0x5555621007e0 msgr2=0x555562107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.784+0000 7fe525824740 1 -- 172.21.15.83:0/2677687747 shutdown_connections 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe525824740 1 -- 172.21.15.83:0/2677687747 wait complete. 2024-03-11T19:53:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe525824740 20 client.27425 trim_cache size 0 max 16384 2024-03-11T19:53:18.794 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:53:18.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 -- 172.21.15.83:0/2810569200 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe50c0a1a30 msgr2=0x7fe50c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe50c0a1a30 0x7fe50c0a3ef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe5140b9660 tx=0x7fe51402c0f0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 -- 172.21.15.83:0/2810569200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c101c70 msgr2=0x7fe51c19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c101c70 0x7fe51c19e790 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe514037040 tx=0x7fe514034d40 comp rx=0 tx=0).stop 2024-03-11T19:53:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 -- 172.21.15.83:0/2810569200 shutdown_connections 2024-03-11T19:53:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe50c0a1a30 0x7fe50c0a3ef0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe51c101c70 0x7fe51c19e790 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe51c1012b0 0x7fe51c197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 --2- 172.21.15.83:0/2810569200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c1005a0 0x7fe51c1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 -- 172.21.15.83:0/2810569200 >> 172.21.15.83:0/2810569200 conn(0x7fe51c0fad70 msgr2=0x7fe51c1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 -- 172.21.15.83:0/2810569200 shutdown_connections 2024-03-11T19:53:18.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.785+0000 7fe522b77640 1 -- 172.21.15.83:0/2810569200 wait complete. 2024-03-11T19:53:18.804 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:53:18.804 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:53:18.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:18 smithi083 ceph-mon[31864]: pgmap v771: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.6 MiB/s wr, 695 op/s 2024-03-11T19:53:18.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:18 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3382007847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:18.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:18 smithi137 ceph-mon[36220]: pgmap v771: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.6 MiB/s wr, 695 op/s 2024-03-11T19:53:18.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:18 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3382007847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:18 smithi204 ceph-mon[36739]: pgmap v771: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.6 MiB/s wr, 695 op/s 2024-03-11T19:53:18.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:18 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3382007847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.957+0000 7f59d084e640 1 -- 172.21.15.83:0/1054107046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc10aaf0 msgr2=0x7f59cc10cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.957+0000 7f59d084e640 1 --2- 172.21.15.83:0/1054107046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc10aaf0 0x7f59cc10cf60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009be0 tx=0x7f59c402bf10 comp rx=0 tx=0).stop 2024-03-11T19:53:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.962+0000 7f59d084e640 1 -- 172.21.15.83:0/1054107046 shutdown_connections 2024-03-11T19:53:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.962+0000 7f59d084e640 1 --2- 172.21.15.83:0/1054107046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59cc10d4a0 0x7f59cc10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.962+0000 7f59d084e640 1 --2- 172.21.15.83:0/1054107046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc10aaf0 0x7f59cc10cf60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.962+0000 7f59d084e640 1 --2- 172.21.15.83:0/1054107046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59cc108180 0x7f59cc10a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.962+0000 7f59d084e640 1 -- 172.21.15.83:0/1054107046 >> 172.21.15.83:0/1054107046 conn(0x7f59cc06dfe0 msgr2=0x7f59cc070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:18.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.968+0000 7f59d084e640 1 -- 172.21.15.83:0/1054107046 shutdown_connections 2024-03-11T19:53:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.970+0000 7f59d084e640 1 -- 172.21.15.83:0/1054107046 wait complete. 2024-03-11T19:53:18.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.971+0000 7f59d084e640 1 Processor -- start 2024-03-11T19:53:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.971+0000 7f59d084e640 1 -- start start 2024-03-11T19:53:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.972+0000 7f59d084e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59cc108180 0x7f59cc114750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.972+0000 7f59d084e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59cc10aaf0 0x7f59cc114c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.972+0000 7f59d084e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc10d4a0 0x7f59cc1b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.972+0000 7f59d084e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc117ce0 con 0x7f59cc10d4a0 2024-03-11T19:53:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.972+0000 7f59d084e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc117e50 con 0x7f59cc108180 2024-03-11T19:53:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.972+0000 7f59d084e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc1b6770 con 0x7f59cc10aaf0 2024-03-11T19:53:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.972+0000 7f59c9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59cc10aaf0 0x7f59cc114c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.972+0000 7f59c9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59cc10aaf0 0x7f59cc114c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45738/0 (socket says 172.21.15.83:45738) 2024-03-11T19:53:18.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.972+0000 7f59c9d9b640 1 -- 172.21.15.83:0/224952607 learned_addr learned my addr 172.21.15.83:0/224952607 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.976+0000 7f59cad9d640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc10d4a0 0x7f59cc1b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.977+0000 7f59ca59c640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59cc108180 0x7f59cc114750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.977+0000 7f59c9d9b640 1 -- 172.21.15.83:0/224952607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59cc108180 msgr2=0x7f59cc114750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.977+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59cc108180 0x7f59cc114750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.977+0000 7f59c9d9b640 1 -- 172.21.15.83:0/224952607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc10d4a0 msgr2=0x7f59cc1b6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.977+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc10d4a0 0x7f59cc1b6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:18.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.977+0000 7f59c9d9b640 1 -- 172.21.15.83:0/224952607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4009840 con 0x7f59cc10aaf0 2024-03-11T19:53:18.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.978+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59cc10aaf0 0x7f59cc114c90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f59c4036040 tx=0x7f59c4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:18.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.978+0000 7f59bb7fe640 1 -- 172.21.15.83:0/224952607 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40047f0 con 0x7f59cc10aaf0 2024-03-11T19:53:18.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.979+0000 7f59bb7fe640 1 -- 172.21.15.83:0/224952607 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59c40349b0 con 0x7f59cc10aaf0 2024-03-11T19:53:18.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.979+0000 7f59bb7fe640 1 -- 172.21.15.83:0/224952607 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4033b60 con 0x7f59cc10aaf0 2024-03-11T19:53:18.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.981+0000 7f59d084e640 1 -- 172.21.15.83:0/224952607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc1b6970 con 0x7f59cc10aaf0 2024-03-11T19:53:18.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.981+0000 7f59d084e640 1 -- 172.21.15.83:0/224952607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc1b6e00 con 0x7f59cc10aaf0 2024-03-11T19:53:18.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.987+0000 7f59bb7fe640 1 -- 172.21.15.83:0/224952607 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f59c4034b20 con 0x7f59cc10aaf0 2024-03-11T19:53:18.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.988+0000 7f59d3426740 1 Processor -- start 2024-03-11T19:53:18.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.988+0000 7f59bb7fe640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f59b40a1b80 0x7f59b40a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.989+0000 7f59bb7fe640 1 -- 172.21.15.83:0/224952607 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f59c402ce40 con 0x7f59cc10aaf0 2024-03-11T19:53:18.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.990+0000 7f59d3426740 1 -- start start 2024-03-11T19:53:18.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.990+0000 7f59d3426740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cb9d270 0x55a77cb9d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.990+0000 7f59d3426740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 0x55a77cba2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.991+0000 7f59d3426740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 0x55a77cba5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:18.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.991+0000 7f59d3426740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77cb59960 con 0x55a77cb9d270 2024-03-11T19:53:18.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.991+0000 7f59c9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 0x55a77cba2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.991+0000 7f59c9d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 0x55a77cba2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45754/0 (socket says 172.21.15.83:45754) 2024-03-11T19:53:18.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.991+0000 7f59c9d9b640 1 -- 172.21.15.83:0/3233240118 learned_addr learned my addr 172.21.15.83:0/3233240118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:18.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.995+0000 7f59cad9d640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f59b40a1b80 0x7f59b40a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.995+0000 7f59cad9d640 1 --2- 172.21.15.83:0/3233240118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 0x55a77cba5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.995+0000 7f59ca59c640 1 --2- 172.21.15.83:0/3233240118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cb9d270 0x55a77cb9d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:18.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.995+0000 7f59d3426740 1 -- 172.21.15.83:0/3233240118 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77cb0d310 con 0x55a77cba2dd0 2024-03-11T19:53:18.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.995+0000 7f59d3426740 1 -- 172.21.15.83:0/3233240118 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77cb0a840 con 0x55a77cb9dc60 2024-03-11T19:53:19.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.996+0000 7f59c9d9b640 1 -- 172.21.15.83:0/3233240118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 msgr2=0x55a77cba5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.996+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/3233240118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 0x55a77cba5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.996+0000 7f59c9d9b640 1 -- 172.21.15.83:0/3233240118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cb9d270 msgr2=0x55a77cb9d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.996+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/3233240118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cb9d270 0x55a77cb9d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.996+0000 7f59c9d9b640 1 -- 172.21.15.83:0/3233240118 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a77cac8060 con 0x55a77cb9dc60 2024-03-11T19:53:19.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:18.996+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/3233240118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 0x55a77cba2760 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f59c40b8e70 tx=0x7f59c4009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3db284ce2fc770f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:19.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.001+0000 7f59b8ff9640 1 -- 172.21.15.83:0/3233240118 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c40bca10 con 0x55a77cb9dc60 2024-03-11T19:53:19.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.001+0000 7f59b8ff9640 1 -- 172.21.15.83:0/3233240118 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59c40c0b60 con 0x55a77cb9dc60 2024-03-11T19:53:19.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.001+0000 7f59cad9d640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f59b40a1b80 0x7f59b40a4040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f59c0005d50 tx=0x7f59c000aa00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:19.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.002+0000 7f59d3426740 1 -- 172.21.15.83:0/3233240118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 msgr2=0x55a77cba2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.002+0000 7f59d3426740 1 --2- 172.21.15.83:0/3233240118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 0x55a77cba2760 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f59c40b8e70 tx=0x7f59c4009500 comp rx=0 tx=0).stop 2024-03-11T19:53:19.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.004+0000 7f59d3426740 1 -- 172.21.15.83:0/3233240118 shutdown_connections 2024-03-11T19:53:19.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.004+0000 7f59d3426740 1 --2- 172.21.15.83:0/3233240118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 0x55a77cba5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.004+0000 7f59d3426740 1 --2- 172.21.15.83:0/3233240118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 0x55a77cba2760 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.004+0000 7f59d3426740 1 --2- 172.21.15.83:0/3233240118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cb9d270 0x55a77cb9d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.004+0000 7f59d3426740 1 -- 172.21.15.83:0/3233240118 >> 172.21.15.83:0/3233240118 conn(0x55a77cb987e0 msgr2=0x55a77cb98be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:19.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.010+0000 7f59d3426740 1 -- 172.21.15.83:0/3233240118 shutdown_connections 2024-03-11T19:53:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.011+0000 7f59d3426740 1 -- 172.21.15.83:0/3233240118 wait complete. 2024-03-11T19:53:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.012+0000 7f59d3426740 1 -- start start 2024-03-11T19:53:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.012+0000 7f59d3426740 1 Processor -- start 2024-03-11T19:53:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.012+0000 7f59d3426740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9d270 0x55a77cba7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.012+0000 7f59d3426740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cb9dc60 0x55a77cba7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.012+0000 7f59d3426740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 0x55a77cbae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.013+0000 7f59d3426740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77cb06020 con 0x55a77cba2dd0 2024-03-11T19:53:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.013+0000 7f59d3426740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77c7d9430 con 0x55a77cb9dc60 2024-03-11T19:53:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.013+0000 7f59d3426740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77c795e00 con 0x55a77cb9d270 2024-03-11T19:53:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.013+0000 7f59d3426740 10 client.0 _ll_register_callbacks cb 0x55a77caadcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.013+0000 7f59cad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 0x55a77cbae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.013+0000 7f59cad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 0x55a77cbae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40714/0 (socket says 172.21.15.83:40714) 2024-03-11T19:53:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.013+0000 7f59cad9d640 1 -- 172.21.15.83:0/21783373 learned_addr learned my addr 172.21.15.83:0/21783373 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.014+0000 7f59ca59c640 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9d270 0x55a77cba7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.014+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cb9dc60 0x55a77cba7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.015+0000 7f59cad9d640 1 -- 172.21.15.83:0/21783373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9d270 msgr2=0x55a77cba7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.015+0000 7f59cad9d640 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9d270 0x55a77cba7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.015+0000 7f59cad9d640 1 -- 172.21.15.83:0/21783373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cb9dc60 msgr2=0x55a77cba7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.015+0000 7f59cad9d640 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cb9dc60 0x55a77cba7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.015+0000 7f59cad9d640 1 -- 172.21.15.83:0/21783373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc1b6e00 con 0x55a77cba2dd0 2024-03-11T19:53:19.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.016+0000 7f59cad9d640 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 0x55a77cbae730 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f59c000fe20 tx=0x7f59c0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fcd05067aed2da server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:19.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.016+0000 7f59b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:53:19.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.016+0000 7f59b1ffb640 1 -- 172.21.15.83:0/21783373 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c001e600 con 0x55a77cba2dd0 2024-03-11T19:53:19.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.016+0000 7f59b1ffb640 1 -- 172.21.15.83:0/21783373 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f59c001e760 con 0x55a77cba2dd0 2024-03-11T19:53:19.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.016+0000 7f59d3426740 1 -- 172.21.15.83:0/21783373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a77c842720 con 0x55a77cba2dd0 2024-03-11T19:53:19.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.017+0000 7f59b1ffb640 1 -- 172.21.15.83:0/21783373 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59c000fe50 con 0x55a77cba2dd0 2024-03-11T19:53:19.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.017+0000 7f59d3426740 10 client.17979 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:19.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.017+0000 7f59d3426740 1 -- 172.21.15.83:0/21783373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a77cb046a0 con 0x55a77cba2dd0 2024-03-11T19:53:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.017+0000 7f59b1ffb640 1 -- 172.21.15.83:0/21783373 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a77c842720 con 0x55a77cba2dd0 2024-03-11T19:53:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.017+0000 7f59b1ffb640 1 -- 172.21.15.83:0/21783373 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59c0031bc0 con 0x55a77cba2dd0 2024-03-11T19:53:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.017+0000 7f59d3426740 10 client.17979 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.017+0000 7f59d3426740 10 client.17979 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:53:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.017+0000 7f59d3426740 20 client.17979 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.017+0000 7f59d3426740 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a77cbb34a0 0x55a77cbb58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.017+0000 7f59d3426740 4 client.17979 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:19.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.017+0000 7f59d3426740 1 -- 172.21.15.83:0/21783373 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a77c891890 con 0x55a77cbb34a0 2024-03-11T19:53:19.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.022+0000 7f59ca59c640 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a77cbb34a0 0x55a77cbb58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.023+0000 7f59ca59c640 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a77cbb34a0 0x55a77cbb58c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=168c85c8ed14dfe2 server_cookie=3bdad1b246b6f3 in_seq=0 out_seq=0 2024-03-11T19:53:19.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.023+0000 7f59b1ffb640 10 client.17979 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:53:19.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.024+0000 7f59b1ffb640 1 -- 172.21.15.83:0/21783373 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a77c891890 con 0x55a77cbb34a0 2024-03-11T19:53:19.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.024+0000 7f59b1ffb640 10 client.17979 handle_command_reply: tid=0 2024-03-11T19:53:19.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.024+0000 7f59d3426740 1 client.17979 shutdown 2024-03-11T19:53:19.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.025+0000 7f59d3426740 1 -- 172.21.15.83:0/21783373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 msgr2=0x55a77cbae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.025+0000 7f59d3426740 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 0x55a77cbae730 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f59c000fe20 tx=0x7f59c0019000 comp rx=0 tx=0).stop 2024-03-11T19:53:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.026+0000 7f59d3426740 1 -- 172.21.15.83:0/21783373 shutdown_connections 2024-03-11T19:53:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.026+0000 7f59d3426740 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a77cbb34a0 0x55a77cbb58c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.026+0000 7f59d3426740 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 0x55a77cbae730 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.026+0000 7f59d3426740 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cb9dc60 0x55a77cba7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.026+0000 7f59d3426740 1 --2- 172.21.15.83:0/21783373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9d270 0x55a77cba7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.026+0000 7f59d3426740 1 -- 172.21.15.83:0/21783373 >> 172.21.15.83:0/21783373 conn(0x55a77cb987e0 msgr2=0x55a77cb97b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.026+0000 7f59b1ffb640 0 client.17979 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:53:19.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.026+0000 7f59d3426740 1 -- 172.21.15.83:0/21783373 shutdown_connections 2024-03-11T19:53:19.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.026+0000 7f59d3426740 1 -- 172.21.15.83:0/21783373 wait complete. 2024-03-11T19:53:19.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.027+0000 7f59d3426740 20 client.17979 trim_cache size 0 max 16384 2024-03-11T19:53:19.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.048+0000 7f59d3426740 1 Processor -- start 2024-03-11T19:53:19.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.048+0000 7f59d3426740 1 -- start start 2024-03-11T19:53:19.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.048+0000 7f59d3426740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cb9d270 0x55a77cba7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.049+0000 7f59d3426740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 0x55a77cba7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.049+0000 7f59d3426740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 0x55a77cbae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.049+0000 7f59d3426740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77ca8b130 con 0x55a77cb9d270 2024-03-11T19:53:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.049+0000 7f59d3426740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77c891890 con 0x55a77cba2dd0 2024-03-11T19:53:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.049+0000 7f59cad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 0x55a77cbae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.049+0000 7f59cad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 0x55a77cbae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39228/0 (socket says 172.21.15.83:39228) 2024-03-11T19:53:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.049+0000 7f59cad9d640 1 -- 172.21.15.83:0/3905315553 learned_addr learned my addr 172.21.15.83:0/3905315553 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.049+0000 7f59ca59c640 1 --2- 172.21.15.83:0/3905315553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cb9d270 0x55a77cba7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.049+0000 7f59d3426740 1 -- 172.21.15.83:0/3905315553 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77c842720 con 0x55a77cb9dc60 2024-03-11T19:53:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.050+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/3905315553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 0x55a77cba7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.050+0000 7f59ca59c640 1 -- 172.21.15.83:0/3905315553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 msgr2=0x55a77cba7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.050+0000 7f59ca59c640 1 --2- 172.21.15.83:0/3905315553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 0x55a77cba7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.050+0000 7f59ca59c640 1 -- 172.21.15.83:0/3905315553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 msgr2=0x55a77cbae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.050+0000 7f59ca59c640 1 --2- 172.21.15.83:0/3905315553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 0x55a77cbae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.050+0000 7f59ca59c640 1 -- 172.21.15.83:0/3905315553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a77caa0c20 con 0x55a77cb9d270 2024-03-11T19:53:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.050+0000 7f59cad9d640 1 --2- 172.21.15.83:0/3905315553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 0x55a77cbae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:19.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.050+0000 7f59ca59c640 1 --2- 172.21.15.83:0/3905315553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cb9d270 0x55a77cba7220 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x55a77cbb61c0 tx=0x7f59bc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795f0ecd3a0f34a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:19.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.050+0000 7f59b3fff640 1 -- 172.21.15.83:0/3905315553 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc017040 con 0x55a77cb9d270 2024-03-11T19:53:19.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.051+0000 7f59b3fff640 1 -- 172.21.15.83:0/3905315553 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59bc01d900 con 0x55a77cb9d270 2024-03-11T19:53:19.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.051+0000 7f59b3fff640 1 -- 172.21.15.83:0/3905315553 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc004a90 con 0x55a77cb9d270 2024-03-11T19:53:19.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.051+0000 7f59d3426740 1 -- 172.21.15.83:0/3905315553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cb9d270 msgr2=0x55a77cba7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.051+0000 7f59d3426740 1 --2- 172.21.15.83:0/3905315553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cb9d270 0x55a77cba7220 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x55a77cbb61c0 tx=0x7f59bc012820 comp rx=0 tx=0).stop 2024-03-11T19:53:19.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.052+0000 7f59d3426740 1 -- 172.21.15.83:0/3905315553 shutdown_connections 2024-03-11T19:53:19.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.052+0000 7f59d3426740 1 --2- 172.21.15.83:0/3905315553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cba2dd0 0x55a77cbae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.052+0000 7f59d3426740 1 --2- 172.21.15.83:0/3905315553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9dc60 0x55a77cba7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.052+0000 7f59d3426740 1 --2- 172.21.15.83:0/3905315553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cb9d270 0x55a77cba7220 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.052+0000 7f59d3426740 1 -- 172.21.15.83:0/3905315553 >> 172.21.15.83:0/3905315553 conn(0x55a77cb987e0 msgr2=0x55a77cba5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.052+0000 7f59d3426740 1 -- 172.21.15.83:0/3905315553 shutdown_connections 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.052+0000 7f59d3426740 1 -- 172.21.15.83:0/3905315553 wait complete. 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.052+0000 7f59d3426740 1 -- start start 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.052+0000 7f59d3426740 1 Processor -- start 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.052+0000 7f59d3426740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9d270 0x55a77cbae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59d3426740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cb9dc60 0x55a77cbae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59d3426740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 0x55a77cbac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59d3426740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77cbacc80 con 0x55a77cba2dd0 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59d3426740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77cbacdf0 con 0x55a77cb9dc60 2024-03-11T19:53:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59d3426740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77cbacfb0 con 0x55a77cb9d270 2024-03-11T19:53:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59ca59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9d270 0x55a77cbae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59cad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 0x55a77cbac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59ca59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9d270 0x55a77cbae010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45782/0 (socket says 172.21.15.83:45782) 2024-03-11T19:53:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59ca59c640 1 -- 172.21.15.83:0/3024949995 learned_addr learned my addr 172.21.15.83:0/3024949995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cb9dc60 0x55a77cbae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59d3426740 10 client.0 _ll_register_callbacks cb 0x7f59c40b8610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59c9d9b640 1 -- 172.21.15.83:0/3024949995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9d270 msgr2=0x55a77cbae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9d270 0x55a77cbae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59c9d9b640 1 -- 172.21.15.83:0/3024949995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 msgr2=0x55a77cbac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 0x55a77cbac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59c9d9b640 1 -- 172.21.15.83:0/3024949995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a77cb046a0 con 0x55a77cb9dc60 2024-03-11T19:53:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59cad9d640 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 0x55a77cbac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59c9d9b640 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cb9dc60 0x55a77cbae550 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55a77cb9e060 tx=0x7f59c4009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbdc9d3cb23f7f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.053+0000 7f59b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:53:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59b1ffb640 1 -- 172.21.15.83:0/3024949995 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4033ec0 con 0x55a77cb9dc60 2024-03-11T19:53:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59b1ffb640 1 -- 172.21.15.83:0/3024949995 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f59c40bca10 con 0x55a77cb9dc60 2024-03-11T19:53:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59d3426740 1 -- 172.21.15.83:0/3024949995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a77c8057e0 con 0x55a77cb9dc60 2024-03-11T19:53:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59b1ffb640 1 -- 172.21.15.83:0/3024949995 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59c40b9080 con 0x55a77cb9dc60 2024-03-11T19:53:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59d3426740 10 client.27440 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59d3426740 1 -- 172.21.15.83:0/3024949995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f59bc0099b0 con 0x55a77cb9dc60 2024-03-11T19:53:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59b1ffb640 1 -- 172.21.15.83:0/3024949995 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a77c8057e0 con 0x55a77cb9dc60 2024-03-11T19:53:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59b1ffb640 1 -- 172.21.15.83:0/3024949995 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59c403c040 con 0x55a77cb9dc60 2024-03-11T19:53:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59d3426740 10 client.27440 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59d3426740 10 client.27440 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:53:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59d3426740 20 client.27440 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:19.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.054+0000 7f59d3426740 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a77cc0a950 0x55a77cc0cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.055+0000 7f59d3426740 4 client.27440 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:53:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.055+0000 7f59d3426740 1 -- 172.21.15.83:0/3024949995 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a77c7b1700 con 0x55a77cc0a950 2024-03-11T19:53:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.055+0000 7f59ca59c640 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a77cc0a950 0x55a77cc0cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.057+0000 7f59ca59c640 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a77cc0a950 0x55a77cc0cd40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=764b0ddb19cad3e3 server_cookie=560cce006d1f2c82 in_seq=0 out_seq=0 2024-03-11T19:53:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.058+0000 7f59b1ffb640 10 client.27440 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:53:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.058+0000 7f59b1ffb640 1 -- 172.21.15.83:0/3024949995 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a77c7b1700 con 0x55a77cc0a950 2024-03-11T19:53:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.058+0000 7f59b1ffb640 10 client.27440 handle_command_reply: tid=0 2024-03-11T19:53:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.058+0000 7f59d3426740 1 client.27440 shutdown 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.058+0000 7f59d3426740 1 -- 172.21.15.83:0/3024949995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cb9dc60 msgr2=0x55a77cbae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.058+0000 7f59d3426740 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cb9dc60 0x55a77cbae550 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55a77cb9e060 tx=0x7f59c4009590 comp rx=0 tx=0).stop 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.059+0000 7f59d3426740 1 -- 172.21.15.83:0/3024949995 shutdown_connections 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.059+0000 7f59d3426740 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55a77cc0a950 0x55a77cc0cd40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.059+0000 7f59d3426740 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a77cba2dd0 0x55a77cbac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.059+0000 7f59d3426740 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a77cb9dc60 0x55a77cbae550 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.059+0000 7f59d3426740 1 --2- 172.21.15.83:0/3024949995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a77cb9d270 0x55a77cbae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.059+0000 7f59d3426740 1 -- 172.21.15.83:0/3024949995 >> 172.21.15.83:0/3024949995 conn(0x55a77cb987e0 msgr2=0x55a77cb9f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.059+0000 7f59b1ffb640 0 client.27440 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.059+0000 7f59d3426740 1 -- 172.21.15.83:0/3024949995 shutdown_connections 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.059+0000 7f59d3426740 1 -- 172.21.15.83:0/3024949995 wait complete. 2024-03-11T19:53:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.059+0000 7f59d3426740 20 client.27440 trim_cache size 0 max 16384 2024-03-11T19:53:19.069 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:53:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 -- 172.21.15.83:0/224952607 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f59b40a1b80 msgr2=0x7f59b40a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f59b40a1b80 0x7f59b40a4040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f59c0005d50 tx=0x7f59c000aa00 comp rx=0 tx=0).stop 2024-03-11T19:53:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 -- 172.21.15.83:0/224952607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59cc10aaf0 msgr2=0x7f59cc114c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59cc10aaf0 0x7f59cc114c90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f59c4036040 tx=0x7f59c4004270 comp rx=0 tx=0).stop 2024-03-11T19:53:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 -- 172.21.15.83:0/224952607 shutdown_connections 2024-03-11T19:53:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f59b40a1b80 0x7f59b40a4040 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59cc10d4a0 0x7f59cc1b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59cc10aaf0 0x7f59cc114c90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 --2- 172.21.15.83:0/224952607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59cc108180 0x7f59cc114750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 -- 172.21.15.83:0/224952607 >> 172.21.15.83:0/224952607 conn(0x7f59cc06dfe0 msgr2=0x7f59cc1097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 -- 172.21.15.83:0/224952607 shutdown_connections 2024-03-11T19:53:19.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.060+0000 7f59d084e640 1 -- 172.21.15.83:0/224952607 wait complete. 2024-03-11T19:53:19.079 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:53:19.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:53:19.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.293+0000 7fc7220df640 1 -- 172.21.15.83:0/1326238478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc71c102380 msgr2=0x7fc71c102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.293+0000 7fc7220df640 1 --2- 172.21.15.83:0/1326238478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc71c102380 0x7fc71c102800 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fc710009900 tx=0x7fc71002bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:19.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.294+0000 7fc7220df640 1 -- 172.21.15.83:0/1326238478 shutdown_connections 2024-03-11T19:53:19.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.294+0000 7fc7220df640 1 --2- 172.21.15.83:0/1326238478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc71c102d40 0x7fc71c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.294+0000 7fc7220df640 1 --2- 172.21.15.83:0/1326238478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc71c102380 0x7fc71c102800 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.294+0000 7fc7220df640 1 --2- 172.21.15.83:0/1326238478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc71c101190 0x7fc71c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.294+0000 7fc7220df640 1 -- 172.21.15.83:0/1326238478 >> 172.21.15.83:0/1326238478 conn(0x7fc71c0fc720 msgr2=0x7fc71c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:19.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.294+0000 7fc7220df640 1 -- 172.21.15.83:0/1326238478 shutdown_connections 2024-03-11T19:53:19.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.295+0000 7fc7220df640 1 -- 172.21.15.83:0/1326238478 wait complete. 2024-03-11T19:53:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.295+0000 7fc7220df640 1 Processor -- start 2024-03-11T19:53:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc7220df640 1 -- start start 2024-03-11T19:53:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc7220df640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc71c101190 0x7fc71c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc7220df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc71c102380 0x7fc71c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc7220df640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc71c102d40 0x7fc71c1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc71b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc71c101190 0x7fc71c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc71bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc71c102d40 0x7fc71c1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc71b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc71c101190 0x7fc71c199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45790/0 (socket says 172.21.15.83:45790) 2024-03-11T19:53:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc71b7fe640 1 -- 172.21.15.83:0/3647745535 learned_addr learned my addr 172.21.15.83:0/3647745535 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc7220df640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c1a0ef0 con 0x7fc71c102d40 2024-03-11T19:53:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc71affd640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc71c102380 0x7fc71c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc7220df640 1 -- 172.21.15.83:0/3647745535 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c1a1060 con 0x7fc71c102380 2024-03-11T19:53:19.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.296+0000 7fc7220df640 1 -- 172.21.15.83:0/3647745535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c1a1220 con 0x7fc71c101190 2024-03-11T19:53:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.297+0000 7fc71b7fe640 1 -- 172.21.15.83:0/3647745535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc71c102380 msgr2=0x7fc71c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.297+0000 7fc71b7fe640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc71c102380 0x7fc71c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.297+0000 7fc71b7fe640 1 -- 172.21.15.83:0/3647745535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc71c102d40 msgr2=0x7fc71c1a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.297+0000 7fc71b7fe640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc71c102d40 0x7fc71c1a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.297+0000 7fc71b7fe640 1 -- 172.21.15.83:0/3647745535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc710009560 con 0x7fc71c101190 2024-03-11T19:53:19.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.297+0000 7fc71bfff640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc71c102d40 0x7fc71c1a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:19.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.298+0000 7fc71b7fe640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc71c101190 0x7fc71c199400 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d770 tx=0x7fc70c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:19.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.298+0000 7fc718ff9640 1 -- 172.21.15.83:0/3647745535 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c009c30 con 0x7fc71c101190 2024-03-11T19:53:19.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.298+0000 7fc7220df640 1 -- 172.21.15.83:0/3647745535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71c1a1500 con 0x7fc71c101190 2024-03-11T19:53:19.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.298+0000 7fc7220df640 1 -- 172.21.15.83:0/3647745535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71c1a1a30 con 0x7fc71c101190 2024-03-11T19:53:19.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.298+0000 7fc718ff9640 1 -- 172.21.15.83:0/3647745535 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc70c00de80 con 0x7fc71c101190 2024-03-11T19:53:19.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.298+0000 7fc718ff9640 1 -- 172.21.15.83:0/3647745535 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c010460 con 0x7fc71c101190 2024-03-11T19:53:19.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.300+0000 7fc718ff9640 1 -- 172.21.15.83:0/3647745535 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc70c0105c0 con 0x7fc71c101190 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.301+0000 7fc724d8d740 1 Processor -- start 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.301+0000 7fc718ff9640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7040a1a30 0x7fc7040a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.301+0000 7fc724d8d740 1 -- start start 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.301+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497412a80 0x55f497412ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497413470 0x55f497417f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f49741aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc724d8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4973c6960 con 0x55f497412a80 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc724d8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49737a310 con 0x55f4974185e0 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc724d8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497377840 con 0x55f497413470 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc71affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497413470 0x55f497417f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc71affd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497413470 0x55f497417f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:45800/0 (socket says 172.21.15.83:45800) 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc71affd640 1 -- 172.21.15.83:0/2363606032 learned_addr learned my addr 172.21.15.83:0/2363606032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:19.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc71bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f49741aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc71bfff640 1 -- 172.21.15.83:0/2363606032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497413470 msgr2=0x55f497417f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc71bfff640 1 --2- 172.21.15.83:0/2363606032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497413470 0x55f497417f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc71bfff640 1 -- 172.21.15.83:0/2363606032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497412a80 msgr2=0x55f497412ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:53:19.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc71bfff640 1 --2- 172.21.15.83:0/2363606032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497412a80 0x55f497412ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.302+0000 7fc71bfff640 1 -- 172.21.15.83:0/2363606032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f497335060 con 0x55f4974185e0 2024-03-11T19:53:19.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.303+0000 7fc718ff9640 1 -- 172.21.15.83:0/3647745535 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc70c09e150 con 0x7fc71c101190 2024-03-11T19:53:19.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.303+0000 7fc71bfff640 1 --2- 172.21.15.83:0/2363606032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f49741aa70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc714011700 tx=0x7fc714011bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9372121c7e403486 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:19.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.303+0000 7fc71bfff640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7040a1a30 0x7fc7040a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.303+0000 7fc701ffb640 1 -- 172.21.15.83:0/2363606032 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71400fd50 con 0x55f4974185e0 2024-03-11T19:53:19.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.303+0000 7fc701ffb640 1 -- 172.21.15.83:0/2363606032 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc714011e10 con 0x55f4974185e0 2024-03-11T19:53:19.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.303+0000 7fc701ffb640 1 -- 172.21.15.83:0/2363606032 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714012810 con 0x55f4974185e0 2024-03-11T19:53:19.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.304+0000 7fc724d8d740 1 -- 172.21.15.83:0/2363606032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 msgr2=0x55f49741aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.304+0000 7fc71bfff640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7040a1a30 0x7fc7040a3ef0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc714000f80 tx=0x7fc714012ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:19.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.304+0000 7fc724d8d740 1 --2- 172.21.15.83:0/2363606032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f49741aa70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc714011700 tx=0x7fc714011bc0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.304+0000 7fc724d8d740 1 -- 172.21.15.83:0/2363606032 shutdown_connections 2024-03-11T19:53:19.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.304+0000 7fc724d8d740 1 --2- 172.21.15.83:0/2363606032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f49741aa70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.304+0000 7fc724d8d740 1 --2- 172.21.15.83:0/2363606032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497413470 0x55f497417f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.304+0000 7fc724d8d740 1 --2- 172.21.15.83:0/2363606032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497412a80 0x55f497412ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.304+0000 7fc724d8d740 1 -- 172.21.15.83:0/2363606032 >> 172.21.15.83:0/2363606032 conn(0x55f49740dff0 msgr2=0x55f49740e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.304+0000 7fc724d8d740 1 -- 172.21.15.83:0/2363606032 shutdown_connections 2024-03-11T19:53:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.304+0000 7fc724d8d740 1 -- 172.21.15.83:0/2363606032 wait complete. 2024-03-11T19:53:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc724d8d740 1 -- start start 2024-03-11T19:53:19.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc724d8d740 1 Processor -- start 2024-03-11T19:53:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f49741cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 0x55f49741d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f4974242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc724d8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497373020 con 0x55f497413470 2024-03-11T19:53:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc724d8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497046430 con 0x55f4974185e0 2024-03-11T19:53:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc724d8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497002e00 con 0x55f497412a80 2024-03-11T19:53:19.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc724d8d740 10 client.0 _ll_register_callbacks cb 0x55f49731acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc71affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 0x55f49741d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc71bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f4974242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc71bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f4974242d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:39264/0 (socket says 172.21.15.83:39264) 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc71bfff640 1 -- 172.21.15.83:0/1804473335 learned_addr learned my addr 172.21.15.83:0/1804473335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc71b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f49741cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.305+0000 7fc71affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 0x55f49741d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40740/0 (socket says 172.21.15.83:40740) 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc71bfff640 1 -- 172.21.15.83:0/1804473335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 msgr2=0x55f49741cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc71bfff640 1 --2- 172.21.15.83:0/1804473335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f49741cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc71bfff640 1 -- 172.21.15.83:0/1804473335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 msgr2=0x55f49741d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc71bfff640 1 --2- 172.21.15.83:0/1804473335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 0x55f49741d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc71bfff640 1 -- 172.21.15.83:0/1804473335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7140113e0 con 0x55f4974185e0 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc71bfff640 1 --2- 172.21.15.83:0/1804473335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f4974242d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc71400f730 tx=0x7fc714012a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e196424d99608c6f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:19.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc6faffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc6faffd640 1 -- 172.21.15.83:0/1804473335 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71401f040 con 0x55f4974185e0 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc6faffd640 1 -- 172.21.15.83:0/1804473335 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc714019e30 con 0x55f4974185e0 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc724d8d740 1 -- 172.21.15.83:0/1804473335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4970727e0 con 0x55f4974185e0 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc6faffd640 1 -- 172.21.15.83:0/1804473335 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc71401bce0 con 0x55f4974185e0 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc724d8d740 10 client.27449 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.306+0000 7fc724d8d740 1 -- 172.21.15.83:0/1804473335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f4973716a0 con 0x55f4974185e0 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.307+0000 7fc6faffd640 1 -- 172.21.15.83:0/1804473335 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f4970727e0 con 0x55f4974185e0 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.307+0000 7fc6faffd640 1 -- 172.21.15.83:0/1804473335 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc714004040 con 0x55f4974185e0 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.307+0000 7fc724d8d740 10 client.27449 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.307+0000 7fc724d8d740 10 client.27449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.307+0000 7fc724d8d740 20 client.27449 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.307+0000 7fc724d8d740 1 --2- 172.21.15.83:0/1804473335 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f497428f30 0x55f49742b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.307+0000 7fc724d8d740 4 client.27449 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.307+0000 7fc724d8d740 1 -- 172.21.15.83:0/1804473335 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4970af720 con 0x55f497428f30 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.307+0000 7fc71b7fe640 1 --2- 172.21.15.83:0/1804473335 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f497428f30 0x55f49742b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.308+0000 7fc6faffd640 10 client.27449 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.308+0000 7fc71b7fe640 1 --2- 172.21.15.83:0/1804473335 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f497428f30 0x55f49742b350 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d64346605cdbe1 server_cookie=624635ee55e60dd8 in_seq=0 out_seq=0 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.308+0000 7fc6faffd640 1 -- 172.21.15.83:0/1804473335 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4970af720 con 0x55f497428f30 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.308+0000 7fc6faffd640 10 client.27449 handle_command_reply: tid=0 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.309+0000 7fc724d8d740 1 client.27449 shutdown 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc724d8d740 1 -- 172.21.15.83:0/1804473335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 msgr2=0x55f4974242d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc724d8d740 1 --2- 172.21.15.83:0/1804473335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f4974242d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc71400f730 tx=0x7fc714012a30 comp rx=0 tx=0).stop 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc724d8d740 1 -- 172.21.15.83:0/1804473335 shutdown_connections 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc724d8d740 1 --2- 172.21.15.83:0/1804473335 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f497428f30 0x55f49742b350 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc724d8d740 1 --2- 172.21.15.83:0/1804473335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f4974242d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc6faffd640 0 client.27449 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:19.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc724d8d740 1 --2- 172.21.15.83:0/1804473335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 0x55f49741d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc724d8d740 1 --2- 172.21.15.83:0/1804473335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f49741cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc724d8d740 1 -- 172.21.15.83:0/1804473335 >> 172.21.15.83:0/1804473335 conn(0x55f49740dff0 msgr2=0x55f49740f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:19.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc724d8d740 1 -- 172.21.15.83:0/1804473335 shutdown_connections 2024-03-11T19:53:19.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc724d8d740 1 -- 172.21.15.83:0/1804473335 wait complete. 2024-03-11T19:53:19.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.310+0000 7fc724d8d740 20 client.27449 trim_cache size 0 max 16384 2024-03-11T19:53:19.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.327+0000 7fc724d8d740 1 Processor -- start 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.327+0000 7fc724d8d740 1 -- start start 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f49741cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f497413470 0x55f49741d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4974185e0 0x55f497424290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc724d8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4972f8130 con 0x55f4974185e0 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc724d8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4970727e0 con 0x55f497413470 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc724d8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4973f7a40 con 0x55f497412a80 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc71bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4974185e0 0x55f497424290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc71affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f497413470 0x55f49741d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc71b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f49741cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc71bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4974185e0 0x55f497424290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40750/0 (socket says 172.21.15.83:40750) 2024-03-11T19:53:19.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc71bfff640 1 -- 172.21.15.83:0/2424440130 learned_addr learned my addr 172.21.15.83:0/2424440130 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc71bfff640 1 -- 172.21.15.83:0/2424440130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 msgr2=0x55f49741cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.328+0000 7fc71bfff640 1 --2- 172.21.15.83:0/2424440130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f49741cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.329+0000 7fc71bfff640 1 -- 172.21.15.83:0/2424440130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f497413470 msgr2=0x55f49741d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.329+0000 7fc71bfff640 1 --2- 172.21.15.83:0/2424440130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f497413470 0x55f49741d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.329+0000 7fc71bfff640 1 -- 172.21.15.83:0/2424440130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f49730dc20 con 0x55f4974185e0 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.329+0000 7fc71bfff640 1 --2- 172.21.15.83:0/2424440130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4974185e0 0x55f497424290 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc714011c00 tx=0x7fc714007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53571f3dac5e5a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.329+0000 7fc71affd640 1 --2- 172.21.15.83:0/2424440130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f497413470 0x55f49741d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.329+0000 7fc71b7fe640 1 --2- 172.21.15.83:0/2424440130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f49741cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.329+0000 7fc7017fa640 1 -- 172.21.15.83:0/2424440130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7140049a0 con 0x55f4974185e0 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.329+0000 7fc7017fa640 1 -- 172.21.15.83:0/2424440130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc71401f030 con 0x55f4974185e0 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.329+0000 7fc7017fa640 1 -- 172.21.15.83:0/2424440130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714003b80 con 0x55f4974185e0 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.330+0000 7fc724d8d740 1 -- 172.21.15.83:0/2424440130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4974185e0 msgr2=0x55f497424290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.330+0000 7fc724d8d740 1 --2- 172.21.15.83:0/2424440130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4974185e0 0x55f497424290 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc714011c00 tx=0x7fc714007f50 comp rx=0 tx=0).stop 2024-03-11T19:53:19.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.330+0000 7fc724d8d740 1 -- 172.21.15.83:0/2424440130 shutdown_connections 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.330+0000 7fc724d8d740 1 --2- 172.21.15.83:0/2424440130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4974185e0 0x55f497424290 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.330+0000 7fc724d8d740 1 --2- 172.21.15.83:0/2424440130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f497413470 0x55f49741d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.330+0000 7fc724d8d740 1 --2- 172.21.15.83:0/2424440130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f49741cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.330+0000 7fc724d8d740 1 -- 172.21.15.83:0/2424440130 >> 172.21.15.83:0/2424440130 conn(0x55f49740dff0 msgr2=0x55f49741b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.330+0000 7fc724d8d740 1 -- 172.21.15.83:0/2424440130 shutdown_connections 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.331+0000 7fc724d8d740 1 -- 172.21.15.83:0/2424440130 wait complete. 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.331+0000 7fc724d8d740 1 -- start start 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.331+0000 7fc724d8d740 1 Processor -- start 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f4974217e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 0x55f497421d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc724d8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f497422cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc724d8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497423200 con 0x55f497413470 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc724d8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497423370 con 0x55f4974185e0 2024-03-11T19:53:19.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc724d8d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f497423530 con 0x55f497412a80 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc724d8d740 10 client.0 _ll_register_callbacks cb 0x7fc714007cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 0x55f497421d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f4974217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 0x55f497421d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40764/0 (socket says 172.21.15.83:40764) 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71affd640 1 -- 172.21.15.83:0/4279384585 learned_addr learned my addr 172.21.15.83:0/4279384585 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71bfff640 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f497422cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71b7fe640 1 -- 172.21.15.83:0/4279384585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 msgr2=0x55f497422cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71b7fe640 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f497422cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71b7fe640 1 -- 172.21.15.83:0/4279384585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 msgr2=0x55f497421d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71b7fe640 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 0x55f497421d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71b7fe640 1 -- 172.21.15.83:0/4279384585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4973716a0 con 0x55f497412a80 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71affd640 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 0x55f497421d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:19.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.332+0000 7fc71bfff640 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f497422cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:19.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.333+0000 7fc71b7fe640 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f4974217e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc70c094b80 tx=0x7fc70c0a22a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10b0f421e0144a9a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:19.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.333+0000 7fc6faffd640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:53:19.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.333+0000 7fc6faffd640 1 -- 172.21.15.83:0/4279384585 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c099e80 con 0x55f497412a80 2024-03-11T19:53:19.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.333+0000 7fc724d8d740 1 -- 172.21.15.83:0/4279384585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc714011e10 con 0x55f497412a80 2024-03-11T19:53:19.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.333+0000 7fc6faffd640 1 -- 172.21.15.83:0/4279384585 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc70c09e150 con 0x55f497412a80 2024-03-11T19:53:19.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.334+0000 7fc6faffd640 1 -- 172.21.15.83:0/4279384585 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc70c0b1050 con 0x55f497412a80 2024-03-11T19:53:19.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.337+0000 7fc724d8d740 10 client.27559 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:19.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.337+0000 7fc724d8d740 1 -- 172.21.15.83:0/4279384585 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f4973338c0 con 0x55f497412a80 2024-03-11T19:53:19.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.341+0000 7fc6faffd640 1 -- 172.21.15.83:0/4279384585 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc714011e10 con 0x55f497412a80 2024-03-11T19:53:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.347+0000 7fc6faffd640 1 -- 172.21.15.83:0/4279384585 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc70c023070 con 0x55f497412a80 2024-03-11T19:53:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.347+0000 7fc724d8d740 10 client.27559 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.347+0000 7fc724d8d740 10 client.27559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.347+0000 7fc724d8d740 20 client.27559 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.347+0000 7fc724d8d740 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f497428f30 0x55f49740c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.347+0000 7fc724d8d740 4 client.27559 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:53:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.347+0000 7fc724d8d740 1 -- 172.21.15.83:0/4279384585 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f4970fe890 con 0x55f497428f30 2024-03-11T19:53:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.350+0000 7fc71affd640 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f497428f30 0x55f49740c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:19.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.360+0000 7fc71affd640 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f497428f30 0x55f49740c8e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f07a3a5c1311cb server_cookie=42ee7c656298ef72 in_seq=0 out_seq=0 2024-03-11T19:53:19.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.360+0000 7fc6faffd640 10 client.27559 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.363+0000 7fc6faffd640 1 -- 172.21.15.83:0/4279384585 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f4970fe890 con 0x55f497428f30 2024-03-11T19:53:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.363+0000 7fc6faffd640 10 client.27559 handle_command_reply: tid=0 2024-03-11T19:53:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.363+0000 7fc724d8d740 1 client.27559 shutdown 2024-03-11T19:53:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.366+0000 7fc724d8d740 1 -- 172.21.15.83:0/4279384585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 msgr2=0x55f4974217e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.366+0000 7fc724d8d740 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f4974217e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc70c094b80 tx=0x7fc70c0a22a0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.367+0000 7fc724d8d740 1 -- 172.21.15.83:0/4279384585 shutdown_connections 2024-03-11T19:53:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.367+0000 7fc724d8d740 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55f497428f30 0x55f49740c8e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.367+0000 7fc724d8d740 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f4974185e0 0x55f497422cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.367+0000 7fc724d8d740 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f497413470 0x55f497421d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.367+0000 7fc724d8d740 1 --2- 172.21.15.83:0/4279384585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f497412a80 0x55f4974217e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.367+0000 7fc724d8d740 1 -- 172.21.15.83:0/4279384585 >> 172.21.15.83:0/4279384585 conn(0x55f49740dff0 msgr2=0x55f4974149c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.367+0000 7fc6faffd640 0 client.27559 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.367+0000 7fc724d8d740 1 -- 172.21.15.83:0/4279384585 shutdown_connections 2024-03-11T19:53:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.371+0000 7fc724d8d740 1 -- 172.21.15.83:0/4279384585 wait complete. 2024-03-11T19:53:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.371+0000 7fc724d8d740 20 client.27559 trim_cache size 0 max 16384 2024-03-11T19:53:19.379 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:53:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.375+0000 7fc7220df640 1 -- 172.21.15.83:0/3647745535 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7040a1a30 msgr2=0x7fc7040a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.375+0000 7fc7220df640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7040a1a30 0x7fc7040a3ef0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc714000f80 tx=0x7fc714012ba0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.375+0000 7fc7220df640 1 -- 172.21.15.83:0/3647745535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc71c101190 msgr2=0x7fc71c199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:19.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.375+0000 7fc7220df640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc71c101190 0x7fc71c199400 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d770 tx=0x7fc70c00dc30 comp rx=0 tx=0).stop 2024-03-11T19:53:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.378+0000 7fc7220df640 1 -- 172.21.15.83:0/3647745535 shutdown_connections 2024-03-11T19:53:19.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.378+0000 7fc7220df640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc7040a1a30 0x7fc7040a3ef0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.378+0000 7fc7220df640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc71c102d40 0x7fc71c1a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.378+0000 7fc7220df640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc71c102380 0x7fc71c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.378+0000 7fc7220df640 1 --2- 172.21.15.83:0/3647745535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc71c101190 0x7fc71c199400 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:19.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.378+0000 7fc7220df640 1 -- 172.21.15.83:0/3647745535 >> 172.21.15.83:0/3647745535 conn(0x7fc71c0fc720 msgr2=0x7fc71c0733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:19.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.379+0000 7fc7220df640 1 -- 172.21.15.83:0/3647745535 shutdown_connections 2024-03-11T19:53:19.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:19.380+0000 7fc7220df640 1 -- 172.21.15.83:0/3647745535 wait complete. 2024-03-11T19:53:20.402 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:53:20.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:20.403 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:53:20.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.569+0000 7f2128cfb640 1 -- 172.21.15.83:0/2362423044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2124101280 msgr2=0x7f2124101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.569+0000 7f2128cfb640 1 --2- 172.21.15.83:0/2362423044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2124101280 0x7f2124101700 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f210c009900 tx=0x7f210c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:20.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.570+0000 7f2128cfb640 1 -- 172.21.15.83:0/2362423044 shutdown_connections 2024-03-11T19:53:20.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.570+0000 7f2128cfb640 1 --2- 172.21.15.83:0/2362423044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124101c40 0x7f21241084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.570+0000 7f2128cfb640 1 --2- 172.21.15.83:0/2362423044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2124101280 0x7f2124101700 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.570+0000 7f2128cfb640 1 --2- 172.21.15.83:0/2362423044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2124100090 0x7f21241004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.570+0000 7f2128cfb640 1 -- 172.21.15.83:0/2362423044 >> 172.21.15.83:0/2362423044 conn(0x7f21240fb620 msgr2=0x7f21240fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.570+0000 7f2128cfb640 1 -- 172.21.15.83:0/2362423044 shutdown_connections 2024-03-11T19:53:20.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.570+0000 7f2128cfb640 1 -- 172.21.15.83:0/2362423044 wait complete. 2024-03-11T19:53:20.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.570+0000 7f2128cfb640 1 Processor -- start 2024-03-11T19:53:20.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2128cfb640 1 -- start start 2024-03-11T19:53:20.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2128cfb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2124100090 0x7f21241a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2128cfb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2124101280 0x7f21241a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2128cfb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124101c40 0x7f212406cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2128cfb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212406d360 con 0x7f2124100090 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2128cfb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212406d4d0 con 0x7f2124101c40 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2128cfb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212406d690 con 0x7f2124101280 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2122d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124101c40 0x7f212406cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f212259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2124100090 0x7f21241a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2122d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124101c40 0x7f212406cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:53208/0 (socket says 172.21.15.83:53208) 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2122d9d640 1 -- 172.21.15.83:0/3841760691 learned_addr learned my addr 172.21.15.83:0/3841760691 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f211bfff640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2124101280 0x7f21241a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2122d9d640 1 -- 172.21.15.83:0/3841760691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2124101280 msgr2=0x7f21241a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2122d9d640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2124101280 0x7f21241a2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2122d9d640 1 -- 172.21.15.83:0/3841760691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2124100090 msgr2=0x7f21241a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2122d9d640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2124100090 0x7f21241a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.571+0000 7f2122d9d640 1 -- 172.21.15.83:0/3841760691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c009560 con 0x7f2124101c40 2024-03-11T19:53:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.572+0000 7f211bfff640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2124101280 0x7f21241a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.572+0000 7f2122d9d640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124101c40 0x7f212406cbd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f211c00eb10 tx=0x7f211c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.572+0000 7f211b7fe640 1 -- 172.21.15.83:0/3841760691 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c004b20 con 0x7f2124101c40 2024-03-11T19:53:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.572+0000 7f2128cfb640 1 -- 172.21.15.83:0/3841760691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212406d970 con 0x7f2124101c40 2024-03-11T19:53:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.572+0000 7f211b7fe640 1 -- 172.21.15.83:0/3841760691 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f211c004c80 con 0x7f2124101c40 2024-03-11T19:53:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.572+0000 7f211b7fe640 1 -- 172.21.15.83:0/3841760691 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c010670 con 0x7f2124101c40 2024-03-11T19:53:20.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.572+0000 7f2128cfb640 1 -- 172.21.15.83:0/3841760691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212406de80 con 0x7f2124101c40 2024-03-11T19:53:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.574+0000 7f211b7fe640 1 -- 172.21.15.83:0/3841760691 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f211c004df0 con 0x7f2124101c40 2024-03-11T19:53:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.574+0000 7f211b7fe640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21100a1a30 0x7f21100a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.575+0000 7f212259c640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21100a1a30 0x7f21100a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.575+0000 7f211b7fe640 1 -- 172.21.15.83:0/3841760691 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f211c014070 con 0x7f2124101c40 2024-03-11T19:53:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.575+0000 7f212259c640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21100a1a30 0x7f21100a3ef0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2124108840 tx=0x7f2114005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.575+0000 7f212b9af740 1 Processor -- start 2024-03-11T19:53:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.576+0000 7f212b9af740 1 -- start start 2024-03-11T19:53:20.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.576+0000 7f212b9af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 0x55ceff5938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.576+0000 7f212b9af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff594b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.576+0000 7f212b9af740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff595050 0x55ceff59b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.576+0000 7f211bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff594b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.576+0000 7f211bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff594b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:49998/0 (socket says 172.21.15.83:49998) 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.576+0000 7f212259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 0x55ceff5938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.576+0000 7f211bfff640 1 -- 172.21.15.83:0/2352603277 learned_addr learned my addr 172.21.15.83:0/2352603277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f212b9af740 1 -- 172.21.15.83:0/2352603277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff502f90 con 0x55ceff5934c0 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f212b9af740 1 -- 172.21.15.83:0/2352603277 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff5004c0 con 0x55ceff595050 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f2122d9d640 1 --2- 172.21.15.83:0/2352603277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff595050 0x55ceff59b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f212b9af740 1 -- 172.21.15.83:0/2352603277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff4fbca0 con 0x55ceff594690 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f211bfff640 1 -- 172.21.15.83:0/2352603277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff595050 msgr2=0x55ceff59b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f211bfff640 1 --2- 172.21.15.83:0/2352603277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff595050 0x55ceff59b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f211bfff640 1 -- 172.21.15.83:0/2352603277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 msgr2=0x55ceff5938e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f211bfff640 1 --2- 172.21.15.83:0/2352603277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 0x55ceff5938e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f211bfff640 1 -- 172.21.15.83:0/2352603277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceff4be0b0 con 0x55ceff594690 2024-03-11T19:53:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f2122d9d640 1 --2- 172.21.15.83:0/2352603277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff595050 0x55ceff59b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f211bfff640 1 --2- 172.21.15.83:0/2352603277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff594b10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f210c006010 tx=0x7f210c02c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9215ff217c263a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f2118ff9640 1 -- 172.21.15.83:0/2352603277 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c0046c0 con 0x55ceff594690 2024-03-11T19:53:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.577+0000 7f2118ff9640 1 -- 172.21.15.83:0/2352603277 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f210c004820 con 0x55ceff594690 2024-03-11T19:53:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.578+0000 7f2118ff9640 1 -- 172.21.15.83:0/2352603277 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c005540 con 0x55ceff594690 2024-03-11T19:53:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.578+0000 7f212b9af740 1 -- 172.21.15.83:0/2352603277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 msgr2=0x55ceff594b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.578+0000 7f212b9af740 1 --2- 172.21.15.83:0/2352603277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff594b10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f210c006010 tx=0x7f210c02c8e0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 -- 172.21.15.83:0/2352603277 shutdown_connections 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 --2- 172.21.15.83:0/2352603277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff595050 0x55ceff59b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 --2- 172.21.15.83:0/2352603277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff594b10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 --2- 172.21.15.83:0/2352603277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 0x55ceff5938e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 -- 172.21.15.83:0/2352603277 >> 172.21.15.83:0/2352603277 conn(0x55ceff58e6a0 msgr2=0x55ceff58eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 -- 172.21.15.83:0/2352603277 shutdown_connections 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 -- 172.21.15.83:0/2352603277 wait complete. 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 -- start start 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 Processor -- start 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 0x55ceff59d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff59d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 0x55ceff59d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 0x55ceff59d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56932/0 (socket says 172.21.15.83:56932) 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212259c640 1 -- 172.21.15.83:0/1676317517 learned_addr learned my addr 172.21.15.83:0/1676317517 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.579+0000 7f212b9af740 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff595050 0x55ceff5a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f212b9af740 1 -- 172.21.15.83:0/1676317517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff4a0f40 con 0x55ceff5934c0 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f212b9af740 1 -- 172.21.15.83:0/1676317517 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff1cf4b0 con 0x55ceff595050 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f212b9af740 1 -- 172.21.15.83:0/1676317517 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff481670 con 0x55ceff594690 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f211bfff640 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff59d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f212b9af740 10 client.0 _ll_register_callbacks cb 0x55ceff4eef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f212259c640 1 -- 172.21.15.83:0/1676317517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 msgr2=0x55ceff59d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f212259c640 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff59d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f212259c640 1 -- 172.21.15.83:0/1676317517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff595050 msgr2=0x55ceff5a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f212259c640 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff595050 0x55ceff5a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f212259c640 1 -- 172.21.15.83:0/1676317517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2124108dd0 con 0x55ceff5934c0 2024-03-11T19:53:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f212259c640 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 0x55ceff59d460 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2114005bc0 tx=0x7f2114004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5bc4a45427c436 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.580+0000 7f2109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.581+0000 7f2109ffb640 1 -- 172.21.15.83:0/1676317517 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211400eb30 con 0x55ceff5934c0 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.581+0000 7f2109ffb640 1 -- 172.21.15.83:0/1676317517 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f211401e320 con 0x55ceff5934c0 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.581+0000 7f212b9af740 1 -- 172.21.15.83:0/1676317517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ceff4d5f40 con 0x55ceff5934c0 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.581+0000 7f2109ffb640 1 -- 172.21.15.83:0/1676317517 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f211400ec90 con 0x55ceff5934c0 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.581+0000 7f212b9af740 10 client.18012 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.581+0000 7f212b9af740 1 -- 172.21.15.83:0/1676317517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ceff4fa320 con 0x55ceff5934c0 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.582+0000 7f2109ffb640 1 -- 172.21.15.83:0/1676317517 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ceff4d5f40 con 0x55ceff5934c0 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.582+0000 7f2109ffb640 1 -- 172.21.15.83:0/1676317517 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2114027680 con 0x55ceff5934c0 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.582+0000 7f212b9af740 10 client.18012 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.582+0000 7f212b9af740 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.582+0000 7f212b9af740 20 client.18012 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.582+0000 7f212b9af740 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ceff5a96f0 0x55ceff5abb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.582+0000 7f212b9af740 4 client.18012 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.582+0000 7f212b9af740 1 -- 172.21.15.83:0/1676317517 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ceff4f2310 con 0x55ceff5a96f0 2024-03-11T19:53:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.582+0000 7f211bfff640 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ceff5a96f0 0x55ceff5abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.583+0000 7f211bfff640 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ceff5a96f0 0x55ceff5abb30 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c202a0aeecff156f server_cookie=37f08ff2c533b1ec in_seq=0 out_seq=0 2024-03-11T19:53:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.583+0000 7f2109ffb640 10 client.18012 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.583+0000 7f2109ffb640 1 -- 172.21.15.83:0/1676317517 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ceff4f2310 con 0x55ceff5a96f0 2024-03-11T19:53:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.583+0000 7f2109ffb640 10 client.18012 handle_command_reply: tid=0 2024-03-11T19:53:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.583+0000 7f212b9af740 1 client.18012 shutdown 2024-03-11T19:53:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f212b9af740 1 -- 172.21.15.83:0/1676317517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 msgr2=0x55ceff59d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f212b9af740 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 0x55ceff59d460 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2114005bc0 tx=0x7f2114004730 comp rx=0 tx=0).stop 2024-03-11T19:53:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f212b9af740 1 -- 172.21.15.83:0/1676317517 shutdown_connections 2024-03-11T19:53:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f212b9af740 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ceff5a96f0 0x55ceff5abb30 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f212b9af740 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff595050 0x55ceff5a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f2109ffb640 0 client.18012 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f212b9af740 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff59d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f212b9af740 1 --2- 172.21.15.83:0/1676317517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff5934c0 0x55ceff59d460 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f212b9af740 1 -- 172.21.15.83:0/1676317517 >> 172.21.15.83:0/1676317517 conn(0x55ceff58e6a0 msgr2=0x55ceff58d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f212b9af740 1 -- 172.21.15.83:0/1676317517 shutdown_connections 2024-03-11T19:53:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f212b9af740 1 -- 172.21.15.83:0/1676317517 wait complete. 2024-03-11T19:53:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.584+0000 7f212b9af740 20 client.18012 trim_cache size 0 max 16384 2024-03-11T19:53:20.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.610+0000 7f212b9af740 1 Processor -- start 2024-03-11T19:53:20.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.610+0000 7f212b9af740 1 -- start start 2024-03-11T19:53:20.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.611+0000 7f212b9af740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff5934c0 0x55ceff597fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.611+0000 7f212b9af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff598510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.611+0000 7f212b9af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff595050 0x55ceff5a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.611+0000 7f212b9af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff583350 con 0x55ceff595050 2024-03-11T19:53:20.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.611+0000 7f2122d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff595050 0x55ceff5a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.611+0000 7f2122d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff595050 0x55ceff5a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56936/0 (socket says 172.21.15.83:56936) 2024-03-11T19:53:20.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.611+0000 7f2122d9d640 1 -- 172.21.15.83:0/711154 learned_addr learned my addr 172.21.15.83:0/711154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:20.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.611+0000 7f212b9af740 1 -- 172.21.15.83:0/711154 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff592f00 con 0x55ceff5934c0 2024-03-11T19:53:20.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.611+0000 7f212b9af740 1 -- 172.21.15.83:0/711154 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff4f2310 con 0x55ceff594690 2024-03-11T19:53:20.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.613+0000 7f211bfff640 1 --2- 172.21.15.83:0/711154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff598510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.614+0000 7f212259c640 1 --2- 172.21.15.83:0/711154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff5934c0 0x55ceff597fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.614+0000 7f211bfff640 1 -- 172.21.15.83:0/711154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff5934c0 msgr2=0x55ceff597fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.614+0000 7f211bfff640 1 --2- 172.21.15.83:0/711154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff5934c0 0x55ceff597fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.614+0000 7f211bfff640 1 -- 172.21.15.83:0/711154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff595050 msgr2=0x55ceff5a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.614+0000 7f211bfff640 1 --2- 172.21.15.83:0/711154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff595050 0x55ceff5a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.614+0000 7f211bfff640 1 -- 172.21.15.83:0/711154 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceff597b40 con 0x55ceff594690 2024-03-11T19:53:20.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.620+0000 7f211bfff640 1 --2- 172.21.15.83:0/711154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff598510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f210c02c170 tx=0x7f210c040040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27882d6a1c9b0a7d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.633+0000 7f210bfff640 1 -- 172.21.15.83:0/711154 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c02c9d0 con 0x55ceff594690 2024-03-11T19:53:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.633+0000 7f210bfff640 1 -- 172.21.15.83:0/711154 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f210c0523a0 con 0x55ceff594690 2024-03-11T19:53:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.633+0000 7f210bfff640 1 -- 172.21.15.83:0/711154 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c005350 con 0x55ceff594690 2024-03-11T19:53:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.637+0000 7f212b9af740 1 -- 172.21.15.83:0/711154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 msgr2=0x55ceff598510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.637+0000 7f212b9af740 1 --2- 172.21.15.83:0/711154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff598510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f210c02c170 tx=0x7f210c040040 comp rx=0 tx=0).stop 2024-03-11T19:53:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.645+0000 7f212b9af740 1 -- 172.21.15.83:0/711154 shutdown_connections 2024-03-11T19:53:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.645+0000 7f212b9af740 1 --2- 172.21.15.83:0/711154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff595050 0x55ceff5a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.645+0000 7f212b9af740 1 --2- 172.21.15.83:0/711154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff594690 0x55ceff598510 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.645+0000 7f212b9af740 1 --2- 172.21.15.83:0/711154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff5934c0 0x55ceff597fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.645+0000 7f212b9af740 1 -- 172.21.15.83:0/711154 >> 172.21.15.83:0/711154 conn(0x55ceff58e6a0 msgr2=0x55ceff5b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.652+0000 7f212b9af740 1 -- 172.21.15.83:0/711154 shutdown_connections 2024-03-11T19:53:20.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.655+0000 7f212b9af740 1 -- 172.21.15.83:0/711154 wait complete. 2024-03-11T19:53:20.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.655+0000 7f212b9af740 1 -- start start 2024-03-11T19:53:20.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.655+0000 7f212b9af740 1 Processor -- start 2024-03-11T19:53:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.657+0000 7f212b9af740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff5934c0 0x55ceff5a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.657+0000 7f212b9af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff594690 0x55ceff5a1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.657+0000 7f212b9af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff595050 0x55ceff5a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.657+0000 7f212b9af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff5a2070 con 0x55ceff594690 2024-03-11T19:53:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.657+0000 7f212b9af740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff5ffc40 con 0x55ceff5934c0 2024-03-11T19:53:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.657+0000 7f212b9af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceff5ffdd0 con 0x55ceff595050 2024-03-11T19:53:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.657+0000 7f212b9af740 10 client.0 _ll_register_callbacks cb 0x7f211c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.657+0000 7f211bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff594690 0x55ceff5a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.657+0000 7f211bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff594690 0x55ceff5a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56946/0 (socket says 172.21.15.83:56946) 2024-03-11T19:53:20.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.657+0000 7f211bfff640 1 -- 172.21.15.83:0/2654482300 learned_addr learned my addr 172.21.15.83:0/2654482300 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.661+0000 7f211bfff640 1 -- 172.21.15.83:0/2654482300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff595050 msgr2=0x55ceff5a19a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:53:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.661+0000 7f211bfff640 1 --2- 172.21.15.83:0/2654482300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff595050 0x55ceff5a19a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.661+0000 7f211bfff640 1 -- 172.21.15.83:0/2654482300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff5934c0 msgr2=0x55ceff5a4970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:53:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.661+0000 7f211bfff640 1 --2- 172.21.15.83:0/2654482300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff5934c0 0x55ceff5a4970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.661+0000 7f211bfff640 1 -- 172.21.15.83:0/2654482300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ceff4be0b0 con 0x55ceff594690 2024-03-11T19:53:20.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.664+0000 7f211bfff640 1 --2- 172.21.15.83:0/2654482300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff594690 0x55ceff5a1460 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f210c005970 tx=0x7f210c02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30430e63c3c009b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.664+0000 7f2109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:53:20.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.668+0000 7f2109ffb640 1 -- 172.21.15.83:0/2654482300 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c004990 con 0x55ceff594690 2024-03-11T19:53:20.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.668+0000 7f212b9af740 1 -- 172.21.15.83:0/2654482300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ceff521900 con 0x55ceff594690 2024-03-11T19:53:20.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.668+0000 7f2109ffb640 1 -- 172.21.15.83:0/2654482300 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f210c04e070 con 0x55ceff594690 2024-03-11T19:53:20.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.668+0000 7f2109ffb640 1 -- 172.21.15.83:0/2654482300 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f210c004af0 con 0x55ceff594690 2024-03-11T19:53:20.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.670+0000 7f212b9af740 10 client.18018 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:20.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.670+0000 7f212b9af740 1 -- 172.21.15.83:0/2654482300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ceff496c00 con 0x55ceff594690 2024-03-11T19:53:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.670+0000 7f2109ffb640 1 -- 172.21.15.83:0/2654482300 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f210c06c020 con 0x55ceff594690 2024-03-11T19:53:20.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.670+0000 7f2109ffb640 1 -- 172.21.15.83:0/2654482300 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f210c03c040 con 0x55ceff594690 2024-03-11T19:53:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.670+0000 7f212b9af740 10 client.18018 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.671+0000 7f212b9af740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.671+0000 7f212b9af740 20 client.18018 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.671+0000 7f212b9af740 1 --2- 172.21.15.83:0/2654482300 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ceff601510 0x55ceff603930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.671+0000 7f212b9af740 4 client.18018 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:53:20.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.671+0000 7f212b9af740 1 -- 172.21.15.83:0/2654482300 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ceff1fb860 con 0x55ceff601510 2024-03-11T19:53:20.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.676+0000 7f212259c640 1 --2- 172.21.15.83:0/2654482300 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ceff601510 0x55ceff603930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.683+0000 7f212259c640 1 --2- 172.21.15.83:0/2654482300 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ceff601510 0x55ceff603930 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f115cdabd92f81 server_cookie=1d52a742457bc074 in_seq=0 out_seq=0 2024-03-11T19:53:20.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.683+0000 7f2109ffb640 10 client.18018 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:20.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.685+0000 7f2109ffb640 1 -- 172.21.15.83:0/2654482300 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ceff1fb860 con 0x55ceff601510 2024-03-11T19:53:20.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.685+0000 7f2109ffb640 10 client.18018 handle_command_reply: tid=0 2024-03-11T19:53:20.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.685+0000 7f212b9af740 1 client.18018 shutdown 2024-03-11T19:53:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f212b9af740 1 -- 172.21.15.83:0/2654482300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff594690 msgr2=0x55ceff5a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f212b9af740 1 --2- 172.21.15.83:0/2654482300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff594690 0x55ceff5a1460 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f210c005970 tx=0x7f210c02c8e0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f212b9af740 1 -- 172.21.15.83:0/2654482300 shutdown_connections 2024-03-11T19:53:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f212b9af740 1 --2- 172.21.15.83:0/2654482300 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ceff601510 0x55ceff603930 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f212b9af740 1 --2- 172.21.15.83:0/2654482300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ceff595050 0x55ceff5a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f212b9af740 1 --2- 172.21.15.83:0/2654482300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceff594690 0x55ceff5a1460 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f212b9af740 1 --2- 172.21.15.83:0/2654482300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ceff5934c0 0x55ceff5a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f212b9af740 1 -- 172.21.15.83:0/2654482300 >> 172.21.15.83:0/2654482300 conn(0x55ceff58e6a0 msgr2=0x55ceff597fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f2109ffb640 0 client.18018 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f212b9af740 1 -- 172.21.15.83:0/2654482300 shutdown_connections 2024-03-11T19:53:20.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f212b9af740 1 -- 172.21.15.83:0/2654482300 wait complete. 2024-03-11T19:53:20.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.687+0000 7f212b9af740 20 client.18018 trim_cache size 0 max 16384 2024-03-11T19:53:20.693 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:53:20.693 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:53:20.693 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "a65d039c-1352-4f46-a1a1-3d1ac7dea04d", 2024-03-11T19:53:20.693 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:53:20.693 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:53:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.688+0000 7f2128cfb640 1 -- 172.21.15.83:0/3841760691 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21100a1a30 msgr2=0x7f21100a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.688+0000 7f2128cfb640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21100a1a30 0x7f21100a3ef0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2124108840 tx=0x7f2114005f90 comp rx=0 tx=0).stop 2024-03-11T19:53:20.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.688+0000 7f2128cfb640 1 -- 172.21.15.83:0/3841760691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124101c40 msgr2=0x7f212406cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.688+0000 7f2128cfb640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124101c40 0x7f212406cbd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f211c00eb10 tx=0x7f211c00c2d0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.688+0000 7f2128cfb640 1 -- 172.21.15.83:0/3841760691 shutdown_connections 2024-03-11T19:53:20.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.688+0000 7f2128cfb640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f21100a1a30 0x7f21100a3ef0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.688+0000 7f2128cfb640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124101c40 0x7f212406cbd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.688+0000 7f2128cfb640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2124101280 0x7f21241a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.688+0000 7f2128cfb640 1 --2- 172.21.15.83:0/3841760691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2124100090 0x7f21241a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.688+0000 7f2128cfb640 1 -- 172.21.15.83:0/3841760691 >> 172.21.15.83:0/3841760691 conn(0x7f21240fb620 msgr2=0x7f21240fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.689+0000 7f2128cfb640 1 -- 172.21.15.83:0/3841760691 shutdown_connections 2024-03-11T19:53:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.689+0000 7f2128cfb640 1 -- 172.21.15.83:0/3841760691 wait complete. 2024-03-11T19:53:20.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:53:20.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:53:20.744 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:20 smithi083 ceph-mon[31864]: pgmap v772: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.4 MiB/s wr, 750 op/s 2024-03-11T19:53:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.865+0000 7ffbe23e9640 1 -- 172.21.15.83:0/4004070169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbdc104d10 msgr2=0x7ffbdc105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.865+0000 7ffbe23e9640 1 --2- 172.21.15.83:0/4004070169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbdc104d10 0x7ffbdc105190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ffbd001a910 tx=0x7ffbd003cc80 comp rx=0 tx=0).stop 2024-03-11T19:53:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.866+0000 7ffbe23e9640 1 -- 172.21.15.83:0/4004070169 shutdown_connections 2024-03-11T19:53:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.866+0000 7ffbe23e9640 1 --2- 172.21.15.83:0/4004070169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbdc104d10 0x7ffbdc105190 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.866+0000 7ffbe23e9640 1 --2- 172.21.15.83:0/4004070169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbdc102360 0x7ffbdc1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.866+0000 7ffbe23e9640 1 --2- 172.21.15.83:0/4004070169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbdc0ff9f0 0x7ffbdc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.866+0000 7ffbe23e9640 1 -- 172.21.15.83:0/4004070169 >> 172.21.15.83:0/4004070169 conn(0x7ffbdc0fb5f0 msgr2=0x7ffbdc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.866+0000 7ffbe23e9640 1 -- 172.21.15.83:0/4004070169 shutdown_connections 2024-03-11T19:53:20.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.866+0000 7ffbe23e9640 1 -- 172.21.15.83:0/4004070169 wait complete. 2024-03-11T19:53:20.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.867+0000 7ffbe23e9640 1 Processor -- start 2024-03-11T19:53:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.868+0000 7ffbe23e9640 1 -- start start 2024-03-11T19:53:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.868+0000 7ffbe23e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbdc0ff9f0 0x7ffbdc194ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.868+0000 7ffbe23e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbdc102360 0x7ffbdc195530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.869+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbdc0ff9f0 0x7ffbdc194ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.869+0000 7ffbdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbdc102360 0x7ffbdc195530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.869+0000 7ffbdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbdc102360 0x7ffbdc195530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56958/0 (socket says 172.21.15.83:56958) 2024-03-11T19:53:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.869+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbdc0ff9f0 0x7ffbdc194ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:53258/0 (socket says 172.21.15.83:53258) 2024-03-11T19:53:20.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.869+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/153549979 learned_addr learned my addr 172.21.15.83:0/153549979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.869+0000 7ffbe23e9640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbdc104d10 0x7ffbdc19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.869+0000 7ffbe23e9640 1 -- 172.21.15.83:0/153549979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc19cae0 con 0x7ffbdc102360 2024-03-11T19:53:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.869+0000 7ffbe23e9640 1 -- 172.21.15.83:0/153549979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc19cc50 con 0x7ffbdc0ff9f0 2024-03-11T19:53:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.869+0000 7ffbe23e9640 1 -- 172.21.15.83:0/153549979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc19ce10 con 0x7ffbdc104d10 2024-03-11T19:53:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.869+0000 7ffbe0986640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbdc104d10 0x7ffbdc19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbe0986640 1 -- 172.21.15.83:0/153549979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbdc0ff9f0 msgr2=0x7ffbdc194ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbe0986640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbdc0ff9f0 0x7ffbdc194ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbe0986640 1 -- 172.21.15.83:0/153549979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbdc102360 msgr2=0x7ffbdc195530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbe0986640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbdc102360 0x7ffbdc195530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbe0986640 1 -- 172.21.15.83:0/153549979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd001a570 con 0x7ffbdc104d10 2024-03-11T19:53:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbdc102360 0x7ffbdc195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbe0986640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbdc104d10 0x7ffbdc19c5a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ffbd003d290 tx=0x7ffbd003da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbd97fa640 1 -- 172.21.15.83:0/153549979 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0005d10 con 0x7ffbdc104d10 2024-03-11T19:53:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbd97fa640 1 -- 172.21.15.83:0/153549979 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbd00053e0 con 0x7ffbdc104d10 2024-03-11T19:53:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbe23e9640 1 -- 172.21.15.83:0/153549979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbdc19d090 con 0x7ffbdc104d10 2024-03-11T19:53:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbd97fa640 1 -- 172.21.15.83:0/153549979 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd004e460 con 0x7ffbdc104d10 2024-03-11T19:53:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.870+0000 7ffbe23e9640 1 -- 172.21.15.83:0/153549979 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc19d540 con 0x7ffbdc104d10 2024-03-11T19:53:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbd97fa640 1 -- 172.21.15.83:0/153549979 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ffbd0003680 con 0x7ffbdc104d10 2024-03-11T19:53:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbe4fc2740 1 Processor -- start 2024-03-11T19:53:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbe4fc2740 1 -- start start 2024-03-11T19:53:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 0x55e8d3ce26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3ce77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbe4fc2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d3c52280 con 0x55e8d3ce22d0 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbe4fc2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d3c4f7b0 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbe4fc2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d3c4af90 con 0x55e8d3ce7e30 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 0x55e8d3ce26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbd97fa640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffbc40a1a30 0x7ffbc40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 0x55e8d3ce26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56968/0 (socket says 172.21.15.83:56968) 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/1054504180 learned_addr learned my addr 172.21.15.83:0/1054504180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.874+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/1054504180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.875+0000 7ffbe0986640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffbc40a1a30 0x7ffbc40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.875+0000 7ffbdbfff640 1 --2- 172.21.15.83:0/1054504180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3ce77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.875+0000 7ffbd97fa640 1 -- 172.21.15.83:0/153549979 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ffbd0100890 con 0x7ffbdc104d10 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.875+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/1054504180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 msgr2=0x55e8d3ce77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.875+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/1054504180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3ce77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.875+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/1054504180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 msgr2=0x55e8d3ce26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.875+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/1054504180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 0x55e8d3ce26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.875+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/1054504180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8d3c09570 con 0x55e8d3ce7e30 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.875+0000 7ffbe0986640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffbc40a1a30 0x7ffbc40a3ef0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00cd840 tx=0x7ffbd009cce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.875+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/1054504180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cea2c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009fd0 tx=0x7ffbcc009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=306282f3657d29d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.875+0000 7ffbc27fc640 1 -- 172.21.15.83:0/1054504180 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc017a80 con 0x55e8d3ce7e30 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.876+0000 7ffbc27fc640 1 -- 172.21.15.83:0/1054504180 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc015630 con 0x55e8d3ce7e30 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.876+0000 7ffbc27fc640 1 -- 172.21.15.83:0/1054504180 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0145b0 con 0x55e8d3ce7e30 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.876+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/1054504180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce7e30 msgr2=0x55e8d3cea2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.876+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/1054504180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cea2c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009fd0 tx=0x7ffbcc009190 comp rx=0 tx=0).stop 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.877+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/1054504180 shutdown_connections 2024-03-11T19:53:20.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.877+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/1054504180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cea2c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.877+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/1054504180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3ce77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.877+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/1054504180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 0x55e8d3ce26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.877+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/1054504180 >> 172.21.15.83:0/1054504180 conn(0x55e8d3cdd7f0 msgr2=0x55e8d3cddbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.877+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/1054504180 shutdown_connections 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.877+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/1054504180 wait complete. 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbe4fc2740 1 -- start start 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbe4fc2740 1 Processor -- start 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cf37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbe4fc2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d391e430 con 0x55e8d3ce22d0 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbe4fc2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d38dae00 con 0x55e8d3ce7e30 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbe4fc2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d3bcffd0 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56970/0 (socket says 172.21.15.83:56970) 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/897848634 learned_addr learned my addr 172.21.15.83:0/897848634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.878+0000 7ffbdbfff640 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cf37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbe4fc2740 10 client.0 _ll_register_callbacks cb 0x55e8d3c25fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbdbfff640 1 -- 172.21.15.83:0/897848634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce7e30 msgr2=0x55e8d3cf37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbdbfff640 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cf37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbdbfff640 1 -- 172.21.15.83:0/897848634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 msgr2=0x55e8d3cec270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbdbfff640 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cec270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbdbfff640 1 -- 172.21.15.83:0/897848634 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbcc0157e0 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbdbfff640 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cec7b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55e8d3ce30c0 tx=0x7ffbd4011c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd8b03baad1b38cb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:53:20.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbbb7fe640 1 -- 172.21.15.83:0/897848634 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4004d20 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbbb7fe640 1 -- 172.21.15.83:0/897848634 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ffbd4018070 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/897848634 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8d394a7e0 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbbb7fe640 1 -- 172.21.15.83:0/897848634 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbd4004e80 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbe4fc2740 10 client.27583 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.879+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/897848634 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e8d3c49610 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.880+0000 7ffbbb7fe640 1 -- 172.21.15.83:0/897848634 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e8d394a7e0 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.880+0000 7ffbbb7fe640 1 -- 172.21.15.83:0/897848634 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffbd4025070 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.880+0000 7ffbe4fc2740 10 client.27583 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.880+0000 7ffbe4fc2740 10 client.27583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.880+0000 7ffbe4fc2740 20 client.27583 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.880+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e8d3cf8440 0x55e8d3cfa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.880+0000 7ffbe4fc2740 4 client.27583 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:20.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.880+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/897848634 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8d3987720 con 0x55e8d3cf8440 2024-03-11T19:53:20.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.880+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e8d3cf8440 0x55e8d3cfa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.881+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e8d3cf8440 0x55e8d3cfa8a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91c6c6575af67cde server_cookie=aa7ddcfc307990ea in_seq=0 out_seq=0 2024-03-11T19:53:20.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.881+0000 7ffbbb7fe640 10 client.27583 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:20.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.882+0000 7ffbbb7fe640 1 -- 172.21.15.83:0/897848634 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e8d3987720 con 0x55e8d3cf8440 2024-03-11T19:53:20.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.882+0000 7ffbbb7fe640 10 client.27583 handle_command_reply: tid=0 2024-03-11T19:53:20.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.882+0000 7ffbe4fc2740 1 client.27583 shutdown 2024-03-11T19:53:20.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:20 smithi137 ceph-mon[36220]: pgmap v772: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.4 MiB/s wr, 750 op/s 2024-03-11T19:53:20.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/897848634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce2cc0 msgr2=0x55e8d3cec7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cec7b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55e8d3ce30c0 tx=0x7ffbd4011c30 comp rx=0 tx=0).stop 2024-03-11T19:53:20.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/897848634 shutdown_connections 2024-03-11T19:53:20.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e8d3cf8440 0x55e8d3cfa8a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cf37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cec7b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/897848634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/897848634 >> 172.21.15.83:0/897848634 conn(0x55e8d3cdd7f0 msgr2=0x55e8d3cdca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbbb7fe640 0 client.27583 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:20.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/897848634 shutdown_connections 2024-03-11T19:53:20.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/897848634 wait complete. 2024-03-11T19:53:20.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.883+0000 7ffbe4fc2740 20 client.27583 trim_cache size 0 max 16384 2024-03-11T19:53:20.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:20 smithi204 ceph-mon[36739]: pgmap v772: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.4 MiB/s wr, 750 op/s 2024-03-11T19:53:20.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.905+0000 7ffbe4fc2740 1 Processor -- start 2024-03-11T19:53:20.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.905+0000 7ffbe4fc2740 1 -- start start 2024-03-11T19:53:20.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.905+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.905+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cf37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.905+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.905+0000 7ffbe4fc2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d3cde7b0 con 0x55e8d3ce7e30 2024-03-11T19:53:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.905+0000 7ffbe4fc2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d3ce30f0 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.905+0000 7ffbe4fc2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d3cf6a60 con 0x55e8d3ce22d0 2024-03-11T19:53:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cf37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cf37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:53288/0 (socket says 172.21.15.83:53288) 2024-03-11T19:53:20.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbdbfff640 1 -- 172.21.15.83:0/3383344535 learned_addr learned my addr 172.21.15.83:0/3383344535 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:20.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/3383344535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/3383344535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 msgr2=0x55e8d3cff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/3383344535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cff240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/3383344535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 msgr2=0x55e8d3cf37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/3383344535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cf37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/3383344535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8d3be5ac0 con 0x55e8d3ce7e30 2024-03-11T19:53:20.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/3383344535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cec250 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x55e8d3cfb1c0 tx=0x7ffbcc009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6241bef48bb9fd35 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbc1ffb640 1 -- 172.21.15.83:0/3383344535 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0186f0 con 0x55e8d3ce7e30 2024-03-11T19:53:20.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbc1ffb640 1 -- 172.21.15.83:0/3383344535 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc02b070 con 0x55e8d3ce7e30 2024-03-11T19:53:20.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.906+0000 7ffbc1ffb640 1 -- 172.21.15.83:0/3383344535 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc02d4c0 con 0x55e8d3ce7e30 2024-03-11T19:53:20.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.907+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/3383344535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce7e30 msgr2=0x55e8d3cec250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.907+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/3383344535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cec250 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x55e8d3cfb1c0 tx=0x7ffbcc009fd0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.908+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/3383344535 shutdown_connections 2024-03-11T19:53:20.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.908+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/3383344535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cec250 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.908+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/3383344535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cf37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.908+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/3383344535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.908+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/3383344535 >> 172.21.15.83:0/3383344535 conn(0x55e8d3cdd7f0 msgr2=0x55e8d3cd8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.908+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/3383344535 shutdown_connections 2024-03-11T19:53:20.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.908+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/3383344535 wait complete. 2024-03-11T19:53:20.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.909+0000 7ffbe4fc2740 1 -- start start 2024-03-11T19:53:20.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.909+0000 7ffbe4fc2740 1 Processor -- start 2024-03-11T19:53:20.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.909+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cf3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.909+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cf1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.909+0000 7ffbe4fc2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cf1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.909+0000 7ffbdb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cf3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.909+0000 7ffbdb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cf3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50080/0 (socket says 172.21.15.83:50080) 2024-03-11T19:53:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.909+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cf1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.909+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/2954496665 learned_addr learned my addr 172.21.15.83:0/2954496665 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/2954496665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d3cf3b00 con 0x55e8d3ce7e30 2024-03-11T19:53:20.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/2954496665 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d3cf23a0 con 0x55e8d3ce2cc0 2024-03-11T19:53:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/2954496665 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8d3cf2560 con 0x55e8d3ce22d0 2024-03-11T19:53:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cf1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbe4fc2740 10 client.0 _ll_register_callbacks cb 0x7ffbcc015440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/2954496665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 msgr2=0x55e8d3cf1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cf1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/2954496665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce7e30 msgr2=0x55e8d3cf1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cf1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbdb7fe640 1 -- 172.21.15.83:0/2954496665 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8d3c09570 con 0x55e8d3ce22d0 2024-03-11T19:53:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/2954496665 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffbcc015630 con 0x55e8d3ce22d0 2024-03-11T19:53:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbdb7fe640 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cf3310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc032040 tx=0x7ffbcc02f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a782d0705c9f29a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbbb7fe640 10 client.27589 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:53:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbbb7fe640 1 -- 172.21.15.83:0/2954496665 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc017c80 con 0x55e8d3ce22d0 2024-03-11T19:53:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbbb7fe640 1 -- 172.21.15.83:0/2954496665 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ffbcc017de0 con 0x55e8d3ce22d0 2024-03-11T19:53:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbbb7fe640 1 -- 172.21.15.83:0/2954496665 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbcc018860 con 0x55e8d3ce22d0 2024-03-11T19:53:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbe4fc2740 10 client.27589 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.910+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/2954496665 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x55e8d3c0b6a0 con 0x55e8d3ce22d0 2024-03-11T19:53:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.911+0000 7ffbbb7fe640 1 -- 172.21.15.83:0/2954496665 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffbcc036070 con 0x55e8d3ce22d0 2024-03-11T19:53:20.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.911+0000 7ffbe4fc2740 10 client.27589 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.911+0000 7ffbe4fc2740 10 client.27589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.911+0000 7ffbe4fc2740 20 client.27589 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.911+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e8d3d4f9a0 0x55e8d3d51dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.911+0000 7ffbe4fc2740 4 client.27589 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.911+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/2954496665 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e8d39d6890 con 0x55e8d3d4f9a0 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.911+0000 7ffbdbfff640 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e8d3d4f9a0 0x55e8d3d51dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.912+0000 7ffbdbfff640 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e8d3d4f9a0 0x55e8d3d51dc0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d08976eea259b3 server_cookie=50ff60ea986b5cf7 in_seq=0 out_seq=0 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.912+0000 7ffbbb7fe640 10 client.27589 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.912+0000 7ffbbb7fe640 1 -- 172.21.15.83:0/2954496665 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e8d39d6890 con 0x55e8d3d4f9a0 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.912+0000 7ffbbb7fe640 10 client.27589 handle_command_reply: tid=0 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 1 client.27589 shutdown 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/2954496665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 msgr2=0x55e8d3cf3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cf3310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc032040 tx=0x7ffbcc02f040 comp rx=0 tx=0).stop 2024-03-11T19:53:20.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/2954496665 shutdown_connections 2024-03-11T19:53:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55e8d3d4f9a0 0x55e8d3d51dc0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbbb7fe640 0 client.27589 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8d3ce7e30 0x55e8d3cf1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d3ce2cc0 0x55e8d3cf1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 1 --2- 172.21.15.83:0/2954496665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8d3ce22d0 0x55e8d3cf3310 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/2954496665 >> 172.21.15.83:0/2954496665 conn(0x55e8d3cdd7f0 msgr2=0x55e8d3cff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/2954496665 shutdown_connections 2024-03-11T19:53:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 1 -- 172.21.15.83:0/2954496665 wait complete. 2024-03-11T19:53:20.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.913+0000 7ffbe4fc2740 20 client.27589 trim_cache size 0 max 16384 2024-03-11T19:53:20.928 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:53:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:53:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:53:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "a65d039c-1352-4f46-a1a1-3d1ac7dea04d": { 2024-03-11T19:53:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:53:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "a65d039c-1352-4f46-a1a1-3d1ac7dea04d", 2024-03-11T19:53:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:53:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:53:20.929 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:53:20.929 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:53:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 -- 172.21.15.83:0/153549979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffbc40a1a30 msgr2=0x7ffbc40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffbc40a1a30 0x7ffbc40a3ef0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00cd840 tx=0x7ffbd009cce0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 -- 172.21.15.83:0/153549979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbdc104d10 msgr2=0x7ffbdc19c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbdc104d10 0x7ffbdc19c5a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ffbd003d290 tx=0x7ffbd003da60 comp rx=0 tx=0).stop 2024-03-11T19:53:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 -- 172.21.15.83:0/153549979 shutdown_connections 2024-03-11T19:53:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ffbc40a1a30 0x7ffbc40a3ef0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffbdc104d10 0x7ffbdc19c5a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbdc102360 0x7ffbdc195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 --2- 172.21.15.83:0/153549979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbdc0ff9f0 0x7ffbdc194ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 -- 172.21.15.83:0/153549979 >> 172.21.15.83:0/153549979 conn(0x7ffbdc0fb5f0 msgr2=0x7ffbdc0fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 -- 172.21.15.83:0/153549979 shutdown_connections 2024-03-11T19:53:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:20.914+0000 7ffbe23e9640 1 -- 172.21.15.83:0/153549979 wait complete. 2024-03-11T19:53:20.933 INFO:tasks.cephfs.filesystem:scrub status for tag:a65d039c-1352-4f46-a1a1-3d1ac7dea04d - {'path': '/', 'tag': 'a65d039c-1352-4f46-a1a1-3d1ac7dea04d', 'options': 'recursive,force'} 2024-03-11T19:53:21.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:21 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:53:21.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:21 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:53:21.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:21 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:21.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:21 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:21.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:21 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:53:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:21 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:53:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:21 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:21.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:21 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:21 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:53:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:21 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:53:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:21 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:22.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:21 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 845.0 s, 2008.6 tps, lat 49.816 ms stddev 61.310 2024-03-11T19:53:22.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:22 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:53:22.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:22 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:22.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:22 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:53:22.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:22 smithi137 ceph-mon[36220]: pgmap v773: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 643 op/s 2024-03-11T19:53:22.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:22 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:22.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:22 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:53:22.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:22 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:22.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:22 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:53:22.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:22 smithi204 ceph-mon[36739]: pgmap v773: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 643 op/s 2024-03-11T19:53:22.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:22 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:22 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:53:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:22 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:22 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:53:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:22 smithi083 ceph-mon[31864]: pgmap v773: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.2 MiB/s wr, 643 op/s 2024-03-11T19:53:23.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:22 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:24.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:24 smithi137 ceph-mon[36220]: pgmap v774: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 MiB/s wr, 564 op/s 2024-03-11T19:53:24.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:24 smithi204 ceph-mon[36739]: pgmap v774: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 MiB/s wr, 564 op/s 2024-03-11T19:53:25.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:24 smithi083 ceph-mon[31864]: pgmap v774: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.6 MiB/s wr, 564 op/s 2024-03-11T19:53:26.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:26 smithi137 ceph-mon[36220]: pgmap v775: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 733 op/s 2024-03-11T19:53:26.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:26 smithi204 ceph-mon[36739]: pgmap v775: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 733 op/s 2024-03-11T19:53:27.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:26 smithi083 ceph-mon[31864]: pgmap v775: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 733 op/s 2024-03-11T19:53:27.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 850.0 s, 1838.4 tps, lat 54.391 ms stddev 62.088 2024-03-11T19:53:28.870 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:28 smithi083 ceph-mon[31864]: pgmap v776: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 MiB/s wr, 578 op/s 2024-03-11T19:53:28.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:28 smithi137 ceph-mon[36220]: pgmap v776: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 MiB/s wr, 578 op/s 2024-03-11T19:53:28.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:28 smithi204 ceph-mon[36739]: pgmap v776: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 MiB/s wr, 578 op/s 2024-03-11T19:53:30.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:30 smithi137 ceph-mon[36220]: pgmap v777: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 646 op/s 2024-03-11T19:53:30.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:30 smithi204 ceph-mon[36739]: pgmap v777: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 646 op/s 2024-03-11T19:53:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:30 smithi083 ceph-mon[31864]: pgmap v777: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 MiB/s wr, 646 op/s 2024-03-11T19:53:31.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:31 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:53:31.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:31 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:53:31.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:31 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:53:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:31 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:53:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:31 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:53:31.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:31 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:53:32.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:31 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:53:32.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:31 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:53:32.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:31 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:53:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 855.0 s, 1950.2 tps, lat 50.654 ms stddev 60.385 2024-03-11T19:53:32.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:32 smithi137 ceph-mon[36220]: pgmap v778: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.5 MiB/s wr, 639 op/s 2024-03-11T19:53:32.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:32 smithi204 ceph-mon[36739]: pgmap v778: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.5 MiB/s wr, 639 op/s 2024-03-11T19:53:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:32 smithi083 ceph-mon[31864]: pgmap v778: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.5 MiB/s wr, 639 op/s 2024-03-11T19:53:34.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:34 smithi137 ceph-mon[36220]: pgmap v779: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.1 MiB/s wr, 544 op/s 2024-03-11T19:53:34.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:34 smithi204 ceph-mon[36739]: pgmap v779: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.1 MiB/s wr, 544 op/s 2024-03-11T19:53:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:34 smithi083 ceph-mon[31864]: pgmap v779: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.1 MiB/s wr, 544 op/s 2024-03-11T19:53:36.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:36 smithi137 ceph-mon[36220]: pgmap v780: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 10 MiB/s wr, 735 op/s 2024-03-11T19:53:36.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:36 smithi204 ceph-mon[36739]: pgmap v780: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 10 MiB/s wr, 735 op/s 2024-03-11T19:53:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:36 smithi083 ceph-mon[31864]: pgmap v780: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 10 MiB/s wr, 735 op/s 2024-03-11T19:53:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 860.0 s, 1982.6 tps, lat 50.769 ms stddev 59.610 2024-03-11T19:53:38.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:38 smithi137 ceph-mon[36220]: pgmap v781: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.3 MiB/s wr, 609 op/s 2024-03-11T19:53:38.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:38 smithi204 ceph-mon[36739]: pgmap v781: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.3 MiB/s wr, 609 op/s 2024-03-11T19:53:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:38 smithi083 ceph-mon[31864]: pgmap v781: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.3 MiB/s wr, 609 op/s 2024-03-11T19:53:40.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:40 smithi137 ceph-mon[36220]: pgmap v782: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.7 MiB/s wr, 671 op/s 2024-03-11T19:53:40.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:40 smithi204 ceph-mon[36739]: pgmap v782: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.7 MiB/s wr, 671 op/s 2024-03-11T19:53:41.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:40 smithi083 ceph-mon[31864]: pgmap v782: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.7 MiB/s wr, 671 op/s 2024-03-11T19:53:42.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 865.0 s, 2240.8 tps, lat 44.913 ms stddev 58.676 2024-03-11T19:53:42.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:42 smithi137 ceph-mon[36220]: pgmap v783: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.6 MiB/s wr, 688 op/s 2024-03-11T19:53:42.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:42 smithi204 ceph-mon[36739]: pgmap v783: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.6 MiB/s wr, 688 op/s 2024-03-11T19:53:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:42 smithi083 ceph-mon[31864]: pgmap v783: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.6 MiB/s wr, 688 op/s 2024-03-11T19:53:43.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:43 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:43.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:43 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:44.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:43 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:53:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:44 smithi137 ceph-mon[36220]: pgmap v784: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7.3 MiB/s wr, 599 op/s 2024-03-11T19:53:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:53:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:53:44.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:53:44.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:44 smithi204 ceph-mon[36739]: pgmap v784: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7.3 MiB/s wr, 599 op/s 2024-03-11T19:53:44.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:53:44.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:53:44.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:53:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:44 smithi083 ceph-mon[31864]: pgmap v784: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7.3 MiB/s wr, 599 op/s 2024-03-11T19:53:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:53:45.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:53:45.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:53:46.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:46 smithi204 ceph-mon[36739]: pgmap v785: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.1 MiB/s wr, 783 op/s 2024-03-11T19:53:47.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:46 smithi083 ceph-mon[31864]: pgmap v785: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.1 MiB/s wr, 783 op/s 2024-03-11T19:53:47.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:46 smithi137 ceph-mon[36220]: pgmap v785: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.1 MiB/s wr, 783 op/s 2024-03-11T19:53:47.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 870.0 s, 2338.2 tps, lat 42.334 ms stddev 50.268 2024-03-11T19:53:48.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:48 smithi204 ceph-mon[36739]: pgmap v786: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-03-11T19:53:49.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:48 smithi083 ceph-mon[31864]: pgmap v786: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-03-11T19:53:49.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:48 smithi137 ceph-mon[36220]: pgmap v786: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-03-11T19:53:50.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:50 smithi204 ceph-mon[36739]: pgmap v787: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 755 op/s 2024-03-11T19:53:50.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:53:50.934 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:53:50.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:50 smithi083 ceph-mon[31864]: pgmap v787: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 755 op/s 2024-03-11T19:53:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.099+0000 7fd6a50f4640 1 -- 172.21.15.83:0/997260308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a010d7d0 msgr2=0x7fd6a010fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.099+0000 7fd6a50f4640 1 --2- 172.21.15.83:0/997260308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a010d7d0 0x7fd6a010fbc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd698009ca0 tx=0x7fd69802c120 comp rx=0 tx=0).stop 2024-03-11T19:53:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.100+0000 7fd6a50f4640 1 -- 172.21.15.83:0/997260308 shutdown_connections 2024-03-11T19:53:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.100+0000 7fd6a50f4640 1 --2- 172.21.15.83:0/997260308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a010d7d0 0x7fd6a010fbc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.100+0000 7fd6a50f4640 1 --2- 172.21.15.83:0/997260308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a0073be0 0x7fd6a0074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.100+0000 7fd6a50f4640 1 --2- 172.21.15.83:0/997260308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6a00751c0 0x7fd6a0073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.100+0000 7fd6a50f4640 1 -- 172.21.15.83:0/997260308 >> 172.21.15.83:0/997260308 conn(0x7fd6a00fc750 msgr2=0x7fd6a00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.103+0000 7fd6a50f4640 1 -- 172.21.15.83:0/997260308 shutdown_connections 2024-03-11T19:53:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.105+0000 7fd6a50f4640 1 -- 172.21.15.83:0/997260308 wait complete. 2024-03-11T19:53:51.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.106+0000 7fd6a50f4640 1 Processor -- start 2024-03-11T19:53:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.106+0000 7fd6a50f4640 1 -- start start 2024-03-11T19:53:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.106+0000 7fd6a50f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6a0073be0 0x7fd6a006fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd6a50f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a00751c0 0x7fd6a006ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd6a50f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a010d7d0 0x7fd6a0108f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd6a50f4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a0072fb0 con 0x7fd6a0073be0 2024-03-11T19:53:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd6a50f4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a0073120 con 0x7fd6a00751c0 2024-03-11T19:53:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd6a50f4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a01094b0 con 0x7fd6a010d7d0 2024-03-11T19:53:51.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd69ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6a0073be0 0x7fd6a006fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd69ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6a0073be0 0x7fd6a006fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53342/0 (socket says 172.21.15.83:53342) 2024-03-11T19:53:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd69ed9d640 1 -- 172.21.15.83:0/1680631124 learned_addr learned my addr 172.21.15.83:0/1680631124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd69f59e640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a010d7d0 0x7fd6a0108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd697fff640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a00751c0 0x7fd6a006ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd69ed9d640 1 -- 172.21.15.83:0/1680631124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a010d7d0 msgr2=0x7fd6a0108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd69ed9d640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a010d7d0 0x7fd6a0108f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd69ed9d640 1 -- 172.21.15.83:0/1680631124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a00751c0 msgr2=0x7fd6a006ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd69ed9d640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a00751c0 0x7fd6a006ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd69ed9d640 1 -- 172.21.15.83:0/1680631124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698009900 con 0x7fd6a0073be0 2024-03-11T19:53:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.107+0000 7fd697fff640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a00751c0 0x7fd6a006ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.108+0000 7fd69ed9d640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6a0073be0 0x7fd6a006fa20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd69000e9f0 tx=0x7fd69000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.108+0000 7fd69cd99640 1 -- 172.21.15.83:0/1680631124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690009700 con 0x7fd6a0073be0 2024-03-11T19:53:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.108+0000 7fd69cd99640 1 -- 172.21.15.83:0/1680631124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd69000cd00 con 0x7fd6a0073be0 2024-03-11T19:53:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.108+0000 7fd6a50f4640 1 -- 172.21.15.83:0/1680631124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a0109790 con 0x7fd6a0073be0 2024-03-11T19:53:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.108+0000 7fd69cd99640 1 -- 172.21.15.83:0/1680631124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6900188e0 con 0x7fd6a0073be0 2024-03-11T19:53:51.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.108+0000 7fd6a50f4640 1 -- 172.21.15.83:0/1680631124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a0109cf0 con 0x7fd6a0073be0 2024-03-11T19:53:51.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.111+0000 7fd6a7da5740 1 Processor -- start 2024-03-11T19:53:51.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.114+0000 7fd69cd99640 1 -- 172.21.15.83:0/1680631124 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd690018a40 con 0x7fd6a0073be0 2024-03-11T19:53:51.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.115+0000 7fd6a7da5740 1 -- start start 2024-03-11T19:53:51.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.115+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c552d0 0x563b27c556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.115+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c55cc0 0x563b27c5a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.115+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c5ae30 0x563b27c5d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.115+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b27bc5280 con 0x563b27c5ae30 2024-03-11T19:53:51.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.115+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b27bc27b0 con 0x563b27c55cc0 2024-03-11T19:53:51.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.115+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b27bbdf90 con 0x563b27c552d0 2024-03-11T19:53:51.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.115+0000 7fd69f59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c552d0 0x563b27c556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.115+0000 7fd69f59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c552d0 0x563b27c556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50122/0 (socket says 172.21.15.83:50122) 2024-03-11T19:53:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.115+0000 7fd69f59e640 1 -- 172.21.15.83:0/3805700554 learned_addr learned my addr 172.21.15.83:0/3805700554 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.115+0000 7fd69ed9d640 1 --2- 172.21.15.83:0/3805700554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c55cc0 0x563b27c5a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.116+0000 7fd69f59e640 1 -- 172.21.15.83:0/3805700554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c55cc0 msgr2=0x563b27c5a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.116+0000 7fd69f59e640 1 --2- 172.21.15.83:0/3805700554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c55cc0 0x563b27c5a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.116+0000 7fd69f59e640 1 -- 172.21.15.83:0/3805700554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c5ae30 msgr2=0x563b27c5d2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:53:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.116+0000 7fd69f59e640 1 --2- 172.21.15.83:0/3805700554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c5ae30 0x563b27c5d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.116+0000 7fd69f59e640 1 -- 172.21.15.83:0/3805700554 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b27b7c570 con 0x563b27c552d0 2024-03-11T19:53:51.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.116+0000 7fd69f59e640 1 --2- 172.21.15.83:0/3805700554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c552d0 0x563b27c556f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd69802c730 tx=0x7fd698005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e1363291d594700 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.116+0000 7fd69cd99640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd68c0a1b80 0x7fd68c0a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.117+0000 7fd6957fa640 1 -- 172.21.15.83:0/3805700554 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698032770 con 0x563b27c552d0 2024-03-11T19:53:51.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.117+0000 7fd69ed9d640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd68c0a1b80 0x7fd68c0a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.117+0000 7fd6957fa640 1 -- 172.21.15.83:0/3805700554 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6980328d0 con 0x563b27c552d0 2024-03-11T19:53:51.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.118+0000 7fd6a7da5740 1 -- 172.21.15.83:0/3805700554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c552d0 msgr2=0x563b27c556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.118+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/3805700554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c552d0 0x563b27c556f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd69802c730 tx=0x7fd698005fa0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.118+0000 7fd69cd99640 1 -- 172.21.15.83:0/1680631124 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd690014070 con 0x7fd6a0073be0 2024-03-11T19:53:51.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.119+0000 7fd69ed9d640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd68c0a1b80 0x7fd68c0a4040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x563b27c560c0 tx=0x7fd6900d2040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.119+0000 7fd6a7da5740 1 -- 172.21.15.83:0/3805700554 shutdown_connections 2024-03-11T19:53:51.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.119+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/3805700554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c5ae30 0x563b27c5d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.119+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/3805700554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c55cc0 0x563b27c5a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.119+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/3805700554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c552d0 0x563b27c556f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.119+0000 7fd6a7da5740 1 -- 172.21.15.83:0/3805700554 >> 172.21.15.83:0/3805700554 conn(0x563b27c507f0 msgr2=0x563b27c50bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.120+0000 7fd6a7da5740 1 -- 172.21.15.83:0/3805700554 shutdown_connections 2024-03-11T19:53:51.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.120+0000 7fd6a7da5740 1 -- 172.21.15.83:0/3805700554 wait complete. 2024-03-11T19:53:51.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.120+0000 7fd6a7da5740 1 -- start start 2024-03-11T19:53:51.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.121+0000 7fd6a7da5740 1 Processor -- start 2024-03-11T19:53:51.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.122+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c5f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.122+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 0x563b27c5f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.122+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c5ae30 0x563b27c66720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.122+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b27891430 con 0x563b27c552d0 2024-03-11T19:53:51.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.122+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b2784de00 con 0x563b27c5ae30 2024-03-11T19:53:51.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.122+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b27b42fd0 con 0x563b27c55cc0 2024-03-11T19:53:51.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.122+0000 7fd6a7da5740 10 client.0 _ll_register_callbacks cb 0x563b27b98fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:51.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.122+0000 7fd697fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 0x563b27c5f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.122+0000 7fd697fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 0x563b27c5f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50138/0 (socket says 172.21.15.83:50138) 2024-03-11T19:53:51.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.122+0000 7fd697fff640 1 -- 172.21.15.83:0/1419218088 learned_addr learned my addr 172.21.15.83:0/1419218088 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:51.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.123+0000 7fd69f59e640 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c5f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.123+0000 7fd697fff640 1 -- 172.21.15.83:0/1419218088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c5ae30 msgr2=0x563b27c66720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:53:51.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.123+0000 7fd697fff640 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c5ae30 0x563b27c66720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.123+0000 7fd697fff640 1 -- 172.21.15.83:0/1419218088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 msgr2=0x563b27c5f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.123+0000 7fd697fff640 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c5f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.123+0000 7fd697fff640 1 -- 172.21.15.83:0/1419218088 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a0110510 con 0x563b27c55cc0 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.123+0000 7fd697fff640 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 0x563b27c5f6e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd68800e680 tx=0x7fd68800eb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2119d559c9a7efd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.123+0000 7fd6867fc640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.123+0000 7fd6867fc640 1 -- 172.21.15.83:0/1419218088 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688005220 con 0x563b27c55cc0 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.123+0000 7fd6867fc640 1 -- 172.21.15.83:0/1419218088 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd688005380 con 0x563b27c55cc0 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1419218088 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b278bd7e0 con 0x563b27c55cc0 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6867fc640 1 -- 172.21.15.83:0/1419218088 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd688005540 con 0x563b27c55cc0 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6a7da5740 10 client.27601 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1419218088 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563b27bbc610 con 0x563b27c55cc0 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6867fc640 1 -- 172.21.15.83:0/1419218088 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563b278bd7e0 con 0x563b27c55cc0 2024-03-11T19:53:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6867fc640 1 -- 172.21.15.83:0/1419218088 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd688004be0 con 0x563b27c55cc0 2024-03-11T19:53:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6a7da5740 10 client.27601 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6a7da5740 10 client.27601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6a7da5740 20 client.27601 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b27c6b530 0x563b27c6d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6a7da5740 4 client.27601 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.124+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1419218088 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b278fa720 con 0x563b27c6b530 2024-03-11T19:53:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.125+0000 7fd69f59e640 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c5f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:53:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.125+0000 7fd69f59e640 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b27c6b530 0x563b27c6d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.126+0000 7fd6867fc640 10 client.27601 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.126+0000 7fd69f59e640 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b27c6b530 0x563b27c6d9f0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14cb5c8c1324bb73 server_cookie=3a63e8f80118fee1 in_seq=0 out_seq=0 2024-03-11T19:53:51.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6867fc640 1 -- 172.21.15.83:0/1419218088 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b278fa720 con 0x563b27c6b530 2024-03-11T19:53:51.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6867fc640 10 client.27601 handle_command_reply: tid=0 2024-03-11T19:53:51.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6a7da5740 1 client.27601 shutdown 2024-03-11T19:53:51.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1419218088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 msgr2=0x563b27c5f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 0x563b27c5f6e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd68800e680 tx=0x7fd68800eb40 comp rx=0 tx=0).stop 2024-03-11T19:53:51.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1419218088 shutdown_connections 2024-03-11T19:53:51.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b27c6b530 0x563b27c6d9f0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c5ae30 0x563b27c66720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 0x563b27c5f6e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1419218088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c5f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6867fc640 0 client.27601 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:51.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.130+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1419218088 >> 172.21.15.83:0/1419218088 conn(0x563b27c507f0 msgr2=0x563b27c4fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.131+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1419218088 shutdown_connections 2024-03-11T19:53:51.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.131+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1419218088 wait complete. 2024-03-11T19:53:51.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.131+0000 7fd6a7da5740 20 client.27601 trim_cache size 0 max 16384 2024-03-11T19:53:51.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:50 smithi137 ceph-mon[36220]: pgmap v787: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 755 op/s 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.152+0000 7fd6a7da5740 1 Processor -- start 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.152+0000 7fd6a7da5740 1 -- start start 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.152+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c4f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.152+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 0x563b27c666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.152+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c5ae30 0x563b27c4f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.152+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b27c517b0 con 0x563b27c552d0 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.152+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b27c699d0 con 0x563b27c5ae30 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.152+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b27c5e2c0 con 0x563b27c55cc0 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd69f59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c4f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd697fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 0x563b27c666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd697fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 0x563b27c666e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50148/0 (socket says 172.21.15.83:50148) 2024-03-11T19:53:51.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd697fff640 1 -- 172.21.15.83:0/770993374 learned_addr learned my addr 172.21.15.83:0/770993374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:51.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd69f59e640 1 --2- 172.21.15.83:0/770993374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c5ae30 0x563b27c4f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd69f59e640 1 -- 172.21.15.83:0/770993374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 msgr2=0x563b27c666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd69f59e640 1 --2- 172.21.15.83:0/770993374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 0x563b27c666e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd69f59e640 1 -- 172.21.15.83:0/770993374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c5ae30 msgr2=0x563b27c4f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd69f59e640 1 --2- 172.21.15.83:0/770993374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c5ae30 0x563b27c4f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd69f59e640 1 -- 172.21.15.83:0/770993374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b27b58ac0 con 0x563b27c552d0 2024-03-11T19:53:51.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd69f59e640 1 --2- 172.21.15.83:0/770993374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c4f1c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd698005e30 tx=0x7fd698041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b4a830304ada57 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd694ff9640 1 -- 172.21.15.83:0/770993374 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698052be0 con 0x563b27c552d0 2024-03-11T19:53:51.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd694ff9640 1 -- 172.21.15.83:0/770993374 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd698058070 con 0x563b27c552d0 2024-03-11T19:53:51.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.153+0000 7fd694ff9640 1 -- 172.21.15.83:0/770993374 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698034440 con 0x563b27c552d0 2024-03-11T19:53:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.154+0000 7fd6a7da5740 1 -- 172.21.15.83:0/770993374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 msgr2=0x563b27c4f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.154+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/770993374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c4f1c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd698005e30 tx=0x7fd698041040 comp rx=0 tx=0).stop 2024-03-11T19:53:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.154+0000 7fd6a7da5740 1 -- 172.21.15.83:0/770993374 shutdown_connections 2024-03-11T19:53:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.154+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/770993374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c5ae30 0x563b27c4f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.154+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/770993374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c55cc0 0x563b27c666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.154+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/770993374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c4f1c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.154+0000 7fd6a7da5740 1 -- 172.21.15.83:0/770993374 >> 172.21.15.83:0/770993374 conn(0x563b27c507f0 msgr2=0x563b27c56e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.154+0000 7fd6a7da5740 1 -- 172.21.15.83:0/770993374 shutdown_connections 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.154+0000 7fd6a7da5740 1 -- 172.21.15.83:0/770993374 wait complete. 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd6a7da5740 1 -- start start 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd6a7da5740 1 Processor -- start 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c66420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c55cc0 0x563b27c64a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd6a7da5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c5ae30 0x563b27c64fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b27c65500 con 0x563b27c552d0 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b27c65670 con 0x563b27c55cc0 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd6a7da5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b27c65830 con 0x563b27c5ae30 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd69f59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c66420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd6a7da5740 10 client.0 _ll_register_callbacks cb 0x7fd690005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd69f59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c66420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53372/0 (socket says 172.21.15.83:53372) 2024-03-11T19:53:51.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd69f59e640 1 -- 172.21.15.83:0/1572954364 learned_addr learned my addr 172.21.15.83:0/1572954364 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd69f59e640 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c5ae30 0x563b27c64fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd697fff640 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c55cc0 0x563b27c64a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd69f59e640 1 -- 172.21.15.83:0/1572954364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c5ae30 msgr2=0x563b27c64fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd69f59e640 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c5ae30 0x563b27c64fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd69f59e640 1 -- 172.21.15.83:0/1572954364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c55cc0 msgr2=0x563b27c64a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd69f59e640 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c55cc0 0x563b27c64a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.155+0000 7fd69f59e640 1 -- 172.21.15.83:0/1572954364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b27b7c570 con 0x563b27c552d0 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.156+0000 7fd69f59e640 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c66420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd6980326d0 tx=0x7fd69802c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a95e6d564a1fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.156+0000 7fd6867fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.156+0000 7fd6867fc640 1 -- 172.21.15.83:0/1572954364 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69804d070 con 0x563b27c552d0 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.156+0000 7fd6867fc640 1 -- 172.21.15.83:0/1572954364 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd69804f040 con 0x563b27c552d0 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.156+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1572954364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6980328d0 con 0x563b27c552d0 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.156+0000 7fd6867fc640 1 -- 172.21.15.83:0/1572954364 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd698052de0 con 0x563b27c552d0 2024-03-11T19:53:51.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.156+0000 7fd6a7da5740 10 client.18045 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.156+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1572954364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563b27b7e6a0 con 0x563b27c552d0 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.156+0000 7fd6867fc640 1 -- 172.21.15.83:0/1572954364 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd698065070 con 0x563b27c552d0 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.156+0000 7fd6867fc640 1 -- 172.21.15.83:0/1572954364 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd698068070 con 0x563b27c552d0 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.157+0000 7fd6a7da5740 10 client.18045 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.157+0000 7fd6a7da5740 10 client.18045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.157+0000 7fd6a7da5740 20 client.18045 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.157+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b27cc2cb0 0x563b27cc50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.157+0000 7fd6a7da5740 4 client.18045 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.157+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1572954364 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fd6980328d0 con 0x563b27cc2cb0 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.157+0000 7fd697fff640 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b27cc2cb0 0x563b27cc50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.159+0000 7fd697fff640 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b27cc2cb0 0x563b27cc50d0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6361df1dcfa460c server_cookie=f5557f74aaf3849c in_seq=0 out_seq=0 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.159+0000 7fd6867fc640 10 client.18045 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.159+0000 7fd6867fc640 1 -- 172.21.15.83:0/1572954364 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fd6980328d0 con 0x563b27cc2cb0 2024-03-11T19:53:51.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.159+0000 7fd6867fc640 10 client.18045 handle_command_reply: tid=0 2024-03-11T19:53:51.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.159+0000 7fd6a7da5740 1 client.18045 shutdown 2024-03-11T19:53:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.159+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1572954364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 msgr2=0x563b27c66420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.159+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c66420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd6980326d0 tx=0x7fd69802c120 comp rx=0 tx=0).stop 2024-03-11T19:53:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.160+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1572954364 shutdown_connections 2024-03-11T19:53:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.160+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b27cc2cb0 0x563b27cc50d0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.160+0000 7fd6867fc640 0 client.18045 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.160+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b27c5ae30 0x563b27c64fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.160+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b27c55cc0 0x563b27c64a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.160+0000 7fd6a7da5740 1 --2- 172.21.15.83:0/1572954364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b27c552d0 0x563b27c66420 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.160+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1572954364 >> 172.21.15.83:0/1572954364 conn(0x563b27c507f0 msgr2=0x563b27c53aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.160+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1572954364 shutdown_connections 2024-03-11T19:53:51.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.160+0000 7fd6a7da5740 1 -- 172.21.15.83:0/1572954364 wait complete. 2024-03-11T19:53:51.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.160+0000 7fd6a7da5740 20 client.18045 trim_cache size 0 max 16384 2024-03-11T19:53:51.169 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:53:51.169 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:53:51.169 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:53:51.169 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:53:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 -- 172.21.15.83:0/1680631124 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd68c0a1b80 msgr2=0x7fd68c0a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd68c0a1b80 0x7fd68c0a4040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x563b27c560c0 tx=0x7fd6900d2040 comp rx=0 tx=0).stop 2024-03-11T19:53:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 -- 172.21.15.83:0/1680631124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6a0073be0 msgr2=0x7fd6a006fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6a0073be0 0x7fd6a006fa20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd69000e9f0 tx=0x7fd69000eeb0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 -- 172.21.15.83:0/1680631124 shutdown_connections 2024-03-11T19:53:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd68c0a1b80 0x7fd68c0a4040 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6a010d7d0 0x7fd6a0108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6a00751c0 0x7fd6a006ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 --2- 172.21.15.83:0/1680631124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6a0073be0 0x7fd6a006fa20 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 -- 172.21.15.83:0/1680631124 >> 172.21.15.83:0/1680631124 conn(0x7fd6a00fc750 msgr2=0x7fd6a0100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 -- 172.21.15.83:0/1680631124 shutdown_connections 2024-03-11T19:53:51.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.161+0000 7fd6a50f4640 1 -- 172.21.15.83:0/1680631124 wait complete. 2024-03-11T19:53:51.179 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:53:51.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:53:51.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.357+0000 7f94aaffd640 1 -- 172.21.15.83:0/2174065667 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a40337e0 con 0x7f94ac101c50 2024-03-11T19:53:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.358+0000 7f94b2855640 1 -- 172.21.15.83:0/2174065667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94ac101c50 msgr2=0x7f94ac1084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.358+0000 7f94b2855640 1 --2- 172.21.15.83:0/2174065667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94ac101c50 0x7f94ac1084e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009be0 tx=0x7f94a402bf10 comp rx=0 tx=0).stop 2024-03-11T19:53:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.358+0000 7f94b2855640 1 -- 172.21.15.83:0/2174065667 shutdown_connections 2024-03-11T19:53:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.358+0000 7f94b2855640 1 --2- 172.21.15.83:0/2174065667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94ac101c50 0x7f94ac1084e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.358+0000 7f94b2855640 1 --2- 172.21.15.83:0/2174065667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac101290 0x7f94ac101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.358+0000 7f94b2855640 1 --2- 172.21.15.83:0/2174065667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94ac1000a0 0x7f94ac1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.358+0000 7f94b2855640 1 -- 172.21.15.83:0/2174065667 >> 172.21.15.83:0/2174065667 conn(0x7f94ac0fb610 msgr2=0x7f94ac0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.358+0000 7f94b2855640 1 -- 172.21.15.83:0/2174065667 shutdown_connections 2024-03-11T19:53:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.358+0000 7f94b2855640 1 -- 172.21.15.83:0/2174065667 wait complete. 2024-03-11T19:53:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.358+0000 7f94b2855640 1 Processor -- start 2024-03-11T19:53:51.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94b2855640 1 -- start start 2024-03-11T19:53:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94b2855640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94ac1000a0 0x7f94ac197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94b2855640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac101290 0x7f94ac197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94b2855640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94ac101c50 0x7f94ac19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94b2855640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac19ed40 con 0x7f94ac1000a0 2024-03-11T19:53:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94b2855640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac19eeb0 con 0x7f94ac101c50 2024-03-11T19:53:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94b2855640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac19f070 con 0x7f94ac101290 2024-03-11T19:53:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94abfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94ac1000a0 0x7f94ac197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94abfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94ac1000a0 0x7f94ac197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53384/0 (socket says 172.21.15.83:53384) 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94abfff640 1 -- 172.21.15.83:0/2594633361 learned_addr learned my addr 172.21.15.83:0/2594633361 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94b0df2640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94ac101c50 0x7f94ac19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94ab7fe640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac101290 0x7f94ac197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94abfff640 1 -- 172.21.15.83:0/2594633361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac101290 msgr2=0x7f94ac197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94abfff640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac101290 0x7f94ac197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94abfff640 1 -- 172.21.15.83:0/2594633361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94ac101c50 msgr2=0x7f94ac19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94abfff640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94ac101c50 0x7f94ac19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94abfff640 1 -- 172.21.15.83:0/2594633361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a4009840 con 0x7f94ac1000a0 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.359+0000 7f94ab7fe640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac101290 0x7f94ac197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.360+0000 7f94abfff640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94ac1000a0 0x7f94ac197250 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e9f0 tx=0x7f949c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.360+0000 7f94a97fa640 1 -- 172.21.15.83:0/2594633361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c00ccc0 con 0x7f94ac1000a0 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.360+0000 7f94b2855640 1 -- 172.21.15.83:0/2594633361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac19f350 con 0x7f94ac1000a0 2024-03-11T19:53:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.360+0000 7f94a97fa640 1 -- 172.21.15.83:0/2594633361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f949c00ce20 con 0x7f94ac1000a0 2024-03-11T19:53:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.360+0000 7f94a97fa640 1 -- 172.21.15.83:0/2594633361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c010430 con 0x7f94ac1000a0 2024-03-11T19:53:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.360+0000 7f94b2855640 1 -- 172.21.15.83:0/2594633361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac19f8e0 con 0x7f94ac1000a0 2024-03-11T19:53:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.361+0000 7f94b2855640 1 -- 172.21.15.83:0/2594633361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac107360 con 0x7f94ac1000a0 2024-03-11T19:53:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.363+0000 7f94a97fa640 1 -- 172.21.15.83:0/2594633361 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f949c003fb0 con 0x7f94ac1000a0 2024-03-11T19:53:51.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.363+0000 7f94a97fa640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94940a1a30 0x7f94940a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.364+0000 7f94a97fa640 1 -- 172.21.15.83:0/2594633361 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f949c014070 con 0x7f94ac1000a0 2024-03-11T19:53:51.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.364+0000 7f94ab7fe640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94940a1a30 0x7f94940a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.365+0000 7f94ab7fe640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94940a1a30 0x7f94940a3ef0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f94ac198760 tx=0x7f94a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.367+0000 7f94a97fa640 1 -- 172.21.15.83:0/2594633361 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f949c096ab0 con 0x7f94ac1000a0 2024-03-11T19:53:51.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.715+0000 7f94b2855640 1 -- 172.21.15.83:0/2594633361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94ac10cc60 con 0x7f94ac1000a0 2024-03-11T19:53:51.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.717+0000 7f94a97fa640 1 -- 172.21.15.83:0/2594633361 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f949c09a170 con 0x7f94ac1000a0 2024-03-11T19:53:51.722 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:53:51.722 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:53:51.722 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.720+0000 7f94b2855640 1 -- 172.21.15.83:0/2594633361 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94940a1a30 msgr2=0x7f94940a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.720+0000 7f94b2855640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94940a1a30 0x7f94940a3ef0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f94ac198760 tx=0x7f94a0008040 comp rx=0 tx=0).stop 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.720+0000 7f94b2855640 1 -- 172.21.15.83:0/2594633361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94ac1000a0 msgr2=0x7f94ac197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.720+0000 7f94b2855640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94ac1000a0 0x7f94ac197250 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e9f0 tx=0x7f949c00eeb0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.720+0000 7f94b2855640 1 -- 172.21.15.83:0/2594633361 shutdown_connections 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.720+0000 7f94b2855640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f94940a1a30 0x7f94940a3ef0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.720+0000 7f94b2855640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94ac101c50 0x7f94ac19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.720+0000 7f94b2855640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac101290 0x7f94ac197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.720+0000 7f94b2855640 1 --2- 172.21.15.83:0/2594633361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94ac1000a0 0x7f94ac197250 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.720+0000 7f94b2855640 1 -- 172.21.15.83:0/2594633361 >> 172.21.15.83:0/2594633361 conn(0x7f94ac0fb610 msgr2=0x7f94ac104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.720+0000 7f94b2855640 1 -- 172.21.15.83:0/2594633361 shutdown_connections 2024-03-11T19:53:51.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.721+0000 7f94b2855640 1 -- 172.21.15.83:0/2594633361 wait complete. 2024-03-11T19:53:51.741 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:53:51.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:53:51.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.871+0000 7faafe3e2640 1 -- 172.21.15.83:0/783402733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80751c0 msgr2=0x7faaf8073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.871+0000 7faafe3e2640 1 --2- 172.21.15.83:0/783402733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80751c0 0x7faaf8073610 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7faae8009930 tx=0x7faae802bc20 comp rx=0 tx=0).stop 2024-03-11T19:53:51.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.871+0000 7faafe3e2640 1 -- 172.21.15.83:0/783402733 shutdown_connections 2024-03-11T19:53:51.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.871+0000 7faafe3e2640 1 --2- 172.21.15.83:0/783402733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaf810d760 0x7faaf810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.871+0000 7faafe3e2640 1 --2- 172.21.15.83:0/783402733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf8073be0 0x7faaf8074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.871+0000 7faafe3e2640 1 --2- 172.21.15.83:0/783402733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80751c0 0x7faaf8073610 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.871+0000 7faafe3e2640 1 -- 172.21.15.83:0/783402733 >> 172.21.15.83:0/783402733 conn(0x7faaf80fc6e0 msgr2=0x7faaf80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.872+0000 7faafe3e2640 1 -- 172.21.15.83:0/783402733 shutdown_connections 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.872+0000 7faafe3e2640 1 -- 172.21.15.83:0/783402733 wait complete. 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.872+0000 7faafe3e2640 1 Processor -- start 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.872+0000 7faafe3e2640 1 -- start start 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faafe3e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf8073be0 0x7faaf819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faafe3e2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaf80751c0 0x7faaf819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faafe3e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf810d760 0x7faaf81a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faafe3e2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81a5410 con 0x7faaf8073be0 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faafe3e2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81a5580 con 0x7faaf810d760 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faafe3e2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf81a5740 con 0x7faaf80751c0 2024-03-11T19:53:51.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faaf7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf8073be0 0x7faaf819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faaf77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaf80751c0 0x7faaf819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faaf7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf8073be0 0x7faaf819d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53404/0 (socket says 172.21.15.83:53404) 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faaf7fff640 1 -- 172.21.15.83:0/663112839 learned_addr learned my addr 172.21.15.83:0/663112839 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faafc97f640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf810d760 0x7faaf81a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faaf7fff640 1 -- 172.21.15.83:0/663112839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaf80751c0 msgr2=0x7faaf819deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faaf7fff640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaf80751c0 0x7faaf819deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faaf7fff640 1 -- 172.21.15.83:0/663112839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf810d760 msgr2=0x7faaf81a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faaf7fff640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf810d760 0x7faaf81a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faaf7fff640 1 -- 172.21.15.83:0/663112839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae8009560 con 0x7faaf8073be0 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faaf77fe640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaf80751c0 0x7faaf819deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.873+0000 7faaf7fff640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf8073be0 0x7faaf819d970 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faae8000c00 tx=0x7faae80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.874+0000 7faaf57fa640 1 -- 172.21.15.83:0/663112839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae802dea0 con 0x7faaf8073be0 2024-03-11T19:53:51.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.874+0000 7faaf57fa640 1 -- 172.21.15.83:0/663112839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faae80044c0 con 0x7faaf8073be0 2024-03-11T19:53:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.874+0000 7faafe3e2640 1 -- 172.21.15.83:0/663112839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf81a59c0 con 0x7faaf8073be0 2024-03-11T19:53:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.874+0000 7faaf57fa640 1 -- 172.21.15.83:0/663112839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8003d00 con 0x7faaf8073be0 2024-03-11T19:53:51.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.874+0000 7faafe3e2640 1 -- 172.21.15.83:0/663112839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf81a5f50 con 0x7faaf8073be0 2024-03-11T19:53:51.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.876+0000 7fab00fbb740 1 Processor -- start 2024-03-11T19:53:51.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.876+0000 7fab00fbb740 1 -- start start 2024-03-11T19:53:51.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.876+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334270 0x561966334690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.876+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334c60 0x561966339760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.876+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 0x56196633c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.876+0000 7fab00fbb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619662f0960 con 0x561966334c60 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.876+0000 7fab00fbb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619662a4310 con 0x561966339dd0 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.876+0000 7fab00fbb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619662a1840 con 0x561966334270 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faaf77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 0x56196633c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faafc97f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334270 0x561966334690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faafc97f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334270 0x561966334690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50198/0 (socket says 172.21.15.83:50198) 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faaf77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 0x56196633c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:34796/0 (socket says 172.21.15.83:34796) 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faafc97f640 1 -- 172.21.15.83:0/1658464514 learned_addr learned my addr 172.21.15.83:0/1658464514 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faafc97f640 1 -- 172.21.15.83:0/1658464514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 msgr2=0x56196633c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faaf57fa640 1 -- 172.21.15.83:0/663112839 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7faae8004df0 con 0x7faaf8073be0 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faafc97f640 1 --2- 172.21.15.83:0/1658464514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 0x56196633c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faafc97f640 1 -- 172.21.15.83:0/1658464514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334c60 msgr2=0x561966339760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faafc97f640 1 --2- 172.21.15.83:0/1658464514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334c60 0x561966339760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faafc97f640 1 -- 172.21.15.83:0/1658464514 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56196625f060 con 0x561966334270 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faafc97f640 1 --2- 172.21.15.83:0/1658464514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334270 0x561966334690 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faaf000a430 tx=0x7faaf0009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51d93146cd00ee14 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faade7fc640 1 -- 172.21.15.83:0/1658464514 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0010040 con 0x561966334270 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faade7fc640 1 -- 172.21.15.83:0/1658464514 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faaf0013c70 con 0x561966334270 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faade7fc640 1 -- 172.21.15.83:0/1658464514 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0014db0 con 0x561966334270 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.877+0000 7faaf57fa640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faae00a1a30 0x7faae00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.878+0000 7faaf7fff640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faae00a1a30 0x7faae00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.878+0000 7faaf57fa640 1 -- 172.21.15.83:0/663112839 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7faae80efa00 con 0x7faaf8073be0 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.878+0000 7fab00fbb740 1 -- 172.21.15.83:0/1658464514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334270 msgr2=0x561966334690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.878+0000 7faaf7fff640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faae00a1a30 0x7faae00a3ef0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x561966335060 tx=0x7faae80b2140 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.878+0000 7fab00fbb740 1 --2- 172.21.15.83:0/1658464514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334270 0x561966334690 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faaf000a430 tx=0x7faaf0009650 comp rx=0 tx=0).stop 2024-03-11T19:53:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.879+0000 7fab00fbb740 1 -- 172.21.15.83:0/1658464514 shutdown_connections 2024-03-11T19:53:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.879+0000 7fab00fbb740 1 --2- 172.21.15.83:0/1658464514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 0x56196633c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.879+0000 7fab00fbb740 1 --2- 172.21.15.83:0/1658464514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334c60 0x561966339760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.879+0000 7fab00fbb740 1 --2- 172.21.15.83:0/1658464514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334270 0x561966334690 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.879+0000 7fab00fbb740 1 -- 172.21.15.83:0/1658464514 >> 172.21.15.83:0/1658464514 conn(0x56196632f7e0 msgr2=0x56196632fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.879+0000 7fab00fbb740 1 -- 172.21.15.83:0/1658464514 shutdown_connections 2024-03-11T19:53:51.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.879+0000 7fab00fbb740 1 -- 172.21.15.83:0/1658464514 wait complete. 2024-03-11T19:53:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.879+0000 7fab00fbb740 1 -- start start 2024-03-11T19:53:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.880+0000 7fab00fbb740 1 Processor -- start 2024-03-11T19:53:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.880+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334270 0x56196633e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.880+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334c60 0x56196633e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.880+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 0x5619663459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.880+0000 7fab00fbb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196629d020 con 0x561966334270 2024-03-11T19:53:51.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.880+0000 7fab00fbb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561965f70430 con 0x561966339dd0 2024-03-11T19:53:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.880+0000 7fab00fbb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561965f2ce00 con 0x561966334c60 2024-03-11T19:53:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.880+0000 7fab00fbb740 10 client.0 _ll_register_callbacks cb 0x561966244cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.882+0000 7faafc97f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 0x5619663459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.882+0000 7faafc97f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 0x5619663459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:34812/0 (socket says 172.21.15.83:34812) 2024-03-11T19:53:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.882+0000 7faafc97f640 1 -- 172.21.15.83:0/3638683146 learned_addr learned my addr 172.21.15.83:0/3638683146 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.883+0000 7faafc97f640 1 -- 172.21.15.83:0/3638683146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334c60 msgr2=0x56196633e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.883+0000 7faafc97f640 1 --2- 172.21.15.83:0/3638683146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334c60 0x56196633e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.883+0000 7faafc97f640 1 -- 172.21.15.83:0/3638683146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334270 msgr2=0x56196633e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.883+0000 7faafc97f640 1 --2- 172.21.15.83:0/3638683146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334270 0x56196633e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.883+0000 7faafc97f640 1 -- 172.21.15.83:0/3638683146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf0009c60 con 0x561966339dd0 2024-03-11T19:53:51.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.883+0000 7faafc97f640 1 --2- 172.21.15.83:0/3638683146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 0x5619663459b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faaf00146a0 tx=0x7faaf0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b488fe1849b5fea server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.884+0000 7faad77fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:53:51.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.889+0000 7faad77fe640 1 -- 172.21.15.83:0/3638683146 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0013590 con 0x561966339dd0 2024-03-11T19:53:51.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.889+0000 7faad77fe640 1 -- 172.21.15.83:0/3638683146 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7faaf00136f0 con 0x561966339dd0 2024-03-11T19:53:51.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.889+0000 7fab00fbb740 1 -- 172.21.15.83:0/3638683146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561965f9c7e0 con 0x561966339dd0 2024-03-11T19:53:51.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.891+0000 7faad77fe640 1 -- 172.21.15.83:0/3638683146 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaf00138b0 con 0x561966339dd0 2024-03-11T19:53:51.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.891+0000 7fab00fbb740 10 client.27512 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:51.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.891+0000 7fab00fbb740 1 -- 172.21.15.83:0/3638683146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56196629b6a0 con 0x561966339dd0 2024-03-11T19:53:51.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.896+0000 7faad77fe640 1 -- 172.21.15.83:0/3638683146 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561965f9c7e0 con 0x561966339dd0 2024-03-11T19:53:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.899+0000 7faad77fe640 1 -- 172.21.15.83:0/3638683146 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faaf0008030 con 0x561966339dd0 2024-03-11T19:53:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.899+0000 7fab00fbb740 10 client.27512 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.899+0000 7fab00fbb740 10 client.27512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:53:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.899+0000 7fab00fbb740 20 client.27512 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.899+0000 7fab00fbb740 1 --2- 172.21.15.83:0/3638683146 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x56196634a4a0 0x56196634c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.899+0000 7fab00fbb740 4 client.27512 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:51.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.899+0000 7fab00fbb740 1 -- 172.21.15.83:0/3638683146 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561965fd9720 con 0x56196634a4a0 2024-03-11T19:53:51.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.905+0000 7faaf77fe640 1 --2- 172.21.15.83:0/3638683146 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x56196634a4a0 0x56196634c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.933+0000 7faaf77fe640 1 --2- 172.21.15.83:0/3638683146 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x56196634a4a0 0x56196634c8c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a95f10619adf5b9b server_cookie=3f7b01384b2f083f in_seq=0 out_seq=0 2024-03-11T19:53:51.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.934+0000 7faad77fe640 10 client.27512 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:53:51.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.937+0000 7faad77fe640 1 -- 172.21.15.83:0/3638683146 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561965fd9720 con 0x56196634a4a0 2024-03-11T19:53:51.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.937+0000 7faad77fe640 10 client.27512 handle_command_reply: tid=0 2024-03-11T19:53:51.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.937+0000 7fab00fbb740 1 client.27512 shutdown 2024-03-11T19:53:51.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.938+0000 7fab00fbb740 1 -- 172.21.15.83:0/3638683146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 msgr2=0x5619663459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.938+0000 7fab00fbb740 1 --2- 172.21.15.83:0/3638683146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 0x5619663459b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faaf00146a0 tx=0x7faaf0004a40 comp rx=0 tx=0).stop 2024-03-11T19:53:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.940+0000 7fab00fbb740 1 -- 172.21.15.83:0/3638683146 shutdown_connections 2024-03-11T19:53:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.940+0000 7fab00fbb740 1 --2- 172.21.15.83:0/3638683146 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x56196634a4a0 0x56196634c8c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.940+0000 7fab00fbb740 1 --2- 172.21.15.83:0/3638683146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966339dd0 0x5619663459b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.940+0000 7fab00fbb740 1 --2- 172.21.15.83:0/3638683146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334c60 0x56196633e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.940+0000 7fab00fbb740 1 --2- 172.21.15.83:0/3638683146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334270 0x56196633e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.940+0000 7fab00fbb740 1 -- 172.21.15.83:0/3638683146 >> 172.21.15.83:0/3638683146 conn(0x56196632f7e0 msgr2=0x56196632eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.941+0000 7faad77fe640 0 client.27512 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:53:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.941+0000 7fab00fbb740 1 -- 172.21.15.83:0/3638683146 shutdown_connections 2024-03-11T19:53:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.941+0000 7fab00fbb740 1 -- 172.21.15.83:0/3638683146 wait complete. 2024-03-11T19:53:51.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.941+0000 7fab00fbb740 20 client.27512 trim_cache size 0 max 16384 2024-03-11T19:53:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.965+0000 7fab00fbb740 1 Processor -- start 2024-03-11T19:53:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.965+0000 7fab00fbb740 1 -- start start 2024-03-11T19:53:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.965+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334270 0x5619663360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.965+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966334c60 0x561966345940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.965+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966339dd0 0x561966336620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7fab00fbb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x561966335ca0 con 0x561966334270 2024-03-11T19:53:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7fab00fbb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561966222130 con 0x561966334c60 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7fab00fbb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196633deb0 con 0x561966339dd0 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faafc97f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334270 0x5619663360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faaf77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966334c60 0x561966345940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faafc97f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334270 0x5619663360e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53416/0 (socket says 172.21.15.83:53416) 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faafc97f640 1 -- 172.21.15.83:0/2835736313 learned_addr learned my addr 172.21.15.83:0/2835736313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faafc97f640 1 --2- 172.21.15.83:0/2835736313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966339dd0 0x561966336620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faafc97f640 1 -- 172.21.15.83:0/2835736313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966339dd0 msgr2=0x561966336620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faafc97f640 1 --2- 172.21.15.83:0/2835736313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966339dd0 0x561966336620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faafc97f640 1 -- 172.21.15.83:0/2835736313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966334c60 msgr2=0x561966345940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faafc97f640 1 --2- 172.21.15.83:0/2835736313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966334c60 0x561966345940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faafc97f640 1 -- 172.21.15.83:0/2835736313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561966237c20 con 0x561966334270 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faafc97f640 1 --2- 172.21.15.83:0/2835736313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334270 0x5619663360e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x561965f9c920 tx=0x7faaf0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365d1b2bdbd219c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faaddffb640 1 -- 172.21.15.83:0/2835736313 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0027910 con 0x561966334270 2024-03-11T19:53:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faaddffb640 1 -- 172.21.15.83:0/2835736313 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faaf0010030 con 0x561966334270 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.966+0000 7faaddffb640 1 -- 172.21.15.83:0/2835736313 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0027da0 con 0x561966334270 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.967+0000 7fab00fbb740 1 -- 172.21.15.83:0/2835736313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334270 msgr2=0x5619663360e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.967+0000 7fab00fbb740 1 --2- 172.21.15.83:0/2835736313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334270 0x5619663360e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x561965f9c920 tx=0x7faaf0000f40 comp rx=0 tx=0).stop 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.967+0000 7fab00fbb740 1 -- 172.21.15.83:0/2835736313 shutdown_connections 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.967+0000 7fab00fbb740 1 --2- 172.21.15.83:0/2835736313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966339dd0 0x561966336620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.967+0000 7fab00fbb740 1 --2- 172.21.15.83:0/2835736313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966334c60 0x561966345940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.967+0000 7fab00fbb740 1 --2- 172.21.15.83:0/2835736313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966334270 0x5619663360e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.967+0000 7fab00fbb740 1 -- 172.21.15.83:0/2835736313 >> 172.21.15.83:0/2835736313 conn(0x56196632f7e0 msgr2=0x56196633e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.967+0000 7fab00fbb740 1 -- 172.21.15.83:0/2835736313 shutdown_connections 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.968+0000 7fab00fbb740 1 -- 172.21.15.83:0/2835736313 wait complete. 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.968+0000 7fab00fbb740 1 -- start start 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.968+0000 7fab00fbb740 1 Processor -- start 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.968+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966334270 0x561966344350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.968+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334c60 0x561966344890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7fab00fbb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966339dd0 0x561966344dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7fab00fbb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619663a09c0 con 0x561966339dd0 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faafc97f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966334270 0x561966344350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faaf77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334c60 0x561966344890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faafc97f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966334270 0x561966344350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:34830/0 (socket says 172.21.15.83:34830) 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faafc97f640 1 -- 172.21.15.83:0/4055452803 learned_addr learned my addr 172.21.15.83:0/4055452803 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7fab00fbb740 1 -- 172.21.15.83:0/4055452803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619663a0b30 con 0x561966334270 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7fab00fbb740 1 -- 172.21.15.83:0/4055452803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619663a0cf0 con 0x561966334c60 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faafc97f640 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966339dd0 0x561966344dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7fab00fbb740 10 client.0 _ll_register_callbacks cb 0x7faaf0009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faaf77fe640 1 -- 172.21.15.83:0/4055452803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966334270 msgr2=0x561966344350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faaf77fe640 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966334270 0x561966344350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faaf77fe640 1 -- 172.21.15.83:0/4055452803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966339dd0 msgr2=0x561966344dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faaf77fe640 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966339dd0 0x561966344dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faaf77fe640 1 -- 172.21.15.83:0/4055452803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56196629b6a0 con 0x561966334c60 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faafc97f640 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966339dd0 0x561966344dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faaf77fe640 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334c60 0x561966344890 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x56196634d170 tx=0x7faaec013980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a44a151366358633 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faad77fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faad77fe640 1 -- 172.21.15.83:0/4055452803 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec017040 con 0x561966334c60 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7faad77fe640 1 -- 172.21.15.83:0/4055452803 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7faaec004460 con 0x561966334c60 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.969+0000 7fab00fbb740 1 -- 172.21.15.83:0/4055452803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faaf0013c70 con 0x561966334c60 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7faad77fe640 1 -- 172.21.15.83:0/4055452803 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaec02c030 con 0x561966334c60 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7fab00fbb740 10 client.27619 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7fab00fbb740 1 -- 172.21.15.83:0/4055452803 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56196625d8c0 con 0x561966334c60 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7faad77fe640 1 -- 172.21.15.83:0/4055452803 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faaf0013c70 con 0x561966334c60 2024-03-11T19:53:51.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7faad77fe640 1 -- 172.21.15.83:0/4055452803 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faaec024020 con 0x561966334c60 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7fab00fbb740 10 client.27619 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7fab00fbb740 10 client.27619 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7fab00fbb740 20 client.27619 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7fab00fbb740 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5619663a27c0 0x5619663a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7fab00fbb740 4 client.27619 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7fab00fbb740 1 -- 172.21.15.83:0/4055452803 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561966028890 con 0x5619663a27c0 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.970+0000 7faafc97f640 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5619663a27c0 0x5619663a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7faad77fe640 10 client.27619 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7faafc97f640 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5619663a27c0 0x5619663a4be0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcd6d45bc4c7e99c server_cookie=1428843d0f61c6f6 in_seq=0 out_seq=0 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7faad77fe640 1 -- 172.21.15.83:0/4055452803 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561966028890 con 0x5619663a27c0 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7faad77fe640 10 client.27619 handle_command_reply: tid=0 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7fab00fbb740 1 client.27619 shutdown 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7fab00fbb740 1 -- 172.21.15.83:0/4055452803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334c60 msgr2=0x561966344890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7fab00fbb740 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334c60 0x561966344890 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x56196634d170 tx=0x7faaec013980 comp rx=0 tx=0).stop 2024-03-11T19:53:51.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7fab00fbb740 1 -- 172.21.15.83:0/4055452803 shutdown_connections 2024-03-11T19:53:51.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7fab00fbb740 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x5619663a27c0 0x5619663a4be0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7faad77fe640 0 client.27619 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:53:51.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7fab00fbb740 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561966339dd0 0x561966344dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7fab00fbb740 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561966334c60 0x561966344890 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7fab00fbb740 1 --2- 172.21.15.83:0/4055452803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561966334270 0x561966344350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.971+0000 7fab00fbb740 1 -- 172.21.15.83:0/4055452803 >> 172.21.15.83:0/4055452803 conn(0x56196632f7e0 msgr2=0x5619663360e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7fab00fbb740 1 -- 172.21.15.83:0/4055452803 shutdown_connections 2024-03-11T19:53:51.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7fab00fbb740 1 -- 172.21.15.83:0/4055452803 wait complete. 2024-03-11T19:53:51.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7fab00fbb740 20 client.27619 trim_cache size 0 max 16384 2024-03-11T19:53:51.983 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 -- 172.21.15.83:0/663112839 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faae00a1a30 msgr2=0x7faae00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faae00a1a30 0x7faae00a3ef0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x561966335060 tx=0x7faae80b2140 comp rx=0 tx=0).stop 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 -- 172.21.15.83:0/663112839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf8073be0 msgr2=0x7faaf819d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf8073be0 0x7faaf819d970 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faae8000c00 tx=0x7faae80040c0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 -- 172.21.15.83:0/663112839 shutdown_connections 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7faae00a1a30 0x7faae00a3ef0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faaf810d760 0x7faaf81a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faaf80751c0 0x7faaf819deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 --2- 172.21.15.83:0/663112839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf8073be0 0x7faaf819d970 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 -- 172.21.15.83:0/663112839 >> 172.21.15.83:0/663112839 conn(0x7faaf80fc6e0 msgr2=0x7faaf8100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 -- 172.21.15.83:0/663112839 shutdown_connections 2024-03-11T19:53:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:51.972+0000 7faafe3e2640 1 -- 172.21.15.83:0/663112839 wait complete. 2024-03-11T19:53:51.991 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:53:51.992 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:53:52.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.134+0000 7f0feeebb640 1 -- 172.21.15.83:0/2451410572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe810d4a0 msgr2=0x7f0fe810d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.134+0000 7f0feeebb640 1 --2- 172.21.15.83:0/2451410572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe810d4a0 0x7f0fe810d920 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0fe001acb0 tx=0x7f0fe003d130 comp rx=0 tx=0).stop 2024-03-11T19:53:52.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.137+0000 7f0feeebb640 1 -- 172.21.15.83:0/2451410572 shutdown_connections 2024-03-11T19:53:52.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.137+0000 7f0feeebb640 1 --2- 172.21.15.83:0/2451410572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe810d4a0 0x7f0fe810d920 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.137+0000 7f0feeebb640 1 --2- 172.21.15.83:0/2451410572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe810aaf0 0x7f0fe810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.137+0000 7f0feeebb640 1 --2- 172.21.15.83:0/2451410572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe8108180 0x7f0fe810a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.137+0000 7f0feeebb640 1 -- 172.21.15.83:0/2451410572 >> 172.21.15.83:0/2451410572 conn(0x7f0fe806dfe0 msgr2=0x7f0fe8070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.137+0000 7f0feeebb640 1 -- 172.21.15.83:0/2451410572 shutdown_connections 2024-03-11T19:53:52.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.137+0000 7f0feeebb640 1 -- 172.21.15.83:0/2451410572 wait complete. 2024-03-11T19:53:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0feeebb640 1 Processor -- start 2024-03-11T19:53:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0feeebb640 1 -- start start 2024-03-11T19:53:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0feeebb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe8108180 0x7f0fe819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0feeebb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe810aaf0 0x7f0fe819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0feeebb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810d4a0 0x7f0fe81a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0feeebb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe81a52b0 con 0x7f0fe810d4a0 2024-03-11T19:53:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0feeebb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe81a5420 con 0x7f0fe810aaf0 2024-03-11T19:53:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0feeebb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe81a55e0 con 0x7f0fe8108180 2024-03-11T19:53:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0fecc57640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe8108180 0x7f0fe819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0fecc57640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe8108180 0x7f0fe819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50228/0 (socket says 172.21.15.83:50228) 2024-03-11T19:53:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0fecc57640 1 -- 172.21.15.83:0/1239441785 learned_addr learned my addr 172.21.15.83:0/1239441785 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:52.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.138+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe810aaf0 0x7f0fe819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0fed458640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810d4a0 0x7f0fe81a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0fecc57640 1 -- 172.21.15.83:0/1239441785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe810aaf0 msgr2=0x7f0fe819dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0fecc57640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe810aaf0 0x7f0fe819dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0fecc57640 1 -- 172.21.15.83:0/1239441785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810d4a0 msgr2=0x7f0fe81a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0fecc57640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810d4a0 0x7f0fe81a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0fecc57640 1 -- 172.21.15.83:0/1239441785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe001a910 con 0x7f0fe8108180 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0fed458640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810d4a0 0x7f0fe81a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0fecc57640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe8108180 0x7f0fe819d7c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800e9f0 tx=0x7f0fd800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0fe5ffb640 1 -- 172.21.15.83:0/1239441785 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8009700 con 0x7f0fe8108180 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0feeebb640 1 -- 172.21.15.83:0/1239441785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe81a58c0 con 0x7f0fe8108180 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0fe5ffb640 1 -- 172.21.15.83:0/1239441785 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fd800cd80 con 0x7f0fe8108180 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0fe5ffb640 1 -- 172.21.15.83:0/1239441785 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8018920 con 0x7f0fe8108180 2024-03-11T19:53:52.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.139+0000 7f0feeebb640 1 -- 172.21.15.83:0/1239441785 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe81a5e20 con 0x7f0fe8108180 2024-03-11T19:53:52.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.142+0000 7f0ff1a85740 1 Processor -- start 2024-03-11T19:53:52.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.142+0000 7f0fe5ffb640 1 -- 172.21.15.83:0/1239441785 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0fd8018a80 con 0x7f0fe8108180 2024-03-11T19:53:52.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.142+0000 7f0ff1a85740 1 -- start start 2024-03-11T19:53:52.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7b270 0x558fb1e7b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7bc60 0x558fb1e80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 0x558fb1e83260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0ff1a85740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1e37960 con 0x558fb1e7bc60 2024-03-11T19:53:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0ff1a85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1deb310 con 0x558fb1e7b270 2024-03-11T19:53:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0ff1a85740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1de8840 con 0x558fb1e80dd0 2024-03-11T19:53:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fe7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 0x558fb1e83260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fecc57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7bc60 0x558fb1e80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fed458640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7b270 0x558fb1e7b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fe7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 0x558fb1e83260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50242/0 (socket says 172.21.15.83:50242) 2024-03-11T19:53:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fe7fff640 1 -- 172.21.15.83:0/3698494446 learned_addr learned my addr 172.21.15.83:0/3698494446 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:52.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fe5ffb640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fd00a1a30 0x7f0fd00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fe7fff640 1 -- 172.21.15.83:0/3698494446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7b270 msgr2=0x558fb1e7b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/3698494446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7b270 0x558fb1e7b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fe7fff640 1 -- 172.21.15.83:0/3698494446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7bc60 msgr2=0x558fb1e80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/3698494446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7bc60 0x558fb1e80760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fe7fff640 1 -- 172.21.15.83:0/3698494446 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fb1da6060 con 0x558fb1e80dd0 2024-03-11T19:53:52.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fe5ffb640 1 -- 172.21.15.83:0/1239441785 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0fd8014070 con 0x7f0fe8108180 2024-03-11T19:53:52.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fed458640 1 --2- 172.21.15.83:0/3698494446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7b270 0x558fb1e7b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.143+0000 7f0fed458640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fd00a1a30 0x7f0fd00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.144+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/3698494446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 0x558fb1e83260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009c90 tx=0x7f0fdc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569695ff164a3820 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.144+0000 7f0fceffd640 1 -- 172.21.15.83:0/3698494446 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc019630 con 0x558fb1e80dd0 2024-03-11T19:53:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.144+0000 7f0fceffd640 1 -- 172.21.15.83:0/3698494446 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fdc019c70 con 0x558fb1e80dd0 2024-03-11T19:53:52.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.144+0000 7f0fceffd640 1 -- 172.21.15.83:0/3698494446 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00ebf0 con 0x558fb1e80dd0 2024-03-11T19:53:52.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 -- 172.21.15.83:0/3698494446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 msgr2=0x558fb1e83260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0fed458640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fd00a1a30 0x7f0fd00a3ef0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0fe003d740 tx=0x7f0fe003f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/3698494446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 0x558fb1e83260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009c90 tx=0x7f0fdc011810 comp rx=0 tx=0).stop 2024-03-11T19:53:52.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 -- 172.21.15.83:0/3698494446 shutdown_connections 2024-03-11T19:53:52.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/3698494446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 0x558fb1e83260 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/3698494446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7bc60 0x558fb1e80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/3698494446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7b270 0x558fb1e7b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 -- 172.21.15.83:0/3698494446 >> 172.21.15.83:0/3698494446 conn(0x558fb1e767e0 msgr2=0x558fb1e76be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 -- 172.21.15.83:0/3698494446 shutdown_connections 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 -- 172.21.15.83:0/3698494446 wait complete. 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 -- start start 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 Processor -- start 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7b270 0x558fb1e85450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.145+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e7bc60 0x558fb1e85990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e80dd0 0x558fb1e8c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0ff1a85740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1de4020 con 0x558fb1e7b270 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0ff1a85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1ab7430 con 0x558fb1e80dd0 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0ff1a85740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1a73e00 con 0x558fb1e7bc60 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0ff1a85740 10 client.0 _ll_register_callbacks cb 0x558fb1d8bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fecc57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7b270 0x558fb1e85450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fe7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e7bc60 0x558fb1e85990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fecc57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7b270 0x558fb1e85450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53450/0 (socket says 172.21.15.83:53450) 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fecc57640 1 -- 172.21.15.83:0/369992300 learned_addr learned my addr 172.21.15.83:0/369992300 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fed458640 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e80dd0 0x558fb1e8c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fecc57640 1 -- 172.21.15.83:0/369992300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e7bc60 msgr2=0x558fb1e85990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fecc57640 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e7bc60 0x558fb1e85990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fecc57640 1 -- 172.21.15.83:0/369992300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e80dd0 msgr2=0x558fb1e8c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fecc57640 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e80dd0 0x558fb1e8c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fecc57640 1 -- 172.21.15.83:0/369992300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc0098f0 con 0x558fb1e7b270 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fecc57640 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7b270 0x558fb1e85450 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800c6a0 tx=0x7f0fd809a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354b180154fe6ee6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.146+0000 7f0fc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.147+0000 7f0fc7fff640 1 -- 172.21.15.83:0/369992300 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd809ae90 con 0x558fb1e7b270 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.148+0000 7f0ff1a85740 1 -- 172.21.15.83:0/369992300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fb1ae37e0 con 0x558fb1e7b270 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.148+0000 7f0fc7fff640 1 -- 172.21.15.83:0/369992300 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0fd809a790 con 0x558fb1e7b270 2024-03-11T19:53:52.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.149+0000 7f0fc7fff640 1 -- 172.21.15.83:0/369992300 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fd809e8e0 con 0x558fb1e7b270 2024-03-11T19:53:52.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.149+0000 7f0ff1a85740 10 client.18075 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:52.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.149+0000 7f0ff1a85740 1 -- 172.21.15.83:0/369992300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558fb1de26a0 con 0x558fb1e7b270 2024-03-11T19:53:52.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.149+0000 7f0fc7fff640 1 -- 172.21.15.83:0/369992300 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558fb1ae37e0 con 0x558fb1e7b270 2024-03-11T19:53:52.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.149+0000 7f0fc7fff640 1 -- 172.21.15.83:0/369992300 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0fd80cfb50 con 0x558fb1e7b270 2024-03-11T19:53:52.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.149+0000 7f0ff1a85740 10 client.18075 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:52.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.149+0000 7f0ff1a85740 10 client.18075 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:53:52.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.149+0000 7f0ff1a85740 20 client.18075 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:52.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.149+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558fb1e91620 0x558fb1e93a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.149+0000 7f0ff1a85740 4 client.18075 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:52.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.149+0000 7f0ff1a85740 1 -- 172.21.15.83:0/369992300 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fb1b20720 con 0x558fb1e91620 2024-03-11T19:53:52.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.150+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558fb1e91620 0x558fb1e93a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.150+0000 7f0fc7fff640 10 client.18075 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:53:52.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.150+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558fb1e91620 0x558fb1e93a60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=145d9ea64b3f662c server_cookie=4012e3202fb04f92 in_seq=0 out_seq=0 2024-03-11T19:53:52.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0fc7fff640 1 -- 172.21.15.83:0/369992300 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558fb1b20720 con 0x558fb1e91620 2024-03-11T19:53:52.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0fc7fff640 10 client.18075 handle_command_reply: tid=0 2024-03-11T19:53:52.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0ff1a85740 1 client.18075 shutdown 2024-03-11T19:53:52.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0ff1a85740 1 -- 172.21.15.83:0/369992300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7b270 msgr2=0x558fb1e85450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7b270 0x558fb1e85450 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800c6a0 tx=0x7f0fd809a560 comp rx=0 tx=0).stop 2024-03-11T19:53:52.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0ff1a85740 1 -- 172.21.15.83:0/369992300 shutdown_connections 2024-03-11T19:53:52.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558fb1e91620 0x558fb1e93a60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e80dd0 0x558fb1e8c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e7bc60 0x558fb1e85990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/369992300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7b270 0x558fb1e85450 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0ff1a85740 1 -- 172.21.15.83:0/369992300 >> 172.21.15.83:0/369992300 conn(0x558fb1e767e0 msgr2=0x558fb1e75b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0fc7fff640 0 client.18075 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:53:52.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0ff1a85740 1 -- 172.21.15.83:0/369992300 shutdown_connections 2024-03-11T19:53:52.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.151+0000 7f0ff1a85740 1 -- 172.21.15.83:0/369992300 wait complete. 2024-03-11T19:53:52.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.152+0000 7f0ff1a85740 20 client.18075 trim_cache size 0 max 16384 2024-03-11T19:53:52.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0ff1a85740 1 Processor -- start 2024-03-11T19:53:52.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0ff1a85740 1 -- start start 2024-03-11T19:53:52.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7b270 0x558fb1e7d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e7bc60 0x558fb1e7d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e80dd0 0x558fb1e8c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0ff1a85740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1d69130 con 0x558fb1e80dd0 2024-03-11T19:53:52.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0ff1a85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1b20720 con 0x558fb1e7b270 2024-03-11T19:53:52.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0ff1a85740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1ae37e0 con 0x558fb1e7bc60 2024-03-11T19:53:52.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0fed458640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e80dd0 0x558fb1e8c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0fed458640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e80dd0 0x558fb1e8c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53454/0 (socket says 172.21.15.83:53454) 2024-03-11T19:53:52.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0fed458640 1 -- 172.21.15.83:0/2685929729 learned_addr learned my addr 172.21.15.83:0/2685929729 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:52.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/2685929729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e7bc60 0x558fb1e7d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0fecc57640 1 --2- 172.21.15.83:0/2685929729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7b270 0x558fb1e7d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0fe7fff640 1 -- 172.21.15.83:0/2685929729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7b270 msgr2=0x558fb1e7d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/2685929729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7b270 0x558fb1e7d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0fe7fff640 1 -- 172.21.15.83:0/2685929729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e80dd0 msgr2=0x558fb1e8c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/2685929729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e80dd0 0x558fb1e8c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0fe7fff640 1 -- 172.21.15.83:0/2685929729 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fb1d7ec20 con 0x558fb1e7bc60 2024-03-11T19:53:52.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.166+0000 7f0fed458640 1 --2- 172.21.15.83:0/2685929729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e80dd0 0x558fb1e8c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:52.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.167+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/2685929729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e7bc60 0x558fb1e7d660 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x558fb1e943b0 tx=0x7f0fdc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ada05a3f410ba9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.167+0000 7f0fce7fc640 1 -- 172.21.15.83:0/2685929729 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc02db80 con 0x558fb1e7bc60 2024-03-11T19:53:52.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.167+0000 7f0fce7fc640 1 -- 172.21.15.83:0/2685929729 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fdc02d920 con 0x558fb1e7bc60 2024-03-11T19:53:52.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.167+0000 7f0fce7fc640 1 -- 172.21.15.83:0/2685929729 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc019430 con 0x558fb1e7bc60 2024-03-11T19:53:52.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.167+0000 7f0ff1a85740 1 -- 172.21.15.83:0/2685929729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e7bc60 msgr2=0x558fb1e7d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.167+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/2685929729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e7bc60 0x558fb1e7d660 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x558fb1e943b0 tx=0x7f0fdc000bc0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.167+0000 7f0ff1a85740 1 -- 172.21.15.83:0/2685929729 shutdown_connections 2024-03-11T19:53:52.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.167+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/2685929729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e80dd0 0x558fb1e8c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.167+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/2685929729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e7bc60 0x558fb1e7d660 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.167+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/2685929729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7b270 0x558fb1e7d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.167+0000 7f0ff1a85740 1 -- 172.21.15.83:0/2685929729 >> 172.21.15.83:0/2685929729 conn(0x558fb1e767e0 msgr2=0x558fb1e98400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.168+0000 7f0ff1a85740 1 -- 172.21.15.83:0/2685929729 shutdown_connections 2024-03-11T19:53:52.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.168+0000 7f0ff1a85740 1 -- 172.21.15.83:0/2685929729 wait complete. 2024-03-11T19:53:52.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.168+0000 7f0ff1a85740 1 -- start start 2024-03-11T19:53:52.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.168+0000 7f0ff1a85740 1 Processor -- start 2024-03-11T19:53:52.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.169+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7b270 0x558fb1e8c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.169+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7bc60 0x558fb1e8acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.169+0000 7f0ff1a85740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 0x558fb1e8b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.169+0000 7f0fe7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7bc60 0x558fb1e8acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.169+0000 7f0fe7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7bc60 0x558fb1e8acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:34878/0 (socket says 172.21.15.83:34878) 2024-03-11T19:53:52.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.169+0000 7f0fe7fff640 1 -- 172.21.15.83:0/4262810933 learned_addr learned my addr 172.21.15.83:0/4262810933 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:52.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.169+0000 7f0ff1a85740 1 -- 172.21.15.83:0/4262810933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1e8ce60 con 0x558fb1e7b270 2024-03-11T19:53:52.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.169+0000 7f0ff1a85740 1 -- 172.21.15.83:0/4262810933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1e8b780 con 0x558fb1e7bc60 2024-03-11T19:53:52.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.169+0000 7f0ff1a85740 1 -- 172.21.15.83:0/4262810933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb1e8b940 con 0x558fb1e80dd0 2024-03-11T19:53:52.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.170+0000 7f0fed458640 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 0x558fb1e8b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.170+0000 7f0ff1a85740 10 client.0 _ll_register_callbacks cb 0x7f0fdc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:52.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.170+0000 7f0fe7fff640 1 -- 172.21.15.83:0/4262810933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 msgr2=0x558fb1e8b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.170+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 0x558fb1e8b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.170+0000 7f0fe7fff640 1 -- 172.21.15.83:0/4262810933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7b270 msgr2=0x558fb1e8c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.170+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7b270 0x558fb1e8c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.170+0000 7f0fe7fff640 1 -- 172.21.15.83:0/4262810933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe810e4b0 con 0x558fb1e7bc60 2024-03-11T19:53:52.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.171+0000 7f0fe7fff640 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7bc60 0x558fb1e8acd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009310 tx=0x7f0fdc009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c872b5a6a433d2b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.171+0000 7f0fc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:53:52.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.171+0000 7f0fc7fff640 1 -- 172.21.15.83:0/4262810933 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc005660 con 0x558fb1e7bc60 2024-03-11T19:53:52.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.171+0000 7f0fc7fff640 1 -- 172.21.15.83:0/4262810933 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0fdc029040 con 0x558fb1e7bc60 2024-03-11T19:53:52.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.171+0000 7f0ff1a85740 1 -- 172.21.15.83:0/4262810933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fdc019c70 con 0x558fb1e7bc60 2024-03-11T19:53:52.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.171+0000 7f0fc7fff640 1 -- 172.21.15.83:0/4262810933 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fdc02db80 con 0x558fb1e7bc60 2024-03-11T19:53:52.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.171+0000 7f0ff1a85740 10 client.27536 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:52.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.171+0000 7f0ff1a85740 1 -- 172.21.15.83:0/4262810933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558fb1da6060 con 0x558fb1e7bc60 2024-03-11T19:53:52.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.171+0000 7f0fc7fff640 1 -- 172.21.15.83:0/4262810933 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fdc048020 con 0x558fb1e7bc60 2024-03-11T19:53:52.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.171+0000 7f0fc7fff640 1 -- 172.21.15.83:0/4262810933 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0fdc034040 con 0x558fb1e7bc60 2024-03-11T19:53:52.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0ff1a85740 10 client.27536 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:52.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0ff1a85740 10 client.27536 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:53:52.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0ff1a85740 20 client.27536 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:52.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558fb1ee8dc0 0x558fb1eeb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0ff1a85740 4 client.27536 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:53:52.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0ff1a85740 1 -- 172.21.15.83:0/4262810933 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558fb1b6f890 con 0x558fb1ee8dc0 2024-03-11T19:53:52.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0fecc57640 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558fb1ee8dc0 0x558fb1eeb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0fc7fff640 10 client.27536 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:53:52.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0fecc57640 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558fb1ee8dc0 0x558fb1eeb1e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6d72acb751b4a10 server_cookie=8e557bade30227d7 in_seq=0 out_seq=0 2024-03-11T19:53:52.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0fc7fff640 1 -- 172.21.15.83:0/4262810933 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558fb1b6f890 con 0x558fb1ee8dc0 2024-03-11T19:53:52.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0fc7fff640 10 client.27536 handle_command_reply: tid=0 2024-03-11T19:53:52.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0ff1a85740 1 client.27536 shutdown 2024-03-11T19:53:52.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0ff1a85740 1 -- 172.21.15.83:0/4262810933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7bc60 msgr2=0x558fb1e8acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.172+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7bc60 0x558fb1e8acd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009310 tx=0x7f0fdc009340 comp rx=0 tx=0).stop 2024-03-11T19:53:52.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0ff1a85740 1 -- 172.21.15.83:0/4262810933 shutdown_connections 2024-03-11T19:53:52.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x558fb1ee8dc0 0x558fb1eeb1e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0fc7fff640 0 client.27536 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:53:52.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558fb1e80dd0 0x558fb1e8b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558fb1e7bc60 0x558fb1e8acd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0ff1a85740 1 --2- 172.21.15.83:0/4262810933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558fb1e7b270 0x558fb1e8c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0ff1a85740 1 -- 172.21.15.83:0/4262810933 >> 172.21.15.83:0/4262810933 conn(0x558fb1e767e0 msgr2=0x558fb1e7d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0ff1a85740 1 -- 172.21.15.83:0/4262810933 shutdown_connections 2024-03-11T19:53:52.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0ff1a85740 1 -- 172.21.15.83:0/4262810933 wait complete. 2024-03-11T19:53:52.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0ff1a85740 20 client.27536 trim_cache size 0 max 16384 2024-03-11T19:53:52.190 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:53:52.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0feeebb640 1 -- 172.21.15.83:0/1239441785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fd00a1a30 msgr2=0x7f0fd00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0feeebb640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fd00a1a30 0x7f0fd00a3ef0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0fe003d740 tx=0x7f0fe003f040 comp rx=0 tx=0).stop 2024-03-11T19:53:52.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0feeebb640 1 -- 172.21.15.83:0/1239441785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe8108180 msgr2=0x7f0fe819d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.173+0000 7f0feeebb640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe8108180 0x7f0fe819d7c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800e9f0 tx=0x7f0fd800eeb0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.174+0000 7f0feeebb640 1 -- 172.21.15.83:0/1239441785 shutdown_connections 2024-03-11T19:53:52.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.174+0000 7f0feeebb640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0fd00a1a30 0x7f0fd00a3ef0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.174+0000 7f0feeebb640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe810d4a0 0x7f0fe81a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.174+0000 7f0feeebb640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fe810aaf0 0x7f0fe819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.174+0000 7f0feeebb640 1 --2- 172.21.15.83:0/1239441785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fe8108180 0x7f0fe819d7c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.174+0000 7f0feeebb640 1 -- 172.21.15.83:0/1239441785 >> 172.21.15.83:0/1239441785 conn(0x7f0fe806dfe0 msgr2=0x7f0fe810b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.174+0000 7f0feeebb640 1 -- 172.21.15.83:0/1239441785 shutdown_connections 2024-03-11T19:53:52.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.174+0000 7f0feeebb640 1 -- 172.21.15.83:0/1239441785 wait complete. 2024-03-11T19:53:52.195 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:53:52.195 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:53:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.339+0000 7fa878ef4640 1 -- 172.21.15.83:0/2669281188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa87410aaf0 msgr2=0x7fa87410cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.339+0000 7fa878ef4640 1 --2- 172.21.15.83:0/2669281188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa87410aaf0 0x7fa87410cf60 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa864009900 tx=0x7fa86402bc80 comp rx=0 tx=0).stop 2024-03-11T19:53:52.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.339+0000 7fa878ef4640 1 -- 172.21.15.83:0/2669281188 shutdown_connections 2024-03-11T19:53:52.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.339+0000 7fa878ef4640 1 --2- 172.21.15.83:0/2669281188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87410d4a0 0x7fa87410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.339+0000 7fa878ef4640 1 --2- 172.21.15.83:0/2669281188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa87410aaf0 0x7fa87410cf60 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.339+0000 7fa878ef4640 1 --2- 172.21.15.83:0/2669281188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa874108180 0x7fa87410a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.340+0000 7fa878ef4640 1 -- 172.21.15.83:0/2669281188 >> 172.21.15.83:0/2669281188 conn(0x7fa87406dfe0 msgr2=0x7fa874070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.341+0000 7fa878ef4640 1 -- 172.21.15.83:0/2669281188 shutdown_connections 2024-03-11T19:53:52.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.343+0000 7fa878ef4640 1 -- 172.21.15.83:0/2669281188 wait complete. 2024-03-11T19:53:52.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.344+0000 7fa878ef4640 1 Processor -- start 2024-03-11T19:53:52.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.344+0000 7fa878ef4640 1 -- start start 2024-03-11T19:53:52.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.344+0000 7fa878ef4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa874108180 0x7fa8741179a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa878ef4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa87410aaf0 0x7fa874117ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa878ef4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87410d4a0 0x7fa874118f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa878ef4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa874119460 con 0x7fa874108180 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa878ef4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8741195d0 con 0x7fa87410aaf0 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa878ef4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa874119790 con 0x7fa87410d4a0 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa871d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa87410aaf0 0x7fa874117ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa871d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa87410aaf0 0x7fa874117ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:34890/0 (socket says 172.21.15.83:34890) 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa871d9b640 1 -- 172.21.15.83:0/3385873945 learned_addr learned my addr 172.21.15.83:0/3385873945 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa87259c640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa874108180 0x7fa8741179a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa871d9b640 1 -- 172.21.15.83:0/3385873945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87410d4a0 msgr2=0x7fa874118f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa871d9b640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87410d4a0 0x7fa874118f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa871d9b640 1 -- 172.21.15.83:0/3385873945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa874108180 msgr2=0x7fa8741179a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.345+0000 7fa871d9b640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa874108180 0x7fa8741179a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.346+0000 7fa871d9b640 1 -- 172.21.15.83:0/3385873945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa864009560 con 0x7fa87410aaf0 2024-03-11T19:53:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.346+0000 7fa871d9b640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa87410aaf0 0x7fa874117ee0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa864009230 tx=0x7fa864005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.346+0000 7fa8637fe640 1 -- 172.21.15.83:0/3385873945 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640048c0 con 0x7fa87410aaf0 2024-03-11T19:53:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.346+0000 7fa878ef4640 1 -- 172.21.15.83:0/3385873945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa874119a10 con 0x7fa87410aaf0 2024-03-11T19:53:52.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.346+0000 7fa878ef4640 1 -- 172.21.15.83:0/3385873945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa874119eb0 con 0x7fa87410aaf0 2024-03-11T19:53:52.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.350+0000 7fa8637fe640 1 -- 172.21.15.83:0/3385873945 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa864005670 con 0x7fa87410aaf0 2024-03-11T19:53:52.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.350+0000 7fa8637fe640 1 -- 172.21.15.83:0/3385873945 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86403e460 con 0x7fa87410aaf0 2024-03-11T19:53:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.351+0000 7fa87bba2740 1 Processor -- start 2024-03-11T19:53:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.351+0000 7fa87bba2740 1 -- start start 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa87bba2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95dc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa87bba2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dcc60 0x5574a95e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa87bba2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 0x5574a95e4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa8637fe640 1 -- 172.21.15.83:0/3385873945 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa86403e680 con 0x7fa87410aaf0 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa872d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95dc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa872d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95dc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50286/0 (socket says 172.21.15.83:50286) 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa872d9d640 1 -- 172.21.15.83:0/2293054732 learned_addr learned my addr 172.21.15.83:0/2293054732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa87bba2740 1 -- 172.21.15.83:0/2293054732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a9598960 con 0x5574a95e1dd0 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa87bba2740 1 -- 172.21.15.83:0/2293054732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a954c310 con 0x5574a95dcc60 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa87bba2740 1 -- 172.21.15.83:0/2293054732 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a9549840 con 0x5574a95dc270 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa87259c640 1 --2- 172.21.15.83:0/2293054732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dcc60 0x5574a95e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.352+0000 7fa871d9b640 1 --2- 172.21.15.83:0/2293054732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 0x5574a95e4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.353+0000 7fa872d9d640 1 -- 172.21.15.83:0/2293054732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dcc60 msgr2=0x5574a95e1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.353+0000 7fa872d9d640 1 --2- 172.21.15.83:0/2293054732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dcc60 0x5574a95e1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.353+0000 7fa872d9d640 1 -- 172.21.15.83:0/2293054732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 msgr2=0x5574a95e4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.353+0000 7fa872d9d640 1 --2- 172.21.15.83:0/2293054732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 0x5574a95e4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.353+0000 7fa872d9d640 1 -- 172.21.15.83:0/2293054732 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574a9507060 con 0x5574a95dc270 2024-03-11T19:53:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.357+0000 7fa872d9d640 1 --2- 172.21.15.83:0/2293054732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95dc690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009e70 tx=0x7fa86c00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=451ba3021e687210 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.357+0000 7fa8637fe640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa85c0a1b80 0x7fa85c0a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.357+0000 7fa8637fe640 1 -- 172.21.15.83:0/3385873945 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa8640f0880 con 0x7fa87410aaf0 2024-03-11T19:53:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.357+0000 7fa87259c640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa85c0a1b80 0x7fa85c0a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.358+0000 7fa87259c640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa85c0a1b80 0x7fa85c0a4040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x5574a95dd060 tx=0x7fa868008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.360+0000 7fa860ff9640 1 -- 172.21.15.83:0/2293054732 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c004d20 con 0x5574a95dc270 2024-03-11T19:53:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.360+0000 7fa860ff9640 1 -- 172.21.15.83:0/2293054732 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa86c004e80 con 0x5574a95dc270 2024-03-11T19:53:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.360+0000 7fa860ff9640 1 -- 172.21.15.83:0/2293054732 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c005020 con 0x5574a95dc270 2024-03-11T19:53:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.361+0000 7fa87bba2740 1 -- 172.21.15.83:0/2293054732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 msgr2=0x5574a95dc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.361+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2293054732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95dc690 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009e70 tx=0x7fa86c00e720 comp rx=0 tx=0).stop 2024-03-11T19:53:52.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.361+0000 7fa87bba2740 1 -- 172.21.15.83:0/2293054732 shutdown_connections 2024-03-11T19:53:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.361+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2293054732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 0x5574a95e4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.361+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2293054732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dcc60 0x5574a95e1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.361+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2293054732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95dc690 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.361+0000 7fa87bba2740 1 -- 172.21.15.83:0/2293054732 >> 172.21.15.83:0/2293054732 conn(0x5574a95d77e0 msgr2=0x5574a95d7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.368+0000 7fa87bba2740 1 -- 172.21.15.83:0/2293054732 shutdown_connections 2024-03-11T19:53:52.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.371+0000 7fa87bba2740 1 -- 172.21.15.83:0/2293054732 wait complete. 2024-03-11T19:53:52.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.371+0000 7fa87bba2740 1 -- start start 2024-03-11T19:53:52.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.371+0000 7fa87bba2740 1 Processor -- start 2024-03-11T19:53:52.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.372+0000 7fa87bba2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.372+0000 7fa87bba2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95dcc60 0x5574a95e6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.372+0000 7fa87259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.372+0000 7fa87259c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95e63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50290/0 (socket says 172.21.15.83:50290) 2024-03-11T19:53:52.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.372+0000 7fa87259c640 1 -- 172.21.15.83:0/2885790132 learned_addr learned my addr 172.21.15.83:0/2885790132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:52.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.372+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95e1dd0 0x5574a95ed8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.372+0000 7fa87bba2740 1 -- 172.21.15.83:0/2885790132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a9545020 con 0x5574a95dcc60 2024-03-11T19:53:52.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.372+0000 7fa87bba2740 1 -- 172.21.15.83:0/2885790132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a9218430 con 0x5574a95e1dd0 2024-03-11T19:53:52.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.373+0000 7fa872d9d640 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95e1dd0 0x5574a95ed8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.373+0000 7fa87bba2740 1 -- 172.21.15.83:0/2885790132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a91d4e00 con 0x5574a95dc270 2024-03-11T19:53:52.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.373+0000 7fa87bba2740 10 client.0 _ll_register_callbacks cb 0x5574a94eccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:52.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.373+0000 7fa87259c640 1 -- 172.21.15.83:0/2885790132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95e1dd0 msgr2=0x5574a95ed8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.373+0000 7fa87259c640 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95e1dd0 0x5574a95ed8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.373+0000 7fa87259c640 1 -- 172.21.15.83:0/2885790132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95dcc60 msgr2=0x5574a95e6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.373+0000 7fa87259c640 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95dcc60 0x5574a95e6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.373+0000 7fa87259c640 1 -- 172.21.15.83:0/2885790132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c009b20 con 0x5574a95dc270 2024-03-11T19:53:52.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.375+0000 7fa87259c640 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95e63c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa868006fa0 tx=0x7fa8680046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bde788fa600897e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.375+0000 7fa859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:53:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.379+0000 7fa859ffb640 1 -- 172.21.15.83:0/2885790132 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868014930 con 0x5574a95dc270 2024-03-11T19:53:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.379+0000 7fa859ffb640 1 -- 172.21.15.83:0/2885790132 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa868014a90 con 0x5574a95dc270 2024-03-11T19:53:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.379+0000 7fa87bba2740 1 -- 172.21.15.83:0/2885790132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574a9281720 con 0x5574a95dc270 2024-03-11T19:53:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.379+0000 7fa859ffb640 1 -- 172.21.15.83:0/2885790132 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa868014c50 con 0x5574a95dc270 2024-03-11T19:53:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.379+0000 7fa87bba2740 10 client.27646 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.380+0000 7fa87bba2740 1 -- 172.21.15.83:0/2885790132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5574a95436a0 con 0x5574a95dc270 2024-03-11T19:53:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.380+0000 7fa859ffb640 1 -- 172.21.15.83:0/2885790132 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa868030020 con 0x5574a95dc270 2024-03-11T19:53:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.381+0000 7fa859ffb640 1 -- 172.21.15.83:0/2885790132 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa8680066d0 con 0x5574a95dc270 2024-03-11T19:53:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.381+0000 7fa87bba2740 10 client.27646 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.381+0000 7fa87bba2740 10 client.27646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.381+0000 7fa87bba2740 20 client.27646 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.381+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574a95f25b0 0x5574a95f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.382+0000 7fa87bba2740 4 client.27646 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.382+0000 7fa87bba2740 1 -- 172.21.15.83:0/2885790132 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574a9281720 con 0x5574a95f25b0 2024-03-11T19:53:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.382+0000 7fa871d9b640 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574a95f25b0 0x5574a95f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.383+0000 7fa871d9b640 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574a95f25b0 0x5574a95f4a50 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=324de76267ed00df server_cookie=f11107f36601b04c in_seq=0 out_seq=0 2024-03-11T19:53:52.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.383+0000 7fa859ffb640 10 client.27646 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:52.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.385+0000 7fa859ffb640 1 -- 172.21.15.83:0/2885790132 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574a9281720 con 0x5574a95f25b0 2024-03-11T19:53:52.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.385+0000 7fa859ffb640 10 client.27646 handle_command_reply: tid=0 2024-03-11T19:53:52.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.387+0000 7fa87bba2740 1 client.27646 shutdown 2024-03-11T19:53:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.387+0000 7fa87bba2740 1 -- 172.21.15.83:0/2885790132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 msgr2=0x5574a95e63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.388+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95e63c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa868006fa0 tx=0x7fa8680046e0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.388+0000 7fa87bba2740 1 -- 172.21.15.83:0/2885790132 shutdown_connections 2024-03-11T19:53:52.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.388+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574a95f25b0 0x5574a95f4a50 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.388+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95e1dd0 0x5574a95ed8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.388+0000 7fa859ffb640 0 client.27646 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:52.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.388+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95dcc60 0x5574a95e6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.388+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2885790132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dc270 0x5574a95e63c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.388+0000 7fa87bba2740 1 -- 172.21.15.83:0/2885790132 >> 172.21.15.83:0/2885790132 conn(0x5574a95d77e0 msgr2=0x5574a95d6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.388+0000 7fa87bba2740 1 -- 172.21.15.83:0/2885790132 shutdown_connections 2024-03-11T19:53:52.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.388+0000 7fa87bba2740 1 -- 172.21.15.83:0/2885790132 wait complete. 2024-03-11T19:53:52.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.389+0000 7fa87bba2740 20 client.27646 trim_cache size 0 max 16384 2024-03-11T19:53:52.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa87bba2740 1 Processor -- start 2024-03-11T19:53:52.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa87bba2740 1 -- start start 2024-03-11T19:53:52.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa87bba2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95dc270 0x5574a95ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa87bba2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dcc60 0x5574a95de520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa87bba2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95e1dd0 0x5574a95ed890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa87bba2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a94ca130 con 0x5574a95dc270 2024-03-11T19:53:52.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa87bba2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a95e6200 con 0x5574a95dcc60 2024-03-11T19:53:52.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa87bba2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a9281720 con 0x5574a95e1dd0 2024-03-11T19:53:52.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa87259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95dc270 0x5574a95ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa87259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95dc270 0x5574a95ddfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53516/0 (socket says 172.21.15.83:53516) 2024-03-11T19:53:52.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa87259c640 1 -- 172.21.15.83:0/1929799584 learned_addr learned my addr 172.21.15.83:0/1929799584 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:52.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa872d9d640 1 --2- 172.21.15.83:0/1929799584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95e1dd0 0x5574a95ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.404+0000 7fa871d9b640 1 --2- 172.21.15.83:0/1929799584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dcc60 0x5574a95de520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.405+0000 7fa871d9b640 1 -- 172.21.15.83:0/1929799584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95e1dd0 msgr2=0x5574a95ed890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.405+0000 7fa871d9b640 1 --2- 172.21.15.83:0/1929799584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95e1dd0 0x5574a95ed890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.405+0000 7fa871d9b640 1 -- 172.21.15.83:0/1929799584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95dc270 msgr2=0x5574a95ddfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.405+0000 7fa871d9b640 1 --2- 172.21.15.83:0/1929799584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95dc270 0x5574a95ddfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.405+0000 7fa871d9b640 1 -- 172.21.15.83:0/1929799584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574a94dfc20 con 0x5574a95dcc60 2024-03-11T19:53:52.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.406+0000 7fa871d9b640 1 --2- 172.21.15.83:0/1929799584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dcc60 0x5574a95de520 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa864040000 tx=0x7fa86402bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d403cf813c6920e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.407+0000 7fa85bfff640 1 -- 172.21.15.83:0/1929799584 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640bdfc0 con 0x5574a95dcc60 2024-03-11T19:53:52.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.407+0000 7fa85bfff640 1 -- 172.21.15.83:0/1929799584 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8640bd8c0 con 0x5574a95dcc60 2024-03-11T19:53:52.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.407+0000 7fa85bfff640 1 -- 172.21.15.83:0/1929799584 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8640b9f30 con 0x5574a95dcc60 2024-03-11T19:53:52.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 -- 172.21.15.83:0/1929799584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dcc60 msgr2=0x5574a95de520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 --2- 172.21.15.83:0/1929799584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dcc60 0x5574a95de520 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa864040000 tx=0x7fa86402bc80 comp rx=0 tx=0).stop 2024-03-11T19:53:52.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 -- 172.21.15.83:0/1929799584 shutdown_connections 2024-03-11T19:53:52.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 --2- 172.21.15.83:0/1929799584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95e1dd0 0x5574a95ed890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 --2- 172.21.15.83:0/1929799584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dcc60 0x5574a95de520 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 --2- 172.21.15.83:0/1929799584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95dc270 0x5574a95ddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 -- 172.21.15.83:0/1929799584 >> 172.21.15.83:0/1929799584 conn(0x5574a95d77e0 msgr2=0x5574a95d7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 -- 172.21.15.83:0/1929799584 shutdown_connections 2024-03-11T19:53:52.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 -- 172.21.15.83:0/1929799584 wait complete. 2024-03-11T19:53:52.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 -- start start 2024-03-11T19:53:52.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 Processor -- start 2024-03-11T19:53:52.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dc270 0x5574a95ec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dcc60 0x5574a95ecdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 0x5574a9648b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a95ed4b0 con 0x5574a95e1dd0 2024-03-11T19:53:52.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a9649060 con 0x5574a95dc270 2024-03-11T19:53:52.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574a96491d0 con 0x5574a95dcc60 2024-03-11T19:53:52.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa87bba2740 10 client.0 _ll_register_callbacks cb 0x7fa86c0098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa872d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 0x5574a9648b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.408+0000 7fa871d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dcc60 0x5574a95ecdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa872d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 0x5574a9648b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53532/0 (socket says 172.21.15.83:53532) 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa87259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dc270 0x5574a95ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa872d9d640 1 -- 172.21.15.83:0/2471590121 learned_addr learned my addr 172.21.15.83:0/2471590121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa872d9d640 1 -- 172.21.15.83:0/2471590121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dcc60 msgr2=0x5574a95ecdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa872d9d640 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dcc60 0x5574a95ecdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa872d9d640 1 -- 172.21.15.83:0/2471590121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dc270 msgr2=0x5574a95ec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa872d9d640 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dc270 0x5574a95ec870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa872d9d640 1 -- 172.21.15.83:0/2471590121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa874119eb0 con 0x5574a95e1dd0 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa871d9b640 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dcc60 0x5574a95ecdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa872d9d640 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 0x5574a9648b20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009e70 tx=0x7fa86c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace74389526ba0cb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:52.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa859ffb640 1 -- 172.21.15.83:0/2471590121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c005210 con 0x5574a95e1dd0 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa859ffb640 1 -- 172.21.15.83:0/2471590121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa86c014070 con 0x5574a95e1dd0 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa87bba2740 1 -- 172.21.15.83:0/2471590121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574a92447e0 con 0x5574a95e1dd0 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa859ffb640 1 -- 172.21.15.83:0/2471590121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa86c005370 con 0x5574a95e1dd0 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa87bba2740 10 client.18093 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.409+0000 7fa87bba2740 1 -- 172.21.15.83:0/2471590121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5574a95058c0 con 0x5574a95e1dd0 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.410+0000 7fa859ffb640 1 -- 172.21.15.83:0/2471590121 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa86c036020 con 0x5574a95e1dd0 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.410+0000 7fa859ffb640 1 -- 172.21.15.83:0/2471590121 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa86c032b40 con 0x5574a95e1dd0 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.410+0000 7fa87bba2740 10 client.18093 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.410+0000 7fa87bba2740 10 client.18093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.410+0000 7fa87bba2740 20 client.18093 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.410+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574a95f25b0 0x5574a964ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.410+0000 7fa87bba2740 4 client.18093 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:53:52.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.410+0000 7fa87bba2740 1 -- 172.21.15.83:0/2471590121 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5574a92447e0 con 0x5574a95f25b0 2024-03-11T19:53:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.410+0000 7fa87259c640 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574a95f25b0 0x5574a964ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa859ffb640 10 client.18093 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87259c640 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574a95f25b0 0x5574a964ccb0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd179135b822715b server_cookie=d1f0bfe761ce5741 in_seq=0 out_seq=0 2024-03-11T19:53:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa859ffb640 1 -- 172.21.15.83:0/2471590121 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5574a92447e0 con 0x5574a95f25b0 2024-03-11T19:53:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa859ffb640 10 client.18093 handle_command_reply: tid=0 2024-03-11T19:53:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 1 client.18093 shutdown 2024-03-11T19:53:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 1 -- 172.21.15.83:0/2471590121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 msgr2=0x5574a9648b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 0x5574a9648b20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009e70 tx=0x7fa86c000f40 comp rx=0 tx=0).stop 2024-03-11T19:53:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 1 -- 172.21.15.83:0/2471590121 shutdown_connections 2024-03-11T19:53:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5574a95f25b0 0x5574a964ccb0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa859ffb640 0 client.18093 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5574a95e1dd0 0x5574a9648b20 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574a95dcc60 0x5574a95ecdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 1 --2- 172.21.15.83:0/2471590121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5574a95dc270 0x5574a95ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 1 -- 172.21.15.83:0/2471590121 >> 172.21.15.83:0/2471590121 conn(0x5574a95d77e0 msgr2=0x5574a95ddfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 1 -- 172.21.15.83:0/2471590121 shutdown_connections 2024-03-11T19:53:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 1 -- 172.21.15.83:0/2471590121 wait complete. 2024-03-11T19:53:52.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.411+0000 7fa87bba2740 20 client.18093 trim_cache size 0 max 16384 2024-03-11T19:53:52.428 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:53:52.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 -- 172.21.15.83:0/3385873945 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa85c0a1b80 msgr2=0x7fa85c0a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa85c0a1b80 0x7fa85c0a4040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x5574a95dd060 tx=0x7fa868008040 comp rx=0 tx=0).stop 2024-03-11T19:53:52.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 -- 172.21.15.83:0/3385873945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa87410aaf0 msgr2=0x7fa874117ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:52.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa87410aaf0 0x7fa874117ee0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa864009230 tx=0x7fa864005270 comp rx=0 tx=0).stop 2024-03-11T19:53:52.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 -- 172.21.15.83:0/3385873945 shutdown_connections 2024-03-11T19:53:52.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa85c0a1b80 0x7fa85c0a4040 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87410d4a0 0x7fa874118f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa87410aaf0 0x7fa874117ee0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 --2- 172.21.15.83:0/3385873945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa874108180 0x7fa8741179a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:52.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 -- 172.21.15.83:0/3385873945 >> 172.21.15.83:0/3385873945 conn(0x7fa87406dfe0 msgr2=0x7fa87410b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:52.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 -- 172.21.15.83:0/3385873945 shutdown_connections 2024-03-11T19:53:52.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:52.412+0000 7fa878ef4640 1 -- 172.21.15.83:0/3385873945 wait complete. 2024-03-11T19:53:52.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 875.0 s, 2178.4 tps, lat 46.322 ms stddev 56.138 2024-03-11T19:53:52.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:52 smithi083 ceph-mon[31864]: pgmap v788: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 724 op/s 2024-03-11T19:53:52.818 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:52 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2594633361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:52.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:52 smithi204 ceph-mon[36739]: pgmap v788: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 724 op/s 2024-03-11T19:53:52.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:52 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2594633361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:53.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:52 smithi137 ceph-mon[36220]: pgmap v788: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 724 op/s 2024-03-11T19:53:53.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:52 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2594633361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:53:53.431 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:53:53.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:53:53.432 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:53:53.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.580+0000 7f0d8cf08640 1 -- 172.21.15.83:0/2079311233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d8810d4a0 msgr2=0x7f0d8810d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.580+0000 7f0d8cf08640 1 --2- 172.21.15.83:0/2079311233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d8810d4a0 0x7f0d8810d920 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009900 tx=0x7f0d7802bc30 comp rx=0 tx=0).stop 2024-03-11T19:53:53.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.580+0000 7f0d8cf08640 1 -- 172.21.15.83:0/2079311233 shutdown_connections 2024-03-11T19:53:53.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.580+0000 7f0d8cf08640 1 --2- 172.21.15.83:0/2079311233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d8810d4a0 0x7f0d8810d920 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.580+0000 7f0d8cf08640 1 --2- 172.21.15.83:0/2079311233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8810aaf0 0x7f0d8810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.580+0000 7f0d8cf08640 1 --2- 172.21.15.83:0/2079311233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d88108180 0x7f0d8810a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.580+0000 7f0d8cf08640 1 -- 172.21.15.83:0/2079311233 >> 172.21.15.83:0/2079311233 conn(0x7f0d8806dfe0 msgr2=0x7f0d88070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.583+0000 7f0d8cf08640 1 -- 172.21.15.83:0/2079311233 shutdown_connections 2024-03-11T19:53:53.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.583+0000 7f0d8cf08640 1 -- 172.21.15.83:0/2079311233 wait complete. 2024-03-11T19:53:53.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.584+0000 7f0d8cf08640 1 Processor -- start 2024-03-11T19:53:53.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.584+0000 7f0d8cf08640 1 -- start start 2024-03-11T19:53:53.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.584+0000 7f0d8cf08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d88108180 0x7f0d881a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.584+0000 7f0d8cf08640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8810aaf0 0x7f0d881a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.584+0000 7f0d8cf08640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8810d4a0 0x7f0d881ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.584+0000 7f0d8cf08640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881adb80 con 0x7f0d8810d4a0 2024-03-11T19:53:53.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.584+0000 7f0d8cf08640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881adcf0 con 0x7f0d88108180 2024-03-11T19:53:53.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.584+0000 7f0d8cf08640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881adeb0 con 0x7f0d8810aaf0 2024-03-11T19:53:53.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.585+0000 7f0d85d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8810aaf0 0x7f0d881a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.585+0000 7f0d85d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8810aaf0 0x7f0d881a6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50320/0 (socket says 172.21.15.83:50320) 2024-03-11T19:53:53.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.585+0000 7f0d85d9b640 1 -- 172.21.15.83:0/998712401 learned_addr learned my addr 172.21.15.83:0/998712401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:53.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.585+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8810d4a0 0x7f0d881ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.585+0000 7f0d8659c640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d88108180 0x7f0d881a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.585+0000 7f0d86d9d640 1 -- 172.21.15.83:0/998712401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8810aaf0 msgr2=0x7f0d881a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.585+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8810aaf0 0x7f0d881a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.585+0000 7f0d86d9d640 1 -- 172.21.15.83:0/998712401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d88108180 msgr2=0x7f0d881a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.585+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d88108180 0x7f0d881a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.585+0000 7f0d86d9d640 1 -- 172.21.15.83:0/998712401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d78009560 con 0x7f0d8810d4a0 2024-03-11T19:53:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.586+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8810d4a0 0x7f0d881ad640 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009230 tx=0x7f0d780042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.586+0000 7f0d777fe640 1 -- 172.21.15.83:0/998712401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78004df0 con 0x7f0d8810d4a0 2024-03-11T19:53:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.586+0000 7f0d777fe640 1 -- 172.21.15.83:0/998712401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d78005430 con 0x7f0d8810d4a0 2024-03-11T19:53:53.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.587+0000 7f0d777fe640 1 -- 172.21.15.83:0/998712401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78003e20 con 0x7f0d8810d4a0 2024-03-11T19:53:53.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.587+0000 7f0d8cf08640 1 -- 172.21.15.83:0/998712401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d881ae130 con 0x7f0d8810d4a0 2024-03-11T19:53:53.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.587+0000 7f0d8cf08640 1 -- 172.21.15.83:0/998712401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d881ae640 con 0x7f0d8810d4a0 2024-03-11T19:53:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.591+0000 7f0d8fbb0740 1 Processor -- start 2024-03-11T19:53:53.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.591+0000 7f0d777fe640 1 -- 172.21.15.83:0/998712401 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0d780035f0 con 0x7f0d8810d4a0 2024-03-11T19:53:53.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.592+0000 7f0d777fe640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d700a1bb0 0x7f0d700a4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.593+0000 7f0d777fe640 1 -- 172.21.15.83:0/998712401 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0d780bcd70 con 0x7f0d8810d4a0 2024-03-11T19:53:53.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.593+0000 7f0d8fbb0740 1 -- start start 2024-03-11T19:53:53.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.593+0000 7f0d85d9b640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d700a1bb0 0x7f0d700a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.593+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45eb08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.593+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb1690 0x563b45eb1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.593+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 0x563b45eb88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.593+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45e1ff90 con 0x563b45eb04c0 2024-03-11T19:53:53.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.593+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45e1d4c0 con 0x563b45eb1690 2024-03-11T19:53:53.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.593+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45e18ca0 con 0x563b45eb2050 2024-03-11T19:53:53.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.594+0000 7f0d86d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 0x563b45eb88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.594+0000 7f0d85d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb1690 0x563b45eb1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.594+0000 7f0d86d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 0x563b45eb88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50328/0 (socket says 172.21.15.83:50328) 2024-03-11T19:53:53.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.594+0000 7f0d86d9d640 1 -- 172.21.15.83:0/509881255 learned_addr learned my addr 172.21.15.83:0/509881255 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:53.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.595+0000 7f0d85d9b640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d700a1bb0 0x7f0d700a4070 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0d881a75a0 tx=0x7f0d7c013830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.595+0000 7f0d8659c640 1 --2- 172.21.15.83:0/509881255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45eb08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.595+0000 7f0d86d9d640 1 -- 172.21.15.83:0/509881255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb1690 msgr2=0x563b45eb1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.595+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/509881255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb1690 0x563b45eb1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.595+0000 7f0d86d9d640 1 -- 172.21.15.83:0/509881255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 msgr2=0x563b45eb08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.595+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/509881255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45eb08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.595+0000 7f0d86d9d640 1 -- 172.21.15.83:0/509881255 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b45ddb0b0 con 0x563b45eb2050 2024-03-11T19:53:53.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.595+0000 7f0d8659c640 1 --2- 172.21.15.83:0/509881255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45eb08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:53.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.595+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/509881255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 0x563b45eb88e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0d7802c240 tx=0x7f0d780c1670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=740120131bbebdba server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.595+0000 7f0d74ff9640 1 -- 172.21.15.83:0/509881255 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780bbfd0 con 0x563b45eb2050 2024-03-11T19:53:53.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.595+0000 7f0d74ff9640 1 -- 172.21.15.83:0/509881255 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d780ba7b0 con 0x563b45eb2050 2024-03-11T19:53:53.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.596+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/509881255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 msgr2=0x563b45eb88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.596+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/509881255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 0x563b45eb88e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0d7802c240 tx=0x7f0d780c1670 comp rx=0 tx=0).stop 2024-03-11T19:53:53.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.596+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/509881255 shutdown_connections 2024-03-11T19:53:53.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.596+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/509881255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 0x563b45eb88e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.596+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/509881255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb1690 0x563b45eb1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.596+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/509881255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45eb08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.596+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/509881255 >> 172.21.15.83:0/509881255 conn(0x563b45eab6a0 msgr2=0x563b45eabaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.597+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/509881255 shutdown_connections 2024-03-11T19:53:53.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.597+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/509881255 wait complete. 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.597+0000 7f0d8fbb0740 1 -- start start 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.597+0000 7f0d8fbb0740 1 Processor -- start 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.597+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45ecb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.597+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb1690 0x563b45ecb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.597+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb2050 0x563b45ed2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.597+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45dbdf40 con 0x563b45eb04c0 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.597+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45aec4b0 con 0x563b45eb2050 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.597+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45d9e670 con 0x563b45eb1690 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.597+0000 7f0d8fbb0740 10 client.0 _ll_register_callbacks cb 0x563b45e0bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d8659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45ecb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d8659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45ecb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53556/0 (socket says 172.21.15.83:53556) 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d8659c640 1 -- 172.21.15.83:0/2598192321 learned_addr learned my addr 172.21.15.83:0/2598192321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d85d9b640 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb1690 0x563b45ecb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb2050 0x563b45ed2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d85d9b640 1 -- 172.21.15.83:0/2598192321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb2050 msgr2=0x563b45ed2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d85d9b640 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb2050 0x563b45ed2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d85d9b640 1 -- 172.21.15.83:0/2598192321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 msgr2=0x563b45ecb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d85d9b640 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45ecb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d85d9b640 1 -- 172.21.15.83:0/2598192321 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d881ae640 con 0x563b45eb1690 2024-03-11T19:53:53.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d85d9b640 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb1690 0x563b45ecb8f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00a750 tx=0x7f0d7c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3586fde08ceffe99 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.598+0000 7f0d6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d6dffb640 1 -- 172.21.15.83:0/2598192321 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c010ea0 con 0x563b45eb1690 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d6dffb640 1 -- 172.21.15.83:0/2598192321 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0d7c0265f0 con 0x563b45eb1690 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/2598192321 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b45df2f40 con 0x563b45eb1690 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d6dffb640 1 -- 172.21.15.83:0/2598192321 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d7c0132f0 con 0x563b45eb1690 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d8fbb0740 10 client.27661 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/2598192321 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563b45e17320 con 0x563b45eb1690 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d6dffb640 1 -- 172.21.15.83:0/2598192321 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563b45df2f40 con 0x563b45eb1690 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d6dffb640 1 -- 172.21.15.83:0/2598192321 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0d7c031990 con 0x563b45eb1690 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d8fbb0740 10 client.27661 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d8fbb0740 10 client.27661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d8fbb0740 20 client.27661 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:53.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b45ed75a0 0x563b45ed99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d8fbb0740 4 client.27661 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.599+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/2598192321 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b45e0f310 con 0x563b45ed75a0 2024-03-11T19:53:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.600+0000 7f0d8659c640 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b45ed75a0 0x563b45ed99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.601+0000 7f0d6dffb640 10 client.27661 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:53.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.601+0000 7f0d8659c640 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b45ed75a0 0x563b45ed99e0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4365db2e9ad7b61 server_cookie=36b6e71b69229bdf in_seq=0 out_seq=0 2024-03-11T19:53:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d6dffb640 1 -- 172.21.15.83:0/2598192321 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b45e0f310 con 0x563b45ed75a0 2024-03-11T19:53:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d6dffb640 10 client.27661 handle_command_reply: tid=0 2024-03-11T19:53:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 1 client.27661 shutdown 2024-03-11T19:53:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/2598192321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb1690 msgr2=0x563b45ecb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb1690 0x563b45ecb8f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00a750 tx=0x7f0d7c004730 comp rx=0 tx=0).stop 2024-03-11T19:53:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/2598192321 shutdown_connections 2024-03-11T19:53:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b45ed75a0 0x563b45ed99e0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb2050 0x563b45ed2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb1690 0x563b45ecb8f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/2598192321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45ecb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/2598192321 >> 172.21.15.83:0/2598192321 conn(0x563b45eab6a0 msgr2=0x563b45eaa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d6dffb640 0 client.27661 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/2598192321 shutdown_connections 2024-03-11T19:53:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/2598192321 wait complete. 2024-03-11T19:53:53.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.602+0000 7f0d8fbb0740 20 client.27661 trim_cache size 0 max 16384 2024-03-11T19:53:53.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8fbb0740 1 Processor -- start 2024-03-11T19:53:53.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8fbb0740 1 -- start start 2024-03-11T19:53:53.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45eb5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb1690 0x563b45eb5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb2050 0x563b45ed28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45ea6be0 con 0x563b45eb04c0 2024-03-11T19:53:53.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45ea0350 con 0x563b45eb2050 2024-03-11T19:53:53.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45eb2450 con 0x563b45eb1690 2024-03-11T19:53:53.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45eb5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45eb5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53572/0 (socket says 172.21.15.83:53572) 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8659c640 1 -- 172.21.15.83:0/1073283051 learned_addr learned my addr 172.21.15.83:0/1073283051 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d85d9b640 1 --2- 172.21.15.83:0/1073283051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb1690 0x563b45eb5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/1073283051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb2050 0x563b45ed28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8659c640 1 -- 172.21.15.83:0/1073283051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb1690 msgr2=0x563b45eb5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8659c640 1 --2- 172.21.15.83:0/1073283051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb1690 0x563b45eb5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8659c640 1 -- 172.21.15.83:0/1073283051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb2050 msgr2=0x563b45ed28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8659c640 1 --2- 172.21.15.83:0/1073283051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb2050 0x563b45ed28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d8659c640 1 -- 172.21.15.83:0/1073283051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b45db3c00 con 0x563b45eb04c0 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.629+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/1073283051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb2050 0x563b45ed28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.630+0000 7f0d8659c640 1 --2- 172.21.15.83:0/1073283051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45eb5150 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009fd0 tx=0x7f0d800096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee107cd540a8dae6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.630+0000 7f0d6ffff640 1 -- 172.21.15.83:0/1073283051 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8001c300 con 0x563b45eb04c0 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.630+0000 7f0d6ffff640 1 -- 172.21.15.83:0/1073283051 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d80020070 con 0x563b45eb04c0 2024-03-11T19:53:53.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.630+0000 7f0d6ffff640 1 -- 172.21.15.83:0/1073283051 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80025430 con 0x563b45eb04c0 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.630+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/1073283051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 msgr2=0x563b45eb5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.630+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/1073283051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45eb5150 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009fd0 tx=0x7f0d800096d0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.630+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/1073283051 shutdown_connections 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.630+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/1073283051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb2050 0x563b45ed28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.630+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/1073283051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb1690 0x563b45eb5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.630+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/1073283051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45eb5150 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/1073283051 >> 172.21.15.83:0/1073283051 conn(0x563b45eab6a0 msgr2=0x563b45ecb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/1073283051 shutdown_connections 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/1073283051 wait complete. 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 1 -- start start 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 1 Processor -- start 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45ed10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb1690 0x563b45ed1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 0x563b45ed1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45ed2cf0 con 0x563b45eb04c0 2024-03-11T19:53:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45f2daf0 con 0x563b45eb1690 2024-03-11T19:53:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b45f2dc30 con 0x563b45eb2050 2024-03-11T19:53:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8fbb0740 10 client.0 _ll_register_callbacks cb 0x7f0d80009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45ed10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45ed10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53584/0 (socket says 172.21.15.83:53584) 2024-03-11T19:53:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d8659c640 1 -- 172.21.15.83:0/391131333 learned_addr learned my addr 172.21.15.83:0/391131333 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:53.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d85d9b640 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb1690 0x563b45ed1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.631+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 0x563b45ed1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.632+0000 7f0d8659c640 1 -- 172.21.15.83:0/391131333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 msgr2=0x563b45ed1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.632+0000 7f0d8659c640 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 0x563b45ed1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.632+0000 7f0d8659c640 1 -- 172.21.15.83:0/391131333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb1690 msgr2=0x563b45ed1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.632+0000 7f0d8659c640 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb1690 0x563b45ed1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.632+0000 7f0d8659c640 1 -- 172.21.15.83:0/391131333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d80009e40 con 0x563b45eb04c0 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.632+0000 7f0d85d9b640 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb1690 0x563b45ed1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.632+0000 7f0d86d9d640 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 0x563b45ed1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.632+0000 7f0d8659c640 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45ed10f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009bd0 tx=0x7f0d80019f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d11f3fbf72d688 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.632+0000 7f0d6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.632+0000 7f0d6dffb640 1 -- 172.21.15.83:0/391131333 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8001c500 con 0x563b45eb04c0 2024-03-11T19:53:53.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.632+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/391131333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b45e3e900 con 0x563b45eb04c0 2024-03-11T19:53:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.633+0000 7f0d6dffb640 1 -- 172.21.15.83:0/391131333 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0d8001c660 con 0x563b45eb04c0 2024-03-11T19:53:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.633+0000 7f0d6dffb640 1 -- 172.21.15.83:0/391131333 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d8001c8a0 con 0x563b45eb04c0 2024-03-11T19:53:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.633+0000 7f0d8fbb0740 10 client.18108 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.633+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/391131333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563b45dd9910 con 0x563b45eb04c0 2024-03-11T19:53:53.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.634+0000 7f0d6dffb640 1 -- 172.21.15.83:0/391131333 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d80011040 con 0x563b45eb04c0 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.634+0000 7f0d6dffb640 1 -- 172.21.15.83:0/391131333 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0d80016040 con 0x563b45eb04c0 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.634+0000 7f0d8fbb0740 10 client.18108 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.634+0000 7f0d8fbb0740 10 client.18108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.634+0000 7f0d8fbb0740 20 client.18108 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.634+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b45f2f700 0x563b45f31b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.634+0000 7f0d8fbb0740 4 client.18108 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.634+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/391131333 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563b45e3e900 con 0x563b45f2f700 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.634+0000 7f0d85d9b640 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b45f2f700 0x563b45f31b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.634+0000 7f0d6dffb640 10 client.18108 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.634+0000 7f0d85d9b640 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b45f2f700 0x563b45f31b20 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad59bd1c87b8febe server_cookie=33b1a73459eaeb00 in_seq=0 out_seq=0 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d6dffb640 1 -- 172.21.15.83:0/391131333 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563b45e3e900 con 0x563b45f2f700 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d6dffb640 10 client.18108 handle_command_reply: tid=0 2024-03-11T19:53:53.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d8fbb0740 1 client.18108 shutdown 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/391131333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 msgr2=0x563b45ed10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45ed10f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009bd0 tx=0x7f0d80019f30 comp rx=0 tx=0).stop 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/391131333 shutdown_connections 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x563b45f2f700 0x563b45f31b20 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b45eb2050 0x563b45ed1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b45eb1690 0x563b45ed1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d8fbb0740 1 --2- 172.21.15.83:0/391131333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b45eb04c0 0x563b45ed10f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/391131333 >> 172.21.15.83:0/391131333 conn(0x563b45eab6a0 msgr2=0x563b45ecb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.635+0000 7f0d6dffb640 0 client.18108 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/391131333 shutdown_connections 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8fbb0740 1 -- 172.21.15.83:0/391131333 wait complete. 2024-03-11T19:53:53.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8fbb0740 20 client.18108 trim_cache size 0 max 16384 2024-03-11T19:53:53.647 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:53:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:53:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "50c855c4-e664-46e1-8ff0-ae73bce9f896", 2024-03-11T19:53:53.647 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:53:53.647 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:53:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 -- 172.21.15.83:0/998712401 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d700a1bb0 msgr2=0x7f0d700a4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d700a1bb0 0x7f0d700a4070 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0d881a75a0 tx=0x7f0d7c013830 comp rx=0 tx=0).stop 2024-03-11T19:53:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 -- 172.21.15.83:0/998712401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8810d4a0 msgr2=0x7f0d881ad640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8810d4a0 0x7f0d881ad640 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009230 tx=0x7f0d780042e0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 -- 172.21.15.83:0/998712401 shutdown_connections 2024-03-11T19:53:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d700a1bb0 0x7f0d700a4070 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d8810d4a0 0x7f0d881ad640 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8810aaf0 0x7f0d881a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 --2- 172.21.15.83:0/998712401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d88108180 0x7f0d881a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 -- 172.21.15.83:0/998712401 >> 172.21.15.83:0/998712401 conn(0x7f0d8806dfe0 msgr2=0x7f0d8810c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 -- 172.21.15.83:0/998712401 shutdown_connections 2024-03-11T19:53:53.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.636+0000 7f0d8cf08640 1 -- 172.21.15.83:0/998712401 wait complete. 2024-03-11T19:53:53.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:53:53.655 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:53:53.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.816+0000 7f8938948640 1 -- 172.21.15.83:0/4171654025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f893410aaf0 msgr2=0x7f893410cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.817+0000 7f893159a640 1 -- 172.21.15.83:0/4171654025 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924005220 con 0x7f893410aaf0 2024-03-11T19:53:53.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.817+0000 7f8938948640 1 --2- 172.21.15.83:0/4171654025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f893410aaf0 0x7f893410cf60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8924009900 tx=0x7f892402bc80 comp rx=0 tx=0).stop 2024-03-11T19:53:53.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.819+0000 7f8938948640 1 -- 172.21.15.83:0/4171654025 shutdown_connections 2024-03-11T19:53:53.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.819+0000 7f8938948640 1 --2- 172.21.15.83:0/4171654025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f893410d4a0 0x7f893410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.819+0000 7f8938948640 1 --2- 172.21.15.83:0/4171654025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f893410aaf0 0x7f893410cf60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.819+0000 7f8938948640 1 --2- 172.21.15.83:0/4171654025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8934108180 0x7f893410a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.819+0000 7f8938948640 1 -- 172.21.15.83:0/4171654025 >> 172.21.15.83:0/4171654025 conn(0x7f893406e200 msgr2=0x7f8934070640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.825+0000 7f8938948640 1 -- 172.21.15.83:0/4171654025 shutdown_connections 2024-03-11T19:53:53.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.826+0000 7f8938948640 1 -- 172.21.15.83:0/4171654025 wait complete. 2024-03-11T19:53:53.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.827+0000 7f8938948640 1 Processor -- start 2024-03-11T19:53:53.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.827+0000 7f8938948640 1 -- start start 2024-03-11T19:53:53.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.827+0000 7f8938948640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8934108180 0x7f89341176d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.827+0000 7f8938948640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f893410aaf0 0x7f8934117c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.827+0000 7f8938948640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f893410d4a0 0x7f8934118c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.827+0000 7f8938948640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8934119200 con 0x7f893410d4a0 2024-03-11T19:53:53.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.827+0000 7f8938948640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8934119370 con 0x7f8934108180 2024-03-11T19:53:53.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.827+0000 7f8938948640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8934119530 con 0x7f893410aaf0 2024-03-11T19:53:53.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.828+0000 7f8931d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f893410aaf0 0x7f8934117c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.828+0000 7f8931d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f893410aaf0 0x7f8934117c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:50380/0 (socket says 172.21.15.83:50380) 2024-03-11T19:53:53.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.828+0000 7f8931d9b640 1 -- 172.21.15.83:0/3936396482 learned_addr learned my addr 172.21.15.83:0/3936396482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:53.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.829+0000 7f893259c640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8934108180 0x7f89341176d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.830+0000 7f8932d9d640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f893410d4a0 0x7f8934118c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.830+0000 7f893259c640 1 -- 172.21.15.83:0/3936396482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f893410aaf0 msgr2=0x7f8934117c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.830+0000 7f893259c640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f893410aaf0 0x7f8934117c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.830+0000 7f893259c640 1 -- 172.21.15.83:0/3936396482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f893410d4a0 msgr2=0x7f8934118c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.830+0000 7f893259c640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f893410d4a0 0x7f8934118c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.830+0000 7f893259c640 1 -- 172.21.15.83:0/3936396482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8924009560 con 0x7f8934108180 2024-03-11T19:53:53.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.830+0000 7f893259c640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8934108180 0x7f89341176d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f892c00be20 tx=0x7f892c009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.831+0000 7f89237fe640 1 -- 172.21.15.83:0/3936396482 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c014440 con 0x7f8934108180 2024-03-11T19:53:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.831+0000 7f8938948640 1 -- 172.21.15.83:0/3936396482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8934119810 con 0x7f8934108180 2024-03-11T19:53:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.831+0000 7f8938948640 1 -- 172.21.15.83:0/3936396482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8934119df0 con 0x7f8934108180 2024-03-11T19:53:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.831+0000 7f89237fe640 1 -- 172.21.15.83:0/3936396482 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f892c014a80 con 0x7f8934108180 2024-03-11T19:53:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.831+0000 7f89237fe640 1 -- 172.21.15.83:0/3936396482 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c012ac0 con 0x7f8934108180 2024-03-11T19:53:53.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.836+0000 7f893b5f6740 1 Processor -- start 2024-03-11T19:53:53.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.836+0000 7f893b5f6740 1 -- start start 2024-03-11T19:53:53.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.836+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 0x55d23465c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.836+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465ccc0 0x55d2346617c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.836+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d234661e30 0x55d2346642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.836+0000 7f893b5f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2345cc280 con 0x55d234661e30 2024-03-11T19:53:53.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.836+0000 7f893b5f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2345c97b0 con 0x55d23465c2d0 2024-03-11T19:53:53.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.836+0000 7f893b5f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2345c4f90 con 0x55d23465ccc0 2024-03-11T19:53:53.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.838+0000 7f8931d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d234661e30 0x55d2346642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.838+0000 7f8932d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 0x55d23465c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.838+0000 7f8931d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d234661e30 0x55d2346642c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53616/0 (socket says 172.21.15.83:53616) 2024-03-11T19:53:53.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.838+0000 7f8931d9b640 1 -- 172.21.15.83:0/2399816185 learned_addr learned my addr 172.21.15.83:0/2399816185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:53.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.839+0000 7f8931d9b640 1 -- 172.21.15.83:0/2399816185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465ccc0 msgr2=0x55d2346617c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:53:53.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.839+0000 7f8931d9b640 1 --2- 172.21.15.83:0/2399816185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465ccc0 0x55d2346617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.839+0000 7f8931d9b640 1 -- 172.21.15.83:0/2399816185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 msgr2=0x55d23465c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.839+0000 7f8931d9b640 1 --2- 172.21.15.83:0/2399816185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 0x55d23465c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.839+0000 7f8931d9b640 1 -- 172.21.15.83:0/2399816185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d234583570 con 0x55d234661e30 2024-03-11T19:53:53.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.839+0000 7f8931d9b640 1 --2- 172.21.15.83:0/2399816185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d234661e30 0x55d2346642c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8924036040 tx=0x7f892402ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e89c9ad1609deb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.839+0000 7f89237fe640 1 -- 172.21.15.83:0/3936396482 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f892c012d00 con 0x7f8934108180 2024-03-11T19:53:53.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.840+0000 7f89237fe640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f891c0a1af0 0x7f891c0a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.840+0000 7f89237fe640 1 -- 172.21.15.83:0/3936396482 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f892c0ccf20 con 0x7f8934108180 2024-03-11T19:53:53.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.840+0000 7f8920ff9640 1 -- 172.21.15.83:0/2399816185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924004b00 con 0x55d234661e30 2024-03-11T19:53:53.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.844+0000 7f8920ff9640 1 -- 172.21.15.83:0/2399816185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8924004df0 con 0x55d234661e30 2024-03-11T19:53:53.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.844+0000 7f8920ff9640 1 -- 172.21.15.83:0/2399816185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924003850 con 0x55d234661e30 2024-03-11T19:53:53.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.845+0000 7f893b5f6740 1 -- 172.21.15.83:0/2399816185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d234661e30 msgr2=0x55d2346642c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.845+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2399816185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d234661e30 0x55d2346642c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8924036040 tx=0x7f892402ca60 comp rx=0 tx=0).stop 2024-03-11T19:53:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.847+0000 7f8932d9d640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f891c0a1af0 0x7f891c0a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.848+0000 7f893b5f6740 1 -- 172.21.15.83:0/2399816185 shutdown_connections 2024-03-11T19:53:53.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.848+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2399816185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d234661e30 0x55d2346642c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.848+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2399816185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465ccc0 0x55d2346617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.848+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2399816185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 0x55d23465c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.848+0000 7f893b5f6740 1 -- 172.21.15.83:0/2399816185 >> 172.21.15.83:0/2399816185 conn(0x55d2346577f0 msgr2=0x55d234657bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.848+0000 7f8932d9d640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f891c0a1af0 0x7f891c0a3fb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55d23465d520 tx=0x7f8928008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.851+0000 7f893b5f6740 1 -- 172.21.15.83:0/2399816185 shutdown_connections 2024-03-11T19:53:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.851+0000 7f893b5f6740 1 -- 172.21.15.83:0/2399816185 wait complete. 2024-03-11T19:53:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.851+0000 7f893b5f6740 1 -- start start 2024-03-11T19:53:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.851+0000 7f893b5f6740 1 Processor -- start 2024-03-11T19:53:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465c2d0 0x55d234666300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465ccc0 0x55d234666840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d234661e30 0x55d23466d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f893b5f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234298430 con 0x55d234661e30 2024-03-11T19:53:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f893b5f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234254e00 con 0x55d23465ccc0 2024-03-11T19:53:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f893b5f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234549fd0 con 0x55d23465c2d0 2024-03-11T19:53:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f8931d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465ccc0 0x55d234666840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f8931d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465ccc0 0x55d234666840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:34984/0 (socket says 172.21.15.83:34984) 2024-03-11T19:53:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f8931d9b640 1 -- 172.21.15.83:0/2110671039 learned_addr learned my addr 172.21.15.83:0/2110671039 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f893b5f6740 10 client.0 _ll_register_callbacks cb 0x55d23459ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f893259c640 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465c2d0 0x55d234666300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f8931d9b640 1 -- 172.21.15.83:0/2110671039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465c2d0 msgr2=0x55d234666300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f8931d9b640 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465c2d0 0x55d234666300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f8931d9b640 1 -- 172.21.15.83:0/2110671039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d234661e30 msgr2=0x55d23466d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f8931d9b640 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d234661e30 0x55d23466d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.852+0000 7f8931d9b640 1 -- 172.21.15.83:0/2110671039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893410e460 con 0x55d23465ccc0 2024-03-11T19:53:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.853+0000 7f8931d9b640 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465ccc0 0x55d234666840 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8924006010 tx=0x7f8924005190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=873a5c57f4220fb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.853+0000 7f8919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:53:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.853+0000 7f8919ffb640 1 -- 172.21.15.83:0/2110671039 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924049860 con 0x55d23465ccc0 2024-03-11T19:53:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.853+0000 7f8919ffb640 1 -- 172.21.15.83:0/2110671039 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f892404f070 con 0x55d23465ccc0 2024-03-11T19:53:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.853+0000 7f893b5f6740 1 -- 172.21.15.83:0/2110671039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2342c47e0 con 0x55d23465ccc0 2024-03-11T19:53:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.854+0000 7f8919ffb640 1 -- 172.21.15.83:0/2110671039 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f892405a030 con 0x55d23465ccc0 2024-03-11T19:53:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.854+0000 7f893b5f6740 10 client.27575 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.854+0000 7f893b5f6740 1 -- 172.21.15.83:0/2110671039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d2345c3610 con 0x55d23465ccc0 2024-03-11T19:53:53.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.855+0000 7f8919ffb640 1 -- 172.21.15.83:0/2110671039 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f892404d070 con 0x55d23465ccc0 2024-03-11T19:53:53.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.856+0000 7f8919ffb640 1 -- 172.21.15.83:0/2110671039 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f892403c040 con 0x55d23465ccc0 2024-03-11T19:53:53.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.856+0000 7f893b5f6740 10 client.27575 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:53.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.856+0000 7f893b5f6740 10 client.27575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:53.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.856+0000 7f893b5f6740 20 client.27575 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:53.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.856+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d2346721e0 0x55d234674600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.856+0000 7f893b5f6740 4 client.27575 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:53:53.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.856+0000 7f893b5f6740 1 -- 172.21.15.83:0/2110671039 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2342c47e0 con 0x55d2346721e0 2024-03-11T19:53:53.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.857+0000 7f893259c640 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d2346721e0 0x55d234674600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.858+0000 7f893259c640 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d2346721e0 0x55d234674600 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a5bd3ad5bddb7d0 server_cookie=6c3c998a4c17b728 in_seq=0 out_seq=0 2024-03-11T19:53:53.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.858+0000 7f8919ffb640 10 client.27575 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f8919ffb640 1 -- 172.21.15.83:0/2110671039 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d2342c47e0 con 0x55d2346721e0 2024-03-11T19:53:53.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f8919ffb640 10 client.27575 handle_command_reply: tid=0 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f893b5f6740 1 client.27575 shutdown 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f893b5f6740 1 -- 172.21.15.83:0/2110671039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465ccc0 msgr2=0x55d234666840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465ccc0 0x55d234666840 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8924006010 tx=0x7f8924005190 comp rx=0 tx=0).stop 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f893b5f6740 1 -- 172.21.15.83:0/2110671039 shutdown_connections 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d2346721e0 0x55d234674600 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d234661e30 0x55d23466d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465ccc0 0x55d234666840 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2110671039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465c2d0 0x55d234666300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f893b5f6740 1 -- 172.21.15.83:0/2110671039 >> 172.21.15.83:0/2110671039 conn(0x55d2346577f0 msgr2=0x55d234656a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f8919ffb640 0 client.27575 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f893b5f6740 1 -- 172.21.15.83:0/2110671039 shutdown_connections 2024-03-11T19:53:53.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.859+0000 7f893b5f6740 1 -- 172.21.15.83:0/2110671039 wait complete. 2024-03-11T19:53:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.860+0000 7f893b5f6740 20 client.27575 trim_cache size 0 max 16384 2024-03-11T19:53:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f893b5f6740 1 Processor -- start 2024-03-11T19:53:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f893b5f6740 1 -- start start 2024-03-11T19:53:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 0x55d23465dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d234661e30 0x55d23465e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f893b5f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2342c47e0 con 0x55d23465ccc0 2024-03-11T19:53:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f8931d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f893b5f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234653320 con 0x55d23465c2d0 2024-03-11T19:53:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f893259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 0x55d23465dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f893b5f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d23465d0f0 con 0x55d234661e30 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f8931d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53636/0 (socket says 172.21.15.83:53636) 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f893259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 0x55d23465dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:35000/0 (socket says 172.21.15.83:35000) 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.881+0000 7f8931d9b640 1 -- 172.21.15.83:0/1917372259 learned_addr learned my addr 172.21.15.83:0/1917372259 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.882+0000 7f8932d9d640 1 --2- 172.21.15.83:0/1917372259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d234661e30 0x55d23465e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.882+0000 7f8931d9b640 1 -- 172.21.15.83:0/1917372259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d234661e30 msgr2=0x55d23465e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.882+0000 7f8931d9b640 1 --2- 172.21.15.83:0/1917372259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d234661e30 0x55d23465e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.882+0000 7f8931d9b640 1 -- 172.21.15.83:0/1917372259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 msgr2=0x55d23465dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.882+0000 7f8931d9b640 1 --2- 172.21.15.83:0/1917372259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 0x55d23465dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.882+0000 7f8931d9b640 1 -- 172.21.15.83:0/1917372259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d23455fac0 con 0x55d23465ccc0 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.882+0000 7f893259c640 1 --2- 172.21.15.83:0/1917372259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 0x55d23465dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.882+0000 7f8931d9b640 1 --2- 172.21.15.83:0/1917372259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466d840 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8924054d10 tx=0x7f8924005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ff14885b414143 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.882+0000 7f891bfff640 1 -- 172.21.15.83:0/1917372259 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892405a040 con 0x55d23465ccc0 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.882+0000 7f891bfff640 1 -- 172.21.15.83:0/1917372259 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f892404d040 con 0x55d23465ccc0 2024-03-11T19:53:53.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.882+0000 7f891bfff640 1 -- 172.21.15.83:0/1917372259 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89240346f0 con 0x55d23465ccc0 2024-03-11T19:53:53.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.883+0000 7f893b5f6740 1 -- 172.21.15.83:0/1917372259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 msgr2=0x55d23466d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.883+0000 7f893b5f6740 1 --2- 172.21.15.83:0/1917372259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466d840 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8924054d10 tx=0x7f8924005bd0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.883+0000 7f893b5f6740 1 -- 172.21.15.83:0/1917372259 shutdown_connections 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.883+0000 7f893b5f6740 1 --2- 172.21.15.83:0/1917372259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d234661e30 0x55d23465e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.883+0000 7f893b5f6740 1 --2- 172.21.15.83:0/1917372259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466d840 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.883+0000 7f893b5f6740 1 --2- 172.21.15.83:0/1917372259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d23465c2d0 0x55d23465dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.883+0000 7f893b5f6740 1 -- 172.21.15.83:0/1917372259 >> 172.21.15.83:0/1917372259 conn(0x55d2346577f0 msgr2=0x55d234666300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.883+0000 7f893b5f6740 1 -- 172.21.15.83:0/1917372259 shutdown_connections 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.883+0000 7f893b5f6740 1 -- 172.21.15.83:0/1917372259 wait complete. 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f893b5f6740 1 -- start start 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f893b5f6740 1 Processor -- start 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465c2d0 0x55d23466b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f893b5f6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234661e30 0x55d23466c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f893b5f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d23466cf10 con 0x55d23465ccc0 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f893b5f6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d23466d080 con 0x55d234661e30 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f893b5f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d23466d240 con 0x55d23465c2d0 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f8931d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f893b5f6740 10 client.0 _ll_register_callbacks cb 0x7f8924009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f8931d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53652/0 (socket says 172.21.15.83:53652) 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f8932d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234661e30 0x55d23466c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.884+0000 7f8931d9b640 1 -- 172.21.15.83:0/2663473391 learned_addr learned my addr 172.21.15.83:0/2663473391 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f893259c640 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465c2d0 0x55d23466b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f8931d9b640 1 -- 172.21.15.83:0/2663473391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465c2d0 msgr2=0x55d23466b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f8931d9b640 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465c2d0 0x55d23466b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f8931d9b640 1 -- 172.21.15.83:0/2663473391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234661e30 msgr2=0x55d23466c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f8931d9b640 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234661e30 0x55d23466c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f8931d9b640 1 -- 172.21.15.83:0/2663473391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8934119df0 con 0x55d23465ccc0 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f8931d9b640 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466b9c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8924065040 tx=0x7f8924005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d957444a7b461f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f8919ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f8919ffb640 1 -- 172.21.15.83:0/2663473391 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892405a070 con 0x55d23465ccc0 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f8919ffb640 1 -- 172.21.15.83:0/2663473391 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f892404f050 con 0x55d23465ccc0 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f893b5f6740 1 -- 172.21.15.83:0/2663473391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8924004df0 con 0x55d23465ccc0 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f8919ffb640 1 -- 172.21.15.83:0/2663473391 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8924054520 con 0x55d23465ccc0 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f893b5f6740 10 client.18120 fetch_fsmap learned FSMap version 32 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.885+0000 7f893b5f6740 1 -- 172.21.15.83:0/2663473391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d2345c3610 con 0x55d23465ccc0 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f8919ffb640 1 -- 172.21.15.83:0/2663473391 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8924078020 con 0x55d23465ccc0 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f8919ffb640 1 -- 172.21.15.83:0/2663473391 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8924049940 con 0x55d23465ccc0 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f893b5f6740 10 client.18120 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:53:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f893b5f6740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f893b5f6740 20 client.18120 populate_metadata read hostname 'smithi083' 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d2346c91d0 0x55d2346cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f893b5f6740 4 client.18120 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f893b5f6740 1 -- 172.21.15.83:0/2663473391 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d234301720 con 0x55d2346c91d0 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f893259c640 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d2346c91d0 0x55d2346cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f8919ffb640 10 client.18120 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f893259c640 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d2346c91d0 0x55d2346cb5f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=172d82447cd10eba server_cookie=7bc0d20b713a6102 in_seq=0 out_seq=0 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f8919ffb640 1 -- 172.21.15.83:0/2663473391 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d234301720 con 0x55d2346c91d0 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f8919ffb640 10 client.18120 handle_command_reply: tid=0 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.886+0000 7f893b5f6740 1 client.18120 shutdown 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f893b5f6740 1 -- 172.21.15.83:0/2663473391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 msgr2=0x55d23466b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466b9c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8924065040 tx=0x7f8924005c90 comp rx=0 tx=0).stop 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f893b5f6740 1 -- 172.21.15.83:0/2663473391 shutdown_connections 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55d2346c91d0 0x55d2346cb5f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f8919ffb640 0 client.18120 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:53:53.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d234661e30 0x55d23466c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d23465ccc0 0x55d23466b9c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f893b5f6740 1 --2- 172.21.15.83:0/2663473391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d23465c2d0 0x55d23466b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f893b5f6740 1 -- 172.21.15.83:0/2663473391 >> 172.21.15.83:0/2663473391 conn(0x55d2346577f0 msgr2=0x55d2346662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f893b5f6740 1 -- 172.21.15.83:0/2663473391 shutdown_connections 2024-03-11T19:53:53.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f893b5f6740 1 -- 172.21.15.83:0/2663473391 wait complete. 2024-03-11T19:53:53.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f893b5f6740 20 client.18120 trim_cache size 0 max 16384 2024-03-11T19:53:53.897 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:53:53.897 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-11T19:53:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:53:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "50c855c4-e664-46e1-8ff0-ae73bce9f896": { 2024-03-11T19:53:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:53:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "50c855c4-e664-46e1-8ff0-ae73bce9f896", 2024-03-11T19:53:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:53:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:53:53.898 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:53:53.898 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:53:53.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f8938948640 1 -- 172.21.15.83:0/3936396482 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f891c0a1af0 msgr2=0x7f891c0a3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f8938948640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f891c0a1af0 0x7f891c0a3fb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55d23465d520 tx=0x7f8928008040 comp rx=0 tx=0).stop 2024-03-11T19:53:53.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f8938948640 1 -- 172.21.15.83:0/3936396482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8934108180 msgr2=0x7f89341176d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:53:53.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f8938948640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8934108180 0x7f89341176d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f892c00be20 tx=0x7f892c009f70 comp rx=0 tx=0).stop 2024-03-11T19:53:53.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f8938948640 1 -- 172.21.15.83:0/3936396482 shutdown_connections 2024-03-11T19:53:53.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f8938948640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f891c0a1af0 0x7f891c0a3fb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.887+0000 7f8938948640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f893410d4a0 0x7f8934118c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.888+0000 7f8938948640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f893410aaf0 0x7f8934117c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.888+0000 7f8938948640 1 --2- 172.21.15.83:0/3936396482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8934108180 0x7f89341176d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:53:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.888+0000 7f8938948640 1 -- 172.21.15.83:0/3936396482 >> 172.21.15.83:0/3936396482 conn(0x7f893406e200 msgr2=0x7f893410b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:53:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.890+0000 7f8938948640 1 -- 172.21.15.83:0/3936396482 shutdown_connections 2024-03-11T19:53:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:53:53.890+0000 7f8938948640 1 -- 172.21.15.83:0/3936396482 wait complete. 2024-03-11T19:53:53.906 INFO:tasks.cephfs.filesystem:scrub status for tag:50c855c4-e664-46e1-8ff0-ae73bce9f896 - {'path': '/', 'tag': '50c855c4-e664-46e1-8ff0-ae73bce9f896', 'options': 'recursive,force'} 2024-03-11T19:53:54.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:54 smithi204 ceph-mon[36739]: pgmap v789: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 596 op/s 2024-03-11T19:53:54.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:54 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:53:54.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:54 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:54.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:54 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:53:54.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:54 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:53:54.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:54 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:54.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:54 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:53:54.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:54 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:54.907 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:54 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:54 smithi083 ceph-mon[31864]: pgmap v789: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 596 op/s 2024-03-11T19:53:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:54 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:53:55.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:54 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:55.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:54 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:53:55.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:54 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:53:55.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:54 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:55.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:54 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:53:55.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:54 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:55.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:54 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:54 smithi137 ceph-mon[36220]: pgmap v789: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 596 op/s 2024-03-11T19:53:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:54 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:53:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:54 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:54 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:53:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:54 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:53:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:54 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:54 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:53:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:54 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:55.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:54 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:53:56.906 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:56 smithi204 ceph-mon[36739]: pgmap v790: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.3 MiB/s wr, 714 op/s 2024-03-11T19:53:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:56 smithi083 ceph-mon[31864]: pgmap v790: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.3 MiB/s wr, 714 op/s 2024-03-11T19:53:57.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:56 smithi137 ceph-mon[36220]: pgmap v790: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.3 MiB/s wr, 714 op/s 2024-03-11T19:53:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 880.0 s, 1538.5 tps, lat 62.431 ms stddev 72.994 2024-03-11T19:53:59.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:53:58 smithi083 ceph-mon[31864]: pgmap v791: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 631 op/s 2024-03-11T19:53:59.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:53:58 smithi137 ceph-mon[36220]: pgmap v791: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 631 op/s 2024-03-11T19:53:59.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:53:58 smithi204 ceph-mon[36739]: pgmap v791: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 631 op/s 2024-03-11T19:54:00.942 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:00 smithi137 ceph-mon[36220]: pgmap v792: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.0 MiB/s wr, 646 op/s 2024-03-11T19:54:00.942 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:00 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:54:00.958 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:00 smithi204 ceph-mon[36739]: pgmap v792: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.0 MiB/s wr, 646 op/s 2024-03-11T19:54:00.961 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:00 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:54:01.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:00 smithi083 ceph-mon[31864]: pgmap v792: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.0 MiB/s wr, 646 op/s 2024-03-11T19:54:01.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:00 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:54:02.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:01 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:54:02.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:01 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:54:02.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:01 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:54:02.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:01 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:54:02.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:01 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:54:02.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:01 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:54:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 885.0 s, 1595.2 tps, lat 64.511 ms stddev 77.353 2024-03-11T19:54:03.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:02 smithi083 ceph-mon[31864]: pgmap v793: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.8 MiB/s wr, 597 op/s 2024-03-11T19:54:03.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:02 smithi137 ceph-mon[36220]: pgmap v793: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.8 MiB/s wr, 597 op/s 2024-03-11T19:54:03.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:02 smithi204 ceph-mon[36739]: pgmap v793: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.8 MiB/s wr, 597 op/s 2024-03-11T19:54:05.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:04 smithi083 ceph-mon[31864]: pgmap v794: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.1 MiB/s wr, 560 op/s 2024-03-11T19:54:05.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:04 smithi137 ceph-mon[36220]: pgmap v794: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.1 MiB/s wr, 560 op/s 2024-03-11T19:54:05.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:04 smithi204 ceph-mon[36739]: pgmap v794: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.1 MiB/s wr, 560 op/s 2024-03-11T19:54:07.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:06 smithi083 ceph-mon[31864]: pgmap v795: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.7 MiB/s wr, 716 op/s 2024-03-11T19:54:07.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:06 smithi137 ceph-mon[36220]: pgmap v795: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.7 MiB/s wr, 716 op/s 2024-03-11T19:54:07.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:06 smithi204 ceph-mon[36739]: pgmap v795: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.7 MiB/s wr, 716 op/s 2024-03-11T19:54:07.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 890.0 s, 1418.0 tps, lat 71.310 ms stddev 73.802 2024-03-11T19:54:09.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:08 smithi083 ceph-mon[31864]: pgmap v796: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.7 MiB/s wr, 617 op/s 2024-03-11T19:54:09.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:08 smithi137 ceph-mon[36220]: pgmap v796: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.7 MiB/s wr, 617 op/s 2024-03-11T19:54:09.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:08 smithi204 ceph-mon[36739]: pgmap v796: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.7 MiB/s wr, 617 op/s 2024-03-11T19:54:11.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:10 smithi083 ceph-mon[31864]: pgmap v797: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.7 MiB/s wr, 568 op/s 2024-03-11T19:54:11.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:10 smithi137 ceph-mon[36220]: pgmap v797: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.7 MiB/s wr, 568 op/s 2024-03-11T19:54:11.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:10 smithi204 ceph-mon[36739]: pgmap v797: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.7 MiB/s wr, 568 op/s 2024-03-11T19:54:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 895.0 s, 1417.4 tps, lat 70.024 ms stddev 73.133 2024-03-11T19:54:13.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:12 smithi083 ceph-mon[31864]: pgmap v798: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.9 MiB/s wr, 668 op/s 2024-03-11T19:54:13.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:12 smithi137 ceph-mon[36220]: pgmap v798: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.9 MiB/s wr, 668 op/s 2024-03-11T19:54:13.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:12 smithi204 ceph-mon[36739]: pgmap v798: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.9 MiB/s wr, 668 op/s 2024-03-11T19:54:15.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:14 smithi083 ceph-mon[31864]: pgmap v799: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.1 MiB/s wr, 564 op/s 2024-03-11T19:54:15.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:14 smithi137 ceph-mon[36220]: pgmap v799: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.1 MiB/s wr, 564 op/s 2024-03-11T19:54:15.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:14 smithi204 ceph-mon[36739]: pgmap v799: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.1 MiB/s wr, 564 op/s 2024-03-11T19:54:17.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:16 smithi083 ceph-mon[31864]: pgmap v800: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 680 op/s 2024-03-11T19:54:17.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:16 smithi137 ceph-mon[36220]: pgmap v800: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 680 op/s 2024-03-11T19:54:17.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:16 smithi204 ceph-mon[36739]: pgmap v800: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 680 op/s 2024-03-11T19:54:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:progress: 900.0 s, 2355.6 tps, lat 42.792 ms stddev 55.982 2024-03-11T19:54:17.577 INFO:teuthology.orchestra.run.smithi083.stdout:transaction type: 2024-03-11T19:54:17.578 INFO:teuthology.orchestra.run.smithi083.stdout:scaling factor: 32 2024-03-11T19:54:17.578 INFO:teuthology.orchestra.run.smithi083.stdout:query mode: simple 2024-03-11T19:54:17.578 INFO:teuthology.orchestra.run.smithi083.stdout:number of clients: 100 2024-03-11T19:54:17.578 INFO:teuthology.orchestra.run.smithi083.stdout:number of threads: 4 2024-03-11T19:54:17.578 INFO:teuthology.orchestra.run.smithi083.stdout:duration: 900 s 2024-03-11T19:54:17.578 INFO:teuthology.orchestra.run.smithi083.stdout:number of transactions actually processed: 2744661 2024-03-11T19:54:17.578 INFO:teuthology.orchestra.run.smithi083.stdout:latency average = 32.787 ms 2024-03-11T19:54:17.578 INFO:teuthology.orchestra.run.smithi083.stdout:latency stddev = 44.242 ms 2024-03-11T19:54:17.578 INFO:teuthology.orchestra.run.smithi083.stdout:tps = 3049.354395 (including connections establishing) 2024-03-11T19:54:17.578 INFO:teuthology.orchestra.run.smithi083.stdout:tps = 3049.362089 (excluding connections establishing) 2024-03-11T19:54:17.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-03-11T19:54:19.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:18 smithi083 ceph-mon[31864]: pgmap v801: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-03-11T19:54:19.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:18 smithi137 ceph-mon[36220]: pgmap v801: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-03-11T19:54:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:18 smithi204 ceph-mon[36739]: pgmap v801: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 619 op/s 2024-03-11T19:54:21.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:20 smithi083 ceph-mon[31864]: pgmap v802: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 662 op/s 2024-03-11T19:54:21.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:20 smithi137 ceph-mon[36220]: pgmap v802: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 662 op/s 2024-03-11T19:54:21.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:20 smithi204 ceph-mon[36739]: pgmap v802: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 662 op/s 2024-03-11T19:54:23.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:22 smithi083 ceph-mon[31864]: pgmap v803: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 815 op/s 2024-03-11T19:54:23.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:22 smithi137 ceph-mon[36220]: pgmap v803: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 815 op/s 2024-03-11T19:54:23.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:22 smithi204 ceph-mon[36739]: pgmap v803: 97 pgs: 97 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 MiB/s wr, 815 op/s 2024-03-11T19:54:23.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:54:23.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:54:24.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.065+0000 7f6a40c40640 1 -- 172.21.15.83:0/3337359634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a3c101270 msgr2=0x7f6a3c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.065+0000 7f6a40c40640 1 --2- 172.21.15.83:0/3337359634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a3c101270 0x7f6a3c1016f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009860 tx=0x7f6a3402ffa0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.065+0000 7f6a40c40640 1 -- 172.21.15.83:0/3337359634 shutdown_connections 2024-03-11T19:54:24.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.065+0000 7f6a40c40640 1 --2- 172.21.15.83:0/3337359634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a3c101c30 0x7f6a3c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.065+0000 7f6a40c40640 1 --2- 172.21.15.83:0/3337359634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a3c101270 0x7f6a3c1016f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.065+0000 7f6a40c40640 1 --2- 172.21.15.83:0/3337359634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c100080 0x7f6a3c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.065+0000 7f6a40c40640 1 -- 172.21.15.83:0/3337359634 >> 172.21.15.83:0/3337359634 conn(0x7f6a3c0fb5f0 msgr2=0x7f6a3c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.066+0000 7f6a40c40640 1 -- 172.21.15.83:0/3337359634 shutdown_connections 2024-03-11T19:54:24.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.066+0000 7f6a40c40640 1 -- 172.21.15.83:0/3337359634 wait complete. 2024-03-11T19:54:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.067+0000 7f6a40c40640 1 Processor -- start 2024-03-11T19:54:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.067+0000 7f6a40c40640 1 -- start start 2024-03-11T19:54:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.068+0000 7f6a40c40640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a3c100080 0x7f6a3c1074d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.068+0000 7f6a40c40640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c101270 0x7f6a3c107a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.068+0000 7f6a40c40640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a3c101c30 0x7f6a3c1b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.068+0000 7f6a40c40640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c1b62e0 con 0x7f6a3c101c30 2024-03-11T19:54:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.068+0000 7f6a40c40640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c1b6450 con 0x7f6a3c101270 2024-03-11T19:54:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.068+0000 7f6a40c40640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c1b6610 con 0x7f6a3c100080 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.068+0000 7f6a3a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a3c100080 0x7f6a3c1074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.068+0000 7f6a3a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a3c100080 0x7f6a3c1074d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41338/0 (socket says 172.21.15.83:41338) 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.068+0000 7f6a3a59c640 1 -- 172.21.15.83:0/3428733751 learned_addr learned my addr 172.21.15.83:0/3428733751 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.068+0000 7f6a3ad9d640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a3c101c30 0x7f6a3c1b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.069+0000 7f6a3a59c640 1 -- 172.21.15.83:0/3428733751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c101270 msgr2=0x7f6a3c107a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.069+0000 7f6a3a59c640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c101270 0x7f6a3c107a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.069+0000 7f6a3a59c640 1 -- 172.21.15.83:0/3428733751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a3c101c30 msgr2=0x7f6a3c1b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.069+0000 7f6a3a59c640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a3c101c30 0x7f6a3c1b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.069+0000 7f6a3a59c640 1 -- 172.21.15.83:0/3428733751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a34009490 con 0x7f6a3c100080 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.069+0000 7f6a3a59c640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a3c100080 0x7f6a3c1074d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c01adb0 tx=0x7f6a2c01b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.069+0000 7f6a2b7fe640 1 -- 172.21.15.83:0/3428733751 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c003970 con 0x7f6a3c100080 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.069+0000 7f6a40c40640 1 -- 172.21.15.83:0/3428733751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c1b68f0 con 0x7f6a3c100080 2024-03-11T19:54:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.069+0000 7f6a40c40640 1 -- 172.21.15.83:0/3428733751 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c1b6e50 con 0x7f6a3c100080 2024-03-11T19:54:24.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.069+0000 7f6a2b7fe640 1 -- 172.21.15.83:0/3428733751 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a2c004020 con 0x7f6a3c100080 2024-03-11T19:54:24.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.069+0000 7f6a2b7fe640 1 -- 172.21.15.83:0/3428733751 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00f800 con 0x7f6a3c100080 2024-03-11T19:54:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.072+0000 7f6a2b7fe640 1 -- 172.21.15.83:0/3428733751 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6a2c00f960 con 0x7f6a3c100080 2024-03-11T19:54:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.073+0000 7f6a2b7fe640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a240a1ae0 0x7f6a240a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.073+0000 7f6a43814740 1 Processor -- start 2024-03-11T19:54:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.073+0000 7f6a2b7fe640 1 -- 172.21.15.83:0/3428733751 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6a2c0cd8a0 con 0x7f6a3c100080 2024-03-11T19:54:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a43814740 1 -- start start 2024-03-11T19:54:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a39d9b640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a240a1ae0 0x7f6a240a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a43814740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c8312d0 0x55ec3c8316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a43814740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c831cc0 0x55ec3c8367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a43814740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 0x55ec3c8392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a43814740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c7a1280 con 0x55ec3c836e30 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a43814740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c79e7b0 con 0x55ec3c8312d0 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a43814740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c799f90 con 0x55ec3c831cc0 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a3a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c8312d0 0x55ec3c8316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a3a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c8312d0 0x55ec3c8316f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:43116/0 (socket says 172.21.15.83:43116) 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a3a59c640 1 -- 172.21.15.83:0/3478519078 learned_addr learned my addr 172.21.15.83:0/3478519078 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.074+0000 7f6a39d9b640 1 --2- 172.21.15.83:0/3478519078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c831cc0 0x55ec3c8367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.075+0000 7f6a3ad9d640 1 --2- 172.21.15.83:0/3478519078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 0x55ec3c8392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.075+0000 7f6a39d9b640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a240a1ae0 0x7f6a240a3fa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6a34039fd0 tx=0x7f6a34036440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.075+0000 7f6a39d9b640 1 -- 172.21.15.83:0/3478519078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c8312d0 msgr2=0x55ec3c8316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.075+0000 7f6a39d9b640 1 --2- 172.21.15.83:0/3478519078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c8312d0 0x55ec3c8316f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.075+0000 7f6a39d9b640 1 -- 172.21.15.83:0/3478519078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 msgr2=0x55ec3c8392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.075+0000 7f6a39d9b640 1 --2- 172.21.15.83:0/3478519078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 0x55ec3c8392c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.075+0000 7f6a39d9b640 1 -- 172.21.15.83:0/3478519078 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3c758570 con 0x55ec3c831cc0 2024-03-11T19:54:24.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.075+0000 7f6a39d9b640 1 --2- 172.21.15.83:0/3478519078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c831cc0 0x55ec3c8367c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009830 tx=0x7f6a34048000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d5c680945f6a48 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.075+0000 7f6a28ff9640 1 -- 172.21.15.83:0/3478519078 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34004240 con 0x55ec3c831cc0 2024-03-11T19:54:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.075+0000 7f6a28ff9640 1 -- 172.21.15.83:0/3478519078 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a340367e0 con 0x55ec3c831cc0 2024-03-11T19:54:24.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.075+0000 7f6a28ff9640 1 -- 172.21.15.83:0/3478519078 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34004cc0 con 0x55ec3c831cc0 2024-03-11T19:54:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.076+0000 7f6a43814740 1 -- 172.21.15.83:0/3478519078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c831cc0 msgr2=0x55ec3c8367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.076+0000 7f6a43814740 1 --2- 172.21.15.83:0/3478519078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c831cc0 0x55ec3c8367c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009830 tx=0x7f6a34048000 comp rx=0 tx=0).stop 2024-03-11T19:54:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.076+0000 7f6a43814740 1 -- 172.21.15.83:0/3478519078 shutdown_connections 2024-03-11T19:54:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.076+0000 7f6a43814740 1 --2- 172.21.15.83:0/3478519078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 0x55ec3c8392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.076+0000 7f6a43814740 1 --2- 172.21.15.83:0/3478519078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c831cc0 0x55ec3c8367c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.076+0000 7f6a43814740 1 --2- 172.21.15.83:0/3478519078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c8312d0 0x55ec3c8316f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.076+0000 7f6a43814740 1 -- 172.21.15.83:0/3478519078 >> 172.21.15.83:0/3478519078 conn(0x55ec3c82c7f0 msgr2=0x55ec3c82cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.077+0000 7f6a43814740 1 -- 172.21.15.83:0/3478519078 shutdown_connections 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.077+0000 7f6a43814740 1 -- 172.21.15.83:0/3478519078 wait complete. 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.077+0000 7f6a43814740 1 -- start start 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.077+0000 7f6a43814740 1 Processor -- start 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.077+0000 7f6a43814740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8312d0 0x55ec3c83a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.077+0000 7f6a43814740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c831cc0 0x55ec3c83ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a43814740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c836e30 0x55ec3c841e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a43814740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c46d430 con 0x55ec3c8312d0 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a43814740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c429e00 con 0x55ec3c831cc0 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a43814740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c71efd0 con 0x55ec3c836e30 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a43814740 10 client.0 _ll_register_callbacks cb 0x55ec3c774fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:24.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a3a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8312d0 0x55ec3c83a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a3a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8312d0 0x55ec3c83a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38578/0 (socket says 172.21.15.83:38578) 2024-03-11T19:54:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a3a59c640 1 -- 172.21.15.83:0/247573539 learned_addr learned my addr 172.21.15.83:0/247573539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a39d9b640 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c831cc0 0x55ec3c83ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a3ad9d640 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c836e30 0x55ec3c841e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a39d9b640 1 -- 172.21.15.83:0/247573539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c836e30 msgr2=0x55ec3c841e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a39d9b640 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c836e30 0x55ec3c841e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a39d9b640 1 -- 172.21.15.83:0/247573539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8312d0 msgr2=0x55ec3c83a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.078+0000 7f6a39d9b640 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8312d0 0x55ec3c83a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.079+0000 7f6a39d9b640 1 -- 172.21.15.83:0/247573539 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c108dc0 con 0x55ec3c831cc0 2024-03-11T19:54:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.079+0000 7f6a39d9b640 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c831cc0 0x55ec3c83ae30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6a340304e0 tx=0x7f6a34004cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e5a022fd5249941 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.079+0000 7f6a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:54:24.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.084+0000 7f6a21ffb640 1 -- 172.21.15.83:0/247573539 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34004240 con 0x55ec3c831cc0 2024-03-11T19:54:24.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.084+0000 7f6a43814740 1 -- 172.21.15.83:0/247573539 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec3c4d6720 con 0x55ec3c831cc0 2024-03-11T19:54:24.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.084+0000 7f6a21ffb640 1 -- 172.21.15.83:0/247573539 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6a340043a0 con 0x55ec3c831cc0 2024-03-11T19:54:24.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.085+0000 7f6a21ffb640 1 -- 172.21.15.83:0/247573539 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a340046c0 con 0x55ec3c831cc0 2024-03-11T19:54:24.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.085+0000 7f6a43814740 10 client.27590 fetch_fsmap learned FSMap version 32 2024-03-11T19:54:24.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.085+0000 7f6a43814740 1 -- 172.21.15.83:0/247573539 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ec3c798610 con 0x55ec3c831cc0 2024-03-11T19:54:24.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.086+0000 7f6a21ffb640 1 -- 172.21.15.83:0/247573539 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a34066080 con 0x55ec3c831cc0 2024-03-11T19:54:24.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.086+0000 7f6a21ffb640 1 -- 172.21.15.83:0/247573539 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6a34042040 con 0x55ec3c831cc0 2024-03-11T19:54:24.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.086+0000 7f6a43814740 10 client.27590 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:54:24.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.086+0000 7f6a43814740 10 client.27590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:24.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.086+0000 7f6a43814740 20 client.27590 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:24.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.086+0000 7f6a43814740 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ec3c847120 0x55ec3c849540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.086+0000 7f6a43814740 4 client.27590 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:24.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.086+0000 7f6a43814740 1 -- 172.21.15.83:0/247573539 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec3c525890 con 0x55ec3c847120 2024-03-11T19:54:24.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.086+0000 7f6a3a59c640 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ec3c847120 0x55ec3c849540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.087+0000 7f6a21ffb640 10 client.27590 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:24.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.087+0000 7f6a3a59c640 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ec3c847120 0x55ec3c849540 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b40a0f7730e923e server_cookie=1caf2deeef24f329 in_seq=0 out_seq=0 2024-03-11T19:54:24.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a21ffb640 1 -- 172.21.15.83:0/247573539 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec3c525890 con 0x55ec3c847120 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a21ffb640 10 client.27590 handle_command_reply: tid=0 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 1 client.27590 shutdown 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 1 -- 172.21.15.83:0/247573539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c831cc0 msgr2=0x55ec3c83ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c831cc0 0x55ec3c83ae30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6a340304e0 tx=0x7f6a34004cd0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 1 -- 172.21.15.83:0/247573539 shutdown_connections 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ec3c847120 0x55ec3c849540 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c836e30 0x55ec3c841e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c831cc0 0x55ec3c83ae30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 1 --2- 172.21.15.83:0/247573539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8312d0 0x55ec3c83a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 1 -- 172.21.15.83:0/247573539 >> 172.21.15.83:0/247573539 conn(0x55ec3c82c7f0 msgr2=0x55ec3c82ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a21ffb640 0 client.27590 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 1 -- 172.21.15.83:0/247573539 shutdown_connections 2024-03-11T19:54:24.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 1 -- 172.21.15.83:0/247573539 wait complete. 2024-03-11T19:54:24.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.088+0000 7f6a43814740 20 client.27590 trim_cache size 0 max 16384 2024-03-11T19:54:24.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.117+0000 7f6a43814740 1 Processor -- start 2024-03-11T19:54:24.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.118+0000 7f6a43814740 1 -- start start 2024-03-11T19:54:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.118+0000 7f6a43814740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c8312d0 0x55ec3c832f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.118+0000 7f6a43814740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c831cc0 0x55ec3c841e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.118+0000 7f6a43814740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 0x55ec3c833440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.118+0000 7f6a43814740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c839980 con 0x55ec3c836e30 2024-03-11T19:54:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.118+0000 7f6a43814740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c8323b0 con 0x55ec3c8312d0 2024-03-11T19:54:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.118+0000 7f6a43814740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c525890 con 0x55ec3c831cc0 2024-03-11T19:54:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.119+0000 7f6a3ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 0x55ec3c833440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.119+0000 7f6a3ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 0x55ec3c833440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38592/0 (socket says 172.21.15.83:38592) 2024-03-11T19:54:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.119+0000 7f6a3ad9d640 1 -- 172.21.15.83:0/4037678226 learned_addr learned my addr 172.21.15.83:0/4037678226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.119+0000 7f6a3a59c640 1 --2- 172.21.15.83:0/4037678226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c8312d0 0x55ec3c832f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.120+0000 7f6a39d9b640 1 --2- 172.21.15.83:0/4037678226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c831cc0 0x55ec3c841e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.120+0000 7f6a3ad9d640 1 -- 172.21.15.83:0/4037678226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c831cc0 msgr2=0x55ec3c841e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.120+0000 7f6a3ad9d640 1 --2- 172.21.15.83:0/4037678226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c831cc0 0x55ec3c841e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.120+0000 7f6a3ad9d640 1 -- 172.21.15.83:0/4037678226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c8312d0 msgr2=0x55ec3c832f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.120+0000 7f6a3ad9d640 1 --2- 172.21.15.83:0/4037678226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c8312d0 0x55ec3c832f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.120+0000 7f6a3ad9d640 1 -- 172.21.15.83:0/4037678226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3c734ac0 con 0x55ec3c836e30 2024-03-11T19:54:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.120+0000 7f6a3ad9d640 1 --2- 172.21.15.83:0/4037678226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 0x55ec3c833440 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000da10 tx=0x7f6a30005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdebe955be9b96ed server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.121+0000 7f6a23fff640 1 -- 172.21.15.83:0/4037678226 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3001d070 con 0x55ec3c836e30 2024-03-11T19:54:24.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.121+0000 7f6a23fff640 1 -- 172.21.15.83:0/4037678226 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a30009990 con 0x55ec3c836e30 2024-03-11T19:54:24.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.122+0000 7f6a43814740 1 -- 172.21.15.83:0/4037678226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 msgr2=0x55ec3c833440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.122+0000 7f6a43814740 1 --2- 172.21.15.83:0/4037678226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 0x55ec3c833440 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000da10 tx=0x7f6a30005f90 comp rx=0 tx=0).stop 2024-03-11T19:54:24.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.122+0000 7f6a43814740 1 -- 172.21.15.83:0/4037678226 shutdown_connections 2024-03-11T19:54:24.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.122+0000 7f6a43814740 1 --2- 172.21.15.83:0/4037678226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c836e30 0x55ec3c833440 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.122+0000 7f6a43814740 1 --2- 172.21.15.83:0/4037678226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c831cc0 0x55ec3c841e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.122+0000 7f6a43814740 1 --2- 172.21.15.83:0/4037678226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c8312d0 0x55ec3c832f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.122+0000 7f6a43814740 1 -- 172.21.15.83:0/4037678226 >> 172.21.15.83:0/4037678226 conn(0x55ec3c82c7f0 msgr2=0x55ec3c83a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.122+0000 7f6a43814740 1 -- 172.21.15.83:0/4037678226 shutdown_connections 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.122+0000 7f6a43814740 1 -- 172.21.15.83:0/4037678226 wait complete. 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.122+0000 7f6a43814740 1 -- start start 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.122+0000 7f6a43814740 1 Processor -- start 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.123+0000 7f6a43814740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c8312d0 0x55ec3c83f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.123+0000 7f6a43814740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c831cc0 0x55ec3c83fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.123+0000 7f6a43814740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8409d0 0x55ec3c840e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.123+0000 7f6a43814740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c841390 con 0x55ec3c8409d0 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.123+0000 7f6a43814740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c841500 con 0x55ec3c831cc0 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.123+0000 7f6a43814740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3c8416c0 con 0x55ec3c8312d0 2024-03-11T19:54:24.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.123+0000 7f6a43814740 10 client.0 _ll_register_callbacks cb 0x7f6a3403b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:24.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.123+0000 7f6a3ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8409d0 0x55ec3c840e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.123+0000 7f6a3ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8409d0 0x55ec3c840e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38594/0 (socket says 172.21.15.83:38594) 2024-03-11T19:54:24.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.124+0000 7f6a3ad9d640 1 -- 172.21.15.83:0/1738036523 learned_addr learned my addr 172.21.15.83:0/1738036523 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.124+0000 7f6a3ad9d640 1 -- 172.21.15.83:0/1738036523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c8312d0 msgr2=0x55ec3c83f4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.124+0000 7f6a3ad9d640 1 --2- 172.21.15.83:0/1738036523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c8312d0 0x55ec3c83f4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.124+0000 7f6a3ad9d640 1 -- 172.21.15.83:0/1738036523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c831cc0 msgr2=0x55ec3c83fa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.124+0000 7f6a3ad9d640 1 --2- 172.21.15.83:0/1738036523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c831cc0 0x55ec3c83fa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.124+0000 7f6a3ad9d640 1 -- 172.21.15.83:0/1738036523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3000d6f0 con 0x55ec3c8409d0 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.124+0000 7f6a3ad9d640 1 --2- 172.21.15.83:0/1738036523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8409d0 0x55ec3c840e50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009bf0 tx=0x7f6a30009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503b01c3e2f7278d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.124+0000 7f6a21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.125+0000 7f6a21ffb640 1 -- 172.21.15.83:0/1738036523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3001d070 con 0x55ec3c8409d0 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.125+0000 7f6a21ffb640 1 -- 172.21.15.83:0/1738036523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6a30013070 con 0x55ec3c8409d0 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.125+0000 7f6a43814740 1 -- 172.21.15.83:0/1738036523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec3c4884e0 con 0x55ec3c8409d0 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.125+0000 7f6a21ffb640 1 -- 172.21.15.83:0/1738036523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a300035f0 con 0x55ec3c8409d0 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.125+0000 7f6a43814740 10 client.18138 fetch_fsmap learned FSMap version 32 2024-03-11T19:54:24.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.125+0000 7f6a43814740 1 -- 172.21.15.83:0/1738036523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ec3c75a6a0 con 0x55ec3c8409d0 2024-03-11T19:54:24.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.126+0000 7f6a21ffb640 1 -- 172.21.15.83:0/1738036523 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a30030030 con 0x55ec3c8409d0 2024-03-11T19:54:24.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.126+0000 7f6a21ffb640 1 -- 172.21.15.83:0/1738036523 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6a30018070 con 0x55ec3c8409d0 2024-03-11T19:54:24.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.126+0000 7f6a43814740 10 client.18138 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:54:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.126+0000 7f6a43814740 10 client.18138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.126+0000 7f6a43814740 20 client.18138 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:24.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.126+0000 7f6a43814740 1 --2- 172.21.15.83:0/1738036523 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ec3c8c1110 0x55ec3c8c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.126+0000 7f6a43814740 4 client.18138 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:54:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.126+0000 7f6a43814740 1 -- 172.21.15.83:0/1738036523 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ec3c445700 con 0x55ec3c8c1110 2024-03-11T19:54:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.126+0000 7f6a3a59c640 1 --2- 172.21.15.83:0/1738036523 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ec3c8c1110 0x55ec3c8c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.127+0000 7f6a21ffb640 10 client.18138 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:24.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.127+0000 7f6a3a59c640 1 --2- 172.21.15.83:0/1738036523 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ec3c8c1110 0x55ec3c8c3530 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d3dffd74dd7ec80 server_cookie=7f3be5a36dd4d684 in_seq=0 out_seq=0 2024-03-11T19:54:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a21ffb640 1 -- 172.21.15.83:0/1738036523 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ec3c445700 con 0x55ec3c8c1110 2024-03-11T19:54:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a21ffb640 10 client.18138 handle_command_reply: tid=0 2024-03-11T19:54:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a43814740 1 client.18138 shutdown 2024-03-11T19:54:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a43814740 1 -- 172.21.15.83:0/1738036523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8409d0 msgr2=0x55ec3c840e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a43814740 1 --2- 172.21.15.83:0/1738036523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8409d0 0x55ec3c840e50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009bf0 tx=0x7f6a30009c20 comp rx=0 tx=0).stop 2024-03-11T19:54:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a43814740 1 -- 172.21.15.83:0/1738036523 shutdown_connections 2024-03-11T19:54:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a43814740 1 --2- 172.21.15.83:0/1738036523 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55ec3c8c1110 0x55ec3c8c3530 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a43814740 1 --2- 172.21.15.83:0/1738036523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec3c8409d0 0x55ec3c840e50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a43814740 1 --2- 172.21.15.83:0/1738036523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ec3c831cc0 0x55ec3c83fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a43814740 1 --2- 172.21.15.83:0/1738036523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ec3c8312d0 0x55ec3c83f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a43814740 1 -- 172.21.15.83:0/1738036523 >> 172.21.15.83:0/1738036523 conn(0x55ec3c82c7f0 msgr2=0x55ec3c832f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a21ffb640 0 client.18138 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a43814740 1 -- 172.21.15.83:0/1738036523 shutdown_connections 2024-03-11T19:54:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.132+0000 7f6a43814740 1 -- 172.21.15.83:0/1738036523 wait complete. 2024-03-11T19:54:24.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.134+0000 7f6a43814740 20 client.18138 trim_cache size 0 max 16384 2024-03-11T19:54:24.140 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:54:24.140 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:54:24.140 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:54:24.140 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:54:24.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.135+0000 7f6a40c40640 1 -- 172.21.15.83:0/3428733751 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a240a1ae0 msgr2=0x7f6a240a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.135+0000 7f6a40c40640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a240a1ae0 0x7f6a240a3fa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6a34039fd0 tx=0x7f6a34036440 comp rx=0 tx=0).stop 2024-03-11T19:54:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.135+0000 7f6a40c40640 1 -- 172.21.15.83:0/3428733751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a3c100080 msgr2=0x7f6a3c1074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.135+0000 7f6a40c40640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a3c100080 0x7f6a3c1074d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c01adb0 tx=0x7f6a2c01b6d0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.135+0000 7f6a40c40640 1 -- 172.21.15.83:0/3428733751 shutdown_connections 2024-03-11T19:54:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.135+0000 7f6a40c40640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6a240a1ae0 0x7f6a240a3fa0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.135+0000 7f6a40c40640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a3c101c30 0x7f6a3c1b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.135+0000 7f6a40c40640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c101270 0x7f6a3c107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.135+0000 7f6a40c40640 1 --2- 172.21.15.83:0/3428733751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a3c100080 0x7f6a3c1074d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.135+0000 7f6a40c40640 1 -- 172.21.15.83:0/3428733751 >> 172.21.15.83:0/3428733751 conn(0x7f6a3c0fb5f0 msgr2=0x7f6a3c06f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.136+0000 7f6a40c40640 1 -- 172.21.15.83:0/3428733751 shutdown_connections 2024-03-11T19:54:24.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.136+0000 7f6a40c40640 1 -- 172.21.15.83:0/3428733751 wait complete. 2024-03-11T19:54:24.155 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:54:24.156 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:54:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.291+0000 7f0dac4ec640 1 -- 172.21.15.83:0/680180330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0da41000a0 msgr2=0x7f0da41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.291+0000 7f0dac4ec640 1 --2- 172.21.15.83:0/680180330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0da41000a0 0x7f0da41004c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009900 tx=0x7f0da002bc30 comp rx=0 tx=0).stop 2024-03-11T19:54:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.291+0000 7f0dac4ec640 1 -- 172.21.15.83:0/680180330 shutdown_connections 2024-03-11T19:54:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.291+0000 7f0dac4ec640 1 --2- 172.21.15.83:0/680180330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da4101c50 0x7f0da41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.291+0000 7f0dac4ec640 1 --2- 172.21.15.83:0/680180330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da4101290 0x7f0da4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.291+0000 7f0dac4ec640 1 --2- 172.21.15.83:0/680180330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0da41000a0 0x7f0da41004c0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.291+0000 7f0dac4ec640 1 -- 172.21.15.83:0/680180330 >> 172.21.15.83:0/680180330 conn(0x7f0da40fb610 msgr2=0x7f0da40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.291+0000 7f0dac4ec640 1 -- 172.21.15.83:0/680180330 shutdown_connections 2024-03-11T19:54:24.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.291+0000 7f0dac4ec640 1 -- 172.21.15.83:0/680180330 wait complete. 2024-03-11T19:54:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.292+0000 7f0dac4ec640 1 Processor -- start 2024-03-11T19:54:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.292+0000 7f0dac4ec640 1 -- start start 2024-03-11T19:54:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.292+0000 7f0dac4ec640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da41000a0 0x7f0da4197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.292+0000 7f0dac4ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da4101290 0x7f0da4197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.292+0000 7f0dac4ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0da4101c50 0x7f0da419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.292+0000 7f0dac4ec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da410a450 con 0x7f0da4101c50 2024-03-11T19:54:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.292+0000 7f0dac4ec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da410a5c0 con 0x7f0da4101290 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.292+0000 7f0dac4ec640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da410a780 con 0x7f0da41000a0 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0daaa89640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0da4101c50 0x7f0da419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0daa288640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da41000a0 0x7f0da4197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0daa288640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da41000a0 0x7f0da4197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41416/0 (socket says 172.21.15.83:41416) 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0daaa89640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0da4101c50 0x7f0da419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38616/0 (socket says 172.21.15.83:38616) 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0da9a87640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da4101290 0x7f0da4197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0daa288640 1 -- 172.21.15.83:0/1336480896 learned_addr learned my addr 172.21.15.83:0/1336480896 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0da9a87640 1 -- 172.21.15.83:0/1336480896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da41000a0 msgr2=0x7f0da4197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0da9a87640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da41000a0 0x7f0da4197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0da9a87640 1 -- 172.21.15.83:0/1336480896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0da4101c50 msgr2=0x7f0da419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0da9a87640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0da4101c50 0x7f0da419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0da9a87640 1 -- 172.21.15.83:0/1336480896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0009560 con 0x7f0da4101290 2024-03-11T19:54:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0daa288640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da41000a0 0x7f0da4197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.293+0000 7f0da9a87640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da4101290 0x7f0da4197770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400e9f0 tx=0x7f0d9400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.294+0000 7f0d9b7fe640 1 -- 172.21.15.83:0/1336480896 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9400ccc0 con 0x7f0da4101290 2024-03-11T19:54:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.294+0000 7f0dac4ec640 1 -- 172.21.15.83:0/1336480896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da410aa60 con 0x7f0da4101290 2024-03-11T19:54:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.294+0000 7f0dac4ec640 1 -- 172.21.15.83:0/1336480896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da410afc0 con 0x7f0da4101290 2024-03-11T19:54:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.294+0000 7f0d9b7fe640 1 -- 172.21.15.83:0/1336480896 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d9400ce20 con 0x7f0da4101290 2024-03-11T19:54:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.294+0000 7f0d9b7fe640 1 -- 172.21.15.83:0/1336480896 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94018920 con 0x7f0da4101290 2024-03-11T19:54:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.295+0000 7f0dac4ec640 1 -- 172.21.15.83:0/1336480896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da410c420 con 0x7f0da4101290 2024-03-11T19:54:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.297+0000 7f0d9b7fe640 1 -- 172.21.15.83:0/1336480896 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0d94018a80 con 0x7f0da4101290 2024-03-11T19:54:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.297+0000 7f0d9b7fe640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d900a1a30 0x7f0d900a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.298+0000 7f0d9b7fe640 1 -- 172.21.15.83:0/1336480896 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0d94014070 con 0x7f0da4101290 2024-03-11T19:54:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.298+0000 7f0daa288640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d900a1a30 0x7f0d900a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.298+0000 7f0daa288640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d900a1a30 0x7f0d900a3ef0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0da002c680 tx=0x7f0da002c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.304+0000 7f0d9b7fe640 1 -- 172.21.15.83:0/1336480896 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d94096970 con 0x7f0da4101290 2024-03-11T19:54:24.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.615+0000 7f0dac4ec640 1 -- 172.21.15.83:0/1336480896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0da4106e80 con 0x7f0da4101290 2024-03-11T19:54:24.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.616+0000 7f0d9b7fe640 1 -- 172.21.15.83:0/1336480896 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11349 (secure 0 0 0) 0x7f0d9409a030 con 0x7f0da4101290 2024-03-11T19:54:24.620 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:24.620 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:47:49.158462+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:54:24.621 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-03-11T19:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 -- 172.21.15.83:0/1336480896 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d900a1a30 msgr2=0x7f0d900a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d900a1a30 0x7f0d900a3ef0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0da002c680 tx=0x7f0da002c7f0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 -- 172.21.15.83:0/1336480896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da4101290 msgr2=0x7f0da4197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da4101290 0x7f0da4197770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400e9f0 tx=0x7f0d9400eeb0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 -- 172.21.15.83:0/1336480896 shutdown_connections 2024-03-11T19:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f0d900a1a30 0x7f0d900a3ef0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0da4101c50 0x7f0da419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0da4101290 0x7f0da4197770 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 --2- 172.21.15.83:0/1336480896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0da41000a0 0x7f0da4197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 -- 172.21.15.83:0/1336480896 >> 172.21.15.83:0/1336480896 conn(0x7f0da40fb610 msgr2=0x7f0da41045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 -- 172.21.15.83:0/1336480896 shutdown_connections 2024-03-11T19:54:24.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.618+0000 7f0dac4ec640 1 -- 172.21.15.83:0/1336480896 wait complete. 2024-03-11T19:54:24.635 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:54:24.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:54:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.753+0000 7f4dce839640 1 -- 172.21.15.83:0/3937969671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc8100080 msgr2=0x7f4dc81004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.753+0000 7f4dce839640 1 --2- 172.21.15.83:0/3937969671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc8100080 0x7f4dc81004a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009900 tx=0x7f4db802bc30 comp rx=0 tx=0).stop 2024-03-11T19:54:24.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.753+0000 7f4dce839640 1 -- 172.21.15.83:0/3937969671 shutdown_connections 2024-03-11T19:54:24.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.754+0000 7f4dce839640 1 --2- 172.21.15.83:0/3937969671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc8101c30 0x7f4dc81084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.754+0000 7f4dce839640 1 --2- 172.21.15.83:0/3937969671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4dc8101270 0x7f4dc81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.754+0000 7f4dce839640 1 --2- 172.21.15.83:0/3937969671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc8100080 0x7f4dc81004a0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.754+0000 7f4dce839640 1 -- 172.21.15.83:0/3937969671 >> 172.21.15.83:0/3937969671 conn(0x7f4dc80fb610 msgr2=0x7f4dc80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.754+0000 7f4dce839640 1 -- 172.21.15.83:0/3937969671 shutdown_connections 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.754+0000 7f4dce839640 1 -- 172.21.15.83:0/3937969671 wait complete. 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.754+0000 7f4dce839640 1 Processor -- start 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.754+0000 7f4dce839640 1 -- start start 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dce839640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc8100080 0x7f4dc8197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dce839640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc8101270 0x7f4dc8197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dce839640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4dc8101c30 0x7f4dc819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dce839640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc819ee10 con 0x7f4dc8101270 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dce839640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc819ef80 con 0x7f4dc8100080 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dce839640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc819f140 con 0x7f4dc8101c30 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc8101270 0x7f4dc8197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc8101270 0x7f4dc8197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38624/0 (socket says 172.21.15.83:38624) 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dc77fe640 1 -- 172.21.15.83:0/1537907883 learned_addr learned my addr 172.21.15.83:0/1537907883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dc7fff640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc8100080 0x7f4dc8197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dccdd6640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4dc8101c30 0x7f4dc819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dc77fe640 1 -- 172.21.15.83:0/1537907883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4dc8101c30 msgr2=0x7f4dc819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dc77fe640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4dc8101c30 0x7f4dc819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dc77fe640 1 -- 172.21.15.83:0/1537907883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc8100080 msgr2=0x7f4dc8197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dc77fe640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc8100080 0x7f4dc8197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dc77fe640 1 -- 172.21.15.83:0/1537907883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8009560 con 0x7f4dc8101270 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.755+0000 7f4dc77fe640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc8101270 0x7f4dc8197860 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00e9f0 tx=0x7f4dbc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.756+0000 7f4dc57fa640 1 -- 172.21.15.83:0/1537907883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc009720 con 0x7f4dc8101270 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.756+0000 7f4dce839640 1 -- 172.21.15.83:0/1537907883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc819f420 con 0x7f4dc8101270 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.756+0000 7f4dc57fa640 1 -- 172.21.15.83:0/1537907883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dbc00cd80 con 0x7f4dc8101270 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.756+0000 7f4dc57fa640 1 -- 172.21.15.83:0/1537907883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc010690 con 0x7f4dc8101270 2024-03-11T19:54:24.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.756+0000 7f4dce839640 1 -- 172.21.15.83:0/1537907883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc819f960 con 0x7f4dc8101270 2024-03-11T19:54:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.758+0000 7f4dc57fa640 1 -- 172.21.15.83:0/1537907883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f4dbc0046c0 con 0x7f4dc8101270 2024-03-11T19:54:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dd1407740 1 Processor -- start 2024-03-11T19:54:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dd1407740 1 -- start start 2024-03-11T19:54:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00270 0x55fd40e00690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e05760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e05dd0 0x55fd40e08260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dd1407740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd40dbc960 con 0x55fd40e00270 2024-03-11T19:54:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dd1407740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd40d70310 con 0x55fd40e05dd0 2024-03-11T19:54:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dd1407740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd40d6d840 con 0x55fd40e00c60 2024-03-11T19:54:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dccdd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00270 0x55fd40e00690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dc7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e05760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dc7fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e05760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41448/0 (socket says 172.21.15.83:41448) 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dccdd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00270 0x55fd40e00690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38636/0 (socket says 172.21.15.83:38636) 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dc7fff640 1 -- 172.21.15.83:0/430658510 learned_addr learned my addr 172.21.15.83:0/430658510 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dc57fa640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4db00a1a30 0x7f4db00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dc77fe640 1 --2- 172.21.15.83:0/430658510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e05dd0 0x55fd40e08260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.759+0000 7f4dccdd6640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4db00a1a30 0x7f4db00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.760+0000 7f4dc57fa640 1 -- 172.21.15.83:0/1537907883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4dbc014070 con 0x7f4dc8101270 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.760+0000 7f4dccdd6640 1 -- 172.21.15.83:0/430658510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 msgr2=0x55fd40e05760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.760+0000 7f4dccdd6640 1 --2- 172.21.15.83:0/430658510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e05760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.760+0000 7f4dccdd6640 1 -- 172.21.15.83:0/430658510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e05dd0 msgr2=0x55fd40e08260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.760+0000 7f4dccdd6640 1 --2- 172.21.15.83:0/430658510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e05dd0 0x55fd40e08260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.760+0000 7f4dccdd6640 1 -- 172.21.15.83:0/430658510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd40d2b060 con 0x55fd40e00270 2024-03-11T19:54:24.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.760+0000 7f4dc7fff640 1 --2- 172.21.15.83:0/430658510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e05760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.760+0000 7f4dccdd6640 1 --2- 172.21.15.83:0/430658510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00270 0x55fd40e00690 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000a8b0 tx=0x7f4dc000ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed7caa217127e3c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.760+0000 7f4dae7fc640 1 -- 172.21.15.83:0/430658510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc001ac40 con 0x55fd40e00270 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.760+0000 7f4dae7fc640 1 -- 172.21.15.83:0/430658510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dc0015ec0 con 0x55fd40e00270 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.760+0000 7f4dae7fc640 1 -- 172.21.15.83:0/430658510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc002aef0 con 0x55fd40e00270 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.761+0000 7f4dd1407740 1 -- 172.21.15.83:0/430658510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00270 msgr2=0x55fd40e00690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.761+0000 7f4dccdd6640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4db00a1a30 0x7f4db00a3ef0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0024000 tx=0x7f4dc0020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.761+0000 7f4dd1407740 1 --2- 172.21.15.83:0/430658510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00270 0x55fd40e00690 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000a8b0 tx=0x7f4dc000ad70 comp rx=0 tx=0).stop 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.761+0000 7f4dd1407740 1 -- 172.21.15.83:0/430658510 shutdown_connections 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.761+0000 7f4dd1407740 1 --2- 172.21.15.83:0/430658510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e05dd0 0x55fd40e08260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.761+0000 7f4dd1407740 1 --2- 172.21.15.83:0/430658510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e05760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.761+0000 7f4dd1407740 1 --2- 172.21.15.83:0/430658510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00270 0x55fd40e00690 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.761+0000 7f4dd1407740 1 -- 172.21.15.83:0/430658510 >> 172.21.15.83:0/430658510 conn(0x55fd40dfb7e0 msgr2=0x55fd40dfbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.761+0000 7f4dd1407740 1 -- 172.21.15.83:0/430658510 shutdown_connections 2024-03-11T19:54:24.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.761+0000 7f4dd1407740 1 -- 172.21.15.83:0/430658510 wait complete. 2024-03-11T19:54:24.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dd1407740 1 -- start start 2024-03-11T19:54:24.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dd1407740 1 Processor -- start 2024-03-11T19:54:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e00270 0x55fd40e0a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e0aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e11a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dd1407740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd40d69020 con 0x55fd40e05dd0 2024-03-11T19:54:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dd1407740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd40a3c430 con 0x55fd40e00270 2024-03-11T19:54:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dd1407740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd409f8e00 con 0x55fd40e00c60 2024-03-11T19:54:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dd1407740 10 client.0 _ll_register_callbacks cb 0x55fd40d10cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dccdd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e11a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e0aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dccdd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e11a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38642/0 (socket says 172.21.15.83:38642) 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e0aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41454/0 (socket says 172.21.15.83:41454) 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dccdd6640 1 -- 172.21.15.83:0/1235246732 learned_addr learned my addr 172.21.15.83:0/1235246732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.762+0000 7f4dc7fff640 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e00270 0x55fd40e0a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.763+0000 7f4dc7fff640 1 -- 172.21.15.83:0/1235246732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 msgr2=0x55fd40e0aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.763+0000 7f4dc7fff640 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e0aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.763+0000 7f4dc7fff640 1 -- 172.21.15.83:0/1235246732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 msgr2=0x55fd40e11a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.763+0000 7f4dc7fff640 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e11a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.763+0000 7f4dc7fff640 1 -- 172.21.15.83:0/1235246732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc0013040 con 0x55fd40e00270 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.763+0000 7f4dccdd6640 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e11a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.763+0000 7f4dc7fff640 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e00270 0x55fd40e0a4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55fd40e016d0 tx=0x7f4db80048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=267cf28f903e36c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.763+0000 7f4da77fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.763+0000 7f4da77fe640 1 -- 172.21.15.83:0/1235246732 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8004250 con 0x55fd40e00270 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.763+0000 7f4dd1407740 1 -- 172.21.15.83:0/1235246732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd40aa5720 con 0x55fd40e00270 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.763+0000 7f4da77fe640 1 -- 172.21.15.83:0/1235246732 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4db80043b0 con 0x55fd40e00270 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4da77fe640 1 -- 172.21.15.83:0/1235246732 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4db8004df0 con 0x55fd40e00270 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4dd1407740 10 client.27608 fetch_fsmap learned FSMap version 32 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4dd1407740 1 -- 172.21.15.83:0/1235246732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fd40d676a0 con 0x55fd40e00270 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4da77fe640 1 -- 172.21.15.83:0/1235246732 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fd40aa5720 con 0x55fd40e00270 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4da77fe640 1 -- 172.21.15.83:0/1235246732 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4db803c040 con 0x55fd40e00270 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4dd1407740 10 client.27608 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4dd1407740 10 client.27608 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4dd1407740 20 client.27608 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd40e16650 0x55fd40e18a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4dd1407740 4 client.27608 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4dd1407740 1 -- 172.21.15.83:0/1235246732 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd40af4890 con 0x55fd40e16650 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.764+0000 7f4dc77fe640 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd40e16650 0x55fd40e18a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.765+0000 7f4da77fe640 10 client.27608 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.765+0000 7f4dc77fe640 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd40e16650 0x55fd40e18a60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba9eca07b34b4aee server_cookie=76633a71ff98fca2 in_seq=0 out_seq=0 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4da77fe640 1 -- 172.21.15.83:0/1235246732 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd40af4890 con 0x55fd40e16650 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4da77fe640 10 client.27608 handle_command_reply: tid=0 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 1 client.27608 shutdown 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 1 -- 172.21.15.83:0/1235246732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e00270 msgr2=0x55fd40e0a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e00270 0x55fd40e0a4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55fd40e016d0 tx=0x7f4db80048f0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 1 -- 172.21.15.83:0/1235246732 shutdown_connections 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd40e16650 0x55fd40e18a60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e11a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e0aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1235246732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e00270 0x55fd40e0a4c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4da77fe640 0 client.27608 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:54:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 1 -- 172.21.15.83:0/1235246732 >> 172.21.15.83:0/1235246732 conn(0x55fd40dfb7e0 msgr2=0x55fd40dfab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 1 -- 172.21.15.83:0/1235246732 shutdown_connections 2024-03-11T19:54:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 1 -- 172.21.15.83:0/1235246732 wait complete. 2024-03-11T19:54:24.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.766+0000 7f4dd1407740 20 client.27608 trim_cache size 0 max 16384 2024-03-11T19:54:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.779+0000 7f4dd1407740 1 Processor -- start 2024-03-11T19:54:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.779+0000 7f4dd1407740 1 -- start start 2024-03-11T19:54:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.779+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00270 0x55fd40e021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.779+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00c60 0x55fd40e11a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e05dd0 0x55fd40e02700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dd1407740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd40cee130 con 0x55fd40e00c60 2024-03-11T19:54:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dd1407740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd40e01480 con 0x55fd40e05dd0 2024-03-11T19:54:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dd1407740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd40df72b0 con 0x55fd40e00270 2024-03-11T19:54:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00c60 0x55fd40e11a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00c60 0x55fd40e11a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38650/0 (socket says 172.21.15.83:38650) 2024-03-11T19:54:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dc77fe640 1 -- 172.21.15.83:0/2362989384 learned_addr learned my addr 172.21.15.83:0/2362989384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dc7fff640 1 --2- 172.21.15.83:0/2362989384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00270 0x55fd40e021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dccdd6640 1 --2- 172.21.15.83:0/2362989384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e05dd0 0x55fd40e02700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dc77fe640 1 -- 172.21.15.83:0/2362989384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00270 msgr2=0x55fd40e021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dc77fe640 1 --2- 172.21.15.83:0/2362989384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00270 0x55fd40e021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dc77fe640 1 -- 172.21.15.83:0/2362989384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e05dd0 msgr2=0x55fd40e02700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dc77fe640 1 --2- 172.21.15.83:0/2362989384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e05dd0 0x55fd40e02700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dc77fe640 1 -- 172.21.15.83:0/2362989384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd40d03c20 con 0x55fd40e00c60 2024-03-11T19:54:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dc77fe640 1 --2- 172.21.15.83:0/2362989384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00c60 0x55fd40e11a00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00d010 tx=0x7f4dbc0cb140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3940cbfcb51e2c29 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dadffb640 1 -- 172.21.15.83:0/2362989384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc0a2e80 con 0x55fd40e00c60 2024-03-11T19:54:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dadffb640 1 -- 172.21.15.83:0/2362989384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dbc09a6e0 con 0x55fd40e00c60 2024-03-11T19:54:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.780+0000 7f4dadffb640 1 -- 172.21.15.83:0/2362989384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc09e3a0 con 0x55fd40e00c60 2024-03-11T19:54:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.781+0000 7f4dd1407740 1 -- 172.21.15.83:0/2362989384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00c60 msgr2=0x55fd40e11a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.781+0000 7f4dd1407740 1 --2- 172.21.15.83:0/2362989384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00c60 0x55fd40e11a00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00d010 tx=0x7f4dbc0cb140 comp rx=0 tx=0).stop 2024-03-11T19:54:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.781+0000 7f4dd1407740 1 -- 172.21.15.83:0/2362989384 shutdown_connections 2024-03-11T19:54:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.781+0000 7f4dd1407740 1 --2- 172.21.15.83:0/2362989384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e05dd0 0x55fd40e02700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.781+0000 7f4dd1407740 1 --2- 172.21.15.83:0/2362989384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e00c60 0x55fd40e11a00 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.781+0000 7f4dd1407740 1 --2- 172.21.15.83:0/2362989384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00270 0x55fd40e021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.781+0000 7f4dd1407740 1 -- 172.21.15.83:0/2362989384 >> 172.21.15.83:0/2362989384 conn(0x55fd40dfb7e0 msgr2=0x55fd40df6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.781+0000 7f4dd1407740 1 -- 172.21.15.83:0/2362989384 shutdown_connections 2024-03-11T19:54:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.781+0000 7f4dd1407740 1 -- 172.21.15.83:0/2362989384 wait complete. 2024-03-11T19:54:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dd1407740 1 -- start start 2024-03-11T19:54:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dd1407740 1 Processor -- start 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e00270 0x55fd40e11420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e11960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dd1407740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e0fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dd1407740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd40e10190 con 0x55fd40e05dd0 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dd1407740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd40e10300 con 0x55fd40e00270 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dd1407740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd40e104c0 con 0x55fd40e00c60 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dd1407740 10 client.0 _ll_register_callbacks cb 0x7f4dc00099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dccdd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e0fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e11960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dccdd6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e0fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38654/0 (socket says 172.21.15.83:38654) 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dccdd6640 1 -- 172.21.15.83:0/1088094018 learned_addr learned my addr 172.21.15.83:0/1088094018 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.782+0000 7f4dc7fff640 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e00270 0x55fd40e11420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4dccdd6640 1 -- 172.21.15.83:0/1088094018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 msgr2=0x55fd40e11960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4dccdd6640 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e11960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4dccdd6640 1 -- 172.21.15.83:0/1088094018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e00270 msgr2=0x55fd40e11420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4dccdd6640 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e00270 0x55fd40e11420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4dccdd6640 1 -- 172.21.15.83:0/1088094018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc819f960 con 0x55fd40e05dd0 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4dc77fe640 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e11960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4dccdd6640 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e0fa90 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00070b0 tx=0x7f4dc002aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190ee4f1dd79318 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4da77fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4da77fe640 1 -- 172.21.15.83:0/1088094018 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0036670 con 0x55fd40e05dd0 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4da77fe640 1 -- 172.21.15.83:0/1088094018 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f4dc0030040 con 0x55fd40e05dd0 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4dd1407740 1 -- 172.21.15.83:0/1088094018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd40a687e0 con 0x55fd40e05dd0 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4da77fe640 1 -- 172.21.15.83:0/1088094018 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dc002aed0 con 0x55fd40e05dd0 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4dd1407740 10 client.18162 fetch_fsmap learned FSMap version 32 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4dd1407740 1 -- 172.21.15.83:0/1088094018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55fd40d298c0 con 0x55fd40e05dd0 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.783+0000 7f4da77fe640 1 -- 172.21.15.83:0/1088094018 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dc0045020 con 0x55fd40e05dd0 2024-03-11T19:54:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.784+0000 7f4da77fe640 1 -- 172.21.15.83:0/1088094018 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4dc0034050 con 0x55fd40e05dd0 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.784+0000 7f4dd1407740 10 client.18162 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.784+0000 7f4dd1407740 10 client.18162 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.784+0000 7f4dd1407740 20 client.18162 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.784+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd40e6e060 0x55fd40e70480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.784+0000 7f4dd1407740 4 client.18162 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.784+0000 7f4dd1407740 1 -- 172.21.15.83:0/1088094018 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd40a14700 con 0x55fd40e6e060 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.784+0000 7f4dc7fff640 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd40e6e060 0x55fd40e70480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4dc7fff640 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd40e6e060 0x55fd40e70480 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8dcc53cc8c1b363 server_cookie=981ea04f24e56d10 in_seq=0 out_seq=0 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4da77fe640 10 client.18162 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4da77fe640 1 -- 172.21.15.83:0/1088094018 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd40a14700 con 0x55fd40e6e060 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4da77fe640 10 client.18162 handle_command_reply: tid=0 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4dd1407740 1 client.18162 shutdown 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4dd1407740 1 -- 172.21.15.83:0/1088094018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 msgr2=0x55fd40e0fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e0fa90 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00070b0 tx=0x7f4dc002aa80 comp rx=0 tx=0).stop 2024-03-11T19:54:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4dd1407740 1 -- 172.21.15.83:0/1088094018 shutdown_connections 2024-03-11T19:54:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x55fd40e6e060 0x55fd40e70480 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd40e05dd0 0x55fd40e0fa90 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd40e00c60 0x55fd40e11960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4dd1407740 1 --2- 172.21.15.83:0/1088094018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd40e00270 0x55fd40e11420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4dd1407740 1 -- 172.21.15.83:0/1088094018 >> 172.21.15.83:0/1088094018 conn(0x55fd40dfb7e0 msgr2=0x55fd40e021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.785+0000 7f4da77fe640 0 client.18162 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:54:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.786+0000 7f4dd1407740 1 -- 172.21.15.83:0/1088094018 shutdown_connections 2024-03-11T19:54:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.786+0000 7f4dd1407740 1 -- 172.21.15.83:0/1088094018 wait complete. 2024-03-11T19:54:24.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.786+0000 7f4dd1407740 20 client.18162 trim_cache size 0 max 16384 2024-03-11T19:54:24.795 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:54:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.786+0000 7f4dce839640 1 -- 172.21.15.83:0/1537907883 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4db00a1a30 msgr2=0x7f4db00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.786+0000 7f4dce839640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4db00a1a30 0x7f4db00a3ef0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0024000 tx=0x7f4dc0020000 comp rx=0 tx=0).stop 2024-03-11T19:54:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.786+0000 7f4dce839640 1 -- 172.21.15.83:0/1537907883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc8101270 msgr2=0x7f4dc8197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.786+0000 7f4dce839640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc8101270 0x7f4dc8197860 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00e9f0 tx=0x7f4dbc00edf0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.787+0000 7f4dce839640 1 -- 172.21.15.83:0/1537907883 shutdown_connections 2024-03-11T19:54:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.787+0000 7f4dce839640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f4db00a1a30 0x7f4db00a3ef0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.787+0000 7f4dce839640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4dc8101c30 0x7f4dc819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.787+0000 7f4dce839640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dc8101270 0x7f4dc8197860 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.787+0000 7f4dce839640 1 --2- 172.21.15.83:0/1537907883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc8100080 0x7f4dc8197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.787+0000 7f4dce839640 1 -- 172.21.15.83:0/1537907883 >> 172.21.15.83:0/1537907883 conn(0x7f4dc80fb610 msgr2=0x7f4dc80fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.787+0000 7f4dce839640 1 -- 172.21.15.83:0/1537907883 shutdown_connections 2024-03-11T19:54:24.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.787+0000 7f4dce839640 1 -- 172.21.15.83:0/1537907883 wait complete. 2024-03-11T19:54:24.803 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:54:24.803 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:54:24.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.916+0000 7fac67b79640 1 -- 172.21.15.83:0/1994771819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac60102d40 msgr2=0x7fac601095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.916+0000 7fac67b79640 1 --2- 172.21.15.83:0/1994771819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac60102d40 0x7fac601095d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fac58009ca0 tx=0x7fac5802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.916+0000 7fac67b79640 1 -- 172.21.15.83:0/1994771819 shutdown_connections 2024-03-11T19:54:24.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.916+0000 7fac67b79640 1 --2- 172.21.15.83:0/1994771819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac60102d40 0x7fac601095d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.916+0000 7fac67b79640 1 --2- 172.21.15.83:0/1994771819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac60102380 0x7fac60102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.916+0000 7fac67b79640 1 --2- 172.21.15.83:0/1994771819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac60101190 0x7fac601015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.916+0000 7fac67b79640 1 -- 172.21.15.83:0/1994771819 >> 172.21.15.83:0/1994771819 conn(0x7fac600fc6e0 msgr2=0x7fac600feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.916+0000 7fac67b79640 1 -- 172.21.15.83:0/1994771819 shutdown_connections 2024-03-11T19:54:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.916+0000 7fac67b79640 1 -- 172.21.15.83:0/1994771819 wait complete. 2024-03-11T19:54:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac67b79640 1 Processor -- start 2024-03-11T19:54:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac67b79640 1 -- start start 2024-03-11T19:54:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac67b79640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac60101190 0x7fac60074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac67b79640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac60102380 0x7fac60073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac67b79640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac60102d40 0x7fac60073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac67b79640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac60074210 con 0x7fac60102380 2024-03-11T19:54:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac67b79640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac60074380 con 0x7fac60101190 2024-03-11T19:54:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac67b79640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac601a1220 con 0x7fac60102d40 2024-03-11T19:54:24.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac65114640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac60102380 0x7fac60073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac65114640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac60102380 0x7fac60073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38680/0 (socket says 172.21.15.83:38680) 2024-03-11T19:54:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac65114640 1 -- 172.21.15.83:0/382677981 learned_addr learned my addr 172.21.15.83:0/382677981 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac66116640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac60102d40 0x7fac60073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac65915640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac60101190 0x7fac60074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac65114640 1 -- 172.21.15.83:0/382677981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac60102d40 msgr2=0x7fac60073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac65114640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac60102d40 0x7fac60073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac65114640 1 -- 172.21.15.83:0/382677981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac60101190 msgr2=0x7fac60074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac65114640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac60101190 0x7fac60074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.917+0000 7fac65114640 1 -- 172.21.15.83:0/382677981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac58009900 con 0x7fac60102380 2024-03-11T19:54:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.918+0000 7fac65114640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac60102380 0x7fac60073570 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fac5000ea20 tx=0x7fac5000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.918+0000 7fac56ffd640 1 -- 172.21.15.83:0/382677981 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50009720 con 0x7fac60102380 2024-03-11T19:54:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.918+0000 7fac67b79640 1 -- 172.21.15.83:0/382677981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac601a14d0 con 0x7fac60102380 2024-03-11T19:54:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.918+0000 7fac67b79640 1 -- 172.21.15.83:0/382677981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac601a19e0 con 0x7fac60102380 2024-03-11T19:54:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.918+0000 7fac56ffd640 1 -- 172.21.15.83:0/382677981 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac5000cd80 con 0x7fac60102380 2024-03-11T19:54:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.918+0000 7fac56ffd640 1 -- 172.21.15.83:0/382677981 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50010490 con 0x7fac60102380 2024-03-11T19:54:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.920+0000 7fac56ffd640 1 -- 172.21.15.83:0/382677981 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fac500046c0 con 0x7fac60102380 2024-03-11T19:54:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.921+0000 7fac56ffd640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fac4c0a1ab0 0x7fac4c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.921+0000 7fac56ffd640 1 -- 172.21.15.83:0/382677981 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fac50014070 con 0x7fac60102380 2024-03-11T19:54:24.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.921+0000 7fac65915640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fac4c0a1ab0 0x7fac4c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.921+0000 7fac6a81f740 1 Processor -- start 2024-03-11T19:54:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65915640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fac4c0a1ab0 0x7fac4c0a3f70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fac5c005fd0 tx=0x7fac5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac6a81f740 1 -- start start 2024-03-11T19:54:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d72e270 0x557f5d72e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d733760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d733dd0 0x557f5d736260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac6a81f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d6ea960 con 0x557f5d72ec60 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac6a81f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d69e310 con 0x557f5d72e270 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac6a81f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d69b840 con 0x557f5d733dd0 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65114640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d733760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65114640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d733760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38688/0 (socket says 172.21.15.83:38688) 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65114640 1 -- 172.21.15.83:0/2457287359 learned_addr learned my addr 172.21.15.83:0/2457287359 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65915640 1 --2- 172.21.15.83:0/2457287359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d72e270 0x557f5d72e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac66116640 1 --2- 172.21.15.83:0/2457287359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d733dd0 0x557f5d736260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65114640 1 -- 172.21.15.83:0/2457287359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d733dd0 msgr2=0x557f5d736260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65114640 1 --2- 172.21.15.83:0/2457287359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d733dd0 0x557f5d736260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65114640 1 -- 172.21.15.83:0/2457287359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d72e270 msgr2=0x557f5d72e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65114640 1 --2- 172.21.15.83:0/2457287359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d72e270 0x557f5d72e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65114640 1 -- 172.21.15.83:0/2457287359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f5d659060 con 0x557f5d72ec60 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac66116640 1 --2- 172.21.15.83:0/2457287359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d733dd0 0x557f5d736260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65915640 1 --2- 172.21.15.83:0/2457287359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d72e270 0x557f5d72e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.922+0000 7fac65114640 1 --2- 172.21.15.83:0/2457287359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d733760 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fac5009a4e0 tx=0x7fac5009a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed65740989ce3ab server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.923+0000 7fac4bfff640 1 -- 172.21.15.83:0/2457287359 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5009e6a0 con 0x557f5d72ec60 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.923+0000 7fac4bfff640 1 -- 172.21.15.83:0/2457287359 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac500a2cf0 con 0x557f5d72ec60 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.923+0000 7fac4bfff640 1 -- 172.21.15.83:0/2457287359 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50096bc0 con 0x557f5d72ec60 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.923+0000 7fac6a81f740 1 -- 172.21.15.83:0/2457287359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 msgr2=0x557f5d733760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.923+0000 7fac6a81f740 1 --2- 172.21.15.83:0/2457287359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d733760 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fac5009a4e0 tx=0x7fac5009a150 comp rx=0 tx=0).stop 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.923+0000 7fac6a81f740 1 -- 172.21.15.83:0/2457287359 shutdown_connections 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.923+0000 7fac6a81f740 1 --2- 172.21.15.83:0/2457287359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d733dd0 0x557f5d736260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.923+0000 7fac6a81f740 1 --2- 172.21.15.83:0/2457287359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d733760 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.923+0000 7fac6a81f740 1 --2- 172.21.15.83:0/2457287359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d72e270 0x557f5d72e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 1 -- 172.21.15.83:0/2457287359 >> 172.21.15.83:0/2457287359 conn(0x557f5d7297e0 msgr2=0x557f5d729be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 1 -- 172.21.15.83:0/2457287359 shutdown_connections 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 1 -- 172.21.15.83:0/2457287359 wait complete. 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 1 -- start start 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 1 Processor -- start 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d7384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72ec60 0x557f5d7389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d73f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d697020 con 0x557f5d72e270 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d36a430 con 0x557f5d733dd0 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d326e00 con 0x557f5d72ec60 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.924+0000 7fac6a81f740 10 client.0 _ll_register_callbacks cb 0x557f5d63ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:24.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac65915640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d7384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac65114640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72ec60 0x557f5d7389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac65915640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d7384b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38694/0 (socket says 172.21.15.83:38694) 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac65915640 1 -- 172.21.15.83:0/764850613 learned_addr learned my addr 172.21.15.83:0/764850613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac66116640 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d73f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac65915640 1 -- 172.21.15.83:0/764850613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72ec60 msgr2=0x557f5d7389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac65915640 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72ec60 0x557f5d7389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac65915640 1 -- 172.21.15.83:0/764850613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 msgr2=0x557f5d73f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac65915640 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d73f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac65915640 1 -- 172.21.15.83:0/764850613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac601a19e0 con 0x557f5d72e270 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac65114640 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72ec60 0x557f5d7389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac65915640 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d7384b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fac5c003f50 tx=0x7fac5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117df26081735c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac497fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac497fa640 1 -- 172.21.15.83:0/764850613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c01f840 con 0x557f5d72e270 2024-03-11T19:54:24.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac6a81f740 1 -- 172.21.15.83:0/764850613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f5d3967e0 con 0x557f5d72e270 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac497fa640 1 -- 172.21.15.83:0/764850613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fac5c017070 con 0x557f5d72e270 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac497fa640 1 -- 172.21.15.83:0/764850613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac5c02e030 con 0x557f5d72e270 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac6a81f740 10 client.18174 fetch_fsmap learned FSMap version 32 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.925+0000 7fac6a81f740 1 -- 172.21.15.83:0/764850613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557f5d6956a0 con 0x557f5d72e270 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.926+0000 7fac497fa640 1 -- 172.21.15.83:0/764850613 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557f5d3967e0 con 0x557f5d72e270 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.926+0000 7fac497fa640 1 -- 172.21.15.83:0/764850613 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fac5c028ae0 con 0x557f5d72e270 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.926+0000 7fac6a81f740 10 client.18174 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.926+0000 7fac6a81f740 10 client.18174 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.926+0000 7fac6a81f740 20 client.18174 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.926+0000 7fac6a81f740 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557f5d7446b0 0x557f5d746ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.926+0000 7fac6a81f740 4 client.18174 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.926+0000 7fac6a81f740 1 -- 172.21.15.83:0/764850613 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f5d3d3720 con 0x557f5d7446b0 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.926+0000 7fac65114640 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557f5d7446b0 0x557f5d746ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.927+0000 7fac65114640 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557f5d7446b0 0x557f5d746ad0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=521931919fa7235e server_cookie=fcfff69e9d347c9d in_seq=0 out_seq=0 2024-03-11T19:54:24.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.927+0000 7fac497fa640 10 client.18174 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:54:24.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.927+0000 7fac497fa640 1 -- 172.21.15.83:0/764850613 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f5d3d3720 con 0x557f5d7446b0 2024-03-11T19:54:24.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.927+0000 7fac497fa640 10 client.18174 handle_command_reply: tid=0 2024-03-11T19:54:24.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.927+0000 7fac6a81f740 1 client.18174 shutdown 2024-03-11T19:54:24.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac6a81f740 1 -- 172.21.15.83:0/764850613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 msgr2=0x557f5d7384b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac6a81f740 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d7384b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fac5c003f50 tx=0x7fac5c004730 comp rx=0 tx=0).stop 2024-03-11T19:54:24.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac6a81f740 1 -- 172.21.15.83:0/764850613 shutdown_connections 2024-03-11T19:54:24.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac6a81f740 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557f5d7446b0 0x557f5d746ad0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac6a81f740 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d73f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac6a81f740 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72ec60 0x557f5d7389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac497fa640 0 client.18174 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:54:24.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac6a81f740 1 --2- 172.21.15.83:0/764850613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d7384b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac6a81f740 1 -- 172.21.15.83:0/764850613 >> 172.21.15.83:0/764850613 conn(0x557f5d7297e0 msgr2=0x557f5d728b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac6a81f740 1 -- 172.21.15.83:0/764850613 shutdown_connections 2024-03-11T19:54:24.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac6a81f740 1 -- 172.21.15.83:0/764850613 wait complete. 2024-03-11T19:54:24.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.928+0000 7fac6a81f740 20 client.18174 trim_cache size 0 max 16384 2024-03-11T19:54:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.940+0000 7fac6a81f740 1 Processor -- start 2024-03-11T19:54:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.940+0000 7fac6a81f740 1 -- start start 2024-03-11T19:54:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.940+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72e270 0x557f5d7361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.940+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d73f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.940+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d729bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.940+0000 7fac6a81f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d61c130 con 0x557f5d72ec60 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.940+0000 7fac6a81f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d3967e0 con 0x557f5d733dd0 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.940+0000 7fac6a81f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d71ba40 con 0x557f5d72e270 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac65114640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d73f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac65114640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d73f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38704/0 (socket says 172.21.15.83:38704) 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac65114640 1 -- 172.21.15.83:0/418845378 learned_addr learned my addr 172.21.15.83:0/418845378 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac65915640 1 --2- 172.21.15.83:0/418845378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72e270 0x557f5d7361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac66116640 1 --2- 172.21.15.83:0/418845378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d729bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac65114640 1 -- 172.21.15.83:0/418845378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72e270 msgr2=0x557f5d7361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac65114640 1 --2- 172.21.15.83:0/418845378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72e270 0x557f5d7361c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac65114640 1 -- 172.21.15.83:0/418845378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 msgr2=0x557f5d729bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac65114640 1 --2- 172.21.15.83:0/418845378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d729bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac65114640 1 -- 172.21.15.83:0/418845378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f5d631c20 con 0x557f5d72ec60 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac66116640 1 --2- 172.21.15.83:0/418845378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d729bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac65114640 1 --2- 172.21.15.83:0/418845378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d73f950 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fac500cb130 tx=0x7fac50085050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35553e9489aace6e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac4b7fe640 1 -- 172.21.15.83:0/418845378 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500ba0d0 con 0x557f5d72ec60 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac4b7fe640 1 -- 172.21.15.83:0/418845378 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac50098850 con 0x557f5d72ec60 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.941+0000 7fac4b7fe640 1 -- 172.21.15.83:0/418845378 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500cd110 con 0x557f5d72ec60 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.942+0000 7fac6a81f740 1 -- 172.21.15.83:0/418845378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 msgr2=0x557f5d73f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.942+0000 7fac6a81f740 1 --2- 172.21.15.83:0/418845378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d73f950 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fac500cb130 tx=0x7fac50085050 comp rx=0 tx=0).stop 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.942+0000 7fac6a81f740 1 -- 172.21.15.83:0/418845378 shutdown_connections 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.942+0000 7fac6a81f740 1 --2- 172.21.15.83:0/418845378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d729bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.942+0000 7fac6a81f740 1 --2- 172.21.15.83:0/418845378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72ec60 0x557f5d73f950 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.942+0000 7fac6a81f740 1 --2- 172.21.15.83:0/418845378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72e270 0x557f5d7361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.942+0000 7fac6a81f740 1 -- 172.21.15.83:0/418845378 >> 172.21.15.83:0/418845378 conn(0x557f5d7297e0 msgr2=0x557f5d724a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.942+0000 7fac6a81f740 1 -- 172.21.15.83:0/418845378 shutdown_connections 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.942+0000 7fac6a81f740 1 -- 172.21.15.83:0/418845378 wait complete. 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.942+0000 7fac6a81f740 1 -- start start 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac6a81f740 1 Processor -- start 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d73f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72ec60 0x557f5d73dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac6a81f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d73e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac6a81f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d73fd70 con 0x557f5d72e270 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac6a81f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d73e6f0 con 0x557f5d733dd0 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac6a81f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5d73e880 con 0x557f5d72ec60 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac65915640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d73f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac65915640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d73f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38710/0 (socket says 172.21.15.83:38710) 2024-03-11T19:54:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac65915640 1 -- 172.21.15.83:0/2180906819 learned_addr learned my addr 172.21.15.83:0/2180906819 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac65114640 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72ec60 0x557f5d73dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac6a81f740 10 client.0 _ll_register_callbacks cb 0x7fac50096130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac66116640 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d73e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac65915640 1 -- 172.21.15.83:0/2180906819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72ec60 msgr2=0x557f5d73dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac65915640 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72ec60 0x557f5d73dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac65915640 1 -- 172.21.15.83:0/2180906819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 msgr2=0x557f5d73e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac65915640 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d73e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac65915640 1 -- 172.21.15.83:0/2180906819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f5d659060 con 0x557f5d72e270 2024-03-11T19:54:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.943+0000 7fac66116640 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d73e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac65915640 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d73f580 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fac5c028e60 tx=0x7fac5c028be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee5afadf74c3a3c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac497fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac497fa640 1 -- 172.21.15.83:0/2180906819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c02e040 con 0x557f5d72e270 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac497fa640 1 -- 172.21.15.83:0/2180906819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fac5c017050 con 0x557f5d72e270 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac6a81f740 1 -- 172.21.15.83:0/2180906819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac500a2cf0 con 0x557f5d72e270 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac497fa640 1 -- 172.21.15.83:0/2180906819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac5c003a80 con 0x557f5d72e270 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac6a81f740 10 client.18180 fetch_fsmap learned FSMap version 32 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac6a81f740 1 -- 172.21.15.83:0/2180906819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557f5d6578c0 con 0x557f5d72e270 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac497fa640 1 -- 172.21.15.83:0/2180906819 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac500a2cf0 con 0x557f5d72e270 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac497fa640 1 -- 172.21.15.83:0/2180906819 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fac5c044070 con 0x557f5d72e270 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac6a81f740 10 client.18180 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac6a81f740 10 client.18180 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:54:24.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac6a81f740 20 client.18180 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.944+0000 7fac6a81f740 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557f5d79c140 0x557f5d79e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.945+0000 7fac6a81f740 4 client.18180 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:54:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.945+0000 7fac6a81f740 1 -- 172.21.15.83:0/2180906819 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f5d422890 con 0x557f5d79c140 2024-03-11T19:54:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.945+0000 7fac65114640 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557f5d79c140 0x557f5d79e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.945+0000 7fac497fa640 10 client.18180 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:54:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.945+0000 7fac65114640 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557f5d79c140 0x557f5d79e560 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7bdc024962ef14d server_cookie=2d711c038408956c in_seq=0 out_seq=0 2024-03-11T19:54:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.945+0000 7fac497fa640 1 -- 172.21.15.83:0/2180906819 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f5d422890 con 0x557f5d79c140 2024-03-11T19:54:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.945+0000 7fac497fa640 10 client.18180 handle_command_reply: tid=0 2024-03-11T19:54:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.945+0000 7fac6a81f740 1 client.18180 shutdown 2024-03-11T19:54:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.945+0000 7fac6a81f740 1 -- 172.21.15.83:0/2180906819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 msgr2=0x557f5d73f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.945+0000 7fac6a81f740 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d73f580 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fac5c028e60 tx=0x7fac5c028be0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac6a81f740 1 -- 172.21.15.83:0/2180906819 shutdown_connections 2024-03-11T19:54:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac6a81f740 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557f5d79c140 0x557f5d79e560 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac497fa640 0 client.18180 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:54:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac6a81f740 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557f5d733dd0 0x557f5d73e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac6a81f740 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f5d72ec60 0x557f5d73dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac6a81f740 1 --2- 172.21.15.83:0/2180906819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f5d72e270 0x557f5d73f580 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac6a81f740 1 -- 172.21.15.83:0/2180906819 >> 172.21.15.83:0/2180906819 conn(0x557f5d7297e0 msgr2=0x557f5d729bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac6a81f740 1 -- 172.21.15.83:0/2180906819 shutdown_connections 2024-03-11T19:54:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac6a81f740 1 -- 172.21.15.83:0/2180906819 wait complete. 2024-03-11T19:54:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac6a81f740 20 client.18180 trim_cache size 0 max 16384 2024-03-11T19:54:24.963 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:54:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac67b79640 1 -- 172.21.15.83:0/382677981 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fac4c0a1ab0 msgr2=0x7fac4c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac67b79640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fac4c0a1ab0 0x7fac4c0a3f70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fac5c005fd0 tx=0x7fac5c00b410 comp rx=0 tx=0).stop 2024-03-11T19:54:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac67b79640 1 -- 172.21.15.83:0/382677981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac60102380 msgr2=0x7fac60073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac67b79640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac60102380 0x7fac60073570 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fac5000ea20 tx=0x7fac5000ee20 comp rx=0 tx=0).stop 2024-03-11T19:54:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac67b79640 1 -- 172.21.15.83:0/382677981 shutdown_connections 2024-03-11T19:54:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac67b79640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fac4c0a1ab0 0x7fac4c0a3f70 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac67b79640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fac60102d40 0x7fac60073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac67b79640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac60102380 0x7fac60073570 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac67b79640 1 --2- 172.21.15.83:0/382677981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac60101190 0x7fac60074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.946+0000 7fac67b79640 1 -- 172.21.15.83:0/382677981 >> 172.21.15.83:0/382677981 conn(0x7fac600fc6e0 msgr2=0x7fac601056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.947+0000 7fac67b79640 1 -- 172.21.15.83:0/382677981 shutdown_connections 2024-03-11T19:54:24.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:24.947+0000 7fac67b79640 1 -- 172.21.15.83:0/382677981 wait complete. 2024-03-11T19:54:24.965 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:24 smithi083 ceph-mon[31864]: pgmap v804: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 727 op/s 2024-03-11T19:54:24.965 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:24 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1336480896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:54:24.966 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:54:24.966 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:54:25.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.073+0000 7fdcfb52c640 1 -- 172.21.15.83:0/21990088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf4102d60 msgr2=0x7fdcf41095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:25.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.074+0000 7fdcfb52c640 1 --2- 172.21.15.83:0/21990088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf4102d60 0x7fdcf41095f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009ca0 tx=0x7fdcec02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.074+0000 7fdcfb52c640 1 -- 172.21.15.83:0/21990088 shutdown_connections 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.074+0000 7fdcfb52c640 1 --2- 172.21.15.83:0/21990088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf4102d60 0x7fdcf41095f0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.074+0000 7fdcfb52c640 1 --2- 172.21.15.83:0/21990088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcf41023a0 0x7fdcf4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.074+0000 7fdcfb52c640 1 --2- 172.21.15.83:0/21990088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcf41011b0 0x7fdcf41015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.074+0000 7fdcfb52c640 1 -- 172.21.15.83:0/21990088 >> 172.21.15.83:0/21990088 conn(0x7fdcf40fc720 msgr2=0x7fdcf40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.074+0000 7fdcfb52c640 1 -- 172.21.15.83:0/21990088 shutdown_connections 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.074+0000 7fdcfb52c640 1 -- 172.21.15.83:0/21990088 wait complete. 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.074+0000 7fdcfb52c640 1 Processor -- start 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.074+0000 7fdcfb52c640 1 -- start start 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcfb52c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcf41011b0 0x7fdcf41971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcfb52c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcf41023a0 0x7fdcf4197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcfb52c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf4102d60 0x7fdcf419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcfb52c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf419ecd0 con 0x7fdcf4102d60 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcfb52c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf419ee40 con 0x7fdcf41023a0 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcfb52c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf419f000 con 0x7fdcf41011b0 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcf9ac9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf4102d60 0x7fdcf419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcf9ac9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf4102d60 0x7fdcf419e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38734/0 (socket says 172.21.15.83:38734) 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcf9ac9640 1 -- 172.21.15.83:0/1659428944 learned_addr learned my addr 172.21.15.83:0/1659428944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcf41011b0 0x7fdcf41971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcf41023a0 0x7fdcf4197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcf9ac9640 1 -- 172.21.15.83:0/1659428944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcf41011b0 msgr2=0x7fdcf41971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcf9ac9640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcf41011b0 0x7fdcf41971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcf9ac9640 1 -- 172.21.15.83:0/1659428944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcf41023a0 msgr2=0x7fdcf4197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:25.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcf9ac9640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcf41023a0 0x7fdcf4197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcf9ac9640 1 -- 172.21.15.83:0/1659428944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec009900 con 0x7fdcf4102d60 2024-03-11T19:54:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.075+0000 7fdcf9ac9640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf4102d60 0x7fdcf419e790 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fdcec037040 tx=0x7fdcec009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.076+0000 7fdcea7fc640 1 -- 172.21.15.83:0/1659428944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0329b0 con 0x7fdcf4102d60 2024-03-11T19:54:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.076+0000 7fdcea7fc640 1 -- 172.21.15.83:0/1659428944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcec003e50 con 0x7fdcf4102d60 2024-03-11T19:54:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.076+0000 7fdcea7fc640 1 -- 172.21.15.83:0/1659428944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0048a0 con 0x7fdcf4102d60 2024-03-11T19:54:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.076+0000 7fdcfb52c640 1 -- 172.21.15.83:0/1659428944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf419f280 con 0x7fdcf4102d60 2024-03-11T19:54:25.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.076+0000 7fdcfb52c640 1 -- 172.21.15.83:0/1659428944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf419f700 con 0x7fdcf4102d60 2024-03-11T19:54:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.078+0000 7fdcea7fc640 1 -- 172.21.15.83:0/1659428944 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdcec00a040 con 0x7fdcf4102d60 2024-03-11T19:54:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcfe1db740 1 Processor -- start 2024-03-11T19:54:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcfe1db740 1 -- start start 2024-03-11T19:54:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6a270 0x55c30ad6a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad6f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6fdd0 0x55c30ad72260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30ad26960 con 0x55c30ad6ac60 2024-03-11T19:54:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30acda310 con 0x55c30ad6fdd0 2024-03-11T19:54:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30acd7840 con 0x55c30ad6a270 2024-03-11T19:54:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcea7fc640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdce00a1960 0x7fdce00a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcf92c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad6f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcf92c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad6f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38748/0 (socket says 172.21.15.83:38748) 2024-03-11T19:54:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcf92c8640 1 -- 172.21.15.83:0/1510157390 learned_addr learned my addr 172.21.15.83:0/1510157390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/1510157390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6a270 0x55c30ad6a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcea7fc640 1 -- 172.21.15.83:0/1659428944 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdcec0ef460 con 0x7fdcf4102d60 2024-03-11T19:54:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcf9ac9640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdce00a1960 0x7fdce00a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.079+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/1510157390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6fdd0 0x55c30ad72260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.080+0000 7fdcf92c8640 1 -- 172.21.15.83:0/1510157390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6a270 msgr2=0x55c30ad6a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.080+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/1510157390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6a270 0x55c30ad6a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.080+0000 7fdcf92c8640 1 -- 172.21.15.83:0/1510157390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6fdd0 msgr2=0x55c30ad72260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.080+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/1510157390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6fdd0 0x55c30ad72260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.080+0000 7fdcf92c8640 1 -- 172.21.15.83:0/1510157390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c30ac95060 con 0x55c30ad6ac60 2024-03-11T19:54:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.080+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/1510157390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6fdd0 0x55c30ad72260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.080+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/1510157390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad6f760 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009ca0 tx=0x7fdcf0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcde49b19801816 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.080+0000 7fdcf9ac9640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdce00a1960 0x7fdce00a3e20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009620 tx=0x7fdcec032570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.080+0000 7fdcdf7fe640 1 -- 172.21.15.83:0/1510157390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0004220 con 0x55c30ad6ac60 2024-03-11T19:54:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.080+0000 7fdcdf7fe640 1 -- 172.21.15.83:0/1510157390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcf0004380 con 0x55c30ad6ac60 2024-03-11T19:54:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.080+0000 7fdcdf7fe640 1 -- 172.21.15.83:0/1510157390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf00197a0 con 0x55c30ad6ac60 2024-03-11T19:54:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.081+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1510157390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 msgr2=0x55c30ad6f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.081+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/1510157390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad6f760 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009ca0 tx=0x7fdcf0011810 comp rx=0 tx=0).stop 2024-03-11T19:54:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.081+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1510157390 shutdown_connections 2024-03-11T19:54:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.081+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/1510157390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6fdd0 0x55c30ad72260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.081+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/1510157390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad6f760 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.081+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/1510157390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6a270 0x55c30ad6a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.081+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1510157390 >> 172.21.15.83:0/1510157390 conn(0x55c30ad657e0 msgr2=0x55c30ad65be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.081+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1510157390 shutdown_connections 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.081+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1510157390 wait complete. 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.081+0000 7fdcfe1db740 1 -- start start 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.081+0000 7fdcfe1db740 1 Processor -- start 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6a270 0x55c30ad74460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6ac60 0x55c30ad749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad7b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30acd3020 con 0x55c30ad6a270 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30a9a6430 con 0x55c30ad6ac60 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30a962e00 con 0x55c30ad6fdd0 2024-03-11T19:54:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcfe1db740 10 client.0 _ll_register_callbacks cb 0x55c30ac7acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcf8ac7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6a270 0x55c30ad74460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcf8ac7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6a270 0x55c30ad74460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38764/0 (socket says 172.21.15.83:38764) 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcf8ac7640 1 -- 172.21.15.83:0/1937203374 learned_addr learned my addr 172.21.15.83:0/1937203374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad7b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6ac60 0x55c30ad749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcf8ac7640 1 -- 172.21.15.83:0/1937203374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 msgr2=0x55c30ad7b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad7b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcf8ac7640 1 -- 172.21.15.83:0/1937203374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6ac60 msgr2=0x55c30ad749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6ac60 0x55c30ad749a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcf8ac7640 1 -- 172.21.15.83:0/1937203374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf00098d0 con 0x55c30ad6a270 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6a270 0x55c30ad74460 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fdce400f460 tx=0x7fdce4009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97087ab57f40bf6e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.082+0000 7fdcdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcdcff9640 1 -- 172.21.15.83:0/1937203374 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce400fea0 con 0x55c30ad6a270 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcdcff9640 1 -- 172.21.15.83:0/1937203374 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdce4004250 con 0x55c30ad6a270 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1937203374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c30a9d27e0 con 0x55c30ad6a270 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcdcff9640 1 -- 172.21.15.83:0/1937203374 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdce4004410 con 0x55c30ad6a270 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcfe1db740 10 client.18192 fetch_fsmap learned FSMap version 32 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1937203374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c30acd16a0 con 0x55c30ad6a270 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcdcff9640 1 -- 172.21.15.83:0/1937203374 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c30a9d27e0 con 0x55c30ad6a270 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcdcff9640 1 -- 172.21.15.83:0/1937203374 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdce402a070 con 0x55c30ad6a270 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcfe1db740 10 client.18192 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcfe1db740 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcfe1db740 20 client.18192 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c30ad80660 0x55c30ad82a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcfe1db740 4 client.18192 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.083+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1937203374 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c30aa0f720 con 0x55c30ad80660 2024-03-11T19:54:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.084+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c30ad80660 0x55c30ad82a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.084+0000 7fdcdcff9640 10 client.18192 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.084+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c30ad80660 0x55c30ad82a80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61404ec904e94d1a server_cookie=21deb164bf265a97 in_seq=0 out_seq=0 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcdcff9640 1 -- 172.21.15.83:0/1937203374 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c30aa0f720 con 0x55c30ad80660 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcdcff9640 10 client.18192 handle_command_reply: tid=0 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 1 client.18192 shutdown 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1937203374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6a270 msgr2=0x55c30ad74460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6a270 0x55c30ad74460 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fdce400f460 tx=0x7fdce4009ac0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1937203374 shutdown_connections 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c30ad80660 0x55c30ad82a80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad7b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6ac60 0x55c30ad749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/1937203374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6a270 0x55c30ad74460 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcdcff9640 0 client.18192 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1937203374 >> 172.21.15.83:0/1937203374 conn(0x55c30ad657e0 msgr2=0x55c30ad64b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1937203374 shutdown_connections 2024-03-11T19:54:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 1 -- 172.21.15.83:0/1937203374 wait complete. 2024-03-11T19:54:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.085+0000 7fdcfe1db740 20 client.18192 trim_cache size 0 max 16384 2024-03-11T19:54:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.098+0000 7fdcfe1db740 1 Processor -- start 2024-03-11T19:54:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.098+0000 7fdcfe1db740 1 -- start start 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.098+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6a270 0x55c30ad74070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.098+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.098+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.098+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30ac58130 con 0x55c30ad6ac60 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.098+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30ad72720 con 0x55c30ad6a270 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.098+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30ad6b0e0 con 0x55c30ad6fdd0 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf92c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf92c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38766/0 (socket says 172.21.15.83:38766) 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf92c8640 1 -- 172.21.15.83:0/123189995 learned_addr learned my addr 172.21.15.83:0/123189995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/123189995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/123189995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6a270 0x55c30ad74070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf92c8640 1 -- 172.21.15.83:0/123189995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 msgr2=0x55c30ad745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/123189995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad745b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf92c8640 1 -- 172.21.15.83:0/123189995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6a270 msgr2=0x55c30ad74070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/123189995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6a270 0x55c30ad74070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf92c8640 1 -- 172.21.15.83:0/123189995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c30ac6dc20 con 0x55c30ad6ac60 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/123189995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6a270 0x55c30ad74070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/123189995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b9a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0000c00 tx=0x7fdcf0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed67206e656bb118 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcdeffd640 1 -- 172.21.15.83:0/123189995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0004100 con 0x55c30ad6ac60 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcdeffd640 1 -- 172.21.15.83:0/123189995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcf0005140 con 0x55c30ad6ac60 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.099+0000 7fdcdeffd640 1 -- 172.21.15.83:0/123189995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf002a730 con 0x55c30ad6ac60 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.100+0000 7fdcfe1db740 1 -- 172.21.15.83:0/123189995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 msgr2=0x55c30ad7b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.100+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/123189995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b9a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0000c00 tx=0x7fdcf0011810 comp rx=0 tx=0).stop 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.100+0000 7fdcfe1db740 1 -- 172.21.15.83:0/123189995 shutdown_connections 2024-03-11T19:54:25.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.100+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/123189995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.100+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/123189995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b9a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.100+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/123189995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6a270 0x55c30ad74070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.100+0000 7fdcfe1db740 1 -- 172.21.15.83:0/123189995 >> 172.21.15.83:0/123189995 conn(0x55c30ad657e0 msgr2=0x55c30ad65bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:25.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.100+0000 7fdcfe1db740 1 -- 172.21.15.83:0/123189995 shutdown_connections 2024-03-11T19:54:25.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.100+0000 7fdcfe1db740 1 -- 172.21.15.83:0/123189995 wait complete. 2024-03-11T19:54:25.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.100+0000 7fdcfe1db740 1 -- start start 2024-03-11T19:54:25.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.100+0000 7fdcfe1db740 1 Processor -- start 2024-03-11T19:54:25.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6a270 0x55c30ad7b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcfe1db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad79940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30ad7a040 con 0x55c30ad6ac60 2024-03-11T19:54:25.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30ad7a1b0 con 0x55c30ad6a270 2024-03-11T19:54:25.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcfe1db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30ad7a370 con 0x55c30ad6fdd0 2024-03-11T19:54:25.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcf92c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcf92c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38768/0 (socket says 172.21.15.83:38768) 2024-03-11T19:54:25.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcf92c8640 1 -- 172.21.15.83:0/3541485620 learned_addr learned my addr 172.21.15.83:0/3541485620 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:25.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcfe1db740 10 client.0 _ll_register_callbacks cb 0x7fdcf00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:25.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6a270 0x55c30ad7b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad79940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcf92c8640 1 -- 172.21.15.83:0/3541485620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 msgr2=0x55c30ad79940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad79940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcf92c8640 1 -- 172.21.15.83:0/3541485620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6a270 msgr2=0x55c30ad7b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6a270 0x55c30ad7b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcf92c8640 1 -- 172.21.15.83:0/3541485620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c30ac95060 con 0x55c30ad6ac60 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcf92c8640 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b870 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009f50 tx=0x7fdcf0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e768548d481b63 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.101+0000 7fdcdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcdcff9640 1 -- 172.21.15.83:0/3541485620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf00037f0 con 0x55c30ad6ac60 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcdcff9640 1 -- 172.21.15.83:0/3541485620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdcf0039070 con 0x55c30ad6ac60 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcfe1db740 1 -- 172.21.15.83:0/3541485620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcf0004380 con 0x55c30ad6ac60 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcdcff9640 1 -- 172.21.15.83:0/3541485620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcf0044030 con 0x55c30ad6ac60 2024-03-11T19:54:25.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcfe1db740 10 client.18198 fetch_fsmap learned FSMap version 32 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcfe1db740 1 -- 172.21.15.83:0/3541485620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c30ac938c0 con 0x55c30ad6ac60 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcdcff9640 1 -- 172.21.15.83:0/3541485620 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcf0004380 con 0x55c30ad6ac60 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcdcff9640 1 -- 172.21.15.83:0/3541485620 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdcf0004b50 con 0x55c30ad6ac60 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcfe1db740 10 client.18198 fetch_fsmap finished waiting for FSMap version 32 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcfe1db740 10 client.18198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcfe1db740 20 client.18198 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c30add7d90 0x55c30adda180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcfe1db740 4 client.18198 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcfe1db740 1 -- 172.21.15.83:0/3541485620 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c30aa5e890 con 0x55c30add7d90 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.102+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c30add7d90 0x55c30adda180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.103+0000 7fdcdcff9640 10 client.18198 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.103+0000 7fdcf8ac7640 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c30add7d90 0x55c30adda180 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76dd5d5722c24251 server_cookie=b3df636e4e770f23 in_seq=0 out_seq=0 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.103+0000 7fdcdcff9640 1 -- 172.21.15.83:0/3541485620 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c30aa5e890 con 0x55c30add7d90 2024-03-11T19:54:25.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.103+0000 7fdcdcff9640 10 client.18198 handle_command_reply: tid=0 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.103+0000 7fdcfe1db740 1 client.18198 shutdown 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfe1db740 1 -- 172.21.15.83:0/3541485620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 msgr2=0x55c30ad7b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b870 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009f50 tx=0x7fdcf0011da0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfe1db740 1 -- 172.21.15.83:0/3541485620 shutdown_connections 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c30add7d90 0x55c30adda180 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c30ad6fdd0 0x55c30ad79940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c30ad6ac60 0x55c30ad7b870 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfe1db740 1 --2- 172.21.15.83:0/3541485620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c30ad6a270 0x55c30ad7b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfe1db740 1 -- 172.21.15.83:0/3541485620 >> 172.21.15.83:0/3541485620 conn(0x55c30ad657e0 msgr2=0x55c30ad74070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcdcff9640 0 client.18198 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfe1db740 1 -- 172.21.15.83:0/3541485620 shutdown_connections 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfe1db740 1 -- 172.21.15.83:0/3541485620 wait complete. 2024-03-11T19:54:25.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfe1db740 20 client.18198 trim_cache size 0 max 16384 2024-03-11T19:54:25.115 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:54:25.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfb52c640 1 -- 172.21.15.83:0/1659428944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdce00a1960 msgr2=0x7fdce00a3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfb52c640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdce00a1960 0x7fdce00a3e20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009620 tx=0x7fdcec032570 comp rx=0 tx=0).stop 2024-03-11T19:54:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfb52c640 1 -- 172.21.15.83:0/1659428944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf4102d60 msgr2=0x7fdcf419e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfb52c640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf4102d60 0x7fdcf419e790 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fdcec037040 tx=0x7fdcec009360 comp rx=0 tx=0).stop 2024-03-11T19:54:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfb52c640 1 -- 172.21.15.83:0/1659428944 shutdown_connections 2024-03-11T19:54:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfb52c640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdce00a1960 0x7fdce00a3e20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfb52c640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf4102d60 0x7fdcf419e790 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfb52c640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcf41023a0 0x7fdcf4197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfb52c640 1 --2- 172.21.15.83:0/1659428944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcf41011b0 0x7fdcf41971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.104+0000 7fdcfb52c640 1 -- 172.21.15.83:0/1659428944 >> 172.21.15.83:0/1659428944 conn(0x7fdcf40fc720 msgr2=0x7fdcf4105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.105+0000 7fdcfb52c640 1 -- 172.21.15.83:0/1659428944 shutdown_connections 2024-03-11T19:54:25.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:25.105+0000 7fdcfb52c640 1 -- 172.21.15.83:0/1659428944 wait complete. 2024-03-11T19:54:25.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:24 smithi137 ceph-mon[36220]: pgmap v804: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 727 op/s 2024-03-11T19:54:25.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:24 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1336480896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:54:25.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:24 smithi204 ceph-mon[36739]: pgmap v804: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 727 op/s 2024-03-11T19:54:25.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:24 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1336480896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:54:26.120 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-11T19:54:26.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:54:26.121 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-11T19:54:26.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.328+0000 7f061b002640 1 -- 172.21.15.83:0/3298032065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061410e4f0 msgr2=0x7f06141108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.328+0000 7f061b002640 1 --2- 172.21.15.83:0/3298032065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061410e4f0 0x7f06141108e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f060c009be0 tx=0x7f060c02bf10 comp rx=0 tx=0).stop 2024-03-11T19:54:26.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.328+0000 7f061b002640 1 -- 172.21.15.83:0/3298032065 shutdown_connections 2024-03-11T19:54:26.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.328+0000 7f061b002640 1 --2- 172.21.15.83:0/3298032065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061410e4f0 0x7f06141108e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.328+0000 7f061b002640 1 --2- 172.21.15.83:0/3298032065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061410bbc0 0x7f061410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.328+0000 7f061b002640 1 --2- 172.21.15.83:0/3298032065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614109290 0x7f061410b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.328+0000 7f061b002640 1 -- 172.21.15.83:0/3298032065 >> 172.21.15.83:0/3298032065 conn(0x7f061406e060 msgr2=0x7f06140704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.329+0000 7f061b002640 1 -- 172.21.15.83:0/3298032065 shutdown_connections 2024-03-11T19:54:26.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.329+0000 7f061b002640 1 -- 172.21.15.83:0/3298032065 wait complete. 2024-03-11T19:54:26.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.329+0000 7f061b002640 1 Processor -- start 2024-03-11T19:54:26.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.330+0000 7f061b002640 1 -- start start 2024-03-11T19:54:26.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.330+0000 7f061b002640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614109290 0x7f061419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.330+0000 7f061b002640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061410bbc0 0x7f061419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.330+0000 7f061b002640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061410e4f0 0x7f06141a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.330+0000 7f061b002640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06141a5510 con 0x7f0614109290 2024-03-11T19:54:26.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.330+0000 7f061b002640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06141a5680 con 0x7f061410bbc0 2024-03-11T19:54:26.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.330+0000 7f061b002640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06141a5840 con 0x7f061410e4f0 2024-03-11T19:54:26.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.330+0000 7f061a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614109290 0x7f061419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.330+0000 7f061a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614109290 0x7f061419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38790/0 (socket says 172.21.15.83:38790) 2024-03-11T19:54:26.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.330+0000 7f061a000640 1 -- 172.21.15.83:0/3853956867 learned_addr learned my addr 172.21.15.83:0/3853956867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:26.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.331+0000 7f061a801640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061410e4f0 0x7f06141a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.331+0000 7f06197ff640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061410bbc0 0x7f061419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.331+0000 7f061a000640 1 -- 172.21.15.83:0/3853956867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061410e4f0 msgr2=0x7f06141a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.331+0000 7f061a000640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061410e4f0 0x7f06141a4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.331+0000 7f061a000640 1 -- 172.21.15.83:0/3853956867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061410bbc0 msgr2=0x7f061419ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.331+0000 7f061a000640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061410bbc0 0x7f061419ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.331+0000 7f061a000640 1 -- 172.21.15.83:0/3853956867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0604009560 con 0x7f0614109290 2024-03-11T19:54:26.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.331+0000 7f061a000640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614109290 0x7f061419d990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f060400e9c0 tx=0x7f060400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.333+0000 7f060affd640 1 -- 172.21.15.83:0/3853956867 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060400cde0 con 0x7f0614109290 2024-03-11T19:54:26.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.334+0000 7f061b002640 1 -- 172.21.15.83:0/3853956867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c009840 con 0x7f0614109290 2024-03-11T19:54:26.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.334+0000 7f061b002640 1 -- 172.21.15.83:0/3853956867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06141a5ef0 con 0x7f0614109290 2024-03-11T19:54:26.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.338+0000 7f061ee3a740 1 Processor -- start 2024-03-11T19:54:26.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.338+0000 7f060affd640 1 -- 172.21.15.83:0/3853956867 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06040035f0 con 0x7f0614109290 2024-03-11T19:54:26.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.338+0000 7f060affd640 1 -- 172.21.15.83:0/3853956867 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604010670 con 0x7f0614109290 2024-03-11T19:54:26.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.338+0000 7f060affd640 1 -- 172.21.15.83:0/3853956867 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0604010890 con 0x7f0614109290 2024-03-11T19:54:26.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.338+0000 7f061ee3a740 1 -- start start 2024-03-11T19:54:26.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.339+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2c9aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.339+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9b860 0x55c4d2c9bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.339+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9c220 0x55c4d2ca2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.339+0000 7f061ee3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d2c09f90 con 0x55c4d2c9c220 2024-03-11T19:54:26.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.339+0000 7f061ee3a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d2c074c0 con 0x55c4d2c9b860 2024-03-11T19:54:26.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.339+0000 7f061ee3a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d2c02ca0 con 0x55c4d2c9a690 2024-03-11T19:54:26.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.341+0000 7f060affd640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f06000a1ba0 0x7f06000a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.341+0000 7f061a801640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2c9aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.341+0000 7f060affd640 1 -- 172.21.15.83:0/3853956867 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0604014070 con 0x7f0614109290 2024-03-11T19:54:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.342+0000 7f061a801640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2c9aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41588/0 (socket says 172.21.15.83:41588) 2024-03-11T19:54:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.342+0000 7f06197ff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9c220 0x55c4d2ca2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.342+0000 7f061a801640 1 -- 172.21.15.83:0/1651955313 learned_addr learned my addr 172.21.15.83:0/1651955313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.343+0000 7f061a000640 1 --2- 172.21.15.83:0/1651955313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9b860 0x55c4d2c9bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.343+0000 7f061a801640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f06000a1ba0 0x7f06000a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.347+0000 7f061a801640 1 -- 172.21.15.83:0/1651955313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9b860 msgr2=0x55c4d2c9bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.347+0000 7f061a801640 1 --2- 172.21.15.83:0/1651955313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9b860 0x55c4d2c9bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.347+0000 7f061a801640 1 -- 172.21.15.83:0/1651955313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9c220 msgr2=0x55c4d2ca2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.347+0000 7f061a801640 1 --2- 172.21.15.83:0/1651955313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9c220 0x55c4d2ca2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.347+0000 7f061a801640 1 -- 172.21.15.83:0/1651955313 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4d2bc50b0 con 0x55c4d2c9a690 2024-03-11T19:54:26.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.349+0000 7f061a801640 1 --2- 172.21.15.83:0/1651955313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2c9aab0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f060c009b30 tx=0x7f060c033fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5bd3a6f9da030a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.353+0000 7f05fffff640 1 -- 172.21.15.83:0/1651955313 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c033b80 con 0x55c4d2c9a690 2024-03-11T19:54:26.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.354+0000 7f05fffff640 1 -- 172.21.15.83:0/1651955313 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f060c044d10 con 0x55c4d2c9a690 2024-03-11T19:54:26.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.354+0000 7f05fffff640 1 -- 172.21.15.83:0/1651955313 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c042b70 con 0x55c4d2c9a690 2024-03-11T19:54:26.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.354+0000 7f061a801640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f06000a1ba0 0x7f06000a4060 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f060c039000 tx=0x7f060c049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.355+0000 7f061ee3a740 1 -- 172.21.15.83:0/1651955313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 msgr2=0x55c4d2c9aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.355+0000 7f061ee3a740 1 --2- 172.21.15.83:0/1651955313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2c9aab0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f060c009b30 tx=0x7f060c033fb0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.361+0000 7f061ee3a740 1 -- 172.21.15.83:0/1651955313 shutdown_connections 2024-03-11T19:54:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.361+0000 7f061ee3a740 1 --2- 172.21.15.83:0/1651955313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9c220 0x55c4d2ca2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.361+0000 7f061ee3a740 1 --2- 172.21.15.83:0/1651955313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9b860 0x55c4d2c9bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.361+0000 7f061ee3a740 1 --2- 172.21.15.83:0/1651955313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2c9aab0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.361+0000 7f061ee3a740 1 -- 172.21.15.83:0/1651955313 >> 172.21.15.83:0/1651955313 conn(0x55c4d2c95870 msgr2=0x55c4d2c95c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.364+0000 7f061ee3a740 1 -- 172.21.15.83:0/1651955313 shutdown_connections 2024-03-11T19:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.368+0000 7f061ee3a740 1 -- 172.21.15.83:0/1651955313 wait complete. 2024-03-11T19:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.368+0000 7f061ee3a740 1 -- start start 2024-03-11T19:54:26.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.368+0000 7f061ee3a740 1 Processor -- start 2024-03-11T19:54:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.369+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2ca4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.369+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9b860 0x55c4d2ca4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.369+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9c220 0x55c4d2cabec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2ca4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2ca4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41604/0 (socket says 172.21.15.83:41604) 2024-03-11T19:54:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061a000640 1 -- 172.21.15.83:0/4244767712 learned_addr learned my addr 172.21.15.83:0/4244767712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061ee3a740 1 -- 172.21.15.83:0/4244767712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d2ba7f40 con 0x55c4d2c9c220 2024-03-11T19:54:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061ee3a740 1 -- 172.21.15.83:0/4244767712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d28d64b0 con 0x55c4d2c9b860 2024-03-11T19:54:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061ee3a740 1 -- 172.21.15.83:0/4244767712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d2b88670 con 0x55c4d2c9a690 2024-03-11T19:54:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061a801640 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9c220 0x55c4d2cabec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061ee3a740 10 client.0 _ll_register_callbacks cb 0x55c4d2bf5f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061a000640 1 -- 172.21.15.83:0/4244767712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9b860 msgr2=0x55c4d2ca4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061a000640 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9b860 0x55c4d2ca4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061a000640 1 -- 172.21.15.83:0/4244767712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9c220 msgr2=0x55c4d2cabec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061a000640 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9c220 0x55c4d2cabec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.370+0000 7f061a000640 1 -- 172.21.15.83:0/4244767712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0614111470 con 0x55c4d2c9a690 2024-03-11T19:54:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.371+0000 7f061a000640 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2ca4940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f060400c6a0 tx=0x7f060409b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd1263bbb08e247 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.371+0000 7f05fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:54:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.371+0000 7f061ee3a740 1 -- 172.21.15.83:0/4244767712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4d2bdcf40 con 0x55c4d2c9a690 2024-03-11T19:54:26.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.372+0000 7f05fd7fa640 1 -- 172.21.15.83:0/4244767712 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06040b4220 con 0x55c4d2c9a690 2024-03-11T19:54:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.372+0000 7f05fd7fa640 1 -- 172.21.15.83:0/4244767712 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f06040b4380 con 0x55c4d2c9a690 2024-03-11T19:54:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.372+0000 7f05fd7fa640 1 -- 172.21.15.83:0/4244767712 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f060409bb60 con 0x55c4d2c9a690 2024-03-11T19:54:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.372+0000 7f061ee3a740 10 client.27766 fetch_fsmap learned FSMap version 33 2024-03-11T19:54:26.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.372+0000 7f061ee3a740 1 -- 172.21.15.83:0/4244767712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=33}) v3 -- 0x55c4d2c01320 con 0x55c4d2c9a690 2024-03-11T19:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.373+0000 7f05fd7fa640 1 -- 172.21.15.83:0/4244767712 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 33) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f060401f060 con 0x55c4d2c9a690 2024-03-11T19:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.373+0000 7f061ee3a740 10 client.27766 fetch_fsmap finished waiting for FSMap version 33 2024-03-11T19:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.374+0000 7f061ee3a740 10 client.27766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:26.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.374+0000 7f061ee3a740 20 client.27766 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.374+0000 7f061ee3a740 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4d2cb0980 0x55c4d2cb2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.374+0000 7f061ee3a740 4 client.27766 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.374+0000 7f061ee3a740 1 -- 172.21.15.83:0/4244767712 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4d2bdcf40 con 0x55c4d2cb0980 2024-03-11T19:54:26.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.374+0000 7f06197ff640 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4d2cb0980 0x55c4d2cb2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.391+0000 7f06197ff640 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4d2cb0980 0x55c4d2cb2e40 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4538a1dc7d33ecd7 server_cookie=5def60e7a5ca2204 in_seq=0 out_seq=0 2024-03-11T19:54:26.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.391+0000 7f05fd7fa640 10 client.27766 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:26.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.395+0000 7f05fd7fa640 1 -- 172.21.15.83:0/4244767712 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c4d2bdcf40 con 0x55c4d2cb0980 2024-03-11T19:54:26.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.395+0000 7f05fd7fa640 10 client.27766 handle_command_reply: tid=0 2024-03-11T19:54:26.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.395+0000 7f061ee3a740 1 client.27766 shutdown 2024-03-11T19:54:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.405+0000 7f061ee3a740 1 -- 172.21.15.83:0/4244767712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 msgr2=0x55c4d2ca4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.405+0000 7f061ee3a740 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2ca4940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f060400c6a0 tx=0x7f060409b9c0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.406+0000 7f061ee3a740 1 -- 172.21.15.83:0/4244767712 shutdown_connections 2024-03-11T19:54:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.406+0000 7f061ee3a740 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4d2cb0980 0x55c4d2cb2e40 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.406+0000 7f061ee3a740 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9c220 0x55c4d2cabec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.406+0000 7f061ee3a740 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9b860 0x55c4d2ca4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.406+0000 7f061ee3a740 1 --2- 172.21.15.83:0/4244767712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2ca4940 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.406+0000 7f061ee3a740 1 -- 172.21.15.83:0/4244767712 >> 172.21.15.83:0/4244767712 conn(0x55c4d2c95870 msgr2=0x55c4d2c98ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.406+0000 7f05fd7fa640 0 client.27766 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.407+0000 7f061ee3a740 1 -- 172.21.15.83:0/4244767712 shutdown_connections 2024-03-11T19:54:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.408+0000 7f061ee3a740 1 -- 172.21.15.83:0/4244767712 wait complete. 2024-03-11T19:54:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.408+0000 7f061ee3a740 20 client.27766 trim_cache size 0 max 16384 2024-03-11T19:54:26.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.447+0000 7f061ee3a740 1 Processor -- start 2024-03-11T19:54:26.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.447+0000 7f061ee3a740 1 -- start start 2024-03-11T19:54:26.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.447+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2ca4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.447+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9b860 0x55c4d2cabe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.447+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9c220 0x55c4d2c991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.447+0000 7f061ee3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d2c90d50 con 0x55c4d2c9b860 2024-03-11T19:54:26.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.447+0000 7f061ee3a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d2c8a350 con 0x55c4d2c9c220 2024-03-11T19:54:26.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.447+0000 7f061ee3a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d2cb7e10 con 0x55c4d2c9a690 2024-03-11T19:54:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.449+0000 7f061a801640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9c220 0x55c4d2c991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.450+0000 7f061a801640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9c220 0x55c4d2c991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:43346/0 (socket says 172.21.15.83:43346) 2024-03-11T19:54:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.450+0000 7f061a801640 1 -- 172.21.15.83:0/2866362774 learned_addr learned my addr 172.21.15.83:0/2866362774 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.450+0000 7f061a801640 1 -- 172.21.15.83:0/2866362774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 msgr2=0x55c4d2ca4c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:54:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.450+0000 7f061a801640 1 --2- 172.21.15.83:0/2866362774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2ca4c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.450+0000 7f061a801640 1 -- 172.21.15.83:0/2866362774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9b860 msgr2=0x55c4d2cabe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.450+0000 7f061a801640 1 --2- 172.21.15.83:0/2866362774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9b860 0x55c4d2cabe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.450+0000 7f061a801640 1 -- 172.21.15.83:0/2866362774 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4d2b9dc00 con 0x55c4d2c9c220 2024-03-11T19:54:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.452+0000 7f061a801640 1 --2- 172.21.15.83:0/2866362774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9c220 0x55c4d2c991e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f060c009b30 tx=0x7f060c02c590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd41eb577080ec0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.453+0000 7f05ff7fe640 1 -- 172.21.15.83:0/2866362774 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c033b80 con 0x55c4d2c9c220 2024-03-11T19:54:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.453+0000 7f05ff7fe640 1 -- 172.21.15.83:0/2866362774 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f060c0094c0 con 0x55c4d2c9c220 2024-03-11T19:54:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.454+0000 7f061ee3a740 1 -- 172.21.15.83:0/2866362774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9c220 msgr2=0x55c4d2c991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.454+0000 7f061ee3a740 1 --2- 172.21.15.83:0/2866362774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9c220 0x55c4d2c991e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f060c009b30 tx=0x7f060c02c590 comp rx=0 tx=0).stop 2024-03-11T19:54:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.455+0000 7f061ee3a740 1 -- 172.21.15.83:0/2866362774 shutdown_connections 2024-03-11T19:54:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.455+0000 7f061ee3a740 1 --2- 172.21.15.83:0/2866362774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9c220 0x55c4d2c991e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.455+0000 7f061ee3a740 1 --2- 172.21.15.83:0/2866362774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9b860 0x55c4d2cabe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.455+0000 7f061ee3a740 1 --2- 172.21.15.83:0/2866362774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9a690 0x55c4d2ca4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.455+0000 7f061ee3a740 1 -- 172.21.15.83:0/2866362774 >> 172.21.15.83:0/2866362774 conn(0x55c4d2c95870 msgr2=0x55c4d2c98ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.457+0000 7f061ee3a740 1 -- 172.21.15.83:0/2866362774 shutdown_connections 2024-03-11T19:54:26.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.457+0000 7f061ee3a740 1 -- 172.21.15.83:0/2866362774 wait complete. 2024-03-11T19:54:26.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.457+0000 7f061ee3a740 1 -- start start 2024-03-11T19:54:26.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.457+0000 7f061ee3a740 1 Processor -- start 2024-03-11T19:54:26.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.458+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9a690 0x55c4d2caaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.458+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9b860 0x55c4d2cab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.458+0000 7f061ee3a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9c220 0x55c4d2d070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.459+0000 7f061ee3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d2d075f0 con 0x55c4d2c9b860 2024-03-11T19:54:26.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.459+0000 7f061ee3a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d2d07730 con 0x55c4d2c9a690 2024-03-11T19:54:26.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.459+0000 7f061ee3a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4d2d078a0 con 0x55c4d2c9c220 2024-03-11T19:54:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.459+0000 7f061a801640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9c220 0x55c4d2d070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.459+0000 7f061a801640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9c220 0x55c4d2d070b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41622/0 (socket says 172.21.15.83:41622) 2024-03-11T19:54:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.459+0000 7f061a801640 1 -- 172.21.15.83:0/2334061227 learned_addr learned my addr 172.21.15.83:0/2334061227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.459+0000 7f061ee3a740 10 client.0 _ll_register_callbacks cb 0x7f060c009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.460+0000 7f061a801640 1 -- 172.21.15.83:0/2334061227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9a690 msgr2=0x55c4d2caaff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-11T19:54:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.460+0000 7f061a801640 1 --2- 172.21.15.83:0/2334061227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9a690 0x55c4d2caaff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.460+0000 7f061a801640 1 -- 172.21.15.83:0/2334061227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9b860 msgr2=0x55c4d2cab530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.460+0000 7f061a801640 1 --2- 172.21.15.83:0/2334061227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9b860 0x55c4d2cab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.460+0000 7f061a801640 1 -- 172.21.15.83:0/2334061227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4d2bc50b0 con 0x55c4d2c9c220 2024-03-11T19:54:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.464+0000 7f061a801640 1 --2- 172.21.15.83:0/2334061227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9c220 0x55c4d2d070b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f060c004af0 tx=0x7f060c044a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16608da0e062ac25 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.464+0000 7f05fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-11T19:54:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.464+0000 7f05fd7fa640 1 -- 172.21.15.83:0/2334061227 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c066070 con 0x55c4d2c9c220 2024-03-11T19:54:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.464+0000 7f061ee3a740 1 -- 172.21.15.83:0/2334061227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4d2c28900 con 0x55c4d2c9c220 2024-03-11T19:54:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.465+0000 7f05fd7fa640 1 -- 172.21.15.83:0/2334061227 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f060c033e60 con 0x55c4d2c9c220 2024-03-11T19:54:26.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.465+0000 7f05fd7fa640 1 -- 172.21.15.83:0/2334061227 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f060c033b80 con 0x55c4d2c9c220 2024-03-11T19:54:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.467+0000 7f061ee3a740 10 client.27769 fetch_fsmap learned FSMap version 33 2024-03-11T19:54:26.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.467+0000 7f061ee3a740 1 -- 172.21.15.83:0/2334061227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55c4d2bc3910 con 0x55c4d2c9c220 2024-03-11T19:54:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.468+0000 7f05fd7fa640 1 -- 172.21.15.83:0/2334061227 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c4d2c28900 con 0x55c4d2c9c220 2024-03-11T19:54:26.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.468+0000 7f05fd7fa640 1 -- 172.21.15.83:0/2334061227 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 33) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f060c069030 con 0x55c4d2c9c220 2024-03-11T19:54:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.468+0000 7f061ee3a740 10 client.27769 fetch_fsmap finished waiting for FSMap version 33 2024-03-11T19:54:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.468+0000 7f061ee3a740 10 client.27769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.468+0000 7f061ee3a740 20 client.27769 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.468+0000 7f061ee3a740 1 --2- 172.21.15.83:0/2334061227 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4d2d08f70 0x55c4d2d0b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.468+0000 7f061ee3a740 4 client.27769 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-11T19:54:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.468+0000 7f061ee3a740 1 -- 172.21.15.83:0/2334061227 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c4d2bf9310 con 0x55c4d2d08f70 2024-03-11T19:54:26.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.468+0000 7f061a000640 1 --2- 172.21.15.83:0/2334061227 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4d2d08f70 0x55c4d2d0b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.469+0000 7f061a000640 1 --2- 172.21.15.83:0/2334061227 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4d2d08f70 0x55c4d2d0b390 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45db06f4c5e06e6d server_cookie=8346b21b1f959f01 in_seq=0 out_seq=0 2024-03-11T19:54:26.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.469+0000 7f05fd7fa640 10 client.27769 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:26.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.471+0000 7f05fd7fa640 1 -- 172.21.15.83:0/2334061227 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c4d2bf9310 con 0x55c4d2d08f70 2024-03-11T19:54:26.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.471+0000 7f05fd7fa640 10 client.27769 handle_command_reply: tid=0 2024-03-11T19:54:26.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.471+0000 7f061ee3a740 1 client.27769 shutdown 2024-03-11T19:54:26.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.477+0000 7f061ee3a740 1 -- 172.21.15.83:0/2334061227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9c220 msgr2=0x55c4d2d070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.477+0000 7f061ee3a740 1 --2- 172.21.15.83:0/2334061227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9c220 0x55c4d2d070b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f060c004af0 tx=0x7f060c044a60 comp rx=0 tx=0).stop 2024-03-11T19:54:26.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.478+0000 7f061ee3a740 1 -- 172.21.15.83:0/2334061227 shutdown_connections 2024-03-11T19:54:26.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.478+0000 7f061ee3a740 1 --2- 172.21.15.83:0/2334061227 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c4d2d08f70 0x55c4d2d0b390 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.478+0000 7f061ee3a740 1 --2- 172.21.15.83:0/2334061227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4d2c9c220 0x55c4d2d070b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.478+0000 7f061ee3a740 1 --2- 172.21.15.83:0/2334061227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c4d2c9b860 0x55c4d2cab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.478+0000 7f061ee3a740 1 --2- 172.21.15.83:0/2334061227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4d2c9a690 0x55c4d2caaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.478+0000 7f061ee3a740 1 -- 172.21.15.83:0/2334061227 >> 172.21.15.83:0/2334061227 conn(0x55c4d2c95870 msgr2=0x55c4d2c94c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.481+0000 7f05fd7fa640 0 client.27769 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:26.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.484+0000 7f061ee3a740 1 -- 172.21.15.83:0/2334061227 shutdown_connections 2024-03-11T19:54:26.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.484+0000 7f061ee3a740 1 -- 172.21.15.83:0/2334061227 wait complete. 2024-03-11T19:54:26.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.484+0000 7f061ee3a740 20 client.27769 trim_cache size 0 max 16384 2024-03-11T19:54:26.492 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:54:26.492 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-03-11T19:54:26.492 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "fffb2b2d-4d5a-48fd-8a4a-1cab517021b4", 2024-03-11T19:54:26.492 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-03-11T19:54:26.492 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:54:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.492+0000 7f061b002640 1 -- 172.21.15.83:0/3853956867 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f06000a1ba0 msgr2=0x7f06000a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.492+0000 7f061b002640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f06000a1ba0 0x7f06000a4060 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f060c039000 tx=0x7f060c049000 comp rx=0 tx=0).stop 2024-03-11T19:54:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.492+0000 7f061b002640 1 -- 172.21.15.83:0/3853956867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614109290 msgr2=0x7f061419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.492+0000 7f061b002640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614109290 0x7f061419d990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f060400e9c0 tx=0x7f060400ee80 comp rx=0 tx=0).stop 2024-03-11T19:54:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.493+0000 7f061b002640 1 -- 172.21.15.83:0/3853956867 shutdown_connections 2024-03-11T19:54:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.493+0000 7f061b002640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f06000a1ba0 0x7f06000a4060 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.493+0000 7f061b002640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061410e4f0 0x7f06141a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.493+0000 7f061b002640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061410bbc0 0x7f061419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.493+0000 7f061b002640 1 --2- 172.21.15.83:0/3853956867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614109290 0x7f061419d990 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.493+0000 7f061b002640 1 -- 172.21.15.83:0/3853956867 >> 172.21.15.83:0/3853956867 conn(0x7f061406e060 msgr2=0x7f0614109b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.493+0000 7f061b002640 1 -- 172.21.15.83:0/3853956867 shutdown_connections 2024-03-11T19:54:26.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.493+0000 7f061b002640 1 -- 172.21.15.83:0/3853956867 wait complete. 2024-03-11T19:54:26.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:54:26.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:54:26.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.707+0000 7f7a70df4640 1 -- 172.21.15.83:0/814513239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aaf0 msgr2=0x7f7a6c10cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.707+0000 7f7a70df4640 1 --2- 172.21.15.83:0/814513239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aaf0 0x7f7a6c10cf60 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009ca0 tx=0x7f7a6402c140 comp rx=0 tx=0).stop 2024-03-11T19:54:26.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.710+0000 7f7a70df4640 1 -- 172.21.15.83:0/814513239 shutdown_connections 2024-03-11T19:54:26.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.710+0000 7f7a70df4640 1 --2- 172.21.15.83:0/814513239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a6c10d4a0 0x7f7a6c10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.710+0000 7f7a70df4640 1 --2- 172.21.15.83:0/814513239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10aaf0 0x7f7a6c10cf60 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.710+0000 7f7a70df4640 1 --2- 172.21.15.83:0/814513239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a6c108180 0x7f7a6c10a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.710+0000 7f7a70df4640 1 -- 172.21.15.83:0/814513239 >> 172.21.15.83:0/814513239 conn(0x7f7a6c06e200 msgr2=0x7f7a6c070640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.710+0000 7f7a70df4640 1 -- 172.21.15.83:0/814513239 shutdown_connections 2024-03-11T19:54:26.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.710+0000 7f7a70df4640 1 -- 172.21.15.83:0/814513239 wait complete. 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.711+0000 7f7a70df4640 1 Processor -- start 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.711+0000 7f7a70df4640 1 -- start start 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.712+0000 7f7a70df4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a6c108180 0x7f7a6c114710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.712+0000 7f7a70df4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a6c10aaf0 0x7f7a6c114c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.712+0000 7f7a70df4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10d4a0 0x7f7a6c1a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.712+0000 7f7a70df4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c117cc0 con 0x7f7a6c10d4a0 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.712+0000 7f7a70df4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c117e30 con 0x7f7a6c108180 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.712+0000 7f7a70df4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c1a5750 con 0x7f7a6c10aaf0 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.712+0000 7f7a69d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a6c10aaf0 0x7f7a6c114c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.712+0000 7f7a69d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a6c10aaf0 0x7f7a6c114c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41636/0 (socket says 172.21.15.83:41636) 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.712+0000 7f7a69d9b640 1 -- 172.21.15.83:0/3502781597 learned_addr learned my addr 172.21.15.83:0/3502781597 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.712+0000 7f7a6a59c640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a6c108180 0x7f7a6c114710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.712+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10d4a0 0x7f7a6c1a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.713+0000 7f7a6ad9d640 1 -- 172.21.15.83:0/3502781597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a6c10aaf0 msgr2=0x7f7a6c114c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.713+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a6c10aaf0 0x7f7a6c114c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.713+0000 7f7a6ad9d640 1 -- 172.21.15.83:0/3502781597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a6c108180 msgr2=0x7f7a6c114710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.713+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a6c108180 0x7f7a6c114710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.713+0000 7f7a6ad9d640 1 -- 172.21.15.83:0/3502781597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a64009900 con 0x7f7a6c10d4a0 2024-03-11T19:54:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.713+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10d4a0 0x7f7a6c1a5210 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d680 tx=0x7f7a6000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.714+0000 7f7a5b7fe640 1 -- 172.21.15.83:0/3502781597 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60005220 con 0x7f7a6c10d4a0 2024-03-11T19:54:26.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.714+0000 7f7a5b7fe640 1 -- 172.21.15.83:0/3502781597 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a60009d00 con 0x7f7a6c10d4a0 2024-03-11T19:54:26.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.714+0000 7f7a70df4640 1 -- 172.21.15.83:0/3502781597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6c1a5a30 con 0x7f7a6c10d4a0 2024-03-11T19:54:26.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.715+0000 7f7a5b7fe640 1 -- 172.21.15.83:0/3502781597 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60004ea0 con 0x7f7a6c10d4a0 2024-03-11T19:54:26.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.716+0000 7f7a70df4640 1 -- 172.21.15.83:0/3502781597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6c1a5f40 con 0x7f7a6c10d4a0 2024-03-11T19:54:26.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.720+0000 7f7a5b7fe640 1 -- 172.21.15.83:0/3502781597 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f7a60005380 con 0x7f7a6c10d4a0 2024-03-11T19:54:26.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.721+0000 7f7a5b7fe640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a540a1aa0 0x7f7a540a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.721+0000 7f7a5b7fe640 1 -- 172.21.15.83:0/3502781597 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7a600cced0 con 0x7f7a6c10d4a0 2024-03-11T19:54:26.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.722+0000 7f7a6a59c640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a540a1aa0 0x7f7a540a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.723+0000 7f7a739c8740 1 Processor -- start 2024-03-11T19:54:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.724+0000 7f7a739c8740 1 -- start start 2024-03-11T19:54:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.724+0000 7f7a6a59c640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a540a1aa0 0x7f7a540a3f60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c115c10 tx=0x7f7a5c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.724+0000 7f7a739c8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e32542d0 0x5637e32546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.724+0000 7f7a739c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 0x5637e32597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.724+0000 7f7a739c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e325c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.724+0000 7f7a739c8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e31c4280 con 0x5637e32542d0 2024-03-11T19:54:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.724+0000 7f7a739c8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e31c17b0 con 0x5637e3259e30 2024-03-11T19:54:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.724+0000 7f7a739c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e31bcf90 con 0x5637e3254cc0 2024-03-11T19:54:26.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.724+0000 7f7a69d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 0x5637e32597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.724+0000 7f7a69d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 0x5637e32597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41648/0 (socket says 172.21.15.83:41648) 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.724+0000 7f7a69d9b640 1 -- 172.21.15.83:0/4001019481 learned_addr learned my addr 172.21.15.83:0/4001019481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.725+0000 7f7a6a59c640 1 --2- 172.21.15.83:0/4001019481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e32542d0 0x5637e32546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.725+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/4001019481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e325c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.725+0000 7f7a69d9b640 1 -- 172.21.15.83:0/4001019481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 msgr2=0x5637e325c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.725+0000 7f7a69d9b640 1 --2- 172.21.15.83:0/4001019481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e325c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.725+0000 7f7a69d9b640 1 -- 172.21.15.83:0/4001019481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e32542d0 msgr2=0x5637e32546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.725+0000 7f7a69d9b640 1 --2- 172.21.15.83:0/4001019481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e32542d0 0x5637e32546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.725+0000 7f7a69d9b640 1 -- 172.21.15.83:0/4001019481 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637e317b570 con 0x5637e3254cc0 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.725+0000 7f7a69d9b640 1 --2- 172.21.15.83:0/4001019481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 0x5637e32597c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009dc0 tx=0x7f7a640323f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ee7326d3da40419 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.725+0000 7f7a70df4640 1 -- 172.21.15.83:0/4001019481 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a640326c0 con 0x5637e3254cc0 2024-03-11T19:54:26.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.726+0000 7f7a70df4640 1 -- 172.21.15.83:0/4001019481 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a64032820 con 0x5637e3254cc0 2024-03-11T19:54:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.727+0000 7f7a739c8740 1 -- 172.21.15.83:0/4001019481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 msgr2=0x5637e32597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.727+0000 7f7a739c8740 1 --2- 172.21.15.83:0/4001019481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 0x5637e32597c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009dc0 tx=0x7f7a640323f0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.729+0000 7f7a739c8740 1 -- 172.21.15.83:0/4001019481 shutdown_connections 2024-03-11T19:54:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.729+0000 7f7a739c8740 1 --2- 172.21.15.83:0/4001019481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e325c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.729+0000 7f7a739c8740 1 --2- 172.21.15.83:0/4001019481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 0x5637e32597c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.729+0000 7f7a739c8740 1 --2- 172.21.15.83:0/4001019481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e32542d0 0x5637e32546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.729+0000 7f7a739c8740 1 -- 172.21.15.83:0/4001019481 >> 172.21.15.83:0/4001019481 conn(0x5637e324f9e0 msgr2=0x5637e324fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.729+0000 7f7a739c8740 1 -- 172.21.15.83:0/4001019481 shutdown_connections 2024-03-11T19:54:26.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.732+0000 7f7a739c8740 1 -- 172.21.15.83:0/4001019481 wait complete. 2024-03-11T19:54:26.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.732+0000 7f7a739c8740 1 -- start start 2024-03-11T19:54:26.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.732+0000 7f7a739c8740 1 Processor -- start 2024-03-11T19:54:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.734+0000 7f7a739c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 0x5637e325e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.734+0000 7f7a739c8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e3254cc0 0x5637e325e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.734+0000 7f7a739c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e3265770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.734+0000 7f7a739c8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e2e90430 con 0x5637e3254cc0 2024-03-11T19:54:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.734+0000 7f7a739c8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e2e4ce00 con 0x5637e3259e30 2024-03-11T19:54:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.734+0000 7f7a739c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e3141fd0 con 0x5637e32542d0 2024-03-11T19:54:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.734+0000 7f7a739c8740 10 client.0 _ll_register_callbacks cb 0x5637e3197fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.734+0000 7f7a6a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 0x5637e325e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.734+0000 7f7a6ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e3265770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.734+0000 7f7a6ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e3265770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.83:43390/0 (socket says 172.21.15.83:43390) 2024-03-11T19:54:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.734+0000 7f7a6ad9d640 1 -- 172.21.15.83:0/3255537306 learned_addr learned my addr 172.21.15.83:0/3255537306 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.736+0000 7f7a6ad9d640 1 -- 172.21.15.83:0/3255537306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 msgr2=0x5637e325e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.736+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/3255537306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 0x5637e325e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.736+0000 7f7a6ad9d640 1 -- 172.21.15.83:0/3255537306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e3254cc0 msgr2=0x5637e325e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.736+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/3255537306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e3254cc0 0x5637e325e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.736+0000 7f7a6ad9d640 1 -- 172.21.15.83:0/3255537306 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6c10e460 con 0x5637e3259e30 2024-03-11T19:54:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.737+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/3255537306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e3265770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d650 tx=0x7f7a6009a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c794851bb10ee9a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.737+0000 7f7a51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:54:26.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.739+0000 7f7a51ffb640 1 -- 172.21.15.83:0/3255537306 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a600a2670 con 0x5637e3259e30 2024-03-11T19:54:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.739+0000 7f7a739c8740 1 -- 172.21.15.83:0/3255537306 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637e2ebc7e0 con 0x5637e3259e30 2024-03-11T19:54:26.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.739+0000 7f7a51ffb640 1 -- 172.21.15.83:0/3255537306 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7a60004620 con 0x5637e3259e30 2024-03-11T19:54:26.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.741+0000 7f7a51ffb640 1 -- 172.21.15.83:0/3255537306 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a60096540 con 0x5637e3259e30 2024-03-11T19:54:26.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.741+0000 7f7a739c8740 10 client.27671 fetch_fsmap learned FSMap version 33 2024-03-11T19:54:26.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.741+0000 7f7a739c8740 1 -- 172.21.15.83:0/3255537306 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5637e31bb610 con 0x5637e3259e30 2024-03-11T19:54:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.744+0000 7f7a51ffb640 1 -- 172.21.15.83:0/3255537306 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5637e2ebc7e0 con 0x5637e3259e30 2024-03-11T19:54:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.745+0000 7f7a51ffb640 1 -- 172.21.15.83:0/3255537306 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 33) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7a60009850 con 0x5637e3259e30 2024-03-11T19:54:26.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.745+0000 7f7a739c8740 10 client.27671 fetch_fsmap finished waiting for FSMap version 33 2024-03-11T19:54:26.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.745+0000 7f7a739c8740 10 client.27671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:26.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.745+0000 7f7a739c8740 20 client.27671 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:26.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.745+0000 7f7a739c8740 1 --2- 172.21.15.83:0/3255537306 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5637e326a3a0 0x5637e326c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.746+0000 7f7a739c8740 4 client.27671 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:26.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.746+0000 7f7a739c8740 1 -- 172.21.15.83:0/3255537306 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637e2ef9720 con 0x5637e326a3a0 2024-03-11T19:54:26.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.746+0000 7f7a6a59c640 1 --2- 172.21.15.83:0/3255537306 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5637e326a3a0 0x5637e326c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.747+0000 7f7a6a59c640 1 --2- 172.21.15.83:0/3255537306 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5637e326a3a0 0x5637e326c7c0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c147a38d0769c481 server_cookie=eb014cdb18b80b7f in_seq=0 out_seq=0 2024-03-11T19:54:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.747+0000 7f7a51ffb640 10 client.27671 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.749+0000 7f7a51ffb640 1 -- 172.21.15.83:0/3255537306 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5637e2ef9720 con 0x5637e326a3a0 2024-03-11T19:54:26.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.749+0000 7f7a51ffb640 10 client.27671 handle_command_reply: tid=0 2024-03-11T19:54:26.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.749+0000 7f7a739c8740 1 client.27671 shutdown 2024-03-11T19:54:26.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.756+0000 7f7a739c8740 1 -- 172.21.15.83:0/3255537306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 msgr2=0x5637e3265770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.756+0000 7f7a739c8740 1 --2- 172.21.15.83:0/3255537306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e3265770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d650 tx=0x7f7a6009a5d0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.758+0000 7f7a739c8740 1 -- 172.21.15.83:0/3255537306 shutdown_connections 2024-03-11T19:54:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.758+0000 7f7a739c8740 1 --2- 172.21.15.83:0/3255537306 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5637e326a3a0 0x5637e326c7c0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.758+0000 7f7a739c8740 1 --2- 172.21.15.83:0/3255537306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e3265770 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.758+0000 7f7a739c8740 1 --2- 172.21.15.83:0/3255537306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e3254cc0 0x5637e325e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.758+0000 7f7a739c8740 1 --2- 172.21.15.83:0/3255537306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 0x5637e325e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.758+0000 7f7a739c8740 1 -- 172.21.15.83:0/3255537306 >> 172.21.15.83:0/3255537306 conn(0x5637e324f9e0 msgr2=0x5637e324ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.758+0000 7f7a51ffb640 0 client.27671 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:26.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.766+0000 7f7a739c8740 1 -- 172.21.15.83:0/3255537306 shutdown_connections 2024-03-11T19:54:26.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.766+0000 7f7a739c8740 1 -- 172.21.15.83:0/3255537306 wait complete. 2024-03-11T19:54:26.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.766+0000 7f7a739c8740 20 client.27671 trim_cache size 0 max 16384 2024-03-11T19:54:26.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.809+0000 7f7a739c8740 1 Processor -- start 2024-03-11T19:54:26.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.809+0000 7f7a739c8740 1 -- start start 2024-03-11T19:54:26.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.809+0000 7f7a739c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 0x5637e325e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.810+0000 7f7a739c8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e3254cc0 0x5637e325e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.810+0000 7f7a6a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 0x5637e325e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.810+0000 7f7a6a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 0x5637e325e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41670/0 (socket says 172.21.15.83:41670) 2024-03-11T19:54:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.810+0000 7f7a6a59c640 1 -- 172.21.15.83:0/3712662555 learned_addr learned my addr 172.21.15.83:0/3712662555 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.810+0000 7f7a739c8740 1 --2- 172.21.15.83:0/3712662555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e3265730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.810+0000 7f7a739c8740 1 -- 172.21.15.83:0/3712662555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e2ef9720 con 0x5637e3254cc0 2024-03-11T19:54:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.810+0000 7f7a739c8740 1 -- 172.21.15.83:0/3712662555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e2ebc7e0 con 0x5637e3259e30 2024-03-11T19:54:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.810+0000 7f7a739c8740 1 -- 172.21.15.83:0/3712662555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e325f0f0 con 0x5637e32542d0 2024-03-11T19:54:26.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.811+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/3712662555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e3265730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.811+0000 7f7a6a59c640 1 -- 172.21.15.83:0/3712662555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 msgr2=0x5637e3265730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.811+0000 7f7a6a59c640 1 --2- 172.21.15.83:0/3712662555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e3265730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.811+0000 7f7a6a59c640 1 -- 172.21.15.83:0/3712662555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e3254cc0 msgr2=0x5637e325e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.811+0000 7f7a6a59c640 1 --2- 172.21.15.83:0/3712662555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e3254cc0 0x5637e325e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.811+0000 7f7a6a59c640 1 -- 172.21.15.83:0/3712662555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637e3157ac0 con 0x5637e32542d0 2024-03-11T19:54:26.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.811+0000 7f7a6a59c640 1 --2- 172.21.15.83:0/3712662555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 0x5637e325e1f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0058c0 tx=0x7f7a5c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b900238a8c3cf07d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.813+0000 7f7a53fff640 1 -- 172.21.15.83:0/3712662555 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c024040 con 0x5637e32542d0 2024-03-11T19:54:26.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.813+0000 7f7a53fff640 1 -- 172.21.15.83:0/3712662555 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a5c027830 con 0x5637e32542d0 2024-03-11T19:54:26.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.813+0000 7f7a53fff640 1 -- 172.21.15.83:0/3712662555 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c005680 con 0x5637e32542d0 2024-03-11T19:54:26.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.814+0000 7f7a739c8740 1 -- 172.21.15.83:0/3712662555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 msgr2=0x5637e325e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.814+0000 7f7a739c8740 1 --2- 172.21.15.83:0/3712662555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 0x5637e325e1f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0058c0 tx=0x7f7a5c014000 comp rx=0 tx=0).stop 2024-03-11T19:54:26.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.814+0000 7f7a739c8740 1 -- 172.21.15.83:0/3712662555 shutdown_connections 2024-03-11T19:54:26.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.814+0000 7f7a739c8740 1 --2- 172.21.15.83:0/3712662555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e3265730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.814+0000 7f7a739c8740 1 --2- 172.21.15.83:0/3712662555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e3254cc0 0x5637e325e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.814+0000 7f7a739c8740 1 --2- 172.21.15.83:0/3712662555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e32542d0 0x5637e325e1f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.814+0000 7f7a739c8740 1 -- 172.21.15.83:0/3712662555 >> 172.21.15.83:0/3712662555 conn(0x5637e324f9e0 msgr2=0x5637e324a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:26 smithi083 ceph-mon[31864]: pgmap v805: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-03-11T19:54:26.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:26 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:26.819 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:26 smithi083 ceph-mon[31864]: scrub queued for path: / 2024-03-11T19:54:26.820 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:26 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:26.820 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:26 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:54:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.819+0000 7f7a739c8740 1 -- 172.21.15.83:0/3712662555 shutdown_connections 2024-03-11T19:54:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.820+0000 7f7a739c8740 1 -- 172.21.15.83:0/3712662555 wait complete. 2024-03-11T19:54:26.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.820+0000 7f7a739c8740 1 -- start start 2024-03-11T19:54:26.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.821+0000 7f7a739c8740 1 Processor -- start 2024-03-11T19:54:26.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.821+0000 7f7a739c8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e32542d0 0x5637e32655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.821+0000 7f7a739c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 0x5637e3263bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.821+0000 7f7a739c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e32640f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.822+0000 7f7a739c8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e3264750 con 0x5637e32542d0 2024-03-11T19:54:26.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.822+0000 7f7a69d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 0x5637e3263bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.822+0000 7f7a69d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 0x5637e3263bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.83:41676/0 (socket says 172.21.15.83:41676) 2024-03-11T19:54:26.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.822+0000 7f7a69d9b640 1 -- 172.21.15.83:0/2254404679 learned_addr learned my addr 172.21.15.83:0/2254404679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:26.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.822+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e32640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.822+0000 7f7a739c8740 1 -- 172.21.15.83:0/2254404679 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e32648c0 con 0x5637e3259e30 2024-03-11T19:54:26.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.822+0000 7f7a739c8740 1 -- 172.21.15.83:0/2254404679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637e3264a80 con 0x5637e3254cc0 2024-03-11T19:54:26.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.822+0000 7f7a739c8740 10 client.0 _ll_register_callbacks cb 0x7f7a64009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:26.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.824+0000 7f7a6ad9d640 1 -- 172.21.15.83:0/2254404679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 msgr2=0x5637e3263bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.824+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 0x5637e3263bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.824+0000 7f7a6ad9d640 1 -- 172.21.15.83:0/2254404679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e32542d0 msgr2=0x5637e32655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:26.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.824+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e32542d0 0x5637e32655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.824+0000 7f7a6ad9d640 1 -- 172.21.15.83:0/2254404679 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c00ee60 con 0x5637e3259e30 2024-03-11T19:54:26.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.825+0000 7f7a6ad9d640 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e32640f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7a600968a0 tx=0x7f7a60099db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e3bb3d7aa4a99a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:26.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.825+0000 7f7a51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-11T19:54:26.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.826+0000 7f7a51ffb640 1 -- 172.21.15.83:0/2254404679 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60096170 con 0x5637e3259e30 2024-03-11T19:54:26.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.826+0000 7f7a739c8740 1 -- 172.21.15.83:0/2254404679 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a64032820 con 0x5637e3259e30 2024-03-11T19:54:26.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.826+0000 7f7a51ffb640 1 -- 172.21.15.83:0/2254404679 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f7a60015070 con 0x5637e3259e30 2024-03-11T19:54:26.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.826+0000 7f7a51ffb640 1 -- 172.21.15.83:0/2254404679 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a600a2670 con 0x5637e3259e30 2024-03-11T19:54:26.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.826+0000 7f7a739c8740 10 client.27677 fetch_fsmap learned FSMap version 33 2024-03-11T19:54:26.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.826+0000 7f7a739c8740 1 -- 172.21.15.83:0/2254404679 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5637e317d6a0 con 0x5637e3259e30 2024-03-11T19:54:26.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.827+0000 7f7a51ffb640 1 -- 172.21.15.83:0/2254404679 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a64032820 con 0x5637e3259e30 2024-03-11T19:54:26.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.828+0000 7f7a51ffb640 1 -- 172.21.15.83:0/2254404679 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 33) v1 ==== 7886+0+0 (secure 0 0 0) 0x7f7a6009ef40 con 0x5637e3259e30 2024-03-11T19:54:26.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.828+0000 7f7a739c8740 10 client.27677 fetch_fsmap finished waiting for FSMap version 33 2024-03-11T19:54:26.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.828+0000 7f7a739c8740 10 client.27677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:26.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.828+0000 7f7a739c8740 20 client.27677 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:26.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.828+0000 7f7a739c8740 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5637e32c1da0 0x5637e32c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:26.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.828+0000 7f7a739c8740 4 client.27677 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:54:26.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.828+0000 7f7a739c8740 1 -- 172.21.15.83:0/2254404679 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5637e2f48890 con 0x5637e32c1da0 2024-03-11T19:54:26.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.829+0000 7f7a6a59c640 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5637e32c1da0 0x5637e32c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:26.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.830+0000 7f7a6a59c640 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5637e32c1da0 0x5637e32c41c0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa8f4747bf1acac server_cookie=e9ceb322db6e7315 in_seq=0 out_seq=0 2024-03-11T19:54:26.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.830+0000 7f7a51ffb640 10 client.27677 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:26.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.830+0000 7f7a51ffb640 1 -- 172.21.15.83:0/2254404679 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x5637e2f48890 con 0x5637e32c1da0 2024-03-11T19:54:26.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.830+0000 7f7a51ffb640 10 client.27677 handle_command_reply: tid=0 2024-03-11T19:54:26.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.830+0000 7f7a739c8740 1 client.27677 shutdown 2024-03-11T19:54:26.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.831+0000 7f7a739c8740 1 -- 172.21.15.83:0/2254404679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 msgr2=0x5637e32640f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.831+0000 7f7a739c8740 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e32640f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7a600968a0 tx=0x7f7a60099db0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.831+0000 7f7a739c8740 1 -- 172.21.15.83:0/2254404679 shutdown_connections 2024-03-11T19:54:26.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.831+0000 7f7a739c8740 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5637e32c1da0 0x5637e32c41c0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.831+0000 7f7a739c8740 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5637e3259e30 0x5637e32640f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.831+0000 7f7a739c8740 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5637e3254cc0 0x5637e3263bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.831+0000 7f7a739c8740 1 --2- 172.21.15.83:0/2254404679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637e32542d0 0x5637e32655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.831+0000 7f7a739c8740 1 -- 172.21.15.83:0/2254404679 >> 172.21.15.83:0/2254404679 conn(0x5637e324f9e0 msgr2=0x5637e3255e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.831+0000 7f7a51ffb640 0 client.27677 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:26.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.831+0000 7f7a739c8740 1 -- 172.21.15.83:0/2254404679 shutdown_connections 2024-03-11T19:54:26.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.831+0000 7f7a739c8740 1 -- 172.21.15.83:0/2254404679 wait complete. 2024-03-11T19:54:26.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.832+0000 7f7a739c8740 20 client.27677 trim_cache size 0 max 16384 2024-03-11T19:54:26.840 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:54:26.840 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (15 inodes in the stack)", 2024-03-11T19:54:26.840 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-03-11T19:54:26.840 INFO:teuthology.orchestra.run.smithi083.stdout: "fffb2b2d-4d5a-48fd-8a4a-1cab517021b4": { 2024-03-11T19:54:26.840 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-03-11T19:54:26.840 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "fffb2b2d-4d5a-48fd-8a4a-1cab517021b4", 2024-03-11T19:54:26.840 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-03-11T19:54:26.840 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:54:26.840 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:54:26.840 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:54:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 -- 172.21.15.83:0/3502781597 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a540a1aa0 msgr2=0x7f7a540a3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a540a1aa0 0x7f7a540a3f60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c115c10 tx=0x7f7a5c00a4c0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 -- 172.21.15.83:0/3502781597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10d4a0 msgr2=0x7f7a6c1a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10d4a0 0x7f7a6c1a5210 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d680 tx=0x7f7a6000db40 comp rx=0 tx=0).stop 2024-03-11T19:54:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 -- 172.21.15.83:0/3502781597 shutdown_connections 2024-03-11T19:54:26.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f7a540a1aa0 0x7f7a540a3f60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a6c10d4a0 0x7f7a6c1a5210 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a6c10aaf0 0x7f7a6c114c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 --2- 172.21.15.83:0/3502781597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a6c108180 0x7f7a6c114710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:26.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 -- 172.21.15.83:0/3502781597 >> 172.21.15.83:0/3502781597 conn(0x7f7a6c06e200 msgr2=0x7f7a6c10a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:26.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 -- 172.21.15.83:0/3502781597 shutdown_connections 2024-03-11T19:54:26.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:26.838+0000 7f7a597fa640 1 -- 172.21.15.83:0/3502781597 wait complete. 2024-03-11T19:54:26.866 INFO:tasks.cephfs.filesystem:scrub status for tag:fffb2b2d-4d5a-48fd-8a4a-1cab517021b4 - {'path': '/', 'tag': 'fffb2b2d-4d5a-48fd-8a4a-1cab517021b4', 'options': 'recursive,force'} 2024-03-11T19:54:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:26 smithi137 ceph-mon[36220]: pgmap v805: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-03-11T19:54:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:26 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:26 smithi137 ceph-mon[36220]: scrub queued for path: / 2024-03-11T19:54:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:26 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:27.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:26 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:54:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:26 smithi204 ceph-mon[36739]: pgmap v805: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-03-11T19:54:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:26 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:27.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:26 smithi204 ceph-mon[36739]: scrub queued for path: / 2024-03-11T19:54:27.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:26 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:27.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:26 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:54:27.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-11T19:54:27.729 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/: 2024-03-11T19:54:27.729 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.729 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 . 2024-03-11T19:54:27.729 INFO:teuthology.orchestra.run.smithi083.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 .. 2024-03-11T19:54:27.729 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 data 2024-03-11T19:54:27.729 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.729 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data: 2024-03-11T19:54:27.729 INFO:teuthology.orchestra.run.smithi083.stdout:total 36 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 . 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 .. 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 11 19:39 base 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Mar 11 19:39 current_logfiles 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Mar 11 19:39 global 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 log 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_commit_ts 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_dynshmem 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Mar 11 19:39 pg_hba.conf 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Mar 11 19:39 pg_ident.conf 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:54 pg_logical 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 11 19:39 pg_multixact 2024-03-11T19:54:27.730 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_notify 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_replslot 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_serial 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_snapshots 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:54 pg_stat 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:54 pg_stat_tmp 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:54 pg_subtrans 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_tblspc 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pg_twophase 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 PG_VERSION 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Mar 11 19:54 pg_wal 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:52 pg_xact 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Mar 11 19:39 postgresql.auto.conf 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Mar 11 19:39 postgresql.conf 2024-03-11T19:54:27.731 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Mar 11 19:39 postmaster.opts 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base: 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 11 19:39 . 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 11 19:39 1 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 11 19:39 13493 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Mar 11 19:41 13494 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 pgsql_tmp 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/1: 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:total 7777 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 11 19:39 . 2024-03-11T19:54:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 11 19:39 .. 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 112 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 113 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 1247 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1247_fsm 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1247_vm 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 11 19:39 1249 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1249_fsm 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1249_vm 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 11 19:39 1255 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1255_fsm 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1255_vm 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 11 19:39 1259 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1259_fsm 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1259_vm 2024-03-11T19:54:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 11 19:39 13344 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13344_fsm 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13344_vm 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13346 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13348 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13349_fsm 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349_vm 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13351 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13353 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13354_fsm 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354_vm 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13356 2024-03-11T19:54:27.734 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13358 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13359_fsm 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359_vm 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13361 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13363 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1417 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1418 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 174 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 175 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2187 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2224 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2328 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2336 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2337 2024-03-11T19:54:27.735 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2579 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2600 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2600_fsm 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2600_vm 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2601_fsm 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601_vm 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2602 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2602_fsm 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2602_vm 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2603 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2603_fsm 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2603_vm 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2604 2024-03-11T19:54:27.736 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2605 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2605_fsm 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2605_vm 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2606_fsm 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606_vm 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2607 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2607_fsm 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2607_vm 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 11 19:39 2608 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2608_fsm 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2608_vm 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 11 19:39 2609 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2609_fsm 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2609_vm 2024-03-11T19:54:27.737 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2610 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2610_fsm 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2610_vm 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2611 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2612_fsm 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612_vm 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2613 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2615_fsm 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615_vm 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2616 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2616_fsm 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2616_vm 2024-03-11T19:54:27.738 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 11 19:39 2617 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2617_fsm 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2617_vm 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 11 19:39 2618 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2618_fsm 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2618_vm 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 11 19:39 2619 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2619_fsm 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2619_vm 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2620 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2650 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2651 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2652 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2653 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2654 2024-03-11T19:54:27.739 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2655 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2656 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2657 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 11 19:39 2658 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 2659 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2660 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2661 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2662 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2663 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2664 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2665 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2666 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2667 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2668 2024-03-11T19:54:27.740 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2669 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2670 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 11 19:39 2673 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 11 19:39 2674 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 11 19:39 2675 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2678 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2679 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2680 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2681 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2682 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2683 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2684 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2685 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2686 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2687 2024-03-11T19:54:27.741 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2688 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2689 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 11 19:39 2690 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 11 19:39 2691 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2692 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2693 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2696 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2699 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2701 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2702 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2703 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2704 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2753 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2753_fsm 2024-03-11T19:54:27.742 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2753_vm 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2754 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2755 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2756 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2757 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2830 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2831 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2832 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2833 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2834 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2835 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2836 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2837 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 11 19:39 2838 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2838_fsm 2024-03-11T19:54:27.743 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2838_vm 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2839 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840_fsm 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2840_vm 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2841 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2995 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2996 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3079_fsm 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079_vm 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3080 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3081 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3085 2024-03-11T19:54:27.744 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3118 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3119 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 11 19:39 3164 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3256 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3257 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3258 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3350 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3351 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3379 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3380 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3381 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3394 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3394_fsm 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3394_vm 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3395 2024-03-11T19:54:27.745 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3429 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3430 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3431 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3433 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3439 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3440 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3455 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 11 19:39 3456 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3456_fsm 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3456_vm 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3466 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3467 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3468 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3501 2024-03-11T19:54:27.746 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3502 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3503 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3534 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3541_fsm 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541_vm 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3542 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3574 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3575 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3576 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3596 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3597 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3598 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3599 2024-03-11T19:54:27.747 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3600_fsm 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600_vm 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3601_fsm 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601_vm 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3602_fsm 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602_vm 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603_fsm 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3603_vm 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3604 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3605 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3606 2024-03-11T19:54:27.748 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3607 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3608 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 3609 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3712 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3764_fsm 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764_vm 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3766 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3767 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3997 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4143 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4144 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4145 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4146 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4147 2024-03-11T19:54:27.749 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4148 2024-03-11T19:54:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4149 2024-03-11T19:54:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4150 2024-03-11T19:54:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4151 2024-03-11T19:54:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4152 2024-03-11T19:54:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4153 2024-03-11T19:54:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4154 2024-03-11T19:54:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4155 2024-03-11T19:54:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4156 2024-03-11T19:54:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4157 2024-03-11T19:54:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4158 2024-03-11T19:54:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4159 2024-03-11T19:54:27.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4160 2024-03-11T19:54:27.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4161 2024-03-11T19:54:27.751 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4162 2024-03-11T19:54:27.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4163 2024-03-11T19:54:27.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4164 2024-03-11T19:54:27.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4165 2024-03-11T19:54:27.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4166 2024-03-11T19:54:27.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4167 2024-03-11T19:54:27.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4168 2024-03-11T19:54:27.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4169 2024-03-11T19:54:27.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4170 2024-03-11T19:54:27.752 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4171 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4172 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4173 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4174 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 5002 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 548 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 549 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6102 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6104 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6106 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6110 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6111 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6112 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6113 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6117 2024-03-11T19:54:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 826 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 827 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 828 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 11 19:39 pg_filenode.map 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 PG_VERSION 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:total 7777 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 11 19:39 . 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 11 19:39 .. 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 112 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 113 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 1247 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1247_fsm 2024-03-11T19:54:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1247_vm 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 11 19:39 1249 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1249_fsm 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1249_vm 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 11 19:39 1255 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1255_fsm 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1255_vm 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 11 19:39 1259 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1259_fsm 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1259_vm 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 11 19:39 13344 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13344_fsm 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13344_vm 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13346 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13348 2024-03-11T19:54:27.755 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13349_fsm 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349_vm 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13351 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13353 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13354_fsm 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354_vm 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13356 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13358 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13359_fsm 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359_vm 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13361 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13363 2024-03-11T19:54:27.756 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1417 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1418 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 174 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 175 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2187 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2224 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2328 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2336 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2337 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2579 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2600 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2600_fsm 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2600_vm 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2601_fsm 2024-03-11T19:54:27.757 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601_vm 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2602 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2602_fsm 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2602_vm 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2603 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2603_fsm 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2603_vm 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2604 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2605 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2605_fsm 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2605_vm 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2606_fsm 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606_vm 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2607 2024-03-11T19:54:27.758 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2607_fsm 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2607_vm 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 11 19:39 2608 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2608_fsm 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2608_vm 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 11 19:39 2609 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2609_fsm 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2609_vm 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2610 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2610_fsm 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2610_vm 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2611 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2612_fsm 2024-03-11T19:54:27.759 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612_vm 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2613 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2615_fsm 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615_vm 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2616 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2616_fsm 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2616_vm 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 11 19:39 2617 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2617_fsm 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2617_vm 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 11 19:39 2618 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2618_fsm 2024-03-11T19:54:27.760 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2618_vm 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 11 19:39 2619 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2619_fsm 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2619_vm 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2620 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2650 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2651 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2652 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2653 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2654 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2655 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2656 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2657 2024-03-11T19:54:27.761 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 11 19:39 2658 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 2659 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2660 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2661 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2662 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2663 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2664 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2665 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2666 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2667 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2668 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2669 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2670 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 11 19:39 2673 2024-03-11T19:54:27.762 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 11 19:39 2674 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 11 19:39 2675 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2678 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2679 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2680 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2681 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2682 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2683 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2684 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2685 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2686 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2687 2024-03-11T19:54:27.763 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2688 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2689 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 11 19:39 2690 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 11 19:39 2691 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2692 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2693 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2696 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2699 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2701 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2702 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2703 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2704 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2753 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2753_fsm 2024-03-11T19:54:27.764 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2753_vm 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2754 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2755 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2756 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2757 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2830 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2831 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2832 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2833 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2834 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2835 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2836 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2837 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 11 19:39 2838 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2838_fsm 2024-03-11T19:54:27.765 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2838_vm 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2839 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840_fsm 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2840_vm 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2841 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2995 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2996 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3079_fsm 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079_vm 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3080 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3081 2024-03-11T19:54:27.766 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3085 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3118 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3119 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 11 19:39 3164 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3256 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3257 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3258 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3350 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3351 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3379 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3380 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3381 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3394 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3394_fsm 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3394_vm 2024-03-11T19:54:27.767 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3395 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3429 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3430 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3431 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3433 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3439 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3440 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3455 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 11 19:39 3456 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3456_fsm 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3456_vm 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3466 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3467 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3468 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3501 2024-03-11T19:54:27.768 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3502 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3503 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3534 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3541_fsm 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541_vm 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3542 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3574 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3575 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3576 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3596 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3597 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3598 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3599 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600 2024-03-11T19:54:27.769 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3600_fsm 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600_vm 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3601_fsm 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601_vm 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3602_fsm 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602_vm 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603_fsm 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3603_vm 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3604 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3605 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3606 2024-03-11T19:54:27.770 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3607 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3608 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 3609 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3712 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3764_fsm 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764_vm 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3766 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3767 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3997 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4143 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4144 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4145 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4146 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4147 2024-03-11T19:54:27.771 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4148 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4149 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4150 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4151 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4152 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4153 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4154 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4155 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4156 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4157 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4158 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4159 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4160 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4161 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4162 2024-03-11T19:54:27.772 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4163 2024-03-11T19:54:27.773 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4164 2024-03-11T19:54:27.773 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4165 2024-03-11T19:54:27.773 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4166 2024-03-11T19:54:27.773 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4167 2024-03-11T19:54:27.773 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4168 2024-03-11T19:54:27.773 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4169 2024-03-11T19:54:27.773 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4170 2024-03-11T19:54:27.773 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4171 2024-03-11T19:54:27.773 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4172 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4173 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4174 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 5002 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 548 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 549 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6102 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6104 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6106 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6110 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6111 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6112 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6113 2024-03-11T19:54:27.776 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6117 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 826 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 827 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 828 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 11 19:39 pg_filenode.map 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 PG_VERSION 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:total 662230 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Mar 11 19:41 . 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 11 19:39 .. 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 112 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 113 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 1247 2024-03-11T19:54:27.777 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1247_fsm 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1247_vm 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 11 19:39 1249 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1249_fsm 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1249_vm 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 11 19:39 1255 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1255_fsm 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1255_vm 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 11 19:53 1259 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1259_fsm 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1259_vm 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 11 19:39 13344 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13344_fsm 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13344_vm 2024-03-11T19:54:27.778 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13346 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13348 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13349_fsm 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13349_vm 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13351 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13353 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13354_fsm 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13354_vm 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13356 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13358 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 13359_fsm 2024-03-11T19:54:27.779 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13359_vm 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 13361 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 13363 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1417 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 1418 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 449069056 Mar 11 19:54 16396 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Mar 11 19:54 16396_fsm 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:41 16396_vm 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1916928 Mar 11 19:54 16397 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:54 16397_fsm 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:54 16397_vm 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2629632 Mar 11 19:54 16399 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:54 16399_fsm 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:54 16399_vm 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:54 16400 2024-03-11T19:54:27.780 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 11 19:54 16402 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Mar 11 19:54 16404 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 144105472 Mar 11 19:54 16406 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 11 19:54 16406_fsm 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:49 16406_vm 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 174 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 175 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2187 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2224 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2328 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2336 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2337 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2579 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2600 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2600_fsm 2024-03-11T19:54:27.781 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2600_vm 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2601_fsm 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2601_vm 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2602 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2602_fsm 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2602_vm 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2603 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2603_fsm 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2603_vm 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2604 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2605 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2605_fsm 2024-03-11T19:54:27.782 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2605_vm 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2606_fsm 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2606_vm 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2607 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2607_fsm 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2607_vm 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 11 19:39 2608 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2608_fsm 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2608_vm 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 11 19:39 2609 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2609_fsm 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2609_vm 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2610 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2610_fsm 2024-03-11T19:54:27.783 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2610_vm 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2611 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2612_fsm 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2612_vm 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2613 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2615_fsm 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2615_vm 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2616 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2616_fsm 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2616_vm 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 11 19:39 2617 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2617_fsm 2024-03-11T19:54:27.784 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2617_vm 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 11 19:39 2618 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2618_fsm 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2618_vm 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Mar 11 19:53 2619 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:40 2619_fsm 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2619_vm 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2620 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2650 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2651 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2652 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2653 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2654 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2655 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2656 2024-03-11T19:54:27.785 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2657 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 11 19:39 2658 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 11 19:39 2659 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2660 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2661 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2662 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2663 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2664 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2665 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2666 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2667 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2668 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2669 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2670 2024-03-11T19:54:27.786 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 11 19:39 2673 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 11 19:39 2674 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 11 19:39 2675 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2678 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2679 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2680 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2681 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2682 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2683 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2684 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2685 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2686 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2687 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2688 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 11 19:39 2689 2024-03-11T19:54:27.787 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 11 19:39 2690 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 11 19:39 2691 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2692 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2693 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:44 2696 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2699 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2701 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2702 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2703 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 2704 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2753 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2753_fsm 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2753_vm 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2754 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2755 2024-03-11T19:54:27.788 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2756 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 2757 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2830 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2831 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2832 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2833 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2834 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2835 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2836 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2837 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 11 19:39 2838 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2838_fsm 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2838_vm 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2839 2024-03-11T19:54:27.789 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2840_fsm 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2840_vm 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2841 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2995 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2996 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3079_fsm 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3079_vm 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3080 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3081 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3085 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3118 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3119 2024-03-11T19:54:27.790 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 11 19:39 3164 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3256 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3257 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3258 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3350 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3351 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3379 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3380 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3381 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3394 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3394_fsm 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3394_vm 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3395 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3429 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3430 2024-03-11T19:54:27.791 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3431 2024-03-11T19:54:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3433 2024-03-11T19:54:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3439 2024-03-11T19:54:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3440 2024-03-11T19:54:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 11 19:39 3455 2024-03-11T19:54:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 11 19:39 3456 2024-03-11T19:54:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3456_fsm 2024-03-11T19:54:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3456_vm 2024-03-11T19:54:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3466 2024-03-11T19:54:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3467 2024-03-11T19:54:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3468 2024-03-11T19:54:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3501 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3502 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3503 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3534 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3541_fsm 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3541_vm 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3542 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3574 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3575 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3576 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3596 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3597 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3598 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3599 2024-03-11T19:54:27.793 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3600_fsm 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3600_vm 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3601_fsm 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3601_vm 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3602_fsm 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3602_vm 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3603_fsm 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3603_vm 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3604 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3605 2024-03-11T19:54:27.794 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3606 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3607 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3608 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 11 19:39 3609 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3712 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 3764_fsm 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3764_vm 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3766 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 3767 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3997 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4143 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4144 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4145 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4146 2024-03-11T19:54:27.795 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4147 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4148 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4149 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4150 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4151 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4152 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4153 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4154 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4155 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4156 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4157 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4158 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4159 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4160 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4161 2024-03-11T19:54:27.796 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4162 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4163 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4164 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4165 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4166 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4167 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4168 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4169 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4170 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4171 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4172 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4173 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4174 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 5002 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 548 2024-03-11T19:54:27.797 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 549 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6102 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6104 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6106 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6110 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6111 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6112 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6113 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6117 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 826 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 827 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 828 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 11 19:39 pg_filenode.map 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Mar 11 19:39 pg_internal.init 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:39 PG_VERSION 2024-03-11T19:54:27.798 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 11 19:39 .. 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/global: 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:total 560 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Mar 11 19:39 . 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1213 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1213_fsm 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1213_vm 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1214 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1214_fsm 2024-03-11T19:54:27.799 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1214_vm 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 1232 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 1233 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1260 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1260_fsm 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1260_vm 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1261 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1261_fsm 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1261_vm 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1262 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 1262_fsm 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 1262_vm 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2396 2024-03-11T19:54:27.800 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 11 19:39 2396_fsm 2024-03-11T19:54:27.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2396_vm 2024-03-11T19:54:27.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2397 2024-03-11T19:54:27.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2671 2024-03-11T19:54:27.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2672 2024-03-11T19:54:27.801 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2676 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2677 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2694 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2695 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2697 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 11 19:39 2698 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2846 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2847 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2964 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2965 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 2966 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 2967 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 3592 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 3593 2024-03-11T19:54:27.802 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4060 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4061 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4175 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4176 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4177 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4178 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4181 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4182 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4183 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4184 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 4185 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 4186 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6000 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6001 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6002 2024-03-11T19:54:27.803 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 6100 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6114 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 6115 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:54 pg_control 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 11 19:39 pg_filenode.map 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Mar 11 19:39 pg_internal.init 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/log: 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:total 2 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 . 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1222 Mar 11 19:54 postgresql-Mon.log 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-03-11T19:54:27.804 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:total 1 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:54 . 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 mappings 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Mar 11 19:54 replorigin_checkpoint 2024-03-11T19:54:27.805 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 snapshots 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:54 .. 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:54 .. 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 11 19:39 . 2024-03-11T19:54:27.806 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 members 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 offsets 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 . 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 11 19:39 .. 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:39 0000 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:total 8 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:39 . 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 11 19:39 .. 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 11 19:40 0000 2024-03-11T19:54:27.807 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.808 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.809 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.809 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-03-11T19:54:27.809 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.809 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.809 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.809 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.809 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-03-11T19:54:27.810 INFO:teuthology.orchestra.run.smithi083.stdout:total 15 2024-03-11T19:54:27.810 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:54 . 2024-03-11T19:54:27.810 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.810 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Mar 11 19:54 db_0.stat 2024-03-11T19:54:27.810 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Mar 11 19:54 db_13494.stat 2024-03-11T19:54:27.810 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Mar 11 19:54 global.stat 2024-03-11T19:54:27.810 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.810 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-03-11T19:54:27.810 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.810 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:54 . 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout:total 232 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 11 19:54 . 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 237568 Mar 11 19:54 0029 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.811 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:total 1048576 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Mar 11 19:54 . 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:54 0000000100000000000000F3 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 0000000100000000000000F4 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 0000000100000000000000F5 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 0000000100000000000000F6 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:49 0000000100000000000000F7 2024-03-11T19:54:27.812 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 0000000100000000000000F8 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 0000000100000000000000F9 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 0000000100000000000000FA 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 0000000100000000000000FB 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 0000000100000000000000FC 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:50 0000000100000000000000FD 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:50 0000000100000000000000FE 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:49 0000000100000000000000FF 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 000000010000000100000000 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 000000010000000100000001 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:49 000000010000000100000002 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:50 000000010000000100000003 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 000000010000000100000004 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 000000010000000100000005 2024-03-11T19:54:27.813 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:51 000000010000000100000006 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 000000010000000100000007 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:51 000000010000000100000008 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:49 000000010000000100000009 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:49 00000001000000010000000A 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:49 00000001000000010000000B 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:49 00000001000000010000000C 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:50 00000001000000010000000D 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:51 00000001000000010000000E 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 00000001000000010000000F 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 000000010000000100000010 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:48 000000010000000100000011 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:51 000000010000000100000012 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:53 000000010000000100000013 2024-03-11T19:54:27.814 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000014 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:53 000000010000000100000015 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:54 000000010000000100000016 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000017 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000018 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000019 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:53 00000001000000010000001A 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:53 00000001000000010000001B 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 00000001000000010000001C 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:53 00000001000000010000001D 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 00000001000000010000001E 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 00000001000000010000001F 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:53 000000010000000100000020 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000021 2024-03-11T19:54:27.815 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000022 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000023 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000024 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000025 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000026 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000027 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000028 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000029 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 00000001000000010000002A 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 00000001000000010000002B 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:53 00000001000000010000002C 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:53 00000001000000010000002D 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:53 00000001000000010000002E 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 00000001000000010000002F 2024-03-11T19:54:27.816 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000030 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:53 000000010000000100000031 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 11 19:52 000000010000000100000032 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 archive_status 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:total 0 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 11 19:39 . 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Mar 11 19:54 .. 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:total 672 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 11 19:52 . 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 11 19:54 .. 2024-03-11T19:54:27.817 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Mar 11 19:42 0000 2024-03-11T19:54:27.818 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Mar 11 19:52 0001 2024-03-11T19:54:27.818 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Mar 11 19:54 0002 2024-03-11T19:54:27.818 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-11T19:54:27.830 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-11T19:54:28.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:27 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:54:28.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:27 smithi083 ceph-mon[31864]: scrub summary: active paths [/] 2024-03-11T19:54:28.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:27 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:28.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:27 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:28.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:27 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:54:28.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:27 smithi137 ceph-mon[36220]: scrub summary: active paths [/] 2024-03-11T19:54:28.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:27 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:28.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:27 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:28.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:27 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:54:28.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:27 smithi204 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-11T19:54:28.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:27 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:28.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:27 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:29.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:28 smithi083 ceph-mon[31864]: pgmap v806: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-03-11T19:54:29.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:28 smithi083 ceph-mon[31864]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:29.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:28 smithi137 ceph-mon[36220]: pgmap v806: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-03-11T19:54:29.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:28 smithi137 ceph-mon[36220]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:29.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:28 smithi204 ceph-mon[36739]: pgmap v806: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 1.07k op/s 2024-03-11T19:54:29.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:28 smithi204 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-11T19:54:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:30 smithi083 ceph-mon[31864]: pgmap v807: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2024-03-11T19:54:31.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:30 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:31.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:30 smithi137 ceph-mon[36220]: pgmap v807: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2024-03-11T19:54:31.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:30 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:31.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:30 smithi204 ceph-mon[36739]: pgmap v807: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2024-03-11T19:54:31.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:30 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:32.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:31 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:32.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:31 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:32.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:31 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:33.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:32 smithi083 ceph-mon[31864]: pgmap v808: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 28 MiB/s wr, 1.85k op/s 2024-03-11T19:54:33.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:32 smithi137 ceph-mon[36220]: pgmap v808: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 28 MiB/s wr, 1.85k op/s 2024-03-11T19:54:33.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:32 smithi204 ceph-mon[36739]: pgmap v808: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 28 MiB/s wr, 1.85k op/s 2024-03-11T19:54:35.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:34 smithi083 ceph-mon[31864]: pgmap v809: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 30 MiB/s wr, 1.80k op/s 2024-03-11T19:54:35.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:34 smithi137 ceph-mon[36220]: pgmap v809: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 30 MiB/s wr, 1.80k op/s 2024-03-11T19:54:35.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:34 smithi204 ceph-mon[36739]: pgmap v809: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 30 MiB/s wr, 1.80k op/s 2024-03-11T19:54:36.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:35 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:54:36.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:35 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:54:36.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:35 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:54:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:36 smithi083 ceph-mon[31864]: pgmap v810: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 29 MiB/s wr, 1.70k op/s 2024-03-11T19:54:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:36 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:54:37.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:36 smithi083 ceph-mon[31864]: scrub summary: idle 2024-03-11T19:54:37.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:36 smithi137 ceph-mon[36220]: pgmap v810: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 29 MiB/s wr, 1.70k op/s 2024-03-11T19:54:37.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:36 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:54:37.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:36 smithi137 ceph-mon[36220]: scrub summary: idle 2024-03-11T19:54:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:36 smithi204 ceph-mon[36739]: pgmap v810: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 29 MiB/s wr, 1.70k op/s 2024-03-11T19:54:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:36 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:54:37.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:36 smithi204 ceph-mon[36739]: scrub summary: idle 2024-03-11T19:54:39.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:38 smithi083 ceph-mon[31864]: pgmap v811: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-03-11T19:54:39.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:38 smithi137 ceph-mon[36220]: pgmap v811: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-03-11T19:54:39.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:38 smithi204 ceph-mon[36739]: pgmap v811: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 24 MiB/s wr, 1.28k op/s 2024-03-11T19:54:41.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:40 smithi083 ceph-mon[31864]: pgmap v812: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-03-11T19:54:41.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:40 smithi137 ceph-mon[36220]: pgmap v812: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-03-11T19:54:41.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:40 smithi204 ceph-mon[36739]: pgmap v812: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-03-11T19:54:43.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:42 smithi083 ceph-mon[31864]: pgmap v813: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 887 op/s 2024-03-11T19:54:43.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:42 smithi137 ceph-mon[36220]: pgmap v813: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 887 op/s 2024-03-11T19:54:43.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:42 smithi204 ceph-mon[36739]: pgmap v813: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 887 op/s 2024-03-11T19:54:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:44 smithi083 ceph-mon[31864]: pgmap v814: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 184 op/s 2024-03-11T19:54:45.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:44 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:54:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:44 smithi137 ceph-mon[36220]: pgmap v814: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 184 op/s 2024-03-11T19:54:45.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:44 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:54:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:44 smithi204 ceph-mon[36739]: pgmap v814: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.4 MiB/s wr, 184 op/s 2024-03-11T19:54:45.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:44 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-11T19:54:46.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:54:46.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:54:46.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:45 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:54:46.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:54:46.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:54:46.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:45 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:54:46.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-11T19:54:46.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-11T19:54:46.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:45 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' 2024-03-11T19:54:47.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:46 smithi137 ceph-mon[36220]: pgmap v815: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-03-11T19:54:47.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:46 smithi204 ceph-mon[36739]: pgmap v815: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-03-11T19:54:47.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:46 smithi083 ceph-mon[31864]: pgmap v815: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-03-11T19:54:48.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:47 smithi137 ceph-mon[36220]: pgmap v816: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-03-11T19:54:48.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:47 smithi204 ceph-mon[36739]: pgmap v816: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-03-11T19:54:48.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:47 smithi083 ceph-mon[31864]: pgmap v816: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-03-11T19:54:50.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:49 smithi137 ceph-mon[36220]: pgmap v817: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-03-11T19:54:50.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:49 smithi204 ceph-mon[36739]: pgmap v817: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-03-11T19:54:50.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:49 smithi083 ceph-mon[31864]: pgmap v817: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-03-11T19:54:52.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:51 smithi137 ceph-mon[36220]: pgmap v818: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8.1 KiB/s wr, 8 op/s 2024-03-11T19:54:52.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:51 smithi204 ceph-mon[36739]: pgmap v818: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8.1 KiB/s wr, 8 op/s 2024-03-11T19:54:52.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:51 smithi083 ceph-mon[31864]: pgmap v818: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8.1 KiB/s wr, 8 op/s 2024-03-11T19:54:54.139 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:53 smithi137 ceph-mon[36220]: pgmap v819: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-03-11T19:54:54.140 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:53 smithi137 ceph-mon[36220]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:54.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:53 smithi204 ceph-mon[36739]: pgmap v819: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-03-11T19:54:54.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:53 smithi204 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:54.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:53 smithi083 ceph-mon[31864]: pgmap v819: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9.9 KiB/s wr, 9 op/s 2024-03-11T19:54:54.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:53 smithi083 ceph-mon[31864]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-11T19:54:56.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:55 smithi204 ceph-mon[36739]: pgmap v820: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 47 KiB/s wr, 12 op/s 2024-03-11T19:54:56.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:55 smithi083 ceph-mon[31864]: pgmap v820: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 47 KiB/s wr, 12 op/s 2024-03-11T19:54:56.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:55 smithi137 ceph-mon[36220]: pgmap v820: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 47 KiB/s wr, 12 op/s 2024-03-11T19:54:56.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-11T19:54:56.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-11T19:54:56.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.979+0000 7fa11819a640 1 -- 172.21.15.83:0/3544470600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa110101180 msgr2=0x7fa1101015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.979+0000 7fa11819a640 1 --2- 172.21.15.83:0/3544470600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa110101180 0x7fa1101015a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009900 tx=0x7fa10c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:54:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.979+0000 7fa11819a640 1 -- 172.21.15.83:0/3544470600 shutdown_connections 2024-03-11T19:54:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.979+0000 7fa11819a640 1 --2- 172.21.15.83:0/3544470600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa110102d30 0x7fa1101095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.979+0000 7fa11819a640 1 --2- 172.21.15.83:0/3544470600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa110102370 0x7fa1101027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.979+0000 7fa11819a640 1 --2- 172.21.15.83:0/3544470600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa110101180 0x7fa1101015a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.979+0000 7fa11819a640 1 -- 172.21.15.83:0/3544470600 >> 172.21.15.83:0/3544470600 conn(0x7fa1100fc6f0 msgr2=0x7fa1100feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:56.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa11819a640 1 -- 172.21.15.83:0/3544470600 shutdown_connections 2024-03-11T19:54:56.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa11819a640 1 -- 172.21.15.83:0/3544470600 wait complete. 2024-03-11T19:54:56.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa11819a640 1 Processor -- start 2024-03-11T19:54:56.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa11819a640 1 -- start start 2024-03-11T19:54:56.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa11819a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa110101180 0x7fa1101971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:56.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa11819a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa110102370 0x7fa110197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:56.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa11819a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa110102d30 0x7fa11019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:56.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa11819a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11019ece0 con 0x7fa110102370 2024-03-11T19:54:56.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa11819a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11019ee50 con 0x7fa110101180 2024-03-11T19:54:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa11819a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11019f010 con 0x7fa110102d30 2024-03-11T19:54:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa115735640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa110102370 0x7fa110197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa115735640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa110102370 0x7fa110197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50550/0 (socket says 172.21.15.83:50550) 2024-03-11T19:54:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.980+0000 7fa115735640 1 -- 172.21.15.83:0/885115885 learned_addr learned my addr 172.21.15.83:0/885115885 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa116737640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa110102d30 0x7fa11019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa115f36640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa110101180 0x7fa1101971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa115735640 1 -- 172.21.15.83:0/885115885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa110102d30 msgr2=0x7fa11019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa115735640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa110102d30 0x7fa11019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa115735640 1 -- 172.21.15.83:0/885115885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa110101180 msgr2=0x7fa1101971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa115735640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa110101180 0x7fa1101971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa115735640 1 -- 172.21.15.83:0/885115885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c009560 con 0x7fa110102370 2024-03-11T19:54:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa116737640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa110102d30 0x7fa11019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa115f36640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa110101180 0x7fa1101971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa115735640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa110102370 0x7fa110197730 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa10000d950 tx=0x7fa10000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa106ffd640 1 -- 172.21.15.83:0/885115885 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100009a30 con 0x7fa110102370 2024-03-11T19:54:57.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa106ffd640 1 -- 172.21.15.83:0/885115885 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa100009b90 con 0x7fa110102370 2024-03-11T19:54:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa11819a640 1 -- 172.21.15.83:0/885115885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11019f2f0 con 0x7fa110102370 2024-03-11T19:54:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa106ffd640 1 -- 172.21.15.83:0/885115885 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100004a00 con 0x7fa110102370 2024-03-11T19:54:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.981+0000 7fa11819a640 1 -- 172.21.15.83:0/885115885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11019f800 con 0x7fa110102370 2024-03-11T19:54:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.983+0000 7fa106ffd640 1 -- 172.21.15.83:0/885115885 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa100009d00 con 0x7fa110102370 2024-03-11T19:54:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.983+0000 7fa11ad75740 1 Processor -- start 2024-03-11T19:54:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa11ad75740 1 -- start start 2024-03-11T19:54:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e61cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61cbe30 0x5644e61ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa11ad75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e6136280 con 0x5644e61c62d0 2024-03-11T19:54:57.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa11ad75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e61337b0 con 0x5644e61c6cc0 2024-03-11T19:54:57.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa11ad75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e612ef90 con 0x5644e61cbe30 2024-03-11T19:54:57.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa116737640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa116737640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50566/0 (socket says 172.21.15.83:50566) 2024-03-11T19:54:57.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa116737640 1 -- 172.21.15.83:0/3974473934 learned_addr learned my addr 172.21.15.83:0/3974473934 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa106ffd640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0fc0a1a30 0x7fa0fc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa115f36640 1 --2- 172.21.15.83:0/3974473934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e61cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa115735640 1 --2- 172.21.15.83:0/3974473934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61cbe30 0x5644e61ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa116737640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0fc0a1a30 0x7fa0fc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa106ffd640 1 -- 172.21.15.83:0/885115885 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa1000cceb0 con 0x7fa110102370 2024-03-11T19:54:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa116737640 1 -- 172.21.15.83:0/3974473934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61cbe30 msgr2=0x5644e61ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa116737640 1 --2- 172.21.15.83:0/3974473934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61cbe30 0x5644e61ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa116737640 1 -- 172.21.15.83:0/3974473934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 msgr2=0x5644e61cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.984+0000 7fa116737640 1 --2- 172.21.15.83:0/3974473934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e61cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.985+0000 7fa116737640 1 -- 172.21.15.83:0/3974473934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644e60ed570 con 0x5644e61c62d0 2024-03-11T19:54:57.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.985+0000 7fa115f36640 1 --2- 172.21.15.83:0/3974473934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e61cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:57.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.985+0000 7fa116737640 1 --2- 172.21.15.83:0/3974473934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61c66f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa1080139e0 tx=0x7fa108013ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35778c1ba143183 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.985+0000 7fa0fbfff640 1 -- 172.21.15.83:0/3974473934 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10801fb80 con 0x5644e61c62d0 2024-03-11T19:54:57.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.985+0000 7fa0fbfff640 1 -- 172.21.15.83:0/3974473934 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa108019070 con 0x5644e61c62d0 2024-03-11T19:54:57.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.985+0000 7fa0fbfff640 1 -- 172.21.15.83:0/3974473934 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10801ed70 con 0x5644e61c62d0 2024-03-11T19:54:57.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.985+0000 7fa116737640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0fc0a1a30 0x7fa0fc0a3ef0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa108022000 tx=0x7fa108020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.986+0000 7fa11ad75740 1 -- 172.21.15.83:0/3974473934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 msgr2=0x5644e61c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.986+0000 7fa11ad75740 1 --2- 172.21.15.83:0/3974473934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61c66f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa1080139e0 tx=0x7fa108013ea0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.986+0000 7fa11ad75740 1 -- 172.21.15.83:0/3974473934 shutdown_connections 2024-03-11T19:54:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.986+0000 7fa11ad75740 1 --2- 172.21.15.83:0/3974473934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61cbe30 0x5644e61ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.986+0000 7fa11ad75740 1 --2- 172.21.15.83:0/3974473934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e61cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.986+0000 7fa11ad75740 1 --2- 172.21.15.83:0/3974473934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61c66f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.986+0000 7fa11ad75740 1 -- 172.21.15.83:0/3974473934 >> 172.21.15.83:0/3974473934 conn(0x5644e61c17f0 msgr2=0x5644e61c1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.986+0000 7fa11ad75740 1 -- 172.21.15.83:0/3974473934 shutdown_connections 2024-03-11T19:54:57.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.986+0000 7fa11ad75740 1 -- 172.21.15.83:0/3974473934 wait complete. 2024-03-11T19:54:57.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.986+0000 7fa11ad75740 1 -- start start 2024-03-11T19:54:57.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.986+0000 7fa11ad75740 1 Processor -- start 2024-03-11T19:54:57.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e61d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61cbe30 0x5644e61d7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa11ad75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e5e02430 con 0x5644e61c62d0 2024-03-11T19:54:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa11ad75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e5dbee00 con 0x5644e61c6cc0 2024-03-11T19:54:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa11ad75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e60b3fd0 con 0x5644e61cbe30 2024-03-11T19:54:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa11ad75740 10 client.0 _ll_register_callbacks cb 0x5644e6109fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa115f36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa115f36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50576/0 (socket says 172.21.15.83:50576) 2024-03-11T19:54:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa115f36640 1 -- 172.21.15.83:0/3078224401 learned_addr learned my addr 172.21.15.83:0/3078224401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa115f36640 1 -- 172.21.15.83:0/3078224401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61cbe30 msgr2=0x5644e61d7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa115f36640 1 --2- 172.21.15.83:0/3078224401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61cbe30 0x5644e61d7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa115f36640 1 -- 172.21.15.83:0/3078224401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 msgr2=0x5644e61d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa115f36640 1 --2- 172.21.15.83:0/3078224401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e61d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa115f36640 1 -- 172.21.15.83:0/3078224401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa108013620 con 0x5644e61c62d0 2024-03-11T19:54:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa115f36640 1 --2- 172.21.15.83:0/3078224401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61d02a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009900 tx=0x7fa10c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e20e00328240313 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa0f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa0f97fa640 1 -- 172.21.15.83:0/3078224401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c005d50 con 0x5644e61c62d0 2024-03-11T19:54:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa0f97fa640 1 -- 172.21.15.83:0/3078224401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa10c004250 con 0x5644e61c62d0 2024-03-11T19:54:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa11ad75740 1 -- 172.21.15.83:0/3078224401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644e5e2e7e0 con 0x5644e61c62d0 2024-03-11T19:54:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa0f97fa640 1 -- 172.21.15.83:0/3078224401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa10c004410 con 0x5644e61c62d0 2024-03-11T19:54:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa11ad75740 10 client.18231 fetch_fsmap learned FSMap version 36 2024-03-11T19:54:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.987+0000 7fa11ad75740 1 -- 172.21.15.83:0/3078224401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5644e612d610 con 0x5644e61c62d0 2024-03-11T19:54:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.988+0000 7fa0f97fa640 1 -- 172.21.15.83:0/3078224401 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5644e5e2e7e0 con 0x5644e61c62d0 2024-03-11T19:54:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.988+0000 7fa0f97fa640 1 -- 172.21.15.83:0/3078224401 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa10c050070 con 0x5644e61c62d0 2024-03-11T19:54:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.988+0000 7fa11ad75740 10 client.18231 fetch_fsmap finished waiting for FSMap version 36 2024-03-11T19:54:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.988+0000 7fa11ad75740 10 client.18231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.988+0000 7fa11ad75740 20 client.18231 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:57.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.988+0000 7fa11ad75740 1 --2- 172.21.15.83:0/3078224401 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5644e61dc5e0 0x5644e61dea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.988+0000 7fa11ad75740 4 client.18231 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:57.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.988+0000 7fa11ad75740 1 -- 172.21.15.83:0/3078224401 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644e5e6b720 con 0x5644e61dc5e0 2024-03-11T19:54:57.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.988+0000 7fa115735640 1 --2- 172.21.15.83:0/3078224401 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5644e61dc5e0 0x5644e61dea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa115735640 1 --2- 172.21.15.83:0/3078224401 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5644e61dc5e0 0x5644e61dea20 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a4240407e1232b server_cookie=83331aee138fe4eb in_seq=0 out_seq=0 2024-03-11T19:54:57.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa0f97fa640 10 client.18231 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:57.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa0f97fa640 1 -- 172.21.15.83:0/3078224401 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644e5e6b720 con 0x5644e61dc5e0 2024-03-11T19:54:57.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa0f97fa640 10 client.18231 handle_command_reply: tid=0 2024-03-11T19:54:57.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa11ad75740 1 client.18231 shutdown 2024-03-11T19:54:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa11ad75740 1 -- 172.21.15.83:0/3078224401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 msgr2=0x5644e61d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa11ad75740 1 --2- 172.21.15.83:0/3078224401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61d02a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009900 tx=0x7fa10c004970 comp rx=0 tx=0).stop 2024-03-11T19:54:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa11ad75740 1 -- 172.21.15.83:0/3078224401 shutdown_connections 2024-03-11T19:54:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa11ad75740 1 --2- 172.21.15.83:0/3078224401 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5644e61dc5e0 0x5644e61dea20 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa11ad75740 1 --2- 172.21.15.83:0/3078224401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61cbe30 0x5644e61d7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa0f97fa640 0 client.18231 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa11ad75740 1 --2- 172.21.15.83:0/3078224401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e61d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa11ad75740 1 --2- 172.21.15.83:0/3078224401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c62d0 0x5644e61d02a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa11ad75740 1 -- 172.21.15.83:0/3078224401 >> 172.21.15.83:0/3078224401 conn(0x5644e61c17f0 msgr2=0x5644e61c0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa11ad75740 1 -- 172.21.15.83:0/3078224401 shutdown_connections 2024-03-11T19:54:57.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.989+0000 7fa11ad75740 1 -- 172.21.15.83:0/3078224401 wait complete. 2024-03-11T19:54:57.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:56.990+0000 7fa11ad75740 20 client.18231 trim_cache size 0 max 16384 2024-03-11T19:54:57.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.004+0000 7fa11ad75740 1 Processor -- start 2024-03-11T19:54:57.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.004+0000 7fa11ad75740 1 -- start start 2024-03-11T19:54:57.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.004+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c17f0 0x5644e61c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61c62d0 0x5644e61d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c6cc0 0x5644e61cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa11ad75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e61cba30 con 0x5644e61c6cc0 2024-03-11T19:54:57.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa11ad75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e5e6b720 con 0x5644e61c17f0 2024-03-11T19:54:57.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa11ad75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e5e2e7e0 con 0x5644e61c62d0 2024-03-11T19:54:57.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa116737640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c6cc0 0x5644e61cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa116737640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c6cc0 0x5644e61cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50578/0 (socket says 172.21.15.83:50578) 2024-03-11T19:54:57.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa116737640 1 -- 172.21.15.83:0/1707265650 learned_addr learned my addr 172.21.15.83:0/1707265650 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa115735640 1 --2- 172.21.15.83:0/1707265650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61c62d0 0x5644e61d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa115f36640 1 --2- 172.21.15.83:0/1707265650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c17f0 0x5644e61c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa116737640 1 -- 172.21.15.83:0/1707265650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61c62d0 msgr2=0x5644e61d77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa116737640 1 --2- 172.21.15.83:0/1707265650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61c62d0 0x5644e61d77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa116737640 1 -- 172.21.15.83:0/1707265650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c17f0 msgr2=0x5644e61c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa116737640 1 --2- 172.21.15.83:0/1707265650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c17f0 0x5644e61c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa116737640 1 -- 172.21.15.83:0/1707265650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644e60c9ac0 con 0x5644e61c6cc0 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa115f36640 1 --2- 172.21.15.83:0/1707265650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c17f0 0x5644e61c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa116737640 1 --2- 172.21.15.83:0/1707265650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c6cc0 0x5644e61cf8c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa108005ee0 tx=0x7fa1080093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a672dbc563ac0dde server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa0fb7fe640 1 -- 172.21.15.83:0/1707265650 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10801fb80 con 0x5644e61c6cc0 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa0fb7fe640 1 -- 172.21.15.83:0/1707265650 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa108013480 con 0x5644e61c6cc0 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.005+0000 7fa0fb7fe640 1 -- 172.21.15.83:0/1707265650 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa108005280 con 0x5644e61c6cc0 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.006+0000 7fa11ad75740 1 -- 172.21.15.83:0/1707265650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c6cc0 msgr2=0x5644e61cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.006+0000 7fa11ad75740 1 --2- 172.21.15.83:0/1707265650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c6cc0 0x5644e61cf8c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa108005ee0 tx=0x7fa1080093e0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.006+0000 7fa11ad75740 1 -- 172.21.15.83:0/1707265650 shutdown_connections 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.006+0000 7fa11ad75740 1 --2- 172.21.15.83:0/1707265650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c6cc0 0x5644e61cf8c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.006+0000 7fa11ad75740 1 --2- 172.21.15.83:0/1707265650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61c62d0 0x5644e61d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.006+0000 7fa11ad75740 1 --2- 172.21.15.83:0/1707265650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c17f0 0x5644e61c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.006+0000 7fa11ad75740 1 -- 172.21.15.83:0/1707265650 >> 172.21.15.83:0/1707265650 conn(0x5644e61c78c0 msgr2=0x5644e61c4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.006+0000 7fa11ad75740 1 -- 172.21.15.83:0/1707265650 shutdown_connections 2024-03-11T19:54:57.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.006+0000 7fa11ad75740 1 -- 172.21.15.83:0/1707265650 wait complete. 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.006+0000 7fa11ad75740 1 -- start start 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.006+0000 7fa11ad75740 1 Processor -- start 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c17f0 0x5644e61d6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61c62d0 0x5644e61d6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa11ad75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e6232b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa11ad75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e6233070 con 0x5644e61c17f0 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa11ad75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e62331b0 con 0x5644e61c6cc0 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa11ad75740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644e6233320 con 0x5644e61c62d0 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa11ad75740 10 client.0 _ll_register_callbacks cb 0x7fa108009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa115f36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c17f0 0x5644e61d6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa115f36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c17f0 0x5644e61d6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50590/0 (socket says 172.21.15.83:50590) 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa115f36640 1 -- 172.21.15.83:0/1589814212 learned_addr learned my addr 172.21.15.83:0/1589814212 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa115735640 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61c62d0 0x5644e61d6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa115f36640 1 -- 172.21.15.83:0/1589814212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61c62d0 msgr2=0x5644e61d6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa116737640 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e6232b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa115f36640 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61c62d0 0x5644e61d6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa115f36640 1 -- 172.21.15.83:0/1589814212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 msgr2=0x5644e6232b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa115f36640 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e6232b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa115f36640 1 -- 172.21.15.83:0/1589814212 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644e60ed570 con 0x5644e61c17f0 2024-03-11T19:54:57.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa115f36640 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c17f0 0x5644e61d6850 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa10c02cd90 tx=0x7fa10c0049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52d1cdb74d27793 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa0f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:57.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa0f97fa640 1 -- 172.21.15.83:0/1589814212 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0056a0 con 0x5644e61c17f0 2024-03-11T19:54:57.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa0f97fa640 1 -- 172.21.15.83:0/1589814212 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fa10c000e40 con 0x5644e61c17f0 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.007+0000 7fa11ad75740 1 -- 172.21.15.83:0/1589814212 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa108019070 con 0x5644e61c17f0 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa0f97fa640 1 -- 172.21.15.83:0/1589814212 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa10c05a030 con 0x5644e61c17f0 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa11ad75740 10 client.18237 fetch_fsmap learned FSMap version 36 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa11ad75740 1 -- 172.21.15.83:0/1589814212 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5644e60ef6a0 con 0x5644e61c17f0 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa0f97fa640 1 -- 172.21.15.83:0/1589814212 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa108019070 con 0x5644e61c17f0 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa0f97fa640 1 -- 172.21.15.83:0/1589814212 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fa10c050070 con 0x5644e61c17f0 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa11ad75740 10 client.18237 fetch_fsmap finished waiting for FSMap version 36 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa11ad75740 10 client.18237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa11ad75740 20 client.18237 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa11ad75740 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5644e61cbe30 0x5644e6236c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa11ad75740 4 client.18237 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa11ad75740 1 -- 172.21.15.83:0/1589814212 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5644e5eba890 con 0x5644e61cbe30 2024-03-11T19:54:57.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.008+0000 7fa115735640 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5644e61cbe30 0x5644e6236c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa0f97fa640 10 client.18237 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa115735640 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5644e61cbe30 0x5644e6236c30 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2488a5947d32fa server_cookie=2313126ce5f2c87 in_seq=0 out_seq=0 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa0f97fa640 1 -- 172.21.15.83:0/1589814212 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5644e5eba890 con 0x5644e61cbe30 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa0f97fa640 10 client.18237 handle_command_reply: tid=0 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 1 client.18237 shutdown 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 1 -- 172.21.15.83:0/1589814212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c17f0 msgr2=0x5644e61d6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c17f0 0x5644e61d6850 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa10c02cd90 tx=0x7fa10c0049e0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 1 -- 172.21.15.83:0/1589814212 shutdown_connections 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x5644e61cbe30 0x5644e6236c30 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644e61c6cc0 0x5644e6232b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5644e61c62d0 0x5644e61d6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 1 --2- 172.21.15.83:0/1589814212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644e61c17f0 0x5644e61d6850 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa0f97fa640 0 client.18237 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:57.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 1 -- 172.21.15.83:0/1589814212 >> 172.21.15.83:0/1589814212 conn(0x5644e61c78c0 msgr2=0x5644e61c5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 1 -- 172.21.15.83:0/1589814212 shutdown_connections 2024-03-11T19:54:57.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 1 -- 172.21.15.83:0/1589814212 wait complete. 2024-03-11T19:54:57.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.009+0000 7fa11ad75740 20 client.18237 trim_cache size 0 max 16384 2024-03-11T19:54:57.033 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:54:57.033 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-03-11T19:54:57.033 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-03-11T19:54:57.033 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:54:57.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 -- 172.21.15.83:0/885115885 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0fc0a1a30 msgr2=0x7fa0fc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0fc0a1a30 0x7fa0fc0a3ef0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa108022000 tx=0x7fa108020000 comp rx=0 tx=0).stop 2024-03-11T19:54:57.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 -- 172.21.15.83:0/885115885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa110102370 msgr2=0x7fa110197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa110102370 0x7fa110197730 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa10000d950 tx=0x7fa10000de10 comp rx=0 tx=0).stop 2024-03-11T19:54:57.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 -- 172.21.15.83:0/885115885 shutdown_connections 2024-03-11T19:54:57.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa0fc0a1a30 0x7fa0fc0a3ef0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa110102d30 0x7fa11019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa110102370 0x7fa110197730 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 --2- 172.21.15.83:0/885115885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa110101180 0x7fa1101971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 -- 172.21.15.83:0/885115885 >> 172.21.15.83:0/885115885 conn(0x7fa1100fc6f0 msgr2=0x7fa110105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 -- 172.21.15.83:0/885115885 shutdown_connections 2024-03-11T19:54:57.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.010+0000 7fa11819a640 1 -- 172.21.15.83:0/885115885 wait complete. 2024-03-11T19:54:57.037 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-11T19:54:57.038 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:54:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.137+0000 7fd140fbc640 1 -- 172.21.15.83:0/308018372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c101280 msgr2=0x7fd13c101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.137+0000 7fd140fbc640 1 --2- 172.21.15.83:0/308018372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c101280 0x7fd13c101700 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd124009900 tx=0x7fd12402bc30 comp rx=0 tx=0).stop 2024-03-11T19:54:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.137+0000 7fd140fbc640 1 -- 172.21.15.83:0/308018372 shutdown_connections 2024-03-11T19:54:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.137+0000 7fd140fbc640 1 --2- 172.21.15.83:0/308018372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd13c101c40 0x7fd13c1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.137+0000 7fd140fbc640 1 --2- 172.21.15.83:0/308018372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c101280 0x7fd13c101700 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.137+0000 7fd140fbc640 1 --2- 172.21.15.83:0/308018372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd13c100090 0x7fd13c1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.137+0000 7fd140fbc640 1 -- 172.21.15.83:0/308018372 >> 172.21.15.83:0/308018372 conn(0x7fd13c0fb600 msgr2=0x7fd13c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.137+0000 7fd140fbc640 1 -- 172.21.15.83:0/308018372 shutdown_connections 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.137+0000 7fd140fbc640 1 -- 172.21.15.83:0/308018372 wait complete. 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd140fbc640 1 Processor -- start 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd140fbc640 1 -- start start 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd140fbc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd13c100090 0x7fd13c104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd140fbc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c101280 0x7fd13c102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd140fbc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd13c101c40 0x7fd13c102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd140fbc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c103300 con 0x7fd13c101280 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd140fbc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c103470 con 0x7fd13c101c40 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd140fbc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c1a1420 con 0x7fd13c100090 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd133fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c101280 0x7fd13c102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd133fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c101280 0x7fd13c102660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50610/0 (socket says 172.21.15.83:50610) 2024-03-11T19:54:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd133fff640 1 -- 172.21.15.83:0/1125994711 learned_addr learned my addr 172.21.15.83:0/1125994711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd133fff640 1 -- 172.21.15.83:0/1125994711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd13c100090 msgr2=0x7fd13c104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd13ad9d640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd13c101c40 0x7fd13c102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd13a59c640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd13c100090 0x7fd13c104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd133fff640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd13c100090 0x7fd13c104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd133fff640 1 -- 172.21.15.83:0/1125994711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd13c101c40 msgr2=0x7fd13c102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd133fff640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd13c101c40 0x7fd13c102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd133fff640 1 -- 172.21.15.83:0/1125994711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd124009560 con 0x7fd13c101280 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.138+0000 7fd13a59c640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd13c100090 0x7fd13c104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.139+0000 7fd13ad9d640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd13c101c40 0x7fd13c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.139+0000 7fd133fff640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c101280 0x7fd13c102660 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd124036040 tx=0x7fd1240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.139+0000 7fd1337fe640 1 -- 172.21.15.83:0/1125994711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1240044e0 con 0x7fd13c101280 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.139+0000 7fd140fbc640 1 -- 172.21.15.83:0/1125994711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c1a1670 con 0x7fd13c101280 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.139+0000 7fd1337fe640 1 -- 172.21.15.83:0/1125994711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1240054e0 con 0x7fd13c101280 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.139+0000 7fd1337fe640 1 -- 172.21.15.83:0/1125994711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12403e450 con 0x7fd13c101280 2024-03-11T19:54:57.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.139+0000 7fd140fbc640 1 -- 172.21.15.83:0/1125994711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c1a1c50 con 0x7fd13c101280 2024-03-11T19:54:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.140+0000 7fd140fbc640 1 -- 172.21.15.83:0/1125994711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13c10bb80 con 0x7fd13c101280 2024-03-11T19:54:57.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.141+0000 7fd1337fe640 1 -- 172.21.15.83:0/1125994711 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd124003680 con 0x7fd13c101280 2024-03-11T19:54:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.141+0000 7fd1337fe640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1280a1a30 0x7fd1280a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.142+0000 7fd1337fe640 1 -- 172.21.15.83:0/1125994711 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd1240eff20 con 0x7fd13c101280 2024-03-11T19:54:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.142+0000 7fd13a59c640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1280a1a30 0x7fd1280a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.142+0000 7fd13a59c640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1280a1a30 0x7fd1280a3ef0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd12c006fd0 tx=0x7fd12c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.145+0000 7fd1337fe640 1 -- 172.21.15.83:0/1125994711 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd1240b91e0 con 0x7fd13c101280 2024-03-11T19:54:57.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.354+0000 7fd140fbc640 1 -- 172.21.15.83:0/1125994711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd13c1a1800 con 0x7fd13c101280 2024-03-11T19:54:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.354+0000 7fd1337fe640 1 -- 172.21.15.83:0/1125994711 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11356 (secure 0 0 0) 0x7fd13c1a1800 con 0x7fd13c101280 2024-03-11T19:54:57.358 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:57.358 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:54:52.016845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:54:57.358 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 36 2024-03-11T19:54:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 -- 172.21.15.83:0/1125994711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1280a1a30 msgr2=0x7fd1280a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1280a1a30 0x7fd1280a3ef0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd12c006fd0 tx=0x7fd12c008040 comp rx=0 tx=0).stop 2024-03-11T19:54:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 -- 172.21.15.83:0/1125994711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c101280 msgr2=0x7fd13c102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c101280 0x7fd13c102660 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd124036040 tx=0x7fd1240042f0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 -- 172.21.15.83:0/1125994711 shutdown_connections 2024-03-11T19:54:57.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd1280a1a30 0x7fd1280a3ef0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd13c101c40 0x7fd13c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd13c101280 0x7fd13c102660 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 --2- 172.21.15.83:0/1125994711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd13c100090 0x7fd13c104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 -- 172.21.15.83:0/1125994711 >> 172.21.15.83:0/1125994711 conn(0x7fd13c0fb600 msgr2=0x7fd13c0fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 -- 172.21.15.83:0/1125994711 shutdown_connections 2024-03-11T19:54:57.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.356+0000 7fd140fbc640 1 -- 172.21.15.83:0/1125994711 wait complete. 2024-03-11T19:54:57.372 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:54:57.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-11T19:54:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 -- 172.21.15.83:0/3983952771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b80106e70 msgr2=0x7f9b801092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 --2- 172.21.15.83:0/3983952771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b80106e70 0x7f9b801092c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009ca0 tx=0x7f9b7002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 -- 172.21.15.83:0/3983952771 shutdown_connections 2024-03-11T19:54:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 --2- 172.21.15.83:0/3983952771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b80106e70 0x7f9b801092c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 --2- 172.21.15.83:0/3983952771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b80104540 0x7f9b80106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 --2- 172.21.15.83:0/3983952771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b80101c10 0x7f9b80104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 -- 172.21.15.83:0/3983952771 >> 172.21.15.83:0/3983952771 conn(0x7f9b800fb620 msgr2=0x7f9b800fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 -- 172.21.15.83:0/3983952771 shutdown_connections 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 -- 172.21.15.83:0/3983952771 wait complete. 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 Processor -- start 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 -- start start 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.473+0000 7f9b87f9a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b80101c10 0x7f9b80197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b87f9a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b80104540 0x7f9b801976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b87f9a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b80106e70 0x7f9b8019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b87f9a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8019ec80 con 0x7f9b80104540 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b87f9a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8019edf0 con 0x7f9b80106e70 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b87f9a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8019efb0 con 0x7f9b80101c10 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b85535640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b80104540 0x7f9b801976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b85535640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b80104540 0x7f9b801976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50628/0 (socket says 172.21.15.83:50628) 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b85535640 1 -- 172.21.15.83:0/1706643649 learned_addr learned my addr 172.21.15.83:0/1706643649 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b85d36640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b80101c10 0x7f9b80197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b86537640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b80106e70 0x7f9b8019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b85535640 1 -- 172.21.15.83:0/1706643649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b80101c10 msgr2=0x7f9b80197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b85535640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b80101c10 0x7f9b80197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b85535640 1 -- 172.21.15.83:0/1706643649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b80106e70 msgr2=0x7f9b8019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b85535640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b80106e70 0x7f9b8019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b85535640 1 -- 172.21.15.83:0/1706643649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b70009900 con 0x7f9b80104540 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b86537640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b80106e70 0x7f9b8019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b85535640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b80104540 0x7f9b801976d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c970 tx=0x7f9b7400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b7effd640 1 -- 172.21.15.83:0/1706643649 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74004440 con 0x7f9b80104540 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b7effd640 1 -- 172.21.15.83:0/1706643649 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b74007d40 con 0x7f9b80104540 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b87f9a640 1 -- 172.21.15.83:0/1706643649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8019f290 con 0x7f9b80104540 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b7effd640 1 -- 172.21.15.83:0/1706643649 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7400f450 con 0x7f9b80104540 2024-03-11T19:54:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.474+0000 7f9b87f9a640 1 -- 172.21.15.83:0/1706643649 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8019f7a0 con 0x7f9b80104540 2024-03-11T19:54:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.476+0000 7f9b7effd640 1 -- 172.21.15.83:0/1706643649 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f9b74007860 con 0x7f9b80104540 2024-03-11T19:54:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b8ab73740 1 Processor -- start 2024-03-11T19:54:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b8ab73740 1 -- start start 2024-03-11T19:54:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5d270 0x560a82b5d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b5dc60 0x560a82b62760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82b19960 con 0x560a82b62dd0 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82acd310 con 0x560a82b5dc60 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82aca840 con 0x560a82b5d270 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b85535640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b7effd640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9b680a1a30 0x7f9b680a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b85535640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b65260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50634/0 (socket says 172.21.15.83:50634) 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b85535640 1 -- 172.21.15.83:0/3679750212 learned_addr learned my addr 172.21.15.83:0/3679750212 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b86537640 1 --2- 172.21.15.83:0/3679750212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5d270 0x560a82b5d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b85d36640 1 --2- 172.21.15.83:0/3679750212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b5dc60 0x560a82b62760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b86537640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9b680a1a30 0x7f9b680a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.477+0000 7f9b7effd640 1 -- 172.21.15.83:0/1706643649 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9b740cd4b0 con 0x7f9b80104540 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.478+0000 7f9b85535640 1 -- 172.21.15.83:0/3679750212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5d270 msgr2=0x560a82b5d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.478+0000 7f9b85535640 1 --2- 172.21.15.83:0/3679750212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5d270 0x560a82b5d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.478+0000 7f9b85535640 1 -- 172.21.15.83:0/3679750212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b5dc60 msgr2=0x560a82b62760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.478+0000 7f9b85535640 1 --2- 172.21.15.83:0/3679750212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b5dc60 0x560a82b62760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.478+0000 7f9b85535640 1 -- 172.21.15.83:0/3679750212 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a82a88060 con 0x560a82b62dd0 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.478+0000 7f9b85535640 1 --2- 172.21.15.83:0/3679750212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b65260 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9b74015420 tx=0x7f9b740035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226f3fe6e1dc12bf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.478+0000 7f9b86537640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9b680a1a30 0x7f9b680a3ef0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9b70006010 tx=0x7f9b700095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.478+0000 7f9b67fff640 1 -- 172.21.15.83:0/3679750212 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74099e30 con 0x560a82b62dd0 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.478+0000 7f9b67fff640 1 -- 172.21.15.83:0/3679750212 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b7409e620 con 0x560a82b62dd0 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.478+0000 7f9b67fff640 1 -- 172.21.15.83:0/3679750212 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74098570 con 0x560a82b62dd0 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.479+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3679750212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 msgr2=0x560a82b65260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.479+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/3679750212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b65260 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9b74015420 tx=0x7f9b740035f0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.479+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3679750212 shutdown_connections 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.479+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/3679750212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b65260 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.479+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/3679750212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b5dc60 0x560a82b62760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.479+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/3679750212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5d270 0x560a82b5d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.479+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3679750212 >> 172.21.15.83:0/3679750212 conn(0x560a82b587e0 msgr2=0x560a82b58be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.479+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3679750212 shutdown_connections 2024-03-11T19:54:57.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.479+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3679750212 wait complete. 2024-03-11T19:54:57.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.479+0000 7f9b8ab73740 1 -- start start 2024-03-11T19:54:57.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.479+0000 7f9b8ab73740 1 Processor -- start 2024-03-11T19:54:57.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b5d270 0x560a82b674e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5dc60 0x560a82b67a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b6ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82ac6020 con 0x560a82b62dd0 2024-03-11T19:54:57.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82799430 con 0x560a82b5d270 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82755e00 con 0x560a82b5dc60 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b8ab73740 10 client.0 _ll_register_callbacks cb 0x560a82a6dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b86537640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b6ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b86537640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b6ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50646/0 (socket says 172.21.15.83:50646) 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b86537640 1 -- 172.21.15.83:0/3153017017 learned_addr learned my addr 172.21.15.83:0/3153017017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b85535640 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5dc60 0x560a82b67a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b85d36640 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b5d270 0x560a82b674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b86537640 1 -- 172.21.15.83:0/3153017017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5dc60 msgr2=0x560a82b67a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b86537640 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5dc60 0x560a82b67a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b86537640 1 -- 172.21.15.83:0/3153017017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b5d270 msgr2=0x560a82b674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b86537640 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b5d270 0x560a82b674e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b86537640 1 -- 172.21.15.83:0/3153017017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8019f7a0 con 0x560a82b62dd0 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b86537640 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b6ea60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9b70005fe0 tx=0x7f9b7002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efbb8dd445e308e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b657fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b657fa640 1 -- 172.21.15.83:0/3153017017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70004240 con 0x560a82b62dd0 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b657fa640 1 -- 172.21.15.83:0/3153017017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9b70049040 con 0x560a82b62dd0 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.480+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3153017017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a827c57e0 con 0x560a82b62dd0 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b657fa640 1 -- 172.21.15.83:0/3153017017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b70034740 con 0x560a82b62dd0 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b8ab73740 10 client.18255 fetch_fsmap learned FSMap version 36 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3153017017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560a82ac46a0 con 0x560a82b62dd0 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b657fa640 1 -- 172.21.15.83:0/3153017017 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560a827c57e0 con 0x560a82b62dd0 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b657fa640 1 -- 172.21.15.83:0/3153017017 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9b7003b070 con 0x560a82b62dd0 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b8ab73740 10 client.18255 fetch_fsmap finished waiting for FSMap version 36 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b8ab73740 10 client.18255 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b8ab73740 20 client.18255 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:57.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x560a82b73730 0x560a82b75b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b8ab73740 4 client.18255 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3153017017 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a82802720 con 0x560a82b73730 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.481+0000 7f9b85d36640 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x560a82b73730 0x560a82b75b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b657fa640 10 client.18255 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b85d36640 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x560a82b73730 0x560a82b75b50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=939dfc2f83b99731 server_cookie=f3f3ebc6d62cc8e5 in_seq=0 out_seq=0 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b657fa640 1 -- 172.21.15.83:0/3153017017 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a82802720 con 0x560a82b73730 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b657fa640 10 client.18255 handle_command_reply: tid=0 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b8ab73740 1 client.18255 shutdown 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3153017017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 msgr2=0x560a82b6ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b6ea60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9b70005fe0 tx=0x7f9b7002c0f0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3153017017 shutdown_connections 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x560a82b73730 0x560a82b75b50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b62dd0 0x560a82b6ea60 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5dc60 0x560a82b67a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b657fa640 0 client.18255 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:54:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/3153017017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b5d270 0x560a82b674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3153017017 >> 172.21.15.83:0/3153017017 conn(0x560a82b587e0 msgr2=0x560a82b57b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3153017017 shutdown_connections 2024-03-11T19:54:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.482+0000 7f9b8ab73740 1 -- 172.21.15.83:0/3153017017 wait complete. 2024-03-11T19:54:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.483+0000 7f9b8ab73740 20 client.18255 trim_cache size 0 max 16384 2024-03-11T19:54:57.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.494+0000 7f9b8ab73740 1 Processor -- start 2024-03-11T19:54:57.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.494+0000 7f9b8ab73740 1 -- start start 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.494+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5d270 0x560a82b5e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.494+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5dc60 0x560a82b6ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.494+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b62dd0 0x560a82b5f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.494+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82a4b130 con 0x560a82b5dc60 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.494+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82b53a10 con 0x560a82b62dd0 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.494+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82b5ec40 con 0x560a82b5d270 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b85535640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5dc60 0x560a82b6ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b85535640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5dc60 0x560a82b6ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50662/0 (socket says 172.21.15.83:50662) 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b85535640 1 -- 172.21.15.83:0/1465931892 learned_addr learned my addr 172.21.15.83:0/1465931892 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b86537640 1 --2- 172.21.15.83:0/1465931892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b62dd0 0x560a82b5f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b85d36640 1 --2- 172.21.15.83:0/1465931892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5d270 0x560a82b5e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b85535640 1 -- 172.21.15.83:0/1465931892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5d270 msgr2=0x560a82b5e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b85535640 1 --2- 172.21.15.83:0/1465931892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5d270 0x560a82b5e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b85535640 1 -- 172.21.15.83:0/1465931892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b62dd0 msgr2=0x560a82b5f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b85535640 1 --2- 172.21.15.83:0/1465931892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b62dd0 0x560a82b5f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b85535640 1 -- 172.21.15.83:0/1465931892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a82a60c20 con 0x560a82b5dc60 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b85d36640 1 --2- 172.21.15.83:0/1465931892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5d270 0x560a82b5e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b85535640 1 --2- 172.21.15.83:0/1465931892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5dc60 0x560a82b6ea20 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9b74007fd0 tx=0x7f9b74096790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fbe4eab6f733c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b677fe640 1 -- 172.21.15.83:0/1465931892 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74099e30 con 0x560a82b5dc60 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b677fe640 1 -- 172.21.15.83:0/1465931892 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b74015260 con 0x560a82b5dc60 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.495+0000 7f9b677fe640 1 -- 172.21.15.83:0/1465931892 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74085180 con 0x560a82b5dc60 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.496+0000 7f9b8ab73740 1 -- 172.21.15.83:0/1465931892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5dc60 msgr2=0x560a82b6ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.496+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/1465931892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5dc60 0x560a82b6ea20 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9b74007fd0 tx=0x7f9b74096790 comp rx=0 tx=0).stop 2024-03-11T19:54:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.496+0000 7f9b8ab73740 1 -- 172.21.15.83:0/1465931892 shutdown_connections 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.496+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/1465931892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b62dd0 0x560a82b5f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.496+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/1465931892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5dc60 0x560a82b6ea20 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.496+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/1465931892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5d270 0x560a82b5e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.496+0000 7f9b8ab73740 1 -- 172.21.15.83:0/1465931892 >> 172.21.15.83:0/1465931892 conn(0x560a82b587e0 msgr2=0x560a82b58bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.496+0000 7f9b8ab73740 1 -- 172.21.15.83:0/1465931892 shutdown_connections 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.496+0000 7f9b8ab73740 1 -- 172.21.15.83:0/1465931892 wait complete. 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.496+0000 7f9b8ab73740 1 -- start start 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.496+0000 7f9b8ab73740 1 Processor -- start 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5d270 0x560a82b6c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5dc60 0x560a82b6ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b8ab73740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b62dd0 0x560a82b6de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82b6e370 con 0x560a82b5d270 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82b6e4e0 con 0x560a82b62dd0 2024-03-11T19:54:57.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b8ab73740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a82b6e6a0 con 0x560a82b5dc60 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b8ab73740 10 client.0 _ll_register_callbacks cb 0x7f9b7409a2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b85d36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5d270 0x560a82b6c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b85d36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5d270 0x560a82b6c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50670/0 (socket says 172.21.15.83:50670) 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b85d36640 1 -- 172.21.15.83:0/507997608 learned_addr learned my addr 172.21.15.83:0/507997608 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b86537640 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b62dd0 0x560a82b6de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b85535640 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5dc60 0x560a82b6ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b85d36640 1 -- 172.21.15.83:0/507997608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5dc60 msgr2=0x560a82b6ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b85d36640 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5dc60 0x560a82b6ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b85d36640 1 -- 172.21.15.83:0/507997608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b62dd0 msgr2=0x560a82b6de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b85d36640 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b62dd0 0x560a82b6de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b85d36640 1 -- 172.21.15.83:0/507997608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a82a88060 con 0x560a82b5d270 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b85535640 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5dc60 0x560a82b6ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:57.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b85d36640 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5d270 0x560a82b6c8e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009e50 tx=0x7f9b780044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b4ecb94370189d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b657fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b657fa640 1 -- 172.21.15.83:0/507997608 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78017030 con 0x560a82b5d270 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b657fa640 1 -- 172.21.15.83:0/507997608 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f9b7801e590 con 0x560a82b5d270 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b8ab73740 1 -- 172.21.15.83:0/507997608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b7409e620 con 0x560a82b5d270 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b657fa640 1 -- 172.21.15.83:0/507997608 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b7801e750 con 0x560a82b5d270 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b8ab73740 10 client.18261 fetch_fsmap learned FSMap version 36 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.497+0000 7f9b8ab73740 1 -- 172.21.15.83:0/507997608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x560a82a868c0 con 0x560a82b5d270 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.498+0000 7f9b657fa640 1 -- 172.21.15.83:0/507997608 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b7409e620 con 0x560a82b5d270 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.498+0000 7f9b657fa640 1 -- 172.21.15.83:0/507997608 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9b7801d040 con 0x560a82b5d270 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.498+0000 7f9b8ab73740 10 client.18261 fetch_fsmap finished waiting for FSMap version 36 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.498+0000 7f9b8ab73740 10 client.18261 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.498+0000 7f9b8ab73740 20 client.18261 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.498+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x560a82bcade0 0x560a82bcd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.498+0000 7f9b8ab73740 4 client.18261 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.498+0000 7f9b8ab73740 1 -- 172.21.15.83:0/507997608 --> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560a82851890 con 0x560a82bcade0 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.498+0000 7f9b85535640 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x560a82bcade0 0x560a82bcd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.498+0000 7f9b657fa640 10 client.18261 ms_handle_connect on v2:172.21.15.137:6836/3499358906 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.498+0000 7f9b85535640 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x560a82bcade0 0x560a82bcd200 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6ef42cc505abd73 server_cookie=e3b4c754d44597a1 in_seq=0 out_seq=0 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b657fa640 1 -- 172.21.15.83:0/507997608 <== mds.2 v2:172.21.15.137:6836/3499358906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560a82851890 con 0x560a82bcade0 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b657fa640 10 client.18261 handle_command_reply: tid=0 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 1 client.18261 shutdown 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 1 -- 172.21.15.83:0/507997608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5d270 msgr2=0x560a82b6c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5d270 0x560a82b6c8e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009e50 tx=0x7f9b780044f0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 1 -- 172.21.15.83:0/507997608 shutdown_connections 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.137:6836/3499358906,v1:172.21.15.137:6837/3499358906] conn(0x560a82bcade0 0x560a82bcd200 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a82b62dd0 0x560a82b6de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a82b5dc60 0x560a82b6ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 1 --2- 172.21.15.83:0/507997608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a82b5d270 0x560a82b6c8e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b657fa640 0 client.18261 ms_handle_reset on v2:172.21.15.137:6836/3499358906 2024-03-11T19:54:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 1 -- 172.21.15.83:0/507997608 >> 172.21.15.83:0/507997608 conn(0x560a82b587e0 msgr2=0x560a82b65f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 1 -- 172.21.15.83:0/507997608 shutdown_connections 2024-03-11T19:54:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 1 -- 172.21.15.83:0/507997608 wait complete. 2024-03-11T19:54:57.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b8ab73740 20 client.18261 trim_cache size 0 max 16384 2024-03-11T19:54:57.510 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:54:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b87f9a640 1 -- 172.21.15.83:0/1706643649 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9b680a1a30 msgr2=0x7f9b680a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b87f9a640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9b680a1a30 0x7f9b680a3ef0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9b70006010 tx=0x7f9b700095c0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b87f9a640 1 -- 172.21.15.83:0/1706643649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b80104540 msgr2=0x7f9b801976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b87f9a640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b80104540 0x7f9b801976d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c970 tx=0x7f9b7400ce30 comp rx=0 tx=0).stop 2024-03-11T19:54:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b87f9a640 1 -- 172.21.15.83:0/1706643649 shutdown_connections 2024-03-11T19:54:57.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b87f9a640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f9b680a1a30 0x7f9b680a3ef0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b87f9a640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b80106e70 0x7f9b8019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b87f9a640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b80104540 0x7f9b801976d0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b87f9a640 1 --2- 172.21.15.83:0/1706643649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b80101c10 0x7f9b80197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b87f9a640 1 -- 172.21.15.83:0/1706643649 >> 172.21.15.83:0/1706643649 conn(0x7f9b800fb620 msgr2=0x7f9b80102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.499+0000 7f9b87f9a640 1 -- 172.21.15.83:0/1706643649 shutdown_connections 2024-03-11T19:54:57.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.500+0000 7f9b87f9a640 1 -- 172.21.15.83:0/1706643649 wait complete. 2024-03-11T19:54:57.514 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:54:57.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-11T19:54:57.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.616+0000 7f528570c640 1 -- 172.21.15.83:0/1957812791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280101190 msgr2=0x7f52801015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.616+0000 7f528570c640 1 --2- 172.21.15.83:0/1957812791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280101190 0x7f52801015b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5270009900 tx=0x7f527002bc30 comp rx=0 tx=0).stop 2024-03-11T19:54:57.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.616+0000 7f528570c640 1 -- 172.21.15.83:0/1957812791 shutdown_connections 2024-03-11T19:54:57.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.616+0000 7f528570c640 1 --2- 172.21.15.83:0/1957812791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5280102d40 0x7f52801095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.616+0000 7f528570c640 1 --2- 172.21.15.83:0/1957812791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5280102380 0x7f5280102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.616+0000 7f528570c640 1 --2- 172.21.15.83:0/1957812791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280101190 0x7f52801015b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.616+0000 7f528570c640 1 -- 172.21.15.83:0/1957812791 >> 172.21.15.83:0/1957812791 conn(0x7f52800fc700 msgr2=0x7f52800feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.616+0000 7f528570c640 1 -- 172.21.15.83:0/1957812791 shutdown_connections 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.616+0000 7f528570c640 1 -- 172.21.15.83:0/1957812791 wait complete. 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.616+0000 7f528570c640 1 Processor -- start 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.616+0000 7f528570c640 1 -- start start 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f528570c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5280101190 0x7f5280197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f528570c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5280102380 0x7f52801977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f528570c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280102d40 0x7f528019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f528570c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528019ed70 con 0x7f5280102d40 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f528570c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528019eee0 con 0x7f5280101190 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f528570c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528019f0a0 con 0x7f5280102380 2024-03-11T19:54:57.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f527f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280102d40 0x7f528019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f527f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280102d40 0x7f528019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50690/0 (socket says 172.21.15.83:50690) 2024-03-11T19:54:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f527f7fe640 1 -- 172.21.15.83:0/3021579112 learned_addr learned my addr 172.21.15.83:0/3021579112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f527e7fc640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5280102380 0x7f52801977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f527f7fe640 1 -- 172.21.15.83:0/3021579112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5280102380 msgr2=0x7f52801977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f527f7fe640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5280102380 0x7f52801977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f527f7fe640 1 -- 172.21.15.83:0/3021579112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5280101190 msgr2=0x7f5280197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f527f7fe640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5280101190 0x7f5280197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f527f7fe640 1 -- 172.21.15.83:0/3021579112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5270009560 con 0x7f5280102d40 2024-03-11T19:54:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f527f7fe640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280102d40 0x7f528019e830 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5278009fd0 tx=0x7f527800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f5267fff640 1 -- 172.21.15.83:0/3021579112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527800bea0 con 0x7f5280102d40 2024-03-11T19:54:57.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f5267fff640 1 -- 172.21.15.83:0/3021579112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5278009940 con 0x7f5280102d40 2024-03-11T19:54:57.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f528570c640 1 -- 172.21.15.83:0/3021579112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528019f380 con 0x7f5280102d40 2024-03-11T19:54:57.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f5267fff640 1 -- 172.21.15.83:0/3021579112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52780106c0 con 0x7f5280102d40 2024-03-11T19:54:57.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.617+0000 7f528570c640 1 -- 172.21.15.83:0/3021579112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528019f840 con 0x7f5280102d40 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.619+0000 7f5267fff640 1 -- 172.21.15.83:0/3021579112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5278004240 con 0x7f5280102d40 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f52883c0740 1 Processor -- start 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f52883c0740 1 -- start start 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f52883c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501fe1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f52883c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1c60 0x557501fe6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f52883c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501fe9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f52883c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501f9d960 con 0x557501fe6dd0 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f52883c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501f51310 con 0x557501fe1270 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f52883c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501f4e840 con 0x557501fe1c60 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f5267fff640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1a30 0x7f52680a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501fe9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501fe9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50706/0 (socket says 172.21.15.83:50706) 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527e7fc640 1 -- 172.21.15.83:0/1119493513 learned_addr learned my addr 172.21.15.83:0/1119493513 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527effd640 1 --2- 172.21.15.83:0/1119493513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1c60 0x557501fe6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527f7fe640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1a30 0x7f52680a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527e7fc640 1 --2- 172.21.15.83:0/1119493513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501fe1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f5267fff640 1 -- 172.21.15.83:0/3021579112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f52780cd120 con 0x7f5280102d40 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527e7fc640 1 -- 172.21.15.83:0/1119493513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1c60 msgr2=0x557501fe6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527e7fc640 1 --2- 172.21.15.83:0/1119493513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1c60 0x557501fe6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527e7fc640 1 -- 172.21.15.83:0/1119493513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 msgr2=0x557501fe1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527e7fc640 1 --2- 172.21.15.83:0/1119493513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501fe1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527e7fc640 1 -- 172.21.15.83:0/1119493513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557501f0c060 con 0x557501fe6dd0 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.620+0000 7f527e7fc640 1 --2- 172.21.15.83:0/1119493513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501fe9260 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5274009df0 tx=0x7f5274009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b4b384921b7b6d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f527f7fe640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1a30 0x7f52680a3ef0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f527809a0d0 tx=0x7f5278004640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52657fa640 1 -- 172.21.15.83:0/1119493513 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274004030 con 0x557501fe6dd0 2024-03-11T19:54:57.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52657fa640 1 -- 172.21.15.83:0/1119493513 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52740035f0 con 0x557501fe6dd0 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52657fa640 1 -- 172.21.15.83:0/1119493513 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527400f6e0 con 0x557501fe6dd0 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52883c0740 1 -- 172.21.15.83:0/1119493513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 msgr2=0x557501fe9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52883c0740 1 --2- 172.21.15.83:0/1119493513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501fe9260 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5274009df0 tx=0x7f5274009790 comp rx=0 tx=0).stop 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52883c0740 1 -- 172.21.15.83:0/1119493513 shutdown_connections 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52883c0740 1 --2- 172.21.15.83:0/1119493513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501fe9260 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52883c0740 1 --2- 172.21.15.83:0/1119493513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1c60 0x557501fe6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52883c0740 1 --2- 172.21.15.83:0/1119493513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501fe1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52883c0740 1 -- 172.21.15.83:0/1119493513 >> 172.21.15.83:0/1119493513 conn(0x557501fdc7e0 msgr2=0x557501fdcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52883c0740 1 -- 172.21.15.83:0/1119493513 shutdown_connections 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.621+0000 7f52883c0740 1 -- 172.21.15.83:0/1119493513 wait complete. 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f52883c0740 1 -- start start 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f52883c0740 1 Processor -- start 2024-03-11T19:54:57.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f52883c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501feb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f52883c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501feb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f52883c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe6dd0 0x557501ff29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f52883c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501f4a020 con 0x557501fe1c60 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f52883c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501c1d430 con 0x557501fe1270 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f52883c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501bd9e00 con 0x557501fe6dd0 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f52883c0740 10 client.0 _ll_register_callbacks cb 0x557501ef1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501feb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501feb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50722/0 (socket says 172.21.15.83:50722) 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527effd640 1 -- 172.21.15.83:0/54225954 learned_addr learned my addr 172.21.15.83:0/54225954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527e7fc640 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501feb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527e7fc640 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe6dd0 0x557501ff29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527effd640 1 -- 172.21.15.83:0/54225954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe6dd0 msgr2=0x557501ff29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527effd640 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe6dd0 0x557501ff29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527effd640 1 -- 172.21.15.83:0/54225954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 msgr2=0x557501feb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527effd640 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501feb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527effd640 1 -- 172.21.15.83:0/54225954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52740099e0 con 0x557501fe1c60 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527e7fc640 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe6dd0 0x557501ff29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f527effd640 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501feb9a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5270009230 tx=0x7f52700048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecb6ef8a098f739 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f5262ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f5262ff5640 1 -- 172.21.15.83:0/54225954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270004ae0 con 0x557501fe1c60 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.622+0000 7f5262ff5640 1 -- 172.21.15.83:0/54225954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5270004250 con 0x557501fe1c60 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f52883c0740 1 -- 172.21.15.83:0/54225954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557501c497e0 con 0x557501fe1c60 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f5262ff5640 1 -- 172.21.15.83:0/54225954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5270004410 con 0x557501fe1c60 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f52883c0740 10 client.18273 fetch_fsmap learned FSMap version 36 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f52883c0740 1 -- 172.21.15.83:0/54225954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x557501f486a0 con 0x557501fe1c60 2024-03-11T19:54:57.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f5262ff5640 1 -- 172.21.15.83:0/54225954 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557501c497e0 con 0x557501fe1c60 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f5262ff5640 1 -- 172.21.15.83:0/54225954 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f527003c050 con 0x557501fe1c60 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f52883c0740 10 client.18273 fetch_fsmap finished waiting for FSMap version 36 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f52883c0740 10 client.18273 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f52883c0740 20 client.18273 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f52883c0740 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557501ff7670 0x557501ff9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f52883c0740 4 client.18273 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f52883c0740 1 -- 172.21.15.83:0/54225954 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557501c86720 con 0x557501ff7670 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.623+0000 7f527e7fc640 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557501ff7670 0x557501ff9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.624+0000 7f527e7fc640 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557501ff7670 0x557501ff9a90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=722e87c350ee0661 server_cookie=9d414fe260c46bd2 in_seq=0 out_seq=0 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.624+0000 7f5262ff5640 10 client.18273 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.624+0000 7f5262ff5640 1 -- 172.21.15.83:0/54225954 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557501c86720 con 0x557501ff7670 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.624+0000 7f5262ff5640 10 client.18273 handle_command_reply: tid=0 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.624+0000 7f52883c0740 1 client.18273 shutdown 2024-03-11T19:54:57.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.624+0000 7f52883c0740 1 -- 172.21.15.83:0/54225954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 msgr2=0x557501feb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.625+0000 7f52883c0740 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501feb9a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5270009230 tx=0x7f52700048f0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.625+0000 7f52883c0740 1 -- 172.21.15.83:0/54225954 shutdown_connections 2024-03-11T19:54:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.625+0000 7f52883c0740 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x557501ff7670 0x557501ff9a90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.625+0000 7f52883c0740 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe6dd0 0x557501ff29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.625+0000 7f52883c0740 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501feb9a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.625+0000 7f52883c0740 1 --2- 172.21.15.83:0/54225954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501feb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.625+0000 7f52883c0740 1 -- 172.21.15.83:0/54225954 >> 172.21.15.83:0/54225954 conn(0x557501fdc7e0 msgr2=0x557501fdbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.625+0000 7f5262ff5640 0 client.18273 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:54:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.625+0000 7f52883c0740 1 -- 172.21.15.83:0/54225954 shutdown_connections 2024-03-11T19:54:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.625+0000 7f52883c0740 1 -- 172.21.15.83:0/54225954 wait complete. 2024-03-11T19:54:57.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.625+0000 7f52883c0740 20 client.18273 trim_cache size 0 max 16384 2024-03-11T19:54:57.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f52883c0740 1 Processor -- start 2024-03-11T19:54:57.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f52883c0740 1 -- start start 2024-03-11T19:54:57.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f52883c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501fd7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f52883c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1c60 0x557501ff29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f52883c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501feb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f52883c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501ecf130 con 0x557501fe6dd0 2024-03-11T19:54:57.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f52883c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501c497e0 con 0x557501fe1270 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f52883c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501fcea40 con 0x557501fe1c60 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f527e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501feb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f527e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501feb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50724/0 (socket says 172.21.15.83:50724) 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f527e7fc640 1 -- 172.21.15.83:0/3576583427 learned_addr learned my addr 172.21.15.83:0/3576583427 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.636+0000 7f527e7fc640 1 --2- 172.21.15.83:0/3576583427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501fd7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.637+0000 7f527effd640 1 --2- 172.21.15.83:0/3576583427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1c60 0x557501ff29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.637+0000 7f527e7fc640 1 -- 172.21.15.83:0/3576583427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1c60 msgr2=0x557501ff29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.637+0000 7f527e7fc640 1 --2- 172.21.15.83:0/3576583427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1c60 0x557501ff29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.637+0000 7f527e7fc640 1 -- 172.21.15.83:0/3576583427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 msgr2=0x557501fd7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.637+0000 7f527e7fc640 1 --2- 172.21.15.83:0/3576583427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501fd7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.637+0000 7f527e7fc640 1 -- 172.21.15.83:0/3576583427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557501ee4c20 con 0x557501fe6dd0 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.637+0000 7f527e7fc640 1 --2- 172.21.15.83:0/3576583427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501feb460 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5274013c60 tx=0x7f5274016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cdbb0dea32b0a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.637+0000 7f5264ff9640 1 -- 172.21.15.83:0/3576583427 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274026040 con 0x557501fe6dd0 2024-03-11T19:54:57.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.637+0000 7f5264ff9640 1 -- 172.21.15.83:0/3576583427 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5274005640 con 0x557501fe6dd0 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.637+0000 7f5264ff9640 1 -- 172.21.15.83:0/3576583427 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527402aa90 con 0x557501fe6dd0 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 -- 172.21.15.83:0/3576583427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 msgr2=0x557501feb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 --2- 172.21.15.83:0/3576583427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501feb460 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5274013c60 tx=0x7f5274016040 comp rx=0 tx=0).stop 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 -- 172.21.15.83:0/3576583427 shutdown_connections 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 --2- 172.21.15.83:0/3576583427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe6dd0 0x557501feb460 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 --2- 172.21.15.83:0/3576583427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1c60 0x557501ff29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 --2- 172.21.15.83:0/3576583427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe1270 0x557501fd7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 -- 172.21.15.83:0/3576583427 >> 172.21.15.83:0/3576583427 conn(0x557501fdc7e0 msgr2=0x557501fdcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 -- 172.21.15.83:0/3576583427 shutdown_connections 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 -- 172.21.15.83:0/3576583427 wait complete. 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 -- start start 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 Processor -- start 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1270 0x557501ff2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501ff0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe6dd0 0x557501ff1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501ff2df0 con 0x557501fe1c60 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501ff1690 con 0x557501fe6dd0 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557501ff1850 con 0x557501fe1270 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f52883c0740 10 client.0 _ll_register_callbacks cb 0x7f52740096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f527effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501ff0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f527effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501ff0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50728/0 (socket says 172.21.15.83:50728) 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f527effd640 1 -- 172.21.15.83:0/3588248735 learned_addr learned my addr 172.21.15.83:0/3588248735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f527e7fc640 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1270 0x557501ff2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f527effd640 1 -- 172.21.15.83:0/3588248735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1270 msgr2=0x557501ff2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f527effd640 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1270 0x557501ff2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f527effd640 1 -- 172.21.15.83:0/3588248735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe6dd0 msgr2=0x557501ff1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f527effd640 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe6dd0 0x557501ff1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f527effd640 1 -- 172.21.15.83:0/3588248735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557501f0c060 con 0x557501fe1c60 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.638+0000 7f527e7fc640 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1270 0x557501ff2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f527effd640 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501ff0c10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x557501fd8740 tx=0x7f5270004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7194e1448450e80d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f5262ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f5262ff5640 1 -- 172.21.15.83:0/3588248735 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270004450 con 0x557501fe1c60 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f5262ff5640 1 -- 172.21.15.83:0/3588248735 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5270000e40 con 0x557501fe1c60 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f52883c0740 1 -- 172.21.15.83:0/3588248735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52740035f0 con 0x557501fe1c60 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f5262ff5640 1 -- 172.21.15.83:0/3588248735 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5270005370 con 0x557501fe1c60 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f52883c0740 10 client.18279 fetch_fsmap learned FSMap version 36 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f52883c0740 1 -- 172.21.15.83:0/3588248735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x557501f0a8c0 con 0x557501fe1c60 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f5262ff5640 1 -- 172.21.15.83:0/3588248735 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52740035f0 con 0x557501fe1c60 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f5262ff5640 1 -- 172.21.15.83:0/3588248735 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f527003c050 con 0x557501fe1c60 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f52883c0740 10 client.18279 fetch_fsmap finished waiting for FSMap version 36 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f52883c0740 10 client.18279 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-11T19:54:57.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f52883c0740 20 client.18279 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f52883c0740 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55750204eef0 0x557502051310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f52883c0740 4 client.18279 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.639+0000 7f52883c0740 1 -- 172.21.15.83:0/3588248735 --> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557501cd5890 con 0x55750204eef0 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f527e7fc640 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55750204eef0 0x557502051310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f527e7fc640 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55750204eef0 0x557502051310 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=721c9ccda0843932 server_cookie=4247e59bf3483c08 in_seq=0 out_seq=0 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f5262ff5640 10 client.18279 ms_handle_connect on v2:172.21.15.204:6838/862560872 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f5262ff5640 1 -- 172.21.15.83:0/3588248735 <== mds.1 v2:172.21.15.204:6838/862560872 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557501cd5890 con 0x55750204eef0 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f5262ff5640 10 client.18279 handle_command_reply: tid=0 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 1 client.18279 shutdown 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 1 -- 172.21.15.83:0/3588248735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 msgr2=0x557501ff0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501ff0c10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x557501fd8740 tx=0x7f5270004bf0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 1 -- 172.21.15.83:0/3588248735 shutdown_connections 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.204:6838/862560872,v1:172.21.15.204:6839/862560872] conn(0x55750204eef0 0x557502051310 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557501fe6dd0 0x557501ff1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557501fe1c60 0x557501ff0c10 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f5262ff5640 0 client.18279 ms_handle_reset on v2:172.21.15.204:6838/862560872 2024-03-11T19:54:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 1 --2- 172.21.15.83:0/3588248735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557501fe1270 0x557501ff2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 1 -- 172.21.15.83:0/3588248735 >> 172.21.15.83:0/3588248735 conn(0x557501fdc7e0 msgr2=0x557501fdcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 1 -- 172.21.15.83:0/3588248735 shutdown_connections 2024-03-11T19:54:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 1 -- 172.21.15.83:0/3588248735 wait complete. 2024-03-11T19:54:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.640+0000 7f52883c0740 20 client.18279 trim_cache size 0 max 16384 2024-03-11T19:54:57.653 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:54:57.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 -- 172.21.15.83:0/3021579112 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1a30 msgr2=0x7f52680a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1a30 0x7f52680a3ef0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f527809a0d0 tx=0x7f5278004640 comp rx=0 tx=0).stop 2024-03-11T19:54:57.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 -- 172.21.15.83:0/3021579112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280102d40 msgr2=0x7f528019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280102d40 0x7f528019e830 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5278009fd0 tx=0x7f527800b260 comp rx=0 tx=0).stop 2024-03-11T19:54:57.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 -- 172.21.15.83:0/3021579112 shutdown_connections 2024-03-11T19:54:57.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f52680a1a30 0x7f52680a3ef0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280102d40 0x7f528019e830 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5280102380 0x7f52801977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 --2- 172.21.15.83:0/3021579112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5280101190 0x7f5280197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 -- 172.21.15.83:0/3021579112 >> 172.21.15.83:0/3021579112 conn(0x7f52800fc700 msgr2=0x7f52801056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 -- 172.21.15.83:0/3021579112 shutdown_connections 2024-03-11T19:54:57.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.641+0000 7f528570c640 1 -- 172.21.15.83:0/3021579112 wait complete. 2024-03-11T19:54:57.656 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-11T19:54:57.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-11T19:54:57.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.756+0000 7f224b683640 1 -- 172.21.15.83:0/1029191552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244100080 msgr2=0x7f22441004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.756+0000 7f224b683640 1 --2- 172.21.15.83:0/1029191552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244100080 0x7f22441004a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2234009900 tx=0x7f223402bc30 comp rx=0 tx=0).stop 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.756+0000 7f224b683640 1 -- 172.21.15.83:0/1029191552 shutdown_connections 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.756+0000 7f224b683640 1 --2- 172.21.15.83:0/1029191552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244101c30 0x7f22441084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.756+0000 7f224b683640 1 --2- 172.21.15.83:0/1029191552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2244101270 0x7f22441016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.756+0000 7f224b683640 1 --2- 172.21.15.83:0/1029191552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244100080 0x7f22441004a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.756+0000 7f224b683640 1 -- 172.21.15.83:0/1029191552 >> 172.21.15.83:0/1029191552 conn(0x7f22440fb610 msgr2=0x7f22440fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.756+0000 7f224b683640 1 -- 172.21.15.83:0/1029191552 shutdown_connections 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.756+0000 7f224b683640 1 -- 172.21.15.83:0/1029191552 wait complete. 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.756+0000 7f224b683640 1 Processor -- start 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.756+0000 7f224b683640 1 -- start start 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f224b683640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2244100080 0x7f2244195050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f224b683640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101270 0x7f2244195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f224b683640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244101c30 0x7f224419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f224b683640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224419cb40 con 0x7f2244101270 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f224b683640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224419ccb0 con 0x7f2244100080 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f224b683640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224419ce70 con 0x7f2244101c30 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f2248c1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101270 0x7f2244195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f2248c1e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101270 0x7f2244195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50744/0 (socket says 172.21.15.83:50744) 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f2248c1e640 1 -- 172.21.15.83:0/3588212791 learned_addr learned my addr 172.21.15.83:0/3588212791 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f224941f640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2244100080 0x7f2244195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f2249c20640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244101c30 0x7f224419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f2248c1e640 1 -- 172.21.15.83:0/3588212791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244101c30 msgr2=0x7f224419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f2248c1e640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244101c30 0x7f224419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f2248c1e640 1 -- 172.21.15.83:0/3588212791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2244100080 msgr2=0x7f2244195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f2248c1e640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2244100080 0x7f2244195050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f2248c1e640 1 -- 172.21.15.83:0/3588212791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2234009560 con 0x7f2244101270 2024-03-11T19:54:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f2248c1e640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101270 0x7f2244195590 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f224000d950 tx=0x7f224000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f223a7fc640 1 -- 172.21.15.83:0/3588212791 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240009c30 con 0x7f2244101270 2024-03-11T19:54:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f223a7fc640 1 -- 172.21.15.83:0/3588212791 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2240009d90 con 0x7f2244101270 2024-03-11T19:54:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f224b683640 1 -- 172.21.15.83:0/3588212791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224419d150 con 0x7f2244101270 2024-03-11T19:54:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f223a7fc640 1 -- 172.21.15.83:0/3588212791 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224000f7e0 con 0x7f2244101270 2024-03-11T19:54:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.757+0000 7f224b683640 1 -- 172.21.15.83:0/3588212791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224419d660 con 0x7f2244101270 2024-03-11T19:54:57.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.759+0000 7f223a7fc640 1 -- 172.21.15.83:0/3588212791 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f2240010ad0 con 0x7f2244101270 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f224e24b740 1 Processor -- start 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f224e24b740 1 -- start start 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f224e24b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91270 0x55c6b3e91690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f224e24b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91c60 0x55c6b3e96760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f224e24b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e96dd0 0x55c6b3e99260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f223a7fc640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22300a3c40 0x7f22300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f224e24b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3e4d960 con 0x55c6b3e91c60 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f224e24b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3e01310 con 0x55c6b3e91270 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f224e24b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3dfe840 con 0x55c6b3e96dd0 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f224941f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91c60 0x55c6b3e96760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f224941f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91c60 0x55c6b3e96760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50758/0 (socket says 172.21.15.83:50758) 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f224941f640 1 -- 172.21.15.83:0/376942618 learned_addr learned my addr 172.21.15.83:0/376942618 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.760+0000 7f2249c20640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22300a3c40 0x7f22300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f2249c20640 1 --2- 172.21.15.83:0/376942618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91270 0x55c6b3e91690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f223a7fc640 1 -- 172.21.15.83:0/3588212791 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f22400ccf30 con 0x7f2244101270 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f2248c1e640 1 --2- 172.21.15.83:0/376942618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e96dd0 0x55c6b3e99260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f224941f640 1 -- 172.21.15.83:0/376942618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e96dd0 msgr2=0x55c6b3e99260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f224941f640 1 --2- 172.21.15.83:0/376942618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e96dd0 0x55c6b3e99260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f224941f640 1 -- 172.21.15.83:0/376942618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91270 msgr2=0x55c6b3e91690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f224941f640 1 --2- 172.21.15.83:0/376942618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91270 0x55c6b3e91690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f224941f640 1 -- 172.21.15.83:0/376942618 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6b3dbc060 con 0x55c6b3e91c60 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f2249c20640 1 --2- 172.21.15.83:0/376942618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91270 0x55c6b3e91690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f224941f640 1 --2- 172.21.15.83:0/376942618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91c60 0x55c6b3e96760 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2234009900 tx=0x7f2234005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19873ccf95361e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f2249c20640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22300a3c40 0x7f22300a6100 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x55c6b3e924c0 tx=0x7f223c009b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f222f7fe640 1 -- 172.21.15.83:0/376942618 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223402cbd0 con 0x55c6b3e91c60 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f222f7fe640 1 -- 172.21.15.83:0/376942618 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2234004030 con 0x55c6b3e91c60 2024-03-11T19:54:57.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.761+0000 7f222f7fe640 1 -- 172.21.15.83:0/376942618 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234005270 con 0x55c6b3e91c60 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.762+0000 7f224e24b740 1 -- 172.21.15.83:0/376942618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91c60 msgr2=0x55c6b3e96760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.762+0000 7f224e24b740 1 --2- 172.21.15.83:0/376942618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91c60 0x55c6b3e96760 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2234009900 tx=0x7f2234005c00 comp rx=0 tx=0).stop 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.762+0000 7f224e24b740 1 -- 172.21.15.83:0/376942618 shutdown_connections 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.762+0000 7f224e24b740 1 --2- 172.21.15.83:0/376942618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e96dd0 0x55c6b3e99260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.762+0000 7f224e24b740 1 --2- 172.21.15.83:0/376942618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91c60 0x55c6b3e96760 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.762+0000 7f224e24b740 1 --2- 172.21.15.83:0/376942618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91270 0x55c6b3e91690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.762+0000 7f224e24b740 1 -- 172.21.15.83:0/376942618 >> 172.21.15.83:0/376942618 conn(0x55c6b3e8c7e0 msgr2=0x55c6b3e8cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.762+0000 7f224e24b740 1 -- 172.21.15.83:0/376942618 shutdown_connections 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.762+0000 7f224e24b740 1 -- 172.21.15.83:0/376942618 wait complete. 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.762+0000 7f224e24b740 1 -- start start 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.762+0000 7f224e24b740 1 Processor -- start 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f224e24b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91270 0x55c6b3e9b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f224e24b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e91c60 0x55c6b3e9b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f224e24b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e96dd0 0x55c6b3ea29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f224e24b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3dfa020 con 0x55c6b3e96dd0 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f224e24b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3acd430 con 0x55c6b3e91270 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f224e24b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3a89e00 con 0x55c6b3e91c60 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f224e24b740 10 client.0 _ll_register_callbacks cb 0x55c6b3da1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f2249c20640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e96dd0 0x55c6b3ea29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f2249c20640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e96dd0 0x55c6b3ea29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50772/0 (socket says 172.21.15.83:50772) 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f2249c20640 1 -- 172.21.15.83:0/1073870989 learned_addr learned my addr 172.21.15.83:0/1073870989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f224941f640 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91270 0x55c6b3e9b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f2248c1e640 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e91c60 0x55c6b3e9b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f2249c20640 1 -- 172.21.15.83:0/1073870989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e91c60 msgr2=0x55c6b3e9b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f2249c20640 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e91c60 0x55c6b3e9b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f2249c20640 1 -- 172.21.15.83:0/1073870989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91270 msgr2=0x55c6b3e9b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f2249c20640 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91270 0x55c6b3e9b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f2249c20640 1 -- 172.21.15.83:0/1073870989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2244108dc0 con 0x55c6b3e96dd0 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f2249c20640 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e96dd0 0x55c6b3ea29a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f223c003c20 tx=0x7f223c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8156cb32cd41067 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f222cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f222cff9640 1 -- 172.21.15.83:0/1073870989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c003e50 con 0x55c6b3e96dd0 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.763+0000 7f222cff9640 1 -- 172.21.15.83:0/1073870989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f223c0045b0 con 0x55c6b3e96dd0 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f224e24b740 1 -- 172.21.15.83:0/1073870989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6b3af97e0 con 0x55c6b3e96dd0 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f222cff9640 1 -- 172.21.15.83:0/1073870989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f223c004770 con 0x55c6b3e96dd0 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f224e24b740 10 client.18291 fetch_fsmap learned FSMap version 36 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f224e24b740 1 -- 172.21.15.83:0/1073870989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c6b3df86a0 con 0x55c6b3e96dd0 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f222cff9640 1 -- 172.21.15.83:0/1073870989 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c6b3af97e0 con 0x55c6b3e96dd0 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f222cff9640 1 -- 172.21.15.83:0/1073870989 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f223c020590 con 0x55c6b3e96dd0 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f224e24b740 10 client.18291 fetch_fsmap finished waiting for FSMap version 36 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f224e24b740 10 client.18291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:57.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f224e24b740 20 client.18291 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f224e24b740 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c6b3ea7760 0x55c6b3ea9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f224e24b740 4 client.18291 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f224e24b740 1 -- 172.21.15.83:0/1073870989 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6b3b36720 con 0x55c6b3ea7760 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.764+0000 7f224941f640 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c6b3ea7760 0x55c6b3ea9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.765+0000 7f222cff9640 10 client.18291 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.765+0000 7f224941f640 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c6b3ea7760 0x55c6b3ea9b80 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4de795520695ee1 server_cookie=cbfd2e3585daefe in_seq=0 out_seq=0 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.765+0000 7f222cff9640 1 -- 172.21.15.83:0/1073870989 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6b3b36720 con 0x55c6b3ea7760 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.765+0000 7f222cff9640 10 client.18291 handle_command_reply: tid=0 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.765+0000 7f224e24b740 1 client.18291 shutdown 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f224e24b740 1 -- 172.21.15.83:0/1073870989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e96dd0 msgr2=0x55c6b3ea29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f224e24b740 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e96dd0 0x55c6b3ea29a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f223c003c20 tx=0x7f223c004ab0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f224e24b740 1 -- 172.21.15.83:0/1073870989 shutdown_connections 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f224e24b740 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c6b3ea7760 0x55c6b3ea9b80 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f224e24b740 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e96dd0 0x55c6b3ea29a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f224e24b740 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e91c60 0x55c6b3e9b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f222cff9640 0 client.18291 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:57.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f224e24b740 1 --2- 172.21.15.83:0/1073870989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91270 0x55c6b3e9b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f224e24b740 1 -- 172.21.15.83:0/1073870989 >> 172.21.15.83:0/1073870989 conn(0x55c6b3e8c7e0 msgr2=0x55c6b3e8bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f224e24b740 1 -- 172.21.15.83:0/1073870989 shutdown_connections 2024-03-11T19:54:57.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f224e24b740 1 -- 172.21.15.83:0/1073870989 wait complete. 2024-03-11T19:54:57.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.766+0000 7f224e24b740 20 client.18291 trim_cache size 0 max 16384 2024-03-11T19:54:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.777+0000 7f224e24b740 1 Processor -- start 2024-03-11T19:54:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.777+0000 7f224e24b740 1 -- start start 2024-03-11T19:54:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.777+0000 7f224e24b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3e8cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.777+0000 7f224e24b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91c60 0x55c6b3ea2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.777+0000 7f224e24b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e96dd0 0x55c6b3e931b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.777+0000 7f224e24b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3d7f130 con 0x55c6b3e91270 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.777+0000 7f224e24b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3e9b260 con 0x55c6b3e91c60 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.777+0000 7f224e24b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3b36720 con 0x55c6b3e96dd0 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f224941f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3e8cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f224941f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3e8cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50774/0 (socket says 172.21.15.83:50774) 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f224941f640 1 -- 172.21.15.83:0/1758642194 learned_addr learned my addr 172.21.15.83:0/1758642194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f224941f640 1 -- 172.21.15.83:0/1758642194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e96dd0 msgr2=0x55c6b3e931b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f224941f640 1 --2- 172.21.15.83:0/1758642194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e96dd0 0x55c6b3e931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f224941f640 1 -- 172.21.15.83:0/1758642194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91c60 msgr2=0x55c6b3ea2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f224941f640 1 --2- 172.21.15.83:0/1758642194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91c60 0x55c6b3ea2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f224941f640 1 -- 172.21.15.83:0/1758642194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6b3d94c20 con 0x55c6b3e91270 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f224941f640 1 --2- 172.21.15.83:0/1758642194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3e8cbc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x55c6b3e926d0 tx=0x7f2234041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e2a72fc515b134 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f222effd640 1 -- 172.21.15.83:0/1758642194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340043e0 con 0x55c6b3e91270 2024-03-11T19:54:57.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f222effd640 1 -- 172.21.15.83:0/1758642194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2234050b70 con 0x55c6b3e91270 2024-03-11T19:54:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.778+0000 7f222effd640 1 -- 172.21.15.83:0/1758642194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340508c0 con 0x55c6b3e91270 2024-03-11T19:54:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 -- 172.21.15.83:0/1758642194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 msgr2=0x55c6b3e8cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 --2- 172.21.15.83:0/1758642194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3e8cbc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x55c6b3e926d0 tx=0x7f2234041040 comp rx=0 tx=0).stop 2024-03-11T19:54:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 -- 172.21.15.83:0/1758642194 shutdown_connections 2024-03-11T19:54:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 --2- 172.21.15.83:0/1758642194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e96dd0 0x55c6b3e931b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 --2- 172.21.15.83:0/1758642194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e91c60 0x55c6b3ea2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 --2- 172.21.15.83:0/1758642194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3e8cbc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 -- 172.21.15.83:0/1758642194 >> 172.21.15.83:0/1758642194 conn(0x55c6b3e8c7e0 msgr2=0x55c6b3e87a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 -- 172.21.15.83:0/1758642194 shutdown_connections 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 -- 172.21.15.83:0/1758642194 wait complete. 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 -- start start 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 Processor -- start 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3ea2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e91c60 0x55c6b3ea2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e96dd0 0x55c6b3ea0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3ea1110 con 0x55c6b3e91270 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3ea1280 con 0x55c6b3e96dd0 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6b3ea1440 con 0x55c6b3e91c60 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224e24b740 10 client.0 _ll_register_callbacks cb 0x7f2234005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224941f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3ea2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224941f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3ea2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50780/0 (socket says 172.21.15.83:50780) 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.779+0000 7f224941f640 1 -- 172.21.15.83:0/240388226 learned_addr learned my addr 172.21.15.83:0/240388226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f2249c20640 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e96dd0 0x55c6b3ea0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f224941f640 1 -- 172.21.15.83:0/240388226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e91c60 msgr2=0x55c6b3ea2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f224941f640 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e91c60 0x55c6b3ea2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f224941f640 1 -- 172.21.15.83:0/240388226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e96dd0 msgr2=0x55c6b3ea0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f224941f640 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e96dd0 0x55c6b3ea0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f224941f640 1 -- 172.21.15.83:0/240388226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6b3dbc060 con 0x55c6b3e91270 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f2249c20640 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e96dd0 0x55c6b3ea0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f224941f640 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3ea2400 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f223402bc70 tx=0x7f2234004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e55e28c17c37e03 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f222cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f222cff9640 1 -- 172.21.15.83:0/240388226 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340046c0 con 0x55c6b3e91270 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f222cff9640 1 -- 172.21.15.83:0/240388226 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f223405f070 con 0x55c6b3e91270 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f224e24b740 1 -- 172.21.15.83:0/240388226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2234004030 con 0x55c6b3e91270 2024-03-11T19:54:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f222cff9640 1 -- 172.21.15.83:0/240388226 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2234004820 con 0x55c6b3e91270 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f224e24b740 10 client.18297 fetch_fsmap learned FSMap version 36 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f224e24b740 1 -- 172.21.15.83:0/240388226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c6b3dba8c0 con 0x55c6b3e91270 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f222cff9640 1 -- 172.21.15.83:0/240388226 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2234004030 con 0x55c6b3e91270 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.780+0000 7f222cff9640 1 -- 172.21.15.83:0/240388226 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 36) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f223406d070 con 0x55c6b3e91270 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f224e24b740 10 client.18297 fetch_fsmap finished waiting for FSMap version 36 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f224e24b740 10 client.18297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f224e24b740 20 client.18297 populate_metadata read hostname 'smithi083' 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f224e24b740 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c6b3efed90 0x55c6b3f011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f224e24b740 4 client.18297 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f224e24b740 1 -- 172.21.15.83:0/240388226 --> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6b3b85890 con 0x55c6b3efed90 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f2248c1e640 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c6b3efed90 0x55c6b3f011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f2248c1e640 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c6b3efed90 0x55c6b3f011b0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e73ae7c5fe89eb server_cookie=3ae91ce4512c0747 in_seq=0 out_seq=0 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f222cff9640 10 client.18297 ms_handle_connect on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f222cff9640 1 -- 172.21.15.83:0/240388226 <== mds.0 v2:172.21.15.137:6838/709970923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6b3b85890 con 0x55c6b3efed90 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f222cff9640 10 client.18297 handle_command_reply: tid=0 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.781+0000 7f224e24b740 1 client.18297 shutdown 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224e24b740 1 -- 172.21.15.83:0/240388226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 msgr2=0x55c6b3ea2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224e24b740 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3ea2400 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f223402bc70 tx=0x7f2234004580 comp rx=0 tx=0).stop 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224e24b740 1 -- 172.21.15.83:0/240388226 shutdown_connections 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224e24b740 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.137:6838/709970923,v1:172.21.15.137:6839/709970923] conn(0x55c6b3efed90 0x55c6b3f011b0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224e24b740 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6b3e96dd0 0x55c6b3ea0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f222cff9640 0 client.18297 ms_handle_reset on v2:172.21.15.137:6838/709970923 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224e24b740 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6b3e91c60 0x55c6b3ea2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224e24b740 1 --2- 172.21.15.83:0/240388226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6b3e91270 0x55c6b3ea2400 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224e24b740 1 -- 172.21.15.83:0/240388226 >> 172.21.15.83:0/240388226 conn(0x55c6b3e8c7e0 msgr2=0x55c6b3e8cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224e24b740 1 -- 172.21.15.83:0/240388226 shutdown_connections 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224e24b740 1 -- 172.21.15.83:0/240388226 wait complete. 2024-03-11T19:54:57.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224e24b740 20 client.18297 trim_cache size 0 max 16384 2024-03-11T19:54:57.793 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 -- 172.21.15.83:0/3588212791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22300a3c40 msgr2=0x7f22300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22300a3c40 0x7f22300a6100 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x55c6b3e924c0 tx=0x7f223c009b50 comp rx=0 tx=0).stop 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 -- 172.21.15.83:0/3588212791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101270 msgr2=0x7f2244195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101270 0x7f2244195590 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f224000d950 tx=0x7f224000de10 comp rx=0 tx=0).stop 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 -- 172.21.15.83:0/3588212791 shutdown_connections 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f22300a3c40 0x7f22300a6100 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2244101c30 0x7f224419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244101270 0x7f2244195590 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 --2- 172.21.15.83:0/3588212791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2244100080 0x7f2244195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 -- 172.21.15.83:0/3588212791 >> 172.21.15.83:0/3588212791 conn(0x7f22440fb610 msgr2=0x7f2244104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 -- 172.21.15.83:0/3588212791 shutdown_connections 2024-03-11T19:54:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:57.782+0000 7f224b683640 1 -- 172.21.15.83:0/3588212791 wait complete. 2024-03-11T19:54:58.067 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:57 smithi083 ceph-mon[31864]: pgmap v821: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-03-11T19:54:58.068 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:57 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1125994711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:54:58.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:57 smithi204 ceph-mon[36739]: pgmap v821: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-03-11T19:54:58.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:57 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1125994711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:54:58.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:57 smithi137 ceph-mon[36220]: pgmap v821: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-03-11T19:54:58.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:57 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1125994711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:54:58.798 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-11T19:54:58.798 INFO:tasks.fwd_scrub:done joining 2024-03-11T19:54:58.798 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-11T19:54:58.811 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:54:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.921+0000 7f6f4390d640 1 -- 172.21.15.83:0/4235434871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c1045b0 msgr2=0x7f6f3c104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.921+0000 7f6f4390d640 1 --2- 172.21.15.83:0/4235434871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c1045b0 0x7f6f3c104a30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009900 tx=0x7f6f2c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.921+0000 7f6f4390d640 1 -- 172.21.15.83:0/4235434871 shutdown_connections 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.921+0000 7f6f4390d640 1 --2- 172.21.15.83:0/4235434871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f3c104f70 0x7f6f3c10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.921+0000 7f6f4390d640 1 --2- 172.21.15.83:0/4235434871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c1045b0 0x7f6f3c104a30 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.921+0000 7f6f4390d640 1 --2- 172.21.15.83:0/4235434871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f3c1033c0 0x7f6f3c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.921+0000 7f6f4390d640 1 -- 172.21.15.83:0/4235434871 >> 172.21.15.83:0/4235434871 conn(0x7f6f3c0fe930 msgr2=0x7f6f3c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f4390d640 1 -- 172.21.15.83:0/4235434871 shutdown_connections 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f4390d640 1 -- 172.21.15.83:0/4235434871 wait complete. 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f4390d640 1 Processor -- start 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f4390d640 1 -- start start 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f4390d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c1033c0 0x7f6f3c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f4390d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f3c1045b0 0x7f6f3c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f4390d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f3c104f70 0x7f6f3c1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f4390d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c1a0ed0 con 0x7f6f3c1033c0 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f4390d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c1a1040 con 0x7f6f3c104f70 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f4390d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c1a1200 con 0x7f6f3c1045b0 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f416a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c1033c0 0x7f6f3c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f416a9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c1033c0 0x7f6f3c1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50792/0 (socket says 172.21.15.83:50792) 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.922+0000 7f6f416a9640 1 -- 172.21.15.83:0/2483806737 learned_addr learned my addr 172.21.15.83:0/2483806737 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f41eaa640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f3c104f70 0x7f6f3c1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f416a9640 1 -- 172.21.15.83:0/2483806737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f3c1045b0 msgr2=0x7f6f3c199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f416a9640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f3c1045b0 0x7f6f3c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f416a9640 1 -- 172.21.15.83:0/2483806737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f3c104f70 msgr2=0x7f6f3c1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f416a9640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f3c104f70 0x7f6f3c1a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f416a9640 1 -- 172.21.15.83:0/2483806737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c009560 con 0x7f6f3c1033c0 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f416a9640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c1033c0 0x7f6f3c1993e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800d770 tx=0x7f6f3800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f327fc640 1 -- 172.21.15.83:0/2483806737 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38009c30 con 0x7f6f3c1033c0 2024-03-11T19:54:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f4390d640 1 -- 172.21.15.83:0/2483806737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3c1a14e0 con 0x7f6f3c1033c0 2024-03-11T19:54:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f327fc640 1 -- 172.21.15.83:0/2483806737 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f3800de80 con 0x7f6f3c1033c0 2024-03-11T19:54:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f327fc640 1 -- 172.21.15.83:0/2483806737 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38010460 con 0x7f6f3c1033c0 2024-03-11T19:54:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.923+0000 7f6f4390d640 1 -- 172.21.15.83:0/2483806737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3c1a19f0 con 0x7f6f3c1033c0 2024-03-11T19:54:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.924+0000 7f6f4390d640 1 -- 172.21.15.83:0/2483806737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3c10a6e0 con 0x7f6f3c1033c0 2024-03-11T19:54:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.925+0000 7f6f327fc640 1 -- 172.21.15.83:0/2483806737 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f6f38003f60 con 0x7f6f3c1033c0 2024-03-11T19:54:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.926+0000 7f6f327fc640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f280a1a30 0x7f6f280a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.926+0000 7f6f40ea8640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f280a1a30 0x7f6f280a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.926+0000 7f6f327fc640 1 -- 172.21.15.83:0/2483806737 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f6f380cd130 con 0x7f6f3c1033c0 2024-03-11T19:54:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.926+0000 7f6f40ea8640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f280a1a30 0x7f6f280a3ef0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c02c6a0 tx=0x7f6f2c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:58.929+0000 7f6f327fc640 1 -- 172.21.15.83:0/2483806737 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6f380963f0 con 0x7f6f3c1033c0 2024-03-11T19:54:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.104+0000 7f6f4390d640 1 -- 172.21.15.83:0/2483806737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f3c10edd0 con 0x7f6f3c1033c0 2024-03-11T19:54:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.106+0000 7f6f327fc640 1 -- 172.21.15.83:0/2483806737 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7f6f38017020 con 0x7f6f3c1033c0 2024-03-11T19:54:59.109 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:54:59.110 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:54:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 -- 172.21.15.83:0/2483806737 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f280a1a30 msgr2=0x7f6f280a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f280a1a30 0x7f6f280a3ef0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c02c6a0 tx=0x7f6f2c000bc0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 -- 172.21.15.83:0/2483806737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c1033c0 msgr2=0x7f6f3c1993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c1033c0 0x7f6f3c1993e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800d770 tx=0x7f6f3800dc30 comp rx=0 tx=0).stop 2024-03-11T19:54:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 -- 172.21.15.83:0/2483806737 shutdown_connections 2024-03-11T19:54:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f6f280a1a30 0x7f6f280a3ef0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f3c104f70 0x7f6f3c1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f3c1045b0 0x7f6f3c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 --2- 172.21.15.83:0/2483806737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f3c1033c0 0x7f6f3c1993e0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 -- 172.21.15.83:0/2483806737 >> 172.21.15.83:0/2483806737 conn(0x7f6f3c0fe930 msgr2=0x7f6f3c1078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 -- 172.21.15.83:0/2483806737 shutdown_connections 2024-03-11T19:54:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.108+0000 7f6f4390d640 1 -- 172.21.15.83:0/2483806737 wait complete. 2024-03-11T19:54:59.123 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:37:03.034982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:37:04.125352+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:54:59.124 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:54:59.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 -- 172.21.15.83:0/3502721051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe540101c50 msgr2=0x7fe5401084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 --2- 172.21.15.83:0/3502721051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe540101c50 0x7fe5401084e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fe538009ca0 tx=0x7fe53802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 -- 172.21.15.83:0/3502721051 shutdown_connections 2024-03-11T19:54:59.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 --2- 172.21.15.83:0/3502721051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe540101c50 0x7fe5401084e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 --2- 172.21.15.83:0/3502721051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe540101290 0x7fe540101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 --2- 172.21.15.83:0/3502721051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5401000a0 0x7fe5401004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 -- 172.21.15.83:0/3502721051 >> 172.21.15.83:0/3502721051 conn(0x7fe5400fb630 msgr2=0x7fe5400fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 -- 172.21.15.83:0/3502721051 shutdown_connections 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 -- 172.21.15.83:0/3502721051 wait complete. 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 Processor -- start 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 -- start start 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.225+0000 7fe548325640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5401000a0 0x7fe5401971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe548325640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe540101290 0x7fe540197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe548325640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe540101c50 0x7fe54019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe548325640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54019ece0 con 0x7fe5401000a0 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe548325640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54019ee50 con 0x7fe540101290 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe548325640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54019f010 con 0x7fe540101c50 2024-03-11T19:54:59.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5460c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5401000a0 0x7fe5401971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5460c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5401000a0 0x7fe5401971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50800/0 (socket says 172.21.15.83:50800) 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5460c1640 1 -- 172.21.15.83:0/2949596811 learned_addr learned my addr 172.21.15.83:0/2949596811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5460c1640 1 -- 172.21.15.83:0/2949596811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe540101c50 msgr2=0x7fe54019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5468c2640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe540101c50 0x7fe54019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5460c1640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe540101c50 0x7fe54019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5460c1640 1 -- 172.21.15.83:0/2949596811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe540101290 msgr2=0x7fe540197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5460c1640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe540101290 0x7fe540197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5460c1640 1 -- 172.21.15.83:0/2949596811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe538009900 con 0x7fe5401000a0 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5460c1640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5401000a0 0x7fe5401971f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe53400d950 tx=0x7fe53400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5337fe640 1 -- 172.21.15.83:0/2949596811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534009c30 con 0x7fe5401000a0 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe548325640 1 -- 172.21.15.83:0/2949596811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54019f2f0 con 0x7fe5401000a0 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe548325640 1 -- 172.21.15.83:0/2949596811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54019f850 con 0x7fe5401000a0 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5337fe640 1 -- 172.21.15.83:0/2949596811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe534009d90 con 0x7fe5401000a0 2024-03-11T19:54:59.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.226+0000 7fe5337fe640 1 -- 172.21.15.83:0/2949596811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53400f850 con 0x7fe5401000a0 2024-03-11T19:54:59.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.227+0000 7fe548325640 1 -- 172.21.15.83:0/2949596811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe524005290 con 0x7fe5401000a0 2024-03-11T19:54:59.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.229+0000 7fe5337fe640 1 -- 172.21.15.83:0/2949596811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe534010b60 con 0x7fe5401000a0 2024-03-11T19:54:59.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.229+0000 7fe5337fe640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe52c0a1a30 0x7fe52c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.229+0000 7fe5458c0640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe52c0a1a30 0x7fe52c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:59.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.229+0000 7fe5337fe640 1 -- 172.21.15.83:0/2949596811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe5340cd380 con 0x7fe5401000a0 2024-03-11T19:54:59.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.230+0000 7fe5458c0640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe52c0a1a30 0x7fe52c0a3ef0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe540198700 tx=0x7fe53c005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:59.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.233+0000 7fe5337fe640 1 -- 172.21.15.83:0/2949596811 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe534096640 con 0x7fe5401000a0 2024-03-11T19:54:59.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.406+0000 7fe548325640 1 -- 172.21.15.83:0/2949596811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe524005ac0 con 0x7fe5401000a0 2024-03-11T19:54:59.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.407+0000 7fe5337fe640 1 -- 172.21.15.83:0/2949596811 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe534099d00 con 0x7fe5401000a0 2024-03-11T19:54:59.410 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:54:59.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 -- 172.21.15.83:0/2949596811 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe52c0a1a30 msgr2=0x7fe52c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe52c0a1a30 0x7fe52c0a3ef0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe540198700 tx=0x7fe53c005c00 comp rx=0 tx=0).stop 2024-03-11T19:54:59.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 -- 172.21.15.83:0/2949596811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5401000a0 msgr2=0x7fe5401971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5401000a0 0x7fe5401971f0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe53400d950 tx=0x7fe53400de10 comp rx=0 tx=0).stop 2024-03-11T19:54:59.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 -- 172.21.15.83:0/2949596811 shutdown_connections 2024-03-11T19:54:59.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe52c0a1a30 0x7fe52c0a3ef0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe540101c50 0x7fe54019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe540101290 0x7fe540197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 --2- 172.21.15.83:0/2949596811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5401000a0 0x7fe5401971f0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 -- 172.21.15.83:0/2949596811 >> 172.21.15.83:0/2949596811 conn(0x7fe5400fb630 msgr2=0x7fe5401044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:59.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 -- 172.21.15.83:0/2949596811 shutdown_connections 2024-03-11T19:54:59.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.409+0000 7fe548325640 1 -- 172.21.15.83:0/2949596811 wait complete. 2024-03-11T19:54:59.424 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:54:59.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.525+0000 7f5a95a23640 1 -- 172.21.15.83:0/2017191963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a90101c40 msgr2=0x7f5a901084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.525+0000 7f5a95a23640 1 --2- 172.21.15.83:0/2017191963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a90101c40 0x7f5a901084d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009ca0 tx=0x7f5a8802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.525+0000 7f5a95a23640 1 -- 172.21.15.83:0/2017191963 shutdown_connections 2024-03-11T19:54:59.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.525+0000 7f5a95a23640 1 --2- 172.21.15.83:0/2017191963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a90101c40 0x7f5a901084d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.525+0000 7f5a95a23640 1 --2- 172.21.15.83:0/2017191963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a90101280 0x7f5a90101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.525+0000 7f5a95a23640 1 --2- 172.21.15.83:0/2017191963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a90100090 0x7f5a901004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.525+0000 7f5a95a23640 1 -- 172.21.15.83:0/2017191963 >> 172.21.15.83:0/2017191963 conn(0x7f5a900fb620 msgr2=0x7f5a900fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:59.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.525+0000 7f5a95a23640 1 -- 172.21.15.83:0/2017191963 shutdown_connections 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.525+0000 7f5a95a23640 1 -- 172.21.15.83:0/2017191963 wait complete. 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a95a23640 1 Processor -- start 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a95a23640 1 -- start start 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a95a23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a90100090 0x7f5a9006a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a95a23640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a90101280 0x7f5a9019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a95a23640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a90101c40 0x7f5a9019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a95a23640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9019eca0 con 0x7f5a90100090 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a95a23640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9019ee10 con 0x7f5a90101280 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a95a23640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9019efd0 con 0x7f5a90101c40 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a8effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a90100090 0x7f5a9006a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a8effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a90100090 0x7f5a9006a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37716/0 (socket says 172.21.15.83:37716) 2024-03-11T19:54:59.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a8effd640 1 -- 172.21.15.83:0/3908823971 learned_addr learned my addr 172.21.15.83:0/3908823971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a87fff640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a90101280 0x7f5a9019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a8effd640 1 -- 172.21.15.83:0/3908823971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a90101c40 msgr2=0x7f5a9019e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a8effd640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a90101c40 0x7f5a9019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a8effd640 1 -- 172.21.15.83:0/3908823971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a90101280 msgr2=0x7f5a9019bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a8effd640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a90101280 0x7f5a9019bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a8effd640 1 -- 172.21.15.83:0/3908823971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a88009900 con 0x7f5a90100090 2024-03-11T19:54:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a87fff640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a90101280 0x7f5a9019bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T19:54:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.526+0000 7f5a8effd640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a90100090 0x7f5a9006a2c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000c970 tx=0x7f5a8000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.527+0000 7f5a94a21640 1 -- 172.21.15.83:0/3908823971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80007c10 con 0x7f5a90100090 2024-03-11T19:54:59.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.527+0000 7f5a94a21640 1 -- 172.21.15.83:0/3908823971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a80007d70 con 0x7f5a90100090 2024-03-11T19:54:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.527+0000 7f5a95a23640 1 -- 172.21.15.83:0/3908823971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9019f2b0 con 0x7f5a90100090 2024-03-11T19:54:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.527+0000 7f5a94a21640 1 -- 172.21.15.83:0/3908823971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000f8a0 con 0x7f5a90100090 2024-03-11T19:54:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.527+0000 7f5a95a23640 1 -- 172.21.15.83:0/3908823971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9019f810 con 0x7f5a90100090 2024-03-11T19:54:59.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.528+0000 7f5a95a23640 1 -- 172.21.15.83:0/3908823971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a901072e0 con 0x7f5a90100090 2024-03-11T19:54:59.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.529+0000 7f5a94a21640 1 -- 172.21.15.83:0/3908823971 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f5a80018af0 con 0x7f5a90100090 2024-03-11T19:54:59.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.529+0000 7f5a94a21640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a7c0a1ab0 0x7f5a7c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.529+0000 7f5a87fff640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a7c0a1ab0 0x7f5a7c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:59.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.529+0000 7f5a94a21640 1 -- 172.21.15.83:0/3908823971 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f5a80013070 con 0x7f5a90100090 2024-03-11T19:54:59.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.530+0000 7f5a87fff640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a7c0a1ab0 0x7f5a7c0a3f70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a9006a030 tx=0x7f5a78005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:59.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.533+0000 7f5a94a21640 1 -- 172.21.15.83:0/3908823971 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a80096a60 con 0x7f5a90100090 2024-03-11T19:54:59.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.706+0000 7f5a95a23640 1 -- 172.21.15.83:0/3908823971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a9002ce80 con 0x7f5a90100090 2024-03-11T19:54:59.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.706+0000 7f5a94a21640 1 -- 172.21.15.83:0/3908823971 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a8009a120 con 0x7f5a90100090 2024-03-11T19:54:59.710 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-03-11T19:54:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 -- 172.21.15.83:0/3908823971 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a7c0a1ab0 msgr2=0x7f5a7c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a7c0a1ab0 0x7f5a7c0a3f70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a9006a030 tx=0x7f5a78005d20 comp rx=0 tx=0).stop 2024-03-11T19:54:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 -- 172.21.15.83:0/3908823971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a90100090 msgr2=0x7f5a9006a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a90100090 0x7f5a9006a2c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000c970 tx=0x7f5a8000ce30 comp rx=0 tx=0).stop 2024-03-11T19:54:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 -- 172.21.15.83:0/3908823971 shutdown_connections 2024-03-11T19:54:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f5a7c0a1ab0 0x7f5a7c0a3f70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a90101c40 0x7f5a9019e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a90101280 0x7f5a9019bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 --2- 172.21.15.83:0/3908823971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a90100090 0x7f5a9006a2c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 -- 172.21.15.83:0/3908823971 >> 172.21.15.83:0/3908823971 conn(0x7f5a900fb620 msgr2=0x7f5a901044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 -- 172.21.15.83:0/3908823971 shutdown_connections 2024-03-11T19:54:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.708+0000 7f5a95a23640 1 -- 172.21.15.83:0/3908823971 wait complete. 2024-03-11T19:54:59.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:54:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.824+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3903371168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb58101190 msgr2=0x7fdb581015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.824+0000 7fdb5d85c640 1 --2- 172.21.15.83:0/3903371168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb58101190 0x7fdb581015b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009900 tx=0x7fdb4802bc30 comp rx=0 tx=0).stop 2024-03-11T19:54:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.824+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3903371168 shutdown_connections 2024-03-11T19:54:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.824+0000 7fdb5d85c640 1 --2- 172.21.15.83:0/3903371168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb58102d40 0x7fdb581095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.824+0000 7fdb5d85c640 1 --2- 172.21.15.83:0/3903371168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb58102380 0x7fdb58102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.824+0000 7fdb5d85c640 1 --2- 172.21.15.83:0/3903371168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb58101190 0x7fdb581015b0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.824+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3903371168 >> 172.21.15.83:0/3903371168 conn(0x7fdb580fc720 msgr2=0x7fdb580feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:54:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.824+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3903371168 shutdown_connections 2024-03-11T19:54:59.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.824+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3903371168 wait complete. 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb5d85c640 1 Processor -- start 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb5d85c640 1 -- start start 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb5d85c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb58101190 0x7fdb58197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb5d85c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb58102380 0x7fdb58197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb5d85c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb58102d40 0x7fdb5819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb5d85c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5819ed20 con 0x7fdb58102380 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb5d85c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5819ee90 con 0x7fdb58101190 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb5d85c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5819f050 con 0x7fdb58102d40 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb567fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb58102380 0x7fdb58197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb567fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb58102380 0x7fdb58197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37746/0 (socket says 172.21.15.83:37746) 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb567fc640 1 -- 172.21.15.83:0/3767128521 learned_addr learned my addr 172.21.15.83:0/3767128521 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb56ffd640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb58101190 0x7fdb58197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb577fe640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb58102d40 0x7fdb5819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:59.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb567fc640 1 -- 172.21.15.83:0/3767128521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb58102d40 msgr2=0x7fdb5819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb567fc640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb58102d40 0x7fdb5819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb567fc640 1 -- 172.21.15.83:0/3767128521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb58101190 msgr2=0x7fdb58197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:54:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb567fc640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb58101190 0x7fdb58197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:54:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb567fc640 1 -- 172.21.15.83:0/3767128521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb48009560 con 0x7fdb58102380 2024-03-11T19:54:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.825+0000 7fdb567fc640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb58102380 0x7fdb58197770 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00d950 tx=0x7fdb4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.826+0000 7fdb5c85a640 1 -- 172.21.15.83:0/3767128521 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c009c30 con 0x7fdb58102380 2024-03-11T19:54:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.826+0000 7fdb5c85a640 1 -- 172.21.15.83:0/3767128521 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb4c009d90 con 0x7fdb58102380 2024-03-11T19:54:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.826+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3767128521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5819f330 con 0x7fdb58102380 2024-03-11T19:54:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.826+0000 7fdb5c85a640 1 -- 172.21.15.83:0/3767128521 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00f7e0 con 0x7fdb58102380 2024-03-11T19:54:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.826+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3767128521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5819f840 con 0x7fdb58102380 2024-03-11T19:54:59.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.827+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3767128521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb58108530 con 0x7fdb58102380 2024-03-11T19:54:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.828+0000 7fdb5c85a640 1 -- 172.21.15.83:0/3767128521 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdb4c010ad0 con 0x7fdb58102380 2024-03-11T19:54:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.828+0000 7fdb5c85a640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb400a1a30 0x7fdb400a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:54:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.828+0000 7fdb56ffd640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb400a1a30 0x7fdb400a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:54:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.828+0000 7fdb5c85a640 1 -- 172.21.15.83:0/3767128521 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdb4c0ccfe0 con 0x7fdb58102380 2024-03-11T19:54:59.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.829+0000 7fdb56ffd640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb400a1a30 0x7fdb400a3ef0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdb48005ab0 tx=0x7fdb48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:54:59.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:54:59.832+0000 7fdb5c85a640 1 -- 172.21.15.83:0/3767128521 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb4c0962a0 con 0x7fdb58102380 2024-03-11T19:55:00.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.007+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3767128521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb5810cdf0 con 0x7fdb58102380 2024-03-11T19:55:00.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.007+0000 7fdb5c85a640 1 -- 172.21.15.83:0/3767128521 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb4c099960 con 0x7fdb58102380 2024-03-11T19:55:00.011 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-03-11T19:55:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3767128521 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb400a1a30 msgr2=0x7fdb400a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb400a1a30 0x7fdb400a3ef0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdb48005ab0 tx=0x7fdb48005fd0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3767128521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb58102380 msgr2=0x7fdb58197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb58102380 0x7fdb58197770 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00d950 tx=0x7fdb4c00de10 comp rx=0 tx=0).stop 2024-03-11T19:55:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3767128521 shutdown_connections 2024-03-11T19:55:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdb400a1a30 0x7fdb400a3ef0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb58102d40 0x7fdb5819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb58102380 0x7fdb58197770 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 --2- 172.21.15.83:0/3767128521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb58101190 0x7fdb58197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3767128521 >> 172.21.15.83:0/3767128521 conn(0x7fdb580fc720 msgr2=0x7fdb58105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:00.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3767128521 shutdown_connections 2024-03-11T19:55:00.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.009+0000 7fdb5d85c640 1 -- 172.21.15.83:0/3767128521 wait complete. 2024-03-11T19:55:00.023 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:59 smithi083 ceph-mon[31864]: pgmap v822: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-03-11T19:55:00.023 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:59 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2483806737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:55:00.023 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:59 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2949596811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:55:00.023 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:54:59 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3908823971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:55:00.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-11T19:55:00.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.122+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2047945209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1df41003f0 msgr2=0x7f1df4100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.122+0000 7f1dfc56c640 1 --2- 172.21.15.83:0/2047945209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1df41003f0 0x7f1df4100850 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009900 tx=0x7f1de402bcd0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2047945209 shutdown_connections 2024-03-11T19:55:00.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 --2- 172.21.15.83:0/2047945209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1df4100d90 0x7f1df4101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 --2- 172.21.15.83:0/2047945209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1df41003f0 0x7f1df4100850 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 --2- 172.21.15.83:0/2047945209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df40ffa90 0x7f1df40ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2047945209 >> 172.21.15.83:0/2047945209 conn(0x7f1df40fb610 msgr2=0x7f1df40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2047945209 shutdown_connections 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2047945209 wait complete. 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 Processor -- start 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 -- start start 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1df40ffa90 0x7f1df41994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1df41003f0 0x7f1df4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df4100d90 0x7f1df41a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df41a0fb0 con 0x7f1df40ffa90 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df41a1120 con 0x7f1df41003f0 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfc56c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df41a12e0 con 0x7f1df4100d90 2024-03-11T19:55:00.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfa308640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1df40ffa90 0x7f1df41994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfa308640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1df40ffa90 0x7f1df41994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37778/0 (socket says 172.21.15.83:37778) 2024-03-11T19:55:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.123+0000 7f1dfa308640 1 -- 172.21.15.83:0/2518055830 learned_addr learned my addr 172.21.15.83:0/2518055830 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1df9b07640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1df41003f0 0x7f1df4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1dfa308640 1 -- 172.21.15.83:0/2518055830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df4100d90 msgr2=0x7f1df41a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1dfa308640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df4100d90 0x7f1df41a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1dfa308640 1 -- 172.21.15.83:0/2518055830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1df41003f0 msgr2=0x7f1df4199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1dfa308640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1df41003f0 0x7f1df4199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1dfa308640 1 -- 172.21.15.83:0/2518055830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de4009560 con 0x7f1df40ffa90 2024-03-11T19:55:00.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1df9b07640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1df41003f0 0x7f1df4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:55:00.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1dfa308640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1df40ffa90 0x7f1df41994c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b4a0 tx=0x7f1df000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:00.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1deb7fe640 1 -- 172.21.15.83:0/2518055830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0010610 con 0x7f1df40ffa90 2024-03-11T19:55:00.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2518055830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df41a15c0 con 0x7f1df40ffa90 2024-03-11T19:55:00.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1deb7fe640 1 -- 172.21.15.83:0/2518055830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df0010c50 con 0x7f1df40ffa90 2024-03-11T19:55:00.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1deb7fe640 1 -- 172.21.15.83:0/2518055830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0017e20 con 0x7f1df40ffa90 2024-03-11T19:55:00.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.124+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2518055830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df41a1a80 con 0x7f1df40ffa90 2024-03-11T19:55:00.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.125+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2518055830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df410a960 con 0x7f1df40ffa90 2024-03-11T19:55:00.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.126+0000 7f1deb7fe640 1 -- 172.21.15.83:0/2518055830 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f1df0017480 con 0x7f1df40ffa90 2024-03-11T19:55:00.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.126+0000 7f1deb7fe640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1de00a1a30 0x7f1de00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:00.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.127+0000 7f1deb7fe640 1 -- 172.21.15.83:0/2518055830 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f1df00cceb0 con 0x7f1df40ffa90 2024-03-11T19:55:00.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.127+0000 7f1df9b07640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1de00a1a30 0x7f1de00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:00.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.127+0000 7f1df9b07640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1de00a1a30 0x7f1de00a3ef0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1df419a9d0 tx=0x7f1de402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:00.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.130+0000 7f1deb7fe640 1 -- 172.21.15.83:0/2518055830 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1df00fad20 con 0x7f1df40ffa90 2024-03-11T19:55:00.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:59 smithi204 ceph-mon[36739]: pgmap v822: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-03-11T19:55:00.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:59 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2483806737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:55:00.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:59 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2949596811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:55:00.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:54:59 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3908823971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:55:00.191 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:59 smithi137 ceph-mon[36220]: pgmap v822: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-03-11T19:55:00.191 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:59 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2483806737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:55:00.191 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:59 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2949596811' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:55:00.191 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:54:59 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3908823971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:55:00.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.348+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2518055830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f1df410edf0 con 0x7f1df40ffa90 2024-03-11T19:55:00.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.356+0000 7f1deb7fe640 1 -- 172.21.15.83:0/2518055830 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+734467 (secure 0 0 0) 0x7f1df0181520 con 0x7f1df40ffa90 2024-03-11T19:55:00.369 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:55:00.370 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:55:00.370 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 18, 2024-03-11T19:55:00.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.370 INFO:teuthology.orchestra.run.smithi083.stdout: "active_gid": 14152, 2024-03-11T19:55:00.370 INFO:teuthology.orchestra.run.smithi083.stdout: "active_name": "x", 2024-03-11T19:55:00.370 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addrs": { 2024-03-11T19:55:00.370 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-03-11T19:55:00.370 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.370 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-03-11T19:55:00.370 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6800", 2024-03-11T19:55:00.370 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3477661212 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3477661212 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addr": "172.21.15.83:6801/3477661212", 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: "active_change": "2024-03-11T19:30:08.427972+0000", 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: "active_mgr_features": 4540701547738038271, 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: "available": true, 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: "standbys": [ 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.371 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 24106, 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "z", 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540701547738038271, 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.372 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.373 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.374 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.375 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.376 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.376 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.376 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.376 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.376 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.376 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.378 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.378 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.378 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.378 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.378 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.378 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.378 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.378 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.380 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-03-11T19:55:00.381 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-03-11T19:55:00.381 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-03-11T19:55:00.382 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-03-11T19:55:00.383 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-03-11T19:55:00.384 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-03-11T19:55:00.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-03-11T19:55:00.386 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.387 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.388 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.390 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-03-11T19:55:00.391 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.392 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.392 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.392 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-03-11T19:55:00.392 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:55:00.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.392 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.392 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.393 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.394 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.395 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.396 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.399 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.400 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.401 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.402 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.402 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.402 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.402 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.402 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.402 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.402 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.402 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.403 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.404 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.404 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.404 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.404 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.404 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.404 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-03-11T19:55:00.404 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-03-11T19:55:00.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-03-11T19:55:00.405 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.406 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.407 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.408 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.410 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-03-11T19:55:00.411 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-03-11T19:55:00.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:00.413 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.413 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-03-11T19:55:00.413 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.413 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.413 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.413 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-11T19:55:00.413 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-11T19:55:00.414 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.415 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.416 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.417 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.418 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.419 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.420 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.421 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.423 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.424 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.425 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.426 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-03-11T19:55:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-03-11T19:55:00.429 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-03-11T19:55:00.430 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-03-11T19:55:00.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.432 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.435 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.436 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.437 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-03-11T19:55:00.438 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.439 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-03-11T19:55:00.440 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-03-11T19:55:00.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.440 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.440 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.440 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:00.440 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.440 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.440 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.440 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-11T19:55:00.440 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.440 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.441 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.442 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-03-11T19:55:00.443 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.445 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.446 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.447 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.448 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.449 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-03-11T19:55:00.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.451 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.451 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.451 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.451 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.451 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-11T19:55:00.451 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.451 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.452 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.453 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.453 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.453 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.453 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.453 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.453 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.453 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.453 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.454 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.455 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.456 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.457 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.459 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-03-11T19:55:00.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.461 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.462 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.463 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.465 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-03-11T19:55:00.466 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-03-11T19:55:00.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.466 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.466 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.466 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.466 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.466 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.466 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.466 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-11T19:55:00.466 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.466 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.467 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-11T19:55:00.468 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-03-11T19:55:00.469 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-11T19:55:00.470 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-11T19:55:00.471 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-11T19:55:00.472 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-03-11T19:55:00.473 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-11T19:55:00.474 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-11T19:55:00.475 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.476 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.476 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.476 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.476 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.476 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-03-11T19:55:00.476 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.476 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.476 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.476 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.477 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.478 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.479 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.480 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.481 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.482 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.483 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.484 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.485 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.488 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.489 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.490 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.491 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.492 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.493 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.494 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.495 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.496 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-11T19:55:00.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.498 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.501 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.502 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.503 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.505 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.506 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.507 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.508 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.509 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.510 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.511 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.512 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.513 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.514 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.515 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.516 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.517 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.518 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.519 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.520 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.521 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.522 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.523 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.524 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:00.525 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.526 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.527 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.529 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.531 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.533 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.534 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.535 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.536 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.537 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-03-11T19:55:00.538 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-03-11T19:55:00.539 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-03-11T19:55:00.540 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-03-11T19:55:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-03-11T19:55:00.542 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-03-11T19:55:00.543 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-03-11T19:55:00.544 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.547 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.548 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.549 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.550 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.551 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.552 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.553 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.555 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.556 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.557 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.558 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.559 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.560 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.561 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.562 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.563 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.564 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.565 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.566 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-03-11T19:55:00.567 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-03-11T19:55:00.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:00.569 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.571 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.572 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.573 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-03-11T19:55:00.574 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.575 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.576 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.577 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.578 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.579 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.580 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-03-11T19:55:00.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-03-11T19:55:00.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-03-11T19:55:00.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-03-11T19:55:00.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-03-11T19:55:00.585 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.586 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.587 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.588 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.589 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.590 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.591 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.592 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.593 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.594 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.595 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.596 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.597 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.598 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.599 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.602 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.603 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.604 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.605 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-11T19:55:00.606 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.607 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.608 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.609 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.610 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.611 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.612 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.613 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.614 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.617 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-03-11T19:55:00.617 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-03-11T19:55:00.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.617 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.617 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.617 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.617 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.617 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.618 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-03-11T19:55:00.618 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.618 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.618 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.618 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-03-11T19:55:00.618 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-03-11T19:55:00.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.618 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.618 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.619 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.620 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.621 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.623 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.624 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.626 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.627 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.628 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.630 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.631 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.632 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.633 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.635 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.637 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.638 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.639 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.640 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.641 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.642 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.643 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.644 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.645 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.646 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.647 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.648 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.649 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.650 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.651 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.652 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.653 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.655 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.656 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.657 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.658 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.659 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.660 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:00.661 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-03-11T19:55:00.662 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.663 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.664 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.665 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.666 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.667 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.668 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.669 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.670 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-03-11T19:55:00.671 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:00.672 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.673 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.674 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.676 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.677 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.678 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.679 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.680 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.681 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-11T19:55:00.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-03-11T19:55:00.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-03-11T19:55:00.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-03-11T19:55:00.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-03-11T19:55:00.686 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-03-11T19:55:00.687 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.688 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.689 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.690 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.691 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.692 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.693 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.694 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.697 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-03-11T19:55:00.698 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.699 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.700 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.701 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.702 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.703 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.703 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.703 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.703 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.703 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.703 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.703 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.703 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.704 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.705 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.706 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.708 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.709 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.710 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.711 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.712 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.713 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.714 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.717 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.718 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.719 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.720 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.721 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.722 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.723 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.724 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.725 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.726 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.727 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.729 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.730 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.731 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.732 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.733 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.734 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.735 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.736 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.738 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-03-11T19:55:00.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.741 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.742 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.743 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.744 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.745 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.746 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.749 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.750 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.751 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.752 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.754 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.755 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-03-11T19:55:00.756 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.757 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.758 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.759 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.760 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.761 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.762 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.763 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.764 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.765 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.766 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-03-11T19:55:00.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-03-11T19:55:00.768 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.769 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-03-11T19:55:00.770 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.771 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.773 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.774 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.775 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.776 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.777 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.778 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.779 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.780 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.781 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.782 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.783 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.784 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.785 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.786 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.787 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.789 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.790 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.791 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.792 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.793 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.794 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.795 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.800 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-03-11T19:55:00.801 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.802 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.803 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.804 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.805 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.806 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.807 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.808 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.809 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.810 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.811 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-03-11T19:55:00.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 24122, 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "y", 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540701547738038271, 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.813 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.814 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.815 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.816 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.817 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.818 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-03-11T19:55:00.819 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.821 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.821 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.821 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.821 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.821 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.821 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.821 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-03-11T19:55:00.821 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.821 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.822 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.823 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-03-11T19:55:00.824 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.825 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.826 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.827 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.828 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.829 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:00.831 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:00.832 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-03-11T19:55:00.833 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-03-11T19:55:00.834 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-03-11T19:55:00.835 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.836 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.837 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.838 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.839 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.840 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.841 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.842 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.843 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-03-11T19:55:00.844 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.845 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.846 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.847 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:00.848 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.849 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.850 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.851 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.852 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.853 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.854 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.855 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.856 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.857 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.858 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.859 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.860 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.861 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.862 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.863 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.864 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.865 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.866 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.867 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.868 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-03-11T19:55:00.869 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-03-11T19:55:00.870 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-03-11T19:55:00.871 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-03-11T19:55:00.872 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-03-11T19:55:00.873 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-03-11T19:55:00.874 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-03-11T19:55:00.875 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.876 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.877 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-03-11T19:55:00.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-03-11T19:55:00.879 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.880 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-03-11T19:55:00.881 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-03-11T19:55:00.882 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-03-11T19:55:00.883 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-03-11T19:55:00.884 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.885 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.886 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.887 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.888 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.889 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.890 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.891 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.892 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.893 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.894 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.895 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.896 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.897 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.898 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.899 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.900 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.901 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.902 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.903 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.905 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.908 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.909 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.910 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.911 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.912 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.913 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.914 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.915 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.916 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-11T19:55:00.917 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.918 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.919 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.920 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.921 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.922 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.923 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.924 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.925 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.926 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.928 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.929 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.930 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.931 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.932 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.933 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.934 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.935 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.936 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.937 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.938 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.939 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.940 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.941 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.943 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.944 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.945 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.946 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.947 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.948 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.949 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.950 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.951 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.952 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.955 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.957 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.960 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.961 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.962 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-03-11T19:55:00.963 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-03-11T19:55:00.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-03-11T19:55:00.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.966 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.967 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.968 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.969 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.970 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.976 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-03-11T19:55:00.978 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-03-11T19:55:00.979 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.981 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.982 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.983 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.985 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.986 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.987 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-11T19:55:00.988 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-03-11T19:55:00.989 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-11T19:55:00.990 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-11T19:55:00.991 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.992 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.993 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.994 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:00.997 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:00.998 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-03-11T19:55:00.999 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-03-11T19:55:01.000 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:01.001 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-03-11T19:55:01.002 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.003 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.004 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:01.005 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.006 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.007 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.008 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.009 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.010 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-03-11T19:55:01.011 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-03-11T19:55:01.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-03-11T19:55:01.013 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-03-11T19:55:01.014 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-03-11T19:55:01.015 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-03-11T19:55:01.016 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.017 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.018 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.019 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.020 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.021 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.022 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.023 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.024 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.025 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.026 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.027 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.028 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.029 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-03-11T19:55:01.030 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.031 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.032 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.033 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.034 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.035 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.036 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.037 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.038 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.039 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.040 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.041 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-03-11T19:55:01.042 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-03-11T19:55:01.043 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-03-11T19:55:01.044 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-03-11T19:55:01.045 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-03-11T19:55:01.046 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.047 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.048 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.049 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.050 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.050 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.050 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.050 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.050 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.050 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.050 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.050 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.051 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.052 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.053 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.054 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.055 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.056 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.057 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.058 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.059 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.060 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.061 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.062 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.063 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-03-11T19:55:01.064 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-11T19:55:01.065 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.066 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.068 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.069 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.070 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.071 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-03-11T19:55:01.072 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.073 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.074 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.075 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.077 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.078 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.079 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.081 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.082 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.083 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.084 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.085 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.086 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.087 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.089 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-11T19:55:01.090 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.091 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.093 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.094 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.096 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.097 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.098 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.099 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.100 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.101 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.102 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.103 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.104 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.106 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.107 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-03-11T19:55:01.108 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-11T19:55:01.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-03-11T19:55:01.110 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-03-11T19:55:01.111 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-03-11T19:55:01.112 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-03-11T19:55:01.113 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.114 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.115 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.116 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.117 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.118 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.119 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.120 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.121 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.121 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.121 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.121 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.121 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.121 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.121 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.121 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.121 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.121 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.122 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.123 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.124 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.125 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-03-11T19:55:01.126 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-03-11T19:55:01.127 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.128 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.129 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.130 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.131 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.132 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.134 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.135 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.136 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.137 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.138 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.139 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.140 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.141 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.142 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.143 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.144 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.145 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-03-11T19:55:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.147 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.148 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.149 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.151 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.152 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.153 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.154 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.155 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.156 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:01.157 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-03-11T19:55:01.158 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-03-11T19:55:01.159 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.160 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.161 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.162 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-03-11T19:55:01.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-03-11T19:55:01.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.165 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.166 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.167 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.168 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.169 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.171 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.172 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.173 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.174 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.174 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.174 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.174 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.174 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.174 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.174 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.174 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.174 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.174 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.174 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.175 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.175 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.175 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.175 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.175 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.175 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.175 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.175 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.175 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.177 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.179 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.180 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.181 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.182 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.183 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.184 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-03-11T19:55:01.185 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.186 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.188 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.189 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.190 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.192 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.195 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.196 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.197 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.198 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.199 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-11T19:55:01.200 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.201 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-03-11T19:55:01.202 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-03-11T19:55:01.203 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.204 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.205 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.206 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.209 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.211 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.212 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.213 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.214 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.215 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.216 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.217 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.218 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.220 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.221 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.222 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.223 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.224 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.225 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.226 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.227 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.229 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.230 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.233 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.234 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.235 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.236 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.237 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.238 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.239 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.240 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: "modules": [ 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-03-11T19:55:01.241 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard", 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "iostat", 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "nfs", 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "restful" 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.242 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.243 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.244 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.245 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.246 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.247 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.248 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.249 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-03-11T19:55:01.250 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-03-11T19:55:01.251 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-03-11T19:55:01.252 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-03-11T19:55:01.253 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-03-11T19:55:01.254 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-03-11T19:55:01.255 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.257 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.258 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-03-11T19:55:01.259 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-03-11T19:55:01.260 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.261 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.262 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-03-11T19:55:01.263 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-11T19:55:01.264 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.265 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.266 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.268 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.269 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.270 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.271 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-03-11T19:55:01.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:01.273 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-03-11T19:55:01.274 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-03-11T19:55:01.275 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.276 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.277 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.278 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.279 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.280 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.281 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.282 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.283 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.284 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.286 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.287 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.288 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.289 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.290 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.291 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.292 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.293 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.294 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.295 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.296 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.297 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.298 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.299 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.300 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.301 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.302 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:01.303 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-03-11T19:55:01.304 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.305 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.306 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.307 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:01.308 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.309 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.310 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.311 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.312 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-03-11T19:55:01.313 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.314 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.315 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.316 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.317 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.318 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-03-11T19:55:01.319 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.320 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.321 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.322 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.323 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.324 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-11T19:55:01.325 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.326 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.327 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.328 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.329 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.330 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-03-11T19:55:01.331 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.332 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.333 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.334 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.335 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.336 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.337 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-03-11T19:55:01.338 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.339 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.340 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.341 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.342 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.344 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.346 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.348 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.349 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.350 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.351 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.352 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.353 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.354 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.355 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.356 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.357 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.358 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.359 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.360 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.361 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.362 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.363 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.364 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.364 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.364 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.364 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-11T19:55:01.364 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-11T19:55:01.364 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.364 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.364 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.364 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.364 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-11T19:55:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-11T19:55:01.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-11T19:55:01.366 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.370 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.371 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.372 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.373 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.374 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.375 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.376 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.377 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.378 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.379 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.380 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.381 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.385 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.386 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:01.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.388 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.389 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.390 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.391 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.392 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.393 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.394 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.395 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.396 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.397 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.398 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.399 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.400 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.401 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.402 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.403 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.404 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-03-11T19:55:01.405 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.406 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.407 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.408 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-03-11T19:55:01.410 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.411 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.412 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.413 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.414 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.415 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-11T19:55:01.416 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-11T19:55:01.417 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-03-11T19:55:01.418 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-11T19:55:01.419 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-11T19:55:01.420 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-03-11T19:55:01.421 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-11T19:55:01.422 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.423 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.424 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.425 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.429 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.430 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.431 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-03-11T19:55:01.433 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-03-11T19:55:01.433 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-03-11T19:55:01.433 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.433 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-03-11T19:55:01.433 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-03-11T19:55:01.433 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.433 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.433 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.433 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-03-11T19:55:01.433 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-03-11T19:55:01.434 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.435 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-03-11T19:55:01.436 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.437 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.439 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.440 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.440 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.440 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.440 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.440 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.440 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.440 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.440 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.440 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.440 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.441 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-03-11T19:55:01.442 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-03-11T19:55:01.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-03-11T19:55:01.444 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-03-11T19:55:01.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-03-11T19:55:01.446 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-03-11T19:55:01.447 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.448 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.449 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.451 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.452 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.453 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.454 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.455 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.456 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.457 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.458 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.459 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-03-11T19:55:01.460 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:01.461 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.462 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.463 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.464 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.465 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.466 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.467 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-03-11T19:55:01.468 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.469 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.470 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.472 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.473 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-03-11T19:55:01.474 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.475 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.476 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-11T19:55:01.478 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.479 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.480 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.481 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.482 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.483 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-03-11T19:55:01.484 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.485 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.486 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.487 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.488 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.489 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.490 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.492 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.493 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.494 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.496 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.497 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.498 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.499 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.500 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-03-11T19:55:01.501 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.502 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.503 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.504 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.507 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.507 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.507 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.507 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.507 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.507 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.507 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.507 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.507 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.508 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.509 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.510 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.511 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.512 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.513 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.514 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.515 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.516 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.517 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.518 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.519 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.520 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-03-11T19:55:01.521 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.522 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.523 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.524 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.525 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.526 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.527 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.528 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.529 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.531 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.535 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.536 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.537 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.538 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.539 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.540 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-03-11T19:55:01.541 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.542 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.543 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.544 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-03-11T19:55:01.545 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.546 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.547 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.548 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.549 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.550 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.551 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.552 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.553 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.554 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.555 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.557 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.558 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.559 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.560 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.561 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-03-11T19:55:01.562 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-03-11T19:55:01.563 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.565 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.566 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.567 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.568 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.569 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.570 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.571 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.572 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.573 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.575 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.576 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.577 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.578 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.579 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.580 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.581 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.582 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.583 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.585 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.586 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.587 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.588 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.589 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.590 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.591 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.593 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.594 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.595 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.596 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.597 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.598 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.599 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.601 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.603 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.604 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.605 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.606 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.607 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.608 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.609 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.610 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.611 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.613 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-03-11T19:55:01.614 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.615 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.617 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.618 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.619 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.620 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.622 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.624 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.629 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.631 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.632 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.633 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.634 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.635 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.636 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.637 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.637 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.637 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.637 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.637 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.637 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.637 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.637 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.637 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.637 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.637 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.638 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.639 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.640 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.641 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.642 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.643 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.647 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.649 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.650 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.651 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.652 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-03-11T19:55:01.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-03-11T19:55:01.654 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-03-11T19:55:01.655 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.656 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.658 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.659 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.660 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-03-11T19:55:01.661 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-03-11T19:55:01.662 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-03-11T19:55:01.663 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-03-11T19:55:01.664 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-03-11T19:55:01.665 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-03-11T19:55:01.666 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-03-11T19:55:01.667 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: "services": { 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard": "https://172.21.15.83:8443/" 2024-03-11T19:55:01.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "always_on_modules": { 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "octopus": [ 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-03-11T19:55:01.669 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "pacific": [ 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "quincy": [ 2024-03-11T19:55:01.670 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "reef": [ 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-03-11T19:55:01.671 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "squid": [ 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-03-11T19:55:01.672 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: "last_failure_osd_epoch": 3, 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: "active_clients": [ 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-03-11T19:55:01.673 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1955145343 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "libcephsqlite", 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 42719287 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-03-11T19:55:01.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 487994704 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-03-11T19:55:01.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-03-11T19:55:01.676 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-03-11T19:55:01.676 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-03-11T19:55:01.676 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3164935375 2024-03-11T19:55:01.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.676 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-03-11T19:55:01.676 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:01.676 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-03-11T19:55:01.676 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:55:01.677 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:00 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3767128521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:55:01.677 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:00 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2518055830' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-11T19:55:01.678 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:00 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3767128521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:55:01.678 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:00 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2518055830' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-11T19:55:01.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2518055830 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1de00a1a30 msgr2=0x7f1de00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:01.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1de00a1a30 0x7f1de00a3ef0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1df419a9d0 tx=0x7f1de402c880 comp rx=0 tx=0).stop 2024-03-11T19:55:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2518055830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1df40ffa90 msgr2=0x7f1df41994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1df40ffa90 0x7f1df41994c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b4a0 tx=0x7f1df000b960 comp rx=0 tx=0).stop 2024-03-11T19:55:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2518055830 shutdown_connections 2024-03-11T19:55:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f1de00a1a30 0x7f1de00a3ef0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1df4100d90 0x7f1df41a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1df41003f0 0x7f1df4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 --2- 172.21.15.83:0/2518055830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1df40ffa90 0x7f1df41994c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2518055830 >> 172.21.15.83:0/2518055830 conn(0x7f1df40fb610 msgr2=0x7f1df40fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2518055830 shutdown_connections 2024-03-11T19:55:01.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:00.361+0000 7f1dfc56c640 1 -- 172.21.15.83:0/2518055830 wait complete. 2024-03-11T19:55:01.685 DEBUG:tasks.check_counter:Getting stats from a 2024-03-11T19:55:01.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-11T19:55:01.685 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:00 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3767128521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:55:01.686 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:00 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2518055830' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-11T19:55:01.935 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:55:02.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:01 smithi083 ceph-mon[31864]: pgmap v823: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-03-11T19:55:02.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:01 smithi204 ceph-mon[36739]: pgmap v823: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-03-11T19:55:02.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:01 smithi137 ceph-mon[36220]: pgmap v823: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4, 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 4, 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 2992, 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 448, 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001544123, 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000044894, 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000618614, 2024-03-11T19:55:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000697079, 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000232359 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 294, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 291, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 495344, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 174752, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.043204686, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.017210548, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.015676067, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.009180421, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 290, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.015879748, 2024-03-11T19:55:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000054757 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 495344, 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 174752 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2, 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 222, 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 320, 2024-03-11T19:55:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 193952, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.016360649, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.012511830, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001648961, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 220, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.013466944, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000061213 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 193952 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148744, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-03-11T19:55:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-03-11T19:55:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 296, 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 453790, 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 296, 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-03-11T19:55:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 296, 2024-03-11T19:55:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 453790, 2024-03-11T19:55:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-03-11T19:55:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:03.723 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:55:04.067 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:03 smithi083 ceph-mon[31864]: pgmap v824: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 52 KiB/s wr, 11 op/s 2024-03-11T19:55:04.386 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-11T19:55:04.387 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-11T19:55:04.387 DEBUG:tasks.check_counter:Getting stats from d 2024-03-11T19:55:04.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-11T19:55:04.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:03 smithi137 ceph-mon[36220]: pgmap v824: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 52 KiB/s wr, 11 op/s 2024-03-11T19:55:04.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:03 smithi204 ceph-mon[36739]: pgmap v824: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 52 KiB/s wr, 11 op/s 2024-03-11T19:55:04.638 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:55:06.166 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:05 smithi083 ceph-mon[31864]: pgmap v825: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-03-11T19:55:06.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:05 smithi137 ceph-mon[36220]: pgmap v825: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-03-11T19:55:06.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:05 smithi204 ceph-mon[36739]: pgmap v825: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-03-11T19:55:06.546 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:55:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-03-11T19:55:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-03-11T19:55:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-03-11T19:55:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-03-11T19:55:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-03-11T19:55:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001170343, 2024-03-11T19:55:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T19:55:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000304541, 2024-03-11T19:55:06.547 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:06.548 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 300, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 297, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 496976, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 178496, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.041444285, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.018008049, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.015992671, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.006379239, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 296, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.017483517, 2024-03-11T19:55:06.549 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000059065 2024-03-11T19:55:06.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:06.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:06.550 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:06.550 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:06.550 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:06.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:06.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 496976, 2024-03-11T19:55:06.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 178496 2024-03-11T19:55:06.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:06.550 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:06.550 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 6, 2024-03-11T19:55:06.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 228, 2024-03-11T19:55:06.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3312, 2024-03-11T19:55:06.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 195104, 2024-03-11T19:55:06.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:06.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-03-11T19:55:06.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.016579118, 2024-03-11T19:55:06.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.012501990, 2024-03-11T19:55:06.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001988088, 2024-03-11T19:55:06.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:06.551 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 227, 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.024391660, 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000107452 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 195104 2024-03-11T19:55:06.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:06.553 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:06.554 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:06.555 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148721, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-03-11T19:55:06.557 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 302, 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 454570, 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 302, 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 302, 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 454570, 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:06.558 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:06.559 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:06.559 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:55:07.211 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-11T19:55:07.212 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-11T19:55:07.212 DEBUG:tasks.check_counter:Getting stats from g 2024-03-11T19:55:07.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-11T19:55:07.456 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:55:08.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:07 smithi083 ceph-mon[31864]: pgmap v826: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-03-11T19:55:08.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:07 smithi137 ceph-mon[36220]: pgmap v826: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-03-11T19:55:08.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:07 smithi204 ceph-mon[36739]: pgmap v826: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-03-11T19:55:09.066 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 300, 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 295, 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 498880, 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 176448, 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.042247735, 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.017460293, 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.015774995, 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.006497058, 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 295, 2024-03-11T19:55:09.067 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.019854466, 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000067303 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 498880, 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 176448 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2, 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 223, 2024-03-11T19:55:09.068 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 320, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 194128, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.016870472, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.012673317, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001917490, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 221, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.017804546, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000080563 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:09.069 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 194128 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.000249227, 2024-03-11T19:55:09.070 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T19:55:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000188647, 2024-03-11T19:55:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:09.071 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:09.072 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:09.073 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:09.074 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-03-11T19:55:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:09.075 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148722, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-03-11T19:55:09.076 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-03-11T19:55:09.077 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 298, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 454050, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 298, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 298, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 454050, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:09.078 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:09.079 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:09.079 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:09.079 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:55:09.691 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-11T19:55:09.691 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-11T19:55:09.691 DEBUG:tasks.check_counter:Getting stats from j 2024-03-11T19:55:09.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-11T19:55:09.931 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.a/config 2024-03-11T19:55:10.183 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:09 smithi083 ceph-mon[31864]: pgmap v827: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 7 op/s 2024-03-11T19:55:10.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:09 smithi137 ceph-mon[36220]: pgmap v827: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 7 op/s 2024-03-11T19:55:10.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:09 smithi204 ceph-mon[36739]: pgmap v827: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 7 op/s 2024-03-11T19:55:11.690 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-03-11T19:55:11.691 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:11.691 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-03-11T19:55:11.691 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-03-11T19:55:11.691 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-03-11T19:55:11.691 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-03-11T19:55:11.691 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:11.691 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-03-11T19:55:11.691 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001169175, 2024-03-11T19:55:11.691 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T19:55:11.691 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000347605, 2024-03-11T19:55:11.691 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:11.692 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 302, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 299, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 497520, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 179744, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.042986291, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.018531736, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.016860219, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.006537887, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 298, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.023068076, 2024-03-11T19:55:11.693 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000077409 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 497520, 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 179744 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 6, 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 230, 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3312, 2024-03-11T19:55:11.694 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 195456, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.017552375, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.012796401, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.002282285, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 229, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.013141922, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000057388 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:11.695 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 195456 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:11.696 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:11.697 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:11.698 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148744, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:11.699 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-03-11T19:55:11.700 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 304, 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 454830, 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 304, 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 304, 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 454830, 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-03-11T19:55:11.701 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-03-11T19:55:12.046 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:11 smithi083 ceph-mon[31864]: pgmap v828: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-03-11T19:55:12.361 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-11T19:55:12.361 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-11T19:55:12.361 DEBUG:tasks.check_counter:Getting stats from b 2024-03-11T19:55:12.361 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-11T19:55:12.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:11 smithi137 ceph-mon[36220]: pgmap v828: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-03-11T19:55:12.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:11 smithi204 ceph-mon[36739]: pgmap v828: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-03-11T19:55:12.612 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:55:14.138 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:13 smithi137 ceph-mon[36220]: pgmap v829: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-03-11T19:55:14.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:13 smithi083 ceph-mon[31864]: pgmap v829: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-03-11T19:55:14.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:13 smithi204 ceph-mon[36739]: pgmap v829: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-03-11T19:55:14.513 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 11521, 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 11916, 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 3533390, 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 14382031, 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 241, 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.751361653, 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.240170500, 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.255364757, 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.203706332, 2024-03-11T19:55:14.514 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 11916, 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.471051890, 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000039531 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3428, 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000654528, 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000190 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.515 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 11743, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 12127, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 3959478, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 4457949, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 7, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.653946741, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.213537035, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.220794716, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.192745269, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 12128, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.545517347, 2024-03-11T19:55:14.516 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000044979 2024-03-11T19:55:14.517 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.517 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:14.517 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3330, 2024-03-11T19:55:14.517 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000649132, 2024-03-11T19:55:14.517 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000194 2024-03-11T19:55:14.517 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.517 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 599984, 2024-03-11T19:55:14.517 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 218352 2024-03-11T19:55:14.518 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.518 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:14.518 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 12489, 2024-03-11T19:55:14.518 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 14789, 2024-03-11T19:55:14.518 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 3424255, 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 9035787, 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 16, 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 8, 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.749791576, 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.269354835, 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.357629607, 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.190538674, 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 14784, 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.686841023, 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000046458 2024-03-11T19:55:14.519 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.520 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:14.520 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 7449, 2024-03-11T19:55:14.520 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001920627, 2024-03-11T19:55:14.520 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000257 2024-03-11T19:55:14.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.521 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 60544, 2024-03-11T19:55:14.521 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 596672 2024-03-11T19:55:14.521 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.521 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-11T19:55:14.521 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-03-11T19:55:14.521 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-11T19:55:14.521 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.521 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-03-11T19:55:14.521 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-11T19:55:14.521 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-11T19:55:14.522 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 20266, 2024-03-11T19:55:14.522 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 4.476480504, 2024-03-11T19:55:14.522 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000220886 2024-03-11T19:55:14.522 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:14.522 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.522 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-03-11T19:55:14.522 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-11T19:55:14.522 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-11T19:55:14.522 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 307, 2024-03-11T19:55:14.523 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.010116509, 2024-03-11T19:55:14.523 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000032952 2024-03-11T19:55:14.523 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:14.523 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.523 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-03-11T19:55:14.523 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 2356, 2024-03-11T19:55:14.523 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 2019, 2024-03-11T19:55:14.523 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-03-11T19:55:14.523 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2019, 2024-03-11T19:55:14.523 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 5.097845331, 2024-03-11T19:55:14.523 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002524935 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 337, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 39, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 3, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 534, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 1, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 441, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 92, 2024-03-11T19:55:14.524 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 532, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 1492, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 425, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 425, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 11, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 2942, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 2078, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 333, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 2, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 53, 2024-03-11T19:55:14.525 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 675, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 16, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 1479, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 21, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 864, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 23880, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 1027, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 1769342555, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-03-11T19:55:14.526 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 23880, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 984, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 29, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 955, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 22896, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 4, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 113, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 1609, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:55:14.527 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:55:14.528 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:55:14.528 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 1742, 2024-03-11T19:55:14.528 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 1325, 2024-03-11T19:55:14.528 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 45, 2024-03-11T19:55:14.528 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 270 2024-03-11T19:55:14.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.528 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-03-11T19:55:14.528 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-03-11T19:55:14.528 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-03-11T19:55:14.528 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-03-11T19:55:14.528 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 2, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 13, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 109, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 82, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 27, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:55:14.529 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-03-11T19:55:14.530 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-03-11T19:55:14.530 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 29, 2024-03-11T19:55:14.530 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 16, 2024-03-11T19:55:14.530 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-03-11T19:55:14.530 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 3, 2024-03-11T19:55:14.530 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-03-11T19:55:14.530 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-03-11T19:55:14.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.530 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 3847, 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 3847, 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 6, 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 6, 2024-03-11T19:55:14.531 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-03-11T19:55:14.532 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-03-11T19:55:14.532 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-03-11T19:55:14.532 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-03-11T19:55:14.532 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 14862798, 2024-03-11T19:55:14.532 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-03-11T19:55:14.532 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-03-11T19:55:14.532 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1478, 2024-03-11T19:55:14.532 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 8.819187590, 2024-03-11T19:55:14.532 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.005966974 2024-03-11T19:55:14.532 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.532 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 450, 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 2001, 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 1551, 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 54, 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 77, 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 23, 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 534, 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 2362, 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 1828, 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 425, 2024-03-11T19:55:14.533 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 1482, 2024-03-11T19:55:14.534 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 1057, 2024-03-11T19:55:14.534 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 65576, 2024-03-11T19:55:14.534 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 215420 2024-03-11T19:55:14.534 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.534 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-03-11T19:55:14.534 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 2554, 2024-03-11T19:55:14.534 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 51, 2024-03-11T19:55:14.534 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 2356, 2024-03-11T19:55:14.534 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 373, 2024-03-11T19:55:14.534 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 102, 2024-03-11T19:55:14.535 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-03-11T19:55:14.535 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1075, 2024-03-11T19:55:14.535 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 3.831816830, 2024-03-11T19:55:14.535 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.003564480 2024-03-11T19:55:14.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.535 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-03-11T19:55:14.535 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 21, 2024-03-11T19:55:14.535 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.021689381, 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001032827 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 27, 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.048007791, 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001778066 2024-03-11T19:55:14.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 233, 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.235599314, 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001011155 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-03-11T19:55:14.537 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000322044, 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000161022 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-03-11T19:55:14.539 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 37, 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.339256360, 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.009169090 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.540 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-03-11T19:55:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 414, 2024-03-11T19:55:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.117856737, 2024-03-11T19:55:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000284678 2024-03-11T19:55:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-03-11T19:55:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 52, 2024-03-11T19:55:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.101207799, 2024-03-11T19:55:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001946303 2024-03-11T19:55:14.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.542 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-03-11T19:55:14.542 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 67, 2024-03-11T19:55:14.542 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.244673368, 2024-03-11T19:55:14.542 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.003651841 2024-03-11T19:55:14.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.542 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-03-11T19:55:14.542 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.542 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.542 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.542 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.013094044, 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.006547022 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.543 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1, 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001006198, 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001006198 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.544 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.545 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 7, 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.018262683, 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002608954 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 81, 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.125052782, 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001543861 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:14.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 1, 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 32, 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 32, 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 973, 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.548 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:14.549 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:14.550 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:14.551 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 306900, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 544, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 352, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 4, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7256, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 154, 2024-03-11T19:55:14.552 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 1633691, 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 16777, 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-03-11T19:55:14.553 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-03-11T19:55:14.554 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 26426, 2024-03-11T19:55:14.554 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 11254536, 2024-03-11T19:55:14.554 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 37, 2024-03-11T19:55:14.554 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 26426, 2024-03-11T19:55:14.554 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-03-11T19:55:14.554 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 26426, 2024-03-11T19:55:14.554 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 44.369257094, 2024-03-11T19:55:14.554 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001679000 2024-03-11T19:55:14.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.554 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-03-11T19:55:14.554 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-03-11T19:55:14.555 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 26426, 2024-03-11T19:55:14.555 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 26666 2024-03-11T19:55:14.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.555 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 26426, 2024-03-11T19:55:14.555 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 23972, 2024-03-11T19:55:14.555 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 2454, 2024-03-11T19:55:14.555 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-03-11T19:55:14.555 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-03-11T19:55:14.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-03-11T19:55:14.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 11, 2024-03-11T19:55:14.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 63, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 1512, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 99, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 3, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 610, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 23909, 2024-03-11T19:55:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 22, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-03-11T19:55:14.557 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-03-11T19:55:14.558 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-03-11T19:55:14.558 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-03-11T19:55:14.558 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-03-11T19:55:14.558 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-03-11T19:55:14.558 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-03-11T19:55:14.558 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-03-11T19:55:14.558 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-03-11T19:55:14.558 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-03-11T19:55:14.558 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-03-11T19:55:14.558 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-03-11T19:55:14.558 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 87, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 3, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 356, 2024-03-11T19:55:14.559 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-03-11T19:55:14.560 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 81 2024-03-11T19:55:14.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.560 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-03-11T19:55:14.560 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-03-11T19:55:14.560 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:55:14.560 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-03-11T19:55:14.560 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-03-11T19:55:14.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.560 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-03-11T19:55:14.560 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-03-11T19:55:14.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.561 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-03-11T19:55:14.561 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-03-11T19:55:14.561 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 17, 2024-03-11T19:55:14.561 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-03-11T19:55:14.561 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 10, 2024-03-11T19:55:14.561 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 96, 2024-03-11T19:55:14.561 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 82, 2024-03-11T19:55:14.561 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-03-11T19:55:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-11T19:55:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 35749, 2024-03-11T19:55:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 7757856, 2024-03-11T19:55:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 35749, 2024-03-11T19:55:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-11T19:55:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 35749, 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 7757856, 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-11T19:55:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-11T19:55:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-11T19:55:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 26426, 2024-03-11T19:55:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 11262818, 2024-03-11T19:55:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 26426, 2024-03-11T19:55:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 11262818, 2024-03-11T19:55:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-03-11T19:55:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-03-11T19:55:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-11T19:55:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-11T19:55:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 26426, 2024-03-11T19:55:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 26426, 2024-03-11T19:55:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 26426, 2024-03-11T19:55:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 26426, 2024-03-11T19:55:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:55:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-11T19:55:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 82, 2024-03-11T19:55:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 8282, 2024-03-11T19:55:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 82, 2024-03-11T19:55:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-11T19:55:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-11T19:55:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 34, 2024-03-11T19:55:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 8282, 2024-03-11T19:55:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x558e906586e0": { 2024-03-11T19:55:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-11T19:55:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 3847, 2024-03-11T19:55:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 10668494, 2024-03-11T19:55:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 3847, 2024-03-11T19:55:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-11T19:55:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-11T19:55:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1478, 2024-03-11T19:55:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 10668494, 2024-03-11T19:55:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:14.570 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:14.570 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:14.570 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:14.570 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:14.570 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:14.570 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-11T19:55:15.314 INFO:tasks.check_counter:Daemon mds.b mds.exported=16 2024-03-11T19:55:15.314 INFO:tasks.check_counter:Daemon mds.b mds.imported=21 2024-03-11T19:55:15.314 DEBUG:tasks.check_counter:Getting stats from e 2024-03-11T19:55:15.315 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-11T19:55:15.603 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:55:16.067 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:15 smithi083 ceph-mon[31864]: pgmap v830: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-03-11T19:55:16.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:15 smithi137 ceph-mon[36220]: pgmap v830: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-03-11T19:55:16.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:15 smithi204 ceph-mon[36739]: pgmap v830: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-03-11T19:55:17.292 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-11T19:55:17.292 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:17.292 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3350, 2024-03-11T19:55:17.292 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 3350, 2024-03-11T19:55:17.292 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 4821387, 2024-03-11T19:55:17.292 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 1041061, 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.236484080, 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.065725842, 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.059102981, 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.095611901, 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3349, 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.116516041, 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000034791 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.293 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 4, 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 230, 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 876, 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 562700, 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.012008342, 2024-03-11T19:55:17.294 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.008479988, 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.001665607, 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000074619, 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 228, 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.012070985, 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000052942 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:55:17.295 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 562016 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 325, 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 307, 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 573068, 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 181239, 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.040953124, 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.015630046, 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.018397945, 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.006489118, 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:17.296 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 306, 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.012603219, 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000041186 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 572800, 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 180928 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-03-11T19:55:17.297 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5633, 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.123268559, 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000021883 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.298 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-03-11T19:55:17.299 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 712, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 2, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 710, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 709, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 10, 2024-03-11T19:55:17.300 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-03-11T19:55:17.301 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-03-11T19:55:17.302 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-03-11T19:55:17.303 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-03-11T19:55:17.304 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 1825, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-03-11T19:55:17.305 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-03-11T19:55:17.306 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 3, 2024-03-11T19:55:17.306 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-03-11T19:55:17.306 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-03-11T19:55:17.306 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-03-11T19:55:17.306 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-03-11T19:55:17.307 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 8074056, 2024-03-11T19:55:17.307 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 8071441, 2024-03-11T19:55:17.307 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-03-11T19:55:17.307 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.308 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.308 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.308 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 1825 2024-03-11T19:55:17.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 577, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 1147, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 570, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 36, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 43, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 7, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 712, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 1569, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 857, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-03-11T19:55:17.309 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 56000, 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 215420 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.311 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-03-11T19:55:17.312 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.313 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.313 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.313 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.313 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-03-11T19:55:17.313 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.315 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-03-11T19:55:17.315 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.315 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.315 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.315 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-03-11T19:55:17.315 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.315 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-03-11T19:55:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-03-11T19:55:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.319 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.319 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.319 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-03-11T19:55:17.319 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.319 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.319 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.319 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-03-11T19:55:17.320 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.320 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.320 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.320 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-03-11T19:55:17.320 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.320 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.321 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-03-11T19:55:17.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.321 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.322 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-03-11T19:55:17.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.322 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.322 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-03-11T19:55:17.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.322 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.323 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-03-11T19:55:17.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.323 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.323 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-03-11T19:55:17.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.324 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.324 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-03-11T19:55:17.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.324 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.325 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-03-11T19:55:17.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.325 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.325 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-03-11T19:55:17.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.325 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.326 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:55:17.327 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-03-11T19:55:17.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.328 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.328 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:17.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.328 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-03-11T19:55:17.328 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-03-11T19:55:17.328 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2024-03-11T19:55:17.329 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 1, 2024-03-11T19:55:17.329 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-03-11T19:55:17.329 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-03-11T19:55:17.329 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-03-11T19:55:17.329 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-03-11T19:55:17.329 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 2418, 2024-03-11T19:55:17.330 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-03-11T19:55:17.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.330 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-11T19:55:17.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:17.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:17.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:17.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:17.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:17.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:17.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-11T19:55:17.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:17.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:17.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:17.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:17.334 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 203143, 2024-03-11T19:55:17.334 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 617, 2024-03-11T19:55:17.334 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:55:17.334 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-03-11T19:55:17.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-11T19:55:17.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-11T19:55:17.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:17.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:17.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:17.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:17.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7256, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 154, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 1831722, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 16252, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-03-11T19:55:17.335 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 3349, 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 21, 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 3, 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 3349, 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3349, 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 4.075932513, 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001217059 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3349, 2024-03-11T19:55:17.336 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 3350 2024-03-11T19:55:17.337 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.337 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 3349, 2024-03-11T19:55:17.337 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 3349, 2024-03-11T19:55:17.337 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-03-11T19:55:17.337 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-03-11T19:55:17.337 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-03-11T19:55:17.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 2079, 2024-03-11T19:55:17.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-03-11T19:55:17.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 1268, 2024-03-11T19:55:17.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-03-11T19:55:17.337 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-03-11T19:55:17.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-03-11T19:55:17.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-03-11T19:55:17.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-03-11T19:55:17.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-03-11T19:55:17.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-03-11T19:55:17.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-03-11T19:55:17.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-03-11T19:55:17.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-03-11T19:55:17.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 1, 2024-03-11T19:55:17.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-03-11T19:55:17.338 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-03-11T19:55:17.339 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-03-11T19:55:17.340 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-03-11T19:55:17.340 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-03-11T19:55:17.340 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-03-11T19:55:17.340 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-03-11T19:55:17.340 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-03-11T19:55:17.340 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-03-11T19:55:17.340 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 87, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 2, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 4, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 5, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 1, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.341 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-03-11T19:55:17.342 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 3675, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 5044955, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 3675, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-11T19:55:17.343 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3675, 2024-03-11T19:55:17.344 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 5044955, 2024-03-11T19:55:17.344 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:17.344 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.344 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.344 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.344 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:17.344 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 3349, 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 3879757, 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3349, 2024-03-11T19:55:17.345 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 3879757, 2024-03-11T19:55:17.346 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:17.346 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.346 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.346 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.346 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:17.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.346 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-03-11T19:55:17.346 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:17.346 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-03-11T19:55:17.346 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:17.346 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 3349, 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 3349, 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3349, 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 3349, 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.347 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:55:17.348 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:17.348 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-11T19:55:17.348 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:17.348 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-11T19:55:17.348 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-11T19:55:17.348 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:17.348 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:17.349 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-11T19:55:17.349 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-11T19:55:17.349 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-03-11T19:55:17.349 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-03-11T19:55:17.349 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:17.349 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.349 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.349 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.350 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:17.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:17.350 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x5603f7ad2960": { 2024-03-11T19:55:17.350 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:17.350 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-11T19:55:17.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:17.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-11T19:55:17.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-11T19:55:17.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:17.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:17.351 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-11T19:55:17.351 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-11T19:55:17.351 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-03-11T19:55:17.352 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-03-11T19:55:17.352 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:17.352 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:17.352 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:17.352 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:17.352 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:17.352 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:17.353 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-11T19:55:18.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:17 smithi137 ceph-mon[36220]: pgmap v831: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-03-11T19:55:18.056 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-11T19:55:18.056 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-11T19:55:18.056 DEBUG:tasks.check_counter:Getting stats from h 2024-03-11T19:55:18.057 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-11T19:55:18.295 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:55:18.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:17 smithi083 ceph-mon[31864]: pgmap v831: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-03-11T19:55:18.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:17 smithi204 ceph-mon[36739]: pgmap v831: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2024-03-11T19:55:19.953 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-11T19:55:19.953 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:19.953 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 2809, 2024-03-11T19:55:19.953 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 3953, 2024-03-11T19:55:19.953 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1248333, 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 5456247, 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 69, 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 6, 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.235612781, 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.086109215, 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.114080551, 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.028313519, 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3951, 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.142944727, 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000036179 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:19.954 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2399, 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000675048, 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000281 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 594032, 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 211888 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1217, 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1619, 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 461894, 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 1140799, 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 11, 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-03-11T19:55:19.955 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.089515776, 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.032709956, 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.024625004, 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.024537132, 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1619, 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.076225779, 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000047082 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 344, 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000052170, 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000151 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.956 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 564832 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3683, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 3280, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1402801, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 1479728, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 10, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 6, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.246062053, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.097464854, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.110192548, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.042904965, 2024-03-11T19:55:19.957 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3278, 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.136709229, 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000041705 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3350, 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000710594, 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000212 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-11T19:55:19.958 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2604, 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.897645219, 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000344717 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-11T19:55:19.959 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 692, 2024-03-11T19:55:19.960 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.026580801, 2024-03-11T19:55:19.960 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000038411 2024-03-11T19:55:19.960 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:19.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.960 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-03-11T19:55:19.960 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 842, 2024-03-11T19:55:19.960 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 842, 2024-03-11T19:55:19.960 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-03-11T19:55:19.960 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 842, 2024-03-11T19:55:19.960 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 4.410316136, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.005237905 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 11, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 360, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 347, 2024-03-11T19:55:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 13, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 360, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 362, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 4, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 4, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 6, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 1484, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 1330, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 8, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 71, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 261, 2024-03-11T19:55:19.962 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 10, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 358, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 11, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 699, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 2757, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 1030, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 1767152940, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 2757, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:55:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 2757, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 41, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 609, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 497, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 293, 2024-03-11T19:55:19.964 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 136, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 831 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 8, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 205, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 205, 2024-03-11T19:55:19.965 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 10, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-03-11T19:55:19.966 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 1735, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 1735, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 3, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 3, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-03-11T19:55:19.967 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 7210416, 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 291, 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 4.618775132, 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.015872079 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-03-11T19:55:19.968 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 88, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 639, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 551, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 26, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 32, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 6, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 360, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 927, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 567, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 4, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 642, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 638, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 56780, 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 207228 2024-03-11T19:55:19.969 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 1003, 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 842, 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 21, 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 32, 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 200, 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.169941664, 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000849708 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.970 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 292, 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 3.727484888, 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.012765359 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.972 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.973 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.974 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.974 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-03-11T19:55:19.974 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.974 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.974 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.974 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-03-11T19:55:19.975 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 145, 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.076445834, 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000527212 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.976 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 199, 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.426547959, 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002143457 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.977 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.979 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-03-11T19:55:19.980 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 6, 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.009895791, 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001649298 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.981 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 1, 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 64, 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 64, 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 2421, 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.982 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:19.984 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 306710, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 197, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 704, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 8, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-11T19:55:19.985 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:19.986 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:19.986 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:19.986 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:19.986 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7256, 2024-03-11T19:55:19.986 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 154, 2024-03-11T19:55:19.987 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:19.987 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:19.987 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:19.987 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-11T19:55:19.987 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 509166, 2024-03-11T19:55:19.987 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 18144, 2024-03-11T19:55:19.987 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:19.987 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-11T19:55:19.987 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:19.987 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-11T19:55:19.987 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 3688, 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 3137044, 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 27, 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 3688, 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3688, 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 10.752821495, 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002915624 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-03-11T19:55:19.988 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3688, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 3815 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 3688, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 2842, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 846, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 10, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 85, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 376, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 129, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-03-11T19:55:19.989 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 217, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 2757, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 20, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-03-11T19:55:19.990 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-03-11T19:55:19.991 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 87, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 2, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-03-11T19:55:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 161, 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 60 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-03-11T19:55:19.993 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 10, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 10, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 205, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 205, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 7705, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 2420354, 2024-03-11T19:55:19.994 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 7705, 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 7705, 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2420354, 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-11T19:55:19.995 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 3688, 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 3157749, 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3688, 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 3157749, 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:19.996 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 3688, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 3688, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3688, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 3688, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.997 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 205, 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 20705, 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 205, 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-11T19:55:19.998 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 85, 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 20705, 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x556510e706e0": { 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 1735, 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 3016112, 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:19.999 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 1735, 2024-03-11T19:55:20.000 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-11T19:55:20.000 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-11T19:55:20.000 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 291, 2024-03-11T19:55:20.000 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 3016112, 2024-03-11T19:55:20.000 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:20.000 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:20.000 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:20.000 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:20.000 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:20.000 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:20.000 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-11T19:55:20.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:19 smithi137 ceph-mon[36220]: pgmap v832: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-11T19:55:20.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:19 smithi083 ceph-mon[31864]: pgmap v832: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-11T19:55:20.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:19 smithi204 ceph-mon[36739]: pgmap v832: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-11T19:55:20.772 INFO:tasks.check_counter:Daemon mds.h mds.exported=10 2024-03-11T19:55:20.772 INFO:tasks.check_counter:Daemon mds.h mds.imported=11 2024-03-11T19:55:20.772 DEBUG:tasks.check_counter:Getting stats from k 2024-03-11T19:55:20.772 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-11T19:55:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.b/config 2024-03-11T19:55:22.138 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:21 smithi137 ceph-mon[36220]: pgmap v833: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-11T19:55:22.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:21 smithi083 ceph-mon[31864]: pgmap v833: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-11T19:55:22.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:21 smithi204 ceph-mon[36739]: pgmap v833: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-11T19:55:22.702 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-11T19:55:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 4, 2024-03-11T19:55:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 4, 2024-03-11T19:55:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 2992, 2024-03-11T19:55:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 448, 2024-03-11T19:55:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-03-11T19:55:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.001540983, 2024-03-11T19:55:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000042596, 2024-03-11T19:55:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.000767868, 2024-03-11T19:55:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3, 2024-03-11T19:55:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000750148, 2024-03-11T19:55:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000250049 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 305, 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 302, 2024-03-11T19:55:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 498336, 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 181616, 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.038794893, 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.014544233, 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.017413719, 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.006277446, 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 301, 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.016602539, 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000055157 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 498336, 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 181616 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 2, 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 230, 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 320, 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 195360, 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-03-11T19:55:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.011673659, 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.008404009, 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.001208049, 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 228, 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.013251855, 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000058122 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:55:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 195360 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148721, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 30, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-11T19:55:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-11T19:55:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-11T19:55:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-11T19:55:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-11T19:55:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 307, 2024-03-11T19:55:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 455220, 2024-03-11T19:55:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 307, 2024-03-11T19:55:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-11T19:55:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-11T19:55:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 307, 2024-03-11T19:55:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 455220, 2024-03-11T19:55:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-11T19:55:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-11T19:55:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-11T19:55:22.715 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-11T19:55:22.715 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:22.715 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-11T19:55:22.715 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-11T19:55:23.359 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-11T19:55:23.359 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-11T19:55:23.359 DEBUG:tasks.check_counter:Getting stats from c 2024-03-11T19:55:23.359 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-11T19:55:23.608 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:55:23.990 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:23 smithi204 ceph-mon[36739]: pgmap v834: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-11T19:55:24.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:23 smithi083 ceph-mon[31864]: pgmap v834: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-11T19:55:24.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:23 smithi137 ceph-mon[36220]: pgmap v834: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-11T19:55:25.466 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-03-11T19:55:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1256, 2024-03-11T19:55:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1258, 2024-03-11T19:55:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 4604437, 2024-03-11T19:55:25.466 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 390626, 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.123088626, 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.035886231, 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.035662527, 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.040069941, 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1256, 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.054800772, 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000043631 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:25.467 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 538, 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 764, 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 132934, 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 643793, 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-03-11T19:55:25.468 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.052165169, 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.019038091, 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.010711101, 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.016502134, 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 762, 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.044509580, 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000058411 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.469 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 476720 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2164, 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2144, 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 7526507, 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 752222, 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.168460226, 2024-03-11T19:55:25.470 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.059265386, 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.051311274, 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.047563081, 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2144, 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.093498912, 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000043609 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.471 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 575952, 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 181536 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 6126, 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.144620408, 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000023607 2024-03-11T19:55:25.472 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-03-11T19:55:25.473 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 609, 2024-03-11T19:55:25.474 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 609, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 609, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 11, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-03-11T19:55:25.475 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 1027, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 1769354685, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-03-11T19:55:25.476 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:55:25.477 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-03-11T19:55:25.478 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-03-11T19:55:25.479 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-03-11T19:55:25.480 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 3849, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 6, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 14871815, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 14869827, 2024-03-11T19:55:25.481 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 3849 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 443, 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 1968, 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 1525, 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 44, 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 75, 2024-03-11T19:55:25.482 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 31, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 609, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 2080, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 1471, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 56924, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215420 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-03-11T19:55:25.483 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.484 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.485 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-03-11T19:55:25.486 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-03-11T19:55:25.487 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.488 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.488 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.488 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.488 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.489 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-03-11T19:55:25.490 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.491 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.492 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.493 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.494 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.495 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 2, 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 1, 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 2409, 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:25.496 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:25.497 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:25.498 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:25.498 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:25.498 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:25.499 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:25.499 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:25.499 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:25.499 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:25.499 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:25.499 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 214029, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 483, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:55:25.500 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 7256, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 154, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 1472729, 2024-03-11T19:55:25.501 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 12888, 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 3628, 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 26, 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 4, 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 3628, 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-03-11T19:55:25.502 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3628, 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 4.121944647, 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001136147 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3628, 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 3629 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 3628, 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 3628, 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-03-11T19:55:25.503 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 2168, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 1457, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-03-11T19:55:25.504 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 2, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-03-11T19:55:25.505 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-03-11T19:55:25.506 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 87, 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 2, 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 6, 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 7, 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 1, 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-03-11T19:55:25.507 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-03-11T19:55:25.508 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 3954, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 11888389, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 3954, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 3954, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 11888389, 2024-03-11T19:55:25.509 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:25.510 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 3628, 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 10677676, 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 3628, 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 10677676, 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-03-11T19:55:25.511 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 3628, 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 3628, 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 3628, 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3628, 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:25.512 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-03-11T19:55:25.513 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55af00718960": { 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-11T19:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-03-11T19:55:26.138 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:25 smithi137 ceph-mon[36220]: pgmap v835: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-11T19:55:26.269 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:25 smithi204 ceph-mon[36739]: pgmap v835: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-11T19:55:26.297 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-11T19:55:26.297 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-11T19:55:26.297 DEBUG:tasks.check_counter:Getting stats from f 2024-03-11T19:55:26.298 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-11T19:55:26.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:25 smithi083 ceph-mon[31864]: pgmap v835: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-11T19:55:26.547 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:55:28.222 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:27 smithi204 ceph-mon[36739]: pgmap v836: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-11T19:55:28.272 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 327, 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 309, 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 574178, 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 182493, 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.041757543, 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.016318624, 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.016921207, 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.006700295, 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 308, 2024-03-11T19:55:28.273 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.012146163, 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000039435 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 573344, 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 182176 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 999, 2024-03-11T19:55:28.274 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1226, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 245690, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 787279, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.090904050, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.028272938, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.019578509, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.033537956, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1224, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.043358163, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000035423 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.275 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 476896 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2353, 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2353, 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 3712067, 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 731045, 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:28.276 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.193579340, 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.055438716, 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.055915571, 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.066792774, 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2352, 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.095383546, 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000040554 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.277 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 5631, 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.146706360, 2024-03-11T19:55:28.278 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000026053 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-03-11T19:55:28.279 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 554, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 554, 2024-03-11T19:55:28.280 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 554, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 3, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-03-11T19:55:28.281 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:55:28.282 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-03-11T19:55:28.283 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-03-11T19:55:28.284 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-03-11T19:55:28.285 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 1735, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 3, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-03-11T19:55:28.286 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 7210416, 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 7209842, 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 1735 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 77, 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 637, 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 560, 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 12, 2024-03-11T19:55:28.287 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 30, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 18, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 554, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 925, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 371, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 49532, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215420 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-03-11T19:55:28.288 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.289 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.290 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.291 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.293 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.293 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-03-11T19:55:28.293 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.293 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.293 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.294 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.295 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.296 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.297 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.298 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 2, 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 1, 2024-03-11T19:55:28.300 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 2411, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:28.301 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:28.302 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:28.303 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 192723, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 121, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 7256, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 154, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:28.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 588587, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 17675, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 3348, 2024-03-11T19:55:28.305 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-03-11T19:55:28.306 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-03-11T19:55:28.306 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 3348, 2024-03-11T19:55:28.306 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-03-11T19:55:28.306 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3348, 2024-03-11T19:55:28.306 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 15.044800792, 2024-03-11T19:55:28.306 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.004493668 2024-03-11T19:55:28.306 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.306 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-03-11T19:55:28.306 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-03-11T19:55:28.306 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3348, 2024-03-11T19:55:28.306 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 3349 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 3348, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 3348, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 2081, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 1265, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-03-11T19:55:28.307 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-03-11T19:55:28.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-03-11T19:55:28.309 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 87, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 2, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 5, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 6, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 1, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-03-11T19:55:28.310 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-03-11T19:55:28.311 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 3675, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 4181498, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 3675, 2024-03-11T19:55:28.312 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 3675, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 4181498, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-11T19:55:28.313 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 3348, 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 3016117, 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 3348, 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3016117, 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:28.314 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 3348, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 3348, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 3348, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3348, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.315 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:28.316 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x5619c78ba960": { 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-11T19:55:28.317 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-03-11T19:55:28.318 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-03-11T19:55:28.318 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:28.318 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:28.318 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:28.318 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:28.318 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:28.318 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:28.318 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-03-11T19:55:28.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:27 smithi083 ceph-mon[31864]: pgmap v836: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-11T19:55:28.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:27 smithi137 ceph-mon[36220]: pgmap v836: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-11T19:55:29.105 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-11T19:55:29.105 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-11T19:55:29.105 DEBUG:tasks.check_counter:Getting stats from i 2024-03-11T19:55:29.105 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-11T19:55:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:55:30.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:29 smithi204 ceph-mon[36739]: pgmap v837: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-03-11T19:55:30.211 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:29 smithi137 ceph-mon[36220]: pgmap v837: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-03-11T19:55:30.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:29 smithi083 ceph-mon[31864]: pgmap v837: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-03-11T19:55:31.120 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-03-11T19:55:31.120 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 4453, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 4066, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2195547, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 6250131, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 67, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.292910335, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.102136815, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.132500405, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.058850973, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4064, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.148534177, 2024-03-11T19:55:31.121 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000036548 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3309, 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000549864, 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000166 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1597, 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1644, 2024-03-11T19:55:31.122 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 456741, 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 1188641, 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 13, 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.096306335, 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.036076403, 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.029449155, 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.025642531, 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1642, 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.058939315, 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000035894 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.123 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 372, 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000062786, 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000168 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1904, 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 567680 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 4703, 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 5825, 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1698335, 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 3320982, 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 10, 2024-03-11T19:55:31.124 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.261540901, 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.114655586, 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.151236216, 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.023878144, 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 5825, 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.170448358, 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000029261 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4438, 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001214984, 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000273 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.125 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 597552, 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 209664 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4674, 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.492741439, 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000319371 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:31.126 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000007103, 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000003551 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 1927, 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 1926, 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1926, 2024-03-11T19:55:31.127 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 3.377098862, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001753426 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 1, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 11, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 782, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 44, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 20, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 718, 2024-03-11T19:55:31.128 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 718, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 782, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 566, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 566, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 11, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 4940, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 3695, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 5, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 476, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 25, 2024-03-11T19:55:31.129 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 12, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 846, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 6, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 1120, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 3156, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 1030, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 1767152940, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 3156, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-03-11T19:55:31.130 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 32, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 32, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 3124, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 317, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 703, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 961, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 438, 2024-03-11T19:55:31.131 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 46, 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 1187 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 5, 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 2, 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 279, 2024-03-11T19:55:31.132 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 279, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 12, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-03-11T19:55:31.133 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 1846, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 1846, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 3, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-03-11T19:55:31.134 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 3, 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 8115233, 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1202, 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 4.366479454, 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003632678 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.135 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 650, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 1152, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 502, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 46, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 47, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 1, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 782, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 1574, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 792, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 151, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 1240, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 1089, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 60436, 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 207228 2024-03-11T19:55:31.136 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 3072, 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 1927, 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 572, 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 19, 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 37, 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.033520330, 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000905954 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1, 2024-03-11T19:55:31.137 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000219061, 2024-03-11T19:55:31.138 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000219061 2024-03-11T19:55:31.138 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.138 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-03-11T19:55:31.138 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.138 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.138 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.138 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.139 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-03-11T19:55:31.139 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 279, 2024-03-11T19:55:31.139 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.494425137, 2024-03-11T19:55:31.139 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001772133 2024-03-11T19:55:31.139 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.139 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-03-11T19:55:31.139 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 940, 2024-03-11T19:55:31.139 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.830424862, 2024-03-11T19:55:31.139 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000883430 2024-03-11T19:55:31.139 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.139 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-03-11T19:55:31.140 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.140 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.140 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.140 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.140 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-03-11T19:55:31.140 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.140 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.140 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.140 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.140 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-03-11T19:55:31.141 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.141 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.141 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.141 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.141 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-03-11T19:55:31.141 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.141 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.141 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.141 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.141 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-03-11T19:55:31.141 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.142 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.142 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.142 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.142 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-03-11T19:55:31.142 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.142 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.142 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.142 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-03-11T19:55:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 78, 2024-03-11T19:55:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.089016253, 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001141234 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 33, 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.033391670, 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001011868 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 279, 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.538310790, 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.005513658 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-03-11T19:55:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-03-11T19:55:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 279, 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.357790759, 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001282404 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-03-11T19:55:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 2, 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 1, 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 314, 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 314, 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 2415, 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-03-11T19:55:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 322986, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 748, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-03-11T19:55:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 7256, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 154, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 2311678, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 31925, 2024-03-11T19:55:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-03-11T19:55:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-03-11T19:55:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-03-11T19:55:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-03-11T19:55:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-03-11T19:55:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 4595, 2024-03-11T19:55:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 3968045, 2024-03-11T19:55:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 28, 2024-03-11T19:55:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 4595, 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4595, 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 11.269848900, 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002452633 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4595, 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 4771 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 4595, 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 3156, 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 1439, 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-03-11T19:55:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-03-11T19:55:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-03-11T19:55:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-03-11T19:55:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 0, 2024-03-11T19:55:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 1202, 2024-03-11T19:55:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 61, 2024-03-11T19:55:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-03-11T19:55:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-03-11T19:55:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-03-11T19:55:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 6, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 3156, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-03-11T19:55:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-03-11T19:55:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 87, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 2, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 12, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 12, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 213, 2024-03-11T19:55:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-03-11T19:55:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 103 2024-03-11T19:55:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-03-11T19:55:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-03-11T19:55:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-03-11T19:55:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-03-11T19:55:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-03-11T19:55:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-03-11T19:55:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-11T19:55:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 10749, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 3415928, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 10749, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 10749, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3415928, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 4595, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 3968045, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 4595, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3968045, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 4595, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 4595, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 4595, 2024-03-11T19:55:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 4595, 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-03-11T19:55:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x564690eb66e0": { 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 40603, 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 1846, 2024-03-11T19:55:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 3920929, 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 1846, 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 1202, 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3880326, 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:31.169 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-03-11T19:55:31.912 INFO:tasks.check_counter:Daemon mds.i mds.exported=12 2024-03-11T19:55:31.913 INFO:tasks.check_counter:Daemon mds.i mds.imported=6 2024-03-11T19:55:31.913 DEBUG:tasks.check_counter:Getting stats from l 2024-03-11T19:55:31.913 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf shell --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-11T19:55:32.153 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/mon.c/config 2024-03-11T19:55:32.274 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:31 smithi204 ceph-mon[36739]: pgmap v838: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-11T19:55:32.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:31 smithi083 ceph-mon[31864]: pgmap v838: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-11T19:55:32.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:31 smithi137 ceph-mon[36220]: pgmap v838: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-11T19:55:33.928 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-03-11T19:55:33.928 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 0, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 0, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 0, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 0, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.001166745, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.000000000, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.000359077, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:33.929 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2, 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 233, 2024-03-11T19:55:33.930 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 320, 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 195888, 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.011552914, 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.008304823, 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.001161510, 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 231, 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.010632179, 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000046026 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:33.931 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 195888 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 312, 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 307, 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 502144, 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 183936, 2024-03-11T19:55:33.932 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 2, 2024-03-11T19:55:33.933 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-03-11T19:55:33.933 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.039764929, 2024-03-11T19:55:33.933 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.016187391, 2024-03-11T19:55:33.933 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.015559397, 2024-03-11T19:55:33.933 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.006395048, 2024-03-11T19:55:33.933 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-03-11T19:55:33.933 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 307, 2024-03-11T19:55:33.933 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.018140319, 2024-03-11T19:55:33.933 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000059088 2024-03-11T19:55:33.933 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:33.933 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 502144, 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 183936 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 0, 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:33.934 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-03-11T19:55:33.935 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-03-11T19:55:33.936 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-11T19:55:33.937 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-03-11T19:55:33.937 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-03-11T19:55:33.937 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-03-11T19:55:33.937 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-03-11T19:55:33.937 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-03-11T19:55:33.937 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-03-11T19:55:33.937 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-03-11T19:55:33.937 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-03-11T19:55:33.937 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-03-11T19:55:33.937 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 148761, 2024-03-11T19:55:33.938 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 31, 2024-03-11T19:55:33.938 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-03-11T19:55:33.938 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-03-11T19:55:33.938 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-03-11T19:55:33.938 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-03-11T19:55:33.938 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-03-11T19:55:33.938 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-03-11T19:55:33.938 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-03-11T19:55:33.938 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-03-11T19:55:33.938 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 0, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 0, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 928, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-11T19:55:33.939 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 310, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 455610, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 310, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 310, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 455610, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-03-11T19:55:33.940 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:33.941 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-03-11T19:55:33.941 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-03-11T19:55:34.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:33 smithi204 ceph-mon[36739]: pgmap v839: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:55:34.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:33 smithi083 ceph-mon[31864]: pgmap v839: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:55:34.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:33 smithi137 ceph-mon[36220]: pgmap v839: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-11T19:55:34.620 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-11T19:55:34.620 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-11T19:55:34.620 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-11T19:55:34.633 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-11T19:55:34.634 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:55:34.634 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T19:55:34.656 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-03-11T19:55:34.656 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T19:55:34.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:55:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.862+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/2230866281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9b81000a0 msgr2=0x7fc9b81004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.862+0000 7fc9bf7d7640 1 --2- 172.21.15.83:0/2230866281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9b81000a0 0x7fc9b81004c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009900 tx=0x7fc9b402bc30 comp rx=0 tx=0).stop 2024-03-11T19:55:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.862+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/2230866281 shutdown_connections 2024-03-11T19:55:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.862+0000 7fc9bf7d7640 1 --2- 172.21.15.83:0/2230866281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b8101c50 0x7fc9b81084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.862+0000 7fc9bf7d7640 1 --2- 172.21.15.83:0/2230866281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b8101290 0x7fc9b8101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.862+0000 7fc9bf7d7640 1 --2- 172.21.15.83:0/2230866281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9b81000a0 0x7fc9b81004c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:34.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.862+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/2230866281 >> 172.21.15.83:0/2230866281 conn(0x7fc9b80fb610 msgr2=0x7fc9b80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/2230866281 shutdown_connections 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/2230866281 wait complete. 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bf7d7640 1 Processor -- start 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bf7d7640 1 -- start start 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bf7d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9b81000a0 0x7fc9b8197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bf7d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b8101290 0x7fc9b81977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bf7d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b8101c50 0x7fc9b819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bf7d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b819ed60 con 0x7fc9b81000a0 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bf7d7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b819eed0 con 0x7fc9b8101290 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bf7d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b819f090 con 0x7fc9b8101c50 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bd573640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9b81000a0 0x7fc9b8197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bd573640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9b81000a0 0x7fc9b8197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43698/0 (socket says 172.21.15.83:43698) 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bd573640 1 -- 172.21.15.83:0/1063881870 learned_addr learned my addr 172.21.15.83:0/1063881870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:34.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bdd74640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b8101c50 0x7fc9b819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.863+0000 7fc9bcd72640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b8101290 0x7fc9b81977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.864+0000 7fc9bd573640 1 -- 172.21.15.83:0/1063881870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b8101c50 msgr2=0x7fc9b819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.864+0000 7fc9bd573640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b8101c50 0x7fc9b819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.864+0000 7fc9bd573640 1 -- 172.21.15.83:0/1063881870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b8101290 msgr2=0x7fc9b81977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.864+0000 7fc9bd573640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b8101290 0x7fc9b81977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.864+0000 7fc9bd573640 1 -- 172.21.15.83:0/1063881870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b4009560 con 0x7fc9b81000a0 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.864+0000 7fc9bd573640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9b81000a0 0x7fc9b8197270 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4036040 tx=0x7fc9b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.864+0000 7fc9ae7fc640 1 -- 172.21.15.83:0/1063881870 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40044e0 con 0x7fc9b81000a0 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.864+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/1063881870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b819f310 con 0x7fc9b81000a0 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.864+0000 7fc9ae7fc640 1 -- 172.21.15.83:0/1063881870 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9b40054e0 con 0x7fc9b81000a0 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.864+0000 7fc9ae7fc640 1 -- 172.21.15.83:0/1063881870 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b403e450 con 0x7fc9b81000a0 2024-03-11T19:55:34.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.864+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/1063881870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b819f8a0 con 0x7fc9b81000a0 2024-03-11T19:55:34.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.865+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/1063881870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b8107410 con 0x7fc9b81000a0 2024-03-11T19:55:34.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.866+0000 7fc9ae7fc640 1 -- 172.21.15.83:0/1063881870 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fc9b4003710 con 0x7fc9b81000a0 2024-03-11T19:55:34.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.867+0000 7fc9ae7fc640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc9a40a1a30 0x7fc9a40a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:34.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.867+0000 7fc9bcd72640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc9a40a1a30 0x7fc9a40a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:34.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.867+0000 7fc9ae7fc640 1 -- 172.21.15.83:0/1063881870 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fc9b40efe20 con 0x7fc9b81000a0 2024-03-11T19:55:34.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.867+0000 7fc9bcd72640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc9a40a1a30 0x7fc9a40a3ef0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8198780 tx=0x7fc9a8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:34.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:34.872+0000 7fc9ae7fc640 1 -- 172.21.15.83:0/1063881870 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc9b40b90c0 con 0x7fc9b81000a0 2024-03-11T19:55:35.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.048+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/1063881870 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9b802cdd0 con 0x7fc9b81000a0 2024-03-11T19:55:35.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.050+0000 7fc9ae7fc640 1 -- 172.21.15.83:0/1063881870 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7fc9b40bc780 con 0x7fc9b81000a0 2024-03-11T19:55:35.054 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:55:35.054 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/1063881870 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc9a40a1a30 msgr2=0x7fc9a40a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc9a40a1a30 0x7fc9a40a3ef0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8198780 tx=0x7fc9a8008040 comp rx=0 tx=0).stop 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/1063881870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9b81000a0 msgr2=0x7fc9b8197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9b81000a0 0x7fc9b8197270 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4036040 tx=0x7fc9b40042f0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/1063881870 shutdown_connections 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fc9a40a1a30 0x7fc9a40a3ef0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9b8101c50 0x7fc9b819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9b8101290 0x7fc9b81977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 --2- 172.21.15.83:0/1063881870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9b81000a0 0x7fc9b8197270 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/1063881870 >> 172.21.15.83:0/1063881870 conn(0x7fc9b80fb610 msgr2=0x7fc9b81045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/1063881870 shutdown_connections 2024-03-11T19:55:35.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.052+0000 7fc9bf7d7640 1 -- 172.21.15.83:0/1063881870 wait complete. 2024-03-11T19:55:35.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T19:32:34.761694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T19:37:03.034982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T19:37:04.125352+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T19:55:35.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T19:55:35.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.168+0000 7ff11a26f640 1 -- 172.21.15.83:0/215067236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114104530 msgr2=0x7ff114106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.168+0000 7ff11a26f640 1 --2- 172.21.15.83:0/215067236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114104530 0x7ff114106920 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009900 tx=0x7ff0fc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:55:35.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.168+0000 7ff11a26f640 1 -- 172.21.15.83:0/215067236 shutdown_connections 2024-03-11T19:55:35.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.168+0000 7ff11a26f640 1 --2- 172.21.15.83:0/215067236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114106e60 0x7ff1141092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.168+0000 7ff11a26f640 1 --2- 172.21.15.83:0/215067236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114104530 0x7ff114106920 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.168+0000 7ff11a26f640 1 --2- 172.21.15.83:0/215067236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff114101c00 0x7ff114103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.168+0000 7ff11a26f640 1 -- 172.21.15.83:0/215067236 >> 172.21.15.83:0/215067236 conn(0x7ff1140fb610 msgr2=0x7ff1140fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.168+0000 7ff11a26f640 1 -- 172.21.15.83:0/215067236 shutdown_connections 2024-03-11T19:55:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.168+0000 7ff11a26f640 1 -- 172.21.15.83:0/215067236 wait complete. 2024-03-11T19:55:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff11a26f640 1 Processor -- start 2024-03-11T19:55:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff11a26f640 1 -- start start 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff11a26f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114101c00 0x7ff1141972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff11a26f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114104530 0x7ff114197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff11a26f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff114106e60 0x7ff11419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff11a26f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419ede0 con 0x7ff114101c00 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff11a26f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419ef50 con 0x7ff114104530 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff11a26f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419f110 con 0x7ff114106e60 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff113fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114101c00 0x7ff1141972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff113fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114101c00 0x7ff1141972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43730/0 (socket says 172.21.15.83:43730) 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff113fff640 1 -- 172.21.15.83:0/2852945042 learned_addr learned my addr 172.21.15.83:0/2852945042 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff1137fe640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114104530 0x7ff114197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff113fff640 1 -- 172.21.15.83:0/2852945042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff114106e60 msgr2=0x7ff11419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff113fff640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff114106e60 0x7ff11419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff113fff640 1 -- 172.21.15.83:0/2852945042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114104530 msgr2=0x7ff114197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff113fff640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114104530 0x7ff114197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.169+0000 7ff113fff640 1 -- 172.21.15.83:0/2852945042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc009560 con 0x7ff114101c00 2024-03-11T19:55:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.170+0000 7ff113fff640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114101c00 0x7ff1141972f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff10400b4a0 tx=0x7ff10400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.170+0000 7ff1117fa640 1 -- 172.21.15.83:0/2852945042 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104010610 con 0x7ff114101c00 2024-03-11T19:55:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.170+0000 7ff1117fa640 1 -- 172.21.15.83:0/2852945042 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff104010c50 con 0x7ff114101c00 2024-03-11T19:55:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.170+0000 7ff11a26f640 1 -- 172.21.15.83:0/2852945042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11419f3f0 con 0x7ff114101c00 2024-03-11T19:55:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.170+0000 7ff1117fa640 1 -- 172.21.15.83:0/2852945042 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104017e20 con 0x7ff114101c00 2024-03-11T19:55:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.170+0000 7ff11a26f640 1 -- 172.21.15.83:0/2852945042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11419f950 con 0x7ff114101c00 2024-03-11T19:55:35.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.171+0000 7ff11a26f640 1 -- 172.21.15.83:0/2852945042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1140fd100 con 0x7ff114101c00 2024-03-11T19:55:35.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.172+0000 7ff1117fa640 1 -- 172.21.15.83:0/2852945042 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7ff104017480 con 0x7ff114101c00 2024-03-11T19:55:35.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.172+0000 7ff1117fa640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1000a1a30 0x7ff1000a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.172+0000 7ff1137fe640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1000a1a30 0x7ff1000a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:35.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.172+0000 7ff1117fa640 1 -- 172.21.15.83:0/2852945042 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7ff1040cca50 con 0x7ff114101c00 2024-03-11T19:55:35.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.173+0000 7ff1137fe640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1000a1a30 0x7ff1000a3ef0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc02c240 tx=0x7ff0fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:35.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.177+0000 7ff1117fa640 1 -- 172.21.15.83:0/2852945042 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff104095d10 con 0x7ff114101c00 2024-03-11T19:55:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.349+0000 7ff11a26f640 1 -- 172.21.15.83:0/2852945042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff11410cc20 con 0x7ff114101c00 2024-03-11T19:55:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.349+0000 7ff1117fa640 1 -- 172.21.15.83:0/2852945042 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff1040993d0 con 0x7ff114101c00 2024-03-11T19:55:35.354 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-03-11T19:55:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 -- 172.21.15.83:0/2852945042 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1000a1a30 msgr2=0x7ff1000a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1000a1a30 0x7ff1000a3ef0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc02c240 tx=0x7ff0fc005fd0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 -- 172.21.15.83:0/2852945042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114101c00 msgr2=0x7ff1141972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114101c00 0x7ff1141972f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff10400b4a0 tx=0x7ff10400b960 comp rx=0 tx=0).stop 2024-03-11T19:55:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 -- 172.21.15.83:0/2852945042 shutdown_connections 2024-03-11T19:55:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7ff1000a1a30 0x7ff1000a3ef0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff114106e60 0x7ff11419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114104530 0x7ff114197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 --2- 172.21.15.83:0/2852945042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff114101c00 0x7ff1141972f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 -- 172.21.15.83:0/2852945042 >> 172.21.15.83:0/2852945042 conn(0x7ff1140fb610 msgr2=0x7ff1141024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 -- 172.21.15.83:0/2852945042 shutdown_connections 2024-03-11T19:55:35.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.352+0000 7ff11a26f640 1 -- 172.21.15.83:0/2852945042 wait complete. 2024-03-11T19:55:35.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T19:55:35.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.466+0000 7f56c730d640 1 -- 172.21.15.83:0/2303412449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0100080 msgr2=0x7f56c01004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.466+0000 7f56c730d640 1 --2- 172.21.15.83:0/2303412449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0100080 0x7f56c01004a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009900 tx=0x7f56bc02bc30 comp rx=0 tx=0).stop 2024-03-11T19:55:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.466+0000 7f56c730d640 1 -- 172.21.15.83:0/2303412449 shutdown_connections 2024-03-11T19:55:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.466+0000 7f56c730d640 1 --2- 172.21.15.83:0/2303412449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0101c30 0x7f56c01084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.466+0000 7f56c730d640 1 --2- 172.21.15.83:0/2303412449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101270 0x7f56c01016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.466+0000 7f56c730d640 1 --2- 172.21.15.83:0/2303412449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0100080 0x7f56c01004a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.466+0000 7f56c730d640 1 -- 172.21.15.83:0/2303412449 >> 172.21.15.83:0/2303412449 conn(0x7f56c00fb610 msgr2=0x7f56c00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.466+0000 7f56c730d640 1 -- 172.21.15.83:0/2303412449 shutdown_connections 2024-03-11T19:55:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.466+0000 7f56c730d640 1 -- 172.21.15.83:0/2303412449 wait complete. 2024-03-11T19:55:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c730d640 1 Processor -- start 2024-03-11T19:55:35.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c730d640 1 -- start start 2024-03-11T19:55:35.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c730d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100080 0x7f56c0199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c730d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101270 0x7f56c01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c730d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101c30 0x7f56c01a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c730d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c01a0f10 con 0x7f56c0101c30 2024-03-11T19:55:35.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c730d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c01a1080 con 0x7f56c0100080 2024-03-11T19:55:35.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c730d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c01a1240 con 0x7f56c0101270 2024-03-11T19:55:35.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c58aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101c30 0x7f56c01a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:35.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c58aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101c30 0x7f56c01a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43750/0 (socket says 172.21.15.83:43750) 2024-03-11T19:55:35.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c58aa640 1 -- 172.21.15.83:0/2878950315 learned_addr learned my addr 172.21.15.83:0/2878950315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:35.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c50a9640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100080 0x7f56c0199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:35.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c48a8640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101270 0x7f56c01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c58aa640 1 -- 172.21.15.83:0/2878950315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101270 msgr2=0x7f56c01999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c58aa640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101270 0x7f56c01999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c58aa640 1 -- 172.21.15.83:0/2878950315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100080 msgr2=0x7f56c0199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c58aa640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100080 0x7f56c0199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c58aa640 1 -- 172.21.15.83:0/2878950315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc009560 con 0x7f56c0101c30 2024-03-11T19:55:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.467+0000 7f56c50a9640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100080 0x7f56c0199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:55:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.468+0000 7f56c58aa640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101c30 0x7f56c01a09d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f56b800eaa0 tx=0x7f56b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.468+0000 7f56b67fc640 1 -- 172.21.15.83:0/2878950315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b800cdb0 con 0x7f56c0101c30 2024-03-11T19:55:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.468+0000 7f56c730d640 1 -- 172.21.15.83:0/2878950315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c01a1520 con 0x7f56c0101c30 2024-03-11T19:55:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.468+0000 7f56b67fc640 1 -- 172.21.15.83:0/2878950315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56b8005020 con 0x7f56c0101c30 2024-03-11T19:55:35.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.468+0000 7f56b67fc640 1 -- 172.21.15.83:0/2878950315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80116b0 con 0x7f56c0101c30 2024-03-11T19:55:35.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.468+0000 7f56c730d640 1 -- 172.21.15.83:0/2878950315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c01a19e0 con 0x7f56c0101c30 2024-03-11T19:55:35.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.469+0000 7f56c730d640 1 -- 172.21.15.83:0/2878950315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c010ba90 con 0x7f56c0101c30 2024-03-11T19:55:35.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.470+0000 7f56b67fc640 1 -- 172.21.15.83:0/2878950315 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f56b8005520 con 0x7f56c0101c30 2024-03-11T19:55:35.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.471+0000 7f56b67fc640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56ac0a1a30 0x7f56ac0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.471+0000 7f56c50a9640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56ac0a1a30 0x7f56ac0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:35.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.471+0000 7f56b67fc640 1 -- 172.21.15.83:0/2878950315 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f56b80cda40 con 0x7f56c0101c30 2024-03-11T19:55:35.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.471+0000 7f56c50a9640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56ac0a1a30 0x7f56ac0a3ef0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f56bc006010 tx=0x7f56bc005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:35.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.474+0000 7f56b67fc640 1 -- 172.21.15.83:0/2878950315 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56b8096d00 con 0x7f56c0101c30 2024-03-11T19:55:35.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.658+0000 7f56c730d640 1 -- 172.21.15.83:0/2878950315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56c010bc20 con 0x7f56c0101c30 2024-03-11T19:55:35.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.659+0000 7f56b67fc640 1 -- 172.21.15.83:0/2878950315 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f56b801f0a0 con 0x7f56c0101c30 2024-03-11T19:55:35.662 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-03-11T19:55:35.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.660+0000 7f56c730d640 1 -- 172.21.15.83:0/2878950315 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56ac0a1a30 msgr2=0x7f56ac0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.660+0000 7f56c730d640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56ac0a1a30 0x7f56ac0a3ef0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f56bc006010 tx=0x7f56bc005be0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.660+0000 7f56c730d640 1 -- 172.21.15.83:0/2878950315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101c30 msgr2=0x7f56c01a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.660+0000 7f56c730d640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101c30 0x7f56c01a09d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f56b800eaa0 tx=0x7f56b800ef60 comp rx=0 tx=0).stop 2024-03-11T19:55:35.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.660+0000 7f56c730d640 1 -- 172.21.15.83:0/2878950315 shutdown_connections 2024-03-11T19:55:35.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.660+0000 7f56c730d640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f56ac0a1a30 0x7f56ac0a3ef0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.660+0000 7f56c730d640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56c0101c30 0x7f56c01a09d0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.660+0000 7f56c730d640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56c0101270 0x7f56c01999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.660+0000 7f56c730d640 1 --2- 172.21.15.83:0/2878950315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c0100080 0x7f56c0199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.660+0000 7f56c730d640 1 -- 172.21.15.83:0/2878950315 >> 172.21.15.83:0/2878950315 conn(0x7f56c00fb610 msgr2=0x7f56c00fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:35.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.661+0000 7f56c730d640 1 -- 172.21.15.83:0/2878950315 shutdown_connections 2024-03-11T19:55:35.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.661+0000 7f56c730d640 1 -- 172.21.15.83:0/2878950315 wait complete. 2024-03-11T19:55:35.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 -- 172.21.15.83:0/3337909708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0700ff1f0 msgr2=0x7fb0700ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 --2- 172.21.15.83:0/3337909708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0700ff1f0 0x7fb0700ff670 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fb058009900 tx=0x7fb05802bc30 comp rx=0 tx=0).stop 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 -- 172.21.15.83:0/3337909708 shutdown_connections 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 --2- 172.21.15.83:0/3337909708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0700ffbb0 0x7fb070109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 --2- 172.21.15.83:0/3337909708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0700ff1f0 0x7fb0700ff670 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 --2- 172.21.15.83:0/3337909708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb070102780 0x7fb070102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 -- 172.21.15.83:0/3337909708 >> 172.21.15.83:0/3337909708 conn(0x7fb0700fadb0 msgr2=0x7fb0700fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 -- 172.21.15.83:0/3337909708 shutdown_connections 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 -- 172.21.15.83:0/3337909708 wait complete. 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 Processor -- start 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 -- start start 2024-03-11T19:55:35.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.777+0000 7fb0748ae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0700ff1f0 0x7fb0701973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb0748ae640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0700ffbb0 0x7fb0701978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb0748ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070102780 0x7fb07019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb0748ae640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07019ee90 con 0x7fb070102780 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb0748ae640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07019f000 con 0x7fb0700ff1f0 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb0748ae640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07019f1c0 con 0x7fb0700ffbb0 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb06ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070102780 0x7fb07019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb06ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070102780 0x7fb07019e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43774/0 (socket says 172.21.15.83:43774) 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb06ed9d640 1 -- 172.21.15.83:0/700996016 learned_addr learned my addr 172.21.15.83:0/700996016 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb06ed9d640 1 -- 172.21.15.83:0/700996016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0700ffbb0 msgr2=0x7fb0701978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb06ed9d640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0700ffbb0 0x7fb0701978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb06ed9d640 1 -- 172.21.15.83:0/700996016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0700ff1f0 msgr2=0x7fb0701973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb06ed9d640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0700ff1f0 0x7fb0701973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb06ed9d640 1 -- 172.21.15.83:0/700996016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058009560 con 0x7fb070102780 2024-03-11T19:55:35.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb06ed9d640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070102780 0x7fb07019e950 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb06800b970 tx=0x7fb06800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:35.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb0677fe640 1 -- 172.21.15.83:0/700996016 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06800cb40 con 0x7fb070102780 2024-03-11T19:55:35.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb0748ae640 1 -- 172.21.15.83:0/700996016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07019f4a0 con 0x7fb070102780 2024-03-11T19:55:35.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb0677fe640 1 -- 172.21.15.83:0/700996016 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb06800cca0 con 0x7fb070102780 2024-03-11T19:55:35.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb0677fe640 1 -- 172.21.15.83:0/700996016 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0680077d0 con 0x7fb070102780 2024-03-11T19:55:35.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.778+0000 7fb0748ae640 1 -- 172.21.15.83:0/700996016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07019fa80 con 0x7fb070102780 2024-03-11T19:55:35.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.779+0000 7fb0748ae640 1 -- 172.21.15.83:0/700996016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0701086b0 con 0x7fb070102780 2024-03-11T19:55:35.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.780+0000 7fb0677fe640 1 -- 172.21.15.83:0/700996016 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fb068014440 con 0x7fb070102780 2024-03-11T19:55:35.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.781+0000 7fb0677fe640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb05c0a1a30 0x7fb05c0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:35.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.781+0000 7fb06e59c640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb05c0a1a30 0x7fb05c0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:35.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.781+0000 7fb0677fe640 1 -- 172.21.15.83:0/700996016 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fb0680ccc30 con 0x7fb070102780 2024-03-11T19:55:35.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.781+0000 7fb06e59c640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb05c0a1a30 0x7fb05c0a3ef0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb0700694f0 tx=0x7fb060006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:35.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.786+0000 7fb0677fe640 1 -- 172.21.15.83:0/700996016 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb068095ef0 con 0x7fb070102780 2024-03-11T19:55:35.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:35 smithi083 ceph-mon[31864]: pgmap v840: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-03-11T19:55:35.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:35 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/1063881870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:55:35.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:35 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2852945042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:55:35.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:35 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2878950315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:55:35.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.957+0000 7fb0748ae640 1 -- 172.21.15.83:0/700996016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb07010cca0 con 0x7fb070102780 2024-03-11T19:55:35.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.958+0000 7fb0677fe640 1 -- 172.21.15.83:0/700996016 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb07010cca0 con 0x7fb070102780 2024-03-11T19:55:35.962 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-03-11T19:55:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 -- 172.21.15.83:0/700996016 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb05c0a1a30 msgr2=0x7fb05c0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb05c0a1a30 0x7fb05c0a3ef0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb0700694f0 tx=0x7fb060006c90 comp rx=0 tx=0).stop 2024-03-11T19:55:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 -- 172.21.15.83:0/700996016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070102780 msgr2=0x7fb07019e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070102780 0x7fb07019e950 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb06800b970 tx=0x7fb06800be30 comp rx=0 tx=0).stop 2024-03-11T19:55:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 -- 172.21.15.83:0/700996016 shutdown_connections 2024-03-11T19:55:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fb05c0a1a30 0x7fb05c0a3ef0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb070102780 0x7fb07019e950 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0700ffbb0 0x7fb0701978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 --2- 172.21.15.83:0/700996016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0700ff1f0 0x7fb0701973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 -- 172.21.15.83:0/700996016 >> 172.21.15.83:0/700996016 conn(0x7fb0700fadb0 msgr2=0x7fb070105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 -- 172.21.15.83:0/700996016 shutdown_connections 2024-03-11T19:55:35.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:35.960+0000 7fb0748ae640 1 -- 172.21.15.83:0/700996016 wait complete. 2024-03-11T19:55:35.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-11T19:55:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.074+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/1086190653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5b4104550 msgr2=0x7fe5b4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.074+0000 7fe5ba4f4640 1 --2- 172.21.15.83:0/1086190653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5b4104550 0x7fe5b4106940 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009900 tx=0x7fe5a802bc30 comp rx=0 tx=0).stop 2024-03-11T19:55:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.074+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/1086190653 shutdown_connections 2024-03-11T19:55:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.074+0000 7fe5ba4f4640 1 --2- 172.21.15.83:0/1086190653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5b4106e80 0x7fe5b41092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.074+0000 7fe5ba4f4640 1 --2- 172.21.15.83:0/1086190653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5b4104550 0x7fe5b4106940 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.074+0000 7fe5ba4f4640 1 --2- 172.21.15.83:0/1086190653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b4101c20 0x7fe5b4104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.074+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/1086190653 >> 172.21.15.83:0/1086190653 conn(0x7fe5b40fb630 msgr2=0x7fe5b40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.074+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/1086190653 shutdown_connections 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.074+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/1086190653 wait complete. 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.075+0000 7fe5ba4f4640 1 Processor -- start 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.075+0000 7fe5ba4f4640 1 -- start start 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.075+0000 7fe5ba4f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5b4101c20 0x7fe5b41972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.075+0000 7fe5ba4f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b4104550 0x7fe5b4197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.075+0000 7fe5ba4f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5b4106e80 0x7fe5b419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.075+0000 7fe5ba4f4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b419ede0 con 0x7fe5b4101c20 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.075+0000 7fe5ba4f4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b419ef50 con 0x7fe5b4106e80 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.075+0000 7fe5ba4f4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b419f110 con 0x7fe5b4104550 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.075+0000 7fe5b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5b4101c20 0x7fe5b41972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.075+0000 7fe5b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5b4101c20 0x7fe5b41972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43784/0 (socket says 172.21.15.83:43784) 2024-03-11T19:55:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.075+0000 7fe5b3fff640 1 -- 172.21.15.83:0/3243176095 learned_addr learned my addr 172.21.15.83:0/3243176095 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b37fe640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b4104550 0x7fe5b4197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b8a91640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5b4106e80 0x7fe5b419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b3fff640 1 -- 172.21.15.83:0/3243176095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b4104550 msgr2=0x7fe5b4197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b3fff640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b4104550 0x7fe5b4197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b3fff640 1 -- 172.21.15.83:0/3243176095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5b4106e80 msgr2=0x7fe5b419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b3fff640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5b4106e80 0x7fe5b419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b3fff640 1 -- 172.21.15.83:0/3243176095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a8009560 con 0x7fe5b4101c20 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b37fe640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b4104550 0x7fe5b4197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b3fff640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5b4101c20 0x7fe5b41972f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400b4a0 tx=0x7fe5a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b17fa640 1 -- 172.21.15.83:0/3243176095 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4010610 con 0x7fe5b4101c20 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b17fa640 1 -- 172.21.15.83:0/3243176095 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5a4010c50 con 0x7fe5b4101c20 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/3243176095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b419f3f0 con 0x7fe5b4101c20 2024-03-11T19:55:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5b17fa640 1 -- 172.21.15.83:0/3243176095 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4017e20 con 0x7fe5b4101c20 2024-03-11T19:55:36.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.076+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/3243176095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b419f950 con 0x7fe5b4101c20 2024-03-11T19:55:36.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.077+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/3243176095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b40fd120 con 0x7fe5b4101c20 2024-03-11T19:55:36.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.078+0000 7fe5b17fa640 1 -- 172.21.15.83:0/3243176095 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fe5a4017480 con 0x7fe5b4101c20 2024-03-11T19:55:36.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.079+0000 7fe5b17fa640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe59c0a1ab0 0x7fe59c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:36.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.079+0000 7fe5b37fe640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe59c0a1ab0 0x7fe59c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:36.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.079+0000 7fe5b17fa640 1 -- 172.21.15.83:0/3243176095 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fe5a40cce60 con 0x7fe5b4101c20 2024-03-11T19:55:36.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.079+0000 7fe5b37fe640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe59c0a1ab0 0x7fe59c0a3f70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8006010 tx=0x7fe5a8005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:36.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.082+0000 7fe5b17fa640 1 -- 172.21.15.83:0/3243176095 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe5a4096120 con 0x7fe5b4101c20 2024-03-11T19:55:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.268+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/3243176095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe5b410cbf0 con 0x7fe5b4101c20 2024-03-11T19:55:36.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.268+0000 7fe5b17fa640 1 -- 172.21.15.83:0/3243176095 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+591 (secure 0 0 0) 0x7fe5a401e260 con 0x7fe5b4101c20 2024-03-11T19:55:36.272 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1699838714 2024-03-12T19:30:08.427867+0000 2024-03-11T19:55:36.272 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/2543145311 2024-03-12T19:30:08.427867+0000 2024-03-11T19:55:36.272 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1376534596 2024-03-12T19:30:08.427867+0000 2024-03-11T19:55:36.272 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/4058944299 2024-03-12T19:30:08.427867+0000 2024-03-11T19:55:36.272 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/2370098752 2024-03-12T19:29:41.798770+0000 2024-03-11T19:55:36.272 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/2370098752 2024-03-12T19:29:41.798770+0000 2024-03-11T19:55:36.272 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1957272663 2024-03-12T19:29:41.798770+0000 2024-03-11T19:55:36.272 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/407682476 2024-03-12T19:29:41.798770+0000 2024-03-11T19:55:36.272 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3034283770 2024-03-12T19:29:41.798770+0000 2024-03-11T19:55:36.273 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/4058944299 2024-03-12T19:30:08.427867+0000 2024-03-11T19:55:36.273 INFO:teuthology.orchestra.run.smithi083.stderr:listed 10 entries 2024-03-11T19:55:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.270+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/3243176095 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe59c0a1ab0 msgr2=0x7fe59c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.270+0000 7fe5ba4f4640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe59c0a1ab0 0x7fe59c0a3f70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8006010 tx=0x7fe5a8005be0 comp rx=0 tx=0).stop 2024-03-11T19:55:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.270+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/3243176095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5b4101c20 msgr2=0x7fe5b41972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.270+0000 7fe5ba4f4640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5b4101c20 0x7fe5b41972f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400b4a0 tx=0x7fe5a400b960 comp rx=0 tx=0).stop 2024-03-11T19:55:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.271+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/3243176095 shutdown_connections 2024-03-11T19:55:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.271+0000 7fe5ba4f4640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fe59c0a1ab0 0x7fe59c0a3f70 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:36.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.271+0000 7fe5ba4f4640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5b4106e80 0x7fe5b419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.271+0000 7fe5ba4f4640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b4104550 0x7fe5b4197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.271+0000 7fe5ba4f4640 1 --2- 172.21.15.83:0/3243176095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5b4101c20 0x7fe5b41972f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.271+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/3243176095 >> 172.21.15.83:0/3243176095 conn(0x7fe5b40fb630 msgr2=0x7fe5b41026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.271+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/3243176095 shutdown_connections 2024-03-11T19:55:36.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.271+0000 7fe5ba4f4640 1 -- 172.21.15.83:0/3243176095 wait complete. 2024-03-11T19:55:36.285 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-11T19:55:36.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:55:36.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /tmp/cephfs 2024-03-11T19:55:36.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:35 smithi137 ceph-mon[36220]: pgmap v840: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-03-11T19:55:36.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:35 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/1063881870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:55:36.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:35 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2852945042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:55:36.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:35 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2878950315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:55:36.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:35 smithi204 ceph-mon[36739]: pgmap v840: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-03-11T19:55:36.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:35 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/1063881870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:55:36.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:35 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2852945042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-11T19:55:36.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:35 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2878950315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-11T19:55:36.483 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-03-11T19:55:36.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:55:36.483 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-03-11T19:55:36.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:55:36.543 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-03-11T19:55:36.587 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-03-11T19:55:36.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:55:36.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-03-11T19:55:36.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T19:55:36.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-03-11T19:55:36.773 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-11T19:55:36.784 INFO:tasks.cephadm:Teardown begin 2024-03-11T19:55:36.784 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-11T19:55:36.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:55:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.989+0000 7f018709d640 1 -- 172.21.15.83:0/1638164267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0180101290 msgr2=0x7f0180101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:36.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.989+0000 7f018709d640 1 --2- 172.21.15.83:0/1638164267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0180101290 0x7f0180101710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f017000d910 tx=0x7f017002fe00 comp rx=0 tx=0).stop 2024-03-11T19:55:37.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.995+0000 7f018709d640 1 -- 172.21.15.83:0/1638164267 shutdown_connections 2024-03-11T19:55:37.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.995+0000 7f018709d640 1 --2- 172.21.15.83:0/1638164267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0180101c50 0x7f01801084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.995+0000 7f018709d640 1 --2- 172.21.15.83:0/1638164267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0180101290 0x7f0180101710 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.995+0000 7f018709d640 1 --2- 172.21.15.83:0/1638164267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01801000a0 0x7f01801004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:36.995+0000 7f018709d640 1 -- 172.21.15.83:0/1638164267 >> 172.21.15.83:0/1638164267 conn(0x7f01800fb610 msgr2=0x7f01800fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:37.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.001+0000 7f018709d640 1 -- 172.21.15.83:0/1638164267 shutdown_connections 2024-03-11T19:55:37.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.002+0000 7f018709d640 1 -- 172.21.15.83:0/1638164267 wait complete. 2024-03-11T19:55:37.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.002+0000 7f018709d640 1 Processor -- start 2024-03-11T19:55:37.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.005+0000 7f018709d640 1 -- start start 2024-03-11T19:55:37.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.005+0000 7f018709d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01801000a0 0x7f01800725d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.005+0000 7f018709d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0180101290 0x7f0180072b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.006+0000 7f018709d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0180101c50 0x7f0180070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.006+0000 7f018709d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0180071180 con 0x7f0180101c50 2024-03-11T19:55:37.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.006+0000 7f018709d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01800712f0 con 0x7f0180101290 2024-03-11T19:55:37.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.006+0000 7f018709d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01800714b0 con 0x7f01801000a0 2024-03-11T19:55:37.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.010+0000 7f017ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0180101290 0x7f0180072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.010+0000 7f0184e39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01801000a0 0x7f01800725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.010+0000 7f018563a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0180101c50 0x7f0180070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.010+0000 7f018563a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0180101c50 0x7f0180070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43794/0 (socket says 172.21.15.83:43794) 2024-03-11T19:55:37.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.010+0000 7f018563a640 1 -- 172.21.15.83:0/3142341736 learned_addr learned my addr 172.21.15.83:0/3142341736 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:37.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.010+0000 7f018563a640 1 -- 172.21.15.83:0/3142341736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01801000a0 msgr2=0x7f01800725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.010+0000 7f018563a640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01801000a0 0x7f01800725d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.010+0000 7f018563a640 1 -- 172.21.15.83:0/3142341736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0180101290 msgr2=0x7f0180072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.010+0000 7f018563a640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0180101290 0x7f0180072b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.010+0000 7f018563a640 1 -- 172.21.15.83:0/3142341736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017000d570 con 0x7f0180101c50 2024-03-11T19:55:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.011+0000 7f018563a640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0180101c50 0x7f0180070c10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f017400b3f0 tx=0x7f017400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.011+0000 7f017dffb640 1 -- 172.21.15.83:0/3142341736 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0174011830 con 0x7f0180101c50 2024-03-11T19:55:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.011+0000 7f017dffb640 1 -- 172.21.15.83:0/3142341736 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0174011e70 con 0x7f0180101c50 2024-03-11T19:55:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.011+0000 7f017dffb640 1 -- 172.21.15.83:0/3142341736 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400f2e0 con 0x7f0180101c50 2024-03-11T19:55:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.011+0000 7f018709d640 1 -- 172.21.15.83:0/3142341736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0180071790 con 0x7f0180101c50 2024-03-11T19:55:37.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.012+0000 7f018709d640 1 -- 172.21.15.83:0/3142341736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01801a01a0 con 0x7f0180101c50 2024-03-11T19:55:37.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.015+0000 7f017dffb640 1 -- 172.21.15.83:0/3142341736 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f0174010680 con 0x7f0180101c50 2024-03-11T19:55:37.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.016+0000 7f017dffb640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f01680a1bb0 0x7f01680a4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.016+0000 7f017dffb640 1 -- 172.21.15.83:0/3142341736 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7f0174099880 con 0x7f0180101c50 2024-03-11T19:55:37.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.017+0000 7f018709d640 1 -- 172.21.15.83:0/3142341736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0180067f70 con 0x7f0180101c50 2024-03-11T19:55:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.026+0000 7f0184e39640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f01680a1bb0 0x7f01680a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.028+0000 7f0184e39640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f01680a1bb0 0x7f01680a4070 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f01801010f0 tx=0x7f01780072d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.030+0000 7f017dffb640 1 -- 172.21.15.83:0/3142341736 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0174095420 con 0x7f0180101c50 2024-03-11T19:55:37.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:36 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/700996016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:55:37.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:36 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3243176095' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:55:37.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.371+0000 7f018709d640 1 -- 172.21.15.83:0/3142341736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f018006b5b0 con 0x7f0180101c50 2024-03-11T19:55:37.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.372+0000 7f017dffb640 1 -- 172.21.15.83:0/3142341736 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11356 (secure 0 0 0) 0x7f0174015030 con 0x7f0180101c50 2024-03-11T19:55:37.374 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:55:37.374 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:54:52.016845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:55:37.375 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 36 2024-03-11T19:55:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 -- 172.21.15.83:0/3142341736 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f01680a1bb0 msgr2=0x7f01680a4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f01680a1bb0 0x7f01680a4070 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f01801010f0 tx=0x7f01780072d0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 -- 172.21.15.83:0/3142341736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0180101c50 msgr2=0x7f0180070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0180101c50 0x7f0180070c10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f017400b3f0 tx=0x7f017400b8b0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 -- 172.21.15.83:0/3142341736 shutdown_connections 2024-03-11T19:55:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f01680a1bb0 0x7f01680a4070 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0180101c50 0x7f0180070c10 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0180101290 0x7f0180072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 --2- 172.21.15.83:0/3142341736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01801000a0 0x7f01800725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 -- 172.21.15.83:0/3142341736 >> 172.21.15.83:0/3142341736 conn(0x7f01800fb610 msgr2=0x7f0180104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:37.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 -- 172.21.15.83:0/3142341736 shutdown_connections 2024-03-11T19:55:37.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.374+0000 7f018709d640 1 -- 172.21.15.83:0/3142341736 wait complete. 2024-03-11T19:55:37.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T19:55:37.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:36 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/700996016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:55:37.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:36 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3243176095' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:55:37.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:36 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/700996016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-11T19:55:37.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:36 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3243176095' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-11T19:55:37.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.489+0000 7fd46710d640 1 -- 172.21.15.83:0/336993009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd460101c50 msgr2=0x7fd4601084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.489+0000 7fd46710d640 1 --2- 172.21.15.83:0/336993009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd460101c50 0x7fd4601084e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd458009ca0 tx=0x7fd45802c0f0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.489+0000 7fd46710d640 1 -- 172.21.15.83:0/336993009 shutdown_connections 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.489+0000 7fd46710d640 1 --2- 172.21.15.83:0/336993009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd460101c50 0x7fd4601084e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.489+0000 7fd46710d640 1 --2- 172.21.15.83:0/336993009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd460101290 0x7fd460101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.489+0000 7fd46710d640 1 --2- 172.21.15.83:0/336993009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4601000a0 0x7fd4601004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd46710d640 1 -- 172.21.15.83:0/336993009 >> 172.21.15.83:0/336993009 conn(0x7fd4600fb610 msgr2=0x7fd4600fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd46710d640 1 -- 172.21.15.83:0/336993009 shutdown_connections 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd46710d640 1 -- 172.21.15.83:0/336993009 wait complete. 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd46710d640 1 Processor -- start 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd46710d640 1 -- start start 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd46710d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4601000a0 0x7fd460068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd46710d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd460101290 0x7fd46019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd46710d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd460101c50 0x7fd46019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd46710d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46019ed50 con 0x7fd4601000a0 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd46710d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46019eec0 con 0x7fd460101290 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd46710d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46019f080 con 0x7fd460101c50 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd464ea9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4601000a0 0x7fd460068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd464ea9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4601000a0 0x7fd460068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43818/0 (socket says 172.21.15.83:43818) 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd464ea9640 1 -- 172.21.15.83:0/2934772216 learned_addr learned my addr 172.21.15.83:0/2934772216 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd45ffff640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd460101290 0x7fd46019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.490+0000 7fd4656aa640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd460101c50 0x7fd46019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd464ea9640 1 -- 172.21.15.83:0/2934772216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd460101c50 msgr2=0x7fd46019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd464ea9640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd460101c50 0x7fd46019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd464ea9640 1 -- 172.21.15.83:0/2934772216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd460101290 msgr2=0x7fd46019bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd464ea9640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd460101290 0x7fd46019bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd464ea9640 1 -- 172.21.15.83:0/2934772216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd458009900 con 0x7fd4601000a0 2024-03-11T19:55:37.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd4656aa640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd460101c50 0x7fd46019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:55:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd464ea9640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4601000a0 0x7fd460068e50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e9c0 tx=0x7fd45000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd45dffb640 1 -- 172.21.15.83:0/2934772216 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4500096e0 con 0x7fd4601000a0 2024-03-11T19:55:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd46710d640 1 -- 172.21.15.83:0/2934772216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46019f300 con 0x7fd4601000a0 2024-03-11T19:55:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd45dffb640 1 -- 172.21.15.83:0/2934772216 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd45000cd80 con 0x7fd4601000a0 2024-03-11T19:55:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd45dffb640 1 -- 172.21.15.83:0/2934772216 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450010430 con 0x7fd4601000a0 2024-03-11T19:55:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.491+0000 7fd46710d640 1 -- 172.21.15.83:0/2934772216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46019f810 con 0x7fd4601000a0 2024-03-11T19:55:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.492+0000 7fd46710d640 1 -- 172.21.15.83:0/2934772216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd460107360 con 0x7fd4601000a0 2024-03-11T19:55:37.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.493+0000 7fd45dffb640 1 -- 172.21.15.83:0/2934772216 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fd450003680 con 0x7fd4601000a0 2024-03-11T19:55:37.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.494+0000 7fd45dffb640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd44c0a1ab0 0x7fd44c0a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.494+0000 7fd45ffff640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd44c0a1ab0 0x7fd44c0a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.494+0000 7fd45dffb640 1 -- 172.21.15.83:0/2934772216 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fd450014070 con 0x7fd4601000a0 2024-03-11T19:55:37.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.494+0000 7fd45ffff640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd44c0a1ab0 0x7fd44c0a3f70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd460068bc0 tx=0x7fd448005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:37.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.498+0000 7fd45dffb640 1 -- 172.21.15.83:0/2934772216 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd4500969e0 con 0x7fd4601000a0 2024-03-11T19:55:37.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.711+0000 7fd46710d640 1 -- 172.21.15.83:0/2934772216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd46010bcf0 con 0x7fd4601000a0 2024-03-11T19:55:37.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.711+0000 7fd45dffb640 1 -- 172.21.15.83:0/2934772216 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11356 (secure 0 0 0) 0x7fd45009a0a0 con 0x7fd4601000a0 2024-03-11T19:55:37.714 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:55:37.714 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/4213430163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":4213430163},{"type":"v1","addr":"172.21.15.83:6835","nonce":4213430163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1285695998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1285695998},{"type":"v1","addr":"172.21.15.83:6837","nonce":1285695998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1413247052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1413247052},{"type":"v1","addr":"172.21.15.83:6839","nonce":1413247052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1703400254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1703400254},{"type":"v1","addr":"172.21.15.83:6841","nonce":1703400254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1242693918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1242693918},{"type":"v1","addr":"172.21.15.204:6833","nonce":1242693918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/1480144108","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":1480144108},{"type":"v1","addr":"172.21.15.137:6833","nonce":1480144108}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-11T19:37:04.916864+0000","modified":"2024-03-11T19:54:52.016845+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24527,"mds_1":24502,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24466":{"gid":24466,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2604685946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2604685946},{"type":"v1","addr":"172.21.15.204:6835","nonce":2604685946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24470":{"gid":24470,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1924725912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1924725912},{"type":"v1","addr":"172.21.15.137:6835","nonce":1924725912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/2654992874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":2654992874},{"type":"v1","addr":"172.21.15.204:6837","nonce":2654992874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3499358906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3499358906},{"type":"v1","addr":"172.21.15.137:6837","nonce":3499358906}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.204:6839/862560872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":862560872},{"type":"v1","addr":"172.21.15.204:6839","nonce":862560872}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/709970923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":709970923},{"type":"v1","addr":"172.21.15.137:6839","nonce":709970923}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T19:55:37.714 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 36 2024-03-11T19:55:37.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.713+0000 7fd46710d640 1 -- 172.21.15.83:0/2934772216 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd44c0a1ab0 msgr2=0x7fd44c0a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.713+0000 7fd46710d640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd44c0a1ab0 0x7fd44c0a3f70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd460068bc0 tx=0x7fd448005f90 comp rx=0 tx=0).stop 2024-03-11T19:55:37.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.713+0000 7fd46710d640 1 -- 172.21.15.83:0/2934772216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4601000a0 msgr2=0x7fd460068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.713+0000 7fd46710d640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4601000a0 0x7fd460068e50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e9c0 tx=0x7fd45000ee80 comp rx=0 tx=0).stop 2024-03-11T19:55:37.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.713+0000 7fd46710d640 1 -- 172.21.15.83:0/2934772216 shutdown_connections 2024-03-11T19:55:37.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.713+0000 7fd46710d640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fd44c0a1ab0 0x7fd44c0a3f70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.713+0000 7fd46710d640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd460101c50 0x7fd46019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.713+0000 7fd46710d640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd460101290 0x7fd46019bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.713+0000 7fd46710d640 1 --2- 172.21.15.83:0/2934772216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4601000a0 0x7fd460068e50 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.713+0000 7fd46710d640 1 -- 172.21.15.83:0/2934772216 >> 172.21.15.83:0/2934772216 conn(0x7fd4600fb610 msgr2=0x7fd460104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:37.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.713+0000 7fd46710d640 1 -- 172.21.15.83:0/2934772216 shutdown_connections 2024-03-11T19:55:37.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.714+0000 7fd46710d640 1 -- 172.21.15.83:0/2934772216 wait complete. 2024-03-11T19:55:37.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T19:55:37.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.827+0000 7fdf2b64d640 1 -- 172.21.15.83:0/767354723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf24102df0 msgr2=0x7fdf24109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.827+0000 7fdf2b64d640 1 --2- 172.21.15.83:0/767354723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf24102df0 0x7fdf24109680 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009ca0 tx=0x7fdf1c02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.827+0000 7fdf2b64d640 1 -- 172.21.15.83:0/767354723 shutdown_connections 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.827+0000 7fdf2b64d640 1 --2- 172.21.15.83:0/767354723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf24102df0 0x7fdf24109680 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.827+0000 7fdf2b64d640 1 --2- 172.21.15.83:0/767354723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf24102430 0x7fdf241028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.827+0000 7fdf2b64d640 1 --2- 172.21.15.83:0/767354723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf240ff460 0x7fdf240ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.827+0000 7fdf2b64d640 1 -- 172.21.15.83:0/767354723 >> 172.21.15.83:0/767354723 conn(0x7fdf240fadb0 msgr2=0x7fdf240fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.827+0000 7fdf2b64d640 1 -- 172.21.15.83:0/767354723 shutdown_connections 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.827+0000 7fdf2b64d640 1 -- 172.21.15.83:0/767354723 wait complete. 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf2b64d640 1 Processor -- start 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf2b64d640 1 -- start start 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf2b64d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf240ff460 0x7fdf24197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf2b64d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf24102430 0x7fdf24197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf2b64d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf24102df0 0x7fdf2419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf2b64d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2419ed30 con 0x7fdf24102430 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf2b64d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2419eea0 con 0x7fdf24102df0 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf2b64d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2419f060 con 0x7fdf240ff460 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf28be8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf24102430 0x7fdf24197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf28be8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf24102430 0x7fdf24197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43840/0 (socket says 172.21.15.83:43840) 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf28be8640 1 -- 172.21.15.83:0/904215645 learned_addr learned my addr 172.21.15.83:0/904215645 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf293e9640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf240ff460 0x7fdf24197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf29bea640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf24102df0 0x7fdf2419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf28be8640 1 -- 172.21.15.83:0/904215645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf240ff460 msgr2=0x7fdf24197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf28be8640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf240ff460 0x7fdf24197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf28be8640 1 -- 172.21.15.83:0/904215645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf24102df0 msgr2=0x7fdf2419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:37.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf28be8640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf24102df0 0x7fdf2419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf28be8640 1 -- 172.21.15.83:0/904215645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c009900 con 0x7fdf24102430 2024-03-11T19:55:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf293e9640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf240ff460 0x7fdf24197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:55:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf29bea640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf24102df0 0x7fdf2419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:55:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf28be8640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf24102430 0x7fdf24197780 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000e9f0 tx=0x7fdf2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf1a7fc640 1 -- 172.21.15.83:0/904215645 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20009720 con 0x7fdf24102430 2024-03-11T19:55:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf1a7fc640 1 -- 172.21.15.83:0/904215645 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf2000cd80 con 0x7fdf24102430 2024-03-11T19:55:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf1a7fc640 1 -- 172.21.15.83:0/904215645 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20010690 con 0x7fdf24102430 2024-03-11T19:55:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf2b64d640 1 -- 172.21.15.83:0/904215645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2419f340 con 0x7fdf24102430 2024-03-11T19:55:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.828+0000 7fdf2b64d640 1 -- 172.21.15.83:0/904215645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2419f800 con 0x7fdf24102430 2024-03-11T19:55:37.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.829+0000 7fdf2b64d640 1 -- 172.21.15.83:0/904215645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf24108560 con 0x7fdf24102430 2024-03-11T19:55:37.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.830+0000 7fdf1a7fc640 1 -- 172.21.15.83:0/904215645 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fdf200046c0 con 0x7fdf24102430 2024-03-11T19:55:37.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.831+0000 7fdf1a7fc640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdf100a1a30 0x7fdf100a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:37.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.831+0000 7fdf293e9640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdf100a1a30 0x7fdf100a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:37.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.831+0000 7fdf1a7fc640 1 -- 172.21.15.83:0/904215645 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fdf20014070 con 0x7fdf24102430 2024-03-11T19:55:37.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.831+0000 7fdf293e9640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdf100a1a30 0x7fdf100a3ef0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdf24069650 tx=0x7fdf14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:37.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:37.835+0000 7fdf1a7fc640 1 -- 172.21.15.83:0/904215645 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf20096ad0 con 0x7fdf24102430 2024-03-11T19:55:38.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.013+0000 7fdf2b64d640 1 -- 172.21.15.83:0/904215645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf2410cbc0 con 0x7fdf24102430 2024-03-11T19:55:38.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.015+0000 7fdf1a7fc640 1 -- 172.21.15.83:0/904215645 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22562 (secure 0 0 0) 0x7fdf2009a190 con 0x7fdf24102430 2024-03-11T19:55:38.018 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-03-11T19:55:38.018 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":87,"fsid":"839f93c8-dfdd-11ee-95c8-87774f69a715","created":"2024-03-11T19:29:04.070278+0000","modified":"2024-03-11T19:38:10.141528+0000","last_up_change":"2024-03-11T19:36:17.856833+0000","last_in_change":"2024-03-11T19:35:57.434640+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T19:32:34.761694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T19:37:03.034982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T19:37:04.125352+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4626dde-b940-4692-9e2d-2a2f3f59a70f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6803","nonce":3703691285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6805","nonce":3703691285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6809","nonce":3703691285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":3703691285},{"type":"v1","addr":"172.21.15.83:6807","nonce":3703691285}]},"public_addr":"172.21.15.83:6803/3703691285","cluster_addr":"172.21.15.83:6805/3703691285","heartbeat_back_addr":"172.21.15.83:6809/3703691285","heartbeat_front_addr":"172.21.15.83:6807/3703691285","state":["exists","up"]},{"osd":1,"uuid":"bb07056c-aceb-4fd5-8b42-c1e33fede2b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6801","nonce":415302205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6803","nonce":415302205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6807","nonce":415302205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":415302205},{"type":"v1","addr":"172.21.15.137:6805","nonce":415302205}]},"public_addr":"172.21.15.137:6801/415302205","cluster_addr":"172.21.15.137:6803/415302205","heartbeat_back_addr":"172.21.15.137:6807/415302205","heartbeat_front_addr":"172.21.15.137:6805/415302205","state":["exists","up"]},{"osd":2,"uuid":"c0f4f2c4-3f48-4c0f-a438-de6d8f151438","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6801","nonce":4285299727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6803","nonce":4285299727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6807","nonce":4285299727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4285299727},{"type":"v1","addr":"172.21.15.204:6805","nonce":4285299727}]},"public_addr":"172.21.15.204:6801/4285299727","cluster_addr":"172.21.15.204:6803/4285299727","heartbeat_back_addr":"172.21.15.204:6807/4285299727","heartbeat_front_addr":"172.21.15.204:6805/4285299727","state":["exists","up"]},{"osd":3,"uuid":"12c251e9-dcee-4d62-b731-9a63cd9815fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6811","nonce":780035155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6813","nonce":780035155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6817","nonce":780035155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":780035155},{"type":"v1","addr":"172.21.15.83:6815","nonce":780035155}]},"public_addr":"172.21.15.83:6811/780035155","cluster_addr":"172.21.15.83:6813/780035155","heartbeat_back_addr":"172.21.15.83:6817/780035155","heartbeat_front_addr":"172.21.15.83:6815/780035155","state":["exists","up"]},{"osd":4,"uuid":"438ba247-da85-4ac4-967f-3ded2137b07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6809","nonce":4100342703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6811","nonce":4100342703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6815","nonce":4100342703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":4100342703},{"type":"v1","addr":"172.21.15.137:6813","nonce":4100342703}]},"public_addr":"172.21.15.137:6809/4100342703","cluster_addr":"172.21.15.137:6811/4100342703","heartbeat_back_addr":"172.21.15.137:6815/4100342703","heartbeat_front_addr":"172.21.15.137:6813/4100342703","state":["exists","up"]},{"osd":5,"uuid":"c57d7878-189e-48d6-8d9a-a4b105dea801","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6809","nonce":2553953517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6811","nonce":2553953517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6815","nonce":2553953517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":2553953517},{"type":"v1","addr":"172.21.15.204:6813","nonce":2553953517}]},"public_addr":"172.21.15.204:6809/2553953517","cluster_addr":"172.21.15.204:6811/2553953517","heartbeat_back_addr":"172.21.15.204:6815/2553953517","heartbeat_front_addr":"172.21.15.204:6813/2553953517","state":["exists","up"]},{"osd":6,"uuid":"829db8a4-7f58-4770-8bd7-8cd258b2b1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6819","nonce":3426343907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6821","nonce":3426343907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6825","nonce":3426343907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":3426343907},{"type":"v1","addr":"172.21.15.83:6823","nonce":3426343907}]},"public_addr":"172.21.15.83:6819/3426343907","cluster_addr":"172.21.15.83:6821/3426343907","heartbeat_back_addr":"172.21.15.83:6825/3426343907","heartbeat_front_addr":"172.21.15.83:6823/3426343907","state":["exists","up"]},{"osd":7,"uuid":"2cf6f477-f5c5-4764-9110-98898779c9f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6817","nonce":2113619017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6819","nonce":2113619017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6823","nonce":2113619017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2113619017},{"type":"v1","addr":"172.21.15.137:6821","nonce":2113619017}]},"public_addr":"172.21.15.137:6817/2113619017","cluster_addr":"172.21.15.137:6819/2113619017","heartbeat_back_addr":"172.21.15.137:6823/2113619017","heartbeat_front_addr":"172.21.15.137:6821/2113619017","state":["exists","up"]},{"osd":8,"uuid":"9e808c31-b428-45c6-896d-42eba00a25f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6817","nonce":3550974508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6819","nonce":3550974508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6823","nonce":3550974508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":3550974508},{"type":"v1","addr":"172.21.15.204:6821","nonce":3550974508}]},"public_addr":"172.21.15.204:6817/3550974508","cluster_addr":"172.21.15.204:6819/3550974508","heartbeat_back_addr":"172.21.15.204:6823/3550974508","heartbeat_front_addr":"172.21.15.204:6821/3550974508","state":["exists","up"]},{"osd":9,"uuid":"78fabab9-c022-4e3a-a912-aab73696f4b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6827","nonce":2013526190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6829","nonce":2013526190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6833","nonce":2013526190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":2013526190},{"type":"v1","addr":"172.21.15.83:6831","nonce":2013526190}]},"public_addr":"172.21.15.83:6827/2013526190","cluster_addr":"172.21.15.83:6829/2013526190","heartbeat_back_addr":"172.21.15.83:6833/2013526190","heartbeat_front_addr":"172.21.15.83:6831/2013526190","state":["exists","up"]},{"osd":10,"uuid":"c7f76fae-ccd2-4b2b-8885-fb008d99ad9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6825","nonce":3042743423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6827","nonce":3042743423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6831","nonce":3042743423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3042743423},{"type":"v1","addr":"172.21.15.137:6829","nonce":3042743423}]},"public_addr":"172.21.15.137:6825/3042743423","cluster_addr":"172.21.15.137:6827/3042743423","heartbeat_back_addr":"172.21.15.137:6831/3042743423","heartbeat_front_addr":"172.21.15.137:6829/3042743423","state":["exists","up"]},{"osd":11,"uuid":"7778999c-58d4-487e-a8ae-eb317e078f58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6825","nonce":2753506819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6827","nonce":2753506819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6831","nonce":2753506819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2753506819},{"type":"v1","addr":"172.21.15.204:6829","nonce":2753506819}]},"public_addr":"172.21.15.204:6825/2753506819","cluster_addr":"172.21.15.204:6827/2753506819","heartbeat_back_addr":"172.21.15.204:6831/2753506819","heartbeat_front_addr":"172.21.15.204:6829/2753506819","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:31:42.412644+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:05.861582+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:30.017243+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:32:54.670495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:19.182941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:33:43.981244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:09.570585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:34.217298+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:34:59.022841+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:25.178723+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:35:50.335881+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T19:36:15.790449+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1699838714":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/2543145311":"2024-03-12T19:30:08.427867+0000","172.21.15.83:0/1376534596":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6801/4058944299":"2024-03-12T19:30:08.427867+0000","172.21.15.83:6800/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6801/2370098752":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/1957272663":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/407682476":"2024-03-12T19:29:41.798770+0000","172.21.15.83:0/3034283770":"2024-03-12T19:29:41.798770+0000","172.21.15.83:6800/4058944299":"2024-03-12T19:30:08.427867+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T19:55:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 -- 172.21.15.83:0/904215645 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdf100a1a30 msgr2=0x7fdf100a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:38.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdf100a1a30 0x7fdf100a3ef0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdf24069650 tx=0x7fdf14005f90 comp rx=0 tx=0).stop 2024-03-11T19:55:38.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 -- 172.21.15.83:0/904215645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf24102430 msgr2=0x7fdf24197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:38.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf24102430 0x7fdf24197780 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000e9f0 tx=0x7fdf2000edf0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 -- 172.21.15.83:0/904215645 shutdown_connections 2024-03-11T19:55:38.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fdf100a1a30 0x7fdf100a3ef0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf24102df0 0x7fdf2419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf24102430 0x7fdf24197780 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 --2- 172.21.15.83:0/904215645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf240ff460 0x7fdf24197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 -- 172.21.15.83:0/904215645 >> 172.21.15.83:0/904215645 conn(0x7fdf240fadb0 msgr2=0x7fdf24105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:38.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 -- 172.21.15.83:0/904215645 shutdown_connections 2024-03-11T19:55:38.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.017+0000 7fdf2b64d640 1 -- 172.21.15.83:0/904215645 wait complete. 2024-03-11T19:55:38.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:37 smithi083 ceph-mon[31864]: pgmap v841: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-03-11T19:55:38.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:37 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3142341736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:55:38.030 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:37 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2934772216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:55:38.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-11T19:55:38.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.131+0000 7fad79a1b640 1 -- 172.21.15.83:0/1552795502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad74102430 msgr2=0x7fad741028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:38.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.131+0000 7fad79a1b640 1 --2- 172.21.15.83:0/1552795502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad74102430 0x7fad741028b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009900 tx=0x7fad5c02bc30 comp rx=0 tx=0).stop 2024-03-11T19:55:38.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.131+0000 7fad79a1b640 1 -- 172.21.15.83:0/1552795502 shutdown_connections 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.131+0000 7fad79a1b640 1 --2- 172.21.15.83:0/1552795502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad74102df0 0x7fad74109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.131+0000 7fad79a1b640 1 --2- 172.21.15.83:0/1552795502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad74102430 0x7fad741028b0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.131+0000 7fad79a1b640 1 --2- 172.21.15.83:0/1552795502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad740ff460 0x7fad740ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.131+0000 7fad79a1b640 1 -- 172.21.15.83:0/1552795502 >> 172.21.15.83:0/1552795502 conn(0x7fad740fad90 msgr2=0x7fad740fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad79a1b640 1 -- 172.21.15.83:0/1552795502 shutdown_connections 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad79a1b640 1 -- 172.21.15.83:0/1552795502 wait complete. 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad79a1b640 1 Processor -- start 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad79a1b640 1 -- start start 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad79a1b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad740ff460 0x7fad74197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad79a1b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad74102430 0x7fad74197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad79a1b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad74102df0 0x7fad7419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad79a1b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7419ed10 con 0x7fad740ff460 2024-03-11T19:55:38.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad79a1b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7419ee80 con 0x7fad74102df0 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad79a1b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7419f040 con 0x7fad74102430 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad72ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad740ff460 0x7fad74197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.132+0000 7fad72ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad740ff460 0x7fad74197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43868/0 (socket says 172.21.15.83:43868) 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad72ffd640 1 -- 172.21.15.83:0/2748886054 learned_addr learned my addr 172.21.15.83:0/2748886054 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad727fc640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad74102430 0x7fad74197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad737fe640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad74102df0 0x7fad7419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad72ffd640 1 -- 172.21.15.83:0/2748886054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad74102430 msgr2=0x7fad74197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad72ffd640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad74102430 0x7fad74197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad72ffd640 1 -- 172.21.15.83:0/2748886054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad74102df0 msgr2=0x7fad7419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad72ffd640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad74102df0 0x7fad7419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad72ffd640 1 -- 172.21.15.83:0/2748886054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c009560 con 0x7fad740ff460 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad72ffd640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad740ff460 0x7fad74197220 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00eb70 tx=0x7fad6c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:38.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad78a19640 1 -- 172.21.15.83:0/2748886054 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c003b70 con 0x7fad740ff460 2024-03-11T19:55:38.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad78a19640 1 -- 172.21.15.83:0/2748886054 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad6c00ee70 con 0x7fad740ff460 2024-03-11T19:55:38.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad78a19640 1 -- 172.21.15.83:0/2748886054 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0107f0 con 0x7fad740ff460 2024-03-11T19:55:38.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad79a1b640 1 -- 172.21.15.83:0/2748886054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7419f320 con 0x7fad740ff460 2024-03-11T19:55:38.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.133+0000 7fad79a1b640 1 -- 172.21.15.83:0/2748886054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7419f800 con 0x7fad740ff460 2024-03-11T19:55:38.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.134+0000 7fad79a1b640 1 -- 172.21.15.83:0/2748886054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad74108560 con 0x7fad740ff460 2024-03-11T19:55:38.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.135+0000 7fad78a19640 1 -- 172.21.15.83:0/2748886054 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fad6c004000 con 0x7fad740ff460 2024-03-11T19:55:38.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.136+0000 7fad78a19640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad600a1ab0 0x7fad600a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:38.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.136+0000 7fad727fc640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad600a1ab0 0x7fad600a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:38.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.136+0000 7fad78a19640 1 -- 172.21.15.83:0/2748886054 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8140+0+0 (secure 0 0 0) 0x7fad6c014070 con 0x7fad740ff460 2024-03-11T19:55:38.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.136+0000 7fad727fc640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad600a1ab0 0x7fad600a3f70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009230 tx=0x7fad5c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:38.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.139+0000 7fad78a19640 1 -- 172.21.15.83:0/2748886054 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad6c096c20 con 0x7fad740ff460 2024-03-11T19:55:38.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.348+0000 7fad79a1b640 1 -- 172.21.15.83:0/2748886054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad7410cdb0 con 0x7fad740ff460 2024-03-11T19:55:38.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.350+0000 7fad78a19640 1 -- 172.21.15.83:0/2748886054 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7fad7410cdb0 con 0x7fad740ff460 2024-03-11T19:55:38.353 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-11T19:55:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 -- 172.21.15.83:0/2748886054 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad600a1ab0 msgr2=0x7fad600a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad600a1ab0 0x7fad600a3f70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009230 tx=0x7fad5c005fd0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 -- 172.21.15.83:0/2748886054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad740ff460 msgr2=0x7fad74197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad740ff460 0x7fad74197220 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00eb70 tx=0x7fad6c00c6a0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 -- 172.21.15.83:0/2748886054 shutdown_connections 2024-03-11T19:55:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fad600a1ab0 0x7fad600a3f70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad74102df0 0x7fad7419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad74102430 0x7fad74197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 --2- 172.21.15.83:0/2748886054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad740ff460 0x7fad74197220 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 -- 172.21.15.83:0/2748886054 >> 172.21.15.83:0/2748886054 conn(0x7fad740fad90 msgr2=0x7fad74105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:38.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 -- 172.21.15.83:0/2748886054 shutdown_connections 2024-03-11T19:55:38.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.353+0000 7fad79a1b640 1 -- 172.21.15.83:0/2748886054 wait complete. 2024-03-11T19:55:38.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-11T19:55:38.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:37 smithi137 ceph-mon[36220]: pgmap v841: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-03-11T19:55:38.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:37 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3142341736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:55:38.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:37 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2934772216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:55:38.397 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:37 smithi204 ceph-mon[36739]: pgmap v841: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-03-11T19:55:38.397 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:37 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3142341736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:55:38.397 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:37 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2934772216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-11T19:55:38.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.469+0000 7f670904d640 1 -- 172.21.15.83:0/2582947549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67040ffbc0 msgr2=0x7f6704109770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:38.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.469+0000 7f670904d640 1 --2- 172.21.15.83:0/2582947549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67040ffbc0 0x7f6704109770 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009ca0 tx=0x7f66fc02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.469+0000 7f670904d640 1 -- 172.21.15.83:0/2582947549 shutdown_connections 2024-03-11T19:55:38.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.469+0000 7f670904d640 1 --2- 172.21.15.83:0/2582947549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67040ffbc0 0x7f6704109770 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.469+0000 7f670904d640 1 --2- 172.21.15.83:0/2582947549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67040ff200 0x7f67040ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.469+0000 7f670904d640 1 --2- 172.21.15.83:0/2582947549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6704102790 0x7f6704102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.469+0000 7f670904d640 1 -- 172.21.15.83:0/2582947549 >> 172.21.15.83:0/2582947549 conn(0x7f67040fad80 msgr2=0x7f67040fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:38.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.470+0000 7f670904d640 1 -- 172.21.15.83:0/2582947549 shutdown_connections 2024-03-11T19:55:38.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.470+0000 7f670904d640 1 -- 172.21.15.83:0/2582947549 wait complete. 2024-03-11T19:55:38.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.470+0000 7f670904d640 1 Processor -- start 2024-03-11T19:55:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.470+0000 7f670904d640 1 -- start start 2024-03-11T19:55:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670904d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67040ff200 0x7f67041971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670904d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67040ffbc0 0x7f6704197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670904d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6704102790 0x7f670419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670904d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670419ece0 con 0x7f6704102790 2024-03-11T19:55:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670904d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670419ee50 con 0x7f67040ffbc0 2024-03-11T19:55:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670904d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670419f010 con 0x7f67040ff200 2024-03-11T19:55:38.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6704102790 0x7f670419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6704102790 0x7f670419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43888/0 (socket says 172.21.15.83:43888) 2024-03-11T19:55:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670359e640 1 -- 172.21.15.83:0/3047200172 learned_addr learned my addr 172.21.15.83:0/3047200172 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670259c640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67040ffbc0 0x7f6704197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f6702d9d640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67040ff200 0x7f67041971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670359e640 1 -- 172.21.15.83:0/3047200172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67040ff200 msgr2=0x7f67041971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670359e640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67040ff200 0x7f67041971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670359e640 1 -- 172.21.15.83:0/3047200172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67040ffbc0 msgr2=0x7f6704197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670359e640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67040ffbc0 0x7f6704197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670359e640 1 -- 172.21.15.83:0/3047200172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc009900 con 0x7f6704102790 2024-03-11T19:55:38.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670359e640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6704102790 0x7f670419e7a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f66fc037040 tx=0x7f66fc032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f66fbfff640 1 -- 172.21.15.83:0/3047200172 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc034c90 con 0x7f6704102790 2024-03-11T19:55:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670904d640 1 -- 172.21.15.83:0/3047200172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670419f290 con 0x7f6704102790 2024-03-11T19:55:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f66fbfff640 1 -- 172.21.15.83:0/3047200172 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66fc034df0 con 0x7f6704102790 2024-03-11T19:55:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f66fbfff640 1 -- 172.21.15.83:0/3047200172 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc004890 con 0x7f6704102790 2024-03-11T19:55:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.471+0000 7f670904d640 1 -- 172.21.15.83:0/3047200172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670419f740 con 0x7f6704102790 2024-03-11T19:55:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.472+0000 7f670904d640 1 -- 172.21.15.83:0/3047200172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67041085f0 con 0x7f6704102790 2024-03-11T19:55:38.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.473+0000 7f66fbfff640 1 -- 172.21.15.83:0/3047200172 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f66fc00a040 con 0x7f6704102790 2024-03-11T19:55:38.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.474+0000 7f66fbfff640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66f00a1a30 0x7f66f00a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:38.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.474+0000 7f6702d9d640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66f00a1a30 0x7f66f00a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:38.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.474+0000 7f66fbfff640 1 -- 172.21.15.83:0/3047200172 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8398+0+0 (secure 0 0 0) 0x7f66fc0f0210 con 0x7f6704102790 2024-03-11T19:55:38.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.475+0000 7f6702d9d640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66f00a1a30 0x7f66f00a3ef0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f67040694b0 tx=0x7f66f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:38.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.479+0000 7f66fbfff640 1 -- 172.21.15.83:0/3047200172 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f66fc0b9380 con 0x7f6704102790 2024-03-11T19:55:38.654 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c[67894]: ignoring --setuser ceph since I am not root 2024-03-11T19:55:38.655 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c[67894]: ignoring --setgroup ceph since I am not root 2024-03-11T19:55:38.655 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-mds[67898]: -- 172.21.15.204:0/818945638 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56391084e1a0 con 0x563910824c00 2024-03-11T19:55:38.655 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c[67894]: starting mds.c at 2024-03-11T19:55:38.655 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f[67585]: ignoring --setuser ceph since I am not root 2024-03-11T19:55:38.655 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f[67585]: ignoring --setgroup ceph since I am not root 2024-03-11T19:55:38.655 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-mds[67589]: -- 172.21.15.204:0/3938100142 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c2a75de1a0 con 0x55c2a75b5000 2024-03-11T19:55:38.655 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f[67585]: starting mds.f at 2024-03-11T19:55:38.656 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i[68220]: ignoring --setuser ceph since I am not root 2024-03-11T19:55:38.656 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i[68220]: ignoring --setgroup ceph since I am not root 2024-03-11T19:55:38.656 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-mds[68224]: -- 172.21.15.204:0/2971854795 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d5daa1a1a0 con 0x55d5da9f0800 2024-03-11T19:55:38.656 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i[68220]: starting mds.i at 2024-03-11T19:55:38.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:38.692+0000 7f670904d640 1 -- 172.21.15.83:0/3047200172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f670402ce40 con 0x7f6704102790 2024-03-11T19:55:38.816 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:38 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a[31860]: 2024-03-11T19:55:38.349+0000 7fab9bab7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T19:55:38.888 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h[66505]: ignoring --setuser ceph since I am not root 2024-03-11T19:55:38.888 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h[66505]: ignoring --setgroup ceph since I am not root 2024-03-11T19:55:38.889 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-mds[66509]: -- 172.21.15.137:0/3612435918 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56075793c1a0 con 0x560757912800 2024-03-11T19:55:38.889 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h[66505]: starting mds.h at 2024-03-11T19:55:38.889 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e[66197]: ignoring --setuser ceph since I am not root 2024-03-11T19:55:38.889 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e[66197]: ignoring --setgroup ceph since I am not root 2024-03-11T19:55:38.889 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-mds[66201]: -- 172.21.15.137:0/3180943978 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b2af25e1a0 con 0x55b2af234c00 2024-03-11T19:55:38.889 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e[66197]: starting mds.e at 2024-03-11T19:55:38.889 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-b[66824]: ignoring --setuser ceph since I am not root 2024-03-11T19:55:38.889 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-b[66824]: ignoring --setgroup ceph since I am not root 2024-03-11T19:55:38.890 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-mds[66828]: -- 172.21.15.137:0/2072855181 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5654fdd1a1a0 con 0x5654fdcf0c00 2024-03-11T19:55:38.890 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-b[66824]: starting mds.b at 2024-03-11T19:55:39.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:38 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/904215645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:55:39.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:38 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2748886054' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T19:55:39.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:38 smithi083 ceph-mon[31864]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-11T19:55:39.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:38 smithi083 ceph-mon[31864]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T19:55:39.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:38 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2748886054' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-11T19:55:39.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:38 smithi083 ceph-mon[31864]: osdmap e88: 12 total, 12 up, 12 in 2024-03-11T19:55:39.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:38 smithi083 ceph-mon[31864]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-11T19:55:39.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:38 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3047200172' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:55:39.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.355+0000 7f66fbfff640 1 -- 172.21.15.83:0/3047200172 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7f66fc0bca40 con 0x7f6704102790 2024-03-11T19:55:39.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 -- 172.21.15.83:0/3047200172 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66f00a1a30 msgr2=0x7f66f00a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66f00a1a30 0x7f66f00a3ef0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f67040694b0 tx=0x7f66f4005f90 comp rx=0 tx=0).stop 2024-03-11T19:55:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 -- 172.21.15.83:0/3047200172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6704102790 msgr2=0x7f670419e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6704102790 0x7f670419e7a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f66fc037040 tx=0x7f66fc032fc0 comp rx=0 tx=0).stop 2024-03-11T19:55:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 -- 172.21.15.83:0/3047200172 shutdown_connections 2024-03-11T19:55:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f66f00a1a30 0x7f66f00a3ef0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6704102790 0x7f670419e7a0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67040ffbc0 0x7f6704197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 --2- 172.21.15.83:0/3047200172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67040ff200 0x7f67041971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 -- 172.21.15.83:0/3047200172 >> 172.21.15.83:0/3047200172 conn(0x7f67040fad80 msgr2=0x7f67041057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 -- 172.21.15.83:0/3047200172 shutdown_connections 2024-03-11T19:55:39.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.358+0000 7f670904d640 1 -- 172.21.15.83:0/3047200172 wait complete. 2024-03-11T19:55:39.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-11T19:55:39.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/904215645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:55:39.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2748886054' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T19:55:39.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-mon[36220]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-11T19:55:39.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-mon[36220]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T19:55:39.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2748886054' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-11T19:55:39.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-mon[36220]: osdmap e88: 12 total, 12 up, 12 in 2024-03-11T19:55:39.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-mon[36220]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-11T19:55:39.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:38 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3047200172' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:55:39.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/904215645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-11T19:55:39.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2748886054' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-11T19:55:39.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-mon[36739]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-11T19:55:39.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-mon[36739]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-11T19:55:39.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2748886054' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-11T19:55:39.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-mon[36739]: osdmap e88: 12 total, 12 up, 12 in 2024-03-11T19:55:39.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-mon[36739]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-11T19:55:39.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:38 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3047200172' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-11T19:55:39.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.480+0000 7f8cd797a640 1 -- 172.21.15.83:0/184272782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd01011b0 msgr2=0x7f8cd01015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:39.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.480+0000 7f8cd797a640 1 --2- 172.21.15.83:0/184272782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd01011b0 0x7f8cd01015d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009ca0 tx=0x7f8ccc02c0f0 comp rx=0 tx=0).stop 2024-03-11T19:55:39.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.480+0000 7f8cd797a640 1 -- 172.21.15.83:0/184272782 shutdown_connections 2024-03-11T19:55:39.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.480+0000 7f8cd797a640 1 --2- 172.21.15.83:0/184272782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd0102d60 0x7f8cd01095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:39.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.480+0000 7f8cd797a640 1 --2- 172.21.15.83:0/184272782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd01023a0 0x7f8cd0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:39.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.480+0000 7f8cd797a640 1 --2- 172.21.15.83:0/184272782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd01011b0 0x7f8cd01015d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:39.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.480+0000 7f8cd797a640 1 -- 172.21.15.83:0/184272782 >> 172.21.15.83:0/184272782 conn(0x7f8cd00fc720 msgr2=0x7f8cd00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:39.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd797a640 1 -- 172.21.15.83:0/184272782 shutdown_connections 2024-03-11T19:55:39.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd797a640 1 -- 172.21.15.83:0/184272782 wait complete. 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd797a640 1 Processor -- start 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd797a640 1 -- start start 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd797a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd01011b0 0x7f8cd0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd797a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd01023a0 0x7f8cd0197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd797a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd0102d60 0x7f8cd019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd797a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd019ed40 con 0x7f8cd0102d60 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd797a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd019eeb0 con 0x7f8cd01011b0 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd797a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd019f070 con 0x7f8cd01023a0 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd5f17640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd0102d60 0x7f8cd019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd5f17640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd0102d60 0x7f8cd019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43902/0 (socket says 172.21.15.83:43902) 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd5f17640 1 -- 172.21.15.83:0/2655171574 learned_addr learned my addr 172.21.15.83:0/2655171574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd4f15640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd01023a0 0x7f8cd0197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:39.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.481+0000 7f8cd5716640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd01011b0 0x7f8cd0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cd5f17640 1 -- 172.21.15.83:0/2655171574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd01023a0 msgr2=0x7f8cd0197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cd5f17640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd01023a0 0x7f8cd0197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cd5f17640 1 -- 172.21.15.83:0/2655171574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd01011b0 msgr2=0x7f8cd0197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cd5f17640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd01011b0 0x7f8cd0197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cd5f17640 1 -- 172.21.15.83:0/2655171574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc009900 con 0x7f8cd0102d60 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cd5716640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd01011b0 0x7f8cd0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cd5f17640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd0102d60 0x7f8cd019e800 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800d750 tx=0x7f8cc800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cc67fc640 1 -- 172.21.15.83:0/2655171574 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8009bd0 con 0x7f8cd0102d60 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cc67fc640 1 -- 172.21.15.83:0/2655171574 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cc800de60 con 0x7f8cd0102d60 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cd797a640 1 -- 172.21.15.83:0/2655171574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd019f350 con 0x7f8cd0102d60 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cc67fc640 1 -- 172.21.15.83:0/2655171574 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8004e70 con 0x7f8cd0102d60 2024-03-11T19:55:39.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.482+0000 7f8cd797a640 1 -- 172.21.15.83:0/2655171574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd019f860 con 0x7f8cd0102d60 2024-03-11T19:55:39.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.483+0000 7f8cd797a640 1 -- 172.21.15.83:0/2655171574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd01084d0 con 0x7f8cd0102d60 2024-03-11T19:55:39.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.484+0000 7f8cc67fc640 1 -- 172.21.15.83:0/2655171574 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7f8cc8005020 con 0x7f8cd0102d60 2024-03-11T19:55:39.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.485+0000 7f8cc67fc640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cbc0a1a30 0x7f8cbc0a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:39.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.485+0000 7f8cd5716640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cbc0a1a30 0x7f8cbc0a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:39.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.485+0000 7f8cc67fc640 1 -- 172.21.15.83:0/2655171574 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8398+0+0 (secure 0 0 0) 0x7f8cc80ccdb0 con 0x7f8cd0102d60 2024-03-11T19:55:39.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.485+0000 7f8cd5716640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cbc0a1a30 0x7f8cbc0a3ef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc02c700 tx=0x7f8ccc0326d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:39.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.489+0000 7f8cc67fc640 1 -- 172.21.15.83:0/2655171574 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8cc8099a50 con 0x7f8cd0102d60 2024-03-11T19:55:39.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:39.661+0000 7f8cd797a640 1 -- 172.21.15.83:0/2655171574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8cd002ce40 con 0x7f8cd0102d60 2024-03-11T19:55:40.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: pgmap v843: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-03-11T19:55:40.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-11T19:55:40.317 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: Cluster is now healthy 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/3047200172' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.137:6837/2989164690,v1:172.21.15.137:6839/2989164690] up:boot 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.204:6834/3853844339,v1:172.21.15.204:6835/3853844339] up:boot 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.137:6834/2941719531,v1:172.21.15.137:6835/2941719531] up:boot 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.137:6836/4289942600,v1:172.21.15.137:6838/4289942600] up:boot 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.204:6836/2421449124,v1:172.21.15.204:6838/2421449124] up:boot 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: mds.? [v2:172.21.15.204:6837/3408858696,v1:172.21.15.204:6839/3408858696] up:boot 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: fsmap 12 up:standby 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:55:40.318 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:55:40.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:55:40.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:55:40.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:39 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2655171574' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.357+0000 7f8cc67fc640 1 -- 172.21.15.83:0/2655171574 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8cc80170d0 con 0x7f8cd0102d60 2024-03-11T19:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-03-11T19:55:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.360+0000 7f8cd797a640 1 -- 172.21.15.83:0/2655171574 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cbc0a1a30 msgr2=0x7f8cbc0a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.360+0000 7f8cd797a640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cbc0a1a30 0x7f8cbc0a3ef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc02c700 tx=0x7f8ccc0326d0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.360+0000 7f8cd797a640 1 -- 172.21.15.83:0/2655171574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd0102d60 msgr2=0x7f8cd019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.360+0000 7f8cd797a640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd0102d60 0x7f8cd019e800 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800d750 tx=0x7f8cc800dc10 comp rx=0 tx=0).stop 2024-03-11T19:55:40.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.360+0000 7f8cd797a640 1 -- 172.21.15.83:0/2655171574 shutdown_connections 2024-03-11T19:55:40.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.360+0000 7f8cd797a640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7f8cbc0a1a30 0x7f8cbc0a3ef0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.360+0000 7f8cd797a640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cd0102d60 0x7f8cd019e800 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.360+0000 7f8cd797a640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8cd01023a0 0x7f8cd0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.360+0000 7f8cd797a640 1 --2- 172.21.15.83:0/2655171574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8cd01011b0 0x7f8cd0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.360+0000 7f8cd797a640 1 -- 172.21.15.83:0/2655171574 >> 172.21.15.83:0/2655171574 conn(0x7f8cd00fc720 msgr2=0x7f8cd01056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.361+0000 7f8cd797a640 1 -- 172.21.15.83:0/2655171574 shutdown_connections 2024-03-11T19:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.361+0000 7f8cd797a640 1 -- 172.21.15.83:0/2655171574 wait complete. 2024-03-11T19:55:40.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: pgmap v843: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: Cluster is now healthy 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/3047200172' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.137:6837/2989164690,v1:172.21.15.137:6839/2989164690] up:boot 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.204:6834/3853844339,v1:172.21.15.204:6835/3853844339] up:boot 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.137:6834/2941719531,v1:172.21.15.137:6835/2941719531] up:boot 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.137:6836/4289942600,v1:172.21.15.137:6838/4289942600] up:boot 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.204:6836/2421449124,v1:172.21.15.204:6838/2421449124] up:boot 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: mds.? [v2:172.21.15.204:6837/3408858696,v1:172.21.15.204:6839/3408858696] up:boot 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: fsmap 12 up:standby 2024-03-11T19:55:40.388 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:55:40.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:55:40.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:55:40.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:55:40.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:55:40.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:55:40.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:39 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2655171574' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:55:40.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: pgmap v843: 97 pgs: 97 active+clean; 1.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-03-11T19:55:40.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: Cluster is now healthy 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/3047200172' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.137:6837/2989164690,v1:172.21.15.137:6839/2989164690] up:boot 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.204:6834/3853844339,v1:172.21.15.204:6835/3853844339] up:boot 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.137:6834/2941719531,v1:172.21.15.137:6835/2941719531] up:boot 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.137:6836/4289942600,v1:172.21.15.137:6838/4289942600] up:boot 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.204:6836/2421449124,v1:172.21.15.204:6838/2421449124] up:boot 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: mds.? [v2:172.21.15.204:6837/3408858696,v1:172.21.15.204:6839/3408858696] up:boot 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: fsmap 12 up:standby 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-11T19:55:40.405 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-11T19:55:40.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-11T19:55:40.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: from='mgr.14152 172.21.15.83:0/2681041491' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-11T19:55:40.406 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:39 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2655171574' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:55:40.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.485+0000 7fa53759e640 1 -- 172.21.15.83:0/3306127715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa538101b50 msgr2=0x7fa538103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:40.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.485+0000 7fa53759e640 1 --2- 172.21.15.83:0/3306127715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa538101b50 0x7fa538103f40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa528009900 tx=0x7fa52802bc30 comp rx=0 tx=0).stop 2024-03-11T19:55:40.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.485+0000 7fa53759e640 1 -- 172.21.15.83:0/3306127715 shutdown_connections 2024-03-11T19:55:40.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.485+0000 7fa53759e640 1 --2- 172.21.15.83:0/3306127715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa538106db0 0x7fa5381091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.485+0000 7fa53759e640 1 --2- 172.21.15.83:0/3306127715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa538104480 0x7fa538106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.485+0000 7fa53759e640 1 --2- 172.21.15.83:0/3306127715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa538101b50 0x7fa538103f40 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.485+0000 7fa53759e640 1 -- 172.21.15.83:0/3306127715 >> 172.21.15.83:0/3306127715 conn(0x7fa5380fb500 msgr2=0x7fa5380fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.485+0000 7fa53759e640 1 -- 172.21.15.83:0/3306127715 shutdown_connections 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.485+0000 7fa53759e640 1 -- 172.21.15.83:0/3306127715 wait complete. 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa53759e640 1 Processor -- start 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa53759e640 1 -- start start 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa53759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa538101b50 0x7fa538197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa53759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa538104480 0x7fa5381976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa53759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa538106db0 0x7fa53819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa53759e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53819ec80 con 0x7fa538106db0 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa53759e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53819edf0 con 0x7fa538101b50 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa53759e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53819efb0 con 0x7fa538104480 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa536d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa538106db0 0x7fa53819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa536d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa538106db0 0x7fa53819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53660/0 (socket says 172.21.15.83:53660) 2024-03-11T19:55:40.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa536d9d640 1 -- 172.21.15.83:0/119731061 learned_addr learned my addr 172.21.15.83:0/119731061 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa535d9b640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa538104480 0x7fa5381976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.486+0000 7fa53659c640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa538101b50 0x7fa538197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.487+0000 7fa536d9d640 1 -- 172.21.15.83:0/119731061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa538104480 msgr2=0x7fa5381976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.487+0000 7fa536d9d640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa538104480 0x7fa5381976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.487+0000 7fa536d9d640 1 -- 172.21.15.83:0/119731061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa538101b50 msgr2=0x7fa538197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.487+0000 7fa536d9d640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa538101b50 0x7fa538197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.487+0000 7fa536d9d640 1 -- 172.21.15.83:0/119731061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa528009560 con 0x7fa538106db0 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.487+0000 7fa536d9d640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa538106db0 0x7fa53819e740 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa530009fd0 tx=0x7fa53000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.487+0000 7fa5277fe640 1 -- 172.21.15.83:0/119731061 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53000bdd0 con 0x7fa538106db0 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.487+0000 7fa53759e640 1 -- 172.21.15.83:0/119731061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53819f290 con 0x7fa538106db0 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.487+0000 7fa5277fe640 1 -- 172.21.15.83:0/119731061 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa530009940 con 0x7fa538106db0 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.487+0000 7fa5277fe640 1 -- 172.21.15.83:0/119731061 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300106c0 con 0x7fa538106db0 2024-03-11T19:55:40.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.487+0000 7fa53759e640 1 -- 172.21.15.83:0/119731061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53819f750 con 0x7fa538106db0 2024-03-11T19:55:40.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.488+0000 7fa53759e640 1 -- 172.21.15.83:0/119731061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5380fcff0 con 0x7fa538106db0 2024-03-11T19:55:40.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.489+0000 7fa5277fe640 1 -- 172.21.15.83:0/119731061 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) v1 ==== 136279+0+0 (secure 0 0 0) 0x7fa530004190 con 0x7fa538106db0 2024-03-11T19:55:40.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.489+0000 7fa5277fe640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5200a1a30 0x7fa5200a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T19:55:40.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.490+0000 7fa53659c640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5200a1a30 0x7fa5200a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T19:55:40.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.490+0000 7fa5277fe640 1 -- 172.21.15.83:0/119731061 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 7886+0+0 (secure 0 0 0) 0x7fa5300ccf70 con 0x7fa538106db0 2024-03-11T19:55:40.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.490+0000 7fa53659c640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5200a1a30 0x7fa5200a3ef0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa528009230 tx=0x7fa528005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T19:55:40.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.494+0000 7fa5277fe640 1 -- 172.21.15.83:0/119731061 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa530100d20 con 0x7fa538106db0 2024-03-11T19:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:40.668+0000 7fa53759e640 1 -- 172.21.15.83:0/119731061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa53810cae0 con 0x7fa538106db0 2024-03-11T19:55:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.362+0000 7fa5277fe640 1 -- 172.21.15.83:0/119731061 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa5300170d0 con 0x7fa538106db0 2024-03-11T19:55:41.365 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-03-11T19:55:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 -- 172.21.15.83:0/119731061 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5200a1a30 msgr2=0x7fa5200a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5200a1a30 0x7fa5200a3ef0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa528009230 tx=0x7fa528005fd0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 -- 172.21.15.83:0/119731061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa538106db0 msgr2=0x7fa53819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T19:55:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa538106db0 0x7fa53819e740 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa530009fd0 tx=0x7fa53000def0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 -- 172.21.15.83:0/119731061 shutdown_connections 2024-03-11T19:55:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.83:6800/3477661212,v1:172.21.15.83:6801/3477661212] conn(0x7fa5200a1a30 0x7fa5200a3ef0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa538106db0 0x7fa53819e740 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa538104480 0x7fa5381976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 --2- 172.21.15.83:0/119731061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa538101b50 0x7fa538197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T19:55:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 -- 172.21.15.83:0/119731061 >> 172.21.15.83:0/119731061 conn(0x7fa5380fb500 msgr2=0x7fa538102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T19:55:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 -- 172.21.15.83:0/119731061 shutdown_connections 2024-03-11T19:55:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-03-11T19:55:41.366+0000 7fa53759e640 1 -- 172.21.15.83:0/119731061 wait complete. 2024-03-11T19:55:41.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:55:41.426 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:55:41.476 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:55:41.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:41 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/2655171574' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T19:55:41.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:41 smithi204 ceph-mon[36739]: osdmap e89: 12 total, 12 up, 12 in 2024-03-11T19:55:41.502 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:41 smithi204 ceph-mon[36739]: from='client.? 172.21.15.83:0/119731061' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:55:41.506 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-11T19:55:41.506 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-11T19:55:41.521 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-11T19:55:41.536 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-11T19:55:41.562 INFO:tasks.cephadm:Stopping all daemons... 2024-03-11T19:55:41.562 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-11T19:55:41.562 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.a 2024-03-11T19:55:41.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:41 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/2655171574' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T19:55:41.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:41 smithi137 ceph-mon[36220]: osdmap e89: 12 total, 12 up, 12 in 2024-03-11T19:55:41.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:41 smithi137 ceph-mon[36220]: from='client.? 172.21.15.83:0/119731061' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:55:41.816 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:41 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/2655171574' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-11T19:55:41.816 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:41 smithi083 ceph-mon[31864]: osdmap e89: 12 total, 12 up, 12 in 2024-03-11T19:55:41.816 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:41 smithi083 ceph-mon[31864]: from='client.? 172.21.15.83:0/119731061' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-11T19:55:41.817 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:41 smithi083 systemd[1]: Stopping Ceph mon.a for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:55:42.263 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a[31860]: 2024-03-11T19:55:41.866+0000 7faba12c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:55:42.263 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a[31860]: 2024-03-11T19:55:41.866+0000 7faba12c2700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-11T19:55:42.263 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:41 smithi083 podman[99423]: 2024-03-11 19:55:41.917160608 +0000 UTC m=+0.167681435 container died 5c07b05df3a9d4523dad626fbf69d189090e5ed76926a7f06b278a3894a7ddf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-03-11T19:55:42.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:42 smithi083 podman[99423]: 2024-03-11 19:55:42.349926715 +0000 UTC m=+0.600447549 container cleanup 5c07b05df3a9d4523dad626fbf69d189090e5ed76926a7f06b278a3894a7ddf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-11T19:55:42.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:42 smithi083 bash[99423]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a 2024-03-11T19:55:42.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.a.service' 2024-03-11T19:55:42.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:42 smithi083 podman[99437]: 2024-03-11 19:55:42.584877824 +0000 UTC m=+0.665225029 container remove 5c07b05df3a9d4523dad626fbf69d189090e5ed76926a7f06b278a3894a7ddf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-a, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-11T19:55:42.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:42 smithi083 systemd[1]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-11T19:55:42.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:42 smithi083 systemd[1]: Stopped Ceph mon.a for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:55:42.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Mar 11 19:55:42 smithi083 systemd[1]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.a.service: Consumed 23.597s CPU time. 2024-03-11T19:55:42.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:55:42.865 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-11T19:55:42.865 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-11T19:55:42.865 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.b 2024-03-11T19:55:43.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:42 smithi137 systemd[1]: Stopping Ceph mon.b for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:55:43.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:43 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-b[36216]: 2024-03-11T19:55:43.266+0000 7f493685d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:55:43.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:43 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-b[36216]: 2024-03-11T19:55:43.266+0000 7f493685d700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-11T19:55:43.638 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:43 smithi137 podman[71963]: 2024-03-11 19:55:43.500789675 +0000 UTC m=+0.387606431 container died 055cc9aa4fb3155facff4b7d424ed32c5d808f2470203d26bbf7392bdf238ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-11T19:55:44.119 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:43 smithi137 podman[71963]: 2024-03-11 19:55:43.916193417 +0000 UTC m=+0.803010168 container cleanup 055cc9aa4fb3155facff4b7d424ed32c5d808f2470203d26bbf7392bdf238ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-11T19:55:44.119 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 11 19:55:43 smithi137 bash[71963]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-b 2024-03-11T19:55:44.255 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.b.service' 2024-03-11T19:55:44.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:55:44.313 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-11T19:55:44.314 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-11T19:55:44.314 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.c 2024-03-11T19:55:44.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:44 smithi204 systemd[1]: Stopping Ceph mon.c for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:55:44.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:44 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-c[36723]: 2024-03-11T19:55:44.627+0000 7f82a765b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:55:44.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:44 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-c[36723]: 2024-03-11T19:55:44.627+0000 7f82a765b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-11T19:55:44.904 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:44 smithi204 podman[73739]: 2024-03-11 19:55:44.729265646 +0000 UTC m=+0.232082028 container died f7bf0a33d86343cc8d00a0f914e82e787ae90c5d2a5bf4971a9783dc94900120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-c, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:55:45.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:45 smithi204 podman[73739]: 2024-03-11 19:55:45.069253074 +0000 UTC m=+0.572069447 container cleanup f7bf0a33d86343cc8d00a0f914e82e787ae90c5d2a5bf4971a9783dc94900120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-c, RELEASE=squid-94a5397, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-11T19:55:45.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 11 19:55:45 smithi204 bash[73739]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mon-c 2024-03-11T19:55:45.490 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mon.c.service' 2024-03-11T19:55:45.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:55:45.542 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-11T19:55:45.542 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-11T19:55:45.542 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.x 2024-03-11T19:55:45.864 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:55:45 smithi083 systemd[1]: Stopping Ceph mgr.x for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:55:46.192 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:55:45 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:55:45.862+0000 7f856119b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:55:46.192 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:55:45 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x[32099]: 2024-03-11T19:55:45.862+0000 7f856119b700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-11T19:55:46.192 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:55:45 smithi083 podman[99544]: 2024-03-11 19:55:45.89450586 +0000 UTC m=+0.175772618 container died 23a489d69253aef9bddcef970c0e4b27e94918711d45826b7c4dc2aecedcbba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-11T19:55:46.508 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:55:46 smithi083 podman[99544]: 2024-03-11 19:55:46.289051035 +0000 UTC m=+0.570317905 container cleanup 23a489d69253aef9bddcef970c0e4b27e94918711d45826b7c4dc2aecedcbba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:94a539707733325322964c6a187c6a1b41be1daf, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:55:46.509 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Mar 11 19:55:46 smithi083 bash[99544]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-x 2024-03-11T19:55:46.621 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.x.service' 2024-03-11T19:55:46.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:55:46.659 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-11T19:55:46.659 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-11T19:55:46.659 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.y 2024-03-11T19:55:46.967 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:55:46 smithi137 systemd[1]: Stopping Ceph mgr.y for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:55:47.305 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:55:46 smithi137 podman[72083]: 2024-03-11 19:55:46.965878023 +0000 UTC m=+0.125091910 container died 24e5dc9168b0a5c03548f10462c6f9627febf255c82e65e514e93edce9b9582f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-11T19:55:47.638 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:55:47 smithi137 podman[72083]: 2024-03-11 19:55:47.303702339 +0000 UTC m=+0.462916235 container cleanup 24e5dc9168b0a5c03548f10462c6f9627febf255c82e65e514e93edce9b9582f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:55:47.638 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:55:47 smithi137 bash[72083]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y 2024-03-11T19:55:47.638 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:55:47 smithi137 podman[72097]: 2024-03-11 19:55:47.514176084 +0000 UTC m=+0.546151168 container remove 24e5dc9168b0a5c03548f10462c6f9627febf255c82e65e514e93edce9b9582f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-y, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:55:47.638 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 11 19:55:47 smithi137 systemd[1]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-11T19:55:47.721 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.y.service' 2024-03-11T19:55:47.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:55:47.759 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-11T19:55:47.759 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-11T19:55:47.759 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.z 2024-03-11T19:55:48.096 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:55:47 smithi204 systemd[1]: Stopping Ceph mgr.z for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:55:48.404 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:55:48 smithi204 podman[73860]: 2024-03-11 19:55:48.094994337 +0000 UTC m=+0.147227260 container died 154856ab6423e87e1b0890feeab0e0da53869a5fb5871bce8a602053d44f5040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-11T19:55:48.742 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:55:48 smithi204 podman[73860]: 2024-03-11 19:55:48.516699361 +0000 UTC m=+0.568932303 container cleanup 154856ab6423e87e1b0890feeab0e0da53869a5fb5871bce8a602053d44f5040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-11T19:55:48.743 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:55:48 smithi204 bash[73860]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z 2024-03-11T19:55:48.993 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.z.service' 2024-03-11T19:55:49.017 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:55:48 smithi204 podman[73874]: 2024-03-11 19:55:48.741261015 +0000 UTC m=+0.640197190 container remove 154856ab6423e87e1b0890feeab0e0da53869a5fb5871bce8a602053d44f5040 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-11T19:55:49.017 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:55:48 smithi204 systemd[1]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-11T19:55:49.017 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:55:48 smithi204 systemd[1]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.z.service: Failed with result 'exit-code'. 2024-03-11T19:55:49.017 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:55:48 smithi204 systemd[1]: Stopped Ceph mgr.z for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:55:49.017 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 11 19:55:48 smithi204 systemd[1]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mgr.z.service: Consumed 13.915s CPU time. 2024-03-11T19:55:49.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:55:49.030 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-11T19:55:49.030 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-11T19:55:49.030 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.0 2024-03-11T19:55:49.316 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:49 smithi083 systemd[1]: Stopping Ceph osd.0 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:55:49.816 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:49 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0[43998]: 2024-03-11T19:55:49.334+0000 7f7b0e19f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:55:49.816 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:49 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0[43998]: 2024-03-11T19:55:49.334+0000 7f7b0e19f700 -1 osd.0 90 *** Got signal Terminated *** 2024-03-11T19:55:49.816 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:49 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0[43998]: 2024-03-11T19:55:49.334+0000 7f7b0e19f700 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:55:55.179 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:54 smithi083 podman[99665]: 2024-03-11 19:55:54.899020146 +0000 UTC m=+5.681252033 container died eb967f3ade909921897e5e9c611a45ddbedb73b8511bf4490d6c6d890b724516 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:55:55.546 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:55 smithi083 podman[99665]: 2024-03-11 19:55:55.277090331 +0000 UTC m=+6.059322211 container cleanup eb967f3ade909921897e5e9c611a45ddbedb73b8511bf4490d6c6d890b724516 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-11T19:55:55.547 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:55 smithi083 bash[99665]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0 2024-03-11T19:55:55.816 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:55 smithi083 podman[99682]: 2024-03-11 19:55:55.545104247 +0000 UTC m=+0.641692164 container remove eb967f3ade909921897e5e9c611a45ddbedb73b8511bf4490d6c6d890b724516 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-11T19:55:56.225 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:55 smithi083 podman[99772]: 2024-03-11 19:55:55.783383765 +0000 UTC m=+0.017604853 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:55:56.225 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:55 smithi083 podman[99772]: 2024-03-11 19:55:55.938157146 +0000 UTC m=+0.172378235 container create 4eac862d8a39e48e6c1cbebac7f23bec1d5b7019974bcf93ca34fff886695d93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-11T19:55:56.226 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:56 smithi083 podman[99772]: 2024-03-11 19:55:56.224179493 +0000 UTC m=+0.458400595 container init 4eac862d8a39e48e6c1cbebac7f23bec1d5b7019974bcf93ca34fff886695d93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-11T19:55:56.516 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:56 smithi083 podman[99772]: 2024-03-11 19:55:56.227406814 +0000 UTC m=+0.461627911 container start 4eac862d8a39e48e6c1cbebac7f23bec1d5b7019974bcf93ca34fff886695d93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-11T19:55:56.517 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:56 smithi083 podman[99772]: 2024-03-11 19:55:56.307424845 +0000 UTC m=+0.541645948 container attach 4eac862d8a39e48e6c1cbebac7f23bec1d5b7019974bcf93ca34fff886695d93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-11T19:55:56.811 INFO:journalctl@ceph.osd.0.smithi083.stdout:Mar 11 19:55:56 smithi083 podman[99772]: 2024-03-11 19:55:56.51573374 +0000 UTC m=+0.749954824 container died 4eac862d8a39e48e6c1cbebac7f23bec1d5b7019974bcf93ca34fff886695d93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-11T19:55:57.107 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.0.service' 2024-03-11T19:55:57.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:55:57.148 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-11T19:55:57.148 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-11T19:55:57.148 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.1 2024-03-11T19:55:57.460 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:55:57 smithi137 systemd[1]: Stopping Ceph osd.1 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:55:57.887 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:55:57 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1[40817]: 2024-03-11T19:55:57.457+0000 7f2e0884b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:55:57.887 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:55:57 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1[40817]: 2024-03-11T19:55:57.457+0000 7f2e0884b700 -1 osd.1 90 *** Got signal Terminated *** 2024-03-11T19:55:57.887 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:55:57 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1[40817]: 2024-03-11T19:55:57.457+0000 7f2e0884b700 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:56:03.277 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:56:02 smithi137 podman[72205]: 2024-03-11 19:56:02.982483598 +0000 UTC m=+5.642515436 container died 828157dfc4042ac7a58b169def36767a471f1346736048e62109fa1b2d981296 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-11T19:56:03.627 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:56:03 smithi137 podman[72205]: 2024-03-11 19:56:03.374641426 +0000 UTC m=+6.034673264 container cleanup 828157dfc4042ac7a58b169def36767a471f1346736048e62109fa1b2d981296 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-03-11T19:56:03.627 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:56:03 smithi137 bash[72205]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1 2024-03-11T19:56:03.887 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:56:03 smithi137 podman[72224]: 2024-03-11 19:56:03.626017149 +0000 UTC m=+0.639129234 container remove 828157dfc4042ac7a58b169def36767a471f1346736048e62109fa1b2d981296 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1, RELEASE=squid-94a5397, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-11T19:56:04.238 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:56:03 smithi137 podman[72317]: 2024-03-11 19:56:03.819162966 +0000 UTC m=+0.017878995 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:56:04.239 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:56:03 smithi137 podman[72317]: 2024-03-11 19:56:03.935339437 +0000 UTC m=+0.134055495 container create 1c04003130e920f11e96c8201a7679a5749dd2e2c24d4b88f3b2550364f11645 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-11T19:56:04.239 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:56:04 smithi137 podman[72317]: 2024-03-11 19:56:04.154695062 +0000 UTC m=+0.353411090 container init 1c04003130e920f11e96c8201a7679a5749dd2e2c24d4b88f3b2550364f11645 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:56:04.239 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:56:04 smithi137 podman[72317]: 2024-03-11 19:56:04.157930605 +0000 UTC m=+0.356646628 container start 1c04003130e920f11e96c8201a7679a5749dd2e2c24d4b88f3b2550364f11645 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-11T19:56:04.637 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:56:04 smithi137 podman[72317]: 2024-03-11 19:56:04.237482798 +0000 UTC m=+0.436198835 container attach 1c04003130e920f11e96c8201a7679a5749dd2e2c24d4b88f3b2550364f11645 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:56:04.637 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 11 19:56:04 smithi137 podman[72317]: 2024-03-11 19:56:04.442441638 +0000 UTC m=+0.641157688 container died 1c04003130e920f11e96c8201a7679a5749dd2e2c24d4b88f3b2550364f11645 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-11T19:56:05.005 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.1.service' 2024-03-11T19:56:05.048 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:56:05.048 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-11T19:56:05.048 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-11T19:56:05.049 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.2 2024-03-11T19:56:05.364 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:05 smithi204 systemd[1]: Stopping Ceph osd.2 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:56:05.629 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:05 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2[41904]: 2024-03-11T19:56:05.361+0000 7f9065b94700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:56:05.629 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:05 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2[41904]: 2024-03-11T19:56:05.361+0000 7f9065b94700 -1 osd.2 90 *** Got signal Terminated *** 2024-03-11T19:56:05.629 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:05 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2[41904]: 2024-03-11T19:56:05.361+0000 7f9065b94700 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:56:11.154 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:10 smithi204 podman[73984]: 2024-03-11 19:56:10.858190759 +0000 UTC m=+5.618660340 container died 924d8105635e8e8dab9406207d810c17065f59898abb761c4579c2bb1d7dd18e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:56:11.448 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:11 smithi204 podman[73984]: 2024-03-11 19:56:11.262222385 +0000 UTC m=+6.022691965 container cleanup 924d8105635e8e8dab9406207d810c17065f59898abb761c4579c2bb1d7dd18e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:56:11.448 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:11 smithi204 bash[73984]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2 2024-03-11T19:56:11.871 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:11 smithi204 podman[74014]: 2024-03-11 19:56:11.447032024 +0000 UTC m=+0.580593947 container remove 924d8105635e8e8dab9406207d810c17065f59898abb761c4579c2bb1d7dd18e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS) 2024-03-11T19:56:12.126 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:11 smithi204 podman[74105]: 2024-03-11 19:56:11.773340264 +0000 UTC m=+0.017822535 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:56:12.127 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:11 smithi204 podman[74105]: 2024-03-11 19:56:11.906388543 +0000 UTC m=+0.150870841 container create 78b658365bcbd16f33fc54e6c899689d039ac0b7d49fb75c4303236bc69138f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-11T19:56:12.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:12 smithi204 podman[74105]: 2024-03-11 19:56:12.125960254 +0000 UTC m=+0.370442529 container init 78b658365bcbd16f33fc54e6c899689d039ac0b7d49fb75c4303236bc69138f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397) 2024-03-11T19:56:12.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:12 smithi204 podman[74105]: 2024-03-11 19:56:12.129444926 +0000 UTC m=+0.373927182 container start 78b658365bcbd16f33fc54e6c899689d039ac0b7d49fb75c4303236bc69138f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:56:12.404 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:12 smithi204 podman[74105]: 2024-03-11 19:56:12.208684172 +0000 UTC m=+0.453166441 container attach 78b658365bcbd16f33fc54e6c899689d039ac0b7d49fb75c4303236bc69138f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-11T19:56:12.729 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 11 19:56:12 smithi204 podman[74105]: 2024-03-11 19:56:12.420343149 +0000 UTC m=+0.664825397 container died 78b658365bcbd16f33fc54e6c899689d039ac0b7d49fb75c4303236bc69138f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-11T19:56:13.019 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.2.service' 2024-03-11T19:56:13.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:56:13.062 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-11T19:56:13.062 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-11T19:56:13.062 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.3 2024-03-11T19:56:13.374 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:13 smithi083 systemd[1]: Stopping Ceph osd.3 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:56:13.636 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:13 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3[51002]: 2024-03-11T19:56:13.371+0000 7f2ec62b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:56:13.636 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:13 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3[51002]: 2024-03-11T19:56:13.371+0000 7f2ec62b8700 -1 osd.3 90 *** Got signal Terminated *** 2024-03-11T19:56:13.636 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:13 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3[51002]: 2024-03-11T19:56:13.371+0000 7f2ec62b8700 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:56:16.065 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:15 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3[51002]: 2024-03-11T19:56:15.793+0000 7f2ec30ad700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:49.230682+0000 front 2024-03-11T19:55:49.230709+0000 (oldest deadline 2024-03-11T19:56:15.130960+0000) 2024-03-11T19:56:16.066 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:15 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:15.916+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:16.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:16 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:16.308+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:17.065 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:16 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3[51002]: 2024-03-11T19:56:16.788+0000 7f2ec30ad700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:49.230682+0000 front 2024-03-11T19:55:49.230709+0000 (oldest deadline 2024-03-11T19:56:15.130960+0000) 2024-03-11T19:56:17.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:16 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:16.882+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:17.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:17 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:17.057+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:17.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:17 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:17.271+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:17.868 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:17 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3[51002]: 2024-03-11T19:56:17.741+0000 7f2ec30ad700 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:49.230682+0000 front 2024-03-11T19:55:49.230709+0000 (oldest deadline 2024-03-11T19:56:15.130960+0000) 2024-03-11T19:56:17.868 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:17 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:17.574+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:18.315 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:17 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:17.865+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:18.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:18 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:18.081+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:18.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:18 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:18.291+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:18.903 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:18 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:18.602+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:18.903 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:18 smithi083 podman[99879]: 2024-03-11 19:56:18.791632238 +0000 UTC m=+5.538744935 container died 42abf92700cb804b70c46cbd0ed0dc941785b2460d5e2218eb4cdeeb3e7aa613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:56:19.194 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:18 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:18.901+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:19.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:19.073+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:19.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:19.043+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:19.565 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:19 smithi083 podman[99879]: 2024-03-11 19:56:19.194299435 +0000 UTC m=+5.941412131 container cleanup 42abf92700cb804b70c46cbd0ed0dc941785b2460d5e2218eb4cdeeb3e7aa613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-11T19:56:19.566 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:19 smithi083 bash[99879]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3 2024-03-11T19:56:19.566 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:19 smithi083 podman[99909]: 2024-03-11 19:56:19.415474694 +0000 UTC m=+0.623940555 container remove 42abf92700cb804b70c46cbd0ed0dc941785b2460d5e2218eb4cdeeb3e7aa613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-11T19:56:19.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:19 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:19.264+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:19.878 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:19 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:19.592+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:19.878 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:19 smithi083 podman[100001]: 2024-03-11 19:56:19.651316896 +0000 UTC m=+0.018529094 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:56:19.878 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:19 smithi083 podman[100001]: 2024-03-11 19:56:19.774652271 +0000 UTC m=+0.141864468 container create b6b8766d0b35fdb09224458563f37e298ec634db0320b66c7fb7415e7fa06fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-94a5397, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-11T19:56:20.315 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:19 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:19.876+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:20.315 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:20 smithi083 podman[100001]: 2024-03-11 19:56:20.020131009 +0000 UTC m=+0.387343219 container init b6b8766d0b35fdb09224458563f37e298ec634db0320b66c7fb7415e7fa06fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:56:20.316 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:20 smithi083 podman[100001]: 2024-03-11 19:56:20.023789861 +0000 UTC m=+0.391002072 container start b6b8766d0b35fdb09224458563f37e298ec634db0320b66c7fb7415e7fa06fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-11T19:56:20.316 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:20 smithi083 podman[100001]: 2024-03-11 19:56:20.110731833 +0000 UTC m=+0.477944062 container attach b6b8766d0b35fdb09224458563f37e298ec634db0320b66c7fb7415e7fa06fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-11T19:56:20.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:20.042+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:20.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:20.061+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:20.574 INFO:journalctl@ceph.osd.3.smithi083.stdout:Mar 11 19:56:20 smithi083 podman[100001]: 2024-03-11 19:56:20.317561655 +0000 UTC m=+0.684773855 container died b6b8766d0b35fdb09224458563f37e298ec634db0320b66c7fb7415e7fa06fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-11T19:56:20.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:20 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:20.222+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:20.879 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:20 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:20.572+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:20.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.3.service' 2024-03-11T19:56:20.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:56:20.968 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-11T19:56:20.969 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-11T19:56:20.969 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.4 2024-03-11T19:56:21.294 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:21 smithi137 systemd[1]: Stopping Ceph osd.4 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:56:21.295 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:21 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:21.190+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:21.315 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:20 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:20.878+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:21.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:21 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:21.024+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:21.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:21 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:21.093+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:21.637 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:21 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4[47521]: 2024-03-11T19:56:21.292+0000 7fe2206f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:56:21.637 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:21 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4[47521]: 2024-03-11T19:56:21.292+0000 7fe2206f0700 -1 osd.4 90 *** Got signal Terminated *** 2024-03-11T19:56:21.637 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:21 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4[47521]: 2024-03-11T19:56:21.292+0000 7fe2206f0700 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:56:21.882 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:21 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:21.587+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:22.315 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:21 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:21.881+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:22.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:22 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:22.016+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:22.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:22 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:22.100+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:22.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:22 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:22.148+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:23.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:22 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:22.863+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:23.066 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:22 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:22.624+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:23.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:22 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:22.996+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:23.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:23 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:23.130+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:23.553 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:23 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:23.193+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:23.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:23 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:23.552+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:24.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:23 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:23.821+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:24.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:23 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:23.640+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:24.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:23 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:23.640+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:24.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:23 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:23.984+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:24.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:24 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:24.104+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:24.575 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:24 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:24.144+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:24.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:24 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:24.574+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:25.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:24 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:24.851+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:25.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:24 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:24.661+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:25.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:24 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:24.661+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:25.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:25 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:25.024+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:25.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:25 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:25.024+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.005377+0000 front 2024-03-11T19:56:01.005471+0000 (oldest deadline 2024-03-11T19:56:24.505268+0000) 2024-03-11T19:56:25.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:25 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:25.066+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:25.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:25 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:25.066+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:25.553 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:25 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:25.184+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:25.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:25 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:25.553+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:25.887 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:25 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4[47521]: 2024-03-11T19:56:25.702+0000 7fe2194dd700 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.080854+0000 front 2024-03-11T19:56:02.080886+0000 (oldest deadline 2024-03-11T19:56:24.981055+0000) 2024-03-11T19:56:26.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:25 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:25.808+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:26.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:25 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:25.648+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:26.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:25 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:25.648+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:26.091 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:25 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:25.764+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:26.091 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:26 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:26.010+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:26.092 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:26 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:26.010+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.005377+0000 front 2024-03-11T19:56:01.005471+0000 (oldest deadline 2024-03-11T19:56:24.505268+0000) 2024-03-11T19:56:26.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:26 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:26.090+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:26.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:26 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:26.090+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:26.559 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:26 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:26.222+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:26.559 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:26 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:26.222+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:26.887 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:26 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:26.559+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:26.887 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:26 smithi137 podman[72434]: 2024-03-11 19:56:26.718562841 +0000 UTC m=+5.558332996 container died 5aa5906080c1e3ece3ec90daacf2f6ed65f864ef3f791c7492c098dde37c81e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-11T19:56:27.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:26 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:26.814+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:27.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:26 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:26.814+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:27.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:26 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:26.659+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:27.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:26 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:26.659+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:27.081 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:26 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:26.753+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:27.081 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:27 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:26.999+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:27.081 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:27 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:26.999+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.005377+0000 front 2024-03-11T19:56:01.005471+0000 (oldest deadline 2024-03-11T19:56:24.505268+0000) 2024-03-11T19:56:27.252 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:27 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:27.238+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:27.252 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:27 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:27.238+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:27.253 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:27 smithi137 podman[72434]: 2024-03-11 19:56:27.067496957 +0000 UTC m=+5.907267114 container cleanup 5aa5906080c1e3ece3ec90daacf2f6ed65f864ef3f791c7492c098dde37c81e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-11T19:56:27.253 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:27 smithi137 bash[72434]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4 2024-03-11T19:56:27.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:27 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:27.080+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:27.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:27 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:27.080+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:27.563 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:27 smithi137 podman[72452]: 2024-03-11 19:56:27.25202712 +0000 UTC m=+0.525465726 container remove 5aa5906080c1e3ece3ec90daacf2f6ed65f864ef3f791c7492c098dde37c81e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:56:27.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:27 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:27.563+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:27.887 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:27 smithi137 podman[72548]: 2024-03-11 19:56:27.677848751 +0000 UTC m=+0.113885008 container create 98f851629cc668420af67bee374ea5a8adae3a591018fd975a8844d59cabe7fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-11T19:56:27.887 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:27 smithi137 podman[72548]: 2024-03-11 19:56:27.582201639 +0000 UTC m=+0.018237877 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:56:27.995 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:27 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:27.722+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:28.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:27 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:27.862+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:28.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:27 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:27.862+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:28.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:27 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:27.708+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:28.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:27 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:27.708+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:28.227 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:27 smithi137 podman[72548]: 2024-03-11 19:56:27.927627898 +0000 UTC m=+0.363664136 container init 98f851629cc668420af67bee374ea5a8adae3a591018fd975a8844d59cabe7fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4-deactivate, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-11T19:56:28.228 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:27 smithi137 podman[72548]: 2024-03-11 19:56:27.930687468 +0000 UTC m=+0.366723697 container start 98f851629cc668420af67bee374ea5a8adae3a591018fd975a8844d59cabe7fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:56:28.228 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:28 smithi137 podman[72548]: 2024-03-11 19:56:28.015801816 +0000 UTC m=+0.451838069 container attach 98f851629cc668420af67bee374ea5a8adae3a591018fd975a8844d59cabe7fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:56:28.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:27 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:27.993+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:28.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:27 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:27.993+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.005377+0000 front 2024-03-11T19:56:01.005471+0000 (oldest deadline 2024-03-11T19:56:24.505268+0000) 2024-03-11T19:56:28.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:28 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:28.056+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:28.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:28 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:28.056+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:28.518 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 11 19:56:28 smithi137 podman[72548]: 2024-03-11 19:56:28.227332894 +0000 UTC m=+0.663369133 container died 98f851629cc668420af67bee374ea5a8adae3a591018fd975a8844d59cabe7fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-11T19:56:28.519 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:28 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:28.265+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:28.519 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:28 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:28.265+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:28.777 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:28 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:28.549+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:28.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.4.service' 2024-03-11T19:56:28.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:56:28.854 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-11T19:56:28.854 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-11T19:56:28.854 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.5 2024-03-11T19:56:29.033 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:28 smithi204 systemd[1]: Stopping Ceph osd.5 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:56:29.033 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:28 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:28.707+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:29.033 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:29 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:29.031+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:29.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:28 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:28.847+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:29.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:28 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:28.847+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:29.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:28 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:28.707+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:29.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:28 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:28.707+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:29.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:29 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:29.031+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:29.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:29 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:29.036+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:29.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:29 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:29.036+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.005377+0000 front 2024-03-11T19:56:01.005471+0000 (oldest deadline 2024-03-11T19:56:24.505268+0000) 2024-03-11T19:56:29.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:29 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:29.148+0000 7fc629be2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:56:29.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:29 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:29.148+0000 7fc629be2700 -1 osd.5 90 *** Got signal Terminated *** 2024-03-11T19:56:29.404 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:29 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:29.148+0000 7fc629be2700 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:56:29.520 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:29 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:29.219+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:29.520 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:29 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:29.219+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:29.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:29 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:29.518+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:30.029 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:29 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:29.725+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:30.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:29 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:29.889+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:30.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:29 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:29.889+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:30.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:29 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:29.679+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:30.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:29 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:29.679+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:30.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:30 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:30.059+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:30.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:30 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:30.059+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.005377+0000 front 2024-03-11T19:56:01.005471+0000 (oldest deadline 2024-03-11T19:56:24.505268+0000) 2024-03-11T19:56:30.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:30 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:30.028+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:30.404 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:30 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:30.028+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:30.510 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:30 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:30.173+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:30.511 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:30 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:30.173+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:30.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:30 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:30.509+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:31.013 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:30 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:30.678+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:31.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:30 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:30.929+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:31.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:30 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:30.929+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:31.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:30 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:30.727+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:31.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:30 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:30.727+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:31.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:31 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:31.012+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:31.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:31 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:31.012+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.005377+0000 front 2024-03-11T19:56:01.005471+0000 (oldest deadline 2024-03-11T19:56:24.505268+0000) 2024-03-11T19:56:31.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:31 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:31.059+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:31.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:31 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:31.059+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:31.542 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:31 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:31.157+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:31.543 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:31 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:31.157+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:31.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:31 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:31.541+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:31.903 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:31 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:31.652+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:32.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:31 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:31.741+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:32.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:31 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:31.741+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:32.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:31 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:31.889+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:32.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:31 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:31.889+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:32.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:31 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:31.889+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:06.469030+0000 front 2024-03-11T19:56:06.469118+0000 (oldest deadline 2024-03-11T19:56:31.769102+0000) 2024-03-11T19:56:32.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:31 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:31.965+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:32.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:31 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:31.965+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.005377+0000 front 2024-03-11T19:56:01.005471+0000 (oldest deadline 2024-03-11T19:56:24.505268+0000) 2024-03-11T19:56:32.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:32 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:32.102+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:32.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:32 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:32.102+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:32.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:32 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:32.102+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:32.574 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:32 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:32.183+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:32.574 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:32 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:32.183+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:32.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:32 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:32.573+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:32.887 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:32 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:32.573+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:32.930 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:32 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:32.663+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:32.930 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:32 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:32.663+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:33.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:32 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:32.698+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:33.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:32 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:32.698+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:33.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:32 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:32.857+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:33.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:32 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:32.857+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:33.066 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:32 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:32.857+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:06.469030+0000 front 2024-03-11T19:56:06.469118+0000 (oldest deadline 2024-03-11T19:56:31.769102+0000) 2024-03-11T19:56:33.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:33 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:33.134+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:33.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:33 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:33.134+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:33.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:33 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:33.134+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:33.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:32 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:32.929+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:33.403 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:32 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:32.929+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.005377+0000 front 2024-03-11T19:56:01.005471+0000 (oldest deadline 2024-03-11T19:56:24.505268+0000) 2024-03-11T19:56:33.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:33 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:33.066+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:33.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:33 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:33.066+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:33.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:33 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:33.066+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:33.665 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:33 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:33.532+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:33.665 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:33 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:33.532+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:34.047 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:33 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:33.710+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:34.047 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:33 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:33.710+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:34.048 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:33 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:33.891+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:53.503602+0000 front 2024-03-11T19:55:53.503614+0000 (oldest deadline 2024-03-11T19:56:18.203573+0000) 2024-03-11T19:56:34.048 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:33 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:33.891+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.005377+0000 front 2024-03-11T19:56:01.005471+0000 (oldest deadline 2024-03-11T19:56:24.505268+0000) 2024-03-11T19:56:34.048 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:33 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5[48502]: 2024-03-11T19:56:33.891+0000 7fc6269d7700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.206396+0000 front 2024-03-11T19:56:09.206318+0000 (oldest deadline 2024-03-11T19:56:33.306482+0000) 2024-03-11T19:56:34.064 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:33 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:33.741+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:34.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:33 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:33.741+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:34.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:33 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:33.905+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:34.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:33 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:33.905+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:34.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:33 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:33.905+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:06.469030+0000 front 2024-03-11T19:56:06.469118+0000 (oldest deadline 2024-03-11T19:56:31.769102+0000) 2024-03-11T19:56:34.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:34 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:34.109+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:34.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:34 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:34.109+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:34.387 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:34 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:34.109+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:34.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:34 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:34.046+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:34.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:34 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:34.046+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:34.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:34 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:34.046+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:34.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:34 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:34.500+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:34.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:34 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:34.500+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:35.023 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:34 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:34.671+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:35.023 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:34 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:34.671+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:35.024 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:34 smithi204 podman[74210]: 2024-03-11 19:56:34.763475167 +0000 UTC m=+5.713036707 container stop 1904bc3f651001baed671ee63d0a9062eab6ab9216d4dee152ddaaf830eecae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:56:35.024 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:34 smithi204 podman[74210]: 2024-03-11 19:56:34.788971299 +0000 UTC m=+5.738532856 container died 1904bc3f651001baed671ee63d0a9062eab6ab9216d4dee152ddaaf830eecae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-11T19:56:35.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:34 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:34.881+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:35.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:34 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:34.881+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:35.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:34 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:34.881+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:06.469030+0000 front 2024-03-11T19:56:06.469118+0000 (oldest deadline 2024-03-11T19:56:31.769102+0000) 2024-03-11T19:56:35.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:34 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:34.785+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:35.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:34 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:34.785+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:35.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:34 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:34.785+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:35.317 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:35 smithi204 podman[74210]: 2024-03-11 19:56:35.140298804 +0000 UTC m=+6.089860344 container cleanup 1904bc3f651001baed671ee63d0a9062eab6ab9216d4dee152ddaaf830eecae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-11T19:56:35.317 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:35 smithi204 bash[74210]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5 2024-03-11T19:56:35.317 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:35 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:35.022+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:35.317 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:35 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:35.022+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:35.317 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:35 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:35.022+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:35.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:35 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:35.120+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:35.387 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:35 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:35.120+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:35.387 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:35 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:35.120+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:35.653 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:35 smithi204 podman[74229]: 2024-03-11 19:56:35.31662474 +0000 UTC m=+0.544173999 container remove 1904bc3f651001baed671ee63d0a9062eab6ab9216d4dee152ddaaf830eecae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-94a5397) 2024-03-11T19:56:35.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:35 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:35.472+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:35.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:35 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:35.472+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:36.003 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:35 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:35.673+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:36.003 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:35 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:35.673+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:36.004 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:35 smithi204 podman[74319]: 2024-03-11 19:56:35.640946519 +0000 UTC m=+0.017621461 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:56:36.004 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:35 smithi204 podman[74319]: 2024-03-11 19:56:35.767683849 +0000 UTC m=+0.144358779 container create 7f838edcc87c42e2c59c92a2a78a03869b50bb41ebc2b6556703f871e6de9a1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-11T19:56:36.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:35.841+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:36.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:35.841+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:36.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:35.841+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:06.469030+0000 front 2024-03-11T19:56:06.469118+0000 (oldest deadline 2024-03-11T19:56:31.769102+0000) 2024-03-11T19:56:36.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:35.808+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:36.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:35.808+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:36.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:35.808+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:36.302 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:36 smithi204 podman[74319]: 2024-03-11 19:56:36.003559249 +0000 UTC m=+0.380234176 container init 7f838edcc87c42e2c59c92a2a78a03869b50bb41ebc2b6556703f871e6de9a1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-11T19:56:36.302 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:36 smithi204 podman[74319]: 2024-03-11 19:56:36.006723357 +0000 UTC m=+0.383398278 container start 7f838edcc87c42e2c59c92a2a78a03869b50bb41ebc2b6556703f871e6de9a1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-11T19:56:36.302 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:36 smithi204 podman[74319]: 2024-03-11 19:56:36.078345512 +0000 UTC m=+0.455020462 container attach 7f838edcc87c42e2c59c92a2a78a03869b50bb41ebc2b6556703f871e6de9a1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-11T19:56:36.302 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:36 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:36.028+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:36.302 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:36 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:36.028+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:36.302 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:36 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:36.028+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:36.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:36 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:36.128+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:36.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:36 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:36.128+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:36.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:36 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:36.128+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:36.570 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 11 19:56:36 smithi204 podman[74319]: 2024-03-11 19:56:36.301855251 +0000 UTC m=+0.678530195 container died 7f838edcc87c42e2c59c92a2a78a03869b50bb41ebc2b6556703f871e6de9a1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:56:36.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:36 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:36.429+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:36.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:36 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:36.429+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:36.903 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:36 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:36.712+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:36.903 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:36 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:36.712+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:36.950 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.5.service' 2024-03-11T19:56:36.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:56:36.992 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-11T19:56:36.992 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-11T19:56:36.992 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.6 2024-03-11T19:56:37.021 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:36 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:36.766+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:37.021 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:36 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:36.766+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:37.021 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:36 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:36.766+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:37.021 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:36 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:36.832+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:37.021 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:36 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:36.832+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:37.022 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:36 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:36.832+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:06.469030+0000 front 2024-03-11T19:56:06.469118+0000 (oldest deadline 2024-03-11T19:56:31.769102+0000) 2024-03-11T19:56:37.292 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:37 smithi083 systemd[1]: Stopping Ceph osd.6 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:56:37.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:37 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:37.120+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:37.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:37 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:37.120+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:37.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:37 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:37.120+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:37.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:37 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:37.038+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:37.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:37 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:37.038+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:37.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:37 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:37.038+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:37.565 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:37.291+0000 7fbbbc2b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:56:37.565 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:37.291+0000 7fbbbc2b4700 -1 osd.6 90 *** Got signal Terminated *** 2024-03-11T19:56:37.565 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:37.291+0000 7fbbbc2b4700 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:56:37.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:37 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:37.462+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:37.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:37 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:37.462+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:37.903 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:37 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:37.706+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:37.903 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:37 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:37.706+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:38.064 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:37.861+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:38.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:37.861+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:38.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:37.861+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:06.469030+0000 front 2024-03-11T19:56:06.469118+0000 (oldest deadline 2024-03-11T19:56:31.769102+0000) 2024-03-11T19:56:38.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:37.808+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:38.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:37.808+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:38.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:37 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:37.808+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:38.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:38.104+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:38.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:38.104+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:38.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:38.104+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:38.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:38.062+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:38.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:38.062+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:38.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:38.062+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:38.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:38.468+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:38.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:38 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:38.468+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:39.036 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:38.710+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:39.037 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:38 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:38.710+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:39.315 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:38 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:38.847+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:39.315 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:38 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:38.847+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:39.315 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:38 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:38.847+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:06.469030+0000 front 2024-03-11T19:56:06.469118+0000 (oldest deadline 2024-03-11T19:56:31.769102+0000) 2024-03-11T19:56:39.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:38 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:38.857+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:39.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:38 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:38.857+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:39.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:38 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:38.857+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:39.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:39 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:39.121+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:39.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:39 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:39.121+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:39.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:39 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:39.121+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:39.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:39 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:39.036+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:39.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:39 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:39.036+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:39.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:39 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:39.036+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:39.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:39 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:39.484+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:39.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:39 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:39.484+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:40.048 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:39 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:39.758+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:40.048 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:39 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:39.758+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:40.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:39 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:39.803+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:40.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:39 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:39.803+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:40.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:39 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:39.803+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:06.469030+0000 front 2024-03-11T19:56:06.469118+0000 (oldest deadline 2024-03-11T19:56:31.769102+0000) 2024-03-11T19:56:40.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:39 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:39.884+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:40.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:39 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:39.884+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:40.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:39 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:39.884+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:40.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:39 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:39.884+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:40.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:40 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:40.126+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:40.387 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:40 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:40.126+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:40.387 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:40 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:40.126+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:40.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:40 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:40.047+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:40.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:40 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:40.047+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:40.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:40 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:40.047+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:40.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:40 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:40.482+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:40.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:40 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:40.482+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:41.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:40 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:40.776+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:41.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:40 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:40.776+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:41.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:40 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:40.776+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:06.469030+0000 front 2024-03-11T19:56:06.469118+0000 (oldest deadline 2024-03-11T19:56:31.769102+0000) 2024-03-11T19:56:41.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:40 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:40.889+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:41.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:40 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:40.889+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:41.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:40 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:40.889+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:41.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:40 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:40.889+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:41.073 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:40 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:40.728+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:41.073 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:40 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:40.728+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:41.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:41 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:41.072+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:41.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:41 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:41.072+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:41.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:41 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:41.072+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:41.513 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:41 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:41.142+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:41.514 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:41 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:41.142+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:41.514 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:41 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:41.142+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:41.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:41 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:41.512+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:41.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:41 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:41.512+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:41.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:41 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:41.512+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.879273+0000 front 2024-03-11T19:56:17.879294+0000 (oldest deadline 2024-03-11T19:56:41.379417+0000) 2024-03-11T19:56:42.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:41.770+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.065789+0000 front 2024-03-11T19:55:52.065599+0000 (oldest deadline 2024-03-11T19:56:15.366838+0000) 2024-03-11T19:56:42.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:41.770+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.568507+0000 front 2024-03-11T19:56:00.568620+0000 (oldest deadline 2024-03-11T19:56:26.468631+0000) 2024-03-11T19:56:42.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:41.770+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:06.469030+0000 front 2024-03-11T19:56:06.469118+0000 (oldest deadline 2024-03-11T19:56:31.769102+0000) 2024-03-11T19:56:42.065 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6[57712]: 2024-03-11T19:56:41.770+0000 7fbbb50a1700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.471326+0000 front 2024-03-11T19:56:17.471422+0000 (oldest deadline 2024-03-11T19:56:41.571510+0000) 2024-03-11T19:56:42.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:41.918+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:42.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:41.918+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:42.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:41.918+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:42.065 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:41 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:41.918+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:42.073 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:41 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:41.695+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:42.073 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:41 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:41.695+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:42.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:42 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:42.072+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:42.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:42 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:42.072+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:42.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:42 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:42.072+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:42.503 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:42 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:42.137+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:42.503 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:42 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:42.137+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:42.503 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:42 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:42.137+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:42.503 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:42 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:42.137+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:42.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:42 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:42.502+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:42.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:42 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:42.502+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:42.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:42 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:42.502+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.879273+0000 front 2024-03-11T19:56:17.879294+0000 (oldest deadline 2024-03-11T19:56:41.379417+0000) 2024-03-11T19:56:43.031 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:42 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:42.681+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:43.031 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:42 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:42.681+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:43.168 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:42 smithi083 podman[100106]: 2024-03-11 19:56:42.875586851 +0000 UTC m=+5.691362562 container died 226ebe4cbefe4f5a87aedee96e8187c9c84b1356143a75c24b793e35fa463349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-11T19:56:43.168 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:42 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:42.922+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:43.169 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:42 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:42.922+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:43.169 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:42 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:42.922+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:43.169 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:42 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:42.922+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:43.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:43 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:43.030+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:43.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:43 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:43.030+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:43.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:43 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:43.030+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:43.426 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:43 smithi083 podman[100106]: 2024-03-11 19:56:43.241980467 +0000 UTC m=+6.057756182 container cleanup 226ebe4cbefe4f5a87aedee96e8187c9c84b1356143a75c24b793e35fa463349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-03-11T19:56:43.426 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:43 smithi083 bash[100106]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6 2024-03-11T19:56:43.503 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:43 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:43.184+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:43.504 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:43 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:43.184+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:43.504 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:43 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:43.184+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:43.504 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:43 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:43.184+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:43.814 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:43 smithi083 podman[100124]: 2024-03-11 19:56:43.426612921 +0000 UTC m=+0.543433761 container remove 226ebe4cbefe4f5a87aedee96e8187c9c84b1356143a75c24b793e35fa463349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-11T19:56:43.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:43 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:43.502+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:43.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:43 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:43.502+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:43.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:43 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:43.502+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.879273+0000 front 2024-03-11T19:56:17.879294+0000 (oldest deadline 2024-03-11T19:56:41.379417+0000) 2024-03-11T19:56:43.991 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:43 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:43.698+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:43.991 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:43 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:43.698+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:44.146 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:43 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:43.955+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:44.146 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:43 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:43.955+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:44.146 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:43 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:43.955+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:44.147 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:43 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:43.955+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:44.147 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:43 smithi083 podman[100218]: 2024-03-11 19:56:43.733314453 +0000 UTC m=+0.017857781 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:56:44.147 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:43 smithi083 podman[100218]: 2024-03-11 19:56:43.835953368 +0000 UTC m=+0.120496688 container create 5655ed33eb799a0593cc0cac2b6dd59458aaa7364a1f25cb9741c38bcfa21a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-94a5397, GIT_CLEAN=True) 2024-03-11T19:56:44.147 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:44 smithi083 podman[100218]: 2024-03-11 19:56:44.063397952 +0000 UTC m=+0.347941286 container init 5655ed33eb799a0593cc0cac2b6dd59458aaa7364a1f25cb9741c38bcfa21a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-03-11T19:56:44.147 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:44 smithi083 podman[100218]: 2024-03-11 19:56:44.066398965 +0000 UTC m=+0.350942284 container start 5655ed33eb799a0593cc0cac2b6dd59458aaa7364a1f25cb9741c38bcfa21a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6-deactivate, RELEASE=squid-94a5397, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-11T19:56:44.147 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:44 smithi083 podman[100218]: 2024-03-11 19:56:44.146478392 +0000 UTC m=+0.431021728 container attach 5655ed33eb799a0593cc0cac2b6dd59458aaa7364a1f25cb9741c38bcfa21a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6-deactivate, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-11T19:56:44.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:44 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:44.231+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:44.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:44 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:44.231+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:44.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:44 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:44.231+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:44.386 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:44 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:44.231+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:44.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:43 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:43.990+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:44.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:43 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:43.990+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:44.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:43 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:43.990+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:44.565 INFO:journalctl@ceph.osd.6.smithi083.stdout:Mar 11 19:56:44 smithi083 podman[100218]: 2024-03-11 19:56:44.35365323 +0000 UTC m=+0.638196552 container died 5655ed33eb799a0593cc0cac2b6dd59458aaa7364a1f25cb9741c38bcfa21a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-11T19:56:44.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:44 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:44.466+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:44.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:44 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:44.466+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:44.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:44 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:44.466+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.879273+0000 front 2024-03-11T19:56:17.879294+0000 (oldest deadline 2024-03-11T19:56:41.379417+0000) 2024-03-11T19:56:44.944 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.6.service' 2024-03-11T19:56:44.999 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:44 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:44.734+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:44.999 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:44 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:44.734+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:45.000 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:44 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:44.734+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:45.026 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:56:45.026 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-11T19:56:45.026 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-11T19:56:45.026 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.7 2024-03-11T19:56:45.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:44 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:44.947+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:45.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:44 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:44.947+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:45.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:44 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:44.947+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:45.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:44 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:44.947+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:45.354 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:45 smithi137 systemd[1]: Stopping Ceph osd.7 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:56:45.355 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:45 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:45.248+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:45.355 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:45 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:45.248+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:45.355 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:45 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:45.248+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:45.355 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:45 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:45.248+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:45.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:44 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:44.999+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:45.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:44 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:44.999+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:45.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:44 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:44.999+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:45.636 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:45 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:45.353+0000 7f56fda69700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:56:45.636 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:45 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:45.353+0000 7f56fda69700 -1 osd.7 90 *** Got signal Terminated *** 2024-03-11T19:56:45.636 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:45 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:45.353+0000 7f56fda69700 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:56:45.636 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:45 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:45.472+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:45.636 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:45 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:45.472+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:45.637 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:45 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:45.472+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.879273+0000 front 2024-03-11T19:56:17.879294+0000 (oldest deadline 2024-03-11T19:56:41.379417+0000) 2024-03-11T19:56:46.036 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:45 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:45.693+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:46.036 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:45 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:45.693+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:46.036 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:45 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:45.693+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:46.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:45 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:45.936+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:46.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:45 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:45.936+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:46.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:45 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:45.936+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:46.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:45 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:45.936+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:46.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:46 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:46.034+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:46.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:46 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:46.034+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:46.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:46 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:46.034+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:46.636 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:46 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:46.498+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:46.636 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:46 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:46.498+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:46.636 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:46 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:46.498+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.879273+0000 front 2024-03-11T19:56:17.879294+0000 (oldest deadline 2024-03-11T19:56:41.379417+0000) 2024-03-11T19:56:46.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:46 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:46.273+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:46.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:46 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:46.273+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:46.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:46 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:46.273+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:46.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:46 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:46.273+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:47.004 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:46 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:46.708+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:47.005 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:46 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:46.708+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:47.005 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:46 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:46.708+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:47.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:46.948+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:47.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:46.948+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:47.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:46.948+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:47.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:46 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:46.948+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:47.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:47 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:47.080+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:47.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:47 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:47.080+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:47.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:47 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:47.080+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:47.526 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:47 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:47.267+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:47.527 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:47 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:47.267+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:47.527 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:47 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:47.267+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:47.527 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:47 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:47.267+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:47.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:47 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:47.525+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:47.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:47 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:47.525+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:47.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:47 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:47.525+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.879273+0000 front 2024-03-11T19:56:17.879294+0000 (oldest deadline 2024-03-11T19:56:41.379417+0000) 2024-03-11T19:56:47.903 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:47 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:47.747+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:47.903 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:47 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:47.747+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:47.903 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:47 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:47.747+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:48.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:47 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:47.936+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:48.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:47 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:47.936+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:48.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:47 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:47.936+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:48.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:47 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:47.936+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:48.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:48 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:48.063+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:48.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:48 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:48.063+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:48.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:48 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:48.063+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:48.496 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:48 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:48.226+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:48.496 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:48 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:48.226+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:48.497 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:48 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:48.226+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:48.497 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:48 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:48.226+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:48.497 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:48 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:48.226+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:48.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:48 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:48.495+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:48.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:48 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:48.495+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:48.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:48 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:48.495+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.879273+0000 front 2024-03-11T19:56:17.879294+0000 (oldest deadline 2024-03-11T19:56:41.379417+0000) 2024-03-11T19:56:49.050 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:48 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:48.741+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:49.050 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:48 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:48.741+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:49.051 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:48 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:48.741+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:49.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:48 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:48.955+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:49.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:48 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:48.955+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:49.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:48 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:48.955+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:49.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:48 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:48.955+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:49.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:49 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:49.049+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:49.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:49 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:49.049+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:49.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:49 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:49.049+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:49.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:49 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:49.049+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:56:49.489 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:49 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:49.232+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:49.489 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:49 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:49.232+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:49.489 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:49 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:49.232+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:49.489 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:49 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:49.232+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:49.489 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:49 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:49.232+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:49.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:49 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:49.487+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:01.174931+0000 front 2024-03-11T19:56:01.174903+0000 (oldest deadline 2024-03-11T19:56:23.475061+0000) 2024-03-11T19:56:49.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:49 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:49.487+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:08.577501+0000 front 2024-03-11T19:56:08.577514+0000 (oldest deadline 2024-03-11T19:56:32.077568+0000) 2024-03-11T19:56:49.887 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:49 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:49.487+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.879273+0000 front 2024-03-11T19:56:17.879294+0000 (oldest deadline 2024-03-11T19:56:41.379417+0000) 2024-03-11T19:56:49.887 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:49 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7[54557]: 2024-03-11T19:56:49.487+0000 7f56fa85e700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:21.379726+0000 front 2024-03-11T19:56:21.379744+0000 (oldest deadline 2024-03-11T19:56:48.980419+0000) 2024-03-11T19:56:50.005 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:49 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:49.712+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:50.006 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:49 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:49.712+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:50.006 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:49 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:49.712+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:50.006 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:49 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:49.712+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:23.769083+0000 front 2024-03-11T19:56:23.769217+0000 (oldest deadline 2024-03-11T19:56:49.069021+0000) 2024-03-11T19:56:50.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:49 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:49.960+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:50.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:49 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:49.960+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:50.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:49 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:49.960+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:50.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:49 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:49.960+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:50.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:50 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:50.005+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:50.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:50 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:50.005+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:50.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:50 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:50.005+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:50.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:50 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:50.005+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:56:50.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:50 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:50.205+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:50.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:50 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:50.205+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:50.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:50 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:50.205+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:50.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:50 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:50.205+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:50.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:50 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:50.205+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:51.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:51.002+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:51.153 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:51.002+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:51.153 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:51.002+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:51.153 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:51.002+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:56:51.153 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:50 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:50.761+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:51.153 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:50 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:50.761+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:51.153 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:50 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:50.761+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:51.153 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:50 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:50.761+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:23.769083+0000 front 2024-03-11T19:56:23.769217+0000 (oldest deadline 2024-03-11T19:56:49.069021+0000) 2024-03-11T19:56:51.222 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:50 smithi137 podman[72668]: 2024-03-11 19:56:50.962029228 +0000 UTC m=+5.735483703 container died 89d7bdd7833f2797fa4c523d59e90210347e281cf51a630bd4a390386c7ef4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:56:51.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:50 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:50.973+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:51.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:50 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:50.973+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:51.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:50 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:50.973+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:51.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:50 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:50.973+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:51.496 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:51 smithi137 podman[72668]: 2024-03-11 19:56:51.303597234 +0000 UTC m=+6.077051706 container cleanup 89d7bdd7833f2797fa4c523d59e90210347e281cf51a630bd4a390386c7ef4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-11T19:56:51.497 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:51 smithi137 bash[72668]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7 2024-03-11T19:56:51.497 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:51 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:51.229+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:51.497 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:51 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:51.229+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:51.497 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:51 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:51.229+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:51.497 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:51 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:51.229+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:51.497 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:51 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:51.229+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:51.886 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:51 smithi137 podman[72687]: 2024-03-11 19:56:51.496451143 +0000 UTC m=+0.534003769 container remove 89d7bdd7833f2797fa4c523d59e90210347e281cf51a630bd4a390386c7ef4af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-11T19:56:52.152 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:51.722+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:52.153 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:51.722+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:52.153 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:51.722+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:52.153 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:51.722+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:23.769083+0000 front 2024-03-11T19:56:23.769217+0000 (oldest deadline 2024-03-11T19:56:49.069021+0000) 2024-03-11T19:56:52.153 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:51.956+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:52.153 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:51.956+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:52.153 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:51.956+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:52.153 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:51 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:51.956+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:56:52.218 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:51 smithi137 podman[72778]: 2024-03-11 19:56:51.905705471 +0000 UTC m=+0.113230381 container create 1573e5c10be4e46e5e560af677c16b45fc08ed752cb4ed3289278af4a559b151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=squid-94a5397) 2024-03-11T19:56:52.218 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:51 smithi137 podman[72778]: 2024-03-11 19:56:51.81005286 +0000 UTC m=+0.017577795 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:56:52.218 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:52 smithi137 podman[72778]: 2024-03-11 19:56:52.108020148 +0000 UTC m=+0.315545085 container init 1573e5c10be4e46e5e560af677c16b45fc08ed752cb4ed3289278af4a559b151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-03-11T19:56:52.218 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:52 smithi137 podman[72778]: 2024-03-11 19:56:52.110977464 +0000 UTC m=+0.318502380 container start 1573e5c10be4e46e5e560af677c16b45fc08ed752cb4ed3289278af4a559b151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-11T19:56:52.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:51 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:51.943+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:52.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:51 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:51.943+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:52.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:51 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:51.943+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:52.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:51 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:51.943+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:52.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:51 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:51.943+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:56:52.637 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:52 smithi137 podman[72778]: 2024-03-11 19:56:52.230510098 +0000 UTC m=+0.438035014 container attach 1573e5c10be4e46e5e560af677c16b45fc08ed752cb4ed3289278af4a559b151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-11T19:56:52.637 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 11 19:56:52 smithi137 podman[72778]: 2024-03-11 19:56:52.398839462 +0000 UTC m=+0.606364387 container died 1573e5c10be4e46e5e560af677c16b45fc08ed752cb4ed3289278af4a559b151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:56:52.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:52 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:52.217+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:52.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:52 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:52.217+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:52.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:52 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:52.217+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:52.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:52 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:52.217+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:52.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:52 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:52.217+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:52.988 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.7.service' 2024-03-11T19:56:53.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:56:53.030 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-11T19:56:53.030 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-11T19:56:53.031 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.8 2024-03-11T19:56:53.059 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:52 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:52.737+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:53.059 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:52 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:52.737+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:53.059 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:52 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:52.737+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:53.059 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:52 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:52.737+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:23.769083+0000 front 2024-03-11T19:56:23.769217+0000 (oldest deadline 2024-03-11T19:56:49.069021+0000) 2024-03-11T19:56:53.060 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:52 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:52.957+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:53.060 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:52 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:52.957+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:53.060 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:52 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:52.957+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:53.060 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:52 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:52.957+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:56:53.266 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:53 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:53.182+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:53.267 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:53 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:53.182+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:53.267 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:53 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:53.182+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:53.267 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:53 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:53.182+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:53.267 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:53 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:53.182+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:53.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:52 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:52.969+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:53.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:52 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:52.969+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:53.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:52 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:52.969+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:53.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:52 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:52.969+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:53.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:52 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:52.969+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:56:53.335 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:53 smithi204 systemd[1]: Stopping Ceph osd.8 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:56:53.652 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:53 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:53.334+0000 7ff98c448700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:56:53.653 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:53 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:53.334+0000 7ff98c448700 -1 osd.8 90 *** Got signal Terminated *** 2024-03-11T19:56:53.653 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:53 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:53.334+0000 7ff98c448700 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:56:53.967 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:53 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:53.713+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:53.968 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:53 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:53.713+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:53.968 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:53 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:53.713+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:53.968 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:53 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:53.713+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:23.769083+0000 front 2024-03-11T19:56:23.769217+0000 (oldest deadline 2024-03-11T19:56:49.069021+0000) 2024-03-11T19:56:54.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:53 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:53.962+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:54.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:53 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:53.962+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:54.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:53 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:53.962+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:54.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:53 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:53.963+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:54.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:53 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:53.963+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:56:54.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:53 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:53.967+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:54.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:53 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:53.967+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:54.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:53 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:53.967+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:54.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:53 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:53.967+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:56:54.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:54 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:54.161+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:54.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:54 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:54.161+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:54.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:54 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:54.161+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:54.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:54 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:54.161+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:54.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:54 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:54.161+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:54.937 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:54 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:54.681+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:54.938 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:54 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:54.681+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:54.938 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:54 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:54.681+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:54.938 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:54 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:54.681+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:23.769083+0000 front 2024-03-11T19:56:23.769217+0000 (oldest deadline 2024-03-11T19:56:49.069021+0000) 2024-03-11T19:56:55.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:55 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:55.000+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:55.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:55 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:55.000+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:55.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:55 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:55.000+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:55.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:55 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:55.000+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:55.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:55 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:55.000+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:56:55.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:54 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:54.937+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:55.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:54 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:54.937+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:55.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:54 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:54.937+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:55.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:54 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:54.937+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:56:55.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:55 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:55.159+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:55.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:55 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:55.159+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:55.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:55 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:55.159+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:55.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:55 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:55.159+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:55.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:55 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:55.159+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:55.961 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:55 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:55.701+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:55.961 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:55 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:55.701+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:55.962 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:55 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:55.701+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:55.962 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:55 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:55.701+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:23.769083+0000 front 2024-03-11T19:56:23.769217+0000 (oldest deadline 2024-03-11T19:56:49.069021+0000) 2024-03-11T19:56:56.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.003+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:56.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.003+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:56.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.003+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:56.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.003+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:56.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.003+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:56:56.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.003+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.704552+0000 front 2024-03-11T19:56:31.704537+0000 (oldest deadline 2024-03-11T19:56:55.204362+0000) 2024-03-11T19:56:56.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:55 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:55.961+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:56.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:55 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:55.961+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:56.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:55 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:55.961+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:56.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:55 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:55.961+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:56:56.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:56 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:56.205+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:56.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:56 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:56.205+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:56.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:56 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:56.205+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:56.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:56 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:56.205+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:56.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:56 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:56.205+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:56.989 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:56 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:56.716+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:56.990 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:56 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:56.716+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:56.990 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:56 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:56.716+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:56.990 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:56 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:56.716+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:23.769083+0000 front 2024-03-11T19:56:23.769217+0000 (oldest deadline 2024-03-11T19:56:49.069021+0000) 2024-03-11T19:56:56.990 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:56 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:56.716+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:32.370719+0000 front 2024-03-11T19:56:32.370747+0000 (oldest deadline 2024-03-11T19:56:56.470890+0000) 2024-03-11T19:56:57.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.978+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:57.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.978+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:57.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.978+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:57.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.978+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:57.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.978+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:56:57.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:56 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:56.978+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.704552+0000 front 2024-03-11T19:56:31.704537+0000 (oldest deadline 2024-03-11T19:56:55.204362+0000) 2024-03-11T19:56:57.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:56 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:56.989+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:57.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:56 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:56.989+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:57.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:56 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:56.989+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:57.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:56 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:56.989+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:56:57.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:57 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:57.186+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:57.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:57 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:57.186+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:57.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:57 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:57.186+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:57.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:57 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:57.186+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:57.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:57 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:57.186+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:57.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:57 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:57.186+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:56:58.020 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:57 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:57.697+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:00.965722+0000 front 2024-03-11T19:56:00.965681+0000 (oldest deadline 2024-03-11T19:56:25.665700+0000) 2024-03-11T19:56:58.020 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:57 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:57.697+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.267186+0000 front 2024-03-11T19:56:10.267338+0000 (oldest deadline 2024-03-11T19:56:32.567326+0000) 2024-03-11T19:56:58.020 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:57 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:57.697+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:12.567850+0000 front 2024-03-11T19:56:12.567745+0000 (oldest deadline 2024-03-11T19:56:43.768479+0000) 2024-03-11T19:56:58.020 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:57 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:57.697+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:23.769083+0000 front 2024-03-11T19:56:23.769217+0000 (oldest deadline 2024-03-11T19:56:49.069021+0000) 2024-03-11T19:56:58.020 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:57 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8[55534]: 2024-03-11T19:56:57.697+0000 7ff98923d700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:32.370719+0000 front 2024-03-11T19:56:32.370747+0000 (oldest deadline 2024-03-11T19:56:56.470890+0000) 2024-03-11T19:56:58.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:57 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:57.968+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:58.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:57 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:57.968+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:58.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:57 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:57.968+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:58.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:57 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:57.968+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:58.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:57 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:57.968+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:56:58.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:57 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:57.968+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.704552+0000 front 2024-03-11T19:56:31.704537+0000 (oldest deadline 2024-03-11T19:56:55.204362+0000) 2024-03-11T19:56:58.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:58 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:58.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:58.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:58 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:58.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:58.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:58 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:58.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:58.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:58 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:58.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:56:58.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:58 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:58.181+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:58.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:58 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:58.181+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:58.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:58 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:58.181+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:58.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:58 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:58.181+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:58.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:58 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:58.181+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:58.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:58 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:58.181+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:56:59.027 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:58 smithi204 podman[74438]: 2024-03-11 19:56:58.75399427 +0000 UTC m=+5.528930829 container died 340fa14656f5ce543f39bd8f183bbbbb8f7e5bc2c5ef3fef7c9a1116b5f5fcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-11T19:56:59.027 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:58 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:58.995+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:56:59.028 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:58 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:58.995+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:56:59.028 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:58 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:58.995+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:56:59.028 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:58 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:58.995+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:56:59.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:59 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:59.000+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:56:59.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:59 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:59.000+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:56:59.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:59 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:59.000+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:56:59.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:59 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:59.000+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:56:59.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:59 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:59.000+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:56:59.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:56:59 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:56:59.000+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.704552+0000 front 2024-03-11T19:56:31.704537+0000 (oldest deadline 2024-03-11T19:56:55.204362+0000) 2024-03-11T19:56:59.377 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:59 smithi204 podman[74438]: 2024-03-11 19:56:59.135521468 +0000 UTC m=+5.910458019 container cleanup 340fa14656f5ce543f39bd8f183bbbbb8f7e5bc2c5ef3fef7c9a1116b5f5fcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-11T19:56:59.377 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:59 smithi204 bash[74438]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8 2024-03-11T19:56:59.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:59 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:59.220+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:56:59.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:59 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:59.220+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:56:59.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:59 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:59.220+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:56:59.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:59 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:59.220+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:56:59.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:59 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:59.220+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:56:59.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:56:59 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:56:59.220+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:56:59.652 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:59 smithi204 podman[74455]: 2024-03-11 19:56:59.377535303 +0000 UTC m=+0.619708868 container remove 340fa14656f5ce543f39bd8f183bbbbb8f7e5bc2c5ef3fef7c9a1116b5f5fcaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_BRANCH=HEAD) 2024-03-11T19:56:59.955 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:59 smithi204 podman[74548]: 2024-03-11 19:56:59.599670024 +0000 UTC m=+0.018184509 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:56:59.955 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:59 smithi204 podman[74548]: 2024-03-11 19:56:59.719971731 +0000 UTC m=+0.138486203 container create a7334134c28ad9a59f27c5218d15ac3859b5516f108860d015c49b85d57bda0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.build-date=20240307) 2024-03-11T19:57:00.241 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:59 smithi204 podman[74548]: 2024-03-11 19:56:59.955641704 +0000 UTC m=+0.374156187 container init a7334134c28ad9a59f27c5218d15ac3859b5516f108860d015c49b85d57bda0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-11T19:57:00.241 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:56:59 smithi204 podman[74548]: 2024-03-11 19:56:59.95933165 +0000 UTC m=+0.377846133 container start a7334134c28ad9a59f27c5218d15ac3859b5516f108860d015c49b85d57bda0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, ceph=True) 2024-03-11T19:57:00.242 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:57:00 smithi204 podman[74548]: 2024-03-11 19:57:00.055589582 +0000 UTC m=+0.474104075 container attach a7334134c28ad9a59f27c5218d15ac3859b5516f108860d015c49b85d57bda0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8-deactivate, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-11T19:57:00.242 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:59 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:59.975+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:00.242 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:59 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:59.975+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:00.242 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:59 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:59.975+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:00.242 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:59 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:59.975+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:00.242 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:56:59 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:56:59.975+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:00.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:00 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:00.035+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:57:00.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:00 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:00.035+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:57:00.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:00 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:00.035+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:57:00.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:00 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:00.035+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:57:00.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:00 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:00.035+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:57:00.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:00 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:00.035+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.704552+0000 front 2024-03-11T19:56:31.704537+0000 (oldest deadline 2024-03-11T19:56:55.204362+0000) 2024-03-11T19:57:00.510 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 11 19:57:00 smithi204 podman[74548]: 2024-03-11 19:57:00.241317868 +0000 UTC m=+0.659832359 container died a7334134c28ad9a59f27c5218d15ac3859b5516f108860d015c49b85d57bda0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-8-deactivate, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:57:00.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:00 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:00.191+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:00.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:00 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:00.191+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:00.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:00 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:00.191+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:00.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:00 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:00.191+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:00.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:00 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:00.191+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:00.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:00 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:00.191+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:00.801 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.8.service' 2024-03-11T19:57:00.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:00.883 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-11T19:57:00.883 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-11T19:57:00.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.9 2024-03-11T19:57:01.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:00 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:00.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:01.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:00 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:00.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:01.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:00 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:00.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:01.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:00 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:00.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:01.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:00 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:00.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:01.186 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:00 smithi083 systemd[1]: Stopping Ceph osd.9 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:01.186 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:01 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:01.045+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:57:01.186 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:01 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:01.045+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:57:01.186 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:01 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:01.045+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:57:01.186 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:01 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:01.045+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:57:01.186 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:01 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:01.045+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:57:01.186 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:01 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:01.045+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.704552+0000 front 2024-03-11T19:56:31.704537+0000 (oldest deadline 2024-03-11T19:56:55.204362+0000) 2024-03-11T19:57:01.186 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:01 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:01.184+0000 7faf8cc4d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:01.186 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:01 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:01.184+0000 7faf8cc4d700 -1 osd.9 90 *** Got signal Terminated *** 2024-03-11T19:57:01.186 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:01 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:01.184+0000 7faf8cc4d700 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:57:01.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:01 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:01.216+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:01.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:01 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:01.216+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:01.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:01 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:01.216+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:01.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:01 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:01.216+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:01.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:01 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:01.216+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:01.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:01 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:01.216+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:02.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:02 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:02.039+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:57:02.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:02 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:02.039+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:57:02.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:02 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:02.039+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:57:02.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:02 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:02.039+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:57:02.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:02 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:02.039+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:57:02.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:02 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:02.039+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.704552+0000 front 2024-03-11T19:56:31.704537+0000 (oldest deadline 2024-03-11T19:56:55.204362+0000) 2024-03-11T19:57:02.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:01 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:01.997+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:02.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:01 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:01.997+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:02.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:01 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:01.997+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:02.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:01 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:01.997+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:02.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:01 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:01.997+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:02.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:02 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:02.250+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:02.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:02 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:02.250+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:02.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:02 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:02.250+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:02.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:02 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:02.250+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:02.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:02 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:02.250+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:02.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:02 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:02.250+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:03.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:03.009+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:57:03.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:03.009+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:57:03.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:03.009+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:57:03.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:03.009+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:57:03.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:03.009+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:57:03.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:03 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:03.009+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.704552+0000 front 2024-03-11T19:56:31.704537+0000 (oldest deadline 2024-03-11T19:56:55.204362+0000) 2024-03-11T19:57:03.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:03.014+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:03.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:03.014+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:03.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:03.014+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:03.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:03.014+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:03.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:03.014+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:03.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:03 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:03.295+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:03.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:03 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:03.295+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:03.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:03 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:03.295+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:03.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:03 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:03.295+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:03.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:03 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:03.295+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:03.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:03 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:03.295+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:04.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:04 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:04.054+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:57:04.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:04 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:04.054+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:57:04.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:04 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:04.054+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:57:04.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:04 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:04.054+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:57:04.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:04 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:04.054+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:57:04.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:04 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:04.054+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.704552+0000 front 2024-03-11T19:56:31.704537+0000 (oldest deadline 2024-03-11T19:56:55.204362+0000) 2024-03-11T19:57:04.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:04 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:04.054+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:40.505264+0000 front 2024-03-11T19:56:40.505316+0000 (oldest deadline 2024-03-11T19:57:03.405337+0000) 2024-03-11T19:57:04.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:03.989+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:04.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:03.989+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:04.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:03.989+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:04.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:03.989+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:04.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:03.989+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:04.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:04 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:04.302+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:04.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:04 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:04.303+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:04.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:04 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:04.303+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:04.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:04 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:04.303+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:04.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:04 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:04.303+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:04.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:04 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:04.303+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:04.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:04 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:04.303+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.984242+0000 front 2024-03-11T19:56:41.984383+0000 (oldest deadline 2024-03-11T19:57:03.684184+0000) 2024-03-11T19:57:05.304 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:04 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:04.982+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:05.304 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:04 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:04.982+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:05.304 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:04 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:04.982+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:05.304 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:04 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:04.982+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:05.305 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:04 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:04.982+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:05.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:05.051+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:57:05.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:05.051+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:57:05.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:05.051+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:57:05.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:05.051+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:57:05.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:05.051+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:57:05.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:05.051+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.704552+0000 front 2024-03-11T19:56:31.704537+0000 (oldest deadline 2024-03-11T19:56:55.204362+0000) 2024-03-11T19:57:05.314 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:05 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:05.051+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:40.505264+0000 front 2024-03-11T19:56:40.505316+0000 (oldest deadline 2024-03-11T19:57:03.405337+0000) 2024-03-11T19:57:05.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:05 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:05.280+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:05.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:05 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:05.280+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:05.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:05 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:05.280+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:05.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:05 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:05.280+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:05.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:05 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:05.280+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:05.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:05 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:05.280+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:05.637 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:05 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:05.280+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.984242+0000 front 2024-03-11T19:56:41.984383+0000 (oldest deadline 2024-03-11T19:57:03.684184+0000) 2024-03-11T19:57:06.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:05 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:05.980+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:06.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:05 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:05.980+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:06.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:05 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:05.980+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:06.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:05 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:05.980+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:06.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:05 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:05.980+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:06.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:05 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:05.980+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:06.564 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:06 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:06.100+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:52.897469+0000 front 2024-03-11T19:55:52.897424+0000 (oldest deadline 2024-03-11T19:56:16.997648+0000) 2024-03-11T19:57:06.564 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:06 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:06.100+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:58.098935+0000 front 2024-03-11T19:55:58.098716+0000 (oldest deadline 2024-03-11T19:56:23.398742+0000) 2024-03-11T19:57:06.564 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:06 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:06.100+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:10.300896+0000 front 2024-03-11T19:56:10.301037+0000 (oldest deadline 2024-03-11T19:56:33.800894+0000) 2024-03-11T19:57:06.564 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:06 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:06.100+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:17.802026+0000 front 2024-03-11T19:56:17.802159+0000 (oldest deadline 2024-03-11T19:56:39.502172+0000) 2024-03-11T19:57:06.564 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:06 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:06.100+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:25.303691+0000 front 2024-03-11T19:56:25.303777+0000 (oldest deadline 2024-03-11T19:56:51.203505+0000) 2024-03-11T19:57:06.564 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:06 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:06.100+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.704552+0000 front 2024-03-11T19:56:31.704537+0000 (oldest deadline 2024-03-11T19:56:55.204362+0000) 2024-03-11T19:57:06.564 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:06 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9[64652]: 2024-03-11T19:57:06.100+0000 7faf89a42700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:40.505264+0000 front 2024-03-11T19:56:40.505316+0000 (oldest deadline 2024-03-11T19:57:03.405337+0000) 2024-03-11T19:57:06.635 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:06 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:06.275+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:06.635 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:06 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:06.275+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:06.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:06 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:06.275+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:06.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:06 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:06.275+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:06.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:06 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:06.275+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:06.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:06 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:06.275+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:06.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:06 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:06.275+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.984242+0000 front 2024-03-11T19:56:41.984383+0000 (oldest deadline 2024-03-11T19:57:03.684184+0000) 2024-03-11T19:57:06.827 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:06 smithi083 podman[100324]: 2024-03-11 19:57:06.576531795 +0000 UTC m=+5.501956413 container died d67ec13d4b0ed8ab45b3ec9529a041d7ad8bbda1aba7b8b1ed0bd92eb8d5929e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9, RELEASE=squid-94a5397, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-11T19:57:07.194 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:06 smithi083 podman[100324]: 2024-03-11 19:57:06.927274697 +0000 UTC m=+5.852699315 container cleanup d67ec13d4b0ed8ab45b3ec9529a041d7ad8bbda1aba7b8b1ed0bd92eb8d5929e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS) 2024-03-11T19:57:07.194 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:06 smithi083 bash[100324]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9 2024-03-11T19:57:07.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:06 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:06.941+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:07.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:06 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:06.941+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:07.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:06 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:06.941+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:07.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:06 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:06.941+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:07.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:06 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:06.941+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:07.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:06 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:06.941+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:07.488 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:07 smithi083 podman[100343]: 2024-03-11 19:57:07.195066254 +0000 UTC m=+0.619119575 container remove d67ec13d4b0ed8ab45b3ec9529a041d7ad8bbda1aba7b8b1ed0bd92eb8d5929e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-11T19:57:07.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:07 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:07.318+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:07.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:07 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:07.318+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:07.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:07 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:07.318+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:07.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:07 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:07.318+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:07.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:07 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:07.318+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:07.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:07 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:07.318+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:07.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:07 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:07.318+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.984242+0000 front 2024-03-11T19:56:41.984383+0000 (oldest deadline 2024-03-11T19:57:03.684184+0000) 2024-03-11T19:57:07.798 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:07 smithi083 podman[100433]: 2024-03-11 19:57:07.391811635 +0000 UTC m=+0.017560536 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:57:07.798 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:07 smithi083 podman[100433]: 2024-03-11 19:57:07.504581298 +0000 UTC m=+0.130330241 container create 9ee5afc79e66bd40c82fbb522f067d5dea9d4c3b9cfc54df23341a0d1cacee0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-11T19:57:07.798 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:07 smithi083 podman[100433]: 2024-03-11 19:57:07.731799071 +0000 UTC m=+0.357547972 container init 9ee5afc79e66bd40c82fbb522f067d5dea9d4c3b9cfc54df23341a0d1cacee0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-94a5397) 2024-03-11T19:57:07.798 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:07 smithi083 podman[100433]: 2024-03-11 19:57:07.735066006 +0000 UTC m=+0.360814909 container start 9ee5afc79e66bd40c82fbb522f067d5dea9d4c3b9cfc54df23341a0d1cacee0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-11T19:57:07.798 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:07 smithi083 podman[100433]: 2024-03-11 19:57:07.798454243 +0000 UTC m=+0.424203155 container attach 9ee5afc79e66bd40c82fbb522f067d5dea9d4c3b9cfc54df23341a0d1cacee0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-11T19:57:08.052 INFO:journalctl@ceph.osd.9.smithi083.stdout:Mar 11 19:57:08 smithi083 podman[100433]: 2024-03-11 19:57:08.027236454 +0000 UTC m=+0.652985365 container died 9ee5afc79e66bd40c82fbb522f067d5dea9d4c3b9cfc54df23341a0d1cacee0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-11T19:57:08.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:07 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:07.933+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:08.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:07 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:07.933+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:08.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:07 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:07.933+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:08.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:07 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:07.933+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:08.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:07 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:07.933+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:08.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:07 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:07.933+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:08.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.9.service' 2024-03-11T19:57:08.635 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:08 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:08.325+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:08.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:08 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:08.325+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:08.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:08 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:08.325+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:08.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:08 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:08.325+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:08.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:08 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:08.325+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:08.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:08 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:08.325+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:08.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:08 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:08.325+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.984242+0000 front 2024-03-11T19:56:41.984383+0000 (oldest deadline 2024-03-11T19:57:03.684184+0000) 2024-03-11T19:57:08.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:08.668 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-11T19:57:08.668 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-11T19:57:08.668 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.10 2024-03-11T19:57:08.962 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:08 smithi137 systemd[1]: Stopping Ceph osd.10 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:09.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:08 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:08.899+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:09.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:08 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:08.899+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:09.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:08 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:08.899+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:09.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:08 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:08.899+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:09.153 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:08 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:08.899+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:09.153 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:08 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:08.899+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:09.336 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:08 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:08.962+0000 7f36e8377700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:09.336 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:08 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:08.962+0000 7f36e8377700 -1 osd.10 90 *** Got signal Terminated *** 2024-03-11T19:57:09.336 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:08 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:08.962+0000 7f36e8377700 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:57:09.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:09 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:09.336+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:09.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:09 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:09.336+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:09.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:09 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:09.336+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:09.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:09 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:09.336+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:09.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:09 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:09.336+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:09.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:09 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:09.336+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:09.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:09 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:09.336+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.984242+0000 front 2024-03-11T19:56:41.984383+0000 (oldest deadline 2024-03-11T19:57:03.684184+0000) 2024-03-11T19:57:10.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:09 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:09.863+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:10.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:09 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:09.863+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:10.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:09 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:09.863+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:10.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:09 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:09.863+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:10.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:09 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:09.863+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:10.153 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:09 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:09.863+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:10.635 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:10 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:10.299+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:10.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:10 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:10.299+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:10.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:10 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:10.299+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:10.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:10 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:10.299+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:10.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:10 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:10.299+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:10.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:10 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:10.299+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:10.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:10 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:10.299+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.984242+0000 front 2024-03-11T19:56:41.984383+0000 (oldest deadline 2024-03-11T19:57:03.684184+0000) 2024-03-11T19:57:11.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:10 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:10.886+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:11.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:10 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:10.886+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:11.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:10 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:10.886+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:11.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:10 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:10.886+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:11.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:10 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:10.886+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:11.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:10 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:10.886+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:11.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:11 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:11.307+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:11.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:11 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:11.307+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:11.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:11 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:11.307+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:11.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:11 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:11.307+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:11.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:11 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:11.307+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:11.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:11 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:11.307+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:11.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:11 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:11.307+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.984242+0000 front 2024-03-11T19:56:41.984383+0000 (oldest deadline 2024-03-11T19:57:03.684184+0000) 2024-03-11T19:57:12.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:11 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:11.934+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:12.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:11 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:11.934+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:12.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:11 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:11.934+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:12.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:11 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:11.934+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:12.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:11 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:11.934+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:12.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:11 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:11.934+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:12.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:11 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:11.934+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-11T19:56:48.487500+0000 front 2024-03-11T19:56:48.487533+0000 (oldest deadline 2024-03-11T19:57:11.387515+0000) 2024-03-11T19:57:12.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:12 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:12.327+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:12.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:12 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:12.327+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:12.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:12 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:12.327+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:12.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:12 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:12.327+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:12.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:12 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:12.327+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:12.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:12 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:12.327+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:12.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:12 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:12.327+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.984242+0000 front 2024-03-11T19:56:41.984383+0000 (oldest deadline 2024-03-11T19:57:03.684184+0000) 2024-03-11T19:57:13.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:12 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:12.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:13.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:12 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:12.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:13.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:12 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:12.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:13.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:12 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:12.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:13.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:12 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:12.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:13.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:12 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:12.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:13.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:12 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:12.969+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-11T19:56:48.487500+0000 front 2024-03-11T19:56:48.487533+0000 (oldest deadline 2024-03-11T19:57:11.387515+0000) 2024-03-11T19:57:13.635 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:13 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:13.291+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:50.275199+0000 front 2024-03-11T19:55:50.275156+0000 (oldest deadline 2024-03-11T19:56:16.175119+0000) 2024-03-11T19:57:13.635 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:13 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:13.291+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:55:59.576727+0000 front 2024-03-11T19:55:59.576682+0000 (oldest deadline 2024-03-11T19:56:25.476814+0000) 2024-03-11T19:57:13.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:13 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:13.291+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.478213+0000 front 2024-03-11T19:56:09.478274+0000 (oldest deadline 2024-03-11T19:56:32.378300+0000) 2024-03-11T19:57:13.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:13 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:13.291+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-03-11T19:56:16.880453+0000 front 2024-03-11T19:56:16.880560+0000 (oldest deadline 2024-03-11T19:56:41.580575+0000) 2024-03-11T19:57:13.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:13 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:13.291+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.182133+0000 front 2024-03-11T19:56:26.182190+0000 (oldest deadline 2024-03-11T19:56:47.882195+0000) 2024-03-11T19:57:13.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:13 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:13.291+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:31.983147+0000 front 2024-03-11T19:56:31.983260+0000 (oldest deadline 2024-03-11T19:56:56.683107+0000) 2024-03-11T19:57:13.636 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:13 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10[61229]: 2024-03-11T19:57:13.291+0000 7f36e1164700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.984242+0000 front 2024-03-11T19:56:41.984383+0000 (oldest deadline 2024-03-11T19:57:03.684184+0000) 2024-03-11T19:57:14.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:14 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:14.011+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:14.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:14 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:14.011+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:14.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:14 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:14.011+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:14.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:14 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:14.011+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:14.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:14 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:14.011+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:14.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:14 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:14.011+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:14.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:14 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:14.011+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-11T19:56:48.487500+0000 front 2024-03-11T19:56:48.487533+0000 (oldest deadline 2024-03-11T19:57:11.387515+0000) 2024-03-11T19:57:14.745 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:14 smithi137 podman[72897]: 2024-03-11 19:57:14.495846775 +0000 UTC m=+5.636703763 container died 04751bb7c8e4e2b66f3347cce7f68375146b2308e526d9b55df6991190673d2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-11T19:57:15.004 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:14 smithi137 podman[72897]: 2024-03-11 19:57:14.820302531 +0000 UTC m=+5.961159514 container cleanup 04751bb7c8e4e2b66f3347cce7f68375146b2308e526d9b55df6991190673d2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-11T19:57:15.004 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:14 smithi137 bash[72897]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10 2024-03-11T19:57:15.385 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:15 smithi137 podman[72914]: 2024-03-11 19:57:15.004702906 +0000 UTC m=+0.510820583 container remove 04751bb7c8e4e2b66f3347cce7f68375146b2308e526d9b55df6991190673d2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:57:15.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:15 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:15.044+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:15.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:15 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:15.044+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:15.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:15 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:15.044+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:15.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:15 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:15.044+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:15.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:15 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:15.044+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:15.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:15 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:15.044+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:15.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:15 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:15.044+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-11T19:56:48.487500+0000 front 2024-03-11T19:56:48.487533+0000 (oldest deadline 2024-03-11T19:57:11.387515+0000) 2024-03-11T19:57:15.757 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:15 smithi137 podman[73007]: 2024-03-11 19:57:15.328440752 +0000 UTC m=+0.017526257 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:57:15.757 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:15 smithi137 podman[73007]: 2024-03-11 19:57:15.447130408 +0000 UTC m=+0.136215897 container create d5cdf4145b80a25334a455d61dea5f3355859a548f4dcc8c33a4fc60d103054a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-11T19:57:15.757 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:15 smithi137 podman[73007]: 2024-03-11 19:57:15.665860921 +0000 UTC m=+0.354946418 container init d5cdf4145b80a25334a455d61dea5f3355859a548f4dcc8c33a4fc60d103054a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True) 2024-03-11T19:57:15.757 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:15 smithi137 podman[73007]: 2024-03-11 19:57:15.66924625 +0000 UTC m=+0.358331741 container start d5cdf4145b80a25334a455d61dea5f3355859a548f4dcc8c33a4fc60d103054a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-11T19:57:15.757 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:15 smithi137 podman[73007]: 2024-03-11 19:57:15.757835928 +0000 UTC m=+0.446921439 container attach d5cdf4145b80a25334a455d61dea5f3355859a548f4dcc8c33a4fc60d103054a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-11T19:57:16.135 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 11 19:57:15 smithi137 podman[73007]: 2024-03-11 19:57:15.959975928 +0000 UTC m=+0.649061419 container died d5cdf4145b80a25334a455d61dea5f3355859a548f4dcc8c33a4fc60d103054a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-11T19:57:16.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:16.048+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:16.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:16.048+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:16.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:16.048+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:16.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:16.048+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:16.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:16.048+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:16.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:16.048+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:16.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:16.048+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-11T19:56:48.487500+0000 front 2024-03-11T19:56:48.487533+0000 (oldest deadline 2024-03-11T19:57:11.387515+0000) 2024-03-11T19:57:16.570 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.10.service' 2024-03-11T19:57:16.614 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:16.614 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-11T19:57:16.614 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-11T19:57:16.614 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.11 2024-03-11T19:57:16.902 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:16 smithi204 systemd[1]: Stopping Ceph osd.11 for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:17.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:16.920+0000 7f0e36742700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:17.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:16.920+0000 7f0e36742700 -1 osd.11 90 *** Got signal Terminated *** 2024-03-11T19:57:17.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:16 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:16.920+0000 7f0e36742700 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-11T19:57:17.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:17 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:17.016+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:17.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:17 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:17.016+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:17.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:17 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:17.016+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:17.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:17 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:17.016+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:17.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:17 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:17.016+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:17.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:17 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:17.016+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:17.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:17 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:17.016+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-11T19:56:48.487500+0000 front 2024-03-11T19:56:48.487533+0000 (oldest deadline 2024-03-11T19:57:11.387515+0000) 2024-03-11T19:57:18.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:18 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:18.007+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:18.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:18 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:18.007+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:18.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:18 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:18.007+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:18.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:18 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:18.007+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:18.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:18 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:18.007+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:18.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:18 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:18.007+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:18.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:18 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:18.007+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-11T19:56:48.487500+0000 front 2024-03-11T19:56:48.487533+0000 (oldest deadline 2024-03-11T19:57:11.387515+0000) 2024-03-11T19:57:19.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:19.042+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:19.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:19.042+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:19.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:19.042+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:19.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:19.042+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:19.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:19.042+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:19.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:19.042+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:19.403 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:19 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:19.042+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-11T19:56:48.487500+0000 front 2024-03-11T19:56:48.487533+0000 (oldest deadline 2024-03-11T19:57:11.387515+0000) 2024-03-11T19:57:20.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:20.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:20.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:20.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:20.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:20.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:20.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-11T19:56:48.487500+0000 front 2024-03-11T19:56:48.487533+0000 (oldest deadline 2024-03-11T19:57:11.387515+0000) 2024-03-11T19:57:20.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.019+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-11T19:56:56.989979+0000 front 2024-03-11T19:56:56.990041+0000 (oldest deadline 2024-03-11T19:57:19.889979+0000) 2024-03-11T19:57:21.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.991+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-03-11T19:55:51.873135+0000 front 2024-03-11T19:55:51.873076+0000 (oldest deadline 2024-03-11T19:56:16.573152+0000) 2024-03-11T19:57:21.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.991+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-11T19:56:02.375091+0000 front 2024-03-11T19:56:02.375035+0000 (oldest deadline 2024-03-11T19:56:24.575543+0000) 2024-03-11T19:57:21.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.991+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-11T19:56:09.177001+0000 front 2024-03-11T19:56:09.177035+0000 (oldest deadline 2024-03-11T19:56:31.477224+0000) 2024-03-11T19:57:21.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.991+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-11T19:56:26.083170+0000 front 2024-03-11T19:56:26.083221+0000 (oldest deadline 2024-03-11T19:56:48.383101+0000) 2024-03-11T19:57:21.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.991+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-11T19:56:34.084969+0000 front 2024-03-11T19:56:34.084916+0000 (oldest deadline 2024-03-11T19:56:59.385009+0000) 2024-03-11T19:57:21.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.991+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-03-11T19:56:41.586233+0000 front 2024-03-11T19:56:41.586194+0000 (oldest deadline 2024-03-11T19:57:05.086114+0000) 2024-03-11T19:57:21.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.991+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-11T19:56:48.487500+0000 front 2024-03-11T19:56:48.487533+0000 (oldest deadline 2024-03-11T19:57:11.387515+0000) 2024-03-11T19:57:21.402 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:20 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11[62451]: 2024-03-11T19:57:20.991+0000 7f0e2f52f700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-11T19:56:56.989979+0000 front 2024-03-11T19:56:56.990041+0000 (oldest deadline 2024-03-11T19:57:19.889979+0000) 2024-03-11T19:57:22.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:22 smithi204 podman[74666]: 2024-03-11 19:57:22.316410674 +0000 UTC m=+5.508937560 container died 18cc330a9da2d4cc9c6a24ea8cb4b39f329c460e8670d62f21d4b25a584af6fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-11T19:57:22.902 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:22 smithi204 podman[74666]: 2024-03-11 19:57:22.714668565 +0000 UTC m=+5.907195494 container cleanup 18cc330a9da2d4cc9c6a24ea8cb4b39f329c460e8670d62f21d4b25a584af6fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-11T19:57:22.902 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:22 smithi204 bash[74666]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11 2024-03-11T19:57:23.323 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:22 smithi204 podman[74683]: 2024-03-11 19:57:22.966525745 +0000 UTC m=+0.644999224 container remove 18cc330a9da2d4cc9c6a24ea8cb4b39f329c460e8670d62f21d4b25a584af6fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-11T19:57:23.586 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:23 smithi204 podman[74776]: 2024-03-11 19:57:23.226499678 +0000 UTC m=+0.017549521 image pull a6477a474d71b439ad6caa1ffdec0c8e4b844b029f7201a2e5a0a3cf1e1a739e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b 2024-03-11T19:57:23.587 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:23 smithi204 podman[74776]: 2024-03-11 19:57:23.342341799 +0000 UTC m=+0.133391626 container create 697c9550eb689b2544f4e4a3aae45aeaaf68d3b1d9184b14d099b1561321e0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-11T19:57:23.587 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:23 smithi204 podman[74776]: 2024-03-11 19:57:23.587049574 +0000 UTC m=+0.378099413 container init 697c9550eb689b2544f4e4a3aae45aeaaf68d3b1d9184b14d099b1561321e0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, GIT_BRANCH=HEAD) 2024-03-11T19:57:23.881 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:23 smithi204 podman[74776]: 2024-03-11 19:57:23.590876722 +0000 UTC m=+0.381926545 container start 697c9550eb689b2544f4e4a3aae45aeaaf68d3b1d9184b14d099b1561321e0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-11T19:57:23.881 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:23 smithi204 podman[74776]: 2024-03-11 19:57:23.669657995 +0000 UTC m=+0.460707825 container attach 697c9550eb689b2544f4e4a3aae45aeaaf68d3b1d9184b14d099b1561321e0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True) 2024-03-11T19:57:24.152 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:23 smithi204 podman[74776]: 2024-03-11 19:57:23.882509957 +0000 UTC m=+0.673559797 container died 697c9550eb689b2544f4e4a3aae45aeaaf68d3b1d9184b14d099b1561321e0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-11T19:57:24.423 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 11 19:57:24 smithi204 podman[74802]: 2024-03-11 19:57:24.40649188 +0000 UTC m=+0.518419163 container remove 697c9550eb689b2544f4e4a3aae45aeaaf68d3b1d9184b14d099b1561321e0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-11T19:57:24.429 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@osd.11.service' 2024-03-11T19:57:24.471 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:24.471 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-11T19:57:24.471 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-11T19:57:24.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.a 2024-03-11T19:57:24.788 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:57:24 smithi083 systemd[1]: Stopping Ceph mds.a for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:25.063 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:57:24 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-a[71069]: 2024-03-11T19:57:24.787+0000 7fd0ae53f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:25.063 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:57:24 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-a[71069]: 2024-03-11T19:57:24.787+0000 7fd0ae53f700 -1 mds.a *** got signal Terminated *** 2024-03-11T19:57:29.064 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:57:28 smithi083 podman[100555]: 2024-03-11 19:57:28.793742514 +0000 UTC m=+4.126787735 container died 5d7b2f08a83b4e13d4232a7c4547947ac20fd698fb4e7945688d9d9af6f33d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397) 2024-03-11T19:57:29.562 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:57:29 smithi083 podman[100555]: 2024-03-11 19:57:29.270952885 +0000 UTC m=+4.603998097 container cleanup 5d7b2f08a83b4e13d4232a7c4547947ac20fd698fb4e7945688d9d9af6f33d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-a, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux ) 2024-03-11T19:57:29.562 INFO:journalctl@ceph.mds.a.smithi083.stdout:Mar 11 19:57:29 smithi083 bash[100555]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-a 2024-03-11T19:57:29.714 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.a.service' 2024-03-11T19:57:29.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:29.752 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-11T19:57:29.752 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-11T19:57:29.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.d 2024-03-11T19:57:30.090 INFO:journalctl@ceph.mds.d.smithi083.stdout:Mar 11 19:57:29 smithi083 systemd[1]: Stopping Ceph mds.d for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:30.563 INFO:journalctl@ceph.mds.d.smithi083.stdout:Mar 11 19:57:30 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-d[70117]: 2024-03-11T19:57:30.091+0000 7fecc38fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:30.564 INFO:journalctl@ceph.mds.d.smithi083.stdout:Mar 11 19:57:30 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-d[70117]: 2024-03-11T19:57:30.091+0000 7fecc38fe700 -1 mds.d *** got signal Terminated *** 2024-03-11T19:57:34.399 INFO:journalctl@ceph.mds.d.smithi083.stdout:Mar 11 19:57:34 smithi083 podman[100675]: 2024-03-11 19:57:34.09884333 +0000 UTC m=+4.116288436 container died d5c2d73f3e13d3a86c41bf8dd8d0861f112de8ce903007c9c6739135527dbe7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:57:34.716 INFO:journalctl@ceph.mds.d.smithi083.stdout:Mar 11 19:57:34 smithi083 podman[100675]: 2024-03-11 19:57:34.474705219 +0000 UTC m=+4.492150350 container cleanup d5c2d73f3e13d3a86c41bf8dd8d0861f112de8ce903007c9c6739135527dbe7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-94a5397, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-11T19:57:34.716 INFO:journalctl@ceph.mds.d.smithi083.stdout:Mar 11 19:57:34 smithi083 bash[100675]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-d 2024-03-11T19:57:34.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.d.service' 2024-03-11T19:57:34.889 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:34.889 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-11T19:57:34.889 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-11T19:57:34.889 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.g 2024-03-11T19:57:35.313 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:57:34 smithi083 systemd[1]: Stopping Ceph mds.g for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:35.313 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:57:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-g[70727]: 2024-03-11T19:57:35.211+0000 7f6e4ffae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:35.314 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:57:35 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-g[70727]: 2024-03-11T19:57:35.211+0000 7f6e4ffae700 -1 mds.g *** got signal Terminated *** 2024-03-11T19:57:39.313 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:57:38 smithi083 podman[100796]: 2024-03-11 19:57:38.994046956 +0000 UTC m=+3.888873538 container died 672aa82f8972085d906f86e0fc9fc75a95c0b2bb6e22367a3fe3601ab431c4e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-11T19:57:39.762 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:57:39 smithi083 podman[100796]: 2024-03-11 19:57:39.545571952 +0000 UTC m=+4.440398576 container cleanup 672aa82f8972085d906f86e0fc9fc75a95c0b2bb6e22367a3fe3601ab431c4e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-g, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-94a5397) 2024-03-11T19:57:39.762 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:57:39 smithi083 bash[100796]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-g 2024-03-11T19:57:40.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.g.service' 2024-03-11T19:57:40.032 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:57:39 smithi083 podman[100810]: 2024-03-11 19:57:39.763435565 +0000 UTC m=+0.769389292 container remove 672aa82f8972085d906f86e0fc9fc75a95c0b2bb6e22367a3fe3601ab431c4e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-g, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-03-11T19:57:40.032 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:57:40 smithi083 systemd[1]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.g.service: Deactivated successfully. 2024-03-11T19:57:40.032 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:57:40 smithi083 systemd[1]: Stopped Ceph mds.g for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:57:40.032 INFO:journalctl@ceph.mds.g.smithi083.stdout:Mar 11 19:57:40 smithi083 systemd[1]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.g.service: Consumed 1.074s CPU time. 2024-03-11T19:57:40.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:40.042 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-11T19:57:40.043 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-11T19:57:40.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.j 2024-03-11T19:57:40.313 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:57:40 smithi083 systemd[1]: Stopping Ceph mds.j for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:40.813 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:57:40 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-j[69729]: 2024-03-11T19:57:40.406+0000 7f8b44da5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:40.813 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:57:40 smithi083 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-j[69729]: 2024-03-11T19:57:40.406+0000 7f8b44da5700 -1 mds.j *** got signal Terminated *** 2024-03-11T19:57:43.813 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:57:43 smithi083 podman[100916]: 2024-03-11 19:57:43.469175939 +0000 UTC m=+3.201642870 container died f6dc6086f4c5784d592032a70e8905e82960bec6016f6ceef2d3ab9f2b10c9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397) 2024-03-11T19:57:44.207 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:57:44 smithi083 podman[100916]: 2024-03-11 19:57:44.015794361 +0000 UTC m=+3.748261300 container cleanup f6dc6086f4c5784d592032a70e8905e82960bec6016f6ceef2d3ab9f2b10c9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-j, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-11T19:57:44.207 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:57:44 smithi083 bash[100916]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-j 2024-03-11T19:57:44.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.j.service' 2024-03-11T19:57:44.458 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:57:44 smithi083 podman[100931]: 2024-03-11 19:57:44.208880655 +0000 UTC m=+0.734968001 container remove f6dc6086f4c5784d592032a70e8905e82960bec6016f6ceef2d3ab9f2b10c9b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-11T19:57:44.458 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:57:44 smithi083 systemd[1]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.j.service: Deactivated successfully. 2024-03-11T19:57:44.459 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:57:44 smithi083 systemd[1]: Stopped Ceph mds.j for 839f93c8-dfdd-11ee-95c8-87774f69a715. 2024-03-11T19:57:44.459 INFO:journalctl@ceph.mds.j.smithi083.stdout:Mar 11 19:57:44 smithi083 systemd[1]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.j.service: Consumed 1.063s CPU time. 2024-03-11T19:57:44.468 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:44.468 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-11T19:57:44.468 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-11T19:57:44.469 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.b 2024-03-11T19:57:44.776 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:57:44 smithi137 systemd[1]: Stopping Ceph mds.b for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:45.045 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:57:44 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-b[66824]: 2024-03-11T19:57:44.776+0000 7f3982ec4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:45.045 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:57:44 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-b[66824]: 2024-03-11T19:57:44.776+0000 7f3982ec4700 -1 mds.b *** got signal Terminated *** 2024-03-11T19:57:46.885 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:57:46 smithi137 podman[73115]: 2024-03-11 19:57:46.425977099 +0000 UTC m=+1.769759159 container died 56dd86f430d7682d138b838c0bb1b9a53322149f1b5b873cb7a1c4fab13246d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:57:47.368 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:57:47 smithi137 podman[73115]: 2024-03-11 19:57:47.085262628 +0000 UTC m=+2.429044684 container cleanup 56dd86f430d7682d138b838c0bb1b9a53322149f1b5b873cb7a1c4fab13246d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-94a5397, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:57:47.369 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 11 19:57:47 smithi137 bash[73115]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-b 2024-03-11T19:57:47.501 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.b.service' 2024-03-11T19:57:47.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:47.539 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-11T19:57:47.539 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-11T19:57:47.539 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.e 2024-03-11T19:57:47.634 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:57:47 smithi137 systemd[1]: Stopping Ceph mds.e for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:48.135 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:57:47 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e[66197]: 2024-03-11T19:57:47.846+0000 7ffab7849700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:48.135 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:57:47 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e[66197]: 2024-03-11T19:57:47.846+0000 7ffab7849700 -1 mds.e *** got signal Terminated *** 2024-03-11T19:57:50.788 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:57:50 smithi137 podman[73248]: 2024-03-11 19:57:50.428322445 +0000 UTC m=+2.679545625 container died 3c3a611e31be5c47f60daca5d7f357df9f867964d8e85243c0e933fbfa443372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-11T19:57:51.134 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:57:50 smithi137 podman[73248]: 2024-03-11 19:57:50.88035303 +0000 UTC m=+3.131576253 container cleanup 3c3a611e31be5c47f60daca5d7f357df9f867964d8e85243c0e933fbfa443372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-11T19:57:51.135 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 11 19:57:50 smithi137 bash[73248]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-e 2024-03-11T19:57:51.351 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.e.service' 2024-03-11T19:57:51.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:51.387 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-11T19:57:51.387 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-11T19:57:51.388 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.h 2024-03-11T19:57:51.707 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:57:51 smithi137 systemd[1]: Stopping Ceph mds.h for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:52.134 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:57:51 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h[66505]: 2024-03-11T19:57:51.708+0000 7f711b1bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:52.135 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:57:51 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h[66505]: 2024-03-11T19:57:51.708+0000 7f711b1bf700 -1 mds.h *** got signal Terminated *** 2024-03-11T19:57:54.724 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:57:54 smithi137 podman[73368]: 2024-03-11 19:57:54.414423032 +0000 UTC m=+2.800523683 container died 047e588e571b84798d5df649d2611a1f54231238252004ce0713fd349725b088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:57:55.024 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:57:54 smithi137 podman[73368]: 2024-03-11 19:57:54.825002928 +0000 UTC m=+3.211103572 container cleanup 047e588e571b84798d5df649d2611a1f54231238252004ce0713fd349725b088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-11T19:57:55.024 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 11 19:57:54 smithi137 bash[73368]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-h 2024-03-11T19:57:55.160 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.h.service' 2024-03-11T19:57:55.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:55.194 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-11T19:57:55.194 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-11T19:57:55.194 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.k 2024-03-11T19:57:55.634 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:57:55 smithi137 systemd[1]: Stopping Ceph mds.k for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:55.634 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:57:55 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-k[65875]: 2024-03-11T19:57:55.503+0000 7f40b78cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:55.635 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:57:55 smithi137 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-k[65875]: 2024-03-11T19:57:55.503+0000 7f40b78cd700 -1 mds.k *** got signal Terminated *** 2024-03-11T19:57:56.961 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:57:56 smithi137 podman[73488]: 2024-03-11 19:57:56.699958838 +0000 UTC m=+1.286735915 container died c2a41a73f12acdb7a94039b2b32c37474fe894207df67113ed09e70e720e1d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-k, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-94a5397, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-11T19:57:57.228 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:57:57 smithi137 podman[73488]: 2024-03-11 19:57:57.045531875 +0000 UTC m=+1.632308952 container cleanup c2a41a73f12acdb7a94039b2b32c37474fe894207df67113ed09e70e720e1d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-k, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-94a5397, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:57:57.228 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 11 19:57:57 smithi137 bash[73488]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-k 2024-03-11T19:57:57.441 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.k.service' 2024-03-11T19:57:57.475 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:57:57.475 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-11T19:57:57.475 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-11T19:57:57.475 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.c 2024-03-11T19:57:57.781 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:57:57 smithi204 systemd[1]: Stopping Ceph mds.c for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:57:58.040 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:57:57 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c[67894]: 2024-03-11T19:57:57.781+0000 7fb5c87a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:57:58.040 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:57:57 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c[67894]: 2024-03-11T19:57:57.781+0000 7fb5c87a8700 -1 mds.c *** got signal Terminated *** 2024-03-11T19:58:02.148 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:58:01 smithi204 podman[74883]: 2024-03-11 19:58:01.788212847 +0000 UTC m=+4.123123815 container died 2ad65201b1a866be0434652ed74e91b1e110caff903de84f5381aaf2a0ccb5fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-11T19:58:02.401 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:58:02 smithi204 podman[74883]: 2024-03-11 19:58:02.231521065 +0000 UTC m=+4.566432032 container cleanup 2ad65201b1a866be0434652ed74e91b1e110caff903de84f5381aaf2a0ccb5fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-03-11T19:58:02.401 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:58:02 smithi204 bash[74883]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c 2024-03-11T19:58:02.786 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 11 19:58:02 smithi204 podman[74909]: 2024-03-11 19:58:02.53361783 +0000 UTC m=+0.736434570 container remove 2ad65201b1a866be0434652ed74e91b1e110caff903de84f5381aaf2a0ccb5fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-94a5397, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-11T19:58:02.802 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.c.service' 2024-03-11T19:58:02.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:58:02.840 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-11T19:58:02.840 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-11T19:58:02.841 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.f 2024-03-11T19:58:03.151 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:58:02 smithi204 systemd[1]: Stopping Ceph mds.f for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:58:03.650 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:58:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f[67585]: 2024-03-11T19:58:03.171+0000 7fc286a34700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:58:03.651 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:58:03 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f[67585]: 2024-03-11T19:58:03.171+0000 7fc286a34700 -1 mds.f *** got signal Terminated *** 2024-03-11T19:58:06.719 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:58:06 smithi204 podman[75014]: 2024-03-11 19:58:06.429997529 +0000 UTC m=+3.369486101 container died 055abf3673c8bc86cc20f21c0f02fdb4db373b050e1045dc6ec518aedef48b95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-11T19:58:07.052 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:58:06 smithi204 podman[75014]: 2024-03-11 19:58:06.820060468 +0000 UTC m=+3.759549052 container cleanup 055abf3673c8bc86cc20f21c0f02fdb4db373b050e1045dc6ec518aedef48b95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-94a5397, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-11T19:58:07.052 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 11 19:58:06 smithi204 bash[75014]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-f 2024-03-11T19:58:07.188 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.f.service' 2024-03-11T19:58:07.224 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:58:07.225 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-11T19:58:07.225 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-11T19:58:07.225 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.i 2024-03-11T19:58:07.650 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:58:07 smithi204 systemd[1]: Stopping Ceph mds.i for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:58:07.651 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:58:07 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i[68220]: 2024-03-11T19:58:07.548+0000 7f2cc42d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:58:07.651 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:58:07 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i[68220]: 2024-03-11T19:58:07.548+0000 7f2cc42d4700 -1 mds.i *** got signal Terminated *** 2024-03-11T19:58:10.706 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:58:10 smithi204 podman[75135]: 2024-03-11 19:58:10.41063482 +0000 UTC m=+2.969274888 container died 9db8ac59f9805054e1da7bc414980ccd77e88cd1a17f7804517e8e3de687169f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-94a5397, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-11T19:58:11.023 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:58:10 smithi204 podman[75135]: 2024-03-11 19:58:10.807718853 +0000 UTC m=+3.366358905 container cleanup 9db8ac59f9805054e1da7bc414980ccd77e88cd1a17f7804517e8e3de687169f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-94a5397, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-11T19:58:11.023 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 11 19:58:10 smithi204 bash[75135]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-i 2024-03-11T19:58:11.153 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.i.service' 2024-03-11T19:58:11.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:58:11.188 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-11T19:58:11.188 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-11T19:58:11.188 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.l 2024-03-11T19:58:11.400 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:58:11 smithi204 systemd[1]: Stopping Ceph mds.l for 839f93c8-dfdd-11ee-95c8-87774f69a715... 2024-03-11T19:58:11.901 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:58:11 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-l[67276]: 2024-03-11T19:58:11.502+0000 7f2a4e19f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-11T19:58:11.901 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:58:11 smithi204 ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-l[67276]: 2024-03-11T19:58:11.502+0000 7f2a4e19f700 -1 mds.l *** got signal Terminated *** 2024-03-11T19:58:14.227 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:58:13 smithi204 podman[75260]: 2024-03-11 19:58:13.962517418 +0000 UTC m=+2.558779952 container died 582f1dd65852207c6a876dadd34ef3c2b479e8f7ffaa88b4bcedaff4973e5082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-l, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=squid-94a5397, io.buildah.version=1.33.5) 2024-03-11T19:58:14.586 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:58:14 smithi204 podman[75260]: 2024-03-11 19:58:14.329270784 +0000 UTC m=+2.925533285 container cleanup 582f1dd65852207c6a876dadd34ef3c2b479e8f7ffaa88b4bcedaff4973e5082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bbe755e768640cafd345af8384e9f3b099ff2fb85ebf0601e0883dbf412b887b, name=ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-l, org.label-schema.build-date=20240307, ceph=True, RELEASE=squid-94a5397, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-11T19:58:14.586 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 11 19:58:14 smithi204 bash[75260]: ceph-839f93c8-dfdd-11ee-95c8-87774f69a715-mds-l 2024-03-11T19:58:14.747 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-839f93c8-dfdd-11ee-95c8-87774f69a715@mds.l.service' 2024-03-11T19:58:14.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-11T19:58:14.781 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-11T19:58:14.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-11T19:58:14.971 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:58:17.427 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-11T19:58:17.618 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:58:20.156 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 --force --keep-logs 2024-03-11T19:58:20.350 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:58:22.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:58:22.784 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:58:22.820 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-11T19:58:22.849 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-11T19:58:22.851 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272/remote/smithi083/crash 2024-03-11T19:58:22.852 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/crash -- . 2024-03-11T19:58:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-11T19:58:22.897 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-03-11T19:58:22.899 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272/remote/smithi137/crash 2024-03-11T19:58:22.899 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/crash -- . 2024-03-11T19:58:22.942 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-11T19:58:22.942 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-03-11T19:58:22.943 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272/remote/smithi204/crash 2024-03-11T19:58:22.945 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/crash -- . 2024-03-11T19:58:22.984 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/crash: Cannot open: No such file or directory 2024-03-11T19:58:22.984 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-03-11T19:58:22.985 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-11T19:58:22.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-11T19:58:23.024 INFO:teuthology.orchestra.run.smithi083.stdout:2024-03-11T19:31:07.246522+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:58:23.024 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-11T19:58:23.024 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[SEC\]' /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-11T19:58:23.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]' /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-11T19:58:23.169 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[WRN\]' /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-11T19:58:23.240 INFO:teuthology.orchestra.run.smithi083.stdout:2024-03-11T19:31:07.246522+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-11T19:58:23.240 INFO:tasks.cephadm:Compressing logs... 2024-03-11T19:58:23.240 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-11T19:58:23.283 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-11T19:58:23.286 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-11T19:58:23.307 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/cephadm.log 2024-03-11T19:58:23.307 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-03-11T19:58:23.307 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.6.log 2024-03-11T19:58:23.308 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mon.a.log 2024-03-11T19:58:23.308 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-11T19:58:23.310 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.g.log 2024-03-11T19:58:23.310 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.j.log 2024-03-11T19:58:23.310 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.audit.log: /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.3.log 2024-03-11T19:58:23.310 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mgr.x.log 2024-03-11T19:58:23.310 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.0.log 2024-03-11T19:58:23.311 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mgr.x.log: 90.1% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-11T19:58:23.312 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-11T19:58:23.315 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.0.log: 94.6% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.g.log.gz 2024-03-11T19:58:23.315 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- 94.6% /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log 2024-03-11T19:58:23.315 INFO:teuthology.orchestra.run.smithi083.stderr: -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.j.log.gz 2024-03-11T19:58:23.316 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-volume.log 82.6% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-11T19:58:23.316 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-03-11T19:58:23.316 INFO:teuthology.orchestra.run.smithi083.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-11T19:58:23.316 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.9.log 2024-03-11T19:58:23.316 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.d.log 2024-03-11T19:58:23.335 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-volume.log: /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.9.log: 86.8% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-11T19:58:23.335 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.a.log 2024-03-11T19:58:23.335 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.d.log.gz 2024-03-11T19:58:23.335 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.a.log: 91.0% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-11T19:58:23.335 INFO:teuthology.orchestra.run.smithi083.stderr: 94.5% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.a.log.gz 2024-03-11T19:58:23.339 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-03-11T19:58:23.339 INFO:teuthology.orchestra.run.smithi137.stderr: /var/log/ceph/cephadm.log 2024-03-11T19:58:23.339 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.10.log 2024-03-11T19:58:23.339 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-11T19:58:23.339 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.b.log 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.4.log 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.1.log 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.k.log 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.h.log 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.7.log 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.e.log 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.7.log: 90.3% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mgr.y.log 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.k.log.gz 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mgr.y.log: 93.0% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mgr.y.log.gz 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log 2024-03-11T19:58:23.340 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-11T19:58:23.341 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-11T19:58:23.341 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log: 86.8% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-11T19:58:23.341 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mon.b.log 2024-03-11T19:58:23.342 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-11T19:58:23.342 INFO:teuthology.orchestra.run.smithi204.stderr:: No such file or directory 2024-03-11T19:58:23.342 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.l.log 2024-03-11T19:58:23.342 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.audit.log 2024-03-11T19:58:23.342 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.i.log 2024-03-11T19:58:23.342 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.l.log: /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.11.log 2024-03-11T19:58:23.342 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mgr.z.log 2024-03-11T19:58:23.342 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.i.log: /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.8.log 2024-03-11T19:58:23.342 INFO:teuthology.orchestra.run.smithi204.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.5.log 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.f.log 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.cephadm.log 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.audit.log.gz 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.cephadm.log: 82.4% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.cephadm.log.gz 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.2.log 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log: 94.6% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.l.log.gz 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.c.log 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.2.log: 86.8% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph.log.gz 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-volume.log 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr: 93.1%/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.c.log: -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mgr.z.log.gz 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mon.c.log 2024-03-11T19:58:23.343 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-volume.log: /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-11T19:58:23.352 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-volume.log: /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-volume.log.gz 2024-03-11T19:58:23.544 INFO:teuthology.orchestra.run.smithi204.stderr: 94.0% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.f.log.gz 2024-03-11T19:58:23.573 INFO:teuthology.orchestra.run.smithi137.stderr: 94.7% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.e.log.gz 2024-03-11T19:58:23.624 INFO:teuthology.orchestra.run.smithi083.stderr: 90.2% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mgr.x.log.gz 2024-03-11T19:58:23.790 INFO:teuthology.orchestra.run.smithi204.stderr: 94.3% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.c.log.gz 2024-03-11T19:58:23.814 INFO:teuthology.orchestra.run.smithi137.stderr: 91.9% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mon.b.log.gz 2024-03-11T19:58:23.881 INFO:teuthology.orchestra.run.smithi204.stderr: 91.8% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mon.c.log.gz 2024-03-11T19:58:24.315 INFO:teuthology.orchestra.run.smithi083.stderr: 90.4% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mon.a.log.gz 2024-03-11T19:58:24.965 INFO:teuthology.orchestra.run.smithi137.stderr: 94.2% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.h.log.gz 2024-03-11T19:58:26.082 INFO:teuthology.orchestra.run.smithi204.stderr: 93.9% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.i.log.gz 2024-03-11T19:58:27.256 INFO:teuthology.orchestra.run.smithi137.stderr: 94.2% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-mds.b.log.gz 2024-03-11T19:58:34.351 INFO:teuthology.orchestra.run.smithi083.stderr: 94.2% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.9.log.gz 2024-03-11T19:58:37.438 INFO:teuthology.orchestra.run.smithi204.stderr: 94.3% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.8.log.gz 2024-03-11T19:58:39.741 INFO:teuthology.orchestra.run.smithi137.stderr: 94.2% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.4.log.gz 2024-03-11T19:58:42.652 INFO:teuthology.orchestra.run.smithi204.stderr: 94.3% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.5.log.gz 2024-03-11T19:58:42.835 INFO:teuthology.orchestra.run.smithi137.stderr: 93.9% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.10.log.gz 2024-03-11T19:58:43.488 INFO:teuthology.orchestra.run.smithi083.stderr: 94.1% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.6.log.gz 2024-03-11T19:58:45.422 INFO:teuthology.orchestra.run.smithi204.stderr: 94.4% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.11.log.gz 2024-03-11T19:58:45.620 INFO:teuthology.orchestra.run.smithi204.stderr: 94.0% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.2.log.gz 2024-03-11T19:58:45.622 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-03-11T19:58:45.622 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m22.321s 2024-03-11T19:58:45.622 INFO:teuthology.orchestra.run.smithi204.stderr:user 1m18.877s 2024-03-11T19:58:45.622 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m2.898s 2024-03-11T19:58:45.972 INFO:teuthology.orchestra.run.smithi137.stderr: 94.0% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.1.log.gz 2024-03-11T19:58:52.265 INFO:teuthology.orchestra.run.smithi083.stderr: 94.2% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.0.log.gz 2024-03-11T19:58:52.891 INFO:teuthology.orchestra.run.smithi083.stderr: 94.1% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.3.log.gz 2024-03-11T19:58:52.893 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-03-11T19:58:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m29.600s 2024-03-11T19:58:52.893 INFO:teuthology.orchestra.run.smithi083.stderr:user 1m27.697s 2024-03-11T19:58:52.894 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m3.284s 2024-03-11T19:58:54.556 INFO:teuthology.orchestra.run.smithi137.stderr: 94.1% -- replaced with /var/log/ceph/839f93c8-dfdd-11ee-95c8-87774f69a715/ceph-osd.7.log.gz 2024-03-11T19:58:54.558 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-03-11T19:58:54.559 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m31.259s 2024-03-11T19:58:54.559 INFO:teuthology.orchestra.run.smithi137.stderr:user 1m32.534s 2024-03-11T19:58:54.559 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m3.480s 2024-03-11T19:58:54.559 INFO:tasks.cephadm:Archiving logs... 2024-03-11T19:58:54.559 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272/remote/smithi083/log 2024-03-11T19:58:54.560 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-11T19:59:07.303 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272/remote/smithi137/log 2024-03-11T19:59:07.303 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-11T19:59:20.193 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272/remote/smithi204/log 2024-03-11T19:59:20.194 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-11T19:59:31.114 INFO:tasks.cephadm:Removing cluster... 2024-03-11T19:59:31.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 --force 2024-03-11T19:59:31.307 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:59:31.677 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 --force 2024-03-11T19:59:31.868 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:59:32.344 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 839f93c8-dfdd-11ee-95c8-87774f69a715 --force 2024-03-11T19:59:32.537 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 839f93c8-dfdd-11ee-95c8-87774f69a715 2024-03-11T19:59:32.891 INFO:tasks.cephadm:Removing cephadm ... 2024-03-11T19:59:32.892 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-11T19:59:32.910 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-11T19:59:32.926 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-11T19:59:32.941 INFO:tasks.cephadm:Teardown complete 2024-03-11T19:59:32.941 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-11T19:59:32.952 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-11T19:59:32.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-11T19:59:32.955 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-11T19:59:32.969 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-11T19:59:33.081 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-11T19:59:33.093 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-11T19:59:33.093 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:59:33.096 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:59:33.098 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T19:59:33.110 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:59:33.111 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:59:33.112 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-03-11T19:59:33.114 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:59:33.114 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-03-11T19:59:33.114 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 64 +695us[ +683us] +/- 150ms 2024-03-11T19:59:33.115 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv02.front.sepia.ceph.com 3 6 377 1 -1753us[-1764us] +/- 52ms 2024-03-11T19:59:33.115 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +2501us[+2489us] +/- 88ms 2024-03-11T19:59:33.115 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:59:33.116 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:59:33.116 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-03-11T19:59:33.116 INFO:teuthology.orchestra.run.smithi083.stdout:^- hv01.front.sepia.ceph.com 2 6 377 1 +798us[ +668us] +/- 151ms 2024-03-11T19:59:33.116 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv02.front.sepia.ceph.com 3 6 377 0 -1626us[-1756us] +/- 52ms 2024-03-11T19:59:33.116 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +2627us[+2497us] +/- 88ms 2024-03-11T19:59:33.117 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:59:33.118 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-11T19:59:33.118 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-03-11T19:59:33.118 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 63 +730us[ +733us] +/- 150ms 2024-03-11T19:59:33.118 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -1690us[-1684us] +/- 52ms 2024-03-11T19:59:33.118 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +2577us[+2577us] +/- 88ms 2024-03-11T19:59:33.118 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-11T19:59:33.118 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-11T19:59:33.129 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-11T19:59:33.129 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-11T19:59:33.140 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-11T19:59:33.246 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-11T19:59:33.318 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-11T19:59:33.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:59:33.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:59:33.423 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:59:33.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:59:33.454 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-11T19:59:33.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:59:33.483 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-11T19:59:33.495 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-11T19:59:33.511 INFO:teuthology.task.internal:Duration was 2299.820807 seconds 2024-03-11T19:59:33.512 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-11T19:59:33.522 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-11T19:59:33.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-11T19:59:33.525 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-11T19:59:33.527 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-11T19:59:33.567 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:59:33.568 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:59:33.570 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-11T19:59:33.914 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-11T19:59:33.914 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-03-11T19:59:33.914 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-11T19:59:33.933 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-03-11T19:59:33.934 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-11T19:59:33.983 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-03-11T19:59:33.984 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-11T19:59:34.004 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-11T19:59:34.004 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-11T19:59:34.007 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-11T19:59:34.026 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-11T19:59:34.073 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-11T19:59:34.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-11T19:59:34.075 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-11T19:59:34.094 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-11T19:59:34.458 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-11T19:59:34.471 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-11T19:59:34.471 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-11T19:59:34.497 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-11T19:59:34.525 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-11T19:59:34.552 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-11T19:59:34.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:59:34.567 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:59:34.570 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:59:34.590 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-03-11T19:59:34.593 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-03-11T19:59:34.618 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-03-11T19:59:34.656 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:59:34.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:59:34.670 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:59:34.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:59:34.685 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-11T19:59:34.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T19:59:34.709 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-11T19:59:34.726 INFO:teuthology.task.internal:Transferring archived files... 2024-03-11T19:59:34.728 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272/remote/smithi083 2024-03-11T19:59:34.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-11T19:59:34.778 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272/remote/smithi137 2024-03-11T19:59:34.779 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-11T19:59:34.817 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272/remote/smithi204 2024-03-11T19:59:34.817 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-11T19:59:34.855 INFO:teuthology.task.internal:Removing archive directory... 2024-03-11T19:59:34.856 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-11T19:59:34.858 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-11T19:59:34.860 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-11T19:59:34.904 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-11T19:59:34.917 INFO:teuthology.task.internal:Not uploading archives. 2024-03-11T19:59:34.917 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-11T19:59:34.965 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-11T19:59:34.966 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-11T19:59:34.968 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-11T19:59:34.972 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-11T19:59:34.980 INFO:teuthology.orchestra.run.smithi083.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 11 19:59 /home/ubuntu/cephtest 2024-03-11T19:59:34.983 INFO:teuthology.orchestra.run.smithi137.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 11 19:59 /home/ubuntu/cephtest 2024-03-11T19:59:34.988 INFO:teuthology.orchestra.run.smithi204.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 11 19:59 /home/ubuntu/cephtest 2024-03-11T19:59:34.989 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-11T19:59:35.082 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-11T19:59:35.218 INFO:teuthology.nuke:Checking targets against current locks 2024-03-11T19:59:35.244 DEBUG:teuthology.nuke:shortname: smithi083 2024-03-11T19:59:35.244 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T19:59:35.274 DEBUG:teuthology.nuke:shortname: smithi137 2024-03-11T19:59:35.275 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T19:59:35.279 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:10:49.730995', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLOuY+M2QlviJnbLDcvky6Z2VXTmnRXIgRlRysT1A53w960iPFT5YgOjY06lf6CKf7Nf12BLKe5QWDhcd/OWMM='} 2024-03-11T19:59:35.300 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:10:49.732958', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIRArQaZruB8rQ7ogXnUlBsEfp+U4nllr8h7QeGdH4Sh7q/7ttds73kas0cP1hVFGLNgfveTI+32RPmk/BiTsvw='} 2024-03-11T19:59:35.305 DEBUG:teuthology.nuke:shortname: smithi204 2024-03-11T19:59:35.305 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T19:59:35.329 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591272', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 19:10:49.734627', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEk29REzAi1N7zcW/tXY8+rqq4cQhg/zAIj4G22CDXWyOqOT0bCWdyI0R7KWbkVYYoKifB0PwQIoyxgddFpTuMw='} 2024-03-11T19:59:35.340 INFO:teuthology.orchestra.console.smithi083:Power off 2024-03-11T19:59:35.340 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-11T19:59:35.357 INFO:teuthology.orchestra.console.smithi137:Power off 2024-03-11T19:59:35.357 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-11T19:59:35.367 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2024-03-11T19:59:35.367 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:35.382 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2024-03-11T19:59:35.382 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:35.393 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-03-11T19:59:35.408 INFO:teuthology.orchestra.console.smithi204:Power off 2024-03-11T19:59:35.409 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-11T19:59:35.419 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-11T19:59:35.432 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2024-03-11T19:59:35.432 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:35.457 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-03-11T19:59:39.394 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:39.421 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:39.458 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:39.504 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-03-11T19:59:39.529 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-11T19:59:39.569 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-03-11T19:59:43.506 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:43.531 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:43.570 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:43.616 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-03-11T19:59:43.640 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-11T19:59:43.680 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-03-11T19:59:47.618 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:47.641 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:47.681 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T19:59:47.729 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-03-11T19:59:47.750 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2024-03-11T19:59:47.791 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2024-03-11T19:59:47.830 INFO:teuthology.orchestra.console.smithi083:Power off completed 2024-03-11T19:59:47.851 INFO:teuthology.orchestra.console.smithi137:Power off completed 2024-03-11T19:59:47.892 INFO:teuthology.orchestra.console.smithi204:Power off completed 2024-03-11T19:59:47.997 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} duration: 2299.820806503296 failure_reason: '"2024-03-11T19:31:07.246522+0000 mon.a (mon.0) 190 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-03-11T19:59:47.997 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-11T19:59:48.090 INFO:teuthology.run:FAIL